Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I just ran into the problem for an HR collegue that uses Excel.

Real financial data manage to produce three floats (with only 2 decimal places as usual with finance) that when used together a+b-c produced -0 (actually -0.1336e-20 or something like that).

The result ended up producing a cascade of mismatching results in further calculations seeking for 0 balance.

It’s a known and identified "feature" with a correct workaround which is rounding (for finance) and an horrible evil twin "formatting away and praying" that can lead to futur unfigurable mess for candid users.

So for me I agree that floating point error is not so bad from a mathematical standpoint, but from a technical stand point let us all remind that for most humankind 3/2 = 1.5 (and not 0).



”with a correct workaround which is rounding (for finance) and an horrible evil twin "formatting away and praying" ”

Are you suggesting the “precision as displayed” setting in Excel is that evil twin? Why?


Because financial data (in our use case) have a fixed 2 decimal format. In almost every other context you could impose this constraint by somehow defining a data type and be done with it.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: