19 thoughts on “Quirky behavior of oocalc”

1. For me:

   * both F2 and F4 hold 0.9421052632… which is not quite 0.94

   * both G2 and G4 hold 14.1315789474… which is neither 14.3 not 14.10

   It seems the calculations are correct.

2. That’s just the rounding… this is no bug…

   356/380 = 0.942105263157895
   356/380 != 0.94

3. Uhm, a similar thing appened to Excel.

   It’s all about fractional numbers, decimal digits and their rappresentation, I think…

   In F2 you do =E2/D2, 358/380
   If you do this subdivision with a calculator, the result is 0,94210526315789473684210526315789

   Now, Calc use as default for numbers a format with only two digits after comma. so, you will see an approximation of the value (here 0,94)
   In F8 you handwrite 0,94, wich isn’t equal to the result of E2/D2, but it is its rappresentation.
   When you get this two value moltiplied by an huge 48000, the approximation error become bigger and more visible.
   if you select all your cells and do “format”->”cells” and increase “decimal places” to, say, 15, you will see more decimal digit and were the math “went wrong”

   (sorry for my english and typos)

4. Not a bug.

   The value of F2 is 0.9421053…

   It is displayed as 0.94 because the cell isn’t formatted to show more decimal places.

   Similarly, the value of G2 is 14.1315789…

5. You need a better brain.

   Input in any cell:
   =480000*15*358/380

   You’ll get 6783157.89, as in the sample .ods. This is what OOCalc is computing, there is no need for such a complex setup as in your example.

   Now input in another cell:
   =480000*15*0.94

   Obviously, you’ll get 6768000, the same as in your sample document.

   Why is that so? Simply because 358/380 does not make 0.94, what is displayed as “0.94” is the standard number format with 2 digits after the digital point! In full truth, 358/380 = 0.94210526315789473684 or, with the current precision in OOCalc, 0.94210526315789500000, and this is why 6783157.89 != 6768000.

   Folks, your Problem Exists Between Keyboard And Chair, not in OOCalc! What you have displayed in a given cell is one thing, and a formula that gets propagated is another thing.

6. using round() to force the decimal places seems to fix it. eg. change the F2 formula to round(e2/d2;2)

7. You’ve simply lost precision. Calc/Spreadsheets carry more precision in their calculations than may be displayed in the cells.

   Select the column G and click the button to add decimal places a few times. What you copied down is not exactly the same number as was calculated.

8. I don’t see the problem. Yes you get different values, but that’s because your calculated cells (the ones with forumlas) contains higher precision than the ones you entered yourself. Try changing the formatting of the cells and you’ll see the difference.

   I did the calculations on a standard calculator and found no errors…

   or am I missing something?

9. It is not broken.

   Expand the formatting for all of the values in column ‘F’ to multiple decimal places… say, 10

   It is the formatting of the cells and the number decimal places… a common gotcha with spreadsheets where calculated values are used in subsequent computations and then compared to hand entered values with less decimal places..

   What you see on the screen isn’t exactly what the underlying numerical value is.

10. This is not an Error: 358/380=0.942105263 not 0.94!
    Ckeck your cell formatting.

11. The result is not accurate, 358 / 380 = 0,942105263…

12. Isn’t what I see on the screen supposed to be the underlying numerical value ? 🙂

13. Nope you are not. As a comment states, it is best to use round().

14. I would be very pissed if what i see on the screen was the underlying numerical value.

    I would say you are confusing content with presentation.

15. Well, not having used a spreadsheet in a while, I would have expected the underlying calculated value to be what gets displayed as the presented value. But, figured out what was wrong 🙂

16. This is not a bug.
    F2 != F4
    G2 != G4

    You can put some checks to make sure that your assumptions are correction. Just like assertions in source code.

    For example, to check if cell G4 and G6 are really equal, following check could be used.
    =IF(G4=G6;”SAME”;”DIFFERENT”)

    ———————–
    Also try changing precision (decimal places) used for display purpose.

    Tools -> Options –> OO.Calc -> View

    Also make sure to increase column widths to properly display higher precision values.

This site uses Akismet to reduce spam. Learn how your comment data is processed.