Comments on: Quirky behavior of oocalc

By: vcp

vcp — Thu, 16 Oct 2008 10:08:23 +0000

This is not a bug.
F2 != F4
G2 != G4

You can put some checks to make sure that your assumptions are correction. Just like assertions in source code.

For example, to check if cell G4 and G6 are really equal, following check could be used.
=IF(G4=G6;”SAME”;”DIFFERENT”)

———————–
Also try changing precision (decimal places) used for display purpose.

Tools -> Options –> OO.Calc -> View

Also make sure to increase column widths to properly display higher precision values.

By: sankarshan

sankarshan — Thu, 16 Oct 2008 06:25:23 +0000

Well, not having used a spreadsheet in a while, I would have expected the underlying calculated value to be what gets displayed as the presented value. But, figured out what was wrong

By: Tarique Sani

Tarique Sani — Thu, 16 Oct 2008 04:36:13 +0000

I would be very pissed if what i see on the screen was the underlying numerical value.

I would say you are confusing content with presentation.

By: sankarshan

sankarshan — Wed, 15 Oct 2008 23:28:36 +0000

Nope you are not. As a comment states, it is best to use round().

By: sankarshan

sankarshan — Wed, 15 Oct 2008 23:27:50 +0000

Isn’t what I see on the screen supposed to be the underlying numerical value ?

By: smeagol

smeagol — Wed, 15 Oct 2008 21:08:48 +0000

The result is not accurate, 358 / 380 = 0,942105263…

By: noName

noName — Wed, 15 Oct 2008 17:06:34 +0000

This is not an Error: 358/380=0.942105263 not 0.94!
Ckeck your cell formatting.

By: John Poelstra

John Poelstra — Wed, 15 Oct 2008 16:10:44 +0000

It is not broken.

Expand the formatting for all of the values in column ‘F’ to multiple decimal places… say, 10

It is the formatting of the cells and the number decimal places… a common gotcha with spreadsheets where calculated values are used in subsequent computations and then compared to hand entered values with less decimal places..

What you see on the screen isn’t exactly what the underlying numerical value is.

By: Isak Savo

Isak Savo — Wed, 15 Oct 2008 15:51:26 +0000

I don’t see the problem. Yes you get different values, but that’s because your calculated cells (the ones with forumlas) contains higher precision than the ones you entered yourself. Try changing the formatting of the cells and you’ll see the difference.

I did the calculations on a standard calculator and found no errors…

or am I missing something?

By: markrian

markrian — Wed, 15 Oct 2008 15:48:12 +0000

You’ve simply lost precision. Calc/Spreadsheets carry more precision in their calculations than may be displayed in the cells.

Select the column G and click the button to add decimal places a few times. What you copied down is not exactly the same number as was calculated.