Number – Talk
(precision?) |
Lou Montana (talk | contribs) m (Text replacement - "<code>" to "<code style="display: block">") |
||
(7 intermediate revisions by 6 users not shown) | |||
Line 1: | Line 1: | ||
I delete a reference to | I delete a reference to ''integer'', because there is no such type in OFP scripting. | ||
--[[User:Suma|Suma]] 11:36, 30 June 2006 (CEST) | --[[User:Suma|Suma]] 11:36, 30 June 2006 (CEST) | ||
---- | ---- | ||
Is anyone sure on the precision | Is anyone sure on the precision to which numbers can be stored as variables? | ||
OFP example : | |||
var1 = | var1 = 123456789 | ||
var1 = var1 - 123000000 | |||
var1 = var1 - | |||
var1 will return 456'''792''' | var1 will return 456'''792''' | ||
As a non-programmer, I find this behaviour quite strange. --[[User:Ceeeb|Ceeeb]] 08:03, 15 February 2007 (CET) | |||
---- | |||
A test script:<br> | |||
player globalchat format["Precision Test: %1, %2, %3, %4, %5, %6, %7, %8, %9", 1234567 - 1234566, 1234567, 999998+1, 999998+2, 999998+3, (9999999+6)-(10999999), (10000000)-(10999999), 9999999/2 - 4999998, 9999999/2 - 4999998.4];<br> | |||
Which gives:<br> | |||
PRECISION TEST: 1, 1.23457E+006, 999999, 1E+006, 1E+006, -999994, -999999, 1.5, 1<br> | |||
This seems to suggest that you can only store a maximum value of 999999 in a variable. (6 digits)<br> | |||
But, you can use up to 8 digits in integer equations, provided the output is 6 digits or less.<br> | |||
Please do not take the above two statements as fact, since I have done only a few hours of limited testing. | |||
--[[User:hellop|hellop]] 09:30, 11 July 2008 (PST) | |||
== Precision loss == | |||
There are problems with adding big numbers, for example (16 777 216 + 1) - 16 777 216 = 0 but it is 1... so you loss precision... | |||
The result describe from which number it is not possible to add without loss precision. | |||
<code style="display: block">_value = 0; | |||
_array = [100000, 10000, 1000, 100, 10, 1]; | |||
for "_i" from 1 to 1000000000 step 1000000 do | |||
{ | |||
{ | |||
_value = (_i + _x) - _i; | |||
if (_value != _x) exitWith | |||
{ | |||
for "_k" from (_i - 1000000) to _i step _x do | |||
{ | |||
_value = (_k + _x) - _k; | |||
if (_value != _x) exitWith {diag_log format["precision loss from %1 with adding %2", [_k] call BIS_fnc_numberText, _x]}; | |||
}; | |||
_array = _array - [_x]; | |||
}; | |||
} | |||
forEach _array; | |||
};</code> | |||
Result: | |||
# precision loss from 16 777 216 with adding 1 | |||
# precision loss from 33 554 440 with adding 10 | |||
# precision loss from 67 108 800 with adding 100 | |||
# precision loss from 134 218 000 with adding 1000 | |||
# precision loss from 268 440 000 with adding 10000 | |||
# precision loss from 536 800 000 with adding 100000 | |||
--[[User:Will|Will]] ([[User talk:Will|talk]]) 13:56, 28 May 2016 (CEST) |
Latest revision as of 11:53, 11 January 2023
I delete a reference to integer, because there is no such type in OFP scripting.
--Suma 11:36, 30 June 2006 (CEST)
Is anyone sure on the precision to which numbers can be stored as variables?
OFP example :
var1 = 123456789 var1 = var1 - 123000000 var1 will return 456792
As a non-programmer, I find this behaviour quite strange. --Ceeeb 08:03, 15 February 2007 (CET)
A test script:
player globalchat format["Precision Test: %1, %2, %3, %4, %5, %6, %7, %8, %9", 1234567 - 1234566, 1234567, 999998+1, 999998+2, 999998+3, (9999999+6)-(10999999), (10000000)-(10999999), 9999999/2 - 4999998, 9999999/2 - 4999998.4];
Which gives:
PRECISION TEST: 1, 1.23457E+006, 999999, 1E+006, 1E+006, -999994, -999999, 1.5, 1
This seems to suggest that you can only store a maximum value of 999999 in a variable. (6 digits)
But, you can use up to 8 digits in integer equations, provided the output is 6 digits or less.
Please do not take the above two statements as fact, since I have done only a few hours of limited testing.
--hellop 09:30, 11 July 2008 (PST)
Precision loss
There are problems with adding big numbers, for example (16 777 216 + 1) - 16 777 216 = 0 but it is 1... so you loss precision... The result describe from which number it is not possible to add without loss precision.
_value = 0;
_array = [100000, 10000, 1000, 100, 10, 1];
for "_i" from 1 to 1000000000 step 1000000 do
{
{
_value = (_i + _x) - _i;
if (_value != _x) exitWith
{
for "_k" from (_i - 1000000) to _i step _x do
{
_value = (_k + _x) - _k;
if (_value != _x) exitWith {diag_log format["precision loss from %1 with adding %2", [_k] call BIS_fnc_numberText, _x]};
};
_array = _array - [_x];
};
}
forEach _array;
};
Result:
- precision loss from 16 777 216 with adding 1
- precision loss from 33 554 440 with adding 10
- precision loss from 67 108 800 with adding 100
- precision loss from 134 218 000 with adding 1000
- precision loss from 268 440 000 with adding 10000
- precision loss from 536 800 000 with adding 100000