Number – Talk

From Bohemia Interactive Community
Jump to navigation Jump to search
(precision?)
m (Text replacement - "<code>" to "<code style="display: block">")
 
(7 intermediate revisions by 6 users not shown)
Line 1: Line 1:
I delete a reference to [[Integer]], because there is no such type in OFP scripting.
I delete a reference to ''integer'', because there is no such type in OFP scripting.


--[[User:Suma|Suma]] 11:36, 30 June 2006 (CEST)
--[[User:Suma|Suma]] 11:36, 30 June 2006 (CEST)
----
----
Is anyone sure on the precision at which numbers can be stored as variables? As a non-programmer, I find this behaviour quite strange.
Is anyone sure on the precision to which numbers can be stored as variables?


OFP example : var1 = 123456789
OFP example :
  var1 = var1 - 100000000
  var1 = 123456789
var1 = var1 - 20000000
  var1 = var1 - 123000000
  var1 = var1 - 3000000
  var1 will return 456'''792'''
  var1 will return 456'''792'''
As a non-programmer, I find this behaviour quite strange. --[[User:Ceeeb|Ceeeb]] 08:03, 15 February 2007 (CET)
----
A test script:<br>
player globalchat format["Precision Test: %1, %2, %3, %4, %5, %6, %7, %8, %9", 1234567 - 1234566, 1234567, 999998+1, 999998+2, 999998+3, (9999999+6)-(10999999), (10000000)-(10999999), 9999999/2 - 4999998, 9999999/2 - 4999998.4];<br>
Which gives:<br>
PRECISION TEST: 1, 1.23457E+006, 999999, 1E+006, 1E+006, -999994, -999999, 1.5, 1<br>
This seems to suggest that you can only store a maximum value of 999999 in a variable.  (6 digits)<br>
But, you can use up to 8 digits in integer equations, provided the output is 6 digits or less.<br>
Please do not take the above two statements as fact, since I have done only a few hours of limited testing.
--[[User:hellop|hellop]] 09:30, 11 July 2008 (PST)
== Precision loss ==
There are problems with adding big numbers, for example (16 777 216  + 1) - 16 777 216 = 0 but it is 1... so you loss precision...
The result describe from which number it is not possible to add without loss precision.
<code style="display: block">_value = 0;
_array = [100000, 10000, 1000, 100, 10, 1];
for "_i" from 1 to 1000000000 step 1000000 do
{
{
_value = (_i + _x) - _i;
if (_value != _x) exitWith
{
for "_k" from (_i - 1000000) to _i step _x do
{
_value = (_k + _x) - _k;
if (_value != _x) exitWith {diag_log format["precision loss from %1 with adding %2", [_k] call BIS_fnc_numberText, _x]};
};
_array = _array - [_x];
};
}
forEach _array;
};</code>
Result:
# precision loss from 16 777 216 with adding 1
# precision loss from 33 554 440 with adding 10
# precision loss from 67 108 800 with adding 100
# precision loss from 134 218 000 with adding 1000
# precision loss from 268 440 000 with adding 10000
# precision loss from 536 800 000 with adding 100000
--[[User:Will|Will]] ([[User talk:Will|talk]]) 13:56, 28 May 2016 (CEST)

Latest revision as of 11:53, 11 January 2023

I delete a reference to integer, because there is no such type in OFP scripting.

--Suma 11:36, 30 June 2006 (CEST)


Is anyone sure on the precision to which numbers can be stored as variables?

OFP example :

var1 = 123456789
var1 = var1 - 123000000
var1 will return 456792

As a non-programmer, I find this behaviour quite strange. --Ceeeb 08:03, 15 February 2007 (CET)


A test script:
player globalchat format["Precision Test: %1, %2, %3, %4, %5, %6, %7, %8, %9", 1234567 - 1234566, 1234567, 999998+1, 999998+2, 999998+3, (9999999+6)-(10999999), (10000000)-(10999999), 9999999/2 - 4999998, 9999999/2 - 4999998.4];

Which gives:
PRECISION TEST: 1, 1.23457E+006, 999999, 1E+006, 1E+006, -999994, -999999, 1.5, 1

This seems to suggest that you can only store a maximum value of 999999 in a variable. (6 digits)
But, you can use up to 8 digits in integer equations, provided the output is 6 digits or less.
Please do not take the above two statements as fact, since I have done only a few hours of limited testing.

--hellop 09:30, 11 July 2008 (PST)

Precision loss

There are problems with adding big numbers, for example (16 777 216 + 1) - 16 777 216 = 0 but it is 1... so you loss precision... The result describe from which number it is not possible to add without loss precision.

_value = 0; _array = [100000, 10000, 1000, 100, 10, 1]; for "_i" from 1 to 1000000000 step 1000000 do { { _value = (_i + _x) - _i; if (_value != _x) exitWith { for "_k" from (_i - 1000000) to _i step _x do { _value = (_k + _x) - _k; if (_value != _x) exitWith {diag_log format["precision loss from %1 with adding %2", [_k] call BIS_fnc_numberText, _x]}; }; _array = _array - [_x]; }; } forEach _array; };

Result:

  1. precision loss from 16 777 216 with adding 1
  2. precision loss from 33 554 440 with adding 10
  3. precision loss from 67 108 800 with adding 100
  4. precision loss from 134 218 000 with adding 1000
  5. precision loss from 268 440 000 with adding 10000
  6. precision loss from 536 800 000 with adding 100000

--Will (talk) 13:56, 28 May 2016 (CEST)