Modify

Opened 10 years ago

Closed 10 years ago

#2810 closed Bug (No Bug)

handling of some float numbers

Reported by: anonymous Owned by:
Milestone: Component: AutoIt
Version: 3.3.12.0 Severity: None
Keywords: Cc:

Description

wierd internal rounding.

$n1=0.9999999999999995
$n2=0.9999999999999996
$n3=0.99999999999999994449919099999
$n4=0.999999999999999944499191
ConsoleWrite('IsInt($n1)='&IsInt($n1)&'; $n1='&$n1&@cr) ;IsInt($n1)=0; $n1=0.999999999999999
ConsoleWrite('IsInt($n2)='&IsInt($n2)&'; $n2='&$n2&@cr) ;IsInt($n2)=0; $n2=1
ConsoleWrite('IsInt($n3)='&IsInt($n3)&'; $n3='&$n3&@cr) ;IsInt($n3)=0; $n3=1
ConsoleWrite('IsInt($n4)='&IsInt($n4)&'; $n4='&$n4&@cr) ;IsInt($n4)=1; $n4=1

Attachments (0)

Change History (1)

comment:1 Changed 10 years ago by jchd18

  • Resolution set to No Bug
  • Status changed from new to closed

Again there is no bug here.

You can invent an infinite number of cases where the result of a floating point calculation differs from what you'd expect. Again it has nothing to do with AutoIt, which uses the relies on C++ compiler to provide "correct" results, "correct" being taken modulo what IEEE754 dictates. Please read http://en.wikipedia.org/wiki/Floating_point

IsInt($n) = 0 applied to a floating point value $n means that $n is not exactly represented as an integer inside the double.

Now, displaying $n by default conversion to string implies rounding to a fix number of decimals, which is not based on the machine epsilon (http://en.wikipedia.org/wiki/Machine_epsilon), nor on the ULP nearby (http://en.wikipedia.org/wiki/Unit_in_the_last_place).

Would you seriously want the following to display anything else than 0.1?

$n = 1/10
ConsoleWrite('Hex($n) = ' & Hex($n) & '; IsInt($n) = ' & IsInt($n) & '; $n = ' & $n & @LF)

Guidelines for posting comments:

  • You cannot re-open a ticket but you may still leave a comment if you have additional information to add.
  • In-depth discussions should take place on the forum.

For more information see the full version of the ticket guidelines here.

Add Comment

Modify Ticket

Action
as closed The ticket will remain with no owner.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.