Or just change the way you calculate to get at the level of accuracy
that you want/need:
irb> ("291.15".to_f * 1000.0).to_i/10
=> 29115
Jeff
On Dec 22, 7:41 pm, Marnen Laibow-Koser <li...@ruby-forum.com> wrote:
> TomRossi7 wrote in post #970207:
>
> > Any idea why this calculates the integer the way it does?
>
> > irb> ("291.15".to_f * 100.0).to_i
> > => 29114
>
> Because of the vagaries of floating-point arithmetic:
> irb> ("291.15".to_f * 100.0) < 29115
> => true
> irb> printf "%.30f", ("291.15".to_f * 100.0)
> 29114.999999999996362021192908287048
>
> It bears repeating: never, ever use Floats for arithmetic. If you can't
> use fixed-point math, then use BigDecimal.
>
>
>
> > Thanks,
> > Tom
>
> Best,
> --
> Marnen Laibow-Koserhttp://www.marnen.org
> mar...@marnen.org
>
> --
> Posted viahttp://www.ruby-forum.com/.
--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
No comments:
Post a Comment