ruby on rails - Converting string to float or decimal sometimes results in long precision -


i have string want convert decimal.
eg.

tax_rate = "0.07" tax_rate.to_d

in cases converts 0.07 converts 0.07000000000000001. why?

if helps, when @ log of when insert value db, relevant part:

["tax_rate", #<bigdecimal:7f9b6221d7c0,'0.7000000000 000001e-1',18(45)>]

hopefully makes sense someone.

disclaimer:
have feeling going ask why i'm doing this.
i've created simple settings model, user can update global settings.
setting model has name, value , var_type columns.
value column string. use helper method retrieve value in appropriate format depending on value of var_type column.

i cannot explain why there chance can tell how avoid having kind of trouble when dealing numbers: use rationals.

here documentation: http://ruby-doc.org/core-1.9.3/rational.html

as stated, rational give exact number want , avoid rounding errors.

from doc:

10.times.inject(0){|t,| t + 0.1}              #=> 0.9999999999999999 10.times.inject(0){|t,| t + rational('0.1')}  #=> (1/1)

let me know if solves problem. : )


Comments

Post a Comment

Popular posts from this blog

linux - Does gcc have any options to add version info in ELF binary file? -

i mean whether gcc can insert source code version infor elf binary section or similar. not want change source file, add info gcc option in makefile. you can emit version info text file, turn text file object file statically link executable. the first step simple have write code: script or write version info in format plain text file. write makefile rule produce version.o version.txt, using objcopy . you'll have object file 2 useful symbols defined in it: beginning , end of textual version info. add generated object executable, , you'll able access version 2 ways: running strings on binary, or writing code in application print version string (you'll need declare start , end symbols variables in header file).
Read more

java - Are there any classes that implement javax.persistence.Parameter<T>? -

i'm using named queries fetch entities database using jpa , hibernate , came across puzzling me: how come there no default implementations of javax.persistence.parameter interface? i rather impressed how @staticmetamodel used criteriabuilder in jpa2 meaning type safety , don't have mess around setting string names. wanted apply similar of jpa1 ejb's i'm working on: dealergroup_.class: public dealergroup_ { public parameter<integer> id = new parameterimpl<integer>("id"); /// etc... } dealergroupfacade.class: // ... public dealergroup fetch(integer id) { typedquery<dealergroup> query = em.createnamedquery("dealergroup.fetchbyid", dealergroup.class); query.setparameter(dealergroup_.id, query); return query.getsingleresult(); } // ... i surprised find there no implementations of parameter interface - of course, implement myself absence makes me suspicious may mistake using in way? should implementing parameter...
Read more

android - send complex objects as post php java -

is possible send contact numbers via post php class inside eclipse ? or post limited primitive types , have find work arounds send them string ? i propose: send them json in post request, befor start, have @ gson project , keep in mind set content application -> json. luck
Read more