Data modified when passing to C from Ruby using NUM2LL and NUM2INT -
i have c module testing ruby test extending ruby piece of c code. number above 2 ** 24 -1 being modified. need pass 64-bit values , forth between ruby , c , should bit-precise. thoughts appreciated.
in c extension: long int sig_val; sig_val = num2ll(sig) // sig type value , ruby number passed on c
. . . few example of data passed:
- (2^34 -1) * 16 ==> c receives (2^34) * 16
- (2^34 +1) * 16 ==> c receives (2^34) * 16
- (2^24 +1) * 16 ==> c receives (2^24) * 16
- (2^25 -1) ==> c receives (2^25)
- (2^24 -1) ==> c receives (2^24 -1) (correct)
any number below correct.
thank you!
copying answer comments in order remove question "unanswered" filter:
i realized in code, there typecast float causes ruby limit number of bits 24 (as per single point ieee754 guess). can pass 64-bit wide numbers using num2ll , ll2num. thank you
~ answer per aalavi
Comments
Post a Comment