Bug - but mine or ESPbasic's?!
Posted: Mon Sep 12, 2016 12:59 pmOK - as an exercise I wrote a simple program to convert a hex string to an integer. (Yes, I know there's an internal function to do this!) The program seemed to run fine, but I started getting some odd results. This is the code:
No, it's not the neatest but, like I say, it was an exercise to test various functions.
Anyway, all seemed to go well; put in FABC and get 64188. Put in FABCD and get 1027021.
Put in FABCDEF and get 262917616 - oops! That should be 262917615.
Problem is, put in anything between FABCDE9 and FABCDEF and you get the same result. Put in anything between FABCDE0 and FABCDE8 and you get 262917600 for each.
Because I thought my logic might be screwy I put in the line
Now, maybe there's a limit on integer values that was unaware of, but in this case, why did the answer 'flip up' by 16 at the 8/9 point?
Anyone got any thoughts?
Phil
Code: Select all
instr$ = ""
do
do
if serial.available() > 0 then instr$ = instr$ & serial.read.chr()
loop until right(instr$,1) = chr(13)
if left(instr$, 1) = "*" then
serialflush
end
else
instr$ = left(instr$, len(instr$) - 1)
end if
gosub [strtohex]
print "Value = " & str(rval)
loop while rval <> 1
[strtohex]
rval = 0
if left(instr$, 2) = "0x" then instr$ = mid(instr$, 3, 99)
instr$ = upper(instr$)
for i = 1 to len(instr$)
t = asc(mid(instr$, i, 1))
if t > 57 then
t = t - 55
else
t = t - 48
end if
rval = rval * 16 + t
next i
instr$ = ""
return
No, it's not the neatest but, like I say, it was an exercise to test various functions.
Anyway, all seemed to go well; put in FABC and get 64188. Put in FABCD and get 1027021.
Put in FABCDEF and get 262917616 - oops! That should be 262917615.
Problem is, put in anything between FABCDE9 and FABCDEF and you get the same result. Put in anything between FABCDE0 and FABCDE8 and you get 262917600 for each.
Because I thought my logic might be screwy I put in the line
Code: Select all
immediately before the gosub - and got exactly the same result with that! print "HexToInt gives " & hextoint(instr$)Now, maybe there's a limit on integer values that was unaware of, but in this case, why did the answer 'flip up' by 16 at the 8/9 point?
Anyone got any thoughts?
Phil