Unsigned Integer Literals?

[Magicks M]

x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua demo)
the number is cast to a float, is that supposed to happen?

I wrote the following helper function(s) to "tonumber" strings which may be
UInts, but I was wondering I could do it by a more direct method as
discussed in the manual:
local function by10(n, i) i = i or 1
for _ = 1, i do n = (n << 4) - (n << 2) - (n << 1) end
return n
end

function integer(s) s = s:reverse()
local n = 0
for i = 0, #s -1 do
n = n + by10(s:sub(i+1,i+1)|0, i)
end
return n
end

[Philippe Verdy]

First, you can easily avoid using s:reverse(), which creates a new string,
instead reverse the direction of the loop.
But in fact you can easily detect that a double is returned for
-4611686018427387904.0 (and only this integer value) and avoid the costly
loop performing custom multiplication by 10:

The integer (-4611686018427387904) is simply the expression (-
4611686018427387903 - 1), where all numbers are already integers in
expressions (you may also avoid the cost of computing this expression by
keeping it in a cache within the function closure

Normally a Lua compiler should be able to preevaluate that expression
because all items are constant. Usually (but not generally) the function
context does not override the unary and binary minus operators; there
should be a clean way to declare to the Lua interpret or compiler that it
should ignore the context and use only builtin operators, or a way to use
those buoting operators using their implied functional form under a global
non-overridable scope, using a formal path, but visibly all is made in Lua
to forbid any function to get a granted acces to the global scope, so it's
difficult to determine if the result of the expression (-
4611686018427387903 - 1) is really constant or if it has other side
effects, even if all its parameters and operator names look as constants;
the constant status is true for the two integers but not for the two
operators). Your attempt using these two functions are also unable to
assert that the expressions used will really be integers as it uses more
operators and some additional "global" functions supposed to work on
strings and retuirn strings, plus the "|0" construct which is supposed to
return an integer (but still not warrantied as "|" is also overridable by
the context).

There's no well-defined way in Lua to select specific types to use for
evaluating expressions using their attached method.

Post by Magicks M

x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua
demo) the number is cast to a float, is that supposed to happen?
I wrote the following helper function(s) to "tonumber" strings which may
be UInts, but I was wondering I could do it by a more direct method as
local function by10(n, i) i = i or 1
for _ = 1, i do n = (n << 4) - (n << 2) - (n << 1) end
return n
end
function integer(s) s = s:reverse()
local n = 0
for i = 0, #s -1 do
n = n + by10(s:sub(i+1,i+1)|0, i)
end
return n
end

[Philippe Verdy]

This would be easier if Lua (like Javascript which uses it for its "strict"
model) included the support of "directives" when defining a chunk of code
(between begin..end, or do..end, or repeat...until, or function()...end):
we should be able to have a pseudo-instruction at start of the bloc
specifying such directive as a string constant. E.g.

function getMinint()
"using lua.core";
return (-4611686018427387903 - 1)
end

Which would instruct to compile/interpret the code using only core Lua
definitions and ignoring any override in scope: this function would not be
collable at all in a context where such access is restricted, or that
function would throw an error("undefined") to the caller if access is
rejected. So "lua.core" (indicated in this directive) should be strictly
limited (and should not include any function of the standard library, not
even string, io, os, or math, but possibly the basic support to table
operators (excluding methods like setmetatable, sort, insert, add..., but
including the "#" and "[]" operators, and the "{}" table constructor, plus
some native metafunctions like __index and possibly __newindex).

Post by Philippe Verdy
First, you can easily avoid using s:reverse(), which creates a new string,
instead reverse the direction of the loop.
But in fact you can easily detect that a double is returned for
-4611686018427387904.0 (and only this integer value) and avoid the costly
The integer (-4611686018427387904) is simply the expression (-
4611686018427387903 - 1), where all numbers are already integers in
expressions (you may also avoid the cost of computing this expression by
keeping it in a cache within the function closure
Normally a Lua compiler should be able to preevaluate that expression
because all items are constant. Usually (but not generally) the function
context does not override the unary and binary minus operators; there
should be a clean way to declare to the Lua interpret or compiler that it
should ignore the context and use only builtin operators, or a way to use
those buoting operators using their implied functional form under a global
non-overridable scope, using a formal path, but visibly all is made in Lua
to forbid any function to get a granted acces to the global scope, so it's
difficult to determine if the result of the expression (-
4611686018427387903 - 1) is really constant or if it has other side
effects, even if all its parameters and operator names look as constants;
the constant status is true for the two integers but not for the two
operators). Your attempt using these two functions are also unable to
assert that the expressions used will really be integers as it uses more
operators and some additional "global" functions supposed to work on
strings and retuirn strings, plus the "|0" construct which is supposed to
return an integer (but still not warrantied as "|" is also overridable by
the context).
There's no well-defined way in Lua to select specific types to use for
evaluating expressions using their attached method.

Post by Magicks M

x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua
demo) the number is cast to a float, is that supposed to happen?
I wrote the following helper function(s) to "tonumber" strings which may
be UInts, but I was wondering I could do it by a more direct method as
local function by10(n, i) i = i or 1
for _ = 1, i do n = (n << 4) - (n << 2) - (n << 1) end
return n
end
function integer(s) s = s:reverse()
local n = 0
for i = 0, #s -1 do
n = n + by10(s:sub(i+1,i+1)|0, i)
end
return n
end

[Muh Muhten]

Post by Philippe Verdy
This would be easier if

Can your proposal be defined to solve more than one problem?

[Muh Muhten]

Post by Magicks M

x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua demo)
the number is cast to a float, is that supposed to happen?

Since this numeric constant has neither radix point nor exponent, and

Post by Magicks M
A numeric constant with a radix point or an exponent denotes a float; otherwise, if its value fits in an integer, it denotes an integer.

The claim in PIL is consistent with 5.3.0-2, but changed in 5.3.3. The
manual text in that section is the same. Of course, pre-5.3 versions
generally have the "new" behaviour due to using only floating-point
numbers.

[Philippe Verdy]

Post by Muh Muhten

Post by Magicks M

We can write constants larger than 2^63 -1 directly, despite
x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua

demo)

Post by Magicks M
the number is cast to a float, is that supposed to happen?

Since this numeric constant has neither radix point nor exponent, and

Post by Magicks M
A numeric constant with a radix point or an exponent denotes a float;

otherwise, if its value fits in an integer, it denotes an integer.
The claim in PIL is consistent with 5.3.0-2, but changed in 5.3.3. The
manual text in that section is the same. Of course, pre-5.3 versions
generally have the "new" behaviour due to using only floating-point
numbers.

But this integer exists ! The expression (-2147483647 - 1) uses only
integers within the positive subset, and produces an integer, even if its
negation would produce a double with the Lua operator. This is the only
integer (assuming integers are 32 bits) that has no distinct negation as an
integer.

If integers are compiled as 64-bit (normally the case on 64-bit target
architecture where the C "int" type is 64-bit), then the unique integer
without distinct negation is (-9223372036854775807-1).

You may still have not understood this issue.

[Muh Muhten]

Post by Philippe Verdy

Post by Muh Muhten

Post by Magicks M

We can write constants larger than 2^63 -1 directly, despite
x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua

demo)

Post by Magicks M
the number is cast to a float, is that supposed to happen?

Since this numeric constant has neither radix point nor exponent, and

Post by Magicks M
A numeric constant with a radix point or an exponent denotes a float;

otherwise, if its value fits in an integer, it denotes an integer.
The claim in PIL is consistent with 5.3.0-2, but changed in 5.3.3. The
manual text in that section is the same. Of course, pre-5.3 versions
generally have the "new" behaviour due to using only floating-point
numbers.

But this integer exists ! The expression (-2147483647 - 1) uses only
integers within the positive subset, and produces an integer, even if its
negation would produce a double with the Lua operator. This is the only
integer (assuming integers are 32 bits) that has no distinct negation as an
integer.
If integers are compiled as 64-bit (normally the case on 64-bit target
architecture where the C "int" type is 64-bit), then the unique integer
without distinct negation is (-9223372036854775807-1).
You may still have not understood this issue.

13835058055282163712 does not fit in a 64-bit integer.

[Magicks M]

Okay so basically what happens is the negative numbers are used to
represent more positive numbers, since there's the same amount of numbers
in each range it works. (we encode the extra positive numbers as negative
int64s if you like) The issue I have is the fact that the PiL section on
uints clearly shows 13835058055282163712 being interpreted as a int64 and
being represented as -4611686018427387904 which does not happen in my
interpreter I get 1.3835058055282e+019 this is not a negative integer.

x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

It does not describe what happens. I want to know if there's a reason this
is claimed to happen when in the demo, my interpreter and an ubuntu build
of lua, I get a float.
It should be noted that 3 << 62 does indeed produce the negative integer
you expect, and ("%u"):format(3 << 62) confirms that it is the number in
question.

Post by Philippe Verdy

Post by Muh Muhten

Post by Magicks M

We can write constants larger than 2^63 -1 directly, despite
x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua

demo)

Post by Magicks M
the number is cast to a float, is that supposed to happen?

Since this numeric constant has neither radix point nor exponent, and

Post by Magicks M
A numeric constant with a radix point or an exponent denotes a float;

otherwise, if its value fits in an integer, it denotes an integer.
The claim in PIL is consistent with 5.3.0-2, but changed in 5.3.3. The
manual text in that section is the same. Of course, pre-5.3 versions
generally have the "new" behaviour due to using only floating-point
numbers.

But this integer exists ! The expression (-2147483647 - 1) uses only
integers within the positive subset, and produces an integer, even if its
negation would produce a double with the Lua operator. This is the only
integer (assuming integers are 32 bits) that has no distinct negation as

an

Post by Philippe Verdy
integer.
If integers are compiled as 64-bit (normally the case on 64-bit target
architecture where the C "int" type is 64-bit), then the unique integer
without distinct negation is (-9223372036854775807-1).
You may still have not understood this issue.

13835058055282163712 does not fit in a 64-bit integer.

[Dirk Laurie]

Post by Muh Muhten

Post by Magicks M

x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua demo)
the number is cast to a float, is that supposed to happen?

The claim in PIL is consistent with 5.3.0-2, but changed in 5.3.3. The
manual text in that section is the same. Of course, pre-5.3 versions
generally have the "new" behaviour due to using only floating-point
numbers.

My "make linux" directly from source has it only in 5.3.4.

$ /usr/local/src/lua-5.3.3/src/lua
Lua 5.3.3 Copyright (C) 1994-2016 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

-4611686018427387904
$ /usr/local/src/lua-5.3.4/src/lua
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

1.3835058055282e+19

We argued this question ad nauseam on the lista few years ago. It
belongs to the category "What should Lua do if it can't do what you
ask?" There is no answer that satisfies everybody. The answer finally
selected serves the purpose of giving a visual clue that there has
been overflow.

Note that only decimal input is affected: hex is still the same;
numbers generated by arithmetic (except /) on integers are integers.

Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=0xc000000000000000; print(n)

-4611686018427387904

Personally I dont mind hex. I can't recognize numbers in written
decimal as powers of 2 when they go past 65536. I don't think the
readibilty of code is served by writing numbers like
13835058055282163712.

[Magicks M]

5.3.5 is also affected, I'm guessing 5.3.4 introduced this and it's a bug
then?

Post by Dirk Laurie

Post by Muh Muhten

Post by Magicks M

We can write constants larger than 2^63 -1 directly, despite
x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua

demo)

Post by Muh Muhten

Post by Magicks M
the number is cast to a float, is that supposed to happen?

The claim in PIL is consistent with 5.3.0-2, but changed in 5.3.3. The
manual text in that section is the same. Of course, pre-5.3 versions
generally have the "new" behaviour due to using only floating-point
numbers.

My "make linux" directly from source has it only in 5.3.4.
$ /usr/local/src/lua-5.3.3/src/lua
Lua 5.3.3 Copyright (C) 1994-2016 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

-4611686018427387904
$ /usr/local/src/lua-5.3.4/src/lua
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

1.3835058055282e+19
We argued this question ad nauseam on the lista few years ago. It
belongs to the category "What should Lua do if it can't do what you
ask?" There is no answer that satisfies everybody. The answer finally
selected serves the purpose of giving a visual clue that there has
been overflow.
Note that only decimal input is affected: hex is still the same;
numbers generated by arithmetic (except /) on integers are integers.
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=0xc000000000000000; print(n)

-4611686018427387904
Personally I dont mind hex. I can't recognize numbers in written
decimal as powers of 2 when they go past 65536. I don't think the
readibilty of code is served by writing numbers like
13835058055282163712.

[Dirk Laurie]

No. "Do not call something 'bug' that does not behave as you wanted or
expected. — Roberto"

It was a deliberate decision, after debate on the list. It is less
confusing for the type of user who does not quite understand two's
complements if decimal literals became floats rather than
underflowing, and those that do understand it also understand hex, so
they do not need decimal notation.

PiL 4 appeared in August 2016, before Lua got to 5.3.4, and anyway it
is not the User's Manual. You can't expect a once-per-version book to
be kept in sync with minor releases.

5.3.5 is also affected, I'm guessing 5.3.4 introduced this and it's a bug then?

Post by Dirk Laurie

Post by Muh Muhten

Post by Magicks M

x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the lua demo)
the number is cast to a float, is that supposed to happen?

The claim in PIL is consistent with 5.3.0-2, but changed in 5.3.3. The
manual text in that section is the same. Of course, pre-5.3 versions
generally have the "new" behaviour due to using only floating-point
numbers.

My "make linux" directly from source has it only in 5.3.4.
$ /usr/local/src/lua-5.3.3/src/lua
Lua 5.3.3 Copyright (C) 1994-2016 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

-4611686018427387904
$ /usr/local/src/lua-5.3.4/src/lua
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

1.3835058055282e+19
We argued this question ad nauseam on the lista few years ago. It
belongs to the category "What should Lua do if it can't do what you
ask?" There is no answer that satisfies everybody. The answer finally
selected serves the purpose of giving a visual clue that there has
been overflow.
Note that only decimal input is affected: hex is still the same;
numbers generated by arithmetic (except /) on integers are integers.
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=0xc000000000000000; print(n)

-4611686018427387904
Personally I dont mind hex. I can't recognize numbers in written
decimal as powers of 2 when they go past 65536. I don't think the
readibilty of code is served by writing numbers like
13835058055282163712.

[Magicks M]

Okay, I'm somewhat surprised that this was removed on purpose since it kind
of introduces a breaking change but thanks for explaining. I totally get
why this isn't something you'd necessarily want. If you could link to the
previous list on the archive that would be cool

Post by Dirk Laurie
No. "Do not call something 'bug' that does not behave as you wanted or
expected. — Roberto"
It was a deliberate decision, after debate on the list. It is less
confusing for the type of user who does not quite understand two's
complements if decimal literals became floats rather than
underflowing, and those that do understand it also understand hex, so
they do not need decimal notation.
PiL 4 appeared in August 2016, before Lua got to 5.3.4, and anyway it
is not the User's Manual. You can't expect a once-per-version book to
be kept in sync with minor releases.

Post by Magicks M
5.3.5 is also affected, I'm guessing 5.3.4 introduced this and it's a

bug then?

Post by Magicks M

Post by Dirk Laurie

Post by Muh Muhten

Post by Magicks M

We can write constants larger than 2^63 -1 directly, despite
x = 13835058055282163712 -- 3 << 62
x --> -4611686018427387904

When I enter this example into an interpreter (you can try in the

lua demo)

Post by Magicks M

Post by Dirk Laurie

Post by Muh Muhten

Post by Magicks M
the number is cast to a float, is that supposed to happen?

The claim in PIL is consistent with 5.3.0-2, but changed in 5.3.3. The
manual text in that section is the same. Of course, pre-5.3 versions
generally have the "new" behaviour due to using only floating-point
numbers.

My "make linux" directly from source has it only in 5.3.4.
$ /usr/local/src/lua-5.3.3/src/lua
Lua 5.3.3 Copyright (C) 1994-2016 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

-4611686018427387904
$ /usr/local/src/lua-5.3.4/src/lua
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

1.3835058055282e+19
We argued this question ad nauseam on the lista few years ago. It
belongs to the category "What should Lua do if it can't do what you
ask?" There is no answer that satisfies everybody. The answer finally
selected serves the purpose of giving a visual clue that there has
been overflow.
Note that only decimal input is affected: hex is still the same;
numbers generated by arithmetic (except /) on integers are integers.
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=0xc000000000000000; print(n)

-4611686018427387904
Personally I dont mind hex. I can't recognize numbers in written
decimal as powers of 2 when they go past 65536. I don't think the
readibilty of code is served by writing numbers like
13835058055282163712.

[Dirk Laurie]

Okay, I'm somewhat surprised that this was removed on purpose since it kind of introduces a breaking change but thanks for explaining. I totally get why this isn't something you'd necessarily want. If you could link to the previous list on the archive that would be cool

I would have to Google it with site:lua-users.org. Last time I did
that, I posted the result as a lmgtfy link and someone told me
off-list that it is considered politically incorrect to post lmgtfy
links, so I won't do it this time.

[Sean Conner]

Post by Dirk Laurie

Okay, I'm somewhat surprised that this was removed on purpose since it kind of introduces a breaking change but thanks for explaining. I totally get why this isn't something you'd necessarily want. If you could link to the previous list on the archive that would be cool

I would have to Google it with site:lua-users.org. Last time I did
that, I posted the result as a lmgtfy link and someone told me
off-list that it is considered politically incorrect to post lmgtfy
links, so I won't do it this time.

Besides, there are those of us who keep a full archive of the email list
who will ocasionally do searches for previous threads. This one here
appears to be the thread Dirk alluded to:

http://lua-users.org/lists/lua-l/2017-02/msg00302.html

And here's the message Roberto sent that gives the last word on the
matter:

http://lua-users.org/lists/lua-l/2017-02/msg00314.html

-spc (Currently stuck on a production deployment that's going long ... )

[Jay Carlson]

Post by Dirk Laurie
I would have to Google it with site:lua-users.org. Last time I did
that, I posted the result as a lmgtfy link and someone told me
off-list that it is considered politically incorrect to post lmgtfy
links, so I won't do it this time.

LMGTFY has negative connotations, but there is benefit in learning how other people searched for something. We built qggl.co as a fun (and DNS-abusive) way for people on forums to show their searches:

lua-users.jay-carlson.printf.qggl.it

Some sites will linkify URL-like things, but often .it (Italy) is not on the TLD list. In that case, you either need a leading

http://lua-users.jay-carlson.printf.qggl.it

or sometimes the "www." prefix will cue linkification:

www.lua-users.jay-carlson.printf.qggl.it

[Philippe Verdy]

Post by Dirk Laurie
$ /usr/local/src/lua-5.3.3/src/lua
Lua 5.3.3 Copyright (C) 1994-2016 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

-4611686018427387904

[...]

Post by Dirk Laurie
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=0xc000000000000000; print(n)

-4611686018427387904

Personally I call these two examples "bugs" because they generate the
incorrect values.

Lua treats numbers only (there's a single exposed type), and is supposed to
rewrite integers using doubles only if they don't fit the expected limited
range, but should NEVER underflow.

Because it confuses the treatment of doubles as well without being even
consistant with integers. Integers in Lua should be internal optimization
of doubles, each time this is possible. It should not depend on the fact we
use a ".0" or exponential notation or none when writing constants or
serializing them as strings and parsing strings.

[Dirk Laurie]

Post by Dirk Laurie
$ /usr/local/src/lua-5.3.3/src/lua
Lua 5.3.3 Copyright (C) 1994-2016 Lua.org, PUC-Rio

Post by Muh Muhten
n=13835058055282163712; print(n)

-4611686018427387904

[...]

Post by Dirk Laurie
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio

Post by Muh Muhten
n=0xc000000000000000; print(n)

-4611686018427387904

Personally I call these two examples "bugs" because they generate the incorrect values.

You could argue that the Lua designers thought so too, because:
1. They changed it in te next minor release.
2. Minor releases are bugfix releases.

[Jim]

having not only unsigned integers but also the corresponding data type
could be a good idea if one has to do integer arithmetic in that range
since the bits representing negative integers would be used for
positive integers and thus the possible range for pure integer
arithmetic is extended.

[Roberto Ierusalimschy]

Post by Jim
having not only unsigned integers but also the corresponding data type
could be a good idea if one has to do integer arithmetic in that range
since the bits representing negative integers would be used for
positive integers and thus the possible range for pure integer
arithmetic is extended.

You can go a long way using unsigned integers in Lua 5.3. For equality,
addition, subtraction, and multiplication, the standard operations do
the job. For constants, you better use hexadecimal. For printing, "%u"
does the job. For order, 'math.ult' does unsigned comparisons. Division
and conversions between floats and unsigned needs some more work, but
can be done. (For instance, "x % 2^64" converts an unsigned integer 'x'
to float.)

PiL 4 has an entire section (13.2) dedicated to unsigned integers
(with the caveat about the change in decimal literals).

-- Roberto