Proposal: Trailing comma in function calls

[Axel Kittenberger]

Lua accepts since ever (as far I know) trailing commas in table definitions.

And this is good, since like most people experience sooner or later, when
doing multilined stuff taking care of the last element is kinda a hassle.

However, I never understood, why trailing commas are not allowed in
function calls.

Reasons are the same, sometimes a function call spawns several lines. This
happens for me especially with functions taking variable length arguments
like print() or write()

For example:

print(
'this is a string',
'another string',
'yet another string', -- note the comma here
)

I don't see any ambiguity added by this, or is there?

Patch is simple, maybe this one is a little naive, since explist() is
called on various places. But works as far I tested*

--- lua-5.2.3/src/lparser.c 2013-04-12 20:48:47.000000000 +0200
+++ lua-5.2.3-comma/src/lparser.c 2014-06-04 16:24:35.958824924 +0200
@@ -812,6 +812,7 @@
int n = 1; /* at least one expression */
expr(ls, v);
while (testnext(ls, ',')) {
+ if( ls->t.token == ')' ) break;
luaK_exp2nextreg(ls->fs, v);
expr(ls, v);
n++;

Kind regards, Axel

* test suite for available for 5.2.2 hangs my computer with extreme memory
usage at the same position "testing strings and string library" with or
without patch, dunno if my system is just too weak for it, I too impatient
or I'm doing something wrong.

[Javier Guerra Giraldez]

However, I never understood, why trailing commas are not allowed in function
calls.

I'd like to see that too.

(or to know if there's any reason behind it)

[Thiago L.]

Post by Axel Kittenberger
Lua accepts since ever (as far I know) trailing commas in table definitions.
And this is good, since like most people experience sooner or later,
when doing multilined stuff taking care of the last element is kinda a
hassle.
However, I never understood, why trailing commas are not allowed in
function calls.
Reasons are the same, sometimes a function call spawns several lines.
This happens for me especially with functions taking variable length
arguments like print() or write()
print(
'this is a string',
'another string',
'yet another string', -- note the comma here
)
I don't see any ambiguity added by this, or is there?
Patch is simple, maybe this one is a little naive, since explist() is
called on various places. But works as far I tested*
--- lua-5.2.3/src/lparser.c2013-04-12 20:48:47.000000000 +0200
+++ lua-5.2.3-comma/src/lparser.c2014-06-04 16:24:35.958824924 +0200
@@ -812,6 +812,7 @@
int n = 1; /* at least one expression */
expr(ls, v);
while (testnext(ls, ',')) {
+ if( ls->t.token == ')' ) break;
luaK_exp2nextreg(ls->fs, v);
expr(ls, v);
n++;
Kind regards, Axel
* test suite for available for 5.2.2 hangs my computer with extreme
memory usage at the same position "testing strings and string library"
with or without patch, dunno if my system is just too weak for it, I
too impatient or I'm doing something wrong.

That just drops the comma no? I think something like
"print(somemultireturnfunc(),)" should do the same as
"print((somemultireturnfunc()))" instead...

[Hisham]

Post by Thiago L.
That just drops the comma no? I think something like
"print(somemultireturnfunc(),)" should do the same as
"print((somemultireturnfunc()))" instead...

The presence of an extra comma would leave me second-guessing the
behavior on multiple returns too. In any case, it should do the same
as the table constructor. Went to check what the table constructor
does:

Lua 5.1.2 Copyright (C) 1994-2007 Lua.org, PUC-Rio

Post by Thiago L.
function f() return 1, 2 end
t = { f() }
print(t[1], t[2])

1 2

Post by Thiago L.
u = { f(), }
print(u[1], u[2])

1 2

Post by Thiago L.
v = { f(), 3 }
print(v[1], v[2])

1 3
So, yes, if function calls were to accept a trailing comma, I think
they should just drop the comma and not impose special behavior on
multiple returns, for consistency with the table constructor.

-- Hisham

[Thiago L.]

Post by Hisham

Post by Thiago L.
That just drops the comma no? I think something like
"print(somemultireturnfunc(),)" should do the same as
"print((somemultireturnfunc()))" instead...

The presence of an extra comma would leave me second-guessing the
behavior on multiple returns too. In any case, it should do the same
as the table constructor. Went to check what the table constructor
Lua 5.1.2 Copyright (C) 1994-2007 Lua.org, PUC-Rio

Post by Thiago L.
function f() return 1, 2 end
t = { f() }
print(t[1], t[2])

1 2

Post by Thiago L.
u = { f(), }
print(u[1], u[2])

1 2

Post by Thiago L.
v = { f(), 3 }
print(v[1], v[2])

1 3
So, yes, if function calls were to accept a trailing comma, I think
they should just drop the comma and not impose special behavior on
multiple returns, for consistency with the table constructor.
-- Hisham

But code golfing... :(

[Dirk Laurie]

Post by Axel Kittenberger
Lua accepts since ever (as far I know) trailing commas in table definitions.
And this is good, since like most people experience sooner or later, when
doing multilined stuff taking care of the last element is kinda a hassle.
However, I never understood, why trailing commas are not allowed in function
calls.
Reasons are the same, sometimes a function call spawns several lines. This
happens for me especially with functions taking variable length arguments
like print() or write()
print(
'this is a string',
'another string',
'yet another string', -- note the comma here
)
I don't see any ambiguity added by this, or is there?

More generally, "lexical void" inside an argument list could mean nil.
That is something that I (when in a bikeshedding mood) have sometimes
missed, with some library functions where I use seldom use the early
arguments, e.g.

load(source,,,MY_ENV)

But now, in your example, I don't suppose you want

print(
'this is a string',
'another string',
'yet another string', -- note the comma here
)

to mean

print('this is a string', 'another string', 'yet another string', nil )

or would you?

[Tom N Harris]

Post by Dirk Laurie
More generally, "lexical void" inside an argument list could mean nil.
That is something that I (when in a bikeshedding mood) have sometimes
missed, with some library functions where I use seldom use the early
arguments, e.g.
load(source,,,MY_ENV)

I like this. And on the left side too.

first,,third = something()

Assigns the first and third values while dropping the second without having to
assign a dummy variable.

Post by Dirk Laurie
But now, in your example, I don't suppose you want
to mean
print('this is a string', 'another string', 'yet another string', nil )
or would you?

You can have both. The formal definition of an unnamed element in a comma list
is "nothing". That is,

print() -- is not the same as
print(nil) -- thus
print(,) -- is not the same as
print(nil,nil)

It's not that you're ignoring the last comma when there's no variable after
it, you're implying the last comma after a variable name. Then the number of
arguments to the function is equal to the number of commas.

[Luiz Henrique de Figueiredo]

Post by Dirk Laurie
load(source,,,MY_ENV)
first,,third = something()

The idiom in these cases would be

load(source,_,_,MY_ENV)
first,_,third = something()

Perhaps it'd be helpful to have a second predeclared local named "_",
as a cheap throw-away variable...

[Mike Nelson]

Post by Luiz Henrique de Figueiredo

Post by Dirk Laurie
load(source,,,MY_ENV)
first,,third = something()

The idiom in these cases would be
load(source,_,_,MY_ENV)
first,_,third = something()
Perhaps it'd be helpful to have a second predeclared local named "_",
as a cheap throw-away variable...

Rather like Go, where _ is predeclared for the same purpose (would be
surprised if they didn't get the idea from Lua). What does everyone
think about also outlawing _ as an rvalue, as Go does?
This would firmly establish _ as a throw-away variable, but is it worth
the internal complexity to enforce it? Perhaps the middle way of
predeclaring but not enforcing is best.

[Elias Barrionovo]

What does everyone think about also outlawing _ as an rvalue, as Go does?

It would go against the "mechanisms, not policies" mantra of the language.

[Coda Highland]

Post by Mike Nelson

Post by Luiz Henrique de Figueiredo

Post by Dirk Laurie
load(source,,,MY_ENV)
first,,third = something()

The idiom in these cases would be
load(source,_,_,MY_ENV)
first,_,third = something()
Perhaps it'd be helpful to have a second predeclared local named "_",
as a cheap throw-away variable...

Rather like Go, where _ is predeclared for the same purpose (would be
surprised if they didn't get the idea from Lua). What does everyone think
about also outlawing _ as an rvalue, as Go does?
This would firmly establish _ as a throw-away variable, but is it worth the
internal complexity to enforce it? Perhaps the middle way of predeclaring
but not enforcing is best.

The first of Luiz's examples uses _ as an rvalue.

/s/ Adam

[Philipp Janda]

Post by Coda Highland

Post by Mike Nelson

Post by Luiz Henrique de Figueiredo

Post by Dirk Laurie
load(source,,,MY_ENV)
first,,third = something()

The idiom in these cases would be
load(source,_,_,MY_ENV)
first,_,third = something()
Perhaps it'd be helpful to have a second predeclared local named "_",
as a cheap throw-away variable...

Rather like Go, where _ is predeclared for the same purpose (would be
surprised if they didn't get the idea from Lua). What does everyone think
about also outlawing _ as an rvalue, as Go does?
This would firmly establish _ as a throw-away variable, but is it worth the
internal complexity to enforce it? Perhaps the middle way of predeclaring
but not enforcing is best.

The first of Luiz's examples uses _ as an rvalue.

... and if you swap those two examples you will get weird results, so
that would actually be an argument _against_ using `_` as an rvalue (at
least for normal code).

And predeclaring is not really useful. Usually you do

local first,_,third = something()

anyway, and if you predeclare `first` and `third`, you can add `_` to
the list there as well ...

Post by Coda Highland
/s/ Adam

Philipp

[Tom N Harris]

Post by Luiz Henrique de Figueiredo
The idiom in these cases would be
load(source,_,_,MY_ENV)
first,_,third = something()

Of course. But if the lines were reversed you'd have a problem. I'm sure
everyone knows not to trust `_` as a rvalue and explicitly writes `nil`. If
`_` were reserved as a placeholder name then as a lvalue it's immediately
forgotten, and as a rvalue it evaluates to `nil`. But then what's the
difference between this and what Dirk and I suggested with empty items in the
comma list? It's not merely syntactic sugar because you're indicating the
value is going to be discarded immediately. The compiler then knows it doesn't
need to keep that register slot around unlike the `local _`. Not to mention
gcing the value earlier.

I'm pretty sure there are sources out there that use `_` for something other
than a placeholder lvalue. Any change to how it's handled would break
someone's code.

Alternately allow `nil` as a lvalue that acts as the bit bucket.

[Coda Highland]

Post by Tom N Harris
Alternately allow `nil` as a lvalue that acts as the bit bucket.

I like this best.

/s/ Adam

[Axel Kittenberger]

Post by Tom N Harris
Alternately allow `nil` as a lvalue that acts as the bit bucket.

While I'd appreciate something like that can we please keep some focus on
the original proposal? And you know the rule, make a proposal, provide a
patch.

This was just a very simply lexical relaxation, it doesn't even the warrant
"extension", where the patch consists of just one line and albeit it
happens less frequently than the trailing comma in tables it still bites
some people as there has been provided some anecdotal evidence beside me in
this conversation (thanks).

There is no implicit nil by lexical void in Lua, so there is no ambiguity
and even if there is ever going to be, the solution would be simple as been
explained by hisham to define lexical voids to exist only before commas and
locial as one comma creates one void and not two.

- Axel

[Dirk Laurie]

Post by Axel Kittenberger
While I'd appreciate something like that can we please keep some focus on
the original proposal?

I'm happy with the table constructor allowing an extra delimiter precisely
because {a,b,c}, {a,b,c,} and {a,b,c,nil} all do the same thing, and I am
not happy with the original proposal because for f(...), the difference
between f(a,b,c) and f(a,b,c,nil) can be detected by select('#',...).

[Andrew Starks]

Post by Dirk Laurie

Post by Axel Kittenberger
While I'd appreciate something like that can we please keep some focus on
the original proposal?

I'm happy with the table constructor allowing an extra delimiter precisely
because {a,b,c}, {a,b,c,} and {a,b,c,nil} all do the same thing, and I am
not happy with the original proposal because for f(...), the difference
between f(a,b,c) and f(a,b,c,nil) can be detected by select('#',...).

His suggestion is that the trailing comma does not result in an extra nil.

[Axel Kittenberger]

Post by Dirk Laurie
I'm happy with the table constructor allowing an extra delimiter precisely
because {a,b,c}, {a,b,c,} and {a,b,c,nil} all do the same thing, and I am
not happy with the original proposal because for f(...), the difference
between f(a,b,c) and f(a,b,c,nil) can be detected by select('#',...).

This is a strawman you created yourself you are now tearing down. There is
not implicit nil, there never was, it is not in the proposal and if there
ever will be, it will be no difficulty to properly define where implicit
nils will be inserted, as been explained, never more nils than there are
commas.

( Instead of implicit nils, I'd rather have the comma completely optionally
implicit, that would be very nice for Lua data files. but unfortunately
there are cases where this might be ambiguous, like the unary minus, or the
lua function call semantics without brackets. Thus I could not suggest that)

[Jay Carlson]

Post by Axel Kittenberger

Post by Dirk Laurie
I'm happy with the table constructor allowing an extra delimiter precisely
because {a,b,c}, {a,b,c,} and {a,b,c,nil} all do the same thing, and I am
not happy with the original proposal because for f(...), the difference
between f(a,b,c) and f(a,b,c,nil) can be detected by select('#',...).

This is a strawman you created yourself you are now tearing down. There

is not implicit nil [...] in the proposal

No, but what if there were? It's very Lua that nil and tables perfectly fit
together in a way you don't have to remember which way the language design
went for trailing comma. You can have either mental model, even if one is
incorrect.

Trailing comma is a very traumatic subject for many of us, because
JavaScript does the wrong thing. Switching between languages, I forget
semicolons too. The C compiler catches that case, at least...

Anyway, given how often I screw up with f(a, b, c, g()) where g returns
multiple values, I'm not sure trailing nil would hurt me much.

[Tom N Harris]

Post by Axel Kittenberger

Post by Tom N Harris
Alternately allow `nil` as a lvalue that acts as the bit bucket.

While I'd appreciate something like that can we please keep some focus on
the original proposal? And you know the rule, make a proposal, provide a
patch.

Actually, I can't say I know that rule. But ignorance of the law is not an
excuse.

In any case, I hacked a bit and got this to (seemingly) work

a,nil,c = foo()

nil in a `local` declaration list is going to take a while longer because of
the optimization that occurs. And I haven't looked at `for` yet. But this is
what I'm aiming towards:

stat ::= varlist '=' explist |
local varnamelist ['=' explist] |
for varnamelist in explist do block end

varlist ::= var {',' var}

var ::= Name | prefixexp '[' exp ']' | prefixexp '.' Name | nil

varnamelist ::= varname {',' varname}

varname ::= Name | nil

And the patch right now (against Lua 5.3work2):

diff --git a/src/lcode.c b/src/lcode.c
index 2e8b3fb..6d719ee 100644
--- a/src/lcode.c
+++ b/src/lcode.c
@@ -613,6 +613,10 @@ luaK_storevar
luaK_codeABC(fs, op, var->u.ind.t, var->u.ind.idx, e);
break;
}
+ case VNIL: {
+ freeexp(fs, ex);
+ return;
+ }
default: {
lua_assert(0); /* invalid var kind to store */
break;
diff --git a/src/lparser.c b/src/lparser.c
index 9349149..cec3198 100644
--- a/src/lparser.c
+++ b/src/lparser.c
@@ -871,6 +871,11 @@ primaryexp
luaK_dischargevars(ls->fs, v);
return;
}
+ case TK_NIL: {
+ init_exp(v, VNIL, 0);
+ luaX_next(ls);
+ return;
+ }
case TK_NAME: {
singlevar(ls, v);
return;
@@ -1138,7 +1143,7 @@

static void assignment (LexState *ls, struct LHS_assign *lh, int nvars) {
expdesc e;
- check_condition(ls, vkisvar(lh->v.k), "syntax error");
+ check_condition(ls, (vkisvar(lh->v.k) | (lh->v.k==VNIL)), "syntax error");
if (testnext(ls, ',')) { /* assignment -> ',' suffixedexp assignment */
struct LHS_assign nv;
nv.prev = lh;

[Tom N Harris]

Brief summary in case you haven't been following along. I'm hacking Lua to
accept 'nil' instead of a variable name in an assignment. Like the way 'undef'
is used in Perl. It lets you drop values from a list without using a dummy
variable.

I've reached a stable point in the patch. Everything (seems to) work as
expected.

1. nil as an lvalue in regular assignment statements. This does what you'd
expect, it skips the corresponding rvalue without creating an unnecessary
local variable as assigning to '_' would.

2. nil as an lvalue in local statements. A local statement without 'nil' acts
exactly the same as before. When there is a 'nil', it acts like a local
declaration followed by an assignment statement. i.e.

local a,nil,b = 1,2,3

is the same as

local a,b
a,nil,b = 1,2,3

3. nil as the lvalue in a for statement. A dummy variable(s) with the name
'(nil)' is created so there's no upside to doing it, except for consistency
with the other uses.

It doesn't generate the most ideal bytecode but that wasn't a primary goal. I
did see some opportunities for improving, particularly with for loops. The
current handling of loops uses a hidden control variable in addition to the
declared variable. The value is implicitly copied during the FORLOOP opcode.
If the control variable is not named, by assigning it to nil, then there is
only need to copy it. My idea is to make the copying from the hidden variable
explicit in the bytecode.

4 [1] FORPREP 0 3 ; to 8
5 [1] MOVE 3 0
...
8 [1] FORLOOP 0 -4 ; to 5

When the loop variable is unnamed, the MOVE opcode can be omitted.

The next question becomes, can we eliminate the assignment list in the syntax?

for 1,10 do end

for in io.lines() do end

Or is that getting too carried away?

Anyway, attached patch made against Lua 5.3work2
src/lcode.c | 4 +++
src/lparser.c | 102
++++++++++++++++++++++++++++++++++++++++++++++------------
2 files changed, 86 insertions(+), 20 deletions(-)

[Andrew Starks]

Post by Coda Highland

Post by Tom N Harris
Alternately allow `nil` as a lvalue that acts as the bit bucket.

I like this best.
/s/ Adam

I remember trying this once, hoping it the "way it worked." So, I guess I
like it too.

[Luiz Henrique de Figueiredo]

Post by Axel Kittenberger
However, I never understood, why trailing commas are not allowed in
function calls.

One argument for not supporting this is that argument lists are almost never
generated automatically, which was the original motivation for supporting
trailing commas in table constructors, as this simplifies code generation.

Another is that argument lists in practice are never very long.

[Andrew Starks]

On Wed, Jun 4, 2014 at 11:15 AM, Luiz Henrique de Figueiredo <

Post by Luiz Henrique de Figueiredo

Post by Axel Kittenberger
However, I never understood, why trailing commas are not allowed in
function calls.

One argument for not supporting this is that argument lists are almost never
generated automatically, which was the original motivation for supporting
trailing commas in table constructors, as this simplifies code generation.
Another is that argument lists in practice are never very long.

Tables don't store nil values, so there is no ambiguity there.

With argument lists, it would be ambiguous as to whether or not a nil was
being placed there and that is not the same thing.

local function foo(...)
return select("#", ...)
end

assert( foo() < foo(nil))

-Andrew

[Axel Kittenberger]

Andrew> With argument lists, it would be ambiguous as to whether or not a
nil was being placed there and that is not the same thing.

There is no implicit nil beyond the last comma, by definition. It was never
valid code before, so there is no breakage. What Dirk posted before as
lexical void meaning nil, was a suggestion. I'm not hot for that, since I
don't think it helps readability, but I wouldn't mind either if others like
it.

Luiz> ... as this simplifies code generation.

I don't know what people are autogenerating, but why wouldn't they
autogenerate function calls when generating code, but only tables? I at
least do the former :-) But honestly I never cared so much about that, the
extra if to prepend a comma in the output loop when index is greater than 1
never was that much a hassle for me. Its more hand written code where I
don't like to have failing code if forgetting to remove the trailing comma
when removing what was previously the last entry.

Luiz> Another is that argument lists in practice are never very long.

Well there is only one trailing comma independently if the list was short
or long. Its always one that is or isn't an issue. But I guess you mean
cases where going multilined. write() is a case where it happens often,
especially in luatex I experienced long argument lists with write. And
there maintaining the last comma there or not there by hand is cumbersome.

Anyway, its all not too important, going to live a happy life even without
trailing commas in functions :-) Just thought this would be something
rather easy to fix. I also don't know of any other language that supports
this, but Lua was cooler than most others anyway.

[Andrew Starks]

Post by Axel Kittenberger
There is no implicit nil beyond the last comma, by definition. It was
never valid code before, so there is no breakage. What Dirk posted before
as lexical void meaning nil, was a suggestion. I'm not hot for that, since
I don't think it helps readability, but I wouldn't mind either if others
like it.

Cool. I don't mean to convey a desire to shoot this down. I do think that
if you're looking at `foo(x, y, )`, and you don't have Lua's specification
committed to memory, and you happen to care[1], then it would be a quick
trip to the reference guide.

It's up to whomever to decide if that _slight_ ambiguity is worth it. I've
gotten my share of "unexpected symbol near ')" errors and would probably
favor the change, if its presence meant no change to the argument count.

-Andrew


[1] An edge case to be sure, but I have code that uses select and does
about the argument count, independent of the last nil.

[Javier Guerra Giraldez]

I also don't know of any other language that supports this

python does:

def s(a,b):
return a+b

s(1,2)
=> 3

s(1,2,)
=> 3

s(1,2,3)
=> TypeError: s() takes exactly 2 arguments (3 given)

[Florian Weimer]

Post by Luiz Henrique de Figueiredo
One argument for not supporting this is that argument lists are almost never
generated automatically, which was the original motivation for supporting
trailing commas in table constructors, as this simplifies code generation.
Another is that argument lists in practice are never very long.

And they are rarely split as one item per line. For tables, the
trailing comma reduces visual clutter in diffs if there's one item per
line.

[Thomas Jericke]

Post by Luiz Henrique de Figueiredo

Post by Axel Kittenberger
However, I never understood, why trailing commas are not allowed in
function calls.

One argument for not supporting this is that argument lists are almost never
generated automatically, which was the original motivation for supporting
trailing commas in table constructors, as this simplifies code generation.
Another is that argument lists in practice are never very long.

Both assumtions are wrong for many Lua scripts I know.

I actually know explicit cases were I run in exactly this problem. Of
course I can always resort in using a table as argument list for all API
functions.

--
Thomas