math.max/math.min given NaN

[David Manura]

$ lua
Lua 5.1.3 Copyright (C) 1994-2008 Lua.org, PUC-Rio

=math.max(0/0),math.min(0/0)

nan nan

=math.max(0/0,1),math.min(0/0,1)

nan nan

=math.max(1,0/0),math.min(1,0/0)

1 1

Is that behavior defined? Should it be defined?

[Mike Pall]

Post by David Manura
$ lua
Lua 5.1.3 Copyright (C) 1994-2008 Lua.org, PUC-Rio

=math.max(0/0),math.min(0/0)

nan nan

=math.max(0/0,1),math.min(0/0,1)

nan nan

=math.max(1,0/0),math.min(1,0/0)

1 1
Is that behavior defined? Should it be defined?

I've discovered that, too, while reimplementing it. I doubt it
would be useful in practice to have "perfect" NaN propagation
semantics for min/max. It needs extra effort in software to do
these checks -- with little payoff.

Even the hardware designers from Intel/AMD haven't bothered to
add the functionality. The SSE2 minsd/maxsd just propagate the
2nd operand if any operand is NaN:
minsd(1, NaN) = maxsd(1, NaN) = NaN
minsd(NaN, 1) = maxsd(NaN, 1) = 1

Few programs rely on perfect NaN propagation. This takes extra
care and mandates hand-crafted conditionals, anyway. IMHO a small
remark in the manual would suffice.

--Mike

[Roberto Ierusalimschy]

Post by David Manura
$ lua
Lua 5.1.3 Copyright (C) 1994-2008 Lua.org, PUC-Rio

=math.max(0/0),math.min(0/0)

nan nan

=math.max(0/0,1),math.min(0/0,1)

nan nan

=math.max(1,0/0),math.min(1,0/0)

1 1
Is that behavior defined? Should it be defined?

I don't see how to define it unless there is a specific rule for nan.
Both "x such that x >= y[i]" or "x such that (not y[i] > x)" seem
undefined when nan is present. (In the first case no 'x' satisfies
the rule; in the second case more than one 'x' may satisfy it.)

-- Roberto

[David Manura]

Post by Roberto Ierusalimschy

Post by David Manura

=math.max(1,0/0),math.min(1,0/0)

1 1
Is that behavior defined? Should it be defined?

I don't see how to define it unless there is a specific rule for nan.
Both "x such that x >= y[i]" or "x such that (not y[i] > x)" seem
undefined when nan is present.

One might do this (not that I'm recommending it):

$ diff -u lmathlib.c~ lmathlib.c
--- lmathlib.c~ 2007-12-27 08:02:25.000000000 -0500
+++ lmathlib.c 2008-02-26 20:06:37.125000000 -0500
@@ -158,6 +158,9 @@
lua_Number d = luaL_checknumber(L, i);
if (d < dmin)
dmin = d;
+ else if (d != d) { /* NaN */
+ dmin = d; break;
+ }
}
lua_pushnumber(L, dmin);
return 1;
@@ -172,6 +175,9 @@
lua_Number d = luaL_checknumber(L, i);
if (d > dmax)
dmax = d;
+ else if (d != d) { /* NaN */
+ dmax = d; break;
+ }
}
lua_pushnumber(L, dmax);
return 1;

But I now see this: "In the IEEE floating-point standard, arithmetic operations
involving NaN always produce NaN....In the proposed IEEE 754r revision of that
standard the same rule applies, except that a few anomalous functions (such as
the maxnum function, which returns the maximum of two operands which are
expected to be numbers) favour numbers—if just one of the operands is a NaN then
the value of the other operand is returned." -- http://en.wikipedia.org/wiki/NaN

Post by Roberto Ierusalimschy
Few programs rely on perfect NaN propagation. This takes extra
care and mandates hand-crafted conditionals, anyway. IMHO a small
remark in the manual would suffice.

This was seen with code that had a logic error causing a NaN, but the
propagation of that NaN wasn't being detected, so that wasn't a case where
perfect NaN propagation was required.