extension for lua bytecode files

[Tony Finch]

Is there a standard extension for Lua bytecode files? I've been using .luo
(short for Lua object code).

I'm writing a program that embeds Lua and which will have significant
parts written in Lua. In order to make it easier to deploy I'm embedding
the bytecodes in the executable, which I am doing by a .lua -> .luo -> .c
translation. The C file just declares an array and initializes it with
the contents of the Lua bytecodes so it can be passed to luaL_loadbuffer.
I guess lots of people do something like this?

Tony.

[Thomas Lauer]

Post by Tony Finch
Is there a standard extension for Lua bytecode files? I've been using .luo
(short for Lua object code).

I don't think there's a standard. FWIW, I use .lbc but .luo sounds
nicer.

Post by Tony Finch
I'm writing a program that embeds Lua and which will have significant
parts written in Lua. In order to make it easier to deploy I'm embedding
the bytecodes in the executable, which I am doing by a .lua -> .luo -> .c
translation. The C file just declares an array and initializes it with
the contents of the Lua bytecodes so it can be passed to luaL_loadbuffer.
I guess lots of people do something like this?

This works very well. I have a 'shell' along the lines of lua.c which
loads three or four byte-code libraries either from internal byte arrays
or from external .lbc files.

Just watch out for strlen() and friends ... there's a function in lua.c
that calls this before loading the buffer. That's fine if the string is
text but if it's precompiled byte code strlen() just won't cut the
mustard.

[Tony Finch]

Post by Thomas Lauer
This works very well. I have a 'shell' along the lines of lua.c which
loads three or four byte-code libraries either from internal byte arrays
or from external .lbc files.

I have been using a helper program to make this kind of array, but I would
have thought that it's a common enough requirement to build in to luac:

--- luac.c.orig Fri Jun 2 17:37:11 2006
+++ luac.c Sat Jun 16 18:26:12 2007
@@ -30,6 +30,7 @@
static int dumping=1; /* dump bytecodes? */
static int stripping=0; /* strip debug information? */
static char Output[]={ OUTPUT }; /* default output file name */
+static const char* Ccode; /* C wrapper name */
static const char* output=Output; /* actual output file name */
static const char* progname=PROGNAME; /* actual program name */

@@ -55,6 +56,7 @@
"usage: %s [options] [filenames].\n"
"Available options are:\n"
" - process stdin\n"
+ " -c name output C wrapper\n"
" -l list\n"
" -o name output to file " LUA_QL("name") " (default is \"%s\")\n"
" -p parse only\n"
@@ -84,6 +86,11 @@
}
else if (IS("-")) /* end of options; use stdin */
break;
+ else if (IS("-c")) /* output C wrapper */
+ {
+ Ccode=argv[++i];
+ if (Ccode==NULL || *Ccode==0) usage(LUA_QL("-c") " needs argument");
+ }
else if (IS("-l")) /* list */
++listing;
else if (IS("-o")) /* output file */
@@ -144,6 +151,33 @@
}
}

+static void Cprologue(FILE *D)
+{
+ fprintf(D,"/* auto-generated by luac */\n"
+ "void luaload_%s(lua_State *L) {\n"
+ " static const char bytecode[] = {\n ", Ccode);
+}
+
+static void Cepilogue(FILE *D)
+{
+ fprintf(D,"};\n if(luaL_loadbuffer(L,bytecode,sizeof(bytecode),"
+ "\"%s\"))\n lua_error(L);\n}\n", Ccode);
+}
+
+static int Cwriter(lua_State* L, const void* p, size_t size, void* u)
+{
+ UNUSED(L);
+ while (size > 0) {
+ if (fprintf((FILE*)u, "%d,", *(unsigned char *)p) < 0)
+ return 1;
+ p++; size--;
+ if (size % 20 == 0)
+ if (fprintf((FILE*)u, "\n ") < 0)
+ return 1;
+ }
+ return 0;
+}
+
static int writer(lua_State* L, const void* p, size_t size, void* u)
{
UNUSED(L);
@@ -174,9 +208,11 @@
{
FILE* D= (output==NULL) ? stdout : fopen(output,"wb");
if (D==NULL) cannot("open");
+ if (Ccode != NULL) Cprologue(D);
lua_lock(L);
- luaU_dump(L,f,writer,D,stripping);
+ luaU_dump(L,f, (Ccode != NULL) ? Cwriter : writer, D,stripping);
lua_unlock(L);
+ if (Ccode != NULL) Cepilogue(D);
if (ferror(D)) cannot("write");
if (fclose(D)) cannot("close");
}


Tony.

[Luiz Henrique de Figueiredo]

Post by Tony Finch
I have been using a helper program to make this kind of array, but I would

Under Linux, you can use "xxd -i".

[Tony Finch]

Post by Luiz Henrique de Figueiredo
Under Linux, you can use "xxd -i".

Thanks for pointing that out. Sadly it doesn't exist on all unixes.

Tony.

[Rici Lake]

Post by Tony Finch

Post by Luiz Henrique de Figueiredo
Under Linux, you can use "xxd -i".

Thanks for pointing that out. Sadly it doesn't exist on all unixes.

It's part of the vim distribution.

[Mark Edgar]

Post by Tony Finch
I have been using a helper program to make this kind of array, but I would

There's bin2c.c in Lua 5.0 distributions, and this one for Lua 5.1:
http://lua-users.org/wiki/BinTwoCee

-Mark

[Sherry Zhang]

Post by Mark Edgar

Post by Tony Finch
I have been using a helper program to make this kind of array, but I would

http://lua-users.org/wiki/BinTwoCee
-Mark

well, use the oringinal bin2c, then change lua_dobuffer to
luaL_loadbuffer, then pcall it will solve the problem.

[Doug Rogers]

Post by Thomas Lauer
I don't think there's a standard. FWIW, I use .lbc but .luo sounds
nicer.

Long ago I used .lc, but I like Tony's .luo better. If I ever embed byte
codes directly again, I'll use .lua.

Nowadays I put the .lua files on a flash drive and just recompile them
when the code loads. Long ago the byte code was portable so I could
compile using Solaris/sparc and include it in a cross-compile to
LynxOS/PowerPC and also for my x86 applications. But now I just include
the Lua source and recompile it.

In the future I may go with Ashwin Hirschi's method of zipping
everything up in a package.

Doug

[Ashwin Hirschi]

Post by Tony Finch
I'm writing a program that embeds Lua and which will have significant
parts written in Lua. In order to make it easier to deploy I'm embedding
the bytecodes in the executable, which I am doing by a .lua -> .luo -> .c
translation. The C file just declares an array and initializes it with
the contents of the Lua bytecodes so it can be passed to luaL_loadbuffer.
I guess lots of people do something like this?

Instead of embedding Lua byte code you might consider keeping all your Lua logic separate from your (compiled) C program.

If indeed a significant part of your application is written in Lua and you have a fair number of scripts, you could compile & bundle them in, for instance, an archive or database format [preferably one you can easily access from your C program... ;-)].

My company has been deploying software this way for years. Mostly we use "application packages" based on zip archives or SQLite databases. And this has worked very well for is.

The approach makes a lot of sense, especially if you have a fairly stable Lua-host/runtime and are developing the better part of your solution in Lua.

For such a scenario updates are often mostly enhancements/tweaks to your Lua logic. So, all you need to do to deploy any changes is recompile and send out the repackaged Lua scripts, without the need to rebuild your entire solution and distribute executables (and such).

Anyways, consider it some food for thought on a Sunday afternoon.

Ashwin.