Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide all of Lua 5.3/5.4 and LuaJIT in a single package #196

Closed
romanalexander opened this issue Oct 31, 2021 · 8 comments · Fixed by #207
Closed

Provide all of Lua 5.3/5.4 and LuaJIT in a single package #196

romanalexander opened this issue Oct 31, 2021 · 8 comments · Fixed by #207
Milestone

Comments

@romanalexander
Copy link

The distributions from pip use the bundled Lua 5.3 implementation, not LuaJIT2.0.5 as the readme suggests

@scoder
Copy link
Owner

scoder commented Nov 1, 2021 via email

@scoder scoder changed the title Windows wheels use Lua 5.3 not LuaJIT Provide all of Lua 5.3/5.4 and LuaJIT in a single package Feb 23, 2022
@scoder
Copy link
Owner

scoder commented Feb 23, 2022

Module names to provide:

  • lupa - from .lua54 import * (i.e. delegating to latest Lua module, as it already does now)
  • lupa.lua53 - Lua 5.3.x
  • lupa.lua54 - Lua 5.4.x
  • lupa.luajit20 - LuaJIT 2.0.x
  • lupa.luajit21 - LuaJIT 2.1.x

Basically, the setup.py script needs to create multiple Extension instances (instead of just one) for the same lupa/_lupa.pyx file but with different extension module names.

It would probably also be a good idea to create a separate git submodule under third-party/ for each of the Lua/LuaJIT versions, so that they can be separately pinned and built from there.

@tatuylonen
Copy link

I strongly support making lupa available with different Lua versions! I'm using lupa extensively for Wiktionary extraction (https://github.com/tatuylonen/wiktextract, or actually its subpackage https://github.com/tatuylonen/wikitextprocessor). Wiktionary uses Lua 5.2, and I have no way of upgrading the Wiktionary modules to Lua 5.3 or 5.4. I've coped with Lua 5.3 by patching the Lua source code using essentially a few dozen regexp-replaces, but it is a horrible kludge. Now with Lua 5.4, it has again changed in several incompatible ways, including changes in supported format strings and apparently the operation of metatables has also changed in subtle ways. It's coming to a point where I'll have to start building and releasing my own version of lupa compiled for an older version unless there is support for either selecting the version to use when creating a Lua context or installing a separate package. The proposed way of, e.g., lupa.lua53 works, but I would very much prefer using Lua 5.2. The English Wiktionary alone contains 322,000 lines of Lua 5.2 code, and when counting the other language editions and Wikipedia, it is probably millions of lines of Lua 5.2 code in Mediawiki projects alone. It is not going to get upgraded from Lua 5.2 any time soon, I think.

I came to this repository with the intention of submitting an issue to request this feature.

@scoder
Copy link
Owner

scoder commented Mar 6, 2022

See #207

@scoder
Copy link
Owner

scoder commented Mar 6, 2022

There is currently no build procedure for LuaJIT (although the sources are bundled) and building Lua 5.2.3 also fails.
Help with that is welcome, otherwise I'd just keep it as it is for now.

@scoder
Copy link
Owner

scoder commented Mar 9, 2022

I got LuaJIT to build at least on Linux. That way, it can be used with a conditional import and a fallback to normal Lua.

@scoder
Copy link
Owner

scoder commented Mar 9, 2022

The PR is mostly working now. I'm not sure about the interface, though. It seems clumsy to write

import lupa.lupa_lua54 as lupa

Also, import lupa currently imports the Lua 5.4 version, so import lupa.lupa_lua53 loads both Lua 5.3 and Lua 5.4.

Rather, it would be nice to say

import lupa.lua53 as lupa   # or ... as lua ?

and have import lupa basically do nothing. But that's backwards incompatible (which would be acceptable for the intended Lupa 2.0 but not great) and also seems a bit less clear – from lupa import lua53 returns the same kind of module as from lupa import lua54, just with a different Lua version. There could still be a submodule (e.g. lupa.lua) that would always refer to the latest (or only) Lua/LuaJIT module.

Comments welcome.

@scoder
Copy link
Owner

scoder commented Mar 10, 2022

The incompatibility can be resolved with PEP 562 (module level __getattr__), that's done.

That makes from lupa import lua53 more appealing now. I changed the module names as well.

scoder added a commit that referenced this issue Aug 3, 2022
* Build multiple Lupa extension modules for the different bundled Lua versions.
* Test all Lupa variants with all Lua versions, not just the latest.
* setup.py: Make sure we always remove all known options from the command line, not just the ones that happen to be relevant.
* Disable Py27 builds in appveyor due to certificate issues with the old git version. We still have Windows builds in Github Actions.
* Improve output in case of LuaJIT build failures.
* Import the latest extension module only on demand from "import lupa" (on Py3.7 and later, which have PEP 562: module __getattr__).
* Update versions of dependencies and build matrix.

Closes #196
@scoder scoder added this to the 2.0 milestone Aug 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants