Rethink extensions

[flying-sheep]

continued from this discussion.

---

hmm, many python plugin systems are done by placing stuff into a plugin subdirectory. (e.g. the flask.ext module for flask extensions)

plugins don’t run independently, so they do belong to their main project. and python packages can install submodules to some subdir. but apparently that’s a problem, so flask instead accepts plugins with names beginning with flask_, e.g. flask_admin, flask_assets, …

look how flask does it: flask/ext/__init__.py, ExtensionImporter

[Berserker66]

"plugins don’t run independently" - not entirely true. There's a tetris out there that can either run if python + pygame is installed, or if dropped into the plugins folder and started by omnitool. It also uses no Terraria resources whatsoever and simply uses omnitool as "a version of python with dependencies installed".

Additionally, flask projects aren't frozen. The extension modules never need to be loaded post freezing and I expect major problems if you try.

What might be ok, is to make the plugins folder into just that, a folder; remove the __init__.py, start the plugin subprocess and append pluginname_lib to it's path. But then, if you want plugins that share a module or package, it gets complicated again.

[flying-sheep]

how about forcing each plugin to be in the PYTHONPATH and have the the name omnitool_<pluginname>.

then we can do [name for finder, name, ispkg in pkgutil.iter_modules() if name.startswith('omnitool_')] to get all plugins.

that should also work with freezing!

[Berserker66]

that just feels clunkier than the current implementation.

What is wrong with the way it's currently done?

[flying-sheep]

• the current way has a toplevel module called plugins, which is too generic.
• by adding the plugins folder itself to the PYTHONPATH, every plugin is available as toplevel module.
• by only allowing them in one folder (which isn’t in the PYTHONPATH), there can’t be e.g. user-installed 3rd party plugins and a systemwide installed omnitool.
• if "." is in the PYTHONPATH, that means the current working directory, as does os.listdir("plugins"). starting omnitool from elsewhere won’t find the plugins.

[Berserker66]

• There is currently no relevant python module also called plugins - there's only a project on PyPi.
• quite possible with the current way plugins are found, if desired. Just complicates shared resources between plugins though,
• having multiple folders for plugins just sounds like a conflict waiting to happen, not keen on that.
• until the restructure omnitool chdir'd to sys.executable anyway, when frozen. That was lost in updates somewhere, but can certainly be added back in. It would fail loading the themes anyway, when run in a different folder.

Additionally, allowing plugins to reside "whereever" just complicates if they try to access non-python omnitool resources, like tImages.zip.

[flying-sheep]

having multiple folders for plugins just sounds like a conflict waiting to happen, not keen on that.

could you elaborate? multiple folders is how pretty much all plugin systems work: you have one or more systemwide paths and one or more per-user ones. that’s true for most text editors and most browsers.

e.g.: ~/.mozilla/firefox/abcdefgh.default/extensions/ and /usr/lib/firefox/browser/extensions/

---

Additionally, allowing plugins to reside "whereever" just complicates if they try to access non-python omnitool resources, like tImages.zip.

this is why applications have data directories that they export. applications don’t rely on the CWD being somewhere, but instead have (on windows) their installation directory (C:\Program Files\omnitool) and on linux several (libs: /usr/lib/omnitool, data: /usr/share/omnitool, systemwide config: /etc/omnitool, user config: $XDG_CONFIG_HOME/omnitool, cache: $XDG_CACHE_HOME/omnitool, …)

on linux, the plugin directories should probably be /usr/lib/omnitool/plugins ~/.local/lib/omnitool/plugins

[Berserker66]

The conflict would be people installing different versions of a plugin into multiple folders and accidentally using an old version.

The plugin system of omnitool is meant to be as simple as possible. A lot of the Terraria community is already overtasked with finding the omnitool folder, to put plugins into.

A real improvement would be, to make plugins packages with their own extensions, say otp (omnitool package) - which would be a pyzip file with meta-info json. Omnitool's own are then in omnitool -> plugins. Third party ones in shared.appdata / plugins - automatically installed there when a otp was double clicked. Omnitool would then also have to make sure to remove older versions of a plugin, if a newer is installed. and the meta-info should be able to contain some sort of update link, so that plugins can be auto updated.

That's basically my dream system. But it's a lot of work and considering there's basically no third party plugin development anymore, it just isn't a high priority. A lot more people want, for example, a new 1.3 planetoids.