module.save_pretrained() and load_from_pretrained() implementations for xpm torch models

[VictorMorand]

I have a use case where I woud like to save a torch module (which also dereives from xpm Config) state_dict along with its configuration. (could be useful for quick testing of a model trained by a wide grid search, or if we wish to push the model on huggingface for instance)

It would look like this:

c_model = ModelConfig(...)
model = c_model.instance() #get the instance of the class, automatically done if in an xpm task
model.train(...)
model.save_pretrained(path)
...
#further loading
model = load_from_pretrained(path)

• experimaestro loadand save functions enable to easily save configs.
• The problem is that when the model is instanciated, we cannot retrieve its config !

there is a hacky way by doing the following

keys = self.C().__xpm__.__dict__['values'].keys()
config = {key: getattr(self, key) for key in keys}

What would be the best way of implementing this ? if possible without having to keep track of the config and the instance separately, which I find heavy

[bpiwowar]

The best way is to keep the configuration and save it, i.e.

from experimaestro import load, save

c_model = Model()
model = c_model.instance()

save(c_model, "/my/directory")

# Load (as instance)
model = load("/my/directory", as_instance=True)

Note that usually this is done within an experimental plan (so no instance there), so what is your use case?

[VictorMorand]

If I'm not wrong, this will not save the weights but only the config,
I would like to avoid saving them separaterly, and have something as easy to use as the HF PyTorchModelHubMixin that jointly saves the config and the weigths in a folder.
So when I want to quicky test or share a model, I only have to do model = load_from_pretrained(path)

[bpiwowar]

When saving, all the files (corresponding to Param[Path] or Meta[Path]) are also serialized. From a coding perspective, it is not a good idea to include the path into the model (but this can be done), the best would be to rely on another configuration/task that is in charge of loading the parameters and holding the path.

[VictorMorand]

As far as I understand, xpm is not done so that we can save models in a task, this would however be very useful IMHO e.g for quickly loading and testing a model trained in a given task.

• a quick fix would be to keep the original Config in the model instance (not the case now, don't really understand why) so that we could serialize the config from the model, and save it alongside with its paramerters/state_dict for easy reloading afterwards.
• we could then have a generic model_loader, which would make the Parameter_loader trick more easy to understand in the demo for instance.

[VictorMorand]

One solution can be found in #111.