New way to handle other versions

[jaudiger]

It would have taken time, but here we go. This PR is here to open the discussion on how to handle multiple versions of a recipe. I'm not quite happy with what we were doing, and I'm still not happy with this proposal.

Basic user story

Having a way to choose the version of a recipe when we want to install it locally or when we want to use it in an other recipes.

How do we do it today ?

export const project = {
  name: "python",
  version: "3.14.0",
  repository: "https://github.com/python/cpython",
  extra: {
    otherVersions: {
      "3.14": "3.14.0",
      "3.13": "3.13.9",
      "3.12": "3.12.12",
    },
  },
};

Basically, we have project.version that serves as the latest version of a recipe, and we can have an optional key project.extra.otherVersions which exposes other possible versions that can be used for this recipe. Today, the concept of other versions could be defined as: "X(.Y)?" where X is the major number and Y the optional minor number. Here we don't want Z to be part of the tag.

What we really want ?

This is not yet completely decided, but here is a good summary from @kylewlacy.

The CLI interface could be defined as is:

# From the registry
brioche install -r sqlx_cli@tag

Where tag is optional and could take one of these values:

• latest
• X(.Y)?
• a hash

Questions

• Do we really want to have a tag latest which is the same thing as just not setting the tag ?
• How do we handle it when installing from a local path ? We cannot use @ since it's a valid character for a folder name

# From a local path
brioche install -p ./packages/sqlx_cli@tag

• Do we want to offer the same level of pinning when using a recipe in another recipe ? (aka latest, X(.Y)? and a hash)
• If we let choose a hash, how can we discover all the hashes for a recipe, and more importantly, how can we see the content of a recipe for a specific hash ?

[jaudiger]

What does this PR ?

• Remove the doublon version between project.version and project.extra.otherVersions to make a difference between latest and the other versions
• Introduce a tag latest when using a recipe that exposes multiple versions
• Add a method to convert from a type version (such as PythonVersion) into the real version (with or without the patch number)
• Fix an issue with Go testing where we were always testing the latest version of Go

[jaudiger]

Alternatively, we could inline the field otherVersion, there is no need to consider otherVersion as an extra field. Extra fields can be considered as some sort of recipe's metadata:

export const project = {
  name: "python",
  version: "3.14.0",
  otherVersions: {
    "3.13": "3.13.9",
    "3.12": "3.12.12",
  },
  repository: "https://github.com/python/cpython",
};

But it's not clear enough (written this way) that latest map to 3.14.0. And, the more I think about it, the more I'm telling me that we should also have the value 3.14 when it's the latest major release. So we could install the recipe with all these methods:

brioche install -r python
brioche install -r python@latest
brioche install -r python@3.14
brioche install -r python@3.13
brioche install -r python@3.12

Other package managers tend to resolve this "mapping" issue by having one recipe per major version as demonstrated here in the Brew registry:

[kylewlacy]

Oh this is interesting! Thanks for getting this PR in place for discussion too! First impressions: I think I would like to not merge this PR though (...unless we have a use-case where we need major.minor / latest tags), but definitely nice to see a sketch to talk it through! Definitely interested in hashing out the design of multiple versions.

Bit of background... my current feeling is that the current way we handle multiple versions is duct tape: good enough to hold us over until we go with a proper solution. I was originally hoping that we could avoid multiple versions entirely until we had the proper solution, but enough conflicts popped up around different dependency versions that we needed something to unblock us. For the final design, I still feel pretty happy overall with the design I outlined here: brioche-dev/brioche#97 (comment) (if you asked me a year ago, I definitely would've hoped to at least have a prototype of this feature by now though!)

Also lots of good question:

Do we really want to have a tag latest which is the same thing as just not setting the tag ?

...we kind of already have this today, kind of! Every time the client resolves a package from the registry, it gets it with a tag name. Today, that's hard-coded as latest.

Personally, having a specific "default" name makes a lot of sense to me intuitively, but it's bit hard to describe why. I guess it's harder for me to imagine what things would look like without some kind of default tag. Docker uses latest as the default tag, git uses main or master as the default branch, most package managers support using * as a version to grab the latest, etc.

How do we handle it when installing from a local path ? We cannot use @ since it's a valid character for a folder name

I think this ties in with brioche-dev/brioche#320. Honestly, I would still push for using @. Yes, it can conflict with a directory name which is a tradeoff, but realistically most projects in the real world will use alphanumeric or -_ as names, and I think it's reasonable to enforce that if it's better ergonomics for the 99.9% of users who'd never even try to use @ or another special character in their folder name.

...actually I decided to check Cargo, and cargo new foo@1 fails by default (but they do let you use the --name flag if you want to use the dir name foo@1 anyway).

Do we want to offer the same level of pinning when using a recipe in another recipe ? (aka latest, X(.Y)? and a hash)

Yes, I definitely want this! That's honestly a big part of the motivation to me: letting packages pick the right version of their dependencies. If aws-cli says they only support Python 3.13 and below, then we should be able to do import python from "python@3.13" and get the right version ideally

If we let choose a hash, how can we discover all the hashes for a recipe, and more importantly, how can we see the content of a recipe for a specific hash ?

Today, https://brioche.dev/packages/ just links to the GitHub repo, but that was mostly done as a placeholder! Eventually, I want a package page that shows everything you'd want about a package: the current version and hash, version history, TypeScript docs, details about what's in the default export, etc. That's a big task on its own though!

Shorter term, I think it'd be reasonable to add brioche subcommands to query the registry: search packages by name (or other fields), show info and versions for a package, fetch a specific version of a package, etc. This would require registry work too, since we don't have APIs for any of these yet though.

Alternatively, we could inline the field otherVersion, there is no need to consider otherVersion as an extra field. Extra fields can be considered as some sort of recipe's metadata

Agreed at a high level, but the reason it's under extra today is so when we update Brioche to handle the otherVersions field (or whatever structure we decide on), the schema won't conflict with any packages we have today. So I want to save the top-level otherVersions until we're super sure we know what we want the schema to be!

And, the more I think about it, the more I'm telling me that we should also have the value 3.14 when it's the latest major release. So we could install the recipe with all these methods:

Also agreed 100%. I do think this should be something that Brioche itself + the registry support though, I'd prefer if packages themselves didn't need to manually do these tag aliases each time, and instead we just handled it automatically (same with latest too-- packages should ideally never need to see the word latest in their implementation)

[jaudiger]

For the final design, I still feel pretty happy overall with the design I outlined here: brioche-dev/brioche#97 (comment)

The only downside of this proposal is that if a recipe depends on another recipe and we need to override its version as in the aws_cli recipe, it'll make things harder when the dependency is upsated:

Today, we have that:

export default function awsCli(): std.Recipe<std.Directory> {
  const pythonVersion = "3.13";
  const managedPython = python({ version: pythonVersion });

  let venv = std.recipe(managedPython);
}

With the proposal, we'll have:

export default function awsCli(): std.Recipe<std.Directory> {
  const pythonVersion = "3.13.9";
  const managedPython = python({ version: pythonVersion });

  let venv = std.recipe(managedPython);
}

Which means every time, the path version if Python changes, we need to update all the downstream recipes. I would more see something that is a mix of today and your proposal:

export const project = {
  name: "postgresql",
  version: "16.0.0",
  versions: [
    "15": "15.0.0",
    "14": "14.0.0",
  ],
};

But again with this proposal, it means the tag is 15, 14 and latest (which redirects to 16.0.0). But how to handle the tag 16 ? Should we put it in the key versions ? A bit as we have today (before this PR):

export const project = {
  name: "postgresql",
  version: "16.0.0",
  versions: [
    "16": "16.0.0",
    "15": "15.0.0",
    "14": "14.0.0",
  ],
};

Honestly, I would still push for using @. Yes, it can conflict with a directory name which is a tradeoff, but realistically most projects in the real world will use alphanumeric or -_ as names, and I think it's reasonable to enforce that if it's better ergonomics for the 99.9% of users who'd never even try to use @ or another special character in their folder name.

As far as this character is forbidden in the name of a recipe, I guess, it can work. I agree that @ is the most common character to represent what we want here.

Shorter term, I think it'd be reasonable to add brioche subcommands to query the registry: search packages by name (or other fields), show info and versions for a package, fetch a specific version of a package, etc. This would require registry work too, since we don't have APIs for any of these yet though.

I agree for the MVP, we should have a way to request the available versions (and all its hashes ?) from the CLI. The webpage could come later.

So I want to save the top-level otherVersions until we're super sure we know what we want the schema to be!

Sure, we can experiment in the extra key, and move the work in the root key once we're enough confident with the design.

(same with latest too-- packages should ideally never need to see the word latest in their implementation)

Agree, that's why I wasn't feeling comfortable with the current approach of this PR.

I think I would like to not merge this PR though (...unless we have a use-case where we need major.minor / latest tags)

Sure, I wasn't expected to be merged anytime soon ! I'll just extract the fix for the go recipe, plus a quick uniformization we could have.

[jaudiger]

Also, after a second thought, it's maybe just me, but I'm not a big fan of having both version and versions keys in the project. Let's say we have nothing yet, and we can start from a blank page (just for the discussion). What would be the best way to support multiple versions while in the same time single version recipe ?

1

export const project = {
  name: "single",
  versions: [
    "1": "1.0.0",
  ],
};

export const project = {
  name: "multiple",
  versions: [
    "1": "1.0.0",
    "2": "2.0.0",
  ],
};

Cons:

• For single recipe, we shouldn't have a named tag (the only acceptable tag should be latest)
• For multiple recipe, the latest tag is a bit hidden. It'll be the highest number, but it's not obvious by just reading the recipe definition.

2

export const project = {
  name: "single",
  version: "1.1.0",
};

export const project = {
  name: "multiple",
  versions: [
    "1": "1.1.0",
    "2": "2.0.0",
  ],
};

Cons:

• For multiple recipe, the latest tag is a bit hidden. It'll be the highest number, but it's not obvious by just reading the recipe definition.

3

export const project = {
  name: "single",
  versions: "1.1.0",
};

export const project = {
  name: "multiple",
  versions: [
    "1.1.0",
    "2.0.0",
  ],
};

Cons:

• How to define named tags 1, 2 for multiple recipe ?
• For multiple recipe, the latest tag is a bit hidden. It'll be the highest number, but it's not obvious by just reading the recipe definition.

4

export const project = {
  name: "single",
  version: "1.1.0",
};

export const project = {
  name: "multiple",
  versions: [
    "1.1.0",
    "2.0.0",
  ],
  tags: [
    "1": "1.1.0",
    "2": "2.0.0",
  ],
};

Cons:

• We have two keys in multiple recipe
• For multiple recipe, the latest tag is a bit hidden. It'll be the highest number, but it's not obvious by just reading the recipe definition.

5

Any other ideas ?

[jaudiger]

I kind of like (4), since it's not a breaking change for 98% of current recipes.

For recipe with one version, we should only allow latest as tag, and latest will forward to project.version.

For recipe with multiple versions, latest will be the max of project.versions. And additional tags (numerical or alphabetical could be defined in project.tags, we could imagine something like that in the future:

export const project = {
  name: "multiple",
  versions: [
    "1.1.0",
    "2.0.0-rc2",
    "3.0.0-2025-11-01"
  ],
  tags: [
    "1": "1.1.0",
    "beta": "2.0.0-rc2",
    "nightly": "3.0.0-2025-11-01"
  ],
};

So it means a such recipe could be installed with these different commands:

brioche install multiple
brioche install multiple@latest
brioche install multiple@1
brioche install multiple@beta
brioche install multiple@nightly

Edit: Another use case of having name tags would be for the dev environment. It's pretty common to import a tool this way. Based on your comment here, we could have something like that:

import * as std from "std";
import rust_analyzer from "rust_analyzer";
import rust from "rust";

export const devTools = std.merge(
    rust({ nightly }),
    rust_analyzer({ nightly }),
);

While still relying on the lock file to have pinned version of packages.

I know this example is not that realistic, since most of the time, a user would also want to choose the nightly version, a bit like here where I pinned my Rust nightly toolchain. Another example is the zig-overlay, we can choose which version of master (same concept as nightly) we want or we can stick to the latest master, if no version is chosen, as I did here.

Edit 2: I said earlier that latest will be the max of project.versions, but it doesn't work if a recipe also contains nightly version. Since we don't these versions to be used in the compute of latest. For such recipes, I think, it could make sense to manually set the latest tag:

export const project = {
  name: "multiple",
  versions: [
    "1.1.0",
    "2.0.0-2025-11-01"
  ],
  tags: [
    "1": "1.1.0",
    "latest": "1.1.0",
    "nightly": "2.0.0-2025-11-01"
  ],
};

[kylewlacy]

@jaudiger I like the last example you gave, I think that's the one that resonates with me the most, but yeah it gets tricky when you start considering how to handle things like "setting latest by default" vs. "mixing stable + nightly versions". I like the idea of having latest default to the max version with the option to override it explicitly. Along the same lines, it might be good to automatically add the tags like 1.75 and 1 when the latest version is 1.75.2, for example

...oh, and I can't remember if we ever discussed how the imports could look on the JS side before? e.g. for using the latest nightly version of Brioche, I think it'd be pretty nice to allow this:

import rust from "rust@nightly";

...which means you wouldn't need to call it like rust({ version: "nightly" }) or equivalent. Or, a similar option would be to use import attribute syntax:

import rust from "rust" with { version: "nightly" };

This gets somewhat more complicated to handle on the runtime side, but I feel like this syntax would make it pretty intuitive to use a specific pinned version of something!

[jaudiger]

...oh, and I can't remember if we ever discussed how the imports could look on the JS side before?

No we haven't, but I kinda like your first proposition, it seems more natural for people coming from other package managers.

I like the last example you gave, I think that's the one that resonates with me the most,

Ok, so let me push this direction with this PR to see what it could do. I'll do that in the coming days