Skip to content

Latest commit

 

History

History
651 lines (502 loc) · 23 KB

File metadata and controls

651 lines (502 loc) · 23 KB

Contributing to NativeLink

NativeLink welcomes contribution from everyone. Here are the guidelines if you are thinking of helping us:

Contributions

Contributions to NativeLink or its dependencies should be made in the form of GitHub pull requests. Each pull request will be reviewed by a core contributor (someone with permission to land patches) and either landed in the main tree or given feedback for changes that would be required. All contributions should follow this format, even those from core contributors.

Should you wish to work on an issue, please claim it first by commenting on the GitHub issue that you want to work on it. This is to prevent duplicated efforts from contributors on the same issue.

AI-assisted contributions

NativeLink builds infrastructure for coding agents, so we won't pretend they don't write code. Agents are welcome contributors when a person stands behind the change. Four rules, in the spirit of the projects whose contribution culture we admire:

  1. You must understand your change. Using an agent to write code is fine; interrogate it about the codebase until you grasp every edge case and effect of the diff. Submitting code you cannot explain is not fine. A reviewer may ask about any line, and "the agent wrote it" is not an answer.
  2. Disclose it. The pull request template has an "AI assistance" section. Say which tools helped and how much: none, autocomplete, an agent drafted it and you reworked it, an agent wrote it and you reviewed every line. "None" is a complete answer. The section is required, and the PR template check asks for it.
  3. No slop. Unreviewed generated code, generated issue or discussion text that no person edited, and generated media (artwork, video, audio) are closed without review. Write issues and comments in your own voice; generated prose runs long, sounds confident, and often misses the point. Repeated submissions of this kind cost the ability to contribute.
  4. Agents follow AGENTS.md. It maps the repository, names the checks, and says which document must follow which change. An agent working here runs the same checks a person would before opening a pull request, and never opens one on its own initiative.

Maintainers use agents at their discretion; the difference is that they have already shown they understand the code. None of this is an anti-AI stance. Review attention is the scarcest resource an open project has, and generated text made it cheap to produce contributions that look finished and are not. The rules keep the reviewer's time for changes a person is answerable for.

Everything the repository offers an agent (the llms.txt files at the repository root and the skills under .claude/skills/) is listed at https://nativelink.com/agents and in AGENTS.md.

Git setup

NativeLink has a somewhat specific contribution process to ensure consistent quality across all commits. If anything in the following guide is unclear to you please raise an issue so that we can clarify this document.

  1. In your GitHub settings, set up distinct authentication and signing keys. For further information see:

  2. Fork the TraceMachina/nativelink repository by clicking on the Fork button.

  3. Clone your fork:

    git clone git@github.com:yourusername/nativelink

Warning

Don't use

git clone git@github.com:TraceMachina/nativelink
  1. cd into the cloned repository:

    cd nativelink
  2. Add TraceMachina/nativelink as a remote repository and call it upstream:

    git remote add upstream git@github.com:TraceMachina/nativelink

Tip

To verify whether the setup was successful, run git remote -v. The output should look like this:

origin   git@github.com:yourusername/nativelink (fetch)
origin   git@github.com:yourusername/nativelink (push)
upstream     git@github.com:TraceMachina/nativelink (fetch)
upstream     git@github.com:TraceMachina/nativelink (push)
  1. Finally, configure git to sign your commits with the keys you set up previously. Create a .gitconfig file in your home directory like so:

    [user]
         name = Your Full Name
         email = the-email-you-use-on-github@example.com
         signingkey = ~/.ssh/your_private_signing_key
    
    [gpg]
         format = ssh  # Or gpg if you use a GPG key.
    
    [commit]
         gpgsign = true
    
    [tag]
         gpgsign = true

Local development setup

NativeLink ships almost all of its tooling in a nix flake which is configured via a flake.nix file in the root of the repository. While it's possible to work on some parts of the codebase without this environment, it'll make your life much easier since it lets you reproduce most of CI locally.

  1. Install Nix with flakes: https://github.com/NixOS/experimental-nix-installer For further information on Nix Flakes see: https://nixos.wiki/wiki/Flakes.

  2. Optionally (but highly recommended), install direnv and hook it into your shell:

    nix profile install nixpkgs#direnv
    
    # Add the right hook for your shell: https://direnv.net/docs/hook.html
    
    # Restart your terminal.
    
    # When you `cd` into the nativelink repository, you should see a message
    # asking you to run `direnv allow`. Do this and you're good to go.

Note

If you don't want to use direnv, you'll need to enter the flake manually with nix develop every time you enter the nativelink directory, switch branches or make changes to the nix files.

Tip

To verify that the environment is active, run env | grep NIX. You should see several *NIX_* environment variables.

  1. The environment doesn't ship a full C++ toolchain yet. Install a recent version of Clang manually:

    # Use your distros preferred package manager (pacman, emerge, apt etc), or
    # install via nix:
    nix profile install nixpkgs#clang

Common workflows

These are some common workflows that you'll encounter during development on NativeLink.

Creating pull requests

NativeLink doesn't allow direct commits or human-created side branches in the TraceMachina/nativelink repository. This holds for contributors in the TraceMachina organization. To create a pull request:

  1. Ensure that your personal fork is up-to-date with upstream:

    git switch main  # Switch to your local main branch
    git pull -r upstream main  # Fetch upstream and rebase your current branch
                               # against upstream's main branch
    git push  # Push your updated main to your own fork

Tip

Use git log to check that your branches are in order:

# A `git log` should look like this:
commit ... (HEAD -> main, upstream/main, origin/main, origin/HEAD)
...
  1. Create a new branch for the change you want to make:

    git switch -c some-feature
  2. After making changes to the source code create a commit with git commit. To keep commits and the git history uniform and readable keep the following rules in mind:

    • Use a capital letter to start the commit and use an imperative tone for the title.
    • Don't end the title with a period.
    • Keep the first line as short as possible. If your feature is complex, add additional information in the commit message body.
    • If you feel like you need the word and in the commit title, the commit might try to do too many things at once and you should consider splitting it into separate commits.
    • The commit message body should have a maximum line length of 72 characters. This is to keep the git log readable with raw terminals.
    # Good.
    Add some feature
    
    # Bad - trailing period
    Add some feature.
    
    # Bad - not imperative
    Adds some feature
    
    # Bad - details should be in the body
    Add some complex feature and try to put all info in the title
    
    # Bad - commit should be split
    Add some feature and actually some other feature as well
  3. Push your commit with git push. This will prompt you to set a remote branch for the commit:

    git push --set-upstream origin some-feature
  4. Go to https://github.com/TraceMachina/nativelink/pulls where you should see a button that you can click to create a new pull request from your fork to the main repository.

  5. Once you opened the pull request, click on the purple Reviewable button in the GitHub page for the pull request to add reviewers with +@somereviewer.

    The reviewers will take it from there and guide you through any potential remaining issues. Feel free to ask for help if you have trouble getting CI for your pull request green.

  6. If you need to make additional changes, don't use a regular git commit on the pull request branch. Instead use git commit --amend and git push -f to update the commit in-place. The changes between the commit versions will remain visible in the Reviewable UI.

Using git rebase

When you start working on a feature your git log looks something like this:

gitGraph
    commit id: "aaa"
    branch origin/main
    branch origin/some-complex-feature
    commit id: "Add some complex feature"
Loading

For complex features your commit might become outdated over time:

gitGraph
    commit id: "aaa"
    branch origin/main
    branch origin/some-complex-feature
    commit id: "Add some complex feature"
    checkout main
    commit id: "bbb"
    commit id: "ccc"
Loading

To get up-to-date with the latest branch, get the latest upstream commit and rebase your branch onto the new main branch:

git switch main                 # Switch to the origin/main branch
git pull -r upstream main       # Sync the local main branch with upstream/main
git push                        # Push the new local main to origin/main
git switch some-complex-feature # Go back to the feature branch
git rebase main                 # Rebase the feature branch onto the local main
git push -f                     # Update the PR on GitHub

After this the history will be fine again:

gitGraph
    commit id: "aaa"
    commit id: "bbb"
    commit id: "ccc"
    branch origin/main
    branch origin/some-complex-feature
    commit id: "Add some complex feature"
Loading

Fixing rust formatting

When working on Rust code bazel test commands automatically run rustfmt on all source files. If you get errors from these checks, run the rustfmt Bazel target to format the sources.

bazel run --config=rustfmt @rules_rust//:rustfmt

Running pre-commit hooks

Ensure you're in the Nix development environment as described in the Local Development Setup. To run the hooks:

pre-commit run -a

This will automatically apply some fixes like automated line fixes and format changes. Note that changed files aren't automatically staged. Use git add to add the changed files manually to the staging area.

Setting up rust-analyzer

rust-analyzer works reasonably well out of the box due to picking up the manifest for the nativelink crate, but it isn't integrated with Bazel by default. In order to generate a project configuration for rust-analyzer, run the @rules_rust//tools/rust_analyzer:gen_rust_project target:

bazel run @rules_rust//tools/rust_analyzer:gen_rust_project

This will generate a rust-project.json file in the root directory. This file needs to be regenerated every time new files or dependencies are added in order to stay up-to-date. You can configure rust-analyzer can pick it up by setting the rust-analyzer.linkedProjects configuration option.

If you use VS Code, you can configure the following tasks.json file to automatically generate this file when you open the editor:

{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "Generate rust-project.json",
      "command": "bazel",
      "args": ["run", "@rules_rust//tools/rust_analyzer:gen_rust_project"],
      "options": {
        "cwd": "${workspaceFolder}"
      },
      "group": "build",
      "problemMatcher": [],
      "presentation": {
        "reveal": "never",
        "panel": "dedicated"
      },
      "runOptions": {
        "runOn": "folderOpen"
      },
      "dependsOn": "Build nativelink"
    },
    {
      "label": "Build nativelink",
      "command": "bazel",
      "args": ["build", "//:nativelink"],
      "options": {
        "cwd": "${workspaceFolder}"
      },
      "group": "build",
      "presentation": {
        "reveal": "silent",
        "panel": "shared"
      }
    }
  ]
}

And to configure rust-analyzer to use that project, set this in your .vscode/settings.json:

{
     "rust-analyzer.linkedProjects": ["rust-project.json"]
}

Generating documentation

Automatically generated documentation is still under construction. To view the documentation for the nativelink-* crates, run the docs command in the nix flake:

docs

To build individual crate-level docs:

# All docs
bazel build docs

# A single crate
bazel build nativelink-config:docs

To run documentation tests with Bazel:

bazel test doctests

Writing documentation

NativeLink largely follows the Microsoft Style Guide.

NativeLink implements its documentation style guide via Vale. The pre-commit hooks forbid errors but permit warnings and suggestions. To view all of Vale's suggestions invoke it directly:

vale somefile

Contributing to Docs or the Website

The documentation site (docs.nativelink.com) and the marketing site (nativelink.com) live in the web/ directory — a Bun + Turborepo workspace with two Next.js apps: web/apps/docs (the docs, built with Fumadocs) and web/apps/web (the website). This replaces the old web/platform layout; the bun setup, bun docs, and bun preview commands from that era no longer exist.

The web/ workspace is not part of the Bazel build — it's built with Bun only. Install Bun (https://bun.sh, or pkgs.bun is available in the nix dev shell), then:

cd web
bun install

# Live dev server for just the docs at http://localhost:3001
bun dev:docs

# Production build + preview of both apps, the successor to the old
# `rm -r dist && bun run build` and `bun preview`. DOCS_URL is baked into
# the marketing app's /docs redirect at build time.
DOCS_URL=http://localhost:3001 bun run build
bun --filter @nativelink/web start    # nativelink.com build on port 3000
bun --filter @nativelink/docs start   # docs build on port 3001

Docs content is MDX under web/apps/docs/content/docs/. Every page needs title and description frontmatter; sidebar order comes from each section's meta.json (the top-level content/docs/meta.json orders the sections themselves). Authoring conventions, available MDX components, and style rules are documented on the site itself under Contribute → Working on documentation — which is also the page to update if this workflow changes.

The configuration reference pages under web/apps/docs/content/docs/reference/nativelink-config/ are autogenerated from the Rust config crate — don't edit them by hand; regenerate with bun --filter @nativelink/docs gen:config-reference from web/.

Creating releases

To keep the release process in line with best practices for open source repositories, not all steps are automated. Specifically, tags should be signed and pushed manually and the release notes should be human readable beyond what most automatically generated changelogs provide.

  1. Bump the current version in the following files:

    • MODULE.bazel
    • Cargo.toml
    • The Cargo.toml of every nativelink-* crate — including nested ones like nativelink-metric/nativelink-metric-macro-derive/Cargo.toml — and the regenerated lock files (Cargo.lock, nativelink-test/fuzz/Cargo.lock)

    As a sanity check, a standard release touches 17 files.

  2. Update the changelog by prepending the new release section to the existing CHANGELOG.md. Don't regenerate the whole file — that rewrites previous release entries and drops manual curation.

    First make sure your local tags match upstream, since --unreleased means "commits not contained in any tag":

    git fetch upstream --tags

    Sanity check which commits will form the new release section:

    git log --oneline "$(git describe --tags --abbrev=0)"..HEAD

    Then prepend the new section:

    git cliff --unreleased --tag=v1.x.y --prepend CHANGELOG.md

    Verify with git diff CHANGELOG.md that the change is purely additive: only added lines, with all previous release entries untouched. You might need to make manual adjustments to cliff.toml if git-cliff doesn't put a commit in the right subsection.

  3. Create the commit and PR. Call it Release NativeLink v1.x.y.

  4. Once the PR is merged, update your local repository and origin:

    git switch main
    git pull -r upstream main
    git push
  5. Create a signed tag on the release commit and give it the same tag message as the name of the tag. This tag should be the version number with a v prefix:

    git tag -s v1.x.y
    
    # tag message should be: v1.x.y
  6. Push the signed tag to the origin repository:

    git push origin v1.x.y
  7. Pushing the tag triggers an additional GHA workflow which should create the container images in your own fork. Check that this workflow is functional. If the CI job in your fork passes, push the tag to upstream:

    git push upstream v1.x.y
  8. Regenerate the latest config reference docs now that the upstream tag exists. Pushing the tag upstream triggers the Regenerate config reference workflow, which regenerates the docs and opens a PR against main with auto-merge enabled. Check that the PR appeared and merged; if the workflow failed, either re-run it from Actions → Regenerate config reference → Run workflow (entering the tag) or fall back to the manual procedure below.

    Passing the new tag updates web/apps/docs/lib/config-versions.ts, which determines the latest version shown in the docs UI, rewrites web/apps/docs/content/docs/reference/nativelink-config/index.mdx from that tag, and creates a versioned page for the previous latest release. You do not need to regenerate every historical version.

    git fetch --tags upstream
    cd web
    bun --filter @nativelink/docs gen:config-reference v1.x.y
    cd ..

    Confirm that web/apps/docs/lib/config-versions.ts marks v1.x.y as the latest release and that web/apps/docs/content/docs/reference/nativelink-config/index.mdx says it was sourced from nativelink-config @ v1.x.y. Then run the docs lint and commit the generated docs update:

    nix develop -c vale web/apps/docs/content/docs/reference/nativelink-config/*.mdx
  9. The images for the release are now being created. Go to the Tags tab in GitHub and double-check that the tag has a green Verified marker next to it. If it does, select Create a release from tag and create release notes. You can use previous release notes as template by clicking on the "Edit" button on a previous release and copy-pasting the contents into the new release notes.

    Attribute every entry to its author. Keep the same category headings as the changelog and write each line as <summary> by @author in (#PR) - (short-sha), then close the notes with a New Contributors section and a Full Changelog link. To pull the author list and first-time contributors for the range, run:

    gh api -X POST repos/TraceMachina/nativelink/releases/generate-notes \
      -f tag_name=v1.x.y -f previous_tag_name=v1.x.z

    Fold that output into the categorized notes rather than pasting it verbatim, since the changelog groups commits by type.

    Make sure to include migration instructions for all breaking changes.

    Explicitly list whatever changes you think are worth mentioning as Major changes. This is a fairly free-form section that doesn't have any explicit requirements other than being a best-effort summary of notable changes.

  10. Once all notes are in line, click Publish Release.

  11. Publishing the release triggers the Signed release artifacts workflow. It builds the nativelink binary for every supported target, signs each asset with keyless Sigstore cosign, generates an SBOM, and produces SLSA Build Level 3 provenance, attaching all of it to the GitHub Release. These signed assets are what the OpenSSF Scorecard Signed-Releases check inspects — the cosign signatures on the GHCR container images are not visible to that check.

    Wait for the workflow to finish, then confirm the release page lists, for each target, a *.tar.gz plus its *.sig/*.pem, an *.spdx.json SBOM, and a *.intoto.jsonl provenance file. You can verify any asset locally:

    # Verify the SLSA provenance covers the artifact.
    slsa-verifier verify-artifact nativelink-1.x.y-x86_64-unknown-linux-musl.tar.gz \
      --provenance-path nativelink-1.x.y.intoto.jsonl \
      --source-uri github.com/TraceMachina/nativelink \
      --source-tag v1.x.y
    
    # Verify the cosign signature.
    cosign verify-blob nativelink-1.x.y-x86_64-unknown-linux-musl.tar.gz \
      --signature nativelink-1.x.y-x86_64-unknown-linux-musl.tar.gz.sig \
      --certificate nativelink-1.x.y-x86_64-unknown-linux-musl.tar.gz.pem \
      --certificate-identity-regexp '^https://github.com/TraceMachina/nativelink/' \
      --certificate-oidc-issuer https://token.actions.githubusercontent.com

    If a release ever ships without signed assets (for example a release created before this workflow existed), re-run the workflow manually against the tag: Actions → Signed release artifacts → Run workflow, entering the tag (e.g. v1.x.y). It will rebuild, re-sign, and attach the assets to that existing release.

Conduct

NativeLink Code of Conduct is available in the CODE_OF_CONDUCT file.

Generating code coverage

You can generate branch-based coverage reports via:

nix build .#nativelinkCoverageForHost

The result symlink contains a webpage with the visualized report.