Proposal: add source patch as a third stdlib compatibility mechanism

[cpunion]

Summary

llgo currently has two main mechanisms for Go stdlib compatibility work:

• alt patch
• overlay

Both are useful, but they solve different problems and both have hard limits.

This proposal introduces a third mechanism, source patch, intended to sit between them:

• earlier than alt patch
• much smaller-grained than overlay
• preserves the original stdlib package identity
• keeps most compatibility work in normal Go source files with build tags

The goal is not to replace every existing mechanism. The goal is to use:

• alt patch for symbol-level and runtime-coupled compatibility
• overlay only as a last resort
• source patch for packages that need source-level replacement while still keeping the original stdlib package identity

Background

Today we mainly rely on two strategies.

1. alt patch

This is the preferred mechanism when possible.

It works well for:

• adding missing symbols
• replacing a few functions or variables
• version-gated compatibility via normal Go build tags
• selective skipping via //llgo:skip / //llgo:skipall

Its strengths are:

• symbol-level control
• easy version/platform gating
• implemented as normal Go code
• does not require replacing stdlib files directly

But it also has a limit:

• it is still fundamentally a patch applied to an already-loaded package shape

That becomes fragile when compatibility depends on:

• original package identity
• unexported stdlib types and methods
• method sets
• generic instantiation boundaries
• loaders/type checking seeing the patched source from the beginning

This is where alt patch starts to become too late in the pipeline.

2. overlay

overlay solves that by replacing files in the original stdlib package before load/typecheck.

Its main strength is:

• the package stays the original stdlib package

This is sometimes exactly what we need.

But overlay is also expensive:

• file-level replacement is coarse
• it is harder to keep compatible across Go versions
• it tends to depend on exact upstream file layout
• it is easy to over-copy stdlib code
• it increases build/load complexity

So overlay is powerful, but not a good default.

Problem

There is a real gap between alt patch and overlay.

Some packages need more than symbol-level patching, but less than full file replacement.

Typical examples are packages where we need to:

• replace a few declarations inside the original stdlib package
• keep the original package identity
• avoid copying whole upstream files
• still write compatibility logic as ordinary Go source

This has shown up most clearly in packages such as:

• internal/sync
• iter
• internal/reflectlite

These cases are awkward for alt patch, but overlay is heavier than necessary.

Proposal

Add a source patch mechanism.

Core idea

Before packages.Load / type checking, llgo builds a patched source view for selected stdlib packages:

• original stdlib source remains the base
• patch files come from runtime/_patch/<pkg>
• selected original declarations are commented out or stubbed in the original stdlib files
• patch code is emitted as separate generated z_llgo_patch_*.go files in the original stdlib package
• the package path remains the original stdlib package path

This gives us:

• original package identity
• load-time visibility of patched declarations
• much smaller scope than file-level overlay

Source patch semantics

Patch files are normal Go files, with build tags as usual.

The current directive model is intentionally small:

• default: a patch file is active if selected for the current target
• //llgo:skipall
  • reduce the original stdlib files for that package to package stubs
• //llgo:skip name1 name2 ...
  • comment out specific original declarations
• same-name declarations in the patch file automatically override original declarations

This means we do not need a separate explicit replace directive. In practice:

• if a patch file declares the same symbol
• the original declaration is commented out in the stdlib source view
• the patch declaration becomes the one seen by the loader/type checker via the generated patch file

Supported top-level declarations

The current implementation supports these top-level declaration kinds:

• type
• const
• var
• top-level func
• methods, keyed as T.M and (*T).M

For type, const, and var, matching is by the declared top-level name.

For methods, matching is by receiver-plus-name key, for example:

• T.M
• (*T).M

const / var granularity

There is one important detail for const and var.

If the original code is written as separate specs inside a grouped declaration, each spec can be handled independently:

const (
    A = 1
    B = 2
)

In that case, replacing or skipping A does not require removing B.

However, if the original code is written as a single multi-name ValueSpec, the current implementation does not split it:

const A, B = 1, 2
var X, Y = f(), g()

If only one name from such a spec is replaced or skipped, the whole original spec is commented out. This keeps the implementation simple and deterministic, but it is a known limitation.

Multiple patch files

If multiple patch files are active for the same package:

• their skip directives are unioned
• same-name declarations from any patch file participate in the override set
• each patch file is emitted as its own generated z_llgo_patch_<name>.go
• if any active patch file declares //llgo:skipall, the original stdlib files are reduced to package stubs and all patch-side code comes from the generated patch files

This keeps patch files independent while still letting the original stdlib source be filtered consistently.

Why this is better than overlay

Compared with overlay, source patch:

• does not require whole-file replacement in the common case
• does not require copying full upstream files just to replace a few declarations
• is less sensitive to upstream file layout churn
• keeps compatibility logic in ordinary Go patch files
• is easier to review and version-gate

So source patch should be the preferred choice when:

• package identity must be preserved
• but only a small subset of declarations actually needs replacement

Why this is better than alt patch for some packages

Compared with alt patch, source patch runs earlier.

That matters when correctness depends on the loader/type checker seeing the patched package shape from the beginning, not after the package shape is already fixed.

This is especially relevant for packages involving:

• unexported stdlib implementation details
• generic instantiation boundaries
• method-set-sensitive behavior
• packages tightly used by other stdlib packages

Important limitation

Source patch must not become a way to import arbitrary llgo internals into stdlib packages.

A source-patched stdlib package should not import:

• github.com/goplus/llgo/runtime/internal/...

That breaks the model and creates module/internal visibility problems.

Instead, source patch code should be limited to:

• standard library packages
• standard library internal/... packages that are already valid for that package
• non-internal llgo compatibility surfaces such as:
  • github.com/goplus/llgo/runtime/abi
• //go:linkname for runtime hooks where needed

If a package fundamentally depends on llgo/runtime/internal/..., it is a strong sign that it should remain an alt patch, not a source patch.

Positioning relative to existing mechanisms

This proposal does not remove alt patch or overlay.

The intended layering is:

Keep using alt patch for

• symbol additions
• function/variable replacement
• runtime/linkname shims
• packages strongly coupled to llgo runtime internals

Likely examples:

• runtime
• reflect
• internal/abi
• sync/atomic
• internal/runtime/maps
• internal/runtime/sys

Use source patch for

• packages that need source-level replacement
• package identity must remain the original stdlib package
• patch logic can stay in ordinary Go files without importing runtime/internal/...

Good candidates include packages like:

• iter
• internal/sync
• internal/reflectlite
• unique where appropriate

Keep overlay only as a last resort

Only for cases where even declaration-level source patch is not enough and file-level replacement is truly required.

Implementation outline

A minimal implementation can work like this:

1. Declare source-patched package paths centrally in runtime/build.go
2. Before the first packages.Load, build a patched source view for each selected package
3. For each active patch file:
   • evaluate Go build tags against the current target
   • parse source patch directives
   • collect declarations provided by the patch
4. Comment out or stub original declarations in the original stdlib source view
5. Emit each active patch file as a generated z_llgo_patch_*.go in the original stdlib package
6. Load/typecheck the package as the original stdlib package

Diagnostics and line mapping

This part is important.

Source patch must preserve diagnostics quality as much as possible.

Two rules are especially important:

• source patch directives should be consumed only once
• sanitizing directive lines should preserve line count

Also, the current implementation keeps diagnostics simple:

• original stdlib files keep their original file paths
• skip and same-name replacement comment out the old declarations there
• skipall reduces the original stdlib files to package stubs
• each generated patch file gets a file-level //line <patch-file>:1

This keeps diagnostics understandable:

• errors in untouched stdlib code still point to the original stdlib file
• errors in patch-side code point back to the original patch file

Incremental rollout

This should be introduced incrementally.

Recommended rollout:

1. keep the mechanism opt-in by package
2. migrate a few well-bounded packages first
3. validate against the existing CI matrix
4. only migrate more packages when it clearly reduces maintenance cost

The point is not “convert everything”. The point is to reduce the set of packages that currently need file-level overlay or awkward alt patch behavior.

Non-goals

This proposal is not trying to:

• eliminate alt patch
• eliminate overlay
• allow source-patched stdlib code to depend on arbitrary llgo internals
• automatically migrate every compatibility package

Expected result

If accepted, llgo will have a cleaner compatibility stack:

• alt patch for symbol/runtime-level adaptation
• source patch for declaration-level source replacement with preserved package identity
• overlay only for rare file-level fallback cases

That should reduce maintenance cost for Go version compatibility, especially for cases where alt patch is too late and overlay is too coarse.

[cpunion]

A focused example of the current source patch implementation.

Assume the original stdlib file is:

package p

type T struct{}

const (
    A = 1
    B = 2
)

var V = 10

func Keep() {}
func Old() int { return 1 }
func (T) M() int { return 4 }

1. //llgo:skipall

If the patch file starts with:

//llgo:skipall
package p

func Old() int { return 2 }

then the original stdlib file is reduced to a package stub:

package p

and the generated patch file contains the replacement code:

//line /path/to/runtime/_patch/p/patch.go:1
//llgo_skipall
package p

func Old() int { return 2 }

So skipall always clears the original file and moves the new code into the generated patch file.

2. //llgo:skip

With:

//llgo:skip Old V
package p

func Added() int { return 3 }

only the selected original declarations are commented out in the original stdlib file:

package p

type T struct{}

const (
    A = 1
    B = 2
)

// var V = 10

func Keep() {}
// func Old() int { return 1 }
func (T) M() int { return 4 }

and the generated patch file contains only the new patch-side code:

//line /path/to/runtime/_patch/p/patch.go:1
//llgo_skip Old V
package p

func Added() int { return 3 }

3. Replacement by same-name declaration

With:

package p

type T struct{ X int }

const A = 10

var V = 20

func Old() int { return 2 }
func (T) M() int { return 5 }
func Added() int { return 3 }

same-name declarations are treated as overrides.

That means the original declarations are commented out in the stdlib file:

package p

// type T struct{}

const (
    // A = 1
    B = 2
)

// var V = 10

func Keep() {}
// func Old() int { return 1 }
// func (T) M() int { return 4 }

and the generated patch file contains the replacement code:

//line /path/to/runtime/_patch/p/patch.go:1
package p

type T struct{ X int }

const A = 10

var V = 20

func Old() int { return 2 }
func (T) M() int { return 5 }
func Added() int { return 3 }

So the current implementation supports these top-level declaration kinds:

• type
• const
• var
• top-level func
• methods, keyed as T.M and (*T).M

4. const / var granularity details

Grouped declarations with one name per spec work as expected.

For example, the original stdlib code:

const (
    A = 1
    B = 2
)

can replace only A while keeping B unchanged, because A and B are different ValueSpecs.

However, a single multi-name ValueSpec is different:

const A, B = 1, 2
var X, Y = f(), g()

If only one name from such a spec is replaced or skipped, the current implementation does not split the spec. It comments out the whole original line and expects the patch side to reintroduce anything that should remain.

This is a deliberate simplification in the current prototype.

5. Multiple patch files

If multiple patch files are active for the same package, the implementation combines them.

For example:

a.go

//llgo:skip Old
package p

func AddedA() int { return 10 }

b.go

package p

func (T) M() int { return 6 }
func AddedB() int { return 20 }

Then:

• Old is commented out because of a.go
• T.M is commented out because b.go provides a same-name replacement
• each patch file is emitted separately

The original stdlib file becomes:

package p

type T struct{}

const (
    A = 1
    B = 2
)

var V = 10

func Keep() {}
// func Old() int { return 1 }
// func (T) M() int { return 4 }

And the generated patch files are:

//line /path/to/runtime/_patch/p/a.go:1
//llgo_skip Old
package p

func AddedA() int { return 10 }

//line /path/to/runtime/_patch/p/b.go:1
package p

func (T) M() int { return 6 }
func AddedB() int { return 20 }

If any active patch file uses //llgo:skipall, the original stdlib file is reduced to a package stub and all new code comes only from the generated patch files.

Diagnostics behavior

The current implementation is intentionally simple:

• the original stdlib file stays the original file
• skip and same-name replacement comment out the old declarations there
• skipall reduces the original file to a package stub
• all new patch code is emitted into separate generated patch files
• each generated patch file carries a file-level //line <patch-file>:1

So diagnostics work like this:

• errors in untouched stdlib code point to the original stdlib file
• errors in new patch code point to the original patch file via //line

This is the current implementation model; it does not try to emit per-block //line switching inside a single merged source file.