Skip to content

OnigRegexp.@last_match class-IV holds an unmigratable OnigRegion (TT_DATA) — breaks cross-VM hosts #130

Description

@vobloeb

Summary

onig_match_common stores the latest match data as a class-level instance variable on OnigRegexp:

// src/mruby_onig_regexp.c (HEAD `3389bae` ~line 245; `bcfa173e` line 191):
mrb_obj_iv_set(mrb, (struct RObject*)cls_onig_regexp, MRB_IVSYM(last_match),
               MISMATCH_NIL_OR(match_value));

The stored value is OnigMatchData wrapping OnigRegion (the C-side region buffer allocated by onig_region_new). OnigRegion is a TT_DATA with no public cloning / migration story — it's heap-allocated, holds per-VM pointers, and is not shareable across mrb_states.

This makes OnigRegexp carry per-process mutable hidden state that no host walking class IVs can safely copy. Any environment that wants to deep-copy class graphs across mrb_states (e.g. for spawning a worker VM, sandboxing, or snapshotting) hits the gem's class IV and either fails or silently corrupts.

Concrete failure

The first popular host that exercises this is mattn/mruby-thread. Thread.new's default migration walks Object and its descendants, copying class IVs from parent VM to worker VM. When it reaches OnigRegexp.@last_match and tries to migrate the OnigMatchData, it raises:

TypeError: cannot migrate TT_DATA object: OnigRegion(#<OnigMatchData:0x...>)

Repro (mruby + mruby-onig-regexp + mruby-thread):

case "foo=bar"
when /^(\w+)=/ then puts $1   # populates OnigRegexp.@last_match as a side-effect
end

# Worker doesn't touch regexps — but Thread.new still walks Object's class graph,
# encounters OnigRegexp.@last_match, can't migrate it:
t = Thread.new { puts "worker" }
t.join
# → TypeError: cannot migrate TT_DATA object: OnigRegion(#<OnigMatchData:0x...>)

Deterministic. Triggers from any prior =~ / case/when / String#match / etc.

Root cause

Class-level mutable state coupled to a non-shareable C resource.

The IV is used as backing storage for OnigRegexp.last_match (CRuby's Regexp.last_match equivalent):

mrb_define_module_function(mrb, cls_onig_regexp, "last_match",
                           onig_regexp_last_match, MRB_ARGS_NONE());
static mrb_value
onig_regexp_last_match(mrb_state* mrb, mrb_value self) {
  return mrb_obj_iv_get(mrb, (struct RObject*)cls_onig_regexp, MRB_IVSYM(last_match));
}

CRuby's Regexp.last_match is thread-local (Ruby per-Thread state), so the same naming with per-process semantics is also surprising independently of cross-VM concerns: a regex match on one mruby-thread worker overwrites Regexp.last_match observed by the parent (and vice-versa).

Suggested fixes (one of)

Option A: per-VM / per-Thread storage

Move @last_match storage off the OnigRegexp class. Candidates:

  • mrb_state user-data slot (each VM keeps its own).
  • A Fiber-local / Thread-local keyed by current execution context.

Matches CRuby's threading model, eliminates the migration hazard automatically (no class IV → nothing to migrate), and fixes the orthogonal "concurrent matches clobber each other's Regexp.last_match" issue.

Option B: snapshot the match into a migratable Ruby value

OnigMatchData exposes its data via Ruby methods ([], captures, pre_match, post_match, to_a, begin, end, …). At @last_match= time, materialize the relevant fields into a plain Hash/Array/String snapshot (no TT_DATA), store that. Reads through Regexp.last_match would return the snapshot.

Trade-off: extra allocation per match. Probably acceptable since @last_match overwrites itself anyway, and most callers immediately read one of the cached fields.

Option C: mark mrb_onig_region_type as host-shareable

Requires cooperation with the host (e.g., mruby-thread exposes a g_shareable_clone registry the gem could plug into via a clone(OnigRegion*) callback). Brittle, host-coupled.

Option D: opt-out switch

Add OnigRegexp.last_match_enabled = false (analogous to existing OnigRegexp.set_global_variables = false). Users in cross-VM environments would call it at startup to disable @last_match writes. Cheapest patch, but only helps users who know they need it.

Workaround we're using downstream

Since we don't use the OnigRegexp.last_match accessor anywhere (we read $~ / $1-$9 globals via case/when exclusively), we delete the IV-write line:

sed -i '/mrb_obj_iv_set.*"@last_match".*MISMATCH_NIL_OR/d' src/mruby_onig_regexp.c

Sidesteps the migration hazard entirely. Trade-off: OnigRegexp.last_match is permanently nil. Not suitable for upstream (silently breaks documented API), fine for our codebase.

We deliberately did not patch the cross-VM host (mruby-thread) to skip the @last_match symbol — that would couple the host to a specific gem's IV names, which we don't want as a pattern.

Versions

  • mruby-onig-regexp: HEAD 3389bae and bcfa173e (last-good before unrelated regressions in #128, #129) — both affected.
  • mruby: 3.3.0, 4.0.0 — both affected.

Happy to send a PR for A or B if there's interest in fixing this in the gem. Thanks for the gem.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions