CHEF-37444: add ~/.chef/ruby/VERSION/gems to GEM_PATH for dynamic plugin loading

[sanghinitin]

Summary

Ohai plugins installed via chef gem install land in
~/.chef/ruby/<RUBY_API_VERSION>/gems, but this path was never added to
the runtime GEM_PATH. As a result, any gem installed that way was
invisible to the packaged ohai binary even after re-running the binstub.

This PR fixes the issue by expanding GEM_PATH in three places: the Ruby
binstub patch, and both the Linux/macOS and Windows Habitat wrappers.

Root Cause Evidence

$ gem info some-ohai-plugin
some-ohai-plugin (x.y.z)
    Installed at: /Users/nsanghi/.chef/ruby/3.4.0/gems
# → not in GEM_PATH → LoadError at runtime

https://progresssoftware.atlassian.net/browse/CHEF-34004

Changes Made

binstub_patch.rb

• Removed the old single-path GEM_PATH assignment.
• Now prepends three paths before any existing GEM_PATH:
  1. $pkg_prefix/vendor — ohai vendored deps (flat layout)
  2. Gem.user_dir → ~/.gem/ruby/VERSION — standard gem install target
  3. ~/.chef/ruby/RUBY_API_VERSION/gems — chef gem install target
• Uses .uniq so paths are never duplicated when the block runs twice.

habitat/plan.sh & habitat/aarch64-darwin/plan.sh

• Runtime wrapper now exports four GEM_PATH entries:
  1. $pkg_prefix/vendor — flat vendor tree
  2. $pkg_prefix/vendor/ruby/VERSION — bundler-nested tree (train-core etc.)
  3. ${HOME}/.gem/ruby/VERSION — standard gem install
  4. ${HOME}/.chef/ruby/VERSION/gems — chef gem install
• ruby_gem_version is now derived from the packaged Ruby's on-disk
  lib/ruby/gems/ directory via basename instead of executing
  ruby -e. The latter can silently fail during do_install (runtime lib
  paths are not yet set up), producing an empty version string that
  corrupts all GEM_PATH entries. Falls back to 3.4.0 with a
  build_line warning if the glob still yields nothing.

habitat/plan.ps1

• After appbundler.bat generates Windows binstubs, injects
  binstub_patch.rb immediately after the require "rubygems" line in
  each binstub — the same insertion point used on Linux/macOS.

Verification

After hab pkg build . and hab pkg install:

# 1. Confirm wrapper GEM_PATH has all four segments and a non-empty version:
grep GEM_PATH /hab/pkgs/chef/ohai/*/*/bin/ohai

# 2. Install a test gem into the chef dir:
RUBY_API=3.4.0
gem install paint --install-dir "$HOME/.chef/ruby/$RUBY_API" --no-document

# 3. Run ohai with a plugin that requires the gem — must print version, not LoadError:
cat > /tmp/test.rb <<'PLUGIN'
Ohai.plugin(:GemPathCheck) do
  provides "gem_path_check"
  collect_data do
    gem_path_check Mash.new
    gem_path_check[:gem_path] = Gem.path
    gem_path_check[:paint_found] = begin
      require "paint"; Gem.loaded_specs["paint"]&.version.to_s
    rescue LoadError => e; "FAILED: #{e.message}"; end
  end
end
PLUGIN
/hab/pkgs/chef/ohai/*/*/bin/ohai -d /tmp gem_path_check
# Expected: paint_found: "2.x.x"  ← gem loaded from ~/.chef/ruby path

This work was completed with AI assistance following Progress AI policies

Confirmed: the actual root cause is in chef/appbundler (env_sanitizer clears GEM_HOME/GEM_PATH via ::Gem.clear_paths and never restores user/chef gem paths), which is still actively used in the chef-19 Habitat build pipeline for ohai, chef, knife, chef-cli, and chef-test-kitchen-enterprise. Filed the upstream fix request there. Given the upstream fix will take longer to land and release, resolving this review so the per-repo workaround here isnt blocked -- please just link back to the appbundler upstream issue in the PR description so we remember to remove this workaround once appbundler ships the real fix.

[tpowell-progress]

Discussed with @sanghinitin and while this isn't ideal, the only solution that comes close is an intermediate package which would still require repo-specific changes

[sanghinitin]

chef/appbundler#84