Skip to content

fix(cbuffer): handle D3D raw export and vs-in fallback#265

Merged
BANANASJIM merged 1 commit into
masterfrom
fix/issue-224-windows-d3d-cbuffer-vsin
Jul 6, 2026
Merged

fix(cbuffer): handle D3D raw export and vs-in fallback#265
BANANASJIM merged 1 commit into
masterfrom
fix/issue-224-windows-d3d-cbuffer-vsin

Conversation

@BANANASJIM

@BANANASJIM BANANASJIM commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • Decode cbuffer values from the reflected value lane instead of always using f32v, including numeric uint variables reported by D3D captures.
  • Route raw cbuffer exports through stageful VFS paths so vs bN and ps bN can resolve different resources at the same set/binding, with stage validation before temp-file output.
  • Add an IA decode fallback for mesh --stage vs-in when RenderDoc PostVS VSIn data is missing or mirrors VSOut.

Test Evidence

  • uv run --extra dev pytest tests/unit/test_buffer_decode.py tests/unit/test_cbuffer_commands.py tests/unit/test_vfs_router.py tests/unit/test_fix2_vfs_intermediate_dirs.py -q
  • pixi run lint
  • pixi run typecheck
  • pixi run test (3071 passed, 6 skipped, coverage 93.20%)
  • Pre-commit pixi run check
  • GitHub CI is green on 00371d29acfa8bae1da82d7a68604ab410f76493

Notes

  • Refs Add export support for shader constant buffers and VS input meshes #224.
  • Legacy decoded cbuffer paths remain routed; raw CLI export now uses /draws/<eid>/cbuffer/<stage>/<set>/<binding>/data.
  • Local manual replay against real .rdc captures was not run because rdc doctor reports the RenderDoc Python replay module is unavailable in this environment. The unit tests cover the reported failure modes with mocks, but Windows D3D11/D3D12 capture verification remains needed.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Constant-buffer routing now includes shader stage in VFS paths and raw export filenames. Cbuffer value flattening selects numeric lanes by reflected type. mesh_data adds IA POSITION fallback for vs-in when PostVS is unusable, with matching tests.

Changes

Stage-aware cbuffer and mesh vs-in fallback

Layer / File(s) Summary
Stage-aware VFS routes and tree structure
src/rdc/vfs/router.py, src/rdc/vfs/tree_cache.py, tests/unit/test_vfs_router.py, tests/unit/test_fix2_vfs_intermediate_dirs.py
Router patterns for /cbuffer add a {stage} segment; tree cache builds per-stage set/binding directories with data leaves alongside the existing aggregate view; tests verify route resolution and tree structure.
Stage-aware cbuffer export and lane-aware decoding
src/rdc/commands/cbuffer.py, src/rdc/handlers/buffer.py, tests/unit/test_cbuffer_commands.py, tests/unit/test_buffer_decode.py
CLI raw export and internal dump filenames include the stage; flattened cbuffer variable values use lane-aware extraction (uint/sint/f32) instead of a float-only helper; tests verify stage-scoped paths, filenames, and typed value extraction.
Mesh vs-in IA POSITION fallback decoding
src/rdc/handlers/buffer.py, tests/unit/test_buffer_decode.py
mesh_data adds helpers to validate/decode PostVS data and, for vs-in, falls back to decoding POSITION from the input assembler (with index/baseVertex handling) when PostVS is missing or invalid, raising a specific error when neither source has data; tests cover fallback scenarios, bounds, and error messages.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

  • BANANASJIM/rdc-cli#129: Both PRs modify constant-buffer decoding in src/rdc/handlers/buffer.py, changing how shader variables are flattened/decoded.
  • BANANASJIM/rdc-cli#227: Both PRs modify mesh_data decoding for the vs-in stage in src/rdc/handlers/buffer.py.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main changes: stage-aware raw export and vs-in fallback handling.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/issue-224-windows-d3d-cbuffer-vsin

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

fix(cbuffer): handle D3D raw export, uint types, and vs-in IA fallback

🐞 Bug fix ✨ Enhancement 🕐 40+ Minutes

Grey Divider

AI Description

• Fix cbuffer value decoding to select the correct ShaderValue lane (u32v/s32v/f32v) based on
 reflected variable type, fixing D3D uint cbuffer variables that were always decoded as floats.
• Route raw cbuffer exports through stage-qualified VFS paths (/draws//cbuffer////data) so vs bN
 and ps bN can resolve different resources at the same set/binding.
• Add an IA-decode fallback for mesh --stage vs-in when RenderDoc PostVS VSIn data is missing or
 mirrors VSOut, reading directly from the bound vertex/index buffers.
• Refactor _handle_mesh_data by extracting _decode_mesh_postvs, _decode_position_rows, and
 _mesh_data_from_ia as standalone helpers.
• Expand unit test coverage for all three fixes with mocked D3D capture scenarios.
Diagram

graph TD
    CLI["cbuffer CLI\n--raw --stage"] --> VFS_PATH["VFS Path\n/draws/eid/cbuffer/stage/set/binding/data"]
    VFS_PATH --> ROUTER["vfs/router.py\nRoute Table"]
    ROUTER --> HANDLER["handlers/buffer.py\ncbuffer_raw / cbuffer_decode"]
    HANDLER --> DECODE["_shader_variable_values\ntype-aware lane selection"]
    HANDLER --> MESH_IA["_mesh_data_from_ia\nIA fallback for vs-in"]
    HANDLER --> MESH_POSTVS["_decode_mesh_postvs\nPostVS path"]
    TREE["vfs/tree_cache.py\npopulate_draw_subtree"] --> ROUTER
    TREE --> STAGE_NODES["Stage-qualified\nVfsNode subtree"]

    subgraph Legend
      direction LR
      _cli["CLI Command"] ~~~ _svc(["Handler/Service"]) ~~~ _helper["Helper Function"]
    end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Mandatory stage in all cbuffer paths (no legacy routes)
  • ➕ Simpler router with no ambiguous stageless paths
  • ➕ Eliminates dual-path maintenance burden
  • ➖ Breaking change for existing CLI users and scripts
  • ➖ Requires migration of all callers
2. Surface vs-in PostVS failure as a clear error instead of IA fallback
  • ➕ Simpler code, no IA decode complexity
  • ➕ Avoids potential discrepancies between IA and PostVS data
  • ➖ Leaves D3D11/D3D12 users with no usable vs-in data
  • ➖ Worse UX for the reported failure mode

Recommendation: The PR's approach of adding parallel stage-qualified VFS paths while keeping legacy stageless paths is the right call for backward compatibility. One alternative worth noting is a single unified path with stage as a mandatory parameter — but that would break existing integrations. The IA fallback for vs-in is a pragmatic workaround for a RenderDoc limitation on D3D captures; an alternative would be to surface the limitation as a clearer error, but the fallback provides real value for users. The type-aware ShaderValue lane selection is the correct fix — the only alternative (always using f32v and reinterpreting) would lose precision for uint variables.

Files changed (8) +627 / -66

Enhancement (2) +46 / -2
router.pyAdd stage-qualified VFS routes for cbuffer decode and raw export +24/-0

Add stage-qualified VFS routes for cbuffer decode and raw export

• Registers four new regex routes under '/draws/<eid>/cbuffer/<stage>/...' (dir, set dir, binding leaf, and data leaf_bin) so that stage-specific cbuffer paths can be resolved independently from the legacy stageless paths.

src/rdc/vfs/router.py

tree_cache.pyPopulate per-stage cbuffer subtree nodes in VFS tree cache +22/-2

Populate per-stage cbuffer subtree nodes in VFS tree cache

• Tracks cbuffer bindings per stage in 'cbuffer_stage_sets' and creates 'VfsNode' entries for each '<stage>/<set>/<binding>/data' path during 'populate_draw_subtree', making stage-qualified cbuffer paths visible in VFS directory listings.

src/rdc/vfs/tree_cache.py

Bug fix (2) +232 / -56
buffer.pyFix cbuffer uint decode, extract mesh helpers, add vs-in IA fallback +231/-55

Fix cbuffer uint decode, extract mesh helpers, add vs-in IA fallback

• Replaces the hardcoded 'f32v' lane extraction with '_shader_variable_value_kind' / '_shader_variable_values' helpers that select 'u32v', 's32v', or 'f32v' based on the reflected variable type. Refactors '_handle_mesh_data' by extracting '_decode_mesh_postvs', '_decode_position_rows', and '_mesh_data_from_ia' as reusable helpers. Adds a 'vs-in' fallback that reads position data directly from the IA vertex/index buffers when PostVS data is absent or mirrors VSOut.

src/rdc/handlers/buffer.py

cbuffer.pyRoute --raw export through stage-qualified VFS path +1/-1

Route --raw export through stage-qualified VFS path

• Changes the '--raw' export path from '/draws/{eid}/cbuffer/{set}/{binding}/data' to '/draws/{eid}/cbuffer/{stage}/{set}/{binding}/data' so the correct stage-specific resource is fetched.

src/rdc/commands/cbuffer.py

Tests (4) +349 / -8
test_buffer_decode.pyAdd tests for uint decode, stageful raw VFS, vs-in IA fallback +297/-7

Add tests for uint decode, stageful raw VFS, vs-in IA fallback

• Adds 'test_uint_numeric_type_extracts_u32v' for D3D uint cbuffer variables, 'test_stageful_raw_data_node_visible_to_vfs_ls' and 'test_stage_specific_resources_with_same_set_binding' for the new VFS paths, and a full 'TestMeshVsInFallback' class covering empty PostVS, non-indexed draws, indexed draws with base-vertex, and the case where VSIn mirrors VSOut.

tests/unit/test_buffer_decode.py

test_cbuffer_commands.pyUpdate and extend cbuffer CLI raw-export path tests +17/-1

Update and extend cbuffer CLI raw-export path tests

• Updates the existing 'test_raw_output_uses_default_stage' assertion to expect the stage-qualified path and adds 'test_raw_output_uses_requested_stage' to verify that '--stage vs' produces the correct VFS path.

tests/unit/test_cbuffer_commands.py

test_vfs_router.pyAdd router tests for stage-qualified cbuffer decode and raw paths +14/-0

Add router tests for stage-qualified cbuffer decode and raw paths

• Adds 'test_cbuffer_stageful_decode' and 'test_cbuffer_stageful_raw' to verify that '/draws/42/cbuffer/vs/0/3' and '/draws/42/cbuffer/ps/0/3/data' resolve to the correct handlers with the 'stage' argument.

tests/unit/test_vfs_router.py

test_fix2_vfs_intermediate_dirs.pyAdd VFS tree-cache and router tests for stageful cbuffer nodes +21/-0

Add VFS tree-cache and router tests for stageful cbuffer nodes

• Adds 'test_cbuffer_stageful_raw_leaf' to verify router resolution and 'test_cbuffer_stage_nodes_and_data_leaf_created' to verify that 'populate_draw_subtree' creates the full '<stage>/<set>/<binding>/data' node hierarchy.

tests/unit/test_fix2_vfs_intermediate_dirs.py

@qodo-code-review

qodo-code-review Bot commented Jul 6, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 📜 Skill insights (0)

Context used

Grey Divider


Action required

1. Wrong POSITION input fallback ✓ Resolved 🐞 Bug ≡ Correctness
Description
In the vs-in IA fallback, _position_input() returns the first vertex input when it can’t find a
POSITION semantic, causing mesh --stage vs-in to silently decode an arbitrary attribute as positions
and export incorrect geometry. Because _mesh_data_from_ia() only errors when pos_input is None, this
wrong-output case won’t be surfaced to the user as an error.
Code

src/rdc/handlers/buffer.py[R467-472]

+def _position_input(inputs: list[Any]) -> Any | None:
+    for vi in inputs:
+        name = str(getattr(vi, "name", "") or getattr(vi, "semanticName", "")).lower()
+        if "position" in name or name.startswith("pos"):
+            return vi
+    return inputs[0] if inputs else None
Evidence
_position_input() explicitly falls back to the first vertex input when no POSITION is found, and
_mesh_data_from_ia() only raises when the returned value is None—so any non-empty vertex input
list will bypass the POSITION check and be decoded as position data.

src/rdc/handlers/buffer.py[467-489]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`_position_input()` falls back to `inputs[0]` when no POSITION-like attribute is found, so `_mesh_data_from_ia()` will proceed and decode the wrong buffer/format as positions.

### Issue Context
This affects the new `mesh_data` IA fallback for `stage=vs-in` and can produce silently incorrect exports.

### Fix Focus Areas
- src/rdc/handlers/buffer.py[467-489]

### Suggested fix
- Change `_position_input()` to return `None` when it cannot positively identify a POSITION-like input.
- Keep `_mesh_data_from_ia()` raising `ValueError("no vertex input POSITION at this event")` when `pos_input is None`.
- (Optional) If you still want a heuristic fallback, gate it behind an explicit parameter/flag and make the output indicate which semantic was used.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Large IA index reads ✓ Resolved 🐞 Bug ➹ Performance
Description
In _mesh_data_from_ia(), if the ActionDescription is missing (action=None) or provides
numIndices==0, the code may fall back to reading ib.byteSize bytes from the index buffer to infer
referenced vertices, which can trigger unexpectedly large GetBufferData reads and slow/failed
exports. This is reachable because _handle_mesh_data() passes the result of _find_action(..., eid)
directly to _mesh_data_from_ia() without requiring that an action was found.
Code

src/rdc/handlers/buffer.py[R494-515]

+    action_count = int(getattr(action, "numIndices", 0) or 0) if action is not None else 0
+
+    first_vertex = 0
+    local_indices: list[int] = []
+    ib = pipe_state.GetIBuffer()
+    irid = getattr(ib, "resourceId", None)
+    i_stride = getattr(ib, "byteStride", 0)
+    is_indexed = action is None or bool(int(getattr(action, "flags", 0)) & _ACTION_INDEXED)
+    if is_indexed and irid is not None and int(irid) != 0 and i_stride in (1, 2, 4):
+        index_count = action_count
+        i_offset = getattr(ib, "byteOffset", 0)
+        if action is not None:
+            i_offset += int(getattr(action, "indexOffset", 0) or 0) * i_stride
+        i_size = (
+            index_count * i_stride
+            if index_count > 0
+            else _known_byte_size(getattr(ib, "byteSize", 0))
+        )
+        if i_size > 0:
+            iraw = controller.GetBufferData(irid, i_offset, i_size)
+            base_vertex = int(getattr(action, "baseVertex", 0) or 0) if action is not None else 0
+            referenced = [i + base_vertex for i in _decode_index_buffer(iraw, i_stride)]
Evidence
The code sets is_indexed true when action is None, and when index_count is zero it computes
i_size from ib.byteSize and reads that many bytes via GetBufferData. _handle_mesh_data() can
pass action=None when _find_action() returns None, making this path reachable.

src/rdc/handlers/buffer.py[475-552]
src/rdc/handlers/buffer.py[555-587]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The IA fallback reads `ib.byteSize` when `action is None` or when `action_count` (from `action.numIndices`) is zero, which can cause very large reads from the index buffer.

### Issue Context
This is in the new `mesh --stage vs-in` fallback logic. When action metadata is unavailable, the handler can still attempt to decode indices.

### Fix Focus Areas
- src/rdc/handlers/buffer.py[494-552]
- src/rdc/handlers/buffer.py[571-587]

### Suggested fix
- If `action is None`, either:
 - raise `ValueError` (e.g., `"no draw action found for eid"`), or
 - treat as non-indexed and **do not** read the index buffer.
- Only read index data when you have a trustworthy `index_count` (e.g., `action is not None` and `action_count > 0`).
- Avoid using `ib.byteSize` as an unbounded fallback read size; if you keep a fallback, cap it to a reasonable maximum or require an explicit user flag for “best-effort” decoding.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread src/rdc/handlers/buffer.py Outdated
Comment thread src/rdc/handlers/buffer.py Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/rdc/handlers/buffer.py (1)

231-297: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Sanitize stage before using it in the temp filename cbuffer_raw accepts an arbitrary stage, and the current f-string lets / or .. escape state.temp_dir when the file is written. Validate stage_name against STAGE_MAP before building the path.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/rdc/handlers/buffer.py` around lines 231 - 297, The _handle_cbuffer_raw
handler currently uses the unvalidated stage_name directly in the temp file
name, which can allow path traversal when writing under state.temp_dir. In
_handle_cbuffer_raw, validate stage_name against STAGE_MAP before it is used for
the temp_path construction, and reject or normalize any unsupported value so the
generated filename cannot contain path separators or parent-directory segments.
🧹 Nitpick comments (4)
src/rdc/vfs/tree_cache.py (1)

356-379: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Minor duplication between per-stage and aggregate cbuffer directory population.

The per-stage loop (356-371) and the aggregate loop (372-379) build near-identical dir/leaf structures, differing mainly in the extra "data" leaf_bin child. Could be factored into a shared helper parameterized by whether to add the data child, but current duplication is small and low-risk.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/rdc/vfs/tree_cache.py` around lines 356 - 379, The cbuffer tree
population logic in tree_cache.py duplicates almost the same dir/leaf
construction in the per-stage loop and the aggregate loop. Refactor the repeated
VfsNode/subtree-building into a shared helper used by the stage and aggregate
paths, and make it parameterized so the stage variant can optionally add the
"data" leaf_bin child while the aggregate variant omits it; keep the existing
behavior in the code paths around cbuffer_stage_names, cbuffer_stage_sets, and
cbuffer_sets.
src/rdc/handlers/buffer.py (2)

402-419: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

PostVS vertex read size isn't bounds-checked like the new IA path.

v_size = getattr(mesh, "vertexByteSize", 0) feeds GetBufferData(mesh.vertexResourceId, 0, v_size) directly, without the _known_byte_size sentinel guard that _mesh_data_from_ia now applies to BoundVBuffer.byteSize. If PostVS ever reports an unresolved/sentinel size, this could trigger an oversized read. Likely low practical risk since PostVS sizes are usually already resolved by RenderDoc, but worth aligning with the new defensive pattern for consistency.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/rdc/handlers/buffer.py` around lines 402 - 419, The PostVS read in
_decode_mesh_postvs should use the same defensive byte-size handling as the new
IA path, since vertexByteSize may still be an unresolved sentinel. Update the
GetBufferData call to guard v_size with the existing _known_byte_size pattern
before reading mesh.vertexResourceId, and keep the rest of the decoding flow
unchanged. Use _decode_mesh_postvs and _mesh_data_from_ia as the reference
points when aligning the bounds check behavior.

47-76: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Lane-aware ShaderValue extraction verified correct against RenderDoc's row-major convention.

Checked against RenderDoc's official example scripts, which index matrix values as v.value.f32v[r*v.columns + c] — matches the new [lane[ri*c+ci] for ri in range(r) for ci in range(c)] formula. u32v/s32v/f32v lane selection also looks right for the type codes used in tests (4→u32v, 5→s32v).

Only gap: no f64v lane for double-typed cbuffer variables — falls through to f32v default, which would misread double values if RenderDoc ever reflects them. Likely out of scope for this PR (targets uint specifically) but worth a follow-up if double cbuffer members are expected.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/rdc/handlers/buffer.py` around lines 47 - 76, Add support for
double-typed ShaderValue extraction in _shader_variable_value_kind and
_shader_variable_values so reflected cbuffer variables that use doubles are not
misread as f32v. Update the lane selection logic alongside the existing
u32v/s32v/f32v handling in the _shader_variable_value_kind helper, and ensure
_shader_variable_values can read the correct lane from var.value when the
reflected type indicates a 64-bit float.
tests/unit/test_buffer_decode.py (1)

290-311: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Solid new coverage; one gap worth calling out.

Traced the assertions in TestMeshVsInFallback (637-831) against _mesh_data_from_ia and they're internally consistent with the implementation's use of action.indexOffset for both indexed and non-indexed offset math. That consistency is exactly what's masking the indexOffset/vertexOffset field-naming issue flagged in src/rdc/handlers/buffer.py (see comment there) — the mock doesn't model RenderDoc's real distinction between the two fields for non-indexed draws, so this suite can't catch that class of bug. Worth adding a mock field for vertexOffset distinct from indexOffset once the source fix lands, to prevent regression.

Also applies to: 314-326, 339-398, 411-411, 485-485, 533-533, 637-831, 1033-1045

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/test_buffer_decode.py` around lines 290 - 311, Update the buffer
decode test coverage around _mesh_data_from_ia and TestMeshVsInFallback so the
mock reflects RenderDoc’s real distinction between indexOffset and vertexOffset
for non-indexed draws. In the existing test setup that currently only uses
action.indexOffset, add a separate vertexOffset field in the mocked action
object and use it in the non-indexed path assertions, while keeping indexed
behavior tied to indexOffset. This will make the assertions in
TestMeshVsInFallback exercise the field-naming fix in src/rdc/handlers/buffer.py
and prevent the current mock from hiding regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/rdc/handlers/buffer.py`:
- Around line 475-552: The non-indexed path in _mesh_data_from_ia is using
action.indexOffset to choose the starting vertex, but that field only applies to
indexed draws. Update the non-indexed branch to read action.vertexOffset
instead, keeping the indexed logic unchanged, and make sure the vertex range
computation still flows through first_vertex, num_verts, and the final
GetBufferData call.

---

Outside diff comments:
In `@src/rdc/handlers/buffer.py`:
- Around line 231-297: The _handle_cbuffer_raw handler currently uses the
unvalidated stage_name directly in the temp file name, which can allow path
traversal when writing under state.temp_dir. In _handle_cbuffer_raw, validate
stage_name against STAGE_MAP before it is used for the temp_path construction,
and reject or normalize any unsupported value so the generated filename cannot
contain path separators or parent-directory segments.

---

Nitpick comments:
In `@src/rdc/handlers/buffer.py`:
- Around line 402-419: The PostVS read in _decode_mesh_postvs should use the
same defensive byte-size handling as the new IA path, since vertexByteSize may
still be an unresolved sentinel. Update the GetBufferData call to guard v_size
with the existing _known_byte_size pattern before reading mesh.vertexResourceId,
and keep the rest of the decoding flow unchanged. Use _decode_mesh_postvs and
_mesh_data_from_ia as the reference points when aligning the bounds check
behavior.
- Around line 47-76: Add support for double-typed ShaderValue extraction in
_shader_variable_value_kind and _shader_variable_values so reflected cbuffer
variables that use doubles are not misread as f32v. Update the lane selection
logic alongside the existing u32v/s32v/f32v handling in the
_shader_variable_value_kind helper, and ensure _shader_variable_values can read
the correct lane from var.value when the reflected type indicates a 64-bit
float.

In `@src/rdc/vfs/tree_cache.py`:
- Around line 356-379: The cbuffer tree population logic in tree_cache.py
duplicates almost the same dir/leaf construction in the per-stage loop and the
aggregate loop. Refactor the repeated VfsNode/subtree-building into a shared
helper used by the stage and aggregate paths, and make it parameterized so the
stage variant can optionally add the "data" leaf_bin child while the aggregate
variant omits it; keep the existing behavior in the code paths around
cbuffer_stage_names, cbuffer_stage_sets, and cbuffer_sets.

In `@tests/unit/test_buffer_decode.py`:
- Around line 290-311: Update the buffer decode test coverage around
_mesh_data_from_ia and TestMeshVsInFallback so the mock reflects RenderDoc’s
real distinction between indexOffset and vertexOffset for non-indexed draws. In
the existing test setup that currently only uses action.indexOffset, add a
separate vertexOffset field in the mocked action object and use it in the
non-indexed path assertions, while keeping indexed behavior tied to indexOffset.
This will make the assertions in TestMeshVsInFallback exercise the field-naming
fix in src/rdc/handlers/buffer.py and prevent the current mock from hiding
regressions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f32c7721-ec10-4b7a-ab55-ab32ec042f05

📥 Commits

Reviewing files that changed from the base of the PR and between 5329530 and f02187d.

📒 Files selected for processing (8)
  • src/rdc/commands/cbuffer.py
  • src/rdc/handlers/buffer.py
  • src/rdc/vfs/router.py
  • src/rdc/vfs/tree_cache.py
  • tests/unit/test_buffer_decode.py
  • tests/unit/test_cbuffer_commands.py
  • tests/unit/test_fix2_vfs_intermediate_dirs.py
  • tests/unit/test_vfs_router.py

Comment thread src/rdc/handlers/buffer.py
@BANANASJIM BANANASJIM force-pushed the fix/issue-224-windows-d3d-cbuffer-vsin branch from f02187d to a9434bb Compare July 6, 2026 03:57

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/unit/test_buffer_decode.py`:
- Around line 672-698: The non-indexed fallback in buffer decoding is using the
wrong draw field, so update the logic in the buffer handler to read the
vertex-offset field instead of indexOffset when there are no indices. Adjust the
related test in test_non_indexed_fallback_applies_first_vertex, along with any
mocked action setup, to use vertexOffset/FirstVertexLocation semantics so the
expected vertex range matches the real RenderDoc behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d8907a84-4caa-49a2-8f0c-3c05e6fcff5c

📥 Commits

Reviewing files that changed from the base of the PR and between f02187d and a9434bb.

📒 Files selected for processing (8)
  • src/rdc/commands/cbuffer.py
  • src/rdc/handlers/buffer.py
  • src/rdc/vfs/router.py
  • src/rdc/vfs/tree_cache.py
  • tests/unit/test_buffer_decode.py
  • tests/unit/test_cbuffer_commands.py
  • tests/unit/test_fix2_vfs_intermediate_dirs.py
  • tests/unit/test_vfs_router.py
🚧 Files skipped from review as they are similar to previous changes (7)
  • src/rdc/commands/cbuffer.py
  • tests/unit/test_cbuffer_commands.py
  • tests/unit/test_fix2_vfs_intermediate_dirs.py
  • src/rdc/vfs/router.py
  • tests/unit/test_vfs_router.py
  • src/rdc/vfs/tree_cache.py
  • src/rdc/handlers/buffer.py

Comment thread tests/unit/test_buffer_decode.py
@BANANASJIM BANANASJIM force-pushed the fix/issue-224-windows-d3d-cbuffer-vsin branch from a9434bb to 00371d2 Compare July 6, 2026 04:04

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@BANANASJIM BANANASJIM merged commit 01527a8 into master Jul 6, 2026
17 checks passed
@BANANASJIM BANANASJIM deleted the fix/issue-224-windows-d3d-cbuffer-vsin branch July 6, 2026 07:43
@Misaka-Mikoto-Tech

Misaka-Mikoto-Tech commented Jul 6, 2026

Copy link
Copy Markdown

I think there are two different use cases that should be separated:

  1. Full VS-IN / IA attribute export

mesh --stage vs-in currently seems to export only one selected position-like attribute plus indices/faces. That is useful for OBJ-style geometry, but it is not equivalent to RenderDoc's Mesh Viewer Input tab.

VS input is not always a conventional position-backed mesh. For example, UI/sprite/procedural meshes may store only indices, tile IDs, corner IDs, or instance data in VS inputs, and reconstruct the real position in the vertex shader from constants/textures. In those cases there may be no POSITION attribute at all.

So a full VS-IN export should dump all input attributes per vertex/index and should not require any attribute to be named or inferred as POSITION.

  1. OBJ export from VS-IN

For OBJ export, selecting a position attribute is still necessary because OBJ needs vertex positions. But the current name-based lookup is too fragile.

A better position selection strategy would be:

  • Prefer an explicit semantic if present: POSITION, SV_Position, or maybe pos*.
  • Otherwise ignore per-instance attributes and infer from vertex-rate attributes.
  • Prefer float/vector formats, especially float3 or float4.
  • Prefer slot 0 and lower byte offsets.
  • If there are multiple plausible candidates, either choose the highest-ranked candidate with a warning or require an explicit user override.
  • Provide an override such as --position-attribute ATTRIBUTE0, --position-slot 0 --position-offset 0, or --position-index 0.
  • Include the selected attribute in JSON/metadata output, e.g. position_attribute, position_source: semantic|heuristic|user.

For the D3D12 NTE capture I tested, the VS-IN attributes are ATTRIBUTE0 and ATTRIBUTE13; ATTRIBUTE0 is the actual position stream, and ATTRIBUTE13 is instance-rate uint data. A robust OBJ fallback should select ATTRIBUTE0 by heuristic or by user override, not fail just because no input semantic is named POSITION.

Unreal/D3D12 captures often expose inputs as ATTRIBUTE0, ATTRIBUTE1, etc., where the names are not semantically meaningful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants