Decouple extension UI layout from folder structure

[ChrisCrosley]

Description

This PR introduces an optional YAML-based layout system that separates an extension's UI layout (tabs, panels, button arrangement) from its on-disk folder structure. Extensions can now declare their ribbon layout in extension_layout.yaml instead of encoding it in nested .tab/.panel/.stack/ directories. Tool bundles can live in a flat tools/ directory, and admins or end users can override the bundled layout with a custom one — without forking the extension.

The system is fully backward-compatible: extensions with no extension_layout.yaml continue to load via the existing folder-walking parser, unchanged. Extensions that opt in can migrate incrementally — tools/ and legacy .tab/.panel/ directories are both scanned, so tools can be moved over piecemeal.

Example layout

Included as optional presets in .extension/tools/

Layout Builder

What's in this PR

Python-side parser (pyrevitlib/pyrevit/extensions/)

• layout_parser.py — reads extension_layout.yaml / *.panel.yaml, resolves tool references against a tool index, and builds the same component tree uimaker.py already consumes.
• toolindex.py — recursive scanner for tools/ (and the legacy hierarchy) that produces a name → component dictionary. Plain subfolders in tools/ are treated as organizational only.
• layout_cli.py — programmatic API for generate_layout() (emit YAML mirroring an existing legacy structure) and list_tools() (diagnostic listing).
• parser.py — dispatches to layout-based or legacy parsing based on whether a layout file is present.
• components.py — Extension now carries an _assembly_only_commands list so unreferenced tools still get compiled into the DLL (see "assembly-only commands" below).
• userconfig.py — adds a global disable_custom_layouts toggle.

C# loader parser (dev/pyRevitLoader/pyRevitExtensionParser/)

• LayoutParser.cs — mirrors the Python parser for the new-loader code path. Handles tool indexing, custom-layout-path resolution from the INI config, panel layout files, stacks, separators, and slideouts.
• ExtensionParser.cs — dispatches to LayoutParser.ParseLayout when a layout file is detected; exposes a new ParseSingleComponent entry point so the layout path can reuse all existing bundle/script/icon parsing.
• ParsedExtension.cs — carries AssemblyOnlyComponents; adds .extension and tools to the directory-hash inputs so cache invalidation is correct under the new structure.
• PyRevitConfig.cs — reads disable_custom_layouts (global) and custom_layout_path (per extension, in [<name>.extension]).
• PyRevitConsts.cs — adds the matching config key constant.

Layout YAML format

tabs:
  - name: "My Tab"
    panels:
      - name: "My Panel"
        layout:
          - "ToolA"
          - "---"            # separator
          - stack:
              - "Small1"
              - "Small2"
              - "Small3"
          - ">>>"            # slideout
          - "ToolB"
      - name: "Complex Panel"
        layout_file: "Complex.panel.yaml"

Tools are referenced by their folder name without the postfix (MyTool.pushbutton → "MyTool"). Lookups are case-insensitive. Panel layouts can be inline or split out into <Name>.panel.yaml files alongside extension_layout.yaml.

Custom user layouts

• Stored under %APPDATA%\pyRevit\Layouts\<ExtensionName>\.
• Resolved per extension via custom_layout_path in the user INI; fall back to the bundled layout if absent or if the global disable_custom_layouts is set.
• New Extension Layout section in the Settings dialog: per-extension Import / Export / Reset buttons, a Custom / Default status column, and a global "Disable all custom layouts" checkbox.

Layout Builder (new Core tool)

• extensions/pyRevitCore.extension/tools/Layout Builder.pushbutton/ — WPF visual editor. Two-pane UI: left side lists available tools with search and a "Show All / Hide Placed" toggle; right side is the tab/panel/stack/tool tree with add / move-up/down / remove buttons. Saves to the user's custom-layout cache directory and reloads pyRevit. Optional developer-shipped presets (<ext>/layouts/*.layout.yaml) appear via a "Load Preset" button when present.

Dev tools (extensions/pyRevitDevTools.extension/)

• Generate Layout.pushbutton — generates an extension_layout.yaml from any existing legacy extension as a starting point for migration. Shift-click splits panels into separate files.
• List Tools.pushbutton — diagnostic listing of all tools discoverable in an extension.
• Test Layout Parsing.pushbutton — exercises the parser for sanity checks during development.

Migrated bundled extensions

• pyRevitCore.extension — fully migrated: tools moved from pyRevit.tab/pyRevit.panel/*.stack/... to a flat tools/ directory, structural folders removed, layout declared in extension_layout.yaml. Adds the new "Layout Builder" button. The Settings smartbutton picks up new XAML for the Extension Layout section.
• pyRevitTools.extension — opts in with extension_layout.yaml only, demonstrating the hybrid case where tools remain in the legacy folder structure but the ribbon is now layout-driven.
• pyRevitTemplates.extension — minimal example layout plus extension-layout-guide.md and migration-instructions.md for extension authors.

Assembly-only commands

When layout mode is active, any tool found on disk but not referenced by the layout is still compiled into the extension assembly — it just doesn't appear in the ribbon. This was a deliberate design choice so that user-facing layout edits (via the Layout Builder or by hand) don't trigger a full assembly rebuild on the next reload: the command type is already available, the layout just decides whether to surface it. Python side: Extension.set_assembly_only_commands() registers the unreferenced tools, and Extension.get_all_commands() merges them with UI commands for assembly compilation. C# side: ParsedExtension.AssemblyOnlyComponents, also folded into CollectCommandComponents().

Backward compatibility & safety

• Extensions with no extension_layout.yaml use the existing parser unchanged — no behavioral change.
• Extensions opting in can keep tools in the legacy .tab/.panel/ tree; both locations are scanned and merged into the tool index (tools/ wins on duplicate name).
• Unique-name format changes for layout-mode tools (<extensionname>_<toolname>, flat) vs. legacy path-based IDs. Cache directory hashing inputs were updated (.extension and tools added) so stale caches don't mask the new structure. Users with leftover compiled DLLs from older builds may need to clear %APPDATA%\pyRevit\<RevitYear>\*.dll once — this is documented in migration-instructions.md.

Notes for reviewers

• The proposal called for a new optional name field on bundle.yaml, but the implementation instead derives the name from the folder name in all cases. I could go either way here. Open to feedback.
• extensions/pyRevitTemplates.extension/extension-layout-guide.md and migration-instructions.md are the authoritative end-user docs.
• The new Layout Builder button is placed in the core pyRevit tab rather than under Dev Tools because it's user-facing, not developer-only.

---

Checklist

• Code follows the PEP 8 style guide.
• Code has been formatted with Black using the command:

  pipenv run black {source_file_or_directory}

• Changes are tested and verified to work as expected in Revit, with both layout-mode and legacy extensions loaded side by side.

---

Related Issues

• Resolves #[issue number if applicable]

---

Additional Notes

• Pre-built engine DLLs in bin/ were rebuilt against the new LayoutParser.cs / ExtensionParser.cs so the new loader can parse layout-mode extensions out of the box.
• The Layout Builder writes to the user's appdata cache, not to extension sources, so accidentally editing an installed extension's layout is not possible from the UI.
• The hash-input change in ParsedExtension.cs (adding .extension and tools) is required for layout-mode extensions but is also a strict improvement for legacy extensions — the previous hash inputs missed changes inside the extension root folder itself.

[devloai]

PR Summary:

Introduces an optional YAML-based layout system that decouples ribbon UI structure from the on-disk folder hierarchy. Extensions can declare tabs/panels/tools in extension_layout.yaml while tools live in a flat tools/ directory. Fully backward-compatible — legacy extensions are untouched. Includes a WPF Layout Builder tool, per-extension custom layout import/export in Settings, C# loader parity via LayoutParser.cs, and migrations of core bundled extensions.

Review Summary:

The overall architecture is well-thought-out and the backward-compatibility story is solid. The Python-side layout parser and tool indexer are clean and well-documented. The C# LayoutParser correctly mirrors the Python logic with appropriate logging.

Two higher-severity issues were found: (1) the Layout Builder script runs its entire entry-point (extension discovery, SelectFromList dialog, build_tool_index, window creation) at module level — a known anti-pattern in pyRevit's IronPython engine where module-level code persists between executions; (2) the layout cache directory is computed differently in the Layout Builder (%APPDATA%\pyRevit\Layouts\) vs the Settings window (PYREVIT_APP_DIR\Layouts\), which will silently diverge on non-standard installations, causing the builder to write layouts to a location Settings never reads. A latent circular import between toolindex.py and parser.py was also flagged — currently safe due to lazy import ordering but fragile. On the C# side, HasLayoutFile and GetLayoutFilePath duplicate their config-lookup logic independently, and ParseLayout has a null-coalescing fallback to a hard-coded filename after the tool index is already built. Exception handling in the Settings import callback re-raises without a user-visible alert.

Suggestions

• Consolidate the layout cache directory path into a single shared utility function (e.g. in layout_parser.py) imported by both Layout Builder and Settings scripts to eliminate the divergent path computation. Apply
• Move the module-level entry-point logic in Layout Builder.pushbutton/script.py (lines 687–738) into a main() function guarded by if __name__ == '__main__': to follow the pyRevit IronPython anti-stale-global pattern. Apply

[devloai]

Module-level code and stale user_config reference at module level

Lines 687–738 execute at module level (outside any function or class). In pyRevit's IronPython engine, module-level code persists between script executions within the same session. This means:

1. user_config is captured at module level (from pyrevit.userconfig import user_config at line 687), which is fine as a reference, but the entire extension-discovery loop, forms.SelectFromList.show, build_tool_index, and load_layout_tree calls at lines 689–738 also run at import time on every re-execution — including when Revit re-uses the cached module. The WPF dialog will pop up again on subsequent executions, but the ext_dirs/seen variables are re-initialized (since they're re-assigned), so the dialog flow still works. However this is an anti-pattern.

2. More critically: from pyrevit.userconfig import user_config at line 687 is module-level global state per the IronPython patterns checklist — if user_config is replaced or reloaded between executions, the cached reference becomes stale.

Move the extension-discovery and dialog logic into the __main__ guard or a dedicated main() function:

def main():
    from pyrevit.userconfig import user_config
    ext_dirs = []
    seen = set()
    for root_dir in user_config.get_ext_root_dirs():
        ...
    # rest of the script
    window = LayoutBuilderWindow(selected_dir, selected_name, tool_index, roots)
    window.show_dialog()

if __name__ == '__main__':
    main()

actions

Feedback: Rate this comment to help me improve future code reviews:

• 👍 Good - Helpful and accurate
• 👎 Poor - Wrong, unclear, or unhelpful
• Skip if you don't have any strong opinions either way.

[devloai]

Cache-dir path is computed differently here vs Settings.smartbutton

_get_layout_cache_dir builds the path as:

op.join(APPDATA_DIR, "pyRevit", "Layouts", extension_name)

where APPDATA_DIR = os.environ.get("APPDATA", op.expanduser("~")).

LayoutExtensionItem._get_cache_dir in Settings.smartbutton/script.py (line 106) builds it as:

op.join(PYREVIT_APP_DIR, 'Layouts', self.Name)

where PYREVIT_APP_DIR comes from pyrevit.PYREVIT_APP_DIR.

These may diverge if PYREVIT_APP_DIR is not %APPDATA%\pyRevit (e.g., on non-standard installs). Use the same canonical source for both. Prefer the PYREVIT_APP_DIR constant (imported from pyrevit) rather than rolling your own APPDATA + "/pyRevit" construct.

from pyrevit import PYREVIT_APP_DIR

def _get_layout_cache_dir(extension_name):
    return op.join(PYREVIT_APP_DIR, "Layouts", extension_name)

actions

Feedback: Rate this comment to help me improve future code reviews:

• 👍 Good - Helpful and accurate
• 👎 Poor - Wrong, unclear, or unhelpful
• Skip if you don't have any strong opinions either way.

[devloai]

HasLayoutFile returns true for a configured custom path even if the file doesn't exist

var customPath = config?.GetCustomLayoutPath(extName);
if (!string.IsNullOrEmpty(customPath))
    return true;

GetCustomLayoutPath already validates File.Exists(value) and returns null if the file is absent — so this is currently safe only because of that guard. But HasLayoutFile does a second config lookup independently of GetLayoutFilePath, which also calls GetCustomLayoutPath. If someone calls HasLayoutFile without calling GetLayoutFilePath (e.g., in a future path), and the custom-path validation changes, this logic could return true while GetLayoutFilePath returns null (the bundled path), silently enabling layout mode with the wrong file.

Prefer calling GetLayoutFilePath internally:

public static bool HasLayoutFile(string extensionDir)
{
    return GetLayoutFilePath(extensionDir) != null;
}

This also eliminates the duplicated config-access logic between the two methods.

actions

Feedback: Rate this comment to help me improve future code reviews:

• 👍 Good - Helpful and accurate
• 👎 Poor - Wrong, unclear, or unhelpful
• Skip if you don't have any strong opinions either way.

[devloai]

Double blank line between guard clauses inside function — Black formatting violation

Lines 363–365 contain two blank lines between the slideout return and the tool-lookup if block. Black only permits double blank lines between top-level definitions, not inside functions. Running black will collapse this to a single blank line.

    if tool_name == exts.SLIDEOUT_IDENTIFIER:
        slideout = GenericUIComponent()
        slideout.type_id = exts.SLIDEOUT_IDENTIFIER
        slideout.name = exts.SLIDEOUT_IDENTIFIER
        parent.add_component(slideout)
        return

    if tool_name in tool_index:   # single blank line only

actions

Feedback: Rate this comment to help me improve future code reviews:

• 👍 Good - Helpful and accurate
• 👎 Poor - Wrong, unclear, or unhelpful
• Skip if you don't have any strong opinions either way.

[devloai]

ParseLayout falls back to a hard-coded LayoutFileName constant when GetLayoutFilePath returns null

var layoutPath = GetLayoutFilePath(extensionDir)
                 ?? Path.Combine(extensionDir, LayoutFileName);

GetLayoutFilePath already returns the bundled extension_layout.yaml path if it exists, and null if neither the custom path nor the bundled file exists. The null-coalescing fallback here means ParseLayout will try to LoadYaml a non-existent file and silently return an empty LayoutParseResult — but only after it has already built the (potentially large) tool index.

This is a latent issue: ParseLayout is only called from ParseExtension after HasLayoutFile returns true, so in practice the file exists. But the logic is fragile. The ?? Path.Combine(...) fallback should either be removed (trust that the caller checked HasLayoutFile) or guard-failed early:

var layoutPath = GetLayoutFilePath(extensionDir);
if (layoutPath == null)
{
    LogWarning($"Layout: No layout file found for {extensionDir}");
    return new LayoutParseResult();
}

actions

Feedback: Rate this comment to help me improve future code reviews:

• 👍 Good - Helpful and accurate
• 👎 Poor - Wrong, unclear, or unhelpful
• Skip if you don't have any strong opinions either way.

[devloai]

Silent re-raise inside import_layout_for_ext swallows the error from the user's perspective

except Exception:
    shutil.rmtree(tmp_dir, ignore_errors=True)
    raise

The bare except Exception: ... raise is fine for cleanup, but there is no outer handler in this event callback — the raise will propagate out of the WPF event handler with no user-visible message. The user will see the dialog freeze or silently fail.

Wrap the outer call with a user-friendly alert:

except Exception as ex:
    shutil.rmtree(tmp_dir, ignore_errors=True)
    mlogger.error("Failed to import layout: %s", ex)
    forms.alert("Failed to import layout files:\n{}".format(ex))
    return

This also requires changing except Exception: → except Exception as ex:.

actions

Feedback: Rate this comment to help me improve future code reviews:

• 👍 Good - Helpful and accurate
• 👎 Poor - Wrong, unclear, or unhelpful
• Skip if you don't have any strong opinions either way.

[devloai]

Circular import: toolindex.py imports private functions from parser.py, which itself imports from toolindex.py (via layout_parser.py)

Import chain:

• parser.py → layout_parser.py (lazy, inside get_parsed_extension) → toolindex.py
• toolindex.py → parser.py (_parse_for_components, _create_subcomponents, _get_subcomponents_classes) ← at module level

The import in toolindex.py at lines 14–18 is a top-level import of three private (_-prefixed) functions from parser.py. Since parser.py itself lazy-imports toolindex inside a function, the circular dependency is not triggered at startup — but this is fragile and relies on the lazy-import order being preserved. Any future refactoring that hoists the import in parser.py to module level will cause an import error.

Consider either:

1. Moving _parse_for_components / _get_subcomponents_classes to a shared _parseutils.py module that neither parser.py nor toolindex.py imports from each other.
2. Keeping the import of parser inside _index_tool (lazily), consistent with how layout_parser handles it.

actions

Feedback: Rate this comment to help me improve future code reviews:

• 👍 Good - Helpful and accurate
• 👎 Poor - Wrong, unclear, or unhelpful
• Skip if you don't have any strong opinions either way.