feat(gpu): obtain and create a pgpu card's cyborg device profile - #1487
Merged
github-actions[bot] merged 5 commits intoSep 14, 2026
Merged
Conversation
This was referenced Sep 14, 2026
os_device_profile_create built the Cyborg device profile name inline from cyborg's `model` string. #818 needs the same name from a second caller that cannot use cyborg's string at all: a card just carved to pgpu takes up to one cyborg agent period (periodic_interval, default 60s) to appear in the accelerator inventory, so right after the carve there is nothing to read. Extract the derivation into os_device_profile_name_for, accepting either source and normalising both onto the same slug: cyborg's model "NVIDIA Corporation GA106 [RTX A2000]" -> rtx_a2000_1 hex's name "NVIDIA RTX A2000" -> rtx_a2000_1 Two derivations would be worse than one shared one that takes two inputs: if they ever disagreed the UI would show a profile name that does not exist. The cyborg path is byte-for-byte what it was, so existing profile names are unchanged; the test pins them as golden values for that reason. It also now refuses an empty name rather than building "_1", which every unnamed card would otherwise share. Refs #818 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Seki Xu <seki.xu@bigstack.co>
A pgpu card is unusable without its Cyborg device profile name -- that is what a flavor's accel:device_profile has to be set to -- and the only way to obtain it was `openstack accelerator device profile list` or the interactive `hex_cli gpu` menu. A Web UI user could set a card to pgpu and then get no further (#818), which is exactly what the parent epic's "no root shell access needed" goal rules out. sriovVgpu and migBackedVgpu cards need nothing of the sort: their PCI alias is minted by gpu_resource_set and already travels with the card. Only pgpu is missing a way to get its identifier out. gpu_device_profile_map answers for the whole node in one Openstack round trip -- that call costs ~1.2s on a real node, and a per-card loop would put that on every GET of a four-card node. A node with no pgpu card makes no call at all. gpu_device_profile_get is the single-card accessor on top. Two things are deliberate: - A card whose profile does not exist yet reports null, not the name it would be given. A name is only worth showing if it resolves; handing the operator one that was never created sends them to NoValidHost with no clue. - os_device_profile_names fails rather than reporting an empty list when Cyborg cannot be reached, so "no profiles" and "could not ask" stay distinguishable. gpu_vgpu_profile_list made the opposite choice and that was #1247. hex_sdk sources only the module matching a command's first underscore- separated segment, so a gpu_* command never has sdk_os.sh loaded and a call into it dies at runtime with "command not found". _gpu_load_os_module pulls it in on demand -- what sdk_health.sh does for sdk_ovn.sh, except lazily: sdk_os.sh is 3300 lines and drags in os_cinder.sh, and sourcing it at the top of this file would tax gpu_device_list, which the API runs on every listing. The test exercises that load path against a fake SDK_DIR instead of stubbing the callee -- stubbing is why the gap reached a real node once. Refs #818 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Seki Xu <seki.xu@bigstack.co>
gpu_resource_set has to be able to create the profile the instant a carve succeeds, and at that instant Cyborg does not know about the card: its nvidia driver decides which cards are passthrough by reading hex's config.json, but only on the agent period (periodic_interval, default 60s). Asking Cyborg for the card's model -- which is what os_device_profile_create does -- would mean either blocking the carve for up to a minute or creating nothing. Both values are available locally. The model name is in config.json, and the product id is in sysfs, which stays readable after the card is bound to vfio-pci (the same reason gpu_sysfs_pci_addr falls back to it). gpu_device_profile_ensure is idempotent and reports the name; a non-pgpu card is a no-op rather than an error, so the carve path can call it unconditionally. The groups document is now built in one place, os_device_profile_groups_for, shared with os_device_profile_create. Those trait names are wrong on Caracal -- cyborg emits CUSTOM_NVIDIA_<PID> as a single trait there (#1478) -- and this keeps that fix to one function. It matters more than it looks: os_nova_pgpu_host_list_by_instance_id parses the groups back out by field ordinal, so a change in how many traits a group carries breaks live pgpu migration with no error at all. The test deliberately stubs no accelerator-device call, so a future edit that reintroduces a dependency on the inventory fails it. The PCI address comes straight out of config.json rather than through gpu_sysfs_pci_addr. That helper asks nvidia-smi first and falls back to config.json only when the answer is empty -- but nvidia-smi prints "No devices were found" on *stdout* (rc 6, empty stderr) for a vfio-bound card, so the answer is never empty, the fallback never fires, and it hands that sentence back as if it were an address. Every card this function cares about is a pgpu, i.e. exactly the case nvidia-smi cannot see. The helper's own defect is left for its own ticket; the regression test stubs it with the poisoned value so this path cannot drift back onto it. Refs #818 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Seki Xu <seki.xu@bigstack.co>
Carving a card to pgpu left the operator halfway: the card was passed through correctly, but the Cyborg device profile that a flavor's accel:device_profile has to name did not exist, and nothing in the product created one. The only way to get it was the interactive `hex_cli gpu` menu or raw `openstack accelerator` commands -- root shell either way, which is what the parent epic set out to remove (#818). gpu_resource_set now creates it on a successful pgpu carve, and `hex_config gpu_device_profile_ensure <gpu_id>` covers the two cases that path cannot: a carve whose profile creation failed, and a card that was already pgpu before any of this existed. A failure to create the profile does not fail the carve. The card really is a pgpu at that point and config.json, the Nova drop-in and the hardware all agree; what is missing is a name that can be recreated on demand. Rolling the carve back for a label would be the worse trade, and the rollback can itself fail -- a far worse state to end in than "no profile yet". Refs #818 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Seki Xu <seki.xu@bigstack.co>
device_profile_create is node-wide and model-keyed: it walks every NVIDIA device Cyborg can see and makes one profile per model. That is the wrong shape for the thing an operator actually holds -- a card -- and it cannot help a card Cyborg has not rescanned yet, because it reads the model out of the accelerator inventory. device_profile_ensure takes one card and goes through hex_config, which derives the name from config.json and the product id from sysfs, so it works immediately after a carve. It is idempotent and a no-op on a non-pgpu card. The existing two commands are untouched: their names and behaviour are what the public docs describe, and customer sites may already have profiles made by them. Refs #818 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Seki Xu <seki.xu@bigstack.co>
Eandalf-Bigstack
force-pushed
the
feat/818-pgpu-device-profile-api-wip
branch
from
September 14, 2026 06:35
2351ba5 to
a6fc31a
Compare
Eandalf-Bigstack
approved these changes
Sep 14, 2026
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
What this PR does / why we need it
A pgpu card is unusable without its Cyborg device profile name — that is the
string a flavor's
accel:device_profilehas to be set to — and the only ways toget it were
openstack accelerator device profile listor the interactivehex_cli gpumenu. A Web UI user could set a card to pgpu and then get nofurther, which is exactly what the parent epic's "no root shell access needed"
goal rules out.
sriovVgpuandmigBackedVgpuneed nothing of the sort: theirPCI alias is minted by
gpu_resource_setand already travels with the card.Five commits, each self-contained:
refactor(gpu): derive a device profile name through one shared functionfeat(gpu): report the device profile of every pgpu card on a nodegpu_device_profile_map/_get— one Openstack round trip for the whole nodefeat(gpu): create a pgpu card's device profile without asking cyborggpu_device_profile_ensure— idempotent, no dependency on Cyborg's inventoryfeat(gpu): give a pgpu card its device profile when it is carvedconfig_gpu.cppcalls ensure on a successful carvefeat(gpu): add a per-card device_profile_ensure to the gpu CLIcli_gpu.cpp, for a card that predates this changeEnsure does not ask Cyborg for the card's model, because at the instant a carve
succeeds Cyborg does not know about the card yet: its nvidia driver learns which
cards are passthrough by reading hex's
config.json, but only on the agentperiod (default 60s). Asking would mean either blocking the carve for up to a
minute or creating nothing. Both values are available locally — the model name
in
config.json, the product id in sysfs, which stays readable after the cardis bound to
vfio-pci.Which issue(s) this PR fixes
Refs #818
Deliberately
Refs, notFixes— #818 is a User Story whose UI (#876) and API(#886) halves are still open, so merging this must not close it.
Special notes for your reviewer
hex_sdkdoes not load modules across files. It takes the command'sfirst underscore-separated segment as the module name and sources only
sdk_<MOD>*.sh, so agpu_*command never hassdk_os.shloaded and everycall into it died with "command not found" — at runtime, not at parse time,
and not in a unit test that stubs the callee.
_gpu_load_os_modulepulls itin on demand, the way
sdk_health.shdoes forsdk_ovn.sh, except lazily:sdk_os.shis 3300 lines and drags inos_cinder.sh, and sourcing it at thetop of the file would tax
gpu_device_list, which the API runs on everylisting. The tests now exercise the real load path against a fake
SDK_DIR.gpu_sysfs_pci_addrreturns a sentence, not an address, for exactly thecards this feature is about.
nvidia-smiprints "No devices were found" onstdout (rc 6, empty stderr) for a vfio-bound card, so that helper's
[ -z "$addr" ]fallback toconfig.jsonnever fires and it hands thesentence back as an address.
gpu_device_profile_ensurealready reads thecard's record, so it takes the address from there. The helper's own defect
is left alone and gets its own ticket ([Bug] gpu_sysfs_pci_addr returns nvidia-smi's error sentence as a PCI address for vfio-bound cards #1486) — its entire fallback branch is dead
code for the case it exists to handle.
Verification
commit in this PR, not just at the tip.
GET /gpuCards.ensurecreatesrtx_pro_6000_blackwell_server_edition_1with the expected traits; the APIreports it on the pgpu card and
nullon the other three.for that test is back to
IDENTICALconfig.json,numvfs=48,driver=nvidia.Two limitations QA needs to know, neither caused by this PR
PGPU ([Bug]: cyborg advertises vGPU VFs and carved PFs as allocatable PGPU, so pgpu scheduling collides with running VMs #1262's fix only applies on Caracal; cn13 runs a March Yoga ISO), so a
pgpu request can land on a vGPU VF and fail in libvirt.
it obtainable; [Bug] Caracal's cyborg renames GPU traits, so every pgpu device profile fails to schedule #1478 makes it usable. They ship together.
Related PRs: bigstack-oss/cube-cos-openapi#113 (field contract, merges first),
and the
cube-cos-apihandler PR, which opens once #113 is merged and its submodulepointer can be re-pointed at a SHA on
develop. This one can land in any order relative to theAPI side — the field simply reads
nulluntil hex can answer.Additional documentation