Skip to content

fix(redfish): fix Redfish GET /Systems returns empty body after power state transitions#1051

Draft
DevipriyaS17 wants to merge 1 commit into
redfishfrom
redfish-offline-get-fallback-fix
Draft

fix(redfish): fix Redfish GET /Systems returns empty body after power state transitions#1051
DevipriyaS17 wants to merge 1 commit into
redfishfrom
redfish-offline-get-fallback-fix

Conversation

@DevipriyaS17
Copy link
Copy Markdown
Contributor

Problem

After power off or power on actions, GET /redfish/v1/Systems/{id} returns HTTP 200 with an empty response body instead of valid JSON. This affected both power state transitions:

  • Power off → GET System fails
  • Power On → Fails to Power on the device
  • Power on → iGET System fails

Root Causes

  1. HTTP write timeout too short — Server's WriteTimeout was 15s. After power transitions, WS-MAN calls take 18–25s, causing the server to close the connection before writing the response body.

  2. Sequential WS-MAN calls with strict error handling — Power state and hardware info calls were sequential. If either failed (device offline, 401 auth error, connection reset), the code returned nil instead of a partial response, resulting in HTTP 200 with empty body.

hspe@BA38RNL00033:~$ curl -sk -u '<username:password>' -H 'Content-Type: application/json' -X POST -d '{"ResetType":"ForceOff"}' https://localhost:8181/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/ComputerSystem.Reset | python3 -m json.tool
{
    "@odata.context": "/redfish/v1/$metadata#Task.Task",
    "@odata.id": "/redfish/v1/TaskService/Tasks/1780566878907650664",
    "@odata.type": "#Task.v1_6_0.Task",
    "EndTime": "2026-06-04T09:54:38Z",
    "Id": "1780566878907650664",
    "Messages": [
        {
            "Message": "The request completed successfully.",
            "MessageId": "Base.1.22.0.Success",
            "Severity": "OK"
        }
    ],
    "Name": "System Reset Task",
    "StartTime": "2026-06-04T09:54:38Z",
    "TaskState": "Completed",
    "TaskStatus": "OK"
}
hspe@BA38RNL00033:~$ curl -sk -u 'standalone:G@ppm0ym' https://localhost:8181/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b | python3 -m json.tool
{
    "@odata.context": "/redfish/v1/$metadata#ComputerSystem.ComputerSystem",
    "@odata.id": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b",
    "@odata.type": "#ComputerSystem.v1_26_0.ComputerSystem",
    "Actions": {
        "#ComputerSystem.Reset": {
            "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/ComputerSystem.Reset",
            "title": "Reset"
        },
        "Oem": {
            "#Oem.Intel.AMT.CancelKVMConsent": {
                "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/Oem/IntelComputerSystem.CancelKVMConsent",
                "title": "Cancel KVM Consent"
            },
            "#Oem.Intel.AMT.GenerateRedirectionToken": {
                "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/Oem/IntelComputerSystem.GenerateRedirectionToken",
                "title": "Generate Redirection Token"
            },
            "#Oem.Intel.AMT.RequestKVMConsent": {
                "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/Oem/IntelComputerSystem.RequestKVMConsent",
                "title": "Request KVM Consent"
            },
            "#Oem.Intel.AMT.SubmitKVMConsentCode": {
                "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/Oem/IntelComputerSystem.SubmitKVMConsentCode",
                "title": "Submit KVM Consent Code"
            }
        }
    },
    "BiosVersion": "WSADLV57.0088.2023.0505.1555",
    "GraphicalConsole": {
        "ConnectTypesSupported": [
            "KVMIP"
        ],
        "Oem": {
            "Intel": {
                "AMT": {
                    "ControlMode": "CCM",
                    "KVMStatus": "Disabled",
                    "UserConsentStatus": "Requested"
                }
            }
        },
        "Port": 16995,
        "ServiceEnabled": false
    },
    "HostName": null,
    "Id": "8c898b4b-7e7b-42f1-9ccf-8abba18ab20b",
    "Manufacturer": "Intel(R) Client Systems",
    "MemorySummary": {
        "TotalSystemMemoryGiB": 16
    },
    "Model": "NUC12WSHv5",
    "Name": "8c898b4b-7e7b-42f1-9ccf-8abba18ab20b",
    "PowerState": "Off",
    "ProcessorSummary": {
        "CoreCount": null,
        "Count": 1,
        "LogicalProcessorCount": null,
        "Model": "12th Gen Intel(R) Core(TM) i5-1250P"
    },
    "SerialConsole": {
        "MaxConcurrentSessions": 1,
        "Oem": {
            "Intel": {
                "AMT": {
                    "ControlMode": "CCM",
                    "SOLStatus": "Disabled",
                    "UserConsentStatus": "NotRequired"
                }
            }
        },
        "WebSocket": {
            "Interactive": true,
            "ServiceEnabled": true
        }
    },
    "SerialNumber": "WO202845L01S002",
    "SystemType": "Physical"
}
hspe@BA38RNL00033:~$ curl -sk -u 'standalone:G@ppm0ym'   -H 'Content-Type: application/json'   -X POST   -d '{"ResetType":"On"}'   https://localhost:8181/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/ComputerSystem.Reset
{"@odata.context":"/redfish/v1/$metadata#Task.Task","@odata.id":"/redfish/v1/TaskService/Tasks/1780566968246685059","@odata.type":"#Task.v1_6_0.Task","EndTime":"2026-06-04T09:56:08Z","Id":"1780566968246685059","Messages":[{"Message":"The request completed successfully.","MessageId":"Base.1.22.0.Success","Severity":"OK"}],"Name":"System Reset Task","StartTime":"2026-06-04T09:56:08Z","TaskState":"Completed","TaskStatus":"OK"}hspe@BA38RNL00033:~$
hspe@BA38RNL00033:~$
hspe@BA38RNL00033:~$ curl -sk -u 'standalone:G@ppm0ym' https://localhost:8181/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b | python3 -m json.tool
{
    "@odata.context": "/redfish/v1/$metadata#ComputerSystem.ComputerSystem",
    "@odata.id": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b",
    "@odata.type": "#ComputerSystem.v1_26_0.ComputerSystem",
    "Actions": {
        "#ComputerSystem.Reset": {
            "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/ComputerSystem.Reset",
            "title": "Reset"
        },
        "Oem": {
            "#Oem.Intel.AMT.CancelKVMConsent": {
                "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/Oem/IntelComputerSystem.CancelKVMConsent",
                "title": "Cancel KVM Consent"
            },
            "#Oem.Intel.AMT.GenerateRedirectionToken": {
                "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/Oem/IntelComputerSystem.GenerateRedirectionToken",
                "title": "Generate Redirection Token"
            },
            "#Oem.Intel.AMT.RequestKVMConsent": {
                "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/Oem/IntelComputerSystem.RequestKVMConsent",
                "title": "Request KVM Consent"
            },
            "#Oem.Intel.AMT.SubmitKVMConsentCode": {
                "target": "/redfish/v1/Systems/8c898b4b-7e7b-42f1-9ccf-8abba18ab20b/Actions/Oem/IntelComputerSystem.SubmitKVMConsentCode",
                "title": "Submit KVM Consent Code"
            }
        }
    },
    "BiosVersion": null,
    "Boot": {
        "AutomaticRetryAttempts": null,
        "BootNext": null,
        "BootSourceOverrideEnabled": "Disabled",
        "BootSourceOverrideMode": "UEFI",
        "BootSourceOverrideTarget": "None",
        "HttpBootUri": null,
        "RemainingAutomaticRetryAttempts": null,
        "UefiTargetBootSourceOverride": null
    },
    "GraphicalConsole": {
        "ConnectTypesSupported": [
            "KVMIP"
        ],
        "Oem": {
            "Intel": {
                "AMT": {
                    "ControlMode": "CCM",
                    "KVMStatus": "Disabled",
                    "UserConsentStatus": "Requested"
                }
            }
        },
        "Port": 16995,
        "ServiceEnabled": false
    },
    "HostName": null,
    "Id": "8c898b4b-7e7b-42f1-9ccf-8abba18ab20b",
    "Manufacturer": null,
    "Model": null,
    "Name": "8c898b4b-7e7b-42f1-9ccf-8abba18ab20b",
    "PowerState": "On",
    "SerialConsole": {
        "MaxConcurrentSessions": 1,
        "Oem": {
            "Intel": {
                "AMT": {
                    "ControlMode": "CCM",
                    "SOLStatus": "Disabled",
                    "UserConsentStatus": "NotRequired"
                }
            }
        },
        "WebSocket": {
            "Interactive": true,
            "ServiceEnabled": true
        }
    },
    "SerialNumber": null,
    "SystemType": "Physical"
}

@DevipriyaS17 DevipriyaS17 force-pushed the redfish-offline-get-fallback-fix branch from 7cc44bc to 28c451c Compare June 4, 2026 10:28
@DevipriyaS17 DevipriyaS17 changed the title fix(redfish) : fix Redfish GET /Systems returns empty body after power state transitions fix(redfish): fix Redfish GET /Systems returns empty body after power state transitions Jun 4, 2026
@DevipriyaS17 DevipriyaS17 force-pushed the redfish-offline-get-fallback-fix branch 2 times, most recently from ede1b7d to 13e6192 Compare June 4, 2026 10:32
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 4, 2026

Codecov Report

❌ Patch coverage is 7.87402% with 117 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.71%. Comparing base (7f7e804) to head (13e6192).

Files with missing lines Patch % Lines
redfish/internal/usecase/wsman_repo.go 0.84% 116 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           redfish    #1051      +/-   ##
===========================================
- Coverage    42.98%   42.71%   -0.28%     
===========================================
  Files          153      153              
  Lines        14853    14952      +99     
===========================================
+ Hits          6385     6387       +2     
- Misses        7888     7984      +96     
- Partials       580      581       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses cases where GET /redfish/v1/Systems/{id} could return HTTP 200 with an empty body after power state transitions, by increasing HTTP server write timeouts and making Redfish system hydration more resilient to slow/failing WS-MAN calls.

Changes:

  • Increased default HTTP server WriteTimeout to accommodate slow WS-MAN responses after power transitions.
  • Updated WS-MAN system aggregation to fan out calls in parallel and return a partial (non-nil) ComputerSystem when some WS-MAN calls fail/timeout.
  • Added bounded-timeout boot settings retrieval (only when the system is reported as powered on) and adjusted logging to use formatted messages.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
redfish/internal/usecase/wsman_repo.go Parallelizes WS-MAN hydration, adds per-subcall timeouts, cached fallbacks, and context/time-budget helpers to avoid empty responses.
redfish/internal/usecase/computer_system.go Bounds boot settings retrieval with a timeout and skips it when the system is powered off.
redfish/internal/controller/http/v1/handler/systems.go Updates error logging to formatted messages for system GET handlers.
redfish/internal/controller/http/v1/handler/systems_test.go Updates logger message assertions to match the new formatted logging strings.
pkg/httpserver/server.go Increases the default HTTP server write timeout from 15s to 60s.

Comment on lines +116 to +120
// Parallel call and timeout constants for system aggregation.
parallelCallCount = 2
wsmanCallTimeout = 8 * time.Second
consoleBudgetMinimum = 1500 * time.Millisecond
controlModeTimeout = 2 * time.Second
Comment on lines +1087 to +1097
func isContextTimeoutOrCancelError(err error) bool {
if err == nil {
return false
}

errText := strings.ToLower(err.Error())

return strings.Contains(errText, "context deadline exceeded") ||
strings.Contains(errText, "context canceled") ||
strings.Contains(errText, "ctx.done")
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants