[test/reboot] Replace clock-based reboot assertion with /proc/uptime check (#26591) - #1411
Merged
Merged
Conversation
…check (#26591) Replace the clock-based reboot assertion in `tests/common/reboot.py` with a `/proc/uptime` check. The old logic compared absolute timestamps across a reboot boundary and failed on platforms with RTC drift when NTP didn't sync within 120s. The new check is monotonic and immune to clock issues. Observed failure: `test_cold_reboot[gnoi_based-]` device rebooted (tmpfs file gone) but clock assertion fired due to 3.5min RTC drift. - [x] Bug fix - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [x] 202511 - [ ] 202512 - [ ] 202605 - [x] master - master: `test_cold_reboot[gnoi_based]` now passes `/proc/uptime` shows ~180s after reboot, well under the 420s threshold. RTC drift causes false "Device did not reboot" assertion on platforms where NTP doesn't sync within 120s of SSH availability. Read `/proc/uptime` after reboot. A freshly-rebooted device must have uptime < `timeout + wait`. This is independent of wall-clock accuracy. Confirmed `/proc/uptime` reads ~180s on the affected DUT post-reboot. All other reboot tests (cli_based, watchdog, continuous) unaffected. Observed on Arista 7060X6 with ~3.5min RTC drift. Applies to any platform with slow NTP sync after boot. --------- Signed-off-by: Priyansh Tratiya <ptratiya@microsoft.com> (cherry picked from commit 5a5e161) Signed-off-by: bingwang <bingwang@microsoft.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Collaborator
|
/azp run |
|
Azure Pipelines: 1 pipeline(s) were filtered out due to trigger conditions. |
This was referenced Sep 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
Cherry-pick of sonic-net/sonic-mgmt#26591 to Azure/sonic-mgmt.msft:202512.
Replace the clock-based reboot assertion in
tests/common/reboot.pywith a/proc/uptimecheck. The old logic compared absolute timestamps across a reboot boundary and failed on platforms with RTC drift when NTP didn't sync within 120s. The new check is monotonic and immune to clock issues.Fixes # (issue)
Type of change
Back port request
Approach
What is the motivation for this PR?
RTC drift causes false "Device did not reboot" assertion on platforms where NTP doesn't sync within 120s of SSH availability.
How did you do it?
Cherry-picked sonic-net/sonic-mgmt#26591 (squash-merge commit 5a5e161), resolving conflicts caused by this branch not yet having the
reboot_started_event/gNOI-smartswitch-signature feature from a separate, unrelated upstream commit. Kept this branch's existingreboot_smartswitch(duthost, pool, reboot_type)signature andperform_rebootinvocation_type handling, while applying the core fix: replacing the wall-clock (get_now_time/get_up_timecomparison) reboot assertion with a monotonic/proc/uptime-based check (reboot_start_time = time.monotonic()captured before reboot, compared againstduthost.get_uptime()with aREBOOT_UPTIME_GRACE_SECONDSgrace margin), and removing the now-unusedpositive_uptimehelper. Also dropped the unrelatedtests/ha/test_ha_npu_reboot.pychange since that file does not exist on this branch.How did you verify/test it?
Verified the resolved file has no leftover conflict markers, passes
python3 -m py_compile/ AST parse, and passesflake8 --max-line-length=120(only pre-existing, unrelated warnings remain).Any platform specific information?
Generic — affects any testbed running with RTC drift or slow NTP sync after boot.
Supported testbed topology if it's a new test case?
N/A — existing test case, topology unchanged.
Documentation
N/A