Skip to content

Add vendor-independent leak test API - #735

Merged
judyjoseph merged 1 commit into
sonic-net:masterfrom
nexthop-ai:chinmoy.leak-test.api.base
Sep 18, 2026
Merged

judyjoseph merged 1 commit into
sonic-net:masterfrom
nexthop-ai:chinmoy.leak-test.api.base

Conversation

@chinmoy-nexthop

Copy link
Copy Markdown
Contributor

Add a common API for injecting a simulated leak into the leak detection path, so the reporting chain can be validated without wetting hardware.

Injection is non-destructive: an injected leak is published like any other leak, and is additionally flagged through LeakageSensorBase is_test_leak() so consumers must not take a mitigation action on it.

  • leakage_sensor_test_base.py: new LeakageSensorTestBase defining is_leak_test_supported(), set_test_leak(), is_test_leak_enabled() and clear_test_leaks()
  • liquid_cooling_base.py: add is_test_leak() on LeakageSensorBase and get_leak_sensor_test() on LiquidCoolingBase, defaulting to False and None so platforms without injection support are unaffected
  • tests/leak_test_api_base.py: LeakTestApiBase, a reusable conformance suite a platform subclasses to validate its implementation against the common contract

Description

Motivation and Context

How Has This Been Tested?

Additional Information (Optional)

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@judyjoseph

Copy link
Copy Markdown
Contributor

@fraserg-arista to review as well, this could be first step on sonic-net/SONiC#2441

@nikamirrr nikamirrr left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Duplicate submission from a retried API call — superseded by the review below, which carries all 15 inline comments. Please disregard this entry.)

@nikamirrr nikamirrr left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review pass focused on correctness. Fifteen inline comments below.

Findings 1, 2, 6, 7, 8, 9, 10, 11, 14 and 15 were confirmed by executing the code rather than
by reading it — including running the full LeakTestApiBase suite against both a correct and a
deliberately-sloppy implementation.

The through-line is the PR's central safety promise: an injected leak is published like a real one
but must never trigger a mitigation action
. Three independent gaps each break it on their own:

  1. is_test_leak() is decoupled from is_leak(), so a stale flag exempts a real leak from
    mitigation.
  2. The reference clear_test_leaks() erases real leaks on sensors that were never injected.
  3. The flag reaches no consumer and no STATE_DB field, while set_test_leak defaults to CRITICAL
    and system_critical_leak_action defaults to power_off — so a "non-destructive" test can power
    the switch off.

The conformance suite cannot catch any of the three: I built an implementation that is wrong in
exactly the dangerous way and it passes 8/8.

Also worth a look, below the cut: a shared mutable default leakage_sensors_list=[] in
LiquidCoolingBase.__init__; test_injection_is_non_destructive being a tautology that passes when
injection is a no-op; and @abstractmethod ... pass against this repo's own written rule in
.github/copilot-instructions.md ("Abstract methods: Raise NotImplementedError in base class"),
which also makes super().set_test_leak(...) silently return a falsy None.

Comment thread sonic_platform_base/liquid_cooling_base.py Outdated
Comment thread tests/leakage_sensor_test_base_test.py Outdated
Comment thread sonic_platform_base/liquid_cooling_base.py
Comment thread tests/leak_test_api_base.py Outdated
Comment thread sonic_platform_base/leakage_sensor_test_base.py Outdated
Comment thread sonic_platform_base/liquid_cooling_base.py
Comment thread sonic_platform_base/leakage_sensor_test_base.py Outdated
Comment thread tests/leak_test_api_base.py Outdated
Comment thread tests/leak_test_api_base.py Outdated
Comment thread sonic_platform_base/liquid_cooling_base.py
@mssonicbld

Copy link
Copy Markdown
Collaborator

This PR has backport request label(s) for branch(es): msft-202608, but is missing required test information. Please make sure you tick the tested branch(es) in the Tested branch section and provide test evidence (e.g., 202608: <test result>) in the Test result section as well in your PR description.

---Powered by SONiC BuildBot

@chinmoy-nexthop
chinmoy-nexthop force-pushed the chinmoy.leak-test.api.base branch from 508549d to 9ef9f23 Compare August 14, 2026 09:20
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@chinmoy-nexthop

Copy link
Copy Markdown
Contributor Author

(Duplicate submission from a retried API call — superseded by the review below, which carries all 15 inline comments. Please disregard this entry.)

@nikamirrr Thank you.
It is absolutely fine. Maybe your agent was a little too verbose. I have looked into your feedback and taken care of a few things that I felt were appropriate.

Addressed the following:-

  • test_leak is now a class attribute, so is_test_leak() is safe on subclasses that don't chain super().init()
  • is_test_leak() only reports True while the sensor is actually leaking, so a stale flag can't mask a real leak
  • get_leak_severity() returns None when there's no leak, as documented
  • Fixed get_profile() — it was using getattr() on a dict and could never find a profile
  • Added a TYPE_CHECKING import so the LeakageSensorTestBase annotation resolves
  • Moved the conformance suite into sonic_platform_base/ so it ships in the wheel, and added an in-repo subclass so it runs in CI
  • Suite now clears injected leaks on teardown even when an assertion fails, fails loudly if SENSOR_NAMES isn't set, and verifies is_test_leak() is False after withdrawal
  • Reference clear_test_leaks() only touches injected sensors, restores prior sensor state, and returns the aggregated result

@mssonicbld

Copy link
Copy Markdown
Collaborator

Hi, there are workflow run(s) waiting for approval, you may be first-time contributor. I will notify maintainers to help approve once PR is approved. Thanks!

---Powered by SONiC BuildBot

nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 23, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 24, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 25, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 26, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 27, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 27, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 28, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 29, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 29, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 31, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Aug 31, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Sep 1, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Sep 1, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 9ef9f23 [case: upstream:open]
@judyjoseph
judyjoseph requested a review from nikamirrr September 1, 2026 17:14
@chinmoy-nexthop
chinmoy-nexthop force-pushed the chinmoy.leak-test.api.base branch from 9ef9f23 to 7077649 Compare September 16, 2026 15:35
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

judyjoseph
judyjoseph previously approved these changes Sep 16, 2026

@judyjoseph judyjoseph left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok to me ..

One comment on the below defenition

test_leak: bool = False as a class attribute

@mssonicbld

Copy link
Copy Markdown
Collaborator

Hi @sonic-net/sonic-platform-common-maintainer, this approved PR has workflow run(s) waiting for approval. Please help review. Thanks!

---Powered by SONiC BuildBot

nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Sep 17, 2026
rebuild-source: sonic-net#735 @ nexthop-ai/sonic-platform-common 7077649 [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-platform-common that referenced this pull request Sep 17, 2026
rebuild-source: sonic-net/pull/735 @ nexthop-ai/sonic-platform-common 7077649 [case: upstream:open]
@chinmoy-nexthop

Copy link
Copy Markdown
Contributor Author

Looks ok to me ..

One comment on the below defenition

test_leak: bool = False as a class attribute

Thank you @judyjoseph , Removed the redundant test_leak class attribute from LeakageSensorBase

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@nikamirrr nikamirrr left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rework — the is_leak() / get_leak_severity() / is_test_leak() contract landed as we discussed on the Option 1 thread, and the TYPE_CHECKING guard for the LeakageSensorTestBase forward reference is in. CI is green here.

One code item left, plus a process one:

  1. is_test_leak() is no longer safe on subclasses that skip super().__init__(). The reply on the line-34 thread said the class-level test_leak = False was kept as a read fallback; it is not on this head, so a leaking out-of-tree sensor now raises AttributeError. One-line fix, commented inline.
  2. The Option 1 follow-ups. The two sonic-buildimage changes this depends on do not seem to be raised yet, and the caveat that was going into the PR description is not there. Until the ast2700 change lands, is_test_leak() is hardwired to False on that platform — worth stating in the description so it is discoverable.

The third comment is a suggestion on the reference clear_test_leaks(), which vendors will copy.

Happy to re-review as soon as the first one is addressed.

Comment thread sonic_platform_base/liquid_cooling_base.py
Comment thread sonic_platform_base/liquid_cooling_base.py
Comment thread tests/leakage_sensor_test_base_test.py Outdated
Add a common API for injecting a simulated leak into the leak detection
path, so the reporting chain can be validated without wetting hardware.

Injection is non-destructive: an injected leak is published like any
other leak, and is additionally flagged through LeakageSensorBase
is_test_leak() so consumers must not take a mitigation action on it.

- leakage_sensor_test_base.py: new LeakageSensorTestBase defining
  is_leak_test_supported(), set_test_leak(), is_test_leak_enabled() and
  clear_test_leaks()
- liquid_cooling_base.py: add is_test_leak() on LeakageSensorBase and
  get_leak_sensor_test() on LiquidCoolingBase, defaulting to False and
  None so platforms without injection support are unaffected
- tests/leak_test_api_base.py: LeakTestApiBase, a reusable conformance
  suite a platform subclasses to validate its implementation against the
  common contract

Signed-off-by: Chinmoy Dey <chinmoy@nexthop.ai>
@chinmoy-nexthop
chinmoy-nexthop force-pushed the chinmoy.leak-test.api.base branch from 05f8e3a to 7faa761 Compare September 18, 2026 07:41
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@judyjoseph judyjoseph left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@judyjoseph
judyjoseph merged commit 2f4aeaf into sonic-net:master Sep 18, 2026
6 checks passed
@mssonicbld

Copy link
Copy Markdown
Collaborator

Cherry-pick PR to msft-202608: Azure/sonic-platform-common.msft#137

@chinmoy-nexthop

Copy link
Copy Markdown
Contributor Author

Thank you for looking into this, @judyjoseph @nikamirr @parvathi-nexthop . I really appreciate your valuable comments and feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants