Add test gnmi-3 Union Replace

[ijernoiu]

Readme location: https://github.com/openconfig/featureprofiles/blob/main/feature/gnmi/tests/union_replace_test/README.md

Issues identified: https://partnerissuetracker.corp.google.com/issues/509552131

Logs available at: https://partnerissuetracker.corp.google.com/issues/415458482#comment202

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new test suite to validate the gNMI Union Replace operation. The tests ensure that both OpenConfig and CLI-based configurations are handled correctly, including scenarios involving configuration overlaps, hardware mismatches, and error handling for invalid configurations. These changes aim to improve the robustness and reliability of gNMI configuration management on supported devices.

Highlights

• New gNMI Union Replace Test: Added a comprehensive test suite to verify gNMI Union Replace functionality, covering idempotent configuration, additions, changes, deletions, and hardware mismatches.
• Platform Exception Added: Updated metadata.textproto to include platform exceptions for Arista devices, specifically for interface enablement and default network instance settings.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[OpenConfigBot]

Pull Request Functional Test Report for #5419 / 3dbe350

Virtual Devices

Device	Test	Test Documentation	Job	Raw Log
Arista cEOS		gNMI-3: union_replace
Cisco 8000E		gNMI-3: union_replace
Cisco XRd		gNMI-3: union_replace
Juniper ncPTX		gNMI-3: union_replace
Nokia SR Linux		gNMI-3: union_replace
Openconfig Lemming		gNMI-3: union_replace

Hardware Devices

Device	Test	Test Documentation	Raw Log
Arista 7808		gNMI-3: union_replace
Cisco 8808		gNMI-3: union_replace
Juniper PTX10008		gNMI-3: union_replace
Nokia 7250 IXR-10e		gNMI-3: union_replace

Help

[gemini-code-assist]

Code Review

This pull request implements a new test suite for gNMI union_replace functionality, including tests for idempotent configurations, additions, deletions, and overlapping configurations across OpenConfig and CLI origins. The feedback identifies several violations of the repository's style guide regarding the use of RFC 5398 compliant ASNs. Additionally, improvements are suggested to handle asynchronous gNMI state updates more reliably using gnmi.Watch and to ensure interface name parsing is robust against different naming conventions to avoid potential panics.

[ijernoiu]

Added new logs at: https://partnerissuetracker.corp.google.com/issues/517964859

[dplore]

Please do not re-write the pre-existing tests. Leave them in place. You may refactor them to be executed as table driven tests, but don't change the logic.

Regarding gnmi-3.7, the requirements were updated to use a configuration containing an invalid interface. ref:

featureprofiles/feature/gnmi/tests/union_replace_test/README.md

Lines 122 to 152 in 4f2c803

	### gNMI-3.7 - union_replace rejected with error in CLI with OC
	Verify a DUT rejects and rolls back a gnmi.Set union_replace with an invalid
	configuration in origin CLI and origin OC. Verify the original configuration is
	preserved.
	Simple issues like a value out of range or referencing a policy that doesn’t
	exist in the OC case will be caught with a validation of the structs. Such an
	error is likely a different code path in a DUT vs. processing a configuration
	that validates but has some semantic error.
	This test uses a configuration containing an invalid interface.
	Steps used for 3.7.1 and 3.7.2:
	1. Get configuration E.1 from DUT.
	2. Generate a configuration E.2 which includes invalid configuration (see sub
	tests).
	3. Push the configuration E.1 + E.2 to the DUT.
	4. Confirm the DUT rejects the gnmi.Set.
	5. Get configuration E.3 from DUT.
	6. Verify E.1 == E.3 (the configuration is unchanged).
	#### gnmi-3.7.1 Invalid interface in OC
	The invalid configuration is configuring MTU for non-existant interface in OC.
	#### gnmi-3.7.2 Invalid interface in CLI
	The invalid configuration is configuring MTU for non-existant interface in CLI.

[dplore]

When run in our environment I get the following results. 3.1 and 3.3.1,2 were passing. Please revert the logic on those tests.

This PR #5419 on hardware EOS-4.36.1F

--- FAIL: TestUnionReplace (891.69s)
    --- FAIL: TestUnionReplace/gNMI-3.1-IdempotentConfiguration (79.07s).   (earlier code passed, please revert)
    --- FAIL: TestUnionReplace/gNMI-3.2.1-AddConfigurationOC (78.37s). (FAIL is expected)
    --- FAIL: TestUnionReplace/gNMI-3.2.2-AddConfigurationCLI (78.19s). (FAIL is expected)
    --- FAIL: TestUnionReplace/gNMI-3.3.1-ChangeConfigurationOC (78.36s). (earlier code passed, please revert)
    --- FAIL: TestUnionReplace/gNMI-3.3.2-ChangeConfigurationCLI (78.13s) (earlier code passed, please revert)
    --- FAIL: TestUnionReplace/gNMI-3.4.1-DeleteByOmissionOC (78.42s)
    --- FAIL: TestUnionReplace/gNMI-3.4.2-DeleteByOmissionCLI (78.15s)
    --- PASS: TestUnionReplace/gNMI-3.5.1-MoveIPBetweenInterfacesOC (27.72s)
    --- FAIL: TestUnionReplace/gNMI-3.5.2-MoveIPBetweenInterfacesCLI (86.51s)
    --- PASS: TestUnionReplace/gNMI-3.6.1-HardwareMismatchOC (19.25s). (PASS is expected)
    --- PASS: TestUnionReplace/gNMI-3.6.2-HardwareMismatchCLI (19.10s). (PASS is expected)
    --- PASS: TestUnionReplace/gNMI-3.7-RejectedInvalidConfig (13.23s)
        --- PASS: TestUnionReplace/gNMI-3.7-RejectedInvalidConfig/gNMI-3.7.1-InvalidInterfaceOC (1.89s)
        --- PASS: TestUnionReplace/gNMI-3.7-RejectedInvalidConfig/gNMI-3.7.2-InvalidInterfaceCLI (1.08s)
    --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected (45.77s)
        --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected/gNMI-3.8.1-DifferentMTUValues (9.91s)
        --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected/gNMI-3.8.2-SameMTUValues (8.77s)
        --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected/gNMI-3.8.3-BGPModelOverlap (8.55s)
        --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected/gNMI-3.8.4-RoutingPolicyOverlap (8.49s)
    --- FAIL: TestUnionReplace/gNMI-3.9.1-NonOverlapCLIAndOC (79.12s)
    --- PASS: TestUnionReplace/gNMI-3.10.1-MissingHardwareOC (23.08s)
    --- PASS: TestUnionReplace/gNMI-3.10.2-MissingHardwareCLI (20.85s)
FAIL

Previous PR #5451 on hardware EOS-4.36.1F

PASS TestUnionReplace3_1_idempotentConfig
FAIL TestUnionReplace3_2_1_addOCMTU
PASS TestUnionReplace3_2_2_addCLIMTU
PASS TestUnionReplace3_3_1_changeOCConfig
PASS TestUnionReplace3_3_2_changeCLIConfig
PASS TestUnionReplace3_6_1
PASS TestUnionReplace3_6_2

[dplore]

When run in our environment I get the following results. 3.1 and 3.3.1,2 were passing. Please revert the logic on those tests.

@navaneethyv can assume review after those changes are made

[ram-mac]

@ijernoiu - The test is failing, below is the test summary.

The DUT is not returning the description "Await(t) on dut(xxxxxxx) at /interfaces/interface[name=Ethernet20/1]/state/description: context deadline exceeded"

--- FAIL: TestUnionReplace (1142.38s)
    --- FAIL: TestUnionReplace/gNMI-3.1-IdempotentConfiguration (93.23s)
    --- FAIL: TestUnionReplace/gNMI-3.2.1-AddConfigurationOC (91.98s)
    --- FAIL: TestUnionReplace/gNMI-3.2.2-AddConfigurationCLI (92.06s)
    --- FAIL: TestUnionReplace/gNMI-3.3.1-ChangeConfigurationOC (92.02s)
    --- FAIL: TestUnionReplace/gNMI-3.3.2-ChangeConfigurationCLI (92.07s)
    --- FAIL: TestUnionReplace/gNMI-3.4.1-DeleteByOmissionOC (91.94s)
    --- FAIL: TestUnionReplace/gNMI-3.4.2-DeleteByOmissionCLI (92.35s)
    --- PASS: TestUnionReplace/gNMI-3.5.1-MoveIPBetweenInterfacesOC (47.04s)
    --- FAIL: TestUnionReplace/gNMI-3.5.2-MoveIPBetweenInterfacesCLI (105.44s)
    --- PASS: TestUnionReplace/gNMI-3.6.1-HardwareMismatchOC (33.36s)
    --- PASS: TestUnionReplace/gNMI-3.6.2-HardwareMismatchCLI (32.94s)
    --- PASS: TestUnionReplace/gNMI-3.7-RejectedInvalidConfig (21.63s)
        --- PASS: TestUnionReplace/gNMI-3.7-RejectedInvalidConfig/gNMI-3.7.1-InvalidInterfaceOC (1.57s)
        --- PASS: TestUnionReplace/gNMI-3.7-RejectedInvalidConfig/gNMI-3.7.2-InvalidInterfaceCLI (1.53s)
    --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected (81.67s)
        --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected/gNMI-3.8.1-DifferentMTUValues (16.35s)
        --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected/gNMI-3.8.2-SameMTUValues (15.43s)
        --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected/gNMI-3.8.3-BGPModelOverlap (15.28s)
        --- PASS: TestUnionReplace/gNMI-3.8-OverlapRejected/gNMI-3.8.4-RoutingPolicyOverlap (14.43s)
    --- FAIL: TestUnionReplace/gNMI-3.9.1-NonOverlapCLIAndOC (92.15s)
    --- PASS: TestUnionReplace/gNMI-3.10.1-MissingHardwareOC (40.79s)
    --- PASS: TestUnionReplace/gNMI-3.10.2-MissingHardwareCLI (34.84s)
FAIL
I0616 04:17:52.154140       1 ondatra.go:423] Ixia debug package metadata extracted:

[navaneethyv]

@ijernoiu can you please share the latest updates and let us know where we are ?

[ijernoiu]

@navaneethyv Please check https://partnerissuetracker.corp.google.com/issues/509736406 for the current status.

[ram-mac]

@ijernoiu - Please take a look at the latest comments in the bug 509736406 and let me know your feedback on this one.

[ram-mac]

With the latest changes the test is now passing for Arista. All relevant details are captured in bug 509736406