Adding k8s network policy benchmarking test between cilium and npm#599
Adding k8s network policy benchmarking test between cilium and npm#599agrawaliti wants to merge 43 commits into
Conversation
…namespace configurations
… churn benchmarks
agrawaliti
requested review from
alyssa1303,
anson627,
jshr-w,
rafael-mendes-pereira and
sumanthreddy29
as code owners
This pull request updates the Azure AKS CLI preview extension configuration in the Terraform module to specify a version for the extension. * [`modules/terraform/azure/aks-cli/main.tf`](diffhunk://#diff-1c09e32cd63aa3f4a6cfae577b3db448daedf498df32bd8834fd4344f6b86ab4R82-R83): Added the `--version` flag with a value of `14.0.0b2` to ensure a specific version of the `aks-preview` extension is used.
| default: {} | ||
|
|
||
| steps: | ||
| - script: | |
There was a problem hiding this comment.
Same here. Revert as this will affect all pipelines
There was a problem hiding this comment.
is that a problem? its just collecting start time, and show that.
There was a problem hiding this comment.
As a best practice, minimize the code changes to unrelated areas. While this most likely has 0 impact, we should scope changes to impact just our code.
There was a problem hiding this comment.
You can put this in here steps/topology/network-churn/validate-resources.yml. We only put change in framework files if we want to it apply for all tests.
jshr-w
left a comment
jshr-w
left a comment
There was a problem hiding this comment.
Please share the runs on our default Cluster Churn and Service Churn pipelines off this branch first so we can make sure they aren't broken.
…ource management conditions
…and network-churn configurations
…ate_config function
| jq --arg telescope_run_id $RUN_ID --arg start_timestamp $START \ | ||
| -c '. + {telescope_run_id: $telescope_run_id, start_timestamp: $start_timestamp}' $(TEST_RESULTS_FILE) > temp-$RUN_ID.json \ |
There was a problem hiding this comment.
This change affects all tests running in Telescope. Please revert this change and collect the start time as part of the collect implementation for this test.
| - template: /steps/engine/clusterloader2/cilium/scale-cluster.yml | ||
| parameters: | ||
| role: net | ||
| region: ${{ parameters.regions[0] }} | ||
| nodes_per_nodepool: 240 | ||
| enable_autoscale: "false" |
There was a problem hiding this comment.
This step should no longer be needed as we use terraform to create new cluster every run. You should update terraform file to create cluster with your desired number of node
| }, | ||
| { | ||
| name = "userpool0" | ||
| node_count = 0 |
There was a problem hiding this comment.
Just put 240 her and disable auto-scaling. This should create a cluster with 240 nodes for user pool
There was a problem hiding this comment.
This file can go to folder called kubernetes under your scenario folder: scenarios/perf-eval/network-policy-churn/. That's where we usually store k8s files to apply for a specific scenario. Example: file
…nd add timestamp appending to test results
…r network policy churn scenario
…scenario-specific directory
5 participants
Integrate network policy enforcement latency measurement.
Developed pipelines to compare network policy-related metrics between Azure powered by Cilium and Azure powered by CNI Overlay using Network Policy Manager.
All the configuration like nodes, pods, n
This pull request introduces significant changes to support network policy enforcement latency testing and related metrics in the cluster loader configuration. It also includes updates to enable secondary cluster usage in competitive tests. Below is a categorized breakdown of the most important changes:
Network Policy Enforcement Latency Testing:
deployment_template.yamlto support network policy enforcement latency tests, including conditional labels, a new server container, and specific configurations for test pods. [1] [2] [3]net-policy-enforcement-latency.yamlandnet-policy-metrics.yamlto define steps for measuring network policy enforcement latency and collecting related metrics. [1] [2]load-config.yamlto include parameters and steps for enabling and running network policy tests, such as pod creation and policy creation latency measurements. [1] [2] [3] [4] [5]Competitive Test Enhancements:
run_id_2anduse_secondary_clusterparameters tocompetitive-test.ymlto allow tests to optionally use a secondary cluster. Updated job parameters to support these additions. [1] [2] [3]Configuration Updates:
load-config.yamlto dynamically calculate the number of namespaces and pods per namespace based on new parameters.slo.pyto account for network tests, including a hardcoded value for nodes per namespace during network testing.Reconciliation Enhancements:
reconcile-objects.yamlto include network policy-related parameters and integration with the new deployment template. [1] [2]amespaces, no. of policies per namespace can be updated in pipeline.Pipeline: https://dev.azure.com/akstelescope/telescope/_build?definitionId=41
Dashboard with new metrics: https://dataexplorer.azure.com/dashboards/e033bb3b-2cf4-4263-b41b-31597a8c4401?p-_startTime=24hours&p-_endTime=now&p-_cluster=v-cilium_network_churn_main&p-_test-type=v-default-config#5117e0aa-eb12-4f7f-b55d-6ffba1eab4ad