diff --git a/.markdownlint.yaml b/.markdownlint.yaml new file mode 100644 index 00000000..7d4e517c --- /dev/null +++ b/.markdownlint.yaml @@ -0,0 +1,47 @@ +# MD001/heading-increment: +# Heading levels do not need to increment by one level at a time +MD001: false + +# MD012/no-multiple-blanks: +# Two consecutive blank lines are allowed +MD012: + maximum: 2 + +# MD013/line-length: +# Set line length to 150 characters for now, until README.md is restructured +MD013: + line_length: 100 + code_block_line_length: 88 + +# MD014/commands-show-output: +# Dollar signs used before commands without showing output are permitted +MD014: false + +# MD022/blanks-around-headings: +# Headings do not need to be surrounded by blank lines for now +MD022: + lines_above: 0 + lines_below: 0 + +# MD026/no-trailing-punctuation: +# Some trailing punctuation in heading is allowed for now, most not. +MD026: + # Punctuation characters + punctuation: ",;!。，；：！" + +# MD032/blanks-around-lists: +# Lists do not need to be surrounded by blank lines for now +MD032: false + +# MD033/no-inline-html: +# Allow certain inline HTML elements +MD033: + allowed_elements: [img, kbd] + +# MD034/no-bare-urls: +# Bare URLs are allowed for now +MD034: false + +# MD045/no-alt-text: +# Images without alt text are allowed for now +MD045: false diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index ec87fc48..7e8a7938 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -191,6 +191,14 @@ repos: additional_dependencies: [coverage] +- repo: https://github.com/igorshubovych/markdownlint-cli + rev: v0.45.0 + hooks: + - id: markdownlint + # The updated fixes for conftest-README.md are in PR #169 + exclude: tests/unit/conftest-README.md + + - repo: https://github.com/RobertCraigie/pyright-python rev: v1.1.408 hooks: diff --git a/README-Windows-WSL2.md b/README-Windows-WSL2.md index c18621c2..875cacc2 100644 --- a/README-Windows-WSL2.md +++ b/README-Windows-WSL2.md @@ -1,5 +1,5 @@ -## Recommendations for local Development on Windows and WSL2 +# Recommendations for local Development on Windows and WSL2 ### Using Docker on WSL2/22.04 diff --git a/README-python-install.md b/README-python-install.md index a8c4b506..1975b4cc 100644 --- a/README-python-install.md +++ b/README-python-install.md @@ -1,4 +1,4 @@ -## Local Python Development setup on Ubuntu 20.04 / 22.04 +# Local Python Development setup on Ubuntu 20.04 / 22.04 For running the Unit tests and static analysis tools, use a relatively modern Linux VM. Possible are Fedora37+, but the documentation here focuses on @@ -7,6 +7,7 @@ Ubuntu20.04 and 22.04 as they are the defaults of WSL and hosts: ```bash sudo apt update;sudo apt install software-properties-common python{2,3} ``` + Ubuntu no longer packages `python2-pip`, so you have to install `pip2` [this way](https://askubuntu.com/questions/1317353/how-can-i-find-an-older-version-of-pip-that-works-with-python-2-7): @@ -21,7 +22,8 @@ echo 'PATH="~/.local/bin/pip:$PATH"' >>~/.bash_aliases . ~/.bash_aliases ``` -You can install specific Python verisons using this PPA: +You can install specific Python versions using this PPA: + ```bash sudo add-apt-repository -y ppa:deadsnakes/ppa sudo apt-get install -y python3.{8,11}{,-distutils} diff --git a/README.md b/README.md index f92d751f..b30b46b7 100644 --- a/README.md +++ b/README.md @@ -9,15 +9,30 @@ the development environment for `xen-bugtool`, a tool designed to assist with debugging XenServer issues. For more information, see these README files: + +Development practices and guidelines: +- [doc/development.md](doc/development.md): Development guidelines and best practices +- [doc/release.md](doc/release.md): Instructions for creating a new release + +Setting up the development environment: - [README-python-install.md](README-python-install.md) - Preparing your Development VM for running the test suite -- [README-pytest.md](README-pytest.md) - Introduction on the recommended pytest suite for unit tests -- [README-pytest-chroot.md](README-pytest-chroot.md) - Introduction on the `pytest-chroot` test suite - [README-Windows-WSL2.md](README-Windows-WSL2.md) - Windows and WSL2 setup tips -- [doc/pre-commit.md](doc/pre-commmit.md): - Using `pre-commit` to run the test and static analysis checks locally + +Automated tests: +- [doc/testing.md](doc/testing.md): Overview of the different types of tests +- [doc/pre-commit.md](doc/pre-commit.md): + Using `pre-commit` to run tests and static analysis checks locally - [doc/coverage.md](doc/coverage.md): Introduction on **coverage** from `pytest` +Documentation on pytest and its fixtures: +- [README-pytest.md](README-pytest.md): + Introduction on the recommended pytest suite for unit tests +- [README-pytest-chroot.md](README-pytest-chroot.md): + Introduction on the `pytest`-based integration test suite using namespaces +- [tests/unit/conftest-README.md](tests/unit/conftest-README.md): + Introduction on the `pytest` fixtures defined in `tests/unit/conftest.py` + ## Frequently asked Questions ### Why use unit tests, isn't testing optional when we have XenRT? @@ -71,7 +86,8 @@ or 15). Beyond these values, functions tend to be difficult to test and maintain and are thus good candidates for a redesign or refactoring. You should keep in mind that both metrics are independent of the number of lines -of code in your function. If you have 100 consecutive statements with no branches (conditions, loops, etc.), you'll get a value of 1 for both of them. +of code in your function. If you have 100 consecutive statements with no branches +(conditions, loops, etc.), you'll get a value of 1 for both of them. #### Blind spots of complexity metrics ##### Consistent and easy to read Code style and formatting rules @@ -105,48 +121,6 @@ pip install radon # Clone python-libs, and host-installer copy perfmon from xen-api, then run: radon cc xen-bugtool host-installer/ perfmon xcp --total-average -nd --md ``` -##### Output: - -| Filename | Name | Type | Start:End Line | Complexity | Classification | -| -------- | ---- | ---- | -------------- | ---------- | -------------- | -| xen-bugtool | main | F | 777:1359 | 84 | F | -| xen-bugtool | load_plugins | F | 1761:1827 | 27 | D | -| xen-bugtool | collect_data | F | 701:758 | 21 | D | -| host-installer/install.py | go | F | 89:325 | 60 | F | -| host-installer/backend.py | performInstallation | F | 293:446 | 31 | E | -| host-installer/backend.py | partitionTargetDisk | F | 525:587 | 21 | D | -| host-installer/disktools.py | DOSPartitionTool.writeThisPartitionTable | M | 839:912 | 23 | D | -| host-installer/restore.py | restoreFromBackup | F | 17:177 | 33 | E | -| host-installer/product.py | ExistingInstallation._readSettings | M | 101:412 | 75 | F | -| host-installer/diskutil.py | probeDisk | F | 467:530 | 21 | D | -| host-installer/init | main | F | 92:247 | 35 | E | -| host-installer/init | configureNetworking | F | 28:85 | 24 | D | -| host-installer/tui/repo.py | confirm_load_repo | F | 207:283 | 21 | D | -| host-installer/tui/network.py | get_iface_configuration | F | 15:134 | 29 | D | -| host-installer/tui/installer/screens.py | get_name_service_configuration | F | 795:962 | 28 | D | -| perfmon | main | F | 1307:1522 | 38 | E | -| perfmon | VMMonitor.get_default_variable_config | M | 858:917 | 23 | D | -| xcp/cpiofile.py | CpioFile.open | M | 1003:1083 | 22 | D | -| xcp/bootloader.py | Bootloader.readExtLinux | M | 110:194 | 32 | E | -| xcp/bootloader.py | Bootloader.readGrub | M | 197:301 | 28 | D | -| xcp/bootloader.py | Bootloader.readGrub2 | M | 304:463 | 26 | D | -| xcp/bootloader.py | Bootloader.writeGrub2 | M | 557:619 | 23 | D | -| xcp/net/ifrename/dynamic.py | DynamicRules.generate | M | 147:227 | 23 | D | -| xcp/net/ifrename/logic.py | rename_logic | F | 125:366 | 41 | F | -| xcp/net/ifrename/logic.py | rename | F | 368:498 | 35 | E | -| xcp/net/ifrename/static.py | StaticRules.load_and_parse | M | 103:210 | 25 | D | -| xcp/net/ifrename/static.py | StaticRules.generate | M | 212:292 | 23 | D | - -#### Verdict - -As the five 5 conditions do not change the functionality of the main code paths, -Python2 compatibility code does not change these complexity numbers. - -#### Summary on CC (Cyclomatic Complexity) results -Only `host-installer/install.py/go()` (CC=60) comes close `bugtool/main()` (CC=84). - -The other projects are larger, when summarizing their CC into one number, this -indicates that other projects need more tests. #### Conclusion diff --git a/doc/coverage.md b/doc/coverage.md index 4508addf..48019b0d 100644 --- a/doc/coverage.md +++ b/doc/coverage.md @@ -1,4 +1,4 @@ -## Unit tests with Code coverage with Codecov and Coveralls +# Unit tests with Code coverage with Codecov and Coveralls [`pre-commit`](coverage.md) runs `pytest --cov` to collect code coverage in its native SQLite database in the file `.coverage` and to generate diff --git a/doc/pre-commit.md b/doc/pre-commit.md index 3ffb7f07..49c13d16 100644 --- a/doc/pre-commit.md +++ b/doc/pre-commit.md @@ -1,3 +1,5 @@ +# Running checks using pre-commit + ## Running CI checks locally (and in GitHub CI) using `pre-commit` This project uses `pre-commit` to run the tests in a defined, clean environment: @@ -6,10 +8,14 @@ This project uses `pre-commit` to run the tests in a defined, clean environment: - It runs `pytest` with coverage and checks the coverage on the changed lines - It runs `pylint`, `mypy`, `pyright` and `pytype`: +The pre-commit configuration defines how it runs +`pytest`, `pylint` and static analysis using `mypy`, `pyright`, and `pytype`. + Links: -- https://mypy.readthedocs.io/en/stable/ -- https://microsoft.github.io/pyright/ -- https://google.github.io/pytype/user_guide.html + +- +- +- - Because this project is now using Python3.6+, type annotations (PEP 484) no longer need to be in comments, but can be in the code directly. @@ -27,17 +33,19 @@ uv pip install pre-commit . .venv/bin/activate # (if using a virtualenv) pre-commit run -av ``` + Without -a, it would only run hooks for staged files with changes. With `-a`, `pre-commit run -a` runs all fixes and checks. You can skip checks if you commit by passing `SKIP=` in the environment: + ```py export SKIP=mypy,pylint;git commit -m "quick save" # (or for --amend) ``` Only the 1st invocation of pre-commit will be slow as it creates `virtualenv`s for each sub-hook configured in its configuration file -[.pre-commit-config.yaml](.pre-commit-config.yaml). Subsequent runs are fast. +[`.pre-commit-config.yaml`](.pre-commit-config.yaml). Subsequent runs are fast. If a formatting hook like trailing-whitespace fails, just run `git add -p` to stage the whitespace fixes into the index and run pre-commit again. @@ -47,10 +55,13 @@ When you are ready to use pre-commit as a pre-commit hook in this clone, run this command to install it. Until uninstalled from this repo, it will then run on each commit. This is completely optional and just a matter of preference: + ```bash pre-commit install ``` +## Videos about pre-commit + ### The easy way to keep your repos tidy (5 minutes) [![The easy way to keep your repos tidy.](https://img.youtube.com/vi/psjz6rwzMdk/0.jpg)](https://www.youtube.com/watch?v=psjz6rwzMdk) @@ -83,17 +94,19 @@ new commit in the stack work for itself, you can use a git alias: [alias] prebase = rebase -x 'pre-commit run --from-ref HEAD~ --to-ref HEAD' ``` + When using `git prebase -i` instead of `git rebase -i`, pre-commit will run the configured commit hooks for each commit of the rebase. This ensures that tests also pass on each intermediate commit. When, during a `git prebase -i`, a pre-commit hook fails or makes changes, -the `rebase` stops, this is the workflow: -- You'll see the reason of the error and get a shell. -- You can then can check `git diff`, in case formatters fixed up the code. -- You can then fix any errors and stage them -- Then, run `git rebase --continue` to continue the rebase to the next step. - -A very helpful explanation by the Author of a book on git is here: -https://adamj.eu/tech/2022/11/07/pre-commit-run-hooks-rebase/ +the `rebase` stops. These are the steps to fix such failed rebase: + +- Review which errors were raised by the `pre-commit` hooks. +- In case formatters fixed the causes already, check `git diff`. +- Fix the errors and stage the fixes using `git add `. +- Run `git rebase --continue` to try to continue the rebase to the next step. + +A very helpful explanation by the author of a book on git is here: + diff --git a/doc/release.md b/doc/release.md index c4a98444..ed20f01b 100644 --- a/doc/release.md +++ b/doc/release.md @@ -5,6 +5,7 @@ This is the process of manually tagging a new version that was merged to master. There are bump tools which help to automate this process, but they are not added yet. First switch to the master branch and pull the latest merge. + ```bash $ git switch master $ git pull origin @@ -18,20 +19,26 @@ Date: Thu Feb 22 15:49:23 2024 +0100 CA-389135: Fix the off-by-default and hidden direct-fetched VM RRDs ``` + Ensure that your commit hash is identical with the latest commit on master shown in the GitHub web repository. Get the latest tag: + ```bash $ git tag | sort -n | tail -n1 v2.0.1 ``` + Tag a new version using an annotated tag (one which is recorded like a git commit): + ```bash $ git_tag=v2.0.2 $ git tag -m "Tag $git_tag" $git_tag ``` + Confirm that the tag was created on the correct commit: + ```bash $ git show $git_tag|head -15 tag v2.0.2 @@ -50,18 +57,19 @@ Date: Thu Feb 22 15:49:23 2024 +0100 CA-389135: Fix the off-by-default and hidden direct-fetched VM RRDs $ git push --tags ``` -Then wait for some time for the new update to be synced to the mirror repo. +Then wait for some time for the new update to be synced to the mirror repo. ## Creating a new release After tagging a new version, it would also be good for review to create a new release: -- With it, you can generate and edit the release notes based on the commits -in it. + +- With it, you can generate and edit the release notes based on the commits in it. - Reviewers can get a better picture of the changes in the release using the release notes. -Navigate to https://github.com/xenserver/status-report/tags: +Navigate to : + - Visit the new tag: https://github.com/xenserver/status-report/releases/tag/v2.0.2 - Click the button “Create release from tag” diff --git a/doc/testing.md b/doc/testing.md index d66a602e..b54ea6d7 100644 --- a/doc/testing.md +++ b/doc/testing.md @@ -5,19 +5,21 @@ Unit tests, Integration tests, End-to-End tests. This section provides a definition of the terms, how they relate and policies. -### The testing pyramid, which is widely used in the testing space: +## The testing pyramid - [![The testing pyramid - Intro](https://upload.wikimedia.org/wikipedia/commons/a/a4/Testing_Pyramid.png)](https://en.wikipedia.org/wiki/Test_automation) +[![The testing pyramid - Intro](https://upload.wikimedia.org/wikipedia/commons/a/a4/Testing_Pyramid.png)](https://en.wikipedia.org/wiki/Test_automation) - [![The Testing Pyramid: How to Structure Your Test Suite](https://semaphoreci.com/wp-content/uploads/2022/03/pyramid-progression.webp)](https://semaphoreci.com/blog/testing-pyramid) - References: - - https://semaphoreci.com/blog/testing-pyramid - - https://web.dev/articles/ta-strategies - - https://semaphoreci.com/wp-content/uploads/2022/03/pyramid-progression.webp +[![The Testing Pyramid: How to Structure Your Test Suite](https://semaphoreci.com/wp-content/uploads/2022/03/pyramid-progression.webp)](https://semaphoreci.com/blog/testing-pyramid) +References: -### Unit tests +- +- +- + +## Unit tests The bulk of tests should be unit tests: + - Tests one unit of execution (one function, class, or even method of a class) - Very easy to write and update (even AI can be used to generate ideas for them) - Fast to execute (few milliseconds per unit test) @@ -26,17 +28,16 @@ The bulk of tests should be unit tests: because Mocks are lies to tell “all is fine” and thus may only be used outside the test target (function, class, or method). -### Integration tests +## Integration tests - These test the correct integration of a number of functional units (functions, classes, methods) - Still easy to write and update -- Still very fast vast to execute (few milliseconds per unit test) +- Still very fast to execute (few milliseconds per unit test) -### End-to-End Tests +## End-to-End Tests - These test an entire program end-to-end. -- All tests in XenRT are E2E tests, but XenRT tests are even system tests. - Any test that runs a program as a whole to test its functionality is an E2E test. - Using a sufficient test environment, many E2E tests are possible without XenRT. - The final E2E test for `xenserver-status-report` is to get a bug-report through