Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# MD001/heading-increment:
# Heading levels do not need to increment by one level at a time
MD001: false

# MD012/no-multiple-blanks:
# Two consecutive blank lines are allowed
MD012:
maximum: 2

# MD013/line-length:
# Set line length to 150 characters for now, until README.md is restructured
MD013:
line_length: 100
code_block_line_length: 88

# MD014/commands-show-output:
# Dollar signs used before commands without showing output are permitted
MD014: false

# MD022/blanks-around-headings:
# Headings do not need to be surrounded by blank lines for now
MD022:
lines_above: 0
lines_below: 0

# MD026/no-trailing-punctuation:
# Some trailing punctuation in heading is allowed for now, most not.
MD026:
# Punctuation characters
punctuation: ",;!。,;:!"

# MD032/blanks-around-lists:
# Lists do not need to be surrounded by blank lines for now
MD032: false

# MD033/no-inline-html:
# Allow certain inline HTML elements
MD033:
allowed_elements: [img, kbd]

# MD034/no-bare-urls:
# Bare URLs are allowed for now
MD034: false

# MD045/no-alt-text:
# Images without alt text are allowed for now
MD045: false
8 changes: 8 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,14 @@ repos:
additional_dependencies: [coverage]


- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.45.0
hooks:
- id: markdownlint
# The updated fixes for conftest-README.md are in PR #169
exclude: tests/unit/conftest-README.md


- repo: https://github.com/RobertCraigie/pyright-python
rev: v1.1.408
hooks:
Expand Down
2 changes: 1 addition & 1 deletion README-Windows-WSL2.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

## Recommendations for local Development on Windows and WSL2
# Recommendations for local Development on Windows and WSL2

### Using Docker on WSL2/22.04

Expand Down
6 changes: 4 additions & 2 deletions README-python-install.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## Local Python Development setup on Ubuntu 20.04 / 22.04
# Local Python Development setup on Ubuntu 20.04 / 22.04

For running the Unit tests and static analysis tools, use a relatively modern
Linux VM. Possible are Fedora37+, but the documentation here focuses on
Expand All @@ -7,6 +7,7 @@ Ubuntu20.04 and 22.04 as they are the defaults of WSL and hosts:
```bash
sudo apt update;sudo apt install software-properties-common python{2,3}
```

Ubuntu no longer packages `python2-pip`, so you have to install `pip2`
[this way](https://askubuntu.com/questions/1317353/how-can-i-find-an-older-version-of-pip-that-works-with-python-2-7):

Expand All @@ -21,7 +22,8 @@ echo 'PATH="~/.local/bin/pip:$PATH"' >>~/.bash_aliases
. ~/.bash_aliases
```

You can install specific Python verisons using this PPA:
You can install specific Python versions using this PPA:

```bash
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo apt-get install -y python3.{8,11}{,-distutils}
Expand Down
68 changes: 21 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,30 @@ the development environment for `xen-bugtool`,
a tool designed to assist with debugging XenServer issues.

For more information, see these README files:

Development practices and guidelines:
- [doc/development.md](doc/development.md): Development guidelines and best practices
- [doc/release.md](doc/release.md): Instructions for creating a new release

Setting up the development environment:
- [README-python-install.md](README-python-install.md) - Preparing your
Development VM for running the test suite
- [README-pytest.md](README-pytest.md) - Introduction on the recommended pytest suite for unit tests
- [README-pytest-chroot.md](README-pytest-chroot.md) - Introduction on the `pytest-chroot` test suite
- [README-Windows-WSL2.md](README-Windows-WSL2.md) - Windows and WSL2 setup tips
- [doc/pre-commit.md](doc/pre-commmit.md):
Using `pre-commit` to run the test and static analysis checks locally

Automated tests:
- [doc/testing.md](doc/testing.md): Overview of the different types of tests
- [doc/pre-commit.md](doc/pre-commit.md):
Using `pre-commit` to run tests and static analysis checks locally
- [doc/coverage.md](doc/coverage.md): Introduction on **coverage** from `pytest`

Documentation on pytest and its fixtures:
- [README-pytest.md](README-pytest.md):
Introduction on the recommended pytest suite for unit tests
- [README-pytest-chroot.md](README-pytest-chroot.md):
Introduction on the `pytest`-based integration test suite using namespaces
- [tests/unit/conftest-README.md](tests/unit/conftest-README.md):
Introduction on the `pytest` fixtures defined in `tests/unit/conftest.py`

## Frequently asked Questions

### Why use unit tests, isn't testing optional when we have XenRT?
Expand Down Expand Up @@ -71,7 +86,8 @@ or 15). Beyond these values, functions tend to be difficult to test and maintain
and are thus good candidates for a redesign or refactoring.

You should keep in mind that both metrics are independent of the number of lines
of code in your function. If you have 100 consecutive statements with no branches (conditions, loops, etc.), you'll get a value of 1 for both of them.
of code in your function. If you have 100 consecutive statements with no branches
(conditions, loops, etc.), you'll get a value of 1 for both of them.

#### Blind spots of complexity metrics
##### Consistent and easy to read Code style and formatting rules
Expand Down Expand Up @@ -105,48 +121,6 @@ pip install radon
# Clone python-libs, and host-installer copy perfmon from xen-api, then run:
radon cc xen-bugtool host-installer/ perfmon xcp --total-average -nd --md
```
##### Output:

| Filename | Name | Type | Start:End Line | Complexity | Classification |
| -------- | ---- | ---- | -------------- | ---------- | -------------- |
| xen-bugtool | main | F | 777:1359 | 84 | F |
| xen-bugtool | load_plugins | F | 1761:1827 | 27 | D |
| xen-bugtool | collect_data | F | 701:758 | 21 | D |
| host-installer/install.py | go | F | 89:325 | 60 | F |
| host-installer/backend.py | performInstallation | F | 293:446 | 31 | E |
| host-installer/backend.py | partitionTargetDisk | F | 525:587 | 21 | D |
| host-installer/disktools.py | DOSPartitionTool.writeThisPartitionTable | M | 839:912 | 23 | D |
| host-installer/restore.py | restoreFromBackup | F | 17:177 | 33 | E |
| host-installer/product.py | ExistingInstallation._readSettings | M | 101:412 | 75 | F |
| host-installer/diskutil.py | probeDisk | F | 467:530 | 21 | D |
| host-installer/init | main | F | 92:247 | 35 | E |
| host-installer/init | configureNetworking | F | 28:85 | 24 | D |
| host-installer/tui/repo.py | confirm_load_repo | F | 207:283 | 21 | D |
| host-installer/tui/network.py | get_iface_configuration | F | 15:134 | 29 | D |
| host-installer/tui/installer/screens.py | get_name_service_configuration | F | 795:962 | 28 | D |
| perfmon | main | F | 1307:1522 | 38 | E |
| perfmon | VMMonitor.get_default_variable_config | M | 858:917 | 23 | D |
| xcp/cpiofile.py | CpioFile.open | M | 1003:1083 | 22 | D |
| xcp/bootloader.py | Bootloader.readExtLinux | M | 110:194 | 32 | E |
| xcp/bootloader.py | Bootloader.readGrub | M | 197:301 | 28 | D |
| xcp/bootloader.py | Bootloader.readGrub2 | M | 304:463 | 26 | D |
| xcp/bootloader.py | Bootloader.writeGrub2 | M | 557:619 | 23 | D |
| xcp/net/ifrename/dynamic.py | DynamicRules.generate | M | 147:227 | 23 | D |
| xcp/net/ifrename/logic.py | rename_logic | F | 125:366 | 41 | F |
| xcp/net/ifrename/logic.py | rename | F | 368:498 | 35 | E |
| xcp/net/ifrename/static.py | StaticRules.load_and_parse | M | 103:210 | 25 | D |
| xcp/net/ifrename/static.py | StaticRules.generate | M | 212:292 | 23 | D |

#### Verdict

As the five 5 conditions do not change the functionality of the main code paths,
Python2 compatibility code does not change these complexity numbers.

#### Summary on CC (Cyclomatic Complexity) results
Only `host-installer/install.py/go()` (CC=60) comes close `bugtool/main()` (CC=84).

The other projects are larger, when summarizing their CC into one number, this
indicates that other projects need more tests.

#### Conclusion

Expand Down
2 changes: 1 addition & 1 deletion doc/coverage.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## Unit tests with Code coverage with Codecov and Coveralls
# Unit tests with Code coverage with Codecov and Coveralls

[`pre-commit`](coverage.md) runs `pytest --cov` to collect code coverage
in its native SQLite database in the file `.coverage` and to generate
Expand Down
37 changes: 25 additions & 12 deletions doc/pre-commit.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Running checks using pre-commit

## Running CI checks locally (and in GitHub CI) using `pre-commit`

This project uses `pre-commit` to run the tests in a defined, clean environment:
Expand All @@ -6,10 +8,14 @@ This project uses `pre-commit` to run the tests in a defined, clean environment:
- It runs `pytest` with coverage and checks the coverage on the changed lines
- It runs `pylint`, `mypy`, `pyright` and `pytype`:

The pre-commit configuration defines how it runs
`pytest`, `pylint` and static analysis using `mypy`, `pyright`, and `pytype`.

Links:
- https://mypy.readthedocs.io/en/stable/
- https://microsoft.github.io/pyright/
- https://google.github.io/pytype/user_guide.html

- <https://mypy.readthedocs.io/en/stable/>
- <https://microsoft.github.io/pyright/>
- <https://google.github.io/pytype/user_guide.html>

- Because this project is now using Python3.6+, type annotations (PEP 484)
no longer need to be in comments, but can be in the code directly.
Expand All @@ -27,17 +33,19 @@ uv pip install pre-commit
. .venv/bin/activate # (if using a virtualenv)
pre-commit run -av
```

Without -a, it would only run hooks for staged files with changes.
With `-a`, `pre-commit run -a` runs all fixes and checks.

You can skip checks if you commit by passing `SKIP=` in the environment:

```py
export SKIP=mypy,pylint;git commit -m "quick save" # (or for --amend)
```

Only the 1st invocation of pre-commit will be slow as it creates `virtualenv`s
for each sub-hook configured in its configuration file
[.pre-commit-config.yaml](.pre-commit-config.yaml). Subsequent runs are fast.
[`.pre-commit-config.yaml`](.pre-commit-config.yaml). Subsequent runs are fast.

If a formatting hook like trailing-whitespace fails, just run `git add -p` to
stage the whitespace fixes into the index and run pre-commit again.
Expand All @@ -47,10 +55,13 @@ When you are ready to use pre-commit as a pre-commit hook in this clone,
run this command to install it. Until uninstalled from this repo, it will
then run on each commit. This is completely optional and just a matter of
preference:

```bash
pre-commit install
```

## Videos about pre-commit

### The easy way to keep your repos tidy (5 minutes)
[![The easy way to keep your repos tidy.](https://img.youtube.com/vi/psjz6rwzMdk/0.jpg)](https://www.youtube.com/watch?v=psjz6rwzMdk)

Expand Down Expand Up @@ -83,17 +94,19 @@ new commit in the stack work for itself, you can use a git alias:
[alias]
prebase = rebase -x 'pre-commit run --from-ref HEAD~ --to-ref HEAD'
```

When using `git prebase -i` instead of `git rebase -i`, pre-commit will
run the configured commit hooks for each commit of the rebase.

This ensures that tests also pass on each intermediate commit.

When, during a `git prebase -i`, a pre-commit hook fails or makes changes,
the `rebase` stops, this is the workflow:
- You'll see the reason of the error and get a shell.
- You can then can check `git diff`, in case formatters fixed up the code.
- You can then fix any errors and stage them
- Then, run `git rebase --continue` to continue the rebase to the next step.

A very helpful explanation by the Author of a book on git is here:
https://adamj.eu/tech/2022/11/07/pre-commit-run-hooks-rebase/
the `rebase` stops. These are the steps to fix such failed rebase:

- Review which errors were raised by the `pre-commit` hooks.
- In case formatters fixed the causes already, check `git diff`.
- Fix the errors and stage the fixes using `git add <file>`.
- Run `git rebase --continue` to try to continue the rebase to the next step.

A very helpful explanation by the author of a book on git is here:
<https://adamj.eu/tech/2022/11/07/pre-commit-run-hooks-rebase/>
16 changes: 12 additions & 4 deletions doc/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ This is the process of manually tagging a new version that was merged to master.
There are bump tools which help to automate this process, but they are not added yet.

First switch to the master branch and pull the latest merge.

```bash
$ git switch master
$ git pull origin
Expand All @@ -18,20 +19,26 @@ Date: Thu Feb 22 15:49:23 2024 +0100

CA-389135: Fix the off-by-default and hidden direct-fetched VM RRDs
```

Ensure that your commit hash is identical with the latest commit on master
shown in the GitHub web repository.

Get the latest tag:

```bash
$ git tag | sort -n | tail -n1
v2.0.1
```

Tag a new version using an annotated tag (one which is recorded like a git commit):

```bash
$ git_tag=v2.0.2
$ git tag -m "Tag $git_tag" $git_tag
```

Confirm that the tag was created on the correct commit:

```bash
$ git show $git_tag|head -15
tag v2.0.2
Expand All @@ -50,18 +57,19 @@ Date: Thu Feb 22 15:49:23 2024 +0100
CA-389135: Fix the off-by-default and hidden direct-fetched VM RRDs
$ git push --tags
```
Then wait for some time for the new update to be synced to the mirror repo.

Then wait for some time for the new update to be synced to the mirror repo.

## Creating a new release

After tagging a new version, it would also be good for review to create
a new release:
- With it, you can generate and edit the release notes based on the commits
in it.

- With it, you can generate and edit the release notes based on the commits in it.
- Reviewers can get a better picture of the changes in the release using the release notes.

Navigate to https://github.com/xenserver/status-report/tags:
Navigate to <https://github.com/xenserver/status-report/tags>:

- Visit the new tag:
https://github.com/xenserver/status-report/releases/tag/v2.0.2
- Click the button “Create release from tag”
Expand Down
25 changes: 13 additions & 12 deletions doc/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,21 @@ Unit tests, Integration tests, End-to-End tests.

This section provides a definition of the terms, how they relate and policies.

### The testing pyramid, which is widely used in the testing space:
## The testing pyramid

[![The testing pyramid - Intro](https://upload.wikimedia.org/wikipedia/commons/a/a4/Testing_Pyramid.png)](https://en.wikipedia.org/wiki/Test_automation)
[![The testing pyramid - Intro](https://upload.wikimedia.org/wikipedia/commons/a/a4/Testing_Pyramid.png)](https://en.wikipedia.org/wiki/Test_automation)

[![The Testing Pyramid: How to Structure Your Test Suite](https://semaphoreci.com/wp-content/uploads/2022/03/pyramid-progression.webp)](https://semaphoreci.com/blog/testing-pyramid)
References:
- https://semaphoreci.com/blog/testing-pyramid
- https://web.dev/articles/ta-strategies
- https://semaphoreci.com/wp-content/uploads/2022/03/pyramid-progression.webp
[![The Testing Pyramid: How to Structure Your Test Suite](https://semaphoreci.com/wp-content/uploads/2022/03/pyramid-progression.webp)](https://semaphoreci.com/blog/testing-pyramid)
References:

### Unit tests
- <https://semaphoreci.com/blog/testing-pyramid>
- <https://web.dev/articles/ta-strategies>
- <https://semaphoreci.com/wp-content/uploads/2022/03/pyramid-progression.webp>

## Unit tests

The bulk of tests should be unit tests:

- Tests one unit of execution (one function, class, or even method of a class)
- Very easy to write and update (even AI can be used to generate ideas for them)
- Fast to execute (few milliseconds per unit test)
Expand All @@ -26,17 +28,16 @@ The bulk of tests should be unit tests:
because Mocks are lies to tell “all is fine” and thus may only be used outside
the test target (function, class, or method).

### Integration tests
## Integration tests

- These test the correct integration of a number of functional units
(functions, classes, methods)
- Still easy to write and update
- Still very fast vast to execute (few milliseconds per unit test)
- Still very fast to execute (few milliseconds per unit test)

### End-to-End Tests
## End-to-End Tests

- These test an entire program end-to-end.
- All tests in XenRT are E2E tests, but XenRT tests are even system tests.
- Any test that runs a program as a whole to test its functionality is an E2E test.
- Using a sufficient test environment, many E2E tests are possible without XenRT.
- The final E2E test for `xenserver-status-report` is to get a bug-report through
Expand Down