Repository navigation
CI Pipeline Reports Successful Linting Despite Notebook Linter Failing to Start #38
Description
Activity
Approach to Resolving the Issue
Initially, I thought of resolving this issue by saving the script as a file and executing it with valid permissions, or even moving completely from Python and translating the same logic into a bash script.
But while testing out the
pynblintlocally, I realised that this is more of a static analyser than a proper linter.
With that, I started searching for other options that better suit the current requirement, and I ended up finding about
nbqatool that lets us lint Jupyter Notebook with a linter of our choice.Using the
nbqatool also helps us overcome the issue which we faced initially withpynblint, i.e., the TypeError raised by theTyperlibrary that is being internally used by thepynblinttool.And since we're expecting to have more contributions in the form of Jupyter Notebooks compared to Python files, it might be better to choose a more verbose, highly descriptive and strict linter for Jupyter.
My Initial go-to choice was
flake8due to its wide usage in industry-level linting workflows and also 'cause we're already using it to perform linting for Python files. But after a bit of exploration, I felt likepylintmight be a better choice and match the description of the requirement that I just mentioned earlier.
The CI pipeline that performs linting checks is generating false positives, as the success code is 0 while the actual linting task for Jupyter Notebooks fails to start. This problem has existed for some time and was recently observed during the last PR merge.
Analysis
pynblint <path>, which appears to use the typer library to create a CLI for linting notebook files.pynblinttool fails to run, causing the workflow to error out for the linting step.Other observation
Also, using linting tools like
pynblintto perform linting on Jupyter Notebooks might not be the best choice, sincepynblintnever really throws a failed status code to fail the whole linting workflow.Well, it can be configured to throw an error status code to fail the linting jobs when a potential error is found. However, I believe there are many better tools to handle it properly with more flexibility.