Skip to content

feat(docs): Enhance content for Section 2.1 on Imports #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
google-labs-jules wants to merge 2 commits into from
Closed

feat(docs): Enhance content for Section 2.1 on Imports #26

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cb038e9
I've just finished overhauling the documentation for you in Section 2…
google-labs-jules[bot] Aug 7, 2025
e5f86c7
Merge branch 'main' into feat/improve-imports-documentation
fmind Aug 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 66 additions & 76 deletions docs/2. Prototyping/2.1. Imports.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,115 +1,105 @@
---
description: Learn best practices for organizing Python imports in notebooks to ensure clarity, maintainability, and efficient code management.
---
# 📝 2.1. Managing Imports

# 2.1. Imports
Properly managing imports is a cornerstone of a well-structured and maintainable AI/ML project. It ensures that your code is clean, readable, and scalable. In the prototyping phase, especially within notebooks, it's easy to create a tangled web of dependencies. This section will guide you through the best practices for handling imports, moving from common pitfalls to professional standards.

## What are code imports?
## The "Quick and Dirty" Method: `sys.path`

In Python, **[code imports](https://docs.python.org/3/reference/import.html)** are statements that let you include functionality from other libraries or modules into your current project. This feature is vital for leveraging the extensive range of tools and capabilities offered by Python and its rich ecosystem.
When you start working on a project with multiple notebooks or scripts, you'll inevitably need to share code between them. A common first attempt is to manipulate Python's `sys.path`.

As outlined by [PEP 8](https://peps.python.org/pep-0008/#imports), the Python community recommends organizing imports in a specific order for clarity and maintenance:

1. **Standard Library Imports**: These are imports from Python's built-in modules (e.g., `os`, `sys`, `math`). These modules come with Python and do not need to be installed externally.
2. **Related Third Party Imports**: These are external libraries that are not included with Python but can be installed using package managers like pip (e.g., `numpy`, `pandas`). They extend Python's functionality significantly.
3. **Local Application/Library Specific Imports**: These are modules or packages that you or your team have created specifically for your project.

Here's an example to illustrate how imports might look in a Python script or notebook:
For example, if you have a utility function in `src/utils.py` and you want to use it in `notebooks/prototype.ipynb`, you might be tempted to do this:

```python
import os # Standard library module
import sys

import pandas as pd # External library module
# Add the project root to the Python path
sys.path.append("..")

from my_project import my_module # Internal project module
from src import utils
```

## Which packages do you need for your project?
**Why you should avoid this:**

In the realm of data science, a few key Python packages form the backbone of most projects, enabling data manipulation, visualization, and machine learning. Essential packages include:
* **It's brittle:** This approach depends on the file structure. If you move the notebook, the relative path `..` might break.
* **IDE Confusion:** Your Integrated Development Environment (IDE) like VS Code might not recognize these "on-the-fly" path changes, leading to incorrect error highlighting and no autocompletion for your modules.
* **It's not portable:** When someone else clones your project, they might have a different structure, or they might not realize they need to run the notebook from a specific directory.

- **[Pandas](https://pandas.pydata.org/)**: For data manipulation and analysis.
- **[NumPy](https://numpy.org/)**: For numerical computing and array manipulation.
- **[Matplotlib](https://matplotlib.org/) or [Plotly](https://plotly.com/)**: For creating static, interactive, and animated visualizations.
- **[Scikit-learn](https://scikit-learn.org/stable/)**: For machine learning, providing simple and efficient tools for data analysis and modeling.
While `sys.path` manipulation can be a quick fix, it leads to code that is hard to maintain and understand in the long run.

To integrate these packages into your project using uv, you can execute the following command in your terminal:
## The Professional Approach: Editable Installs

A much cleaner and more robust solution is to treat your project as a Python package. By installing your project in "editable" mode, you make your project's modules available everywhere in your environment, just like any other library (e.g., `pandas`, `scikit-learn`).

You can do this with a single command at the root of your project:

```bash
uv add pandas numpy matplotlib scikit-learn plotly
# Using uv (recommended for this course)
uv pip install -e .

# Or using standard pip
pip install -e .
```

This command tells uv to download and install these packages, along with their dependencies, into your project environment, ensuring version compatibility and easy package management.
The `-e` flag stands for "editable," which means that any changes you make to your source code are immediately reflected in the installed package without needing to reinstall it.

## How should you organize your imports to facilitate your work?
**Benefits of this approach:**

Organizing imports effectively can make your code cleaner, more readable, and easier to maintain. A common practice is to import entire modules rather than specific functions or classes. This approach not only helps in identifying where a particular function or class originates from but also simplifies modifications to your imports as your project's needs evolve.
* **Clean Imports:** You can now use absolute imports from anywhere in your project, which is much cleaner and more readable.
```python
# No more sys.path hacks!
from mlops.dataset import load_data
```
* **IDE Friendly:** Your IDE will now correctly recognize your project's structure, giving you features like autocompletion, go-to-definition, and refactoring support.
* **Consistency:** It ensures that your prototyping environment (notebooks) and your production environment (scripts) handle imports in the exact same way.
* **Collaboration:** Anyone who clones your repository can set up their environment with the same single command, ensuring consistency across the team.

Consider the following examples:
This is the standard and recommended way to manage dependencies within a Python project.

```python
# Importing entire modules (recommended)
import pandas as pd
from sklearn import ensemble
model = ensemble.RandomForestClassifier()
## Best Practices for Clean Imports

# Importing specific functions/classes
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier()
```
Once you have your project set up for proper importing, it's important to keep your import statements clean and organized.

Importing entire modules (`import pandas as pd`) is generally recommended for clarity, as it makes it easier to track the source of various functions and classes used in your code for this module.
### 1. Organize Your Imports (PEP 8)

## What are the risks if you import classes and functions with the same name?
[PEP 8](https://peps.python.org/pep-0008/#imports), the official style guide for Python code, recommends grouping imports in the following order:

Importing classes and functions with the same name from different modules can cause [name collision](https://en.wikipedia.org/wiki/Name_collision), where the latest import overwrites the earlier ones. This can lead to unexpected behavior and make debugging more challenging. Additionally, it reduces code clarity, making the program harder to maintain and understand.
1. **Standard library imports** (e.g., `sys`, `os`, `json`)
2. **Third-party library imports** (e.g., `pandas`, `numpy`, `sklearn`)
3. **Local application/library specific imports** (your own project's modules)

For example, consider you import `load` from two different modules in Python:
Here’s an example:

```python
from module1 import load
from module2 import load # overwrite load imported from module1
```
# 1. Standard library
import json
import sys
from pathlib import Path

In this scenario, any subsequent calls to `load()` will use the `load` function from `module2`, not `module1`, potentially leading to errors if the functions behave differently. To avoid such issues, you could use aliases:
# 2. Third-party libraries
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split

```python
from module1 import load as load1
from module2 import load as load2`
# 3. Local application
from mlops.dataset import load_data
from mlops.models import train_model
```

Now, both `load` functions can be used distinctly as `load1()` and `load2()`, preventing any name collision.

## Are there any side effects when importing modules in Python?
This structure makes it easy to see the dependencies of a script at a glance. Tools like `isort` can even automate this for you.

Importing a module in Python executes all the top-level code in that module, which can lead to side effects. These effects can be both intentional and unintentional. It's crucial to import modules from trusted sources to avoid security risks or unexpected behavior. Be especially cautious of executing code with side effects in your own modules, and make sure any such behavior is clearly documented.
### 2. Use Absolute Imports

Consider this cautionary example:
With an editable install, you should favor absolute imports over relative imports.

```python
# A module with a potentially harmful operation
# lib.py
import os
os.system("rm -rf /") # This command is extremely dangerous!

# main.py
import lib # Importing lib.py could lead to data loss
```
* **Absolute import:** `from my_package.my_module import my_function`
* **Relative import:** `from ..my_module import my_function`

## What should you do if packages cannot be imported from your notebook?

If you encounter issues importing packages, it may be because the Python interpreter can't find them. This problem is common when using virtual environments. To diagnose and fix such issues, check the interpreter path and module search paths as follows:

```python
import sys
print("Interpreter path:", sys.executable)
print("Module search paths:", sys.path)
```
Absolute imports are more explicit and prevent ambiguity. You can tell exactly where the module is located within your project structure. Relative imports can be convenient for modules deep within a package, but for top-level scripts and notebooks, absolute imports are much safer and clearer.

Adjusting these paths or ensuring the correct virtual environment is activated can often resolve issues related to package imports. With VS Code, you can [select the Python environment](https://code.visualstudio.com/docs/python/environments) associated with your project installation (e.g., `.venv`).
## Key Takeaways

## Imports additional resources
* **Avoid `sys.path` manipulation.** It's a temporary fix that creates long-term problems.
* **Install your project in editable mode** (`uv pip install -e .`). This is the professional standard.
* **Organize your imports** into three sections: standard library, third-party, and local application.
* **Prefer absolute imports** for clarity and maintainability.

- **[Imports example from the MLOps Python Package](https://github.com/fmind/mlops-python-package/blob/main/notebooks/prototype.ipynb)**
- [Python import: Advanced Techniques and Tips](https://realpython.com/python-import/)
- [The Python import system](https://docs.python.org/3/reference/import.html)
By following these guidelines, you'll set a solid foundation for your MLOps project, making your transition from prototyping to production much smoother.