From 163b8a7c0cb386237d56a098dc8b3ef6642c0f5b Mon Sep 17 00:00:00 2001 From: "google-labs-jules[bot]" <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Thu, 7 Aug 2025 19:41:15 +0000 Subject: [PATCH 1/2] Improve style conventions in Chapter 1.0 This commit improves the style conventions in Chapter 1.0 (from 1.0 to 1.6) of the documentation. - Ensures all the emoji in the title section are at the beginning of the title - Ensures the conventions between files are the same - Use markdownlint rules on the files - Review index.md of chapter 1.0 after that --- docs/1. Initializing/1.0. System.md | 135 +++++++--- docs/1. Initializing/1.1. Python.md | 137 +++++++--- docs/1. Initializing/1.2. uv.md | 133 +++++++--- docs/1. Initializing/1.3. uv (project).md | 138 +++++++--- docs/1. Initializing/1.4. git.md | 148 +++++++---- docs/1. Initializing/1.5. GitHub.md | 167 ++++++++---- docs/1. Initializing/1.6. VS Code.md | 240 +++++++++++++----- docs/1. Initializing/index.md | 45 +++- docs/2. Prototyping/2.2. Configs.md | 1 + docs/2. Prototyping/2.4. Analysis.md | 1 + docs/3. Productionizing/3.1. Modules.md | 4 +- docs/3. Productionizing/3.3. Entrypoints.md | 2 +- .../3. Productionizing/3.4. Configurations.md | 4 +- .../3. Productionizing/3.5. Documentations.md | 18 +- .../3.6. VS Code Workspace.md | 56 ++-- docs/4. Validating/4.5. Formatting.md | 28 +- docs/5. Refining/5.1. Task Automation.md | 5 +- docs/5. Refining/5.3. CI-CD Workflows.md | 24 +- docs/5. Refining/5.5. AI-ML Experiments.md | 7 +- docs/5. Refining/archives/5.1. PyInvoke.md | 1 + docs/6. Sharing/6.0. Repository.md | 18 +- docs/6. Sharing/6.1. License.md | 2 +- docs/6. Sharing/6.6. Contributions.md | 1 - docs/7. Observability/0. Reproducibility.md | 8 +- docs/7. Observability/1. Monitoring.md | 1 + docs/7. Observability/5. Explainability.md | 4 +- docs/index.md | 2 +- 27 files changed, 928 insertions(+), 402 deletions(-) diff --git a/docs/1. Initializing/1.0. System.md b/docs/1. Initializing/1.0. System.md index acee7a65..6ab5a7e7 100644 --- a/docs/1. Initializing/1.0. System.md +++ b/docs/1. Initializing/1.0. System.md @@ -1,72 +1,131 @@ --- -description: This section ensures your system is adequately prepared, outlining the essential prerequisites for installing and effectively running the necessary development tools. We'll discuss which operating systems are compatible, explore the recommended hardware specifications, and guide you through installing the essential software packages. +description: > + This section ensures your system is adequately prepared, outlining the + essential prerequisites for installing and effectively running the necessary + development tools. We'll discuss which operating systems are compatible, + explore the recommended hardware specifications, and guide you through + installing the essential software packages. --- # 1.0. System -Before we dive into the exciting world of machine learning operations, it's crucial to ensure that your system is properly set up. A well-configured environment is foundational for a smooth learning experience, enabling you to focus on the concepts rather than troubleshooting technical issues. This section will guide you through the essential prerequisites, from operating system compatibility to hardware specifications and software installations. +Before we dive into the exciting world of machine learning operations, it's +crucial to ensure that your system is properly set up. A well-configured +environment is foundational for a smooth learning experience, enabling you to +focus on the concepts rather than troubleshooting technical issues. This section +will guide you through the essential prerequisites, from operating system +compatibility to hardware specifications and software installations. ## πŸ’» What system is recommended for this course? -This course is designed to be compatible with a variety of operating systems to ensure broad accessibility. Here are the recommended setups: +This course is designed to be compatible with a variety of operating systems to +ensure broad accessibility. Here are the recommended setups: - **Operating Systems:** - - **Linux** - - **macOS** - - **Windows** with Subsystem for Linux (WSL) 2 - - **Chromebook** with a model that supports Linux applications. + - **Linux** + - **macOS** + - **Windows** with Subsystem for Linux (WSL) 2 + - **Chromebook** with a model that supports Linux applications. ## πŸ§ͺ Can you use [JupyterLab](https://jupyterlab.readthedocs.io/en/latest/) or [Google Colab](https://colab.google/) for this course? -While notebook environments like JupyterLab and Google Colab are excellent for experimentation and the initial stages of a project (as covered in the Prototyping chapter), this course is optimized for [Visual Studio Code (VS Code)](https://code.visualstudio.com/). VS Code provides a more comprehensive development environment that is better suited for the later stages of the MLOps lifecycle, which involve productionizing, validating, and refining code. - -Here's a quick comparison of why VS Code is recommended for the full scope of this course: - -| Feature | VS Code | JupyterLab / Google Colab | -| ------------------------ | ------------------------------------- | ------------------------------------ | -| **Integrated Terminal** | βœ… Yes | ❌ Limited or no direct access | -| **File System Navigation** | βœ… Yes | ❌ Limited and less intuitive | -| **Source Control (Git)** | βœ… Excellent integration | ❌ Basic or external integration | -| **Advanced Debugging** | βœ… Yes | ❌ Limited debugging capabilities | -| **Code Refactoring** | βœ… Extensive support | ❌ Basic or no support | +While notebook environments like JupyterLab and Google Colab are excellent for +experimentation and the initial stages of a project (as covered in the +Prototyping chapter), this course is optimized for +[Visual Studio Code (VS Code)](https://code.visualstudio.com/). VS Code provides +a more comprehensive development environment that is better suited for the later +stages of the MLOps lifecycle, which involve productionizing, validating, and +refining code. + +Here's a quick comparison of why VS Code is recommended for the full scope of +this course: + +| Feature | VS Code | JupyterLab / Google Colab | +| ------------------------ | -------------------------------------- | ------------------------------------ | +| **Integrated Terminal** | βœ… Yes | ❌ Limited or no direct access | +| **File System Navigation** | βœ… Yes | ❌ Limited and less intuitive | +| **Source Control (Git)** | βœ… Excellent integration | ❌ Basic or external integration | +| **Advanced Debugging** | βœ… Yes | ❌ Limited debugging capabilities | +| **Code Refactoring** | βœ… Extensive support | ❌ Basic or no support | | **Full Project Lifecycle** | βœ… Ideal for development to production | βœ… Best for prototyping & analysis | ## πŸ“¦ Are additional software installations required? -Yes, a few key software installations are necessary to follow along with the course. These tools form the backbone of a modern development workflow: +Yes, a few key software installations are necessary to follow along with the +course. These tools form the backbone of a modern development workflow: - **Python:** The primary programming language for all coding activities. -- **uv:** An extremely fast Python package installer and resolver, used for managing project dependencies efficiently. -- **Git:** The standard for version control, essential for tracking changes and collaborating on code. -- **VS Code:** The recommended Integrated Development Environment (IDE) for its powerful features and extensions. +- **uv:** An extremely fast Python package installer and resolver, used for + managing project dependencies efficiently. +- **Git:** The standard for version control, essential for tracking changes and + collaborating on code. +- **VS Code:** The recommended Integrated Development Environment (IDE) for its + powerful features and extensions. -Each of these tools is covered in detail in the upcoming sections of this chapter, with step-by-step installation guides. +Each of these tools is covered in detail in the upcoming sections of this +chapter, with step-by-step installation guides. ## βš™οΈ What are the specific hardware requirements for MLOps projects? -The hardware required for MLOps projects can vary significantly based on the project's complexity and the type of data involved. Here’s a general guide: +The hardware required for MLOps projects can vary significantly based on the +project's complexity and the type of data involved. Here’s a general guide: -- **Basic Projects (e.g., Tabular Data):** For models using libraries like Scikit-learn or XGBoost, a standard local machine is usually sufficient. A GPU is generally not required but can be beneficial for certain models. -- **Intermediate Projects (e.g., Multimedia Data):** When working with images or video, using deep learning frameworks like TensorFlow or PyTorch, access to at least one GPU becomes highly beneficial for faster model training. -- **Advanced Projects (e.g., Large-Scale Models):** For projects involving large models like transformers or extensive parallel processing, you may need multiple GPUs, potentially distributed across several machines. +- **Basic Projects (e.g., Tabular Data):** For models using libraries like + Scikit-learn or XGBoost, a standard local machine is usually sufficient. A GPU + is generally not required but can be beneficial for certain models. +- **Intermediate Projects (e.g., Multimedia Data):** When working with images or + video, using deep learning frameworks like TensorFlow or PyTorch, access to at + least one GPU becomes highly beneficial for faster model training. +- **Advanced Projects (e.g., Large-Scale Models):** For projects involving large + models like transformers or extensive parallel processing, you may need + multiple GPUs, potentially distributed across several machines. -A practical approach is to start with a local setup for initial development and then scale to cloud-based resources as needed for more intensive training and deployment. Cloud platforms also enable running multiple experiments simultaneously, which can expedite the development process. +A practical approach is to start with a local setup for initial development and +then scale to cloud-based resources as needed for more intensive training and +deployment. Cloud platforms also enable running multiple experiments +simultaneously, which can expedite the development process. ## πŸ€” Why System Setup Matters for MLOps? -A well-configured system is more than just a matter of convenience; it's a cornerstone of effective MLOps. Here’s why: - -- **Reproducibility:** A consistent and well-documented setup ensures that your experiments are reproducible. This means you (or your teammates) can run the same code and get the same results, which is crucial for validating models and tracking progress. -- **Scalability:** As your projects grow in complexity, your system needs to be able to scale. A proper setup allows you to transition smoothly from local development to more powerful cloud-based resources without major overhauls. -- **Collaboration:** When working in a team, a standardized development environment ensures that everyone is on the same page. This minimizes the "it works on my machine" problem and streamlines the collaboration process. -- **Efficiency:** A properly configured system with the right tools can significantly boost your productivity. From faster package management to integrated debugging, a good setup lets you focus on building and deploying models, not fighting with your tools. +A well-configured system is more than just a matter of convenience; it's a +cornerstone of effective MLOps. Here’s why: + +- **Reproducibility:** A consistent and well-documented setup ensures that your + experiments are reproducible. This means you (or your teammates) can run the + same code and get the same results, which is crucial for validating models and + tracking progress. +- **Scalability:** As your projects grow in complexity, your system needs to be + able to scale. A proper setup allows you to transition smoothly from local + development to more powerful cloud-based resources without major overhauls. +- **Collaboration:** When working in a team, a standardized development + environment ensures that everyone is on the same page. This minimizes the "it + works on my machine" problem and streamlines the collaboration process. +- **Efficiency:** A properly configured system with the right tools can + significantly boost your productivity. From faster package management to + integrated debugging, a good setup lets you focus on building and deploying + models, not fighting with your tools. ## ☁️ Is it possible to use cloud-based systems? -This course supports both local and [cloud-based development environments](../6. Sharing/6.5. Workstations.md), including options like [GitHub Codespaces](https://github.com/features/codespaces) and [Cloud Workstation](https://cloud.google.com/workstations). Cloud platforms offer considerable benefits, such as standardized development environments for easier team collaboration and enhanced security measures for your data. Nonetheless, it's crucial to understand any specific setup requirements and to manage resources effectively, especially when navigating the limitations of free tiers or usage quotas on these services. +This course supports both local and +[cloud-based development environments](../6. Sharing/6.5. Workstations.md), +including options like +[GitHub Codespaces](https://github.com/features/codespaces) and +[Cloud Workstation](https://cloud.google.com/workstations). Cloud platforms +offer considerable benefits, such as standardized development environments for +easier team collaboration and enhanced security measures for your data. +Nonetheless, it's crucial to understand any specific setup requirements and to +manage resources effectively, especially when navigating the limitations of free +tiers or usage quotas on these services. ## πŸ“š System additional resources -- **[GitHub Codespaces](https://github.com/features/codespaces):** A cloud-based development environment that allows you to code directly from your browser, with pre-configured setups for a seamless experience. -- **[Google Cloud Workstations](https://cloud.google.com/workstations):** A fully managed development environment on Google Cloud, offering a secure and scalable solution for remote development. -- **[MLOps Landscape in 2024: Top Tools and Platforms](https://neptune.ai/blog/mlops-tools-platforms-landscape):** A comprehensive overview of the current MLOps landscape, covering a wide range of tools and platforms that are shaping the industry. +- **[GitHub Codespaces](https://github.com/features/codespaces):** A cloud-based + development environment that allows you to code directly from your browser, + with pre-configured setups for a seamless experience. +- **[Google Cloud Workstations](https://cloud.google.com/workstations):** A + fully managed development environment on Google Cloud, offering a secure and + scalable solution for remote development. +- **[MLOps Landscape in 2024: Top Tools and Platforms](https://neptune.ai/blog/mlops-tools-platforms-landscape):** + A comprehensive overview of the current MLOps landscape, covering a wide range + of tools and platforms that are shaping the industry. diff --git a/docs/1. Initializing/1.1. Python.md b/docs/1. Initializing/1.1. Python.md index 7e98de46..070e44fc 100644 --- a/docs/1. Initializing/1.1. Python.md +++ b/docs/1. Initializing/1.1. Python.md @@ -1,67 +1,123 @@ --- -description: Understand why Python is the preferred language for AI/ML projects and learn how to choose the right Python version and install it for your development environment. +description: > + Understand why Python is the preferred language for AI/ML projects and learn + how to choose the right Python version and install it for your development + environment. --- # 🐍 1.1. Python ## ❓ What is Python? -[Python](https://www.python.org/) is a dynamic, high-level programming language known for its ease of learning and readability, making it a favorite among developers across various disciplines, including web development, automation, data science, and machine learning. It stands out for its simplicity and the vast ecosystem of third-party packages, allowing developers to build applications quickly and efficiently. Given its popularity, reflected in rankings such as the [Tiobe Index](https://www.tiobe.com/tiobe-index/) and [IEEE Spectrum Annual Ranking](https://spectrum.ieee.org/the-top-programming-languages-2023), Python is a staple in the programming world. For beginners and seasoned developers alike, Python offers a balance of readability and power, supported by a rich standard library and an extensive array of packages for diverse application needs. +[Python](https://www.python.org/) is a dynamic, high-level programming language +known for its ease of learning and readability, making it a favorite among +developers across various disciplines, including web development, automation, +data science, and machine learning. It stands out for its simplicity and the +vast ecosystem of third-party packages, allowing developers to build +applications quickly and efficiently. Given its popularity, reflected in +rankings such as the [Tiobe Index](https://www.tiobe.com/tiobe-index/) and +[IEEE Spectrum Annual Ranking](https://spectrum.ieee.org/the-top-programming-languages-2023), +Python is a staple in the programming world. For beginners and seasoned +developers alike, Python offers a balance of readability and power, supported by +a rich standard library and an extensive array of packages for diverse +application needs.
- Python -
Python(source)
+ A person is explaining to another person that they have learned Python and that it is a great language. The other person is impressed and asks how long it took to learn. The first person replies that it only took a few hours. +
+ Python Comic (source) +
## πŸ€– Why is Python preferred for AI/ML projects? -Python's dominance in AI and machine learning is no accident. It stems from a powerful combination of a simple, readable syntax and an unparalleled ecosystem of specialized libraries. This allows developers to move quickly from idea to implementation, a crucial advantage in the fast-paced world of AI/ML. +Python's dominance in AI and machine learning is no accident. It stems from a +powerful combination of a simple, readable syntax and an unparalleled ecosystem +of specialized libraries. This allows developers to move quickly from idea to +implementation, a crucial advantage in the fast-paced world of AI/ML. Here are some of the key reasons for Python's popularity: -- **Vast Ecosystem of Libraries**: Python boasts a rich collection of open-source libraries that cover the entire AI/ML workflow, from data manipulation to model deployment. -- **Ease of Use and Readability**: Python's clean syntax makes it easy to learn and read, which helps in collaborative projects and reduces the cost of maintenance. -- **Flexibility**: Python is a general-purpose language, which means it can be used for a wide range of tasks, including web development, automation, and data analysis, making it a versatile tool for any MLOps professional. -- **Strong Community Support**: Python has a large and active community, which means you can easily find help, tutorials, and documentation. - -Here's a quick look at some of the most popular AI/ML libraries in the Python ecosystem: - -| Library | Category | Description | -| --------------------------------------------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -| [NumPy](https://numpy.org/) | Numerical Computing | The fundamental package for scientific computing with Python, providing support for large, multi-dimensional arrays and matrices. | -| [Pandas](https://pandas.pydata.org/) | Data Manipulation | A powerful library for data analysis and manipulation, offering data structures like DataFrames that are essential for cleaning and preparing data. | -| [Scikit-learn](https://scikit-learn.org/) | Machine Learning | A comprehensive library for classical machine learning algorithms, including classification, regression, clustering, and dimensionality reduction. | -| [TensorFlow](https://www.tensorflow.org/) | Deep Learning | An end-to-end open-source platform for machine learning, developed by Google. | -| [PyTorch](https://pytorch.org/) | Deep Learning | An open-source machine learning library developed by Facebook's AI Research lab, known for its flexibility and ease of use. | -| [Matplotlib](https://matplotlib.org/) & [Seaborn](https://seaborn.pydata.org/) | Data Visualization | Libraries for creating static, animated, and interactive visualizations in Python. | - -Its user-friendly syntax supports rapid prototyping and iterative development, essential in the AI/ML workflow. Additionally, Python's interoperability with high-performance languages like C and C++ enables developers to optimize computational efficiency without sacrificing development speed or ease of use, making it the go-to language for AI/ML endeavors. +- **Vast Ecosystem of Libraries**: Python boasts a rich collection of + open-source libraries that cover the entire AI/ML workflow, from data + manipulation to model deployment. +- **Ease of Use and Readability**: Python's clean syntax makes it easy to learn + and read, which helps in collaborative projects and reduces the cost of + maintenance. +- **Flexibility**: Python is a general-purpose language, which means it can be + used for a wide range of tasks, including web development, automation, and data + analysis, making it a versatile tool for any MLOps professional. +- **Strong Community Support**: Python has a large and active community, which + means you can easily find help, tutorials, and documentation. + +Here's a quick look at some of the most popular AI/ML libraries in the Python +ecosystem: + +| Library | Category | Description | +| ---------------------------------------------------------------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| [NumPy](https://numpy.org/) | Numerical Computing | The fundamental package for scientific computing with Python, providing support for large, multi-dimensional arrays and matrices. | +| [Pandas](https://pandas.pydata.org/) | Data Manipulation | A powerful library for data analysis and manipulation, offering data structures like DataFrames that are essential for cleaning and preparing data. | +| [Scikit-learn](https://scikit-learn.org/) | Machine Learning | A comprehensive library for classical machine learning algorithms, including classification, regression, clustering, and dimensionality reduction. | +| [TensorFlow](https://www.tensorflow.org/) | Deep Learning | An end-to-end open-source platform for machine learning, developed by Google. | +| [PyTorch](https://pytorch.org/) | Deep Learning | An open-source machine learning library developed by Facebook's AI Research lab, known for its flexibility and ease of use. | +| [Matplotlib](https://matplotlib.org/) & [Seaborn](https://seaborn.pydata.org/) | Data Visualization | Libraries for creating static, animated, and interactive visualizations in Python. | + +Its user-friendly syntax supports rapid prototyping and iterative development, +essential in the AI/ML workflow. Additionally, Python's interoperability with +high-performance languages like C and C++ enables developers to optimize +computational efficiency without sacrificing development speed or ease of use, +making it the go-to language for AI/ML endeavors. ## πŸ› οΈ Is Python a good language for MLOps? -Python excels in the MLOps domain, offering a blend of simplicity for algorithm development and the robustness required for operational workflows. The key to maximizing Python's benefits in MLOps lies in adopting best practices for code quality and maintainability. This course covers strategies for effective Python code structuring and validation, ensuring that Python's flexibility and extensive toolkit can be leveraged effectively within MLOps pipelines. +Python excels in the MLOps domain, offering a blend of simplicity for algorithm +development and the robustness required for operational workflows. The key to +maximizing Python's benefits in MLOps lies in adopting best practices for code +quality and maintainability. This course covers strategies for effective Python +code structuring and validation, ensuring that Python's flexibility and +extensive toolkit can be leveraged effectively within MLOps pipelines. ## πŸ”„ Can you use other languages for AI/ML? -While Python dominates the AI/ML landscape, other languages like [R](https://www.r-project.org/) or [Julia](https://julialang.org/) also provide capabilities for statistical analysis and machine learning. Each of these languages brings unique strengths, whether in performance, syntax, or domain specificity. Transitioning to or incorporating these languages in AI/ML projects is possible but requires careful consideration of their ecosystems and how they fit into the broader project goals. +While Python dominates the AI/ML landscape, other languages like +[R](https://www.r-project.org/) or [Julia](https://julialang.org/) also provide +capabilities for statistical analysis and machine learning. Each of these +languages brings unique strengths, whether in performance, syntax, or domain +specificity. Transitioning to or incorporating these languages in AI/ML projects +is possible but requires careful consideration of their ecosystems and how they +fit into the broader project goals. ## πŸ“¦ Which Python version should you use? -Choosing the right Python version is a critical step in setting up your project. While it's tempting to always use the latest version, compatibility with key libraries and your production environment is more important. +Choosing the right Python version is a critical step in setting up your project. +While it's tempting to always use the latest version, compatibility with key +libraries and your production environment is more important. -As a general rule, it's best to **use the latest stable version of Python that is supported by your project's key dependencies**. +As a general rule, it's best to +**use the latest stable version of Python that is supported by your project's +key dependencies**. Here's a simple process for choosing a version: -1. **Check your project's dependencies**: Look at the `pyproject.toml` or `requirements.txt` file to see if a specific Python version is required. -2. **Check library compatibility**: If you're starting a new project, check the documentation for your key libraries (like TensorFlow or PyTorch) to see which Python versions they support. -3. **Consult the Python release schedule**: Check the [official Python developer's guide](https://devguide.python.org/versions/) to see the support status of different versions. Avoid using versions that are no longer supported. +1. **Check your project's dependencies**: Look at the `pyproject.toml` or + `requirements.txt` file to see if a specific Python version is required. +2. **Check library compatibility**: If you're starting a new project, check the + documentation for your key libraries (like TensorFlow or PyTorch) to see + which Python versions they support. +3. **Consult the Python release schedule**: Check the + [official Python developer's guide](https://devguide.python.org/versions/) to + see the support status of different versions. Avoid using versions that are + no longer supported. -For this course, we recommend using **Python 3.11**, as it is widely supported by the AI/ML ecosystem and offers a good balance of new features and stability. +For this course, we recommend using **Python 3.11**, as it is widely supported +by the AI/ML ecosystem and offers a good balance of new features and stability. ## πŸš€ How to install Python for this course? -For this course, we recommend using [`uv`](https://docs.astral.sh/uv/) to install and manage Python versions. `uv` is a modern, fast, and unified toolchain that can handle both Python installation and package management. It's a great replacement for tools like `pyenv` and `pip`. +For this course, we recommend using [`uv`](https://docs.astral.sh/uv/) to +install and manage Python versions. `uv` is a modern, fast, and unified +toolchain that can handle both Python installation and package management. It's +a great replacement for tools like `pyenv` and `pip`. Here's how you can install a specific Python version using `uv`: @@ -70,14 +126,23 @@ Here's how you can install a specific Python version using `uv`: uv python install 3.11 ``` -This command will download and install Python 3.11, making it available for you to use in your projects. You can then create a virtual environment with this version using `uv venv --python 3.11`. This approach provides the flexibility to switch between Python versions on a per-project basis, which is essential for modern MLOps workflows. +This command will download and install Python 3.11, making it available for you +to use in your projects. You can then create a virtual environment with this +version using `uv venv --python 3.11`. This approach provides the flexibility to +switch between Python versions on a per-project basis, which is essential for +modern MLOps workflows. ## πŸ”‘ Key Takeaways -- **Python is the leading language for AI/ML** due to its simplicity, flexibility, and vast ecosystem of libraries. -- **Choosing the right Python version is crucial**. For this course, we recommend **Python 3.11**. -- **Use `uv` to install and manage Python versions**. It's a modern, fast, and unified tool that simplifies the development workflow. -- **Mastering Python is a key skill for any MLOps professional**. A solid understanding of the language and its ecosystem is essential for building robust and maintainable AI/ML systems. +- **Python is the leading language for AI/ML** due to its simplicity, + flexibility, and vast ecosystem of libraries. +- **Choosing the right Python version is crucial**. For this course, we recommend + **Python 3.11**. +- **Use `uv` to install and manage Python versions**. It's a modern, fast, and + unified tool that simplifies the development workflow. +- **Mastering Python is a key skill for any MLOps professional**. A solid + understanding of the language and its ecosystem is essential for building + robust and maintainable AI/ML systems. ## πŸ“š Python additional resources diff --git a/docs/1. Initializing/1.2. uv.md b/docs/1. Initializing/1.2. uv.md index 6e0204b1..34a87b4f 100644 --- a/docs/1. Initializing/1.2. uv.md +++ b/docs/1. Initializing/1.2. uv.md @@ -1,43 +1,74 @@ --- -description: Discover uv, a fast and versatile Python package manager and project manager designed to streamline your development workflow. Learn how to install it and use it as a drop-in replacement for pip, venv, pipx, and pyenv. +description: > + Discover uv, a fast and versatile Python package manager and project manager + designed to streamline your development workflow. Learn how to install it and + use it as a drop-in replacement for pip, venv, pipx, and pyenv. --- -# 1.2. uv ⚑️ +# ⚑️ 1.2. uv ## πŸ€” What is uv? -[uv](https://docs.astral.sh/uv/) is an extremely fast Python package installer and resolver, written in Rust. It's designed as a drop-in replacement for `pip`, `pip-tools`, `pipx`, `venv`, and `virtualenv`, with `pyenv` support coming soon. Created by [Astral](https://astral.sh/), the same team behind the high-performance linter [Ruff](https://docs.astral.sh/ruff/), `uv` aims to significantly speed up and simplify Python project management. It offers a unified toolchain that handles virtual environments, dependency resolution, package installation, and more, all while providing exceptional performance. +[uv](https://docs.astral.sh/uv/) is an extremely fast Python package installer +and resolver, written in Rust. It's designed as a drop-in replacement for `pip`, +`pip-tools`, `pipx`, `venv`, and `virtualenv`, with `pyenv` support coming soon. +Created by [Astral](https://astral.sh/), the same team behind the +high-performance linter [Ruff](https://docs.astral.sh/ruff/), `uv` aims to +significantly speed up and simplify Python project management. It offers a +unified toolchain that handles virtual environments, dependency resolution, +package installation, and more, all while providing exceptional performance. ## πŸš€ Why should you use uv? -`uv` offers several compelling advantages for Python developers, especially in the context of MLOps. - -| Feature | `uv` | `pip`, `venv`, `pipx`, etc. | -| ----------------------- | ---------------------------------- | --------------------------------------- | -| **Speed** | Blazing fast (written in Rust) | Slower (written in Python) | -| **Tooling** | Unified (all-in-one tool) | Fragmented (multiple tools) | -| **Dependency Resolution** | Advanced, high-performance resolver | Basic, can be slow for complex cases | -| **Caching** | Aggressive and efficient | Less effective | -| **Disk Space** | Efficient, uses less space | Can be bloated | - -- **Performance**: `uv` is incredibly fast, often outperforming `pip`, `venv`, and `pyenv` by a significant margin. This speed translates to faster project setup, quicker dependency resolution, and reduced wait times during development and deployment. -- **Unified Toolchain**: `uv` replaces multiple tools, simplifying your development workflow. It can manage virtual environments, install packages, and resolve dependencies, all within a single command-line interface. -- **Drop-in Replacement**: `uv` is designed to be a drop-in replacement for common Python tools. This means you can often substitute `uv` commands for `pip`, `venv`, or `pipx` commands without altering your existing workflows significantly. -- **Active Development**: Backed by Astral, `uv` is under active development with a focus on performance, reliability, and ease of use. -- **Cross-Platform Compatibility**: `uv` works seamlessly across Linux, macOS, and Windows, ensuring a consistent experience regardless of your operating system. -- **Caching**: `uv` implements aggressive caching mechanisms to avoid redundant work, further speeding up operations like package installation and environment setup. +`uv` offers several compelling advantages for Python developers, especially in +the context of MLOps. + +| Feature | `uv` | `pip`, `venv`, `pipx`, etc. | +| ----------------------- | ----------------------------------- | ------------------------------------ | +| **Speed** | Blazing fast (written in Rust) | Slower (written in Python) | +| **Tooling** | Unified (all-in-one tool) | Fragmented (multiple tools) | +| **Dependency Resolution** | Advanced, high-performance resolver | Basic, can be slow for complex cases | +| **Caching** | Aggressive and efficient | Less effective | +| **Disk Space** | Efficient, uses less space | Can be bloated | + +- **Performance**: `uv` is incredibly fast, often outperforming `pip`, `venv`, + and `pyenv` by a significant margin. This speed translates to faster project + setup, quicker dependency resolution, and reduced wait times during + development and deployment. +- **Unified Toolchain**: `uv` replaces multiple tools, simplifying your + development workflow. It can manage virtual environments, install packages, and + resolve dependencies, all within a single command-line interface. +- **Drop-in Replacement**: `uv` is designed to be a drop-in replacement for + common Python tools. This means you can often substitute `uv` commands for + `pip`, `venv`, or `pipx` commands without altering your existing workflows + significantly. +- **Active Development**: Backed by Astral, `uv` is under active development + with a focus on performance, reliability, and ease of use. +- **Cross-Platform Compatibility**: `uv` works seamlessly across Linux, macOS, + and Windows, ensuring a consistent experience regardless of your operating + system. +- **Caching**: `uv` implements aggressive caching mechanisms to avoid redundant + work, further speeding up operations like package installation and environment + setup. ### πŸ’‘ MLOps Benefits In MLOps, reproducibility and speed are critical. `uv` helps on both fronts: -- **Faster CI/CD Pipelines**: By speeding up dependency installation, `uv` can significantly reduce the time your CI/CD pipelines take to run. -- **Consistent Environments**: `uv`'s fast and reliable dependency resolution ensures that you can create consistent and reproducible environments for your machine learning models. -- **Simplified Dependency Management**: No more juggling multiple tools. `uv` simplifies your `pyproject.toml` or `requirements.txt` management. +- **Faster CI/CD Pipelines**: By speeding up dependency installation, `uv` can + significantly reduce the time your CI/CD pipelines take to run. +- **Consistent Environments**: `uv`'s fast and reliable dependency resolution + ensures that you can create consistent and reproducible environments for your + machine learning models. +- **Simplified Dependency Management**: No more juggling multiple tools. `uv` + simplifies your `pyproject.toml` or `requirements.txt` management. ## πŸ“¦ How to install uv? -[Installing `uv` is straightforward](https://docs.astral.sh/uv/getting-started/installation/). The recommended method is to use the official installation script, which automatically detects your operating system and installs the appropriate version: +[Installing `uv` is straightforward](https://docs.astral.sh/uv/getting-started/installation/). +The recommended method is to use the official installation script, which +automatically detects your operating system and installs the appropriate +version: ```bash curl -LsSf https://astral.sh/uv/install.sh | sh @@ -59,25 +90,35 @@ uv --version ## πŸ₯§ How to use uv as a drop-in replacement for pip? -`uv` can be used as a direct replacement for many common `pip` commands. Here's how: +`uv` can be used as a direct replacement for many common `pip` commands. Here's +how: - **Installing packages**: + ```bash uv pip install requests numpy pandas ``` + - **Uninstalling packages**: + ```bash uv pip uninstall requests ``` + - **Listing installed packages**: + ```bash uv pip freeze ``` + - **Updating packages**: + ```bash uv pip install --upgrade requests ``` + - **Installing packages from a `requirements.txt` file**: + ```bash uv pip install -r requirements.txt ``` @@ -87,63 +128,91 @@ uv --version `uv` can also replace `venv` for creating and managing virtual environments: - **Creating a virtual environment**: + ```bash uv venv ``` - This command creates a new virtual environment in the `.venv` directory by default. You can customize the location or specify a Python interpreter: + + This command creates a new virtual environment in the `.venv` directory by + default. You can customize the location or specify a Python interpreter: + ```bash uv venv --python 3.11 ``` + - **Activating the virtual environment**: The activation process is the same as with `venv`. On macOS and Linux: + ```bash source .venv/bin/activate ``` + On Windows: + ```powershell .venv\Scripts\Activate ``` + - **Listing available Python interpreters**: + ```bash uv python list ``` ## πŸ› οΈ How to use uv as a drop-in replacement for pipx? -`uv` can also replace `pipx` for installing and managing globally available Python tools: +`uv` can also replace `pipx` for installing and managing globally available +Python tools: - **Installing a tool globally**: + ```bash uv tool install ruff ``` + - **Listing globally installed tools**: + ```bash uv tool list ``` + - **Running a tool without installing it**: - This is useful for one-off commands without cluttering your global environment. + This is useful for one-off commands without cluttering your global + environment. + ```bash uv tool run ruff --version ``` ## 🐍 How to install a Python version with uv? -`uv` can also be used to install specific Python versions, similar to `pyenv`. This is particularly useful when you need to test your code against different Python environments or when a project requires a specific Python version that is not your system's default. +`uv` can also be used to install specific Python versions, similar to `pyenv`. +This is particularly useful when you need to test your code against different +Python environments or when a project requires a specific Python version that is +not your system's default. - **Installing a specific Python version**: + ```bash uv python install 3.13 ``` - This command downloads and installs Python 3.13. You can then use this version to create virtual environments or run scripts. + + This command downloads and installs Python 3.13. You can then use this version + to create virtual environments or run scripts. - **Listing available Python versions**: + ```bash uv python list ``` + - **Listing installed Python versions**: + ```bash uv python list --only-installed ``` + - **Removing a specific Python version**: + ```bash uv python remove 3.13 ``` @@ -152,12 +221,14 @@ uv --version - **`uv` is a fast, all-in-one tool** for Python package and project management. - It can replace `pip`, `venv`, `pipx`, and more, simplifying your workflow. -- In MLOps, `uv` shines by **speeding up CI/CD pipelines** and ensuring **reproducible environments**. +- In MLOps, `uv` shines by **speeding up CI/CD pipelines** and ensuring + **reproducible environments**. - Adopting `uv` can lead to a more efficient and streamlined development process. ## πŸ“š uv additional resources -- **[uv Documentation](https://docs.astral.sh/uv/)**: The official documentation provides comprehensive information on all `uv` features and commands. +- **[uv Documentation](https://docs.astral.sh/uv/)**: The official documentation + provides comprehensive information on all `uv` features and commands. - **[Poetry Was Good, Uv Is Better: An MLOps Migration Story](https://medium.com/@fmind/poetry-was-good-uv-is-better-an-mlops-migration-story-f52bf0c6c703)** - [uv Installation](https://docs.astral.sh/uv/getting-started/installation/) - [uv Features](https://docs.astral.sh/uv/getting-started/features/) diff --git a/docs/1. Initializing/1.3. uv (project).md b/docs/1. Initializing/1.3. uv (project).md index 150882ac..81f984c9 100644 --- a/docs/1. Initializing/1.3. uv (project).md +++ b/docs/1. Initializing/1.3. uv (project).md @@ -1,14 +1,24 @@ --- -description: Discover how to use uv to manage project dependencies and build Python packages, streamlining the process of creating production-ready artifacts. +description: > + Discover how to use uv to manage project dependencies and build Python + packages, streamlining the process of creating production-ready artifacts. --- # πŸš€ 1.3. uv (project) ## πŸ“¦ What is a package? -A [Python package](https://packaging.python.org/en/latest/) is a collection of Python modules that can be installed and used in your projects. Packages allow you to leverage external libraries and frameworks, extending Python's built-in capabilities. +A [Python package](https://packaging.python.org/en/latest/) is a collection of +Python modules that can be installed and used in your projects. Packages allow +you to leverage external libraries and frameworks, extending Python's built-in +capabilities. -[uv](https://docs.astral.sh/uv/) simplifies the management of these packages by treating them as dependencies. With uv, you can declare the packages your project needs in a `pyproject.toml` file. uv then ensures that the correct versions of these dependencies are installed, creating a stable and conflict-free development environment. Here’s a basic example of how to specify dependencies with uv: +[uv](https://docs.astral.sh/uv/) simplifies the management of these packages by +treating them as dependencies. With uv, you can declare the packages your +project needs in a `pyproject.toml` file. uv then ensures that the correct +versions of these dependencies are installed, creating a stable and +conflict-free development environment. Here’s a basic example of how to specify +dependencies with uv: ```toml # https://docs.astral.sh/uv/reference/pyproject-toml/ @@ -27,38 +37,64 @@ dev = [ ] ``` -You will learn more on how to construct and publish Python Package in the [Package section of this course](../3. Productionizing/3.0. Package.md). +You will learn more on how to construct and publish Python Package in the +[Package section of this course](../3. Productionizing/3.0. Package.md). ## πŸ€” Why do you need a package manager? -In the Python ecosystem, software is typically distributed and installed as packages. These packages, often in Wheel or zip format, contain source code and essential metadata. Manually managing these packages and their dependencies can be a tedious and error-prone process. Package managers like uv automate these tasks, which boosts productivity and ensures that your development and deployment environments are consistent. +In the Python ecosystem, software is typically distributed and installed as +packages. These packages, often in Wheel or zip format, contain source code and +essential metadata. Manually managing these packages and their dependencies can +be a tedious and error-prone process. Package managers like uv automate these +tasks, which boosts productivity and ensures that your development and +deployment environments are consistent.
- Python Environment -
Python Environment(source)
+ A comic showing a person trying to set up a Python environment and getting frustrated with the complexity of the process. +
+ Python Environment Comic (source) +
-By default, uv will download and install Python packages from [PyPI](https://pypi.org/), a repository of software for the Python programming language. If needed, [other Python repositories](https://warehouse.pypa.io/repository-projects/) can be configured to provide extra sources of dependencies. +By default, uv will download and install Python packages from +[PyPI](https://pypi.org/), a repository of software for the Python programming +language. If needed, +[other Python repositories](https://warehouse.pypa.io/repository-projects/) can +be configured to provide extra sources of dependencies. ## ✨ Why should you use uv in your project? Using uv in your project offers several key benefits: -- **Improved Environment Management**: uv simplifies the management of project environments, which helps maintain consistency. -- **Simplified Package Building and Distribution**: It provides a unified workflow for building, distributing, and installing packages, which reduces complexity. -- **Standardized Project Metadata**: uv uses a standard `pyproject.toml` file to define project metadata, such as dependencies, authors, and versioning. This ensures that your project is easy to understand and maintain. +- **Improved Environment Management**: uv simplifies the management of project + environments, which helps maintain consistency. +- **Simplified Package Building and Distribution**: It provides a unified + workflow for building, distributing, and installing packages, which reduces + complexity. +- **Standardized Project Metadata**: uv uses a standard `pyproject.toml` file to + define project metadata, such as dependencies, authors, and versioning. This + ensures that your project is easy to understand and maintain. -Compared to traditional tools like pip and venv, uv provides a more cohesive and user-friendly experience by combining package and environment management into a single, streamlined process. +Compared to traditional tools like pip and venv, uv provides a more cohesive and +user-friendly experience by combining package and environment management into a +single, streamlined process. ## πŸ› οΈ How can you use uv for your MLOps project? -To integrate uv into your MLOps project, you can follow these steps to set up your development environment: +To integrate uv into your MLOps project, you can follow these steps to set up +your development environment: - Begin by creating a new project directory and navigate into it. -- Run `uv init` in your terminal. This command starts an interactive guide to help set initial project parameters, such as package name, version, description, author, and dependencies. This step generates a `pyproject.toml` file, crucial for your project's configuration under uv. -- Run `uv sync` to install the project dependencies and source code. This will let you access your project code through `uv run` and its command-line utilities. +- Run `uv init` in your terminal. This command starts an interactive guide to + help set initial project parameters, such as package name, version, + description, author, and dependencies. This step generates a `pyproject.toml` + file, crucial for your project's configuration under uv. +- Run `uv sync` to install the project dependencies and source code. This will + let you access your project code through `uv run` and its command-line + utilities. -The `pyproject.toml` file is central to defining your project's dependencies and settings. Let's break down a typical example: +The `pyproject.toml` file is central to defining your project's dependencies and +settings. Let's break down a typical example: ```toml # https://docs.astral.sh/uv/reference/settings/ @@ -102,17 +138,26 @@ build-backend = "hatchling.build" Here’s what each section means: -- **`[project]`**: This section contains general metadata about your project, such as its name, version, and description. -- **`dependencies`**: This is a list of the main dependencies that your project needs to run. -- **`[project.urls]`**: This section allows you to include helpful links, such as to the project's homepage or documentation. -- **`[project.scripts]`**: Here, you can define command-line scripts that will be created when your package is installed. -- **`[build-system]`**: This section specifies the build tool that `uv` will use to create your package. - -At the end of the installation process, a `uv.lock` file is generated with all the project dependencies that have been installed. You can remove and regenerate the `uv.lock` file if you wish to update the list of dependencies. +- **`[project]`**: This section contains general metadata about your project, + such as its name, version, and description. +- **`dependencies`**: This is a list of the main dependencies that your project + needs to run. +- **`[project.urls]`**: This section allows you to include helpful links, such + as to the project's homepage or documentation. +- **`[project.scripts]`**: Here, you can define command-line scripts that will + be created when your package is installed. +- **`[build-system]`**: This section specifies the build tool that `uv` will use + to create your package. + +At the end of the installation process, a `uv.lock` file is generated with all +the project dependencies that have been installed. You can remove and regenerate +the `uv.lock` file if you wish to update the list of dependencies. ## βž• How can you install dependencies for your project with uv? -uv distinguishes between main (production) and development dependencies, which allows for a more organized approach to dependency management. To add dependencies, you can use the following commands: +uv distinguishes between main (production) and development dependencies, which +allows for a more organized approach to dependency management. To add +dependencies, you can use the following commands: ```bash # For main dependencies @@ -122,15 +167,23 @@ $ uv add pandas scikit-learn $ uv add --group dev ipykernel ``` -Executing these commands updates the `pyproject.toml` file, accurately managing and versioning your project's dependencies. +Executing these commands updates the `pyproject.toml` file, accurately managing +and versioning your project's dependencies. ## Main vs. Dev Dependencies in uv -In uv, dependencies are categorized into two types: [main dependencies](https://docs.astral.sh/uv/concepts/projects/dependencies/#project-dependencies) and [development (dev) dependencies](https://docs.astral.sh/uv/concepts/projects/dependencies/#development-dependencies). +In uv, dependencies are categorized into two types: +[main dependencies](https://docs.astral.sh/uv/concepts/projects/dependencies/#project-dependencies) +and +[development (dev) dependencies](https://docs.astral.sh/uv/concepts/projects/dependencies/#development-dependencies). -- **Main Dependencies**: These are essential for your project to run in a production environment. For example, libraries like pandas or scikit-learn are typically main dependencies in an MLOps project. +- **Main Dependencies**: These are essential for your project to run in a + production environment. For example, libraries like pandas or scikit-learn are + typically main dependencies in an MLOps project. -- **Development Dependencies**: These are only used for development and testing purposes. Examples include testing frameworks like `pytest` or linters like `ruff`. They are not required for the application to run in production. +- **Development Dependencies**: These are only used for development and testing + purposes. Examples include testing frameworks like `pytest` or linters like + `ruff`. They are not required for the application to run in production. Here’s a simple example in a `pyproject.toml` file: @@ -146,11 +199,17 @@ dev = [ ] ``` -This setup helps keep production environments lean by excluding unnecessary development tools. +This setup helps keep production environments lean by excluding unnecessary +development tools. ## 🏒 Using Custom Repositories -uv supports [custom package repositories](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-sources), including private and organizational ones. This allows you to use proprietary packages alongside those from the public PyPI. You can easily add a custom repository and configure authentication using uv's commands, which ensures secure and flexible dependency management. +uv supports +[custom package repositories](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-sources), +including private and organizational ones. This allows you to use proprietary +packages alongside those from the public PyPI. You can easily add a custom +repository and configure authentication using uv's commands, which ensures +secure and flexible dependency management. ## ⚑ Common `uv` Commands @@ -163,14 +222,23 @@ Here is a quick reference for some of the most common `uv` commands: ## πŸ”’ The `uv.lock` File -The `uv.lock` file is generated after you install your project's dependencies. It records the exact versions of all the packages that were installed, including their dependencies. This is crucial for ensuring that your project is reproducible. When you share your project with others, they can use the `uv.lock` file to create an identical environment, which helps to avoid bugs and inconsistencies. +The `uv.lock` file is generated after you install your project's dependencies. +It records the exact versions of all the packages that were installed, including +their dependencies. This is crucial for ensuring that your project is +reproducible. When you share your project with others, they can use the +`uv.lock` file to create an identical environment, which helps to avoid bugs and +inconsistencies. ## πŸ”‘ Key Takeaways -- **Unified Tool**: `uv` streamlines Python project management by combining the functionalities of `pip`, `venv`, `pipx`, and `pyenv` into a single, fast tool. -- **`pyproject.toml`**: This file is central to `uv` projects, defining metadata, dependencies (main and development), and build configurations. -- **Dependency Management**: `uv` simplifies adding, managing, and synchronizing project dependencies, ensuring consistent environments. -- **Reproducibility**: The `uv.lock` file guarantees reproducible environments by locking exact dependency versions. +- **Unified Tool**: `uv` streamlines Python project management by combining the + functionalities of `pip`, `venv`, `pipx`, and `pyenv` into a single, fast tool. +- **`pyproject.toml`**: This file is central to `uv` projects, defining + metadata, dependencies (main and development), and build configurations. +- **Dependency Management**: `uv` simplifies adding, managing, and synchronizing + project dependencies, ensuring consistent environments. +- **Reproducibility**: The `uv.lock` file guarantees reproducible environments + by locking exact dependency versions. ## πŸ“š Additional Resources diff --git a/docs/1. Initializing/1.4. git.md b/docs/1. Initializing/1.4. git.md index 91aaf2dc..97db29fe 100644 --- a/docs/1. Initializing/1.4. git.md +++ b/docs/1. Initializing/1.4. git.md @@ -1,32 +1,52 @@ --- -description: Learn how to use Git for version control and collaboration in MLOps projects, enabling you to track changes, revert to previous versions, and work with others effectively. +description: > + Learn how to use Git for version control and collaboration in MLOps projects, + enabling you to track changes, revert to previous versions, and work with + others effectively. --- -# 1.4. Git 🌿 +# 🌿 1.4. Git -## What is Git? πŸ€” +## πŸ€” What is Git? -[Git](https://git-scm.com/) is a distributed version control system that is integral for managing both small and large projects effectively. It excels in tracking source code changes during software development, enabling multiple developers to collaborate on the same project without conflicts. Git is highly regarded for its robust data integrity, versatility, and support for complex, nonlinear development workflows. +[Git](https://git-scm.com/) is a distributed version control system that is +integral for managing both small and large projects effectively. It excels in +tracking source code changes during software development, enabling multiple +developers to collaborate on the same project without conflicts. Git is highly +regarded for its robust data integrity, versatility, and support for complex, +nonlinear development workflows.
- Git from XKCD -
Git (source)
+ A comic showing a person using Git and getting frustrated with the complexity of the process. +
+ Git Comic (source) +
-## Why is Git Essential? 🎯 +## 🎯 Why is Git Essential? Git serves several critical purposes in software development: -- **Version Control**: It meticulously tracks and manages changes to your project, offering the ability to revert to previous states, compare changes across timelines, and more. -- **Collaboration**: Git facilitates simultaneous collaboration among multiple contributors on the same project. It supports branching and merging strategies, allowing seamless teamwork without risking data overwrite. -- **Backup and Restore**: With changes stored in a repository, Git acts as a backup mechanism. You can revert your project to a prior state or retrieve lost data as needed. -- **Branching and Merging**: Git enables you to create branches for experimenting or developing new features independently of the main project, which can later be merged back into the mainline without disrupting the ongoing development. - -## Installation & Setup πŸ› οΈ +- **Version Control**: It meticulously tracks and manages changes to your + project, offering the ability to revert to previous states, compare changes + across timelines, and more. +- **Collaboration**: Git facilitates simultaneous collaboration among multiple + contributors on the same project. It supports branching and merging + strategies, allowing seamless teamwork without risking data overwrite. +- **Backup and Restore**: With changes stored in a repository, Git acts as a + backup mechanism. You can revert your project to a prior state or retrieve + lost data as needed. +- **Branching and Merging**: Git enables you to create branches for experimenting + or developing new features independently of the main project, which can later + be merged back into the mainline without disrupting the ongoing development. + +## πŸ› οΈ Installation & Setup ### Installation -To install Git, consult the [Git Installation Guide](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git), which offers comprehensive instructions for a variety of operating systems. +To install Git, consult the +[Git Installation Guide](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git), +which offers comprehensive instructions for a variety of operating systems. ```bash # Installation on macOS with Homebrew @@ -39,7 +59,8 @@ sudo apt install git ### Initial Configuration -Before you start using Git, you need to configure it with your name and email address. This is important because every Git commit uses this information. +Before you start using Git, you need to configure it with your name and email +address. This is important because every Git commit uses this information. ```bash git config --global user.name "Your Name" @@ -47,31 +68,43 @@ git config --global user.email "youremail@example.com" ``` You can verify your configuration with: + ```bash git config --list ``` -## The Git Workflow: A Practical Guide πŸš€ +## πŸš€ The Git Workflow: A Practical Guide -Understanding the basic Git workflow is key to using it effectively. It consists of three main states: +Understanding the basic Git workflow is key to using it effectively. It consists +of three main states: -1. **Working Directory**: This is your local project folder with all its files. -2. **Staging Area (Index)**: This is an intermediate area where you can build up your next commit. It allows you to choose which changes you want to include. -3. **Repository (`.git` directory)**: This is where Git stores the project's history and metadata. A commit takes the files from the staging area and stores that snapshot permanently to your repository. +1. **Working Directory**: This is your local project folder with all its files. +2. **Staging Area (Index)**: This is an intermediate area where you can build up + your next commit. It allows you to choose which changes you want to include. +3. **Repository (`.git` directory)**: This is where Git stores the project's + history and metadata. A commit takes the files from the staging area and + stores that snapshot permanently to your repository. Here’s a step-by-step guide to a common workflow: -1. **Initialize a Repository**: In your project directory, run this command to create a new local Git repository. +1. **Initialize a Repository**: In your project directory, run this command to + create a new local Git repository. + ```bash git init ``` -2. **Check the Status**: Use `git status` to see the state of your working directory and staging area. It shows which files are new, modified, or staged. +2. **Check the Status**: Use `git status` to see the state of your working + directory and staging area. It shows which files are new, modified, or + staged. + ```bash git status ``` -3. **Stage Files**: Add files to the staging area to prepare them for a commit. You can add one file or all of them. +3. **Stage Files**: Add files to the staging area to prepare them for a commit. + You can add one file or all of them. + ```bash # Stage a specific file git add README.md @@ -80,54 +113,74 @@ Here’s a step-by-step guide to a common workflow: git add . ``` -4. **Commit Changes**: Commit the staged files to your repository. The commit message should be a short, descriptive summary of the changes. +4. **Commit Changes**: Commit the staged files to your repository. The commit + message should be a short, descriptive summary of the changes. + ```bash git commit -m "feat: Add initial project files" ``` -5. **View History**: Use `git log` to see the history of commits in your repository. +5. **View History**: Use `git log` to see the history of commits in your + repository. + ```bash git log ``` -## Branching: Working in Parallel 🌿 +## 🌿 Branching: Working in Parallel -Branches allow you to work on different features or experiments without affecting the main codebase (often called the `main` or `master` branch). +Branches allow you to work on different features or experiments without affecting +the main codebase (often called the `main` or `master` branch). - **Create a new branch and switch to it**: + ```bash git checkout -b new-feature-branch ``` + - **Switch back to the main branch**: + ```bash git checkout main ``` + - **Merge the new feature branch into main**: + ```bash git merge new-feature-branch ``` -## Connecting to the World: Remote Repositories 🌐 +## 🌐 Connecting to the World: Remote Repositories -So far, everything has been on your local machine. To collaborate or to have a backup of your code, you use remote repositories (like on GitHub). +So far, everything has been on your local machine. To collaborate or to have a +backup of your code, you use remote repositories (like on GitHub). + +1. **Add a remote repository**: -1. **Add a remote repository**: ```bash git remote add origin ``` -2. **Push your changes to the remote**: + +2. **Push your changes to the remote**: + ```bash git push -u origin main ``` + The `-u` flag sets the upstream branch, so next time you can just run `git push`. -## What Not to Commit: The `.gitignore` File 🚫 +## 🚫 What Not to Commit: The `.gitignore` File -You don't want to commit every file to your repository. A `.gitignore` file tells Git which files or folders to ignore. +You don't want to commit every file to your repository. A `.gitignore` file +tells Git which files or folders to ignore. -- **Exclude Secrets**: Sensitive data like API keys and passwords should never be committed. -- **Manage Large Files**: Use [Git Large File Storage (git-lfs)](https://git-lfs.github.com/) for files over 100MB (like datasets). -- **Omit Temporary Files**: Ignore environment-specific files (`.venv`), cache (`.pytest_cache`), logs, and build artifacts (`/dist`). +- **Exclude Secrets**: Sensitive data like API keys and passwords should never be + committed. +- **Manage Large Files**: Use + [Git Large File Storage (git-lfs)](https://git-lfs.github.com/) for files over + 100MB (like datasets). +- **Omit Temporary Files**: Ignore environment-specific files (`.venv`), cache + (`.pytest_cache`), logs, and build artifacts (`/dist`). Here is an example `.gitignore` file: @@ -163,17 +216,24 @@ __pycache__/ # Python files *.py[cod] ``` -Adhering to these practices ensures your repository remains streamlined and secure. -## πŸ”‘ Key Takeaways +Adhering to these practices ensures your repository remains streamlined and +secure. -- **Version Control is Key**: Git is indispensable for tracking changes, collaborating, and maintaining project history. -- **Understand the Workflow**: Master the cycle of modifying, staging, and committing changes. -- **Branching for Parallel Development**: Use branches to isolate new features or experiments without disrupting the main codebase. -- **Remote Repositories for Collaboration and Backup**: Push your local changes to platforms like GitHub for sharing and safekeeping. -- **`.gitignore` for Clean Repositories**: Properly configure `.gitignore` to exclude unnecessary or sensitive files from your repository. +## πŸ”‘ Key Takeaways -## Additional Resources πŸ“š +- **Version Control is Key**: Git is indispensable for tracking changes, + collaborating, and maintaining project history. +- **Understand the Workflow**: Master the cycle of modifying, staging, and + committing changes. +- **Branching for Parallel Development**: Use branches to isolate new features + or experiments without disrupting the main codebase. +- **Remote Repositories for Collaboration and Backup**: Push your local changes + to platforms like GitHub for sharing and safekeeping. +- **`.gitignore` for Clean Repositories**: Properly configure `.gitignore` to + exclude unnecessary or sensitive files from your repository. + +## πŸ“š Additional Resources - **[`.gitignore` example from the MLOps Python Package](https://github.com/fmind/mlops-python-package/blob/main/.gitignore)** - [About Git](https://docs.github.com/en/get-started/using-git/about-git) diff --git a/docs/1. Initializing/1.5. GitHub.md b/docs/1. Initializing/1.5. GitHub.md index 38f79c58..dfb492ad 100644 --- a/docs/1. Initializing/1.5. GitHub.md +++ b/docs/1. Initializing/1.5. GitHub.md @@ -1,101 +1,156 @@ --- -description: Explore how to leverage GitHub for project hosting, version control, and collaboration, facilitating teamwork and project management within an MLOps environment. +description: > + Explore how to leverage GitHub for project hosting, version control, and + collaboration, facilitating teamwork and project management within an MLOps + environment. --- -# 1.5. πŸ™ GitHub +# πŸ™ 1.5. GitHub -GitHub is an essential platform for modern software development, and it plays a crucial role in MLOps. It's a web-based platform that uses Git for version control, enabling you to store, manage, and track your code changes. But it's much more than just a code repository. It’s a hub for collaboration, a tool for automation, and a place to showcase your work. +GitHub is an essential platform for modern software development, and it plays a +crucial role in MLOps. It's a web-based platform that uses Git for version +control, enabling you to store, manage, and track your code changes. But it's +much more than just a code repository. It’s a hub for collaboration, a tool for +automation, and a place to showcase your work. ## πŸš€ Getting Started -For any MLOps project, using a version control system is non-negotiable. GitHub is the most popular choice, and for good reason. It simplifies collaboration, helps maintain code quality, and provides a backbone for automation. +For any MLOps project, using a version control system is non-negotiable. GitHub +is the most popular choice, and for good reason. It simplifies collaboration, +helps maintain code quality, and provides a backbone for automation. -### Why GitHub is Key for MLOps: +### πŸ€” Why GitHub is Key for MLOps -- **🀝 Collaboration:** GitHub makes it easy for teams to work together on code. You can review changes, discuss ideas, and track progress all in one place. -- **Versioning:** It keeps a complete history of your project. If something goes wrong, you can easily roll back to a previous state. This is critical for reproducibility in MLOps. -- **CI/CD Integration:** GitHub Actions allows you to automate your workflows, from running tests to deploying models. -- **Open Source Community:** It's the largest host of open-source projects. You can learn from others, contribute to projects, and even get contributions to your own. +- **🀝 Collaboration:** GitHub makes it easy for teams to work together on code. + You can review changes, discuss ideas, and track progress all in one place. +- **Versioning:** It keeps a complete history of your project. If something goes + wrong, you can easily roll back to a previous state. This is critical for + reproducibility in MLOps. +- **CI/CD Integration:** GitHub Actions allows you to automate your workflows, + from running tests to deploying models. +- **Open Source Community:** It's the largest host of open-source projects. You + can learn from others, contribute to projects, and even get contributions to + your own. -### Setting up Your MLOps Project on GitHub +### πŸ› οΈ Setting up Your MLOps Project on GitHub -Setting up a project on GitHub is a straightforward process. Here’s how you can get your local project onto GitHub: +Setting up a project on GitHub is a straightforward process. Here’s how you can +get your local project onto GitHub: -1. **Create a GitHub Repository:** - - Navigate to [GitHub](https://github.com/new) and create a new repository. - - Give it a name, a brief description, and choose whether it should be public or private. - - Initialize it with a `README` file, a `.gitignore` file (the Python template is a good start), and a license (e.g., MIT). +1. **Create a GitHub Repository:** + - Navigate to [GitHub](https://github.com/new) and create a new repository. + - Give it a name, a brief description, and choose whether it should be public + or private. + - Initialize it with a `README` file, a `.gitignore` file (the Python + template is a good start), and a license (e.g., MIT). + +2. **Initialize Git Locally:** + - Open your terminal, navigate to your project's root directory, and + initialize a Git repository: -2. **Initialize Git Locally:** - - Open your terminal, navigate to your project's root directory, and initialize a Git repository: ```bash git init ``` - - Add your project files to the staging area: + + - Add your project files to the staging area: + ```bash git add . ``` - - Commit your files with a meaningful message: + + - Commit your files with a meaningful message: + ```bash git commit -m "Initial commit" ``` -3. **Link and Push to GitHub:** - - Connect your local repository to the one you created on GitHub: +3. **Link and Push to GitHub:** + - Connect your local repository to the one you created on GitHub: + ```bash git remote add origin ``` - - Push your local changes to the remote repository: + + - Push your local changes to the remote repository: + ```bash git push -u origin main ``` - Now your project is on GitHub, and you can start collaborating and building your MLOps pipeline. -## ✨ Core Features for MLOps + Now your project is on GitHub, and you can start collaborating and building + your MLOps pipeline. -GitHub offers a suite of features that are particularly useful for MLOps projects. Here are a few you should get familiar with: +## ✨ Core Features for MLOps -- **[GitHub Repositories](https://github.com/new):** This is the foundation. It's where your code, documentation, and data (or pointers to data) will live. -- **[GitHub Actions](https://github.com/features/actions):** This is your automation engine. You can create workflows to automatically test your code, build your models, and even deploy them. For example, you could have an action that triggers a model retraining pipeline whenever new data is available. -- **[GitHub Projects](https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects):** A project management tool to help you organize your work. You can create Kanban boards to track tasks, bugs, and experiments. -- **[GitHub Security](https://github.com/security):** This feature helps you find and fix vulnerabilities in your code. It's crucial for ensuring your MLOps pipeline is secure. -- **[GitHub Packages](https://github.com/features/packages):** A place to host your software packages, including Docker images and Python packages. This is useful for sharing and reusing components of your MLOps pipeline. +GitHub offers a suite of features that are particularly useful for MLOps +projects. Here are a few you should get familiar with: + +- **[GitHub Repositories](https://github.com/new):** This is the foundation. + It's where your code, documentation, and data (or pointers to data) will live. +- **[GitHub Actions](https://github.com/features/actions):** This is your + automation engine. You can create workflows to automatically test your code, + build your models, and even deploy them. For example, you could have an action + that triggers a model retraining pipeline whenever new data is available. +- **[GitHub Projects](https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects):** + A project management tool to help you organize your work. You can create + Kanban boards to track tasks, bugs, and experiments. +- **[GitHub Security](https://github.com/security):** This feature helps you + find and fix vulnerabilities in your code. It's crucial for ensuring your + MLOps pipeline is secure. +- **[GitHub Packages](https://github.com/features/packages):** A place to host + your software packages, including Docker images and Python packages. This is + useful for sharing and reusing components of your MLOps pipeline. ## πŸ† Best Practices for MLOps -To make the most of GitHub in an MLOps context, it's important to follow some best practices: - -- **Branching Strategy:** Use a consistent branching strategy. A simple and effective one is the **feature branching workflow**: - 1. Create a new branch for each new feature or experiment (e.g., `feature/add-new-model` or `experiment/try-different-hyperparameters`). - 2. Do all your work on that branch. - 3. When you're ready, open a pull request to merge your changes into the `main` branch. -- **Pull Requests (PRs):** PRs are the heart of collaboration on GitHub. They are the place to: - - Review code and suggest improvements. - - Run automated checks (like tests and linting) using GitHub Actions. - - Discuss the changes and ensure they align with the project's goals. - - Always require at least one review before merging a PR. -- **Issue Tracking:** Use GitHub Issues to track everything: bugs, new features, ideas for experiments, and more. - - Use labels to categorize your issues (e.g., `bug`, `enhancement`, `documentation`). - - Link issues to PRs to automatically close them when the PR is merged. +To make the most of GitHub in an MLOps context, it's important to follow some +best practices: + +- **Branching Strategy:** Use a consistent branching strategy. A simple and + effective one is the **feature branching workflow**: + 1. Create a new branch for each new feature or experiment (e.g., + `feature/add-new-model` or `experiment/try-different-hyperparameters`). + 2. Do all your work on that branch. + 3. When you're ready, open a pull request to merge your changes into the + `main` branch. +- **Pull Requests (PRs):** PRs are the heart of collaboration on GitHub. They are + the place to: + - Review code and suggest improvements. + - Run automated checks (like tests and linting) using GitHub Actions. + - Discuss the changes and ensure they align with the project's goals. + - Always require at least one review before merging a PR. +- **Issue Tracking:** Use GitHub Issues to track everything: bugs, new features, + ideas for experiments, and more. + - Use labels to categorize your issues (e.g., `bug`, `enhancement`, + `documentation`). + - Link issues to PRs to automatically close them when the PR is merged. ## ↔️ Exploring Alternatives -While GitHub is the most popular platform, there are other excellent options available: +While GitHub is the most popular platform, there are other excellent options +available: -- **[Bitbucket](https://bitbucket.org/product):** Great for teams that use other Atlassian products like Jira and Trello. -- **[GitLab](https://about.gitlab.com/):** Offers a complete DevOps platform in a single application. It has strong CI/CD features built-in. -- **[Azure DevOps](https://azure.microsoft.com/en-us/products/devops), [Cloud Source Repository](https://cloud.google.com/source-repositories/docs), [AWS CodeCommit](https://aws.amazon.com/codecommit/):** These are good options if you are heavily invested in a particular cloud ecosystem. +- **[Bitbucket](https://bitbucket.org/product):** Great for teams that use other + Atlassian products like Jira and Trello. +- **[GitLab](https://about.gitlab.com/):** Offers a complete DevOps platform in a + single application. It has strong CI/CD features built-in. +- **[Azure DevOps](https://azure.microsoft.com/en-us/products/devops), + [Cloud Source Repository](https://cloud.google.com/source-repositories/docs), + [AWS CodeCommit](https://aws.amazon.com/codecommit/):** These are good options + if you are heavily invested in a particular cloud ecosystem. ## πŸ”‘ Key Takeaways -- GitHub is indispensable for MLOps due to its robust version control, collaboration features, and CI/CD capabilities. -- Always use a `.gitignore` file to exclude unnecessary or sensitive files. -- Leverage GitHub Actions for automating your MLOps workflows. -- Adopt a consistent branching strategy and use Pull Requests for code review and collaboration. -- Track all work, including bugs and experiments, using GitHub Issues. +- GitHub is indispensable for MLOps due to its robust version control, + collaboration features, and CI/CD capabilities. +- Always use a `.gitignore` file to exclude unnecessary or sensitive files. +- Leverage GitHub Actions for automating your MLOps workflows. +- Adopt a consistent branching strategy and use Pull Requests for code review and + collaboration. +- Track all work, including bugs and experiments, using GitHub Issues. ## πŸ“š Additional Resources -- **[MLOps Python Package on GitHub](https://github.com/fmind/mlops-python-package)** -- **[GitHub's Official Guides](https://guides.github.com/)** -- **[Introduction to Git and GitHub for Python Developers](https://realpython.com/python-git-github-intro/)** +- **[MLOps Python Package on GitHub](https://github.com/fmind/mlops-python-package)** +- **[GitHub's Official Guides](https://guides.github.com/)** +- **[Introduction to Git and GitHub for Python Developers](https://realpython.com/python-git-github-intro/)** diff --git a/docs/1. Initializing/1.6. VS Code.md b/docs/1. Initializing/1.6. VS Code.md index 4c052e13..1a60d5f6 100644 --- a/docs/1. Initializing/1.6. VS Code.md +++ b/docs/1. Initializing/1.6. VS Code.md @@ -1,22 +1,41 @@ --- -description: Set up Visual Studio Code (VS Code) as your IDE for Python and MLOps projects, customizing it with essential extensions and settings to enhance your coding workflow. +description: > + Set up Visual Studio Code (VS Code) as your IDE for Python and MLOps + projects, customizing it with essential extensions and settings to enhance + your coding workflow. --- -# 1.6. VS Code πŸ‘©β€πŸ’» +# πŸ‘©β€πŸ’» 1.6. VS Code ## πŸ€” What is VS Code? -[Visual Studio Code (VS Code)](https://code.visualstudio.com/) is a versatile and free open-source code editor developed by Microsoft. Known for its efficiency and adaptability, it supports a multitude of programming languages with built-in support for JavaScript, TypeScript, and Node.js. It's celebrated for its extensibility, allowing users to install extensions for additional languages, frameworks, and tools, making it a highly customizable environment for software development. +[Visual Studio Code (VS Code)](https://code.visualstudio.com/) is a versatile +and free open-source code editor developed by Microsoft. Known for its +efficiency and adaptability, it supports a multitude of programming languages +with built-in support for JavaScript, TypeScript, and Node.js. It's celebrated +for its extensibility, allowing users to install extensions for additional +languages, frameworks, and tools, making it a highly customizable environment +for software development. ## 🌟 Why should you use VS Code? VS Code stands out as a top choice for developers due to its: -- **Lightweight Performance**: Offers robust features like [IntelliSense for code completion](https://code.visualstudio.com/docs/editor/intellisense), [code navigation](https://code.visualstudio.com/docs/editor/editingevolved), and real-time syntax highlighting without bogging down your system. -- **Extensibility**: Its [extension marketplace](https://marketplace.visualstudio.com/VSCode) provides support for almost all major programming languages and tools. -- **Integrated Terminal**: Allows you to run shell commands, Git operations, and scripts from within the editor. -- **Debugging Tools**: Comes with [built-in debugging support](https://code.visualstudio.com/docs/editor/debugging) for several languages and the capability to add more via extensions. -- **Source Control Integration**: Features [integrated Git support](https://code.visualstudio.com/docs/sourcecontrol/overview) and extensions for other version control systems, enhancing code management. +- **Lightweight Performance**: Offers robust features like + [IntelliSense for code completion](https://code.visualstudio.com/docs/editor/intellisense), + [code navigation](https://code.visualstudio.com/docs/editor/editingevolved), + and real-time syntax highlighting without bogging down your system. +- **Extensibility**: Its + [extension marketplace](https://marketplace.visualstudio.com/VSCode) provides + support for almost all major programming languages and tools. +- **Integrated Terminal**: Allows you to run shell commands, Git operations, and + scripts from within the editor. +- **Debugging Tools**: Comes with + [built-in debugging support](https://code.visualstudio.com/docs/editor/debugging) + for several languages and the capability to add more via extensions. +- **Source Control Integration**: Features + [integrated Git support](https://code.visualstudio.com/docs/sourcecontrol/overview) + and extensions for other version control systems, enhancing code management. ## πŸ› οΈ How should you install VS Code? @@ -25,9 +44,13 @@ To get started with VS Code: 1. Go to the [Visual Studio Code website](https://code.visualstudio.com/). 2. Download the installer for your operating system (Windows, macOS, or Linux). 3. Execute the installer and follow the prompts to complete the installation. -4. **(Optional but Recommended)** On macOS and Linux, add `code` to your `PATH` so you can open files and folders from the command line. You can do this by opening the Command Palette (`Cmd+Shift+P`) and typing `Shell Command: Install 'code' command in PATH`. +4. **(Optional but Recommended)** On macOS and Linux, add `code` to your `PATH` + so you can open files and folders from the command line. You can do this by + opening the Command Palette (`Cmd+Shift+P`) and typing + `Shell Command: Install 'code' command in PATH`. Once installed, you can open your project folder from the terminal by running: + ```bash code . ``` @@ -36,63 +59,107 @@ code . There are plenty of resources to help you master VS Code: -- **[Official Documentation](https://code.visualstudio.com/docs)**: A thorough guide covering everything from basic to advanced features. -- **[VS Code Tips and Tricks](https://code.visualstudio.com/docs/getstarted/tips-and-tricks)**: Enhance your productivity with these best practices and shortcuts. -- **[Microsoft Learn](https://docs.microsoft.com/en-us/learn/modules/use-vscode/)**: Offers free, interactive tutorials for using VS Code in various scenarios. -- **[YouTube Channels](https://www.youtube.com/@code)**: Search for VS Code tutorials on YouTube to find numerous guides for all skill levels. +- **[Official Documentation](https://code.visualstudio.com/docs)**: A thorough + guide covering everything from basic to advanced features. +- **[VS Code Tips and Tricks](https://code.visualstudio.com/docs/getstarted/tips-and-tricks)**: + Enhance your productivity with these best practices and shortcuts. +- **[Microsoft Learn](https://docs.microsoft.com/en-us/learn/modules/use-vscode/)**: + Offers free, interactive tutorials for using VS Code in various scenarios. +- **[YouTube Channels](https://www.youtube.com/@code)**: Search for VS Code + tutorials on YouTube to find numerous guides for all skill levels. ## 🧩 Which VS Code extensions should you install for MLOps? -This section lists recommended extensions from the [VS Code Marketplace](https://marketplace.visualstudio.com/VSCode), grouped by functionality. +This section lists recommended extensions from the +[VS Code Marketplace](https://marketplace.visualstudio.com/VSCode), grouped by +functionality. ### 🐍 Python Development -- [**Python Extension Pack**](https://marketplace.visualstudio.com/items?itemName=donjayamanne.python-extension-pack): The ultimate pack for Python development. Includes `autoDocString`, `Python`, `Jinja`, `IntelliCode`, `Python Indent`, and `Python Environment Manager`. -- [**Python Environment Manager**](https://marketplace.visualstudio.com/items?itemName=donjayamanne.python-environment-manager): Access and manage your Python environments (e.g., venv, conda). -- [**Ruff**](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff): An extremely fast Python linter and code formatter, written in Rust. -- [**Mypy Type Checker**](https://marketplace.visualstudio.com/items?itemName=ms-python.mypy-type-checker): Static type checking for Python. Great for validating your code and communicating how it should be used. -- [**Pylance**](https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance): An enhanced language server for Python, providing features like static type checking and smart completions. +- [**Python Extension Pack**](https://marketplace.visualstudio.com/items?itemName=donjayamanne.python-extension-pack): + The ultimate pack for Python development. Includes `autoDocString`, `Python`, + `Jinja`, `IntelliCode`, `Python Indent`, and `Python Environment Manager`. +- [**Python Environment Manager**](https://marketplace.visualstudio.com/items?itemName=donjayamanne.python-environment-manager): + Access and manage your Python environments (e.g., venv, conda). +- [**Ruff**](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff): + An extremely fast Python linter and code formatter, written in Rust. +- [**Mypy Type Checker**](https://marketplace.visualstudio.com/items?itemName=ms-python.mypy-type-checker): + Static type checking for Python. Great for validating your code and + communicating how it should be used. +- [**Pylance**](https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance): + An enhanced language server for Python, providing features like static type + checking and smart completions. ### πŸ“Š Data Science & Notebooks -- [**Jupyter**](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter): Extension pack for Jupyter Notebooks, including keymaps, slideshow capabilities, notebook renderers, and cell tags. -- [**Rainbow CSV**](https://marketplace.visualstudio.com/items?itemName=mechatroner.rainbow-csv): Colorizes CSV columns for improved readability. +- [**Jupyter**](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter): + Extension pack for Jupyter Notebooks, including keymaps, slideshow + capabilities, notebook renderers, and cell tags. +- [**Rainbow CSV**](https://marketplace.visualstudio.com/items?itemName=mechatroner.rainbow-csv): + Colorizes CSV columns for improved readability. ### πŸ™ Version Control (Git) -- [**GitLens**](https://marketplace.visualstudio.com/items?itemName=eamodio.gitlens): Supercharges the Git capabilities built into VS Code. Visualize code authorship with Git blame annotations and code lens, seamlessly navigate and explore Git repositories, and more. -- [**Git History**](https://marketplace.visualstudio.com/items?itemName=donjayamanne.githistory): Visualize your Git history (files, branches, commits). -- **[Git Graph](https://marketplace.visualstudio.com/items?itemName=mhutchie.git-graph)**: View a Git Graph of your repository, and easily perform Git actions from the graph. +- [**GitLens**](https://marketplace.visualstudio.com/items?itemName=eamodio.gitlens): + Supercharges the Git capabilities built into VS Code. Visualize code authorship + with Git blame annotations and code lens, seamlessly navigate and explore Git + repositories, and more. +- [**Git History**](https://marketplace.visualstudio.com/items?itemName=donjayamanne.githistory): + Visualize your Git history (files, branches, commits). +- **[Git Graph](https://marketplace.visualstudio.com/items?itemName=mhutchie.git-graph)**: + View a Git Graph of your repository, and easily perform Git actions from the + graph. ### ☁️ Remote Development & Containers -- [**Remote Development Extension Pack**](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack): Essential for remote development. Includes `Dev Containers`, `Remote - SSH`, `Remote - Tunnels`, and `WSL`. -- [**Docker**](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker): Manage and connect to Docker from within VS Code. -- [**Kubernetes**](https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools): Everything you need to develop Kubernetes applications. +- [**Remote Development Extension Pack**](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack): + Essential for remote development. Includes `Dev Containers`, `Remote - SSH`, + `Remote - Tunnels`, and `WSL`. +- [**Docker**](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker): + Manage and connect to Docker from within VS Code. +- [**Kubernetes**](https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools): + Everything you need to develop Kubernetes applications. ### πŸ“ General Productivity & Editing -- [**Project Manager**](https://marketplace.visualstudio.com/items?itemName=alefragnani.project-manager): Organize, manage, and access VS Code workspaces and Git repositories. -- [**Even Better TOML**](https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml): Language support for TOML files, improving the editing experience for `pyproject.toml`. -- [**YAML**](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml): Language support for YAML, a popular configuration file format. -- [**Code Spell Checker**](https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker): Highlights and helps fix spelling mistakes in your code. -- [**Error Lens**](https://marketplace.visualstudio.com/items?itemName=usernamehw.errorlens): Displays warnings and errors inline with your code. -- [**Markdown All in One**](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one): All you need for editing Markdown files (shortcuts, table of contents, etc.). -- [**Better Comments**](https://marketplace.visualstudio.com/items?itemName=aaron-bond.better-comments): Highlights comments based on prefixes like `*`, `!`, `?`, `TODO`. -- [**indent-rainbow**](https://marketplace.visualstudio.com/items?itemName=oderwat.indent-rainbow): Makes code indentation more readable. -- [**Path Intellisense**](https://marketplace.visualstudio.com/items?itemName=christian-kohler.path-intellisense): Autocompletes file paths. -- [**Todo Tree**](https://marketplace.visualstudio.com/items?itemName=Gruntfuggly.todo-tree): Aggregates `TODO`, `FIXME`, and other annotations in a dedicated view. +- [**Project Manager**](https://marketplace.visualstudio.com/items?itemName=alefragnani.project-manager): + Organize, manage, and access VS Code workspaces and Git repositories. +- [**Even Better TOML**](https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml): + Language support for TOML files, improving the editing experience for + `pyproject.toml`. +- [**YAML**](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml): + Language support for YAML, a popular configuration file format. +- [**Code Spell Checker**](https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker): + Highlights and helps fix spelling mistakes in your code. +- [**Error Lens**](https://marketplace.visualstudio.com/items?itemName=usernamehw.errorlens): + Displays warnings and errors inline with your code. +- [**Markdown All in One**](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one): + All you need for editing Markdown files (shortcuts, table of contents, etc.). +- [**Better Comments**](https://marketplace.visualstudio.com/items?itemName=aaron-bond.better-comments): + Highlights comments based on prefixes like `*`, `!`, `?`, `TODO`. +- [**indent-rainbow**](https://marketplace.visualstudio.com/items?itemName=oderwat.indent-rainbow): + Makes code indentation more readable. +- [**Path Intellisense**](https://marketplace.visualstudio.com/items?itemName=christian-kohler.path-intellisense): + Autocompletes file paths. +- [**Todo Tree**](https://marketplace.visualstudio.com/items?itemName=Gruntfuggly.todo-tree): + Aggregates `TODO`, `FIXME`, and other annotations in a dedicated view. ## βš™οΈ How can you configure VS Code settings for MLOps? -You can customize VS Code by modifying its `settings.json` file. Access it via the Command Palette (`Ctrl+Shift+P`) with `Preferences: Open User Settings (JSON)`. +You can customize VS Code by modifying its `settings.json` file. Access it via +the Command Palette (`Ctrl+Shift+P`) with +`Preferences: Open User Settings (JSON)`. ### User vs. Workspace Settings - **User Settings**: Apply globally to all your VS Code instances. -- **Workspace Settings**: Specific to a project and stored in a `.vscode/settings.json` file within the project folder. These override user settings and are useful for project-specific configurations that you can share with your team. +- **Workspace Settings**: Specific to a project and stored in a + `.vscode/settings.json` file within the project folder. These override user + settings and are useful for project-specific configurations that you can share + with your team. -Here are some recommended settings for MLOps development. Each setting is annotated to explain its purpose. +Here are some recommended settings for MLOps development. Each setting is +annotated to explain its purpose. ```json { @@ -190,46 +257,78 @@ Here are some recommended settings for MLOps development. Each setting is annota ## πŸš€ Tips and Tricks for MLOps -- **Multi-root Workspaces**: If your MLOps project involves multiple repositories (e.g., one for infrastructure, one for modeling), use a multi-root workspace to manage them all in a single VS Code window. Go to `File > Add Folder to Workspace...`. -- **Command Palette is Your Best Friend**: `Ctrl+Shift+P` (or `Cmd+Shift+P` on macOS) is the most important shortcut. You can access almost any command or setting from here. -- **Integrated Terminal**: Use the integrated terminal (`Ctrl+\``) to run shell commands without leaving the editor. You can have multiple terminals open at once. -- **Debugging**: Use the built-in debugger for Python. Set breakpoints, inspect variables, and step through your code. The Jupyter extension also provides a great debugging experience for notebooks. +- **Multi-root Workspaces**: If your MLOps project involves multiple + repositories (e.g., one for infrastructure, one for modeling), use a + multi-root workspace to manage them all in a single VS Code window. Go to + `File > Add Folder to Workspace...`. +- **Command Palette is Your Best Friend**: `Ctrl+Shift+P` (or `Cmd+Shift+P` on + macOS) is the most important shortcut. You can access almost any command or + setting from here. +- **Integrated Terminal**: Use the integrated terminal (`Ctrl+\``) to run shell + commands without leaving the editor. You can have multiple terminals open at + once. +- **Debugging**: Use the built-in debugger for Python. Set breakpoints, inspect + variables, and step through your code. The Jupyter extension also provides a + great debugging experience for notebooks. ## ⌨️ Useful Keyboard Shortcuts -| Shortcut (Windows/Linux) | Shortcut (macOS) | Description | -|---|---|---| -| `Ctrl+Shift+P` | `Cmd+Shift+P` | Open Command Palette | -| `Ctrl+P` | `Cmd+P` | Go to File | -| `Ctrl+Shift+F` | `Cmd+Shift+F` | Search across files | -| `Ctrl+\`` | `Cmd+\`` | Toggle integrated terminal | -| `Ctrl+B` | `Cmd+B` | Toggle sidebar visibility | -| `F5` | `F5` | Start debugging | -| `Shift+Alt+O` | `Shift+Option+O` | Organize imports (with Python extension) | -| `Ctrl+K Ctrl+S` | `Cmd+K Cmd+S` | Open Keyboard Shortcuts | - +| Shortcut (Windows/Linux) | Shortcut (macOS) | Description | +| ------------------------ | ------------------ | ------------------------------------------ | +| `Ctrl+Shift+P` | `Cmd+Shift+P` | Open Command Palette | +| `Ctrl+P` | `Cmd+P` | Go to File | +| `Ctrl+Shift+F` | `Cmd+Shift+F` | Search across files | +| `Ctrl+\`` | `Cmd+\`` | Toggle integrated terminal | +| `Ctrl+B` | `Cmd+B` | Toggle sidebar visibility | +| `F5` | `F5` | Start debugging | +| `Shift+Alt+O` | `Shift+Option+O` | Organize imports (with Python extension) | +| `Ctrl+K Ctrl+S` | `Cmd+K Cmd+S` | Open Keyboard Shortcuts | ## ⚑ How to configure VS Code for using the [Jupyter Extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) with [uv](https://docs.astral.sh/uv/)? -To configure VS Code for using the [Jupyter Extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) with [uv](https://docs.astral.sh/uv/), follow these steps to ensure that your uv-managed virtual environment is recognized within VS Code. This allows you to use the Jupyter Extension seamlessly with the Python interpreter provided by uv. - -1. **Install the Jupyter Extension**: First, ensure that the [Jupyter Extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) is installed in VS Code. You can find and install this extension from the [VS Code Marketplace](https://marketplace.visualstudio.com/VSCode). -2. **Open Your Project in VS Code**: Open your project folder in VS Code. If you've just created a new uv project, this will be the directory containing your `pyproject.toml` file. -3. **Select Python Interpreter**: To make VS Code use the Python interpreter from your uv environment: - - Open the Command Palette (`Ctrl+Shift+P` on Windows/Linux, `Cmd+Shift+P` on macOS). +To configure VS Code for using the +[Jupyter Extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) +with [uv](https://docs.astral.sh/uv/), follow these steps to ensure that your +uv-managed virtual environment is recognized within VS Code. This allows you to +use the Jupyter Extension seamlessly with the Python interpreter provided by uv. + +1. **Install the Jupyter Extension**: First, ensure that the + [Jupyter Extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) + is installed in VS Code. You can find and install this extension from the + [VS Code Marketplace](https://marketplace.visualstudio.com/VSCode). +2. **Open Your Project in VS Code**: Open your project folder in VS Code. If + you've just created a new uv project, this will be the directory containing + your `pyproject.toml` file. +3. **Select Python Interpreter**: To make VS Code use the Python interpreter + from your uv environment: + - Open the Command Palette (`Ctrl+Shift+P` on Windows/Linux, `Cmd+Shift+P` on + macOS). - Type `Python: Select Interpreter` and select it. - - Look for the interpreter that corresponds to your uv environment. It will typically be located under the `.venv` path within your project directory or listed as a virtual environment with your project's name. + - Look for the interpreter that corresponds to your uv environment. It will + typically be located under the `.venv` path within your project directory + or listed as a virtual environment with your project's name. - Select the appropriate interpreter. -4. **Verify Jupyter Notebook Configuration**: Create a new Jupyter notebook in VS Code (`*.ipynb` file) and verify that the cells execute using the Python interpreter from your uv environment. You can check the upper-right corner of the notebook interface to see which interpreter is currently active. -5. **Install Necessary Libraries**: If you need additional Python libraries that are not yet part of your uv project, you can add them by running `uv add ` in your terminal or command prompt. This ensures that all dependencies are managed by uv and available in the notebook. +4. **Verify Jupyter Notebook Configuration**: Create a new Jupyter notebook in + VS Code (`*.ipynb` file) and verify that the cells execute using the Python + interpreter from your uv environment. You can check the upper-right corner of + the notebook interface to see which interpreter is currently active. +5. **Install Necessary Libraries**: If you need additional Python libraries that + are not yet part of your uv project, you can add them by running + `uv add ` in your terminal or command prompt. This ensures that + all dependencies are managed by uv and available in the notebook. # πŸ”‘ Key Takeaways -- **VS Code is a powerful and customizable IDE** for MLOps, offering a rich set of features and extensions. -- **Install essential extensions** for Python development, data science, Git, and remote development to enhance your workflow. -- **Configure VS Code settings** to optimize your environment for MLOps, including formatting, linting, and Git integration. -- **Leverage the Command Palette and integrated terminal** for efficient navigation and execution of commands. -- **Ensure proper integration with `uv`** to manage Python environments and dependencies seamlessly within VS Code. +- **VS Code is a powerful and customizable IDE** for MLOps, offering a rich set + of features and extensions. +- **Install essential extensions** for Python development, data science, Git, and + remote development to enhance your workflow. +- **Configure VS Code settings** to optimize your environment for MLOps, + including formatting, linting, and Git integration. +- **Leverage the Command Palette and integrated terminal** for efficient + navigation and execution of commands. +- **Ensure proper integration with `uv`** to manage Python environments and + dependencies seamlessly within VS Code. ## πŸ”— VS Code additional resources @@ -238,4 +337,5 @@ To configure VS Code for using the [Jupyter Extension](https://marketplace.visua - [Advanced Visual Studio Code for Python Developers](https://realpython.com/advanced-visual-studio-code-python/) - [How to configure VS Code for AI, ML and MLOps development in Python](https://fmind.medium.com/how-to-configure-vs-code-for-ai-ml-and-mlops-development-in-python-%EF%B8%8F%EF%B8%8F-8582d8c6ea54) - [Awesome VS Code](https://github.com/viatsko/awesome-vscode) -- [VS Code Can Do That?!](https://vscodecandothat.com/): A collection of tips and tricks for VS Code. +- [VS Code Can Do That?!](https://vscodecandothat.com/): A collection of tips + and tricks for VS Code. diff --git a/docs/1. Initializing/index.md b/docs/1. Initializing/index.md index 7f025bfa..e7a2111c 100644 --- a/docs/1. Initializing/index.md +++ b/docs/1. Initializing/index.md @@ -1,15 +1,40 @@ --- -description: This chapter guides you through setting up a robust development environment for MLOps projects using Python. Learn how to manage Python versions, install dependencies with Poetry, and use Git and GitHub for version control and collaboration. +description: > + This chapter guides you through setting up a robust development environment + for MLOps projects using Python. Learn how to manage Python versions, install + dependencies with Poetry, and use Git and GitHub for version control and + collaboration. --- -# 1. Initializing +# πŸš€ 1. Initializing -The initialization phase is crucial in setting the stage for efficient and streamlined development, particularly for projects centered around Python and MLOps. This chapter aims to guide you through establishing a robust development setup, ensuring that every necessary tool and environment is correctly configured from the get-go. By following these foundational steps, you'll create a solid base for your project, enabling smooth progress and reducing the likelihood of delays caused by environment-related issues. +The initialization phase is crucial in setting the stage for efficient and +streamlined development, particularly for projects centered around Python and +MLOps. This chapter aims to guide you through establishing a robust development +setup, ensuring that every necessary tool and environment is correctly +configured from the get-go. By following these foundational steps, you'll create +a solid base for your project, enabling smooth progress and reducing the +likelihood of delays caused by environment-related issues. -- **[1.0. System](./1.0. System.md)**: This section ensures your system is adequately prepared, outlining the essential prerequisites for installing and effectively running the necessary development tools. -- **[1.1. Python](./1.1. Python.md)**: Here, we introduce how to set up Pythonβ€”the core programming language for our projects. We'll focus on version management and creating isolated environments for each project to avoid conflicts and dependency issues. -- **[1.2. uv](./1.2. uv.md)**: We explore `uv`, an extremely fast Python tool written in Rust. Uv can install Python versions, manage virtual environments, and handle dependencies, making it a versatile tool for MLOps projects. -- **[1.3. uv (project)](./1.3. uv (project).md)**: This part delves into using `uv` for project packaging. It simplifies the process of defining, installing, and updating project metadata and dependencies with ease. -- **[1.4. git](./1.4. git.md)**: Focuses on `git`, the cornerstone version control system integral to GitHub. You'll learn how to initiate and manage repositories effectively, a critical skill for collaborative development. -- **[1.5. GitHub](./1.5. GitHub.md)**: Discusses how to leverage GitHub for project hosting, version control, and collaboration. It's a pivotal component in modern development workflows, facilitating teamwork and project management. -- **[1.6. VS Code](./1.6. VS Code.md)**: Highlights the setup of Visual Studio Code (VS Code), showing how to adapt this versatile editor into an integrated development environment (IDE) customized for Python and MLOps projects. +- **[1.0. System](./1.0. System.md)**: This section ensures your system is + adequately prepared, outlining the essential prerequisites for installing and + effectively running the necessary development tools. +- **[1.1. Python](./1.1. Python.md)**: Here, we introduce how to set up + Pythonβ€”the core programming language for our projects. We'll focus on version + management and creating isolated environments for each project to avoid + conflicts and dependency issues. +- **[1.2. uv](./1.2. uv.md)**: We explore `uv`, an extremely fast Python tool + written in Rust. Uv can install Python versions, manage virtual environments, + and handle dependencies, making it a versatile tool for MLOps projects. +- **[1.3. uv (project)](./1.3. uv (project).md)**: This part delves into using + `uv` for project packaging. It simplifies the process of defining, installing, + and updating project metadata and dependencies with ease. +- **[1.4. git](./1.4. git.md)**: Focuses on `git`, the cornerstone version + control system integral to GitHub. You'll learn how to initiate and manage + repositories effectively, a critical skill for collaborative development. +- **[1.5. GitHub](./1.5. GitHub.md)**: Discusses how to leverage GitHub for + project hosting, version control, and collaboration. It's a pivotal component + in modern development workflows, facilitating teamwork and project management. +- **[1.6. VS Code](./1.6. VS Code.md)**: Highlights the setup of Visual Studio + Code (VS Code), showing how to adapt this versatile editor into an integrated + development environment (IDE) customized for Python and MLOps projects. diff --git a/docs/2. Prototyping/2.2. Configs.md b/docs/2. Prototyping/2.2. Configs.md index 28bc8ac9..dae0dd37 100644 --- a/docs/2. Prototyping/2.2. Configs.md +++ b/docs/2. Prototyping/2.2. Configs.md @@ -103,6 +103,7 @@ pd.options.display.max_columns = None # Adjust sklearn output format sklearn.set_config(transform_output="pandas") ``` + ## Why do you need to pass options? Library defaults may not always cater to your specific needs or the demands of your project. For instance: diff --git a/docs/2. Prototyping/2.4. Analysis.md b/docs/2. Prototyping/2.4. Analysis.md index 58c3d381..8f9cc134 100644 --- a/docs/2. Prototyping/2.4. Analysis.md +++ b/docs/2. Prototyping/2.4. Analysis.md @@ -82,6 +82,7 @@ Handling missing values in datasets is crucial for maintaining data integrity. H 1. **Install MissingNo**: `pip install missingno` 2. **Import and Use**: + ```python import missingno as msno import pandas as pd diff --git a/docs/3. Productionizing/3.1. Modules.md b/docs/3. Productionizing/3.1. Modules.md index 4b366add..f37f7ee6 100644 --- a/docs/3. Productionizing/3.1. Modules.md +++ b/docs/3. Productionizing/3.1. Modules.md @@ -35,8 +35,8 @@ Modules become indispensable as your project grows beyond a simple script. While Creating a Python module is as simple as creating a `.py` file within your project package. For example, in a project structured with a `src` directory, you might organize your modules as follows: ```bash -$ touch src/bikes/models.py -$ touch src/bikes/datasets.py +touch src/bikes/models.py +touch src/bikes/datasets.py ``` This creates two modules, `models.py` and `datasets.py`, under the `bikes` package. Each module can then contain specific functionalities related to your project, such as defining data models or handling dataset loading and preprocessing. diff --git a/docs/3. Productionizing/3.3. Entrypoints.md b/docs/3. Productionizing/3.3. Entrypoints.md index 9f108a0d..4cb9d821 100644 --- a/docs/3. Productionizing/3.3. Entrypoints.md +++ b/docs/3. Productionizing/3.3. Entrypoints.md @@ -83,7 +83,7 @@ bikes = 'bikes.scripts:main' In this syntax, `bikes` represents the command users will enter in the CLI to activate your tool. The path `bikes.scripts:main` directs uv to execute the `main` function found in the `scripts` module of the `bikes` package. Upon installation, uv generates an executable script for this command, integrating your package's functionality seamlessly into the user's command-line environment, alongside other common utilities: ```bash -$ uv run bikes one two three +uv run bikes one two three ``` This snippet run the bikes entrypoint from the CLI and passes 3 positional arguments: one, two, and three. diff --git a/docs/3. Productionizing/3.4. Configurations.md b/docs/3. Productionizing/3.4. Configurations.md index 5b233a81..c7fd27de 100644 --- a/docs/3. Productionizing/3.4. Configurations.md +++ b/docs/3. Productionizing/3.4. Configurations.md @@ -34,7 +34,7 @@ When choosing a format for configuration files, common options include [JSON](ht Passing configuration files to your program typically utilizes the CLI, offering a straightforward method to integrate configurations with additional command options or flags. For example, executing a command like: ```bash -$ bikes defaults.yaml training.yaml --verbose +bikes defaults.yaml training.yaml --verbose ``` This example enables the combination of configuration files with verbosity options for more detailed logging. This flexibility is also extendable to configurations stored on cloud services, provided your application supports such paths. @@ -86,7 +86,7 @@ class TrainTestSplitter(pdt.BaseModel): Environment variables are more suitable for simple configurations or when dealing with sensitive information that shouldn't be stored in files, even though they lack the structure and type-safety of dedicated configuration files. They are universally supported and easily integrated but may become cumbersome for managing complex or numerous settings. ```bash -$ MLFLOW_TRACKING_URI=./mlruns bikes one two three +MLFLOW_TRACKING_URI=./mlruns bikes one two three ``` In this example, the `MLFLOW_TRACKING_URI` is passed as an environment variable to the `bikes` program, while the command also accepts 3 positional arguments: one, two, and three. diff --git a/docs/3. Productionizing/3.5. Documentations.md b/docs/3. Productionizing/3.5. Documentations.md index f7a6c6e0..d8daf67d 100644 --- a/docs/3. Productionizing/3.5. Documentations.md +++ b/docs/3. Productionizing/3.5. Documentations.md @@ -64,23 +64,23 @@ Beyond in-code documentation, complementing it with external documentation (e.g. For creating documentation, you have multiple tools, formats, and conventions at your disposal: - **Tools**: - - **[MkDocs](https://www.mkdocs.org/)**: A fast, simple static site generator designed for project documentation, built with Python. - - **[pdoc](https://pdoc.dev/)**: A tool and library for auto-generating API documentation for Python projects, best for API docs. - - **[Sphinx](https://www.sphinx-doc.org/en/master/)**: A robust tool for creating detailed and beautiful documentation, popular within the Python community, albeit with a steeper setup curve. + - **[MkDocs](https://www.mkdocs.org/)**: A fast, simple static site generator designed for project documentation, built with Python. + - **[pdoc](https://pdoc.dev/)**: A tool and library for auto-generating API documentation for Python projects, best for API docs. + - **[Sphinx](https://www.sphinx-doc.org/en/master/)**: A robust tool for creating detailed and beautiful documentation, popular within the Python community, albeit with a steeper setup curve. - **Formats**: - - **[Markdown](https://www.markdownguide.org/)**: A straightforward text format that converts to HTML, ideal for simpler docs. - - **[reStructuredText (reST)](https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html)**: Offers more features than Markdown, widely used in Python documentation, especially with Sphinx. + - **[Markdown](https://www.markdownguide.org/)**: A straightforward text format that converts to HTML, ideal for simpler docs. + - **[reStructuredText (reST)](https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html)**: Offers more features than Markdown, widely used in Python documentation, especially with Sphinx. - **Convention**: - - **[Numpy Style](https://numpydoc.readthedocs.io/en/latest/format.html)**: Features a clear, structured format for documenting Python functions, classes, and modules, focusing on readability. - - **[Google Style](https://google.github.io/styleguide/pyguide.html)**: Known for its simplicity and ease of use in documenting Python code, emphasizing clarity and brevity. - - **[reStructuredText](https://developer.lsst.io/restructuredtext/style.html)**: Offers a comprehensive set of markup syntax and constructs, ideal for technical documentation that requires detailed structuring and cross-referencing. + - **[Numpy Style](https://numpydoc.readthedocs.io/en/latest/format.html)**: Features a clear, structured format for documenting Python functions, classes, and modules, focusing on readability. + - **[Google Style](https://google.github.io/styleguide/pyguide.html)**: Known for its simplicity and ease of use in documenting Python code, emphasizing clarity and brevity. + - **[reStructuredText](https://developer.lsst.io/restructuredtext/style.html)**: Offers a comprehensive set of markup syntax and constructs, ideal for technical documentation that requires detailed structuring and cross-referencing. For best practices, choose a tool and format that align with your project's needs and complexity. Adopting a widely recognized convention can facilitate consistency and comprehension across your documentation. Generating a simple API documentation can be as simple as calling a tool like [`pdoc`](https://pdoc.dev/) with an input and output directory: ```bash -$ uv run pdoc --docformat=google --output-directory=docs/ src/bikes +uv run pdoc --docformat=google --output-directory=docs/ src/bikes ``` You can also use your IDE or some extensions like [autoDocstring](https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring) to automate the documentation generation process. diff --git a/docs/3. Productionizing/3.6. VS Code Workspace.md b/docs/3. Productionizing/3.6. VS Code Workspace.md index bad1965a..cacb1d8a 100644 --- a/docs/3. Productionizing/3.6. VS Code Workspace.md +++ b/docs/3. Productionizing/3.6. VS Code Workspace.md @@ -30,34 +30,34 @@ VS Code Workspace configurations support [the same settings as user settings](ht ```json { - "folders": [ - { - "path": "." - } - ], - "settings": { - "editor.formatOnSave": true, - "python.defaultInterpreterPath": ".venv/bin/python", - "python.testing.pytestEnabled": true, - "python.testing.pytestArgs": [ - "tests" - ], - "[python]": { - "editor.codeActionsOnSave": { - "source.organizeImports": true - }, - "editor.defaultFormatter": "charliermarsh.ruff", - } - }, - "extensions": { - "recommendations": [ - "charliermarsh.ruff", - "ms-python.mypy-type-checker", - "ms-python.python", - "ms-python.vscode-pylance", - "redhat.vscode-yaml" - ] - } + "folders": [ + { + "path": "." + } + ], + "settings": { + "editor.formatOnSave": true, + "python.defaultInterpreterPath": ".venv/bin/python", + "python.testing.pytestEnabled": true, + "python.testing.pytestArgs": [ + "tests" + ], + "[python]": { + "editor.codeActionsOnSave": { + "source.organizeImports": true + }, + "editor.defaultFormatter": "charliermarsh.ruff", + } + }, + "extensions": { + "recommendations": [ + "charliermarsh.ruff", + "ms-python.mypy-type-checker", + "ms-python.python", + "ms-python.vscode-pylance", + "redhat.vscode-yaml" + ] + } } ``` diff --git a/docs/4. Validating/4.5. Formatting.md b/docs/4. Validating/4.5. Formatting.md index 140f6bde..f482a176 100644 --- a/docs/4. Validating/4.5. Formatting.md +++ b/docs/4. Validating/4.5. Formatting.md @@ -45,20 +45,20 @@ Below is an example configuration for your `[project].code-workspace` file to au ```json { - "settings": { - "editor.formatOnSave": true, - "[python]": { - "editor.codeActionsOnSave": { - "source.organizeImports": true - }, - "editor.defaultFormatter": "charliermarsh.ruff", - }, - }, - "extensions": { - "recommendations": [ - "charliermarsh.ruff", - ] - } + "settings": { + "editor.formatOnSave": true, + "[python]": { + "editor.codeActionsOnSave": { + "source.organizeImports": true + }, + "editor.defaultFormatter": "charliermarsh.ruff", + }, + }, + "extensions": { + "recommendations": [ + "charliermarsh.ruff", + ] + } } ``` diff --git a/docs/5. Refining/5.1. Task Automation.md b/docs/5. Refining/5.1. Task Automation.md index bb86c42b..86926e6b 100644 --- a/docs/5. Refining/5.1. Task Automation.md +++ b/docs/5. Refining/5.1. Task Automation.md @@ -38,12 +38,12 @@ package: package-build # build package constraints [group('package')] package-constraints constraints="constraints.txt": - uv pip compile pyproject.toml --generate-hashes --output-file={{constraints}} + uv pip compile pyproject.toml --generate-hashes --output-file={{constraints}} # build python package [group('package')] package-build constraints="constraints.txt": clean-build package-constraints - uv build --build-constraint={{constraints}} --require-hashes --wheel + uv build --build-constraint={{constraints}} --require-hashes --wheel ``` This example illustrates how tasks can be easily defined and automated using Python, making it accessible for those already familiar with the language. Developers can then execute the task from their terminal: @@ -172,6 +172,7 @@ just check-code check-format # run all the check tasks in the module just check ``` + ## Task automation additional resources - **[Task automation examples from the MLOps Python Package](https://github.com/fmind/mlops-python-package/tree/main/tasks)** diff --git a/docs/5. Refining/5.3. CI-CD Workflows.md b/docs/5. Refining/5.3. CI-CD Workflows.md index 6ecce3c9..74ed3c1a 100644 --- a/docs/5. Refining/5.3. CI-CD Workflows.md +++ b/docs/5. Refining/5.3. CI-CD Workflows.md @@ -67,17 +67,17 @@ Here is a breakdown of each attribute in the workflow: - **on**: Specifies the event that triggers the workflow, in this case, a pull request. - **concurrency**: Manages how workflow runs are handled concurrently. If `cancel-in-progress` is set to `true`, any in-progress runs of the workflow will be canceled when a new run is triggered. - **jobs**: Defines the jobs to be run as part of the workflow. - - **checks**: Identifies a job within the workflow, named "checks". - - **runs-on**: Specifies the type of virtual host machine to run the job on. - - **steps**: Lists the steps to be executed as part of the job. - - **- uses: actions/checkout@v4**: Utilizes the `checkout` action to access the repository code within the job. - - **- uses: ./.github/actions/setup**: Applies a custom action located in the repository to set up the environment. - - **- run: uv sync --group=check**: Executes the command to install dependencies specified under the "checks" group with uv. - - **- run: uv run just check-code**: Executes a task to check code quality. - - **- run: uv run just check-type**: Executes a task to check type annotations. - - **- run: uv run just check-format**: Executes a task to check code formatting. - - **- run: uv run just check-security**: Executes a task to check security vulnerabilities. - - **- run: uv run just check-coverage**: Execute a task to check test coverage. + - **checks**: Identifies a job within the workflow, named "checks". + - **runs-on**: Specifies the type of virtual host machine to run the job on. + - **steps**: Lists the steps to be executed as part of the job. + - **- uses: actions/checkout@v4**: Utilizes the `checkout` action to access the repository code within the job. + - **- uses: ./.github/actions/setup**: Applies a custom action located in the repository to set up the environment. + - **- run: uv sync --group=check**: Executes the command to install dependencies specified under the "checks" group with uv. + - **- run: uv run just check-code**: Executes a task to check code quality. + - **- run: uv run just check-type**: Executes a task to check type annotations. + - **- run: uv run just check-format**: Executes a task to check code formatting. + - **- run: uv run just check-security**: Executes a task to check security vulnerabilities. + - **- run: uv run just check-coverage**: Execute a task to check test coverage. ### [Publication Workflow](https://github.com/fmind/mlops-python-package/blob/main/.github/workflows/publish.yml) @@ -159,7 +159,7 @@ runs: This composite action can then be referenced in multiple workflows, ensuring a [DRY (Don't Repeat Yourself)](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) approach to CI/CD configuration. -You can also find more actions to use in your workflows from GitHub Marketplace: https://github.com/marketplace?type=actions +You can also find more actions to use in your workflows from GitHub Marketplace: ## What are some tips and tricks for using CI/CD workflows for MLOps? diff --git a/docs/5. Refining/5.5. AI-ML Experiments.md b/docs/5. Refining/5.5. AI-ML Experiments.md index 26987592..ac9fbeed 100644 --- a/docs/5. Refining/5.5. AI-ML Experiments.md +++ b/docs/5. Refining/5.5. AI-ML Experiments.md @@ -81,7 +81,7 @@ with mlflow.start_run( [MLflow's autologging](https://mlflow.org/docs/latest/tracking/autolog.html) capability simplifies the tracking of experiments by automatically recording several information. You can complement autologging by [manually logging additional information](https://mlflow.org/docs/latest/tracking/tracking-api.html#manual-logging): - Parameters with [`mlflow.log_param()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_param) for individual key-value pairs, or [`mlflow.log_params()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_params) for multiple parameters. -- Metrics using [`mlflow.log_metric()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_metric) for single key-value metrics, capturing the evolution of metrics over time, or []`mlflow.log_metrics()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_metrics) for multiple metrics. +- Metrics using [`mlflow.log_metric()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_metric) for single key-value metrics, capturing the evolution of metrics over time, or []`mlflow.log_metrics()`]() for multiple metrics. - Input datasets and context with [`mlflow.log_input()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_input), including tags for detailed categorization. - Tags for the active run through [`mlflow.set_tag()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_tag) for single tags or [`mlflow.set_tags()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_tags) for multiple tags. - Artifacts such as files or directories with [`mlflow.log_artifact()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_artifact) or [`mlflow.log_artifacts()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_artifacts) for logging multiple files. @@ -120,7 +120,6 @@ runs_df = mlflow.search_runs(experiment_ids) 3. **Visualize the Comparison**: For a more intuitive comparison, consider visualizing the results using libraries such as [Matplotlib](https://matplotlib.org/) or [Seaborn](https://seaborn.pydata.org/). For example, plotting the performance metrics of different runs can help in visually assessing which configurations performed better. - ```python import matplotlib.pyplot as plt @@ -144,6 +143,7 @@ To maximize the efficacy of AI/ML experiments: - Align logged information with relevant business metrics to ensure experiments are focused on meaningful outcomes. - Use [nested runs](https://mlflow.org/docs/latest/traditional-ml/hyperparameter-tuning-with-child-runs/part1-child-runs.html) to structure experiments hierarchically, facilitating organized exploration of parameter spaces. + ```python with mlflow.start_run() as parent_run: param = [0.01, 0.02, 0.03] @@ -155,11 +155,14 @@ with mlflow.start_run() as parent_run: ... mlflow.log_metric("val_loss", val_loss) ``` + - [Employ tagging extensively](https://mlflow.org/docs/latest/tracking/tracking-api.html#add-tags-to-runs) to enhance the searchability and categorization of experiments. - Track detailed progress by logging steps and timestamps, providing insights into the evolution of model performance. + ```python mlflow.log_metric(key="train_loss", value=train_loss, step=epoch, timestamp=now) ``` + - Regularly log models to the model registry for versioning and to facilitate deployment processes. ## AI-ML Experiment additional resources diff --git a/docs/5. Refining/archives/5.1. PyInvoke.md b/docs/5. Refining/archives/5.1. PyInvoke.md index 4e440372..c9e9da1c 100644 --- a/docs/5. Refining/archives/5.1. PyInvoke.md +++ b/docs/5. Refining/archives/5.1. PyInvoke.md @@ -183,6 +183,7 @@ inv checks.code checks.format # run all the check tasks in the module inv checks ``` + ## Task automation additional resources - **[Task automation examples from the MLOps Python Package](https://github.com/fmind/mlops-python-package/tree/main/tasks)** diff --git a/docs/6. Sharing/6.0. Repository.md b/docs/6. Sharing/6.0. Repository.md index 3049399c..e481fe5f 100644 --- a/docs/6. Sharing/6.0. Repository.md +++ b/docs/6. Sharing/6.0. Repository.md @@ -6,7 +6,7 @@ description: Learn how to set up a repository for your MLOps projects using plat ## What is a code repository? -[A code repository](https://en.wikipedia.org/wiki/Repository_(version_control)) serves as a centralized platform that supports collaboration in software development. It offers tools for version control, managing contribution guidelines, and establishing automation workflows. The key elements defining a code repository include the host platform, the owner (which can be an individual or an organization), and the repository name. For example, a project might be identified by a URL such as https://github.com/fmind/mlops-python-package where additional details on project specifics or documentation could be appended. +[A code repository](https://en.wikipedia.org/wiki/Repository_(version_control)) serves as a centralized platform that supports collaboration in software development. It offers tools for version control, managing contribution guidelines, and establishing automation workflows. The key elements defining a code repository include the host platform, the owner (which can be an individual or an organization), and the repository name. For example, a project might be identified by a URL such as where additional details on project specifics or documentation could be appended. Popular code repositories include [GitHub](https://github.com/), [GitLab](https://about.gitlab.com/), and [Bitbucket](https://bitbucket.org/product), each providing features tailored to different collaboration needs and complexities. [GitHub](https://github.com/) is renowned for its robust support for public repositories, while [GitLab](https://github.com/) and [BitBucket](https://bitbucket.org/product) are preferred by both private and public entities. Moreover, cloud providers like [Google Cloud Platform](https://cloud.google.com/source-repositories/docs), [Azure](https://azure.microsoft.com/en-us/products/devops/repos), and [AWS](https://aws.amazon.com/codecommit/) offer integrated code repository for public and private organizations. @@ -38,18 +38,25 @@ Maintain organizational consistency by following any existing naming conventions 1. **Modify your files or add new ones** within your project directory. 2. **Stage the changes** you want to include in your commit by running: + ```bash git add ``` + To add all changes in the directory, you can use: + ```bash git add . ``` + 3. **Check the status** to see what changes are staged for the next commit: + ```bash git status ``` + 4. **Commit the staged changes** by running: + ```bash git commit -m "Your commit message" ``` @@ -61,13 +68,17 @@ Here, replace "Your commit message" with a brief description of what changes wer [A git branch](https://git-scm.com/docs/git-branch) allows you to develop features, fix bugs, or safely experiment with new ideas in a contained area of your repository. 1. **Switch to the branch** from which you want to base your new branch (commonly the main branch): + ```bash git checkout main ``` + 2. **Create and switch to a new branch** by running: + ```bash git checkout -b ``` + Replace `` with a descriptive name for your branch, such as `feat/add-login`. ### Creating Tags @@ -75,15 +86,20 @@ Replace `` with a descriptive name for your branch, such as `feat/a [A git tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging) is used to mark specific points in repository history as important, typically for release versions. 1. **Check your commit history** to find the commit to which you want to attach a tag: + ```bash git log ``` + 2. **Create an annotated tag** on your current commit by running: + ```bash git tag -a -m "Your tag message" ``` + Replace `` with your version or release identifier, such as `v1.0.0`, and "Your tag message" with a description of what this tag represents. 3. **Push the tag to your remote repository**: + ```bash git push origin ``` diff --git a/docs/6. Sharing/6.1. License.md b/docs/6. Sharing/6.1. License.md index 0f82ec4d..beac3eab 100644 --- a/docs/6. Sharing/6.1. License.md +++ b/docs/6. Sharing/6.1. License.md @@ -46,7 +46,7 @@ Choosing the right software license involves careful consideration of your proje 1. **Create a LICENSE.txt file** in the root directory. 2. **Include the Full License Text**: Ensure the complete, unaltered text of the license is included to make it legally binding. -You can find open-source license text on this website: https://opensource.org/license. +You can find open-source license text on this website: . ## Should you choose a different license for AI/ML models? diff --git a/docs/6. Sharing/6.6. Contributions.md b/docs/6. Sharing/6.6. Contributions.md index d7af59a7..dbab89ad 100644 --- a/docs/6. Sharing/6.6. Contributions.md +++ b/docs/6. Sharing/6.6. Contributions.md @@ -50,7 +50,6 @@ Yes, automating parts of the code review process is possible and can significant For instance, you can enable code review with [Google Gemini Code Assist](https://developers.google.com/gemini-code-assist/docs/review-github-code). After you install the [Gemini Code Assist App](https://github.com/apps/gemini-code-assist) from GitHub marketplace, you can add a [`.gemini/config.yaml`](https://github.com/fmind/mlops-python-package/blob/main/.gemini/config.yaml) file at the root of your repository with the following content: - ```yaml # https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github diff --git a/docs/7. Observability/0. Reproducibility.md b/docs/7. Observability/0. Reproducibility.md index 3e590964..7a759c2e 100644 --- a/docs/7. Observability/0. Reproducibility.md +++ b/docs/7. Observability/0. Reproducibility.md @@ -94,12 +94,12 @@ package: package-build # build package constraints [group('package')] package-constraints constraints="constraints.txt": - uv pip compile pyproject.toml --generate-hashes --output-file={{constraints}} + uv pip compile pyproject.toml --generate-hashes --output-file={{constraints}} # build python package [group('package')] package-build constraints="constraints.txt": clean-build package-constraints - uv build --build-constraint={{constraints}} --require-hashes --wheel + uv build --build-constraint={{constraints}} --require-hashes --wheel ``` By using the `--build-constraint` and `--require-hashes` options, uv ensures that the build process is deterministic. The `--build-constraint` option specifies a constraints file that lists the exact versions of the dependencies to be used during the build, while `--require-hashes` ensures that all dependencies are downloaded with their respective hashes, preventing any variation in the downloaded packages. @@ -129,8 +129,8 @@ In this example: - `name` defines the project name as "bikes". - `python_env` specifies the path to the [python environment file](https://github.com/fmind/mlops-python-package/blob/main/python_env.yaml). - `entry_points` defines entry points, which specify how to run parts of the project. - - `main` is an entry point that accepts one parameters: `conf_file` as a file path. - - The `command` specifies how to execute the entry point, which in this case runs the `bikes` module with the provided parameters. + - `main` is an entry point that accepts one parameters: `conf_file` as a file path. + - The `command` specifies how to execute the entry point, which in this case runs the `bikes` module with the provided parameters. ### Executing an MLflow Project diff --git a/docs/7. Observability/1. Monitoring.md b/docs/7. Observability/1. Monitoring.md index 6f1c09b1..42af5696 100644 --- a/docs/7. Observability/1. Monitoring.md +++ b/docs/7. Observability/1. Monitoring.md @@ -98,6 +98,7 @@ Here's how you can implement this monitoring functionality: with evaluations_job as runner: runner.run() ``` + ![Monitoring Charts](../img/monitoring/charts.png) ## How to integrate AI/ML Monitoring to your data infrastructure? diff --git a/docs/7. Observability/5. Explainability.md b/docs/7. Observability/5. Explainability.md index b6ffd634..a95358ad 100644 --- a/docs/7. Observability/5. Explainability.md +++ b/docs/7. Observability/5. Explainability.md @@ -20,9 +20,9 @@ description: Explore the significance of explainability in AI/ML, learn how to u Explainability methods can be broadly categorized into two types based on their scope of interpretation: - **[Local Explainability](https://censius.ai/blogs/global-local-cohort-explainability#blogpost-toc-13)**: Focuses on understanding individual predictions, providing insights into the specific factors influencing a single output. This approach answers questions like "Why was this loan application rejected?" or "Why did the model predict this patient is at high risk?" - - **Example**: [SHAP (SHapley Additive exPlanations)](https://shap.readthedocs.io/en/latest/) is a widely used technique for local explainability. It assigns a contribution score to each feature for a given prediction, quantifying how each feature influences the outcome. + - **Example**: [SHAP (SHapley Additive exPlanations)](https://shap.readthedocs.io/en/latest/) is a widely used technique for local explainability. It assigns a contribution score to each feature for a given prediction, quantifying how each feature influences the outcome. - **[Global Explainability](https://censius.ai/blogs/global-local-cohort-explainability#blogpost-toc-11)**: Seeks to understand the overall behavior of the model, providing insights into the general relationship between input features and predictions across the entire dataset. This helps answer questions such as "Which features are most important for the model's predictions overall?" or "How does the model generally behave for different customer segments?" - - **Example**: Model feature importance scores, often provided by tree-based models or linear models, represent a form of global explainability. These scores indicate the relative influence of each feature on the model's predictions, averaged across the entire dataset. + - **Example**: Model feature importance scores, often provided by tree-based models or linear models, represent a form of global explainability. These scores indicate the relative influence of each feature on the model's predictions, averaged across the entire dataset. Choosing between local and global explainability depends on the specific goals and questions you are trying to answer. In some scenarios, you might require both types of explanations to gain a complete understanding of your model's behavior and its predictions. diff --git a/docs/index.md b/docs/index.md index 04207fa5..eca12370 100644 --- a/docs/index.md +++ b/docs/index.md @@ -44,6 +44,6 @@ The chapter focuses on sharing and distributing MLOps projects. We explore tools This chapter dives into the essential aspects of observability in MLOps, equipping you with the knowledge and strategies to gain comprehensive insights into the performance, behavior, and health of your deployed models and infrastructure. You'll learn how to ensure reproducibility, implement monitoring and alerting systems, track data and model lineage, manage costs and KPIs, understand model explainability, and monitor infrastructure performance. -## Let's journey together! +## Let's journey together We are excited to have you join us on this journey to mastering MLOps. By the end of this course, you will be well-equipped to manage and execute ML projects with a high degree of professionalism and skill. Let’s get started on transforming your data science capabilities with effective MLOps practices! From 53af3beb0118ef6f29308e74b8ef9c044c27a1df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?M=C3=A9d=C3=A9ric=20Hurier=20=28Fmind=29?= Date: Thu, 7 Aug 2025 21:46:49 +0200 Subject: [PATCH 2/2] review --- docs/1. Initializing/1.0. System.md | 18 ++++++++++++++++-- docs/1. Initializing/1.1. Python.md | 2 +- docs/1. Initializing/1.2. uv.md | 4 ++-- docs/1. Initializing/1.6. VS Code.md | 4 ++-- 4 files changed, 21 insertions(+), 7 deletions(-) diff --git a/docs/1. Initializing/1.0. System.md b/docs/1. Initializing/1.0. System.md index 6ab5a7e7..b3b6fcb5 100644 --- a/docs/1. Initializing/1.0. System.md +++ b/docs/1. Initializing/1.0. System.md @@ -7,7 +7,7 @@ description: > installing the essential software packages. --- -# 1.0. System +# βš™οΈ 1.0. System Before we dive into the exciting world of machine learning operations, it's crucial to ensure that your system is properly set up. A well-configured @@ -118,7 +118,21 @@ Nonetheless, it's crucial to understand any specific setup requirements and to manage resources effectively, especially when navigating the limitations of free tiers or usage quotas on these services. -## πŸ“š System additional resources +## πŸ”‘ Key Takeaways + +- **Operating System**: Linux, macOS, or Windows with WSL 2 are recommended. +- **VS Code**: The primary IDE for this course, offering a comprehensive + development environment beyond what JupyterLab or Google Colab provide. +- **Essential Software**: Python, `uv`, Git, and VS Code are required and will + be covered in detail. +- **Hardware**: Varies by project complexity; start local, scale to cloud as + needed. +- **System Setup Importance**: Crucial for reproducibility, scalability, + collaboration, and efficiency in MLOps. +- **Cloud Support**: Cloud-based environments like GitHub Codespaces and Google + Cloud Workstations are supported. + +## πŸ“š Additional Resources - **[GitHub Codespaces](https://github.com/features/codespaces):** A cloud-based development environment that allows you to code directly from your browser, diff --git a/docs/1. Initializing/1.1. Python.md b/docs/1. Initializing/1.1. Python.md index 070e44fc..b7b77286 100644 --- a/docs/1. Initializing/1.1. Python.md +++ b/docs/1. Initializing/1.1. Python.md @@ -144,7 +144,7 @@ modern MLOps workflows. understanding of the language and its ecosystem is essential for building robust and maintainable AI/ML systems. -## πŸ“š Python additional resources +## πŸ“š Additional Resources - [Python Website](https://www.python.org/) - [Planet Python](https://planetpython.org/) diff --git a/docs/1. Initializing/1.2. uv.md b/docs/1. Initializing/1.2. uv.md index 34a87b4f..bb66815f 100644 --- a/docs/1. Initializing/1.2. uv.md +++ b/docs/1. Initializing/1.2. uv.md @@ -217,7 +217,7 @@ not your system's default. uv python remove 3.13 ``` -## 🎯 Key Takeaways +## πŸ”‘ Key Takeaways - **`uv` is a fast, all-in-one tool** for Python package and project management. - It can replace `pip`, `venv`, `pipx`, and more, simplifying your workflow. @@ -225,7 +225,7 @@ not your system's default. **reproducible environments**. - Adopting `uv` can lead to a more efficient and streamlined development process. -## πŸ“š uv additional resources +## πŸ“š Additional Resources - **[uv Documentation](https://docs.astral.sh/uv/)**: The official documentation provides comprehensive information on all `uv` features and commands. diff --git a/docs/1. Initializing/1.6. VS Code.md b/docs/1. Initializing/1.6. VS Code.md index 1a60d5f6..11d2ddcf 100644 --- a/docs/1. Initializing/1.6. VS Code.md +++ b/docs/1. Initializing/1.6. VS Code.md @@ -317,7 +317,7 @@ use the Jupyter Extension seamlessly with the Python interpreter provided by uv. `uv add ` in your terminal or command prompt. This ensures that all dependencies are managed by uv and available in the notebook. -# πŸ”‘ Key Takeaways +## πŸ”‘ Key Takeaways - **VS Code is a powerful and customizable IDE** for MLOps, offering a rich set of features and extensions. @@ -330,7 +330,7 @@ use the Jupyter Extension seamlessly with the Python interpreter provided by uv. - **Ensure proper integration with `uv`** to manage Python environments and dependencies seamlessly within VS Code. -## πŸ”— VS Code additional resources +## πŸ“š Additional Resources - [Python in Visual Studio Code](https://code.visualstudio.com/docs/languages/python) - [Python Development in Visual Studio Code](https://realpython.com/python-development-visual-studio-code/)