diff --git a/.spellcheck-en-custom.txt b/.spellcheck-en-custom.txt index 7904cac..63cf43e 100644 --- a/.spellcheck-en-custom.txt +++ b/.spellcheck-en-custom.txt @@ -61,6 +61,8 @@ Mellea's mify Milvus Moderna +multimodal +Multimodal ollama Ollama OPENAI @@ -118,5 +120,6 @@ verifiers virtualenv workspaces WSL +Vectorization venv Zhao diff --git a/docs/opentech/docling/images/DoclingDuck.png b/docs/opentech/docling/images/DoclingDuck.png new file mode 100644 index 0000000..8763d71 Binary files /dev/null and b/docs/opentech/docling/images/DoclingDuck.png differ diff --git a/docs/opentech/docling/lab-1/README.md b/docs/opentech/docling/lab-1/README.md new file mode 100644 index 0000000..58259b9 --- /dev/null +++ b/docs/opentech/docling/lab-1/README.md @@ -0,0 +1,34 @@ +--- +title: Docling Workshop Lab 1 +description: Document Conversion with Docling +logo: images/DoclingDuck.png +--- + +# Document Conversion with Docling + +The primary purpose of Docling is document conversion. Docling enables us to convert documents various format into formats that are more useful in AI applications, while preserving document structure. + +This lab walks through the different document conversion options Docling offers, as well as some enrichment features. We will also explore the converted documents to examine how Docling stores metadata to preserve document structure. + +## Prerequisites + +This lab is a [Jupyter notebook](https://jupyter.org/). Please follow the instructions in [prework](../prework/README.md) to run the lab. + +## Lab + +The path of the notebooks directory is relative to the `opentech/docling` folder from the git clone in the [prework](../prework/README.md). + +1. Run the following commands to create `doclingkernel` which has the dependencies from our `pyproject.toml` venv and launch Jupyter Lab. + + ```shell + uv run --directory docling ipython kernel install --user --env VIRTUAL_ENV .venv --name=doclingkernel + uv run --directory docling jupyter lab + ``` + +1. In Jupyter Lab in your browser, walk through the notebook: + + 1. Jupyter Lab will open in your browser + 1. Navigate to the `notebooks` folder + 1. Open `Conversion.ipynb` + 1. Use the play button to walk through the notebook + 1. Be sure to read the text, the code, and the output diff --git a/docs/opentech/docling/lab-2/README.md b/docs/opentech/docling/lab-2/README.md new file mode 100644 index 0000000..1ab0f44 --- /dev/null +++ b/docs/opentech/docling/lab-2/README.md @@ -0,0 +1,33 @@ +--- +title: Docling Workshop Lab 2 +description: Chunking and Vectorization with Docling +logo: images/DoclingDuck.png +--- + +# Chunking with Docling + +[Chunking](https://www.ibm.com/architectures/papers/rag-cookbook/chunking) is the process of splitting large texts into smaller, manageable segments before feeding them into a model. This is an important step because models have a maximum context length, and chunking ensures that relevant information fits within this limit while preserving coherence, improving retrieval accuracy, and avoiding loss of important content during processing. + +In this lab we will explore the importance of chunking and the capabilities Docling has to create more valuable chunks. + +## Prerequisites + +This lab is a [Jupyter notebook](https://jupyter.org/). Please follow the instructions in [prework](../prework/README.md) to run the lab. + +## Lab + +The path of the notebooks directory is relative to the `opentech/docling` folder from the git clone in the [prework](../prework/README.md). + +1. (SKIP IF STILL RUNNING FROM LAB 1) Run the following commands to create `doclingkernel` which has the dependencies from our `pyproject.toml` venv and launch Jupyter Lab. + + ```shell + uv run --directory docling ipython kernel install --user --env VIRTUAL_ENV .venv --name=doclingkernel + uv run --directory docling jupyter lab + ``` + +1. In Jupyter Lab in your browser, walk through the notebook: + + 1. Navigate to the `notebooks` folder + 1. Open `Chunking.ipynb` + 1. Use the play button to walk through the notebook + 1. Be sure to read the text, the code, and the output diff --git a/docs/opentech/docling/lab-3/README.md b/docs/opentech/docling/lab-3/README.md new file mode 100644 index 0000000..0dba1aa --- /dev/null +++ b/docs/opentech/docling/lab-3/README.md @@ -0,0 +1,33 @@ +--- +title: Docling Workshop Lab 3 +description: Multimodal RAG with Docling +logo: images/DoclingDuck.png +--- + +# Multimodal RAG with Docling + +[Retrieval Augmented Generation (RAG)](https://research.ibm.com/blog/retrieval-augmented-generation-RAG) is an architectural pattern that can be used to augment the performance of language models by recalling factual information from a knowledge base, and adding that information to the model query. + +In this lab we will combine the skills we learned in the two previous labs to build a Docling-enhanced RAG system. + +## Prerequisites + +This lab is a [Jupyter notebook](https://jupyter.org/). Please follow the instructions in [prework](../prework/README.md) to run the lab. + +## Lab + +The path of the notebooks directory is relative to the `opentech/docling` folder from the git clone in the [prework](../prework/README.md). + +1. (SKIP IF STILL RUNNING FROM LAB 1) Run the following commands to create `doclingkernel` which has the dependencies from our `pyproject.toml` venv and launch Jupyter Lab. + + ```shell + uv run --directory docling ipython kernel install --user --env VIRTUAL_ENV .venv --name=doclingkernel + uv run --directory docling jupyter lab + ``` + +1. In Jupyter Lab in your browser, walk through the notebook: + + 1. Navigate to the `notebooks` folder + 1. Open `RAG.ipynb` + 1. Use the play button to walk through the notebook + 1. Be sure to read the text, the code, and the output diff --git a/docs/opentech/docling/overview/README.md b/docs/opentech/docling/overview/README.md index e69de29..d67fadc 100644 --- a/docs/opentech/docling/overview/README.md +++ b/docs/opentech/docling/overview/README.md @@ -0,0 +1,38 @@ +--- +title: BeeAI OpenTech Docling Workshop +description: Learn how to process documents with Docling +logo: images/DoclingDuck.png +--- + +# Introduction + +In this workshop we'll be walking through the features of [Docling](https://docling-project.github.io/docling/) + +By the end of this workshop, you will be able to use Docling to: + +* Convert complex documents into various formats suitable for AI +* Understand the various methods of [Chunking](https://www.ibm.com/architectures/papers/rag-cookbook/chunking) and how to use them +* Create a transparent and verifiable multimodal [RAG](https://research.ibm.com/blog/retrieval-augmented-generation-RAG) application + +## Agenda + +| | | +|:---------------------------------------------------------------------|:------------------------------------------------------| +| [Lab 0: Prework](../prework/README.md) | Review the prework for the workshop | +| [Lab 1: Document Conversion with Docling](../lab-1/README.md) | Learn how to use Docling to convert complex documents | +| [Lab 2: Chunking and Vectorization with Docling](../lab-2/README.md) | Learn how Docling enables advanced chunking | +| [Lab 3: Multimodal RAG with Docling](../lab-3/README.md) | Build a verifiable multimodal RAG system | + +## Technology Used + +The technology used in the workshop is as follows: + +* [Docling](https://docling-project.github.io/docling/) +* [Jupyter notebooks](https://jupyter.org/) +* [LangChain](https://www.langchain.com/) +* [Ollama](https://ollama.com/) +* [IBM Granite AI foundation models](https://www.ibm.com/granite) + +## Acknowledgments + +The Docling workshop heavily leverages content from the IBM Granite Community Docling Workshop [here](https://ibm-granite-community.github.io/docling-workshop/). diff --git a/docs/opentech/docling/prework/README.md b/docs/opentech/docling/prework/README.md new file mode 100644 index 0000000..24c9b5f --- /dev/null +++ b/docs/opentech/docling/prework/README.md @@ -0,0 +1,15 @@ +--- +title: Docling Workshop Prework +description: Preparation for the Docling Workshop +logo: images/DoclingDuck.png +--- + +# Prework + +Please do the [prework](../../prework/README.md) for the entire workshop. + + diff --git a/docs/opentech/docling/setup/README.md b/docs/opentech/docling/setup/README.md index 406cf62..45b8659 100644 --- a/docs/opentech/docling/setup/README.md +++ b/docs/opentech/docling/setup/README.md @@ -1,9 +1,14 @@ # Docling notebook setup -```shell -uv run --directory docling ipython kernel install --user --env VIRTUAL_ENV .venv --name=doclingproject -``` +Launch Jupyter with `doclingkernel` which has the dependencies from our `pyproject.toml` venv. ```shell -uv run --directory docling --with jupyter jupyter lab +uv run --directory docling ipython kernel install --user --env VIRTUAL_ENV .venv --name=doclingkernel +uv run --directory docling jupyter lab ``` + +In Jupyter Lab: + +* Double-click the notebooks folder +* Double-click the Conversion.ipynb +* Use the play button to walk through the notebook