A RAG pipeline built on Azure OpenAI and Azure AI Search with no LangChain, no LlamaIndex, and no framework in between. Every step is written out: parse a PDF, split it into overlapping chunks tagged with their section, embed them, index them for hybrid search, retrieve, and answer from the retrieved text only.
Answers cite the handbook section they came from, so you can check them.
The test corpus is a fake employee handbook for a fictional company called Contoso. A script in this repo generates it, so there is no real document or personal data anywhere in the project.
Ingest, run once:
documents/*.pdf
parse_pdf() pypdf pulls out the raw text
split_sections() finds the headings again, 27 sections
chunk_sections() sliding window inside each section, 63 chunks
embed_batch() text-embedding-3-large, 3072 dimensions
into Azure AI Search: content + section + vector
Ask, every question:
embed() question becomes a vector
search() vector kNN and BM25 together, top 5
answer() numbered chunks plus the question, to gpt-5-mini
out: an answer with [1] citations
You need Python 3.9+, an Azure OpenAI resource, and an Azure AI Search service.
git clone https://github.com/immohamedadhil/rag-from-scratch.git
cd rag-from-scratch
pip install -r requirements.txt
cp .env.example .envDeploy two models in Azure AI Foundry: an embedding model (text-embedding-3-large) and a chat model (gpt-5-mini, though any chat deployment works). Put both endpoints, both keys, and your deployment names into .env.
create_index.py needs a Search admin key. The other scripts are fine with a query key.
python ingest.py # parse, chunk, embed
python create_index.py # build the index, upload the chunks
python chatbot.py # ask questionsThe first two only need running again when the documents change.
search.py is a second REPL that prints the retrieved chunks and their scores without generating anything. When an answer is wrong it is the quickest way to find out whether retrieval or generation caused it.
Embedding the sample handbook costs well under a cent.
Building the index:
$ python ingest.py
contoso_employee_handbook.pdf: 27 sections -> 63 chunks
embedding 63 chunks
wrote 63 records (3072 dimensions) to data/chunks_and_vectors.json
$ python create_index.py
dropped existing index 'handbook-index'
created index 'handbook-index'
uploaded 63 / 63 documents
Asking questions:
$ python chatbot.py
Question (or 'quit'): How many vacation days do I get, and does that change over time?
You get 18 vacation days per year for your first two years, then this increases to
24 days per year after two years. Vacation days accrue monthly. [1]
Sources:
[1] 3. Paid Time Off and Leave (contoso_employee_handbook.pdf)
Question (or 'quit'): I lost my work laptop. What am I supposed to do?
Report it as a security incident to the Security team at security@contoso.example
within one hour of discovery. [1][2]
Do not report lost or stolen devices to the IT service desk. [2]
Reporting a suspected incident that turns out to be harmless carries no penalty;
failing to report one does. [1]
Sources:
[1] 6. IT Security and Acceptable Use (contoso_employee_handbook.pdf)
[2] 13. Equipment and Facilities (contoso_employee_handbook.pdf)
Question (or 'quit'): What is the CEO's personal phone number?
I don't have the CEO's personal phone number in the provided context.
The last one is the case worth watching. That answer is not in the handbook, so the model says so rather than inventing a number, and nothing is cited because nothing was used.
The laptop question shows why the chunking works the way it does. The rule is in section 6 and the exception ("not the service desk") is in section 13, so a correct answer needs both, and the citations show which part came from where.
search.py prints what the model was given, without generating anything:
$ python search.py
Question (or 'quit'): how many vacation days do I get?
[1] score=0.0333 3. Paid Time Off and Leave (contoso_employee_handbook.pdf)
New employees accrue 18 days of paid vacation per year during their first two years
of service. After two years, this increases to 24 days per year...
[2] score=0.0325 3. Paid Time Off and Leave (contoso_employee_handbook.pdf)
advance where possible. Public holidays follow the calendar of the employee's
registered country and are additional to the vacation entitlement...
Those scores look small because hybrid results are ranked by reciprocal rank fusion rather than cosine similarity. Only the ordering carries meaning.
The vector half matches on meaning, so asking about "holidays" finds a chunk that only says "vacation". The keyword half matches literal strings, so "401k" finds "401k". Embeddings are weak at that second case, because a rare token carries little meaning of its own and gets washed out. Azure runs both halves and merges the rankings itself. Either one on its own is worse.
The sliding window runs within each section, not across the whole document. That wastes a little space at section ends, and in exchange every chunk belongs to exactly one section. That is what makes the citation possible. If chunks span sections you have to guess which section a chunk mostly belongs to, and the citation stops being reliable.
A chunk that reads ...accrue 18 days of paid vacation... is much easier to find as 3. Paid Time Off and Leave - ...accrue 18 days.... Only the embedding input gets the prefix. The stored text stays clean, and the heading is also its own searchable field so keyword search can hit it.
Overlap is there so an idea that falls across a boundary still shows up whole in one chunk. Without it, half a sentence loses to a chunk that has the entire thing, and the context you get back reads like a line went missing.
When the window lands just past the end of a section it leaves a stub of a few words behind. At search time a stub looks like any other chunk, so it can take one of the five slots and contribute nothing. Anything shorter than the overlap gets folded into the chunk before it.
HNSW is approximate nearest neighbour. It trades a little recall for a lot of speed, which is worth it once the corpus is past the size you would happily scan linearly. OpenAI embeddings come back normalised, so cosine and dot product give the same ranking. Cosine is the default and there was no reason to change it.
Chunk ids are just positions in a list. Re-ingesting an edited document gives a different number of chunks, and any old id without a new counterpart stays in the index. Those leftovers are still searchable and still quote text that no longer exists. Dropping the index first is the only way to know exactly what is in it.
This began as a workaround for SDK trouble on the Python version I was using, and stayed because the call is a single POST with a JSON body. Batching and the 429 retry are easier to see written out than buried in a client object.
The model is told to cite the numbered blocks it used, and the source list is filtered down to those numbers. Printing all five retrieved chunks would suggest the answer used all five.
config.py endpoints, deployments, embed and chat calls, clients
chunking.py PDF to sections to chunks, no network calls
ingest.py runs chunking and embedding, writes data/
create_index.py index schema and upload
search.py hybrid retrieval and a REPL to inspect it
chatbot.py answers with citations
tools/make_handbook.py builds the sample PDF, needs reportlab
documents/ input PDFs
docs/ architecture diagram
MIT, see LICENSE.