An analytics pipeline that detects underperforming test stations in manufacturing/hardware QA data and generates plain-language root-cause narration for the flagged cases. It combines statistical anomaly detection (z-score) with an LLM narration step: the model does not decide which stations are anomalous — that is a statistical determination — it only translates the flagged station's raw failure logs into a concise, human-readable explanation.
The project is delivered as a Jupyter notebook and is self-contained: it generates synthetic test-floor data, computes per-station KPIs with DuckDB, flags statistical outliers, narrates them with a hosted language model, and renders an executive dashboard.
- Background: the problem and the approach
- Pipeline overview
- Core concepts
- Repository contents
- Procedure: running the notebook
- How the LLM call works
- Troubleshooting
- Using other data
- Security and data-handling notes
- Design notes and limitations
- Requirements
- Possible extensions
- Further reading
A manufacturing or hardware QA floor runs many test stations, each producing pass/fail results and free-text failure logs. When a station begins failing units at an unusual rate, it needs to be identified quickly and its likely cause understood. Two distinct tasks are involved: detecting which stations are statistically abnormal, and explaining why — turning terse log lines into an account a supervisor can act on.
This pipeline separates those two tasks deliberately. A statistical method (z-score) makes the detection decision, because that is a numerical question with a rigorous answer. A language model then handles only the narration — converting a flagged station's raw logs into a short explanation and suggested action. This division matters: asking a language model to decide what is anomalous would reintroduce the risk of confident-but-wrong output, whereas asking it only to summarize evidence that has already been statistically established keeps its role grounded.
synthetic/test data
│
▼
[ DuckDB SQL ] ──► per-station KPIs (yield, fail rate)
│
▼
[ z-score ] ──► stations flagged as statistical outliers
│
▼
[ LLM narration ] ──► plain-language root cause + suggested action
│
▼
[ dashboard image ]
| Stage | Tool | Role |
|---|---|---|
| Data | notebook generator | Structured sensor readings plus free-text failure logs |
| KPIs | DuckDB | SQL aggregation of per-station yield and fail rate |
| Detection | z-score | Flags stations whose fail rate is a fleet-wide outlier |
| Narration | hosted LLM | Turns a flagged station's raw logs into a concise explanation |
| Reporting | matplotlib | Executive dashboard summarizing findings |
These concepts explain why each component was chosen; reading them makes the notebook straightforward to follow.
A z-score measures how many standard deviations a value sits from the mean:
z = (x − μ) / σ. A station whose fail rate is several standard deviations above
the fleet average is statistically unusual, rather than merely above an arbitrary
fixed threshold. This is the simplest rigorous form of anomaly detection and the
natural first tool to reach for before more complex methods (isolation forests,
clustering) are justified. Its key assumption is that fail rates are roughly
normally distributed across stations — worth stating explicitly, because it is the
condition under which the z-score is meaningful.
DuckDB is an in-process (serverless) OLAP — online analytical processing —
database, designed for running SQL aggregations over dataframe-sized datasets. It
is faster than a pandas groupby at scale while being as simple to start as
importing a library. It occupies a useful middle ground: heavier analytical queries
than pandas handles comfortably, without the overhead of a full data warehouse.
The narration task — turning a handful of short log lines into a two-to-three sentence explanation — is instruction-following rather than heavy reasoning. A mid-sized instruct model is therefore sufficient; using the largest available model would be wasteful. This reflects a general principle: match model size to task complexity rather than defaulting to the largest option.
An LLM's temperature controls how much randomness enters token selection. A low temperature (0.3 here) keeps the model close to patterns clearly supported by the input — appropriate for a factual-summary task, where an invented root cause would be worse than a plain one. High temperature suits creative generation; this task is the opposite.
The LLM call uses an OpenAI-compatible chat-completions endpoint
(POST /v1/chat/completions), the request/response shape popularized by OpenAI and
now offered by many providers. This compatibility is a de facto industry standard:
client code written against it is portable across providers with minimal change,
which is why the narration step is not locked to any single vendor.
Only the raw failure-log text of a flagged station is sent to the language model — not full sensor records or any identifying information. This is the principle of data minimization (least privilege) applied to a third-party API call: send the minimum needed for the task, and nothing more.
fleet-qa-copilot/
├── fleet_qa_copilot.ipynb # The pipeline notebook (data, KPIs, detection, narration, dashboard)
├── fleet_qa_dashboard.png # Output: executive dashboard image
├── requirements.txt # Python dependencies
├── README.md
└── .gitignore
A local .env file, used to hold the LLM API token, is excluded from version
control (section 9).
The notebook runs in a Python environment with Jupyter (including WSL on Windows).
cd fleet-qa-copilot
pip install -r requirements.txtUsing a virtual environment (python3 -m venv venv && source venv/bin/activate
before installing) is recommended to isolate dependencies.
The narration step calls a hosted LLM and requires an API token, supplied as an
environment variable. Create a .env file in the project directory:
echo 'HF_TOKEN=REPLACE_WITH_TOKEN' > .envThe notebook loads this automatically with load_dotenv(). The .env file is
excluded by .gitignore and must never be committed — it holds a live credential.
Launch Jupyter and open fleet_qa_copilot.ipynb:
jupyter notebook fleet_qa_copilot.ipynbRun the cells in order. The notebook generates synthetic test data, computes per-station KPIs with DuckDB, flags statistical outliers by z-score, sends each flagged station's logs to the LLM for narration, and renders the dashboard image. By default it uses synthetic data, so it runs end to end without any external data source; only the narration step requires the token and internet access.
The narration step sends a flagged station's failure logs to a hosted model through an OpenAI-compatible chat-completions endpoint and receives a short explanation in return. Two details are deliberate:
- Low temperature (0.3). The task is factual summarization, so the model is kept close to the evidence rather than allowed to embellish.
- Minimal payload. Only the raw failure-log text is sent (section 9).
If the narration step is unavailable (no token, or no network), the rest of the pipeline — data generation, KPI computation, and anomaly detection — still runs; only the plain-language narration is skipped.
These are real issues encountered while building and running the project, listed in roughly the order they are likely to occur.
ModuleNotFoundError for duckdb or another library. The dependencies are not
installed in the active environment. Run pip install -r requirements.txt, and
confirm the notebook kernel is using the environment where they were installed.
Token loads as empty even though .env contains it. load_dotenv() looks in
the current working directory by default. Ensure the .env file is in the same
directory the notebook runs from, and that the variable name matches what the
notebook reads.
Connection errors calling the LLM (Max retries exceeded, or 410 Gone). The
Hugging Face api-inference.huggingface.co endpoint was retired in 2026 in favour
of router.huggingface.co (an OpenAI-compatible Inference Providers API). Code
copied from older tutorials often references the retired domain and will fail with a
connection error or an explicit 410 Gone. This project already uses the current
endpoint; if a call still fails after confirming the URL, verify basic reachability
with curl -I https://router.huggingface.co in case a firewall or VPN blocks the
domain.
A specific model name returns 404 / "model not found". Model availability on hosted inference changes over time. Substitute a currently available instruct model of comparable size in the model-name field.
Rate-limit errors (HTTP 429). The free tier allows a few hundred requests per
hour, which can be exceeded when looping over many flagged stations or re-running
frequently. Add a short delay (for example time.sleep(1)) between calls, or reduce
how many stations are narrated per run.
Dashboard image looks empty or text is clipped. The dashboard layout assumes a small number of flagged stations. If more are flagged, adjust the text-spacing value in the dashboard cell, or truncate the per-station summary text.
The pipeline is built around a generic per-station schema (an identifier, pass/fail counts or a fail rate, and free-text failure logs). To run it on a real export rather than synthetic data, replace the data-generation cell with a load of the real dataset mapped to that schema; the KPI, detection, narration, and dashboard stages require no change. Because the detection stage is purely statistical, it adapts to any fleet whose station fail rates are roughly comparable in distribution.
- Credentials. The LLM API token is stored in a
.envfile that is excluded by.gitignoreand must never be committed. If a token is ever exposed, it should be revoked and reissued. - Data minimization. Only the raw failure-log text of a flagged station is sent to the external model — no full sensor records or identifying information — applying least-privilege to the third-party call.
- Secrets in general. Tokens should be kept out of version control and out of
command-line arguments (which can appear in process listings and logs);
environment variables and ignored
.envfiles are the standard mechanisms.
The dataset is synthetic by default, and the anomaly detection uses a z-score, which assumes fail rates are roughly normally distributed across stations; a fleet whose distribution is strongly skewed or multimodal would call for a different method (for example an isolation forest or a robust/quantile-based rule). The narration depends on a hosted model and network access; without them the analytical stages still run but the plain-language explanations are omitted. The project demonstrates a clean separation between statistical detection and language-based explanation rather than a production monitoring system, which would add persistent storage, scheduled runs, alerting, and evaluation of narration quality.
- Python 3.10+ with Jupyter (including WSL on Windows).
- Dependencies in
requirements.txt(including duckdb, pandas, matplotlib, requests, python-dotenv). - For the narration step: an API token (
HF_TOKEN) and internet access to the inference endpoint.
- Add a second detection method (for example isolation forest) and compare its flagged set against the z-score's, to test sensitivity to the normality assumption.
- Persist results across runs and add scheduling so the pipeline can run unattended.
- Add an evaluation step that checks narration outputs against known root causes on a labelled subset.
- Parameterize the z-score threshold and expose it as a configurable operating point.
- Replace the static dashboard image with an interactive dashboard.
- Z-score / standard score: any introductory statistics text, for example OpenIntro Statistics (free online), covers this in its early chapters.
- DuckDB:
duckdb.org/docs— readable official documentation. - Hosted inference APIs:
huggingface.co/docs/inference-providers— reference for the router API used here. - LLM temperature and sampling: provider documentation (for example OpenAI's or Cohere's) gives the clearest practical explanations, as it is an engineering concept more than a research one.
Released under the MIT License. See LICENSE for details.