ner-dspy

In this repo, you will find a small example of using DSPy for a data extraction / Named Entity Recognition (NER) Task. The task is to extract food-related entities from a recipe. One example recipe can be found in the ./data folder.

This workflow illustrates how to use DSPy to extract entities by employing the following prompt engineering techniques:

• Chain-of-Thought (CoT)
• Prompt-Chaining (PC)
• Function Calling using TypedPredictor from DSPy

The output of running the program is a list of FoodEntities:

{
  "entities": [
    {
      "food": "pork belly",
      "quantity": 2,
      "unit": "lb",
      "physical_quality": null,
      "color": ""
    },
    {
      "food": "green onions",
      "quantity": 2,
      "unit": "items",
      "physical_quality": "or 3 if small",
      "color": ""
    },
    ...
  ]

Setup ⚙️

0. Copy .env.example and add any env variables needed e.g. OPENAI_API_KEY
1. Make sure to install poetry on your system see e.g. instructions here
2. Install requirements

   poetry install --no-root

Run notebook 📓

To run the jupyter server use:

poetry run jupyter lab