This repository contains the official dataset and baseline model implementations for our paper, "KRISHI: An Annotated Dataset for Agricultural Text Classification in Malayalam".
Our work aims to address the scarcity of NLP resources for the agricultural domain in low-resource Indian languages, starting with Malayalam.
The KRISHI dataset is a collection of agricultural texts in Malayalam, expertly annotated into five distinct categories. It is designed to serve as a benchmark for developing and evaluating NLP models for agricultural applications in the region.
The five categories are:
AGRI_PRACTICESCROPSDISEASESENVIRONMENTAL_FACTORSLIVESTOCK
The dataset is a living resource and will be updated periodically with new data. You can access the latest version via the Google Sheet linked below.
➡️ Access the KRISHI Dataset (Google Sheet)
We established baseline performance on the KRISHI dataset using three standard machine learning algorithms. Our results highlight that while class imbalance poses a challenge for probabilistic models, margin-based approaches demonstrated significant robustness.
| Model | Accuracy | Weighted F1-Score |
|---|---|---|
| SVM (LinearSVC) | 69% | 0.68 |
| Logistic Regression | 55% | 0.47 |
| Naive Bayes | 45% | 0.30 |
Key Insights:
- LinearSVC achieved the strongest performance, effectively handling minority classes like
DISEASES(F1: 0.91) andLIVESTOCK(F1: 0.80) where other models struggled. - Logistic Regression and Naive Bayes showed a strong bias towards the majority
CROPSclass, resulting in significantly lower performance on underrepresented categories.