CleanNews

This repository represents the implementation of CleanNews - an architecture for real-time fake news detection and mitigation. It combines DeBERTa embeddings, graph embeddings, and immunization strategies to detect and reduce the spread of misinformation on social media datasets (Twitter15 & Twitter16).

---

Packages

The project was tested in Google Colab with Python 3.10.9 and the following packages:

• NetworkX 2.8.8
• node2vec 0.4.6
• numpy 1.25.2
• nltk 3.8.1
• demoji 1.1.0
• torch 2.8.0
• transformers 4.55.4
• sklearn 1.6.1
• seaborn 0.13.2
• matplotlib 3.10.0
• Gensim > 4.1.2
• pandas 2.2.2
• contractions 0.1.73
• pyspellchecker 0.8.1
• Flask 3.0.3
• scipy 1.11.3
• tqdm 4.66.4

---

Repository Structure

CleanNews/
│── Dataset/                # CSVs, vectors, and GraphML files
│── Embeddings/             # Graph embeddings for each dataset
│── Pretrained_Models/      # Pretrained models (Twitter16)
│── templates/              # Flask HTML templates
│   ├── index.html          # Input text UI
│   └── network_immunization.html  # Graph immunization UI
│── file.py                 # Flask app for testing models
│── *.ipynb                 # Jupyter notebooks for preprocessing, embeddings, and modeling
│── README.md

Utilization

1. Get the Datasets

Download the Twitter15 and Twitter16 datasets from the Rumor_RvNN repository.

---

2. Preprocess the Datasets

• Run Create_dataset.ipynb using the datasets above.
• Output: Three CSV files per dataset:
  • tree_data.csv
  • labels.csv
  • source_data.csv

---

3. Generate DeBERTa Embeddings

• Use Deberta_embeddings.ipynb for raw text.
• Use Deberta_embeddings_clean.ipynb for soft and aggressive cleaning.
• Output: embs.npy file per dataset.

---

4. Generate Graph Embeddings

• Run Graph_embeddings.ipynb.
• Output: NetworkX .graphml file per dataset.

---

5. Train & Test Models

• Use Deberta_16.ipynb or Deberta_15.ipynb.
• Each notebook includes:
  • 8 model architectures
  • Grid search for hyperparameter tuning
  • Graph immunization with Random Solver, Sparse Shield, or NetShield

---

6. Test with GloVe Embeddings (Twitter15 only)

• Run GloVe_15.ipynb.
• Requires GloVe embeddings.

---

Flask Application

A minimal Flask app (file.py) is included for testing models on raw text input and graph immunization.

Run the app with:

python file.py

Then open http://127.0.0.1:5000 in your browser.

• index.html → Submit raw text for classification
• network_immunization.html → Apply immunization on graph structures

---

Datasets

• Twitter15 & Twitter16
• GloVe Embeddings