From 409fa877afc8908fe598b1e6df16f22005b2c94f Mon Sep 17 00:00:00 2001
From: MRiganSUSX <M.Rigan@sussex.ac.uk>
Date: Wed, 25 Feb 2026 13:58:49 +0100
Subject: [PATCH 1/2] readme for ers

---
 ers-protobuf-dbwriter/README.md | 76 +++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)
 create mode 100644 ers-protobuf-dbwriter/README.md

diff --git a/ers-protobuf-dbwriter/README.md b/ers-protobuf-dbwriter/README.md
new file mode 100644
index 00000000..890ce9c7
--- /dev/null
+++ b/ers-protobuf-dbwriter/README.md
@@ -0,0 +1,76 @@
+# ERS Database Performance Optimization
+
+## 🚀 The Problem
+The ERS dashboard was slow because the "ERSstream" table lacked indexes. Every refresh triggered a **Sequential Scan**, reading every row on disk. This took ~236 seconds (4 minutes) for only 500k rows.
+
+## 🛠 The Fix
+Apply a **Composite B-Tree Index** on `(session, "time" DESC)`. This allows PostgreSQL to jump directly to specific session logs without a manual sort.
+
+### 1. Optimized Grafana Query
+Update your dashboard SQL to avoid mathematical operations on the "time" column itself.
+
+```sql
+SELECT 
+    severity, 
+    time/1000000 as time, 
+    application_name, 
+    message, 
+    issue_name, 
+    host_name
+FROM "ERSstream"
+WHERE session='${session}' 
+  AND time >= $__from * 1000000 
+  AND time <= $__to * 1000000
+ORDER BY time DESC
+LIMIT 500
+```
+
+### 2. Production Application (Safe)
+Run this inside the `ApplicationDbErrorReporting` database. The `CONCURRENTLY` keyword ensures the DAQ pipeline stays online while the index builds.
+
+```sql
+CREATE INDEX CONCURRENTLY idx_ersstream_session_time 
+ON "ERSstream" (session, "time" DESC);
+```
+
+## 📈 Results
+* **Before Index:** ~236,000 ms (4 minutes)
+* **After Index:** ~24 ms
+* **Improvement:** 10,000x Speedup
+
+---
+
+## 💡 Troubleshooting Tips & Tricks
+
+### How to access the Database Pod
+Use `kubectl` to jump into the running dbwriter or postgres pod:
+```bash
+# Exec into the dbwriter pod to use the bundled psql client
+kubectl exec -it <pod_name> -n ers -- psql $DATABASE_URI
+```
+
+### Useful PostgreSQL Commands
+Once inside the `psql` prompt:
+* **List all databases:** `\l`
+* **Connect to the ERS database:** `\c ApplicationDbErrorReporting`
+* **List all tables:** `\dt`
+* **Show table schema/indexes:** `\d "ERSstream"`
+
+### Testing Performance
+Use these to verify if the index is working:
+
+**Standard Query Example:**
+```sql
+SELECT * FROM "ERSstream" WHERE session = '<session>' LIMIT 10;
+```
+
+**Analyze Performance:**
+Adding `EXPLAIN ANALYZE` before any query shows you exactly how the database is finding the data (Look for "Index Scan" vs "Seq Scan").
+```sql
+EXPLAIN ANALYZE 
+SELECT * FROM "ERSstream" 
+WHERE session = '<session>' 
+ORDER BY time DESC 
+LIMIT 10;
+```
+

From f17896d4a073bc44dc1b563faeecc2eed3787049 Mon Sep 17 00:00:00 2001
From: MRiganSUSX <M.Rigan@sussex.ac.uk>
Date: Wed, 25 Feb 2026 15:17:54 +0100
Subject: [PATCH 2/2] improved readme

---
 ers-protobuf-dbwriter/README.md | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/ers-protobuf-dbwriter/README.md b/ers-protobuf-dbwriter/README.md
index 890ce9c7..ad3541d3 100644
--- a/ers-protobuf-dbwriter/README.md
+++ b/ers-protobuf-dbwriter/README.md
@@ -43,10 +43,10 @@ ON "ERSstream" (session, "time" DESC);
 ## 💡 Troubleshooting Tips & Tricks
 
 ### How to access the Database Pod
-Use `kubectl` to jump into the running dbwriter or postgres pod:
+Access the database using the environment variables already defined in the pod's container.
 ```bash
-# Exec into the dbwriter pod to use the bundled psql client
-kubectl exec -it <pod_name> -n ers -- psql $DATABASE_URI
+# Uses the pod's internal environment variable (injected via K8s secrets) to authenticate automatically
+kubectl exec -it <pod_name> -n ers -- sh -c 'psql $DATABASE_URI'
 ```
 
 ### Useful PostgreSQL Commands
@@ -56,6 +56,11 @@ Once inside the `psql` prompt:
 * **List all tables:** `\dt`
 * **Show table schema/indexes:** `\d "ERSstream"`
 
+### Safe Exits
+* **Exit query results (pager):** Press `q`
+* **Exit psql prompt:** Type `\q` or `Ctrl+D`
+* **Exit pod session:** Type `exit` or `Ctrl+D`
+
 ### Testing Performance
 Use these to verify if the index is working: