Skip to content

[Web] Fix report session truncation #304

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bitflicker64 merged 3 commits into from
Jul 20, 2026
Merged

[Web] Fix report session truncation #304

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a0567d5
[Web] Fix report session truncation
floze-the-genius Jul 18, 2026
1e3811a
test(web): share report session limit
floze-the-genius Jul 18, 2026
90dea0b
fix(web): surface truncated session reports
floze-the-genius Jul 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 40 additions & 7 deletions termstory/web.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@

logger = logging.getLogger(__name__)

WEB_REPORT_SESSION_LIMIT = 500
AI_HIGHLIGHT_LIMIT = 15


def get_web_data(db: Database, start_ts: Optional[int] = None, end_ts: Optional[int] = None) -> dict:
"""Gather stats, project list, timeline, and AI highlights from the database, filtering by date range if provided."""
Expand Down Expand Up @@ -58,10 +61,9 @@ def get_web_data(db: Database, start_ts: Optional[int] = None, end_ts: Optional[

if conditions:
query += " WHERE " + " AND ".join(conditions)
query += " ORDER BY start_time DESC LIMIT 500"
else:
query += " ORDER BY start_time DESC LIMIT 500"

query += " ORDER BY start_time DESC LIMIT ?"
params.append(WEB_REPORT_SESSION_LIMIT)

cursor.execute(query, params)
session_ids = [row[0] for row in cursor.fetchall()]
finally:
Expand Down Expand Up @@ -152,7 +154,7 @@ def get_web_data(db: Database, start_ts: Optional[int] = None, end_ts: Optional[

# 4. AI Summary Highlights
ai_sessions = [s for s in sessions_data if s["ai_summary"]]
if len(ai_sessions) < 15:
if len(ai_sessions) < AI_HIGHLIGHT_LIMIT:
# Fetch more from DB if we don't have enough in the filtered set
conn = db.get_connection()
try:
Expand All @@ -170,7 +172,8 @@ def get_web_data(db: Database, start_ts: Optional[int] = None, end_ts: Optional[
params_ai.append(end_ts)
if conditions_ai:
query_ai += " AND " + " AND ".join(conditions_ai)
query_ai += " ORDER BY start_time DESC LIMIT 15"
query_ai += " ORDER BY start_time DESC LIMIT ?"
params_ai.append(AI_HIGHLIGHT_LIMIT)

cursor.execute(query_ai, params_ai)
extra_ids = [row[0] for row in cursor.fetchall()]
Expand Down Expand Up @@ -200,7 +203,7 @@ def get_web_data(db: Database, start_ts: Optional[int] = None, end_ts: Optional[
})

ai_sessions.sort(key=lambda x: x["start_time"], reverse=True)
highlights_data = ai_sessions[:15]
highlights_data = ai_sessions[:AI_HIGHLIGHT_LIMIT]

# 5. Calculate daily activity for the last 90 days for the heatmap
now_dt = datetime.now()
Expand Down Expand Up @@ -242,10 +245,19 @@ def get_web_data(db: Database, start_ts: Optional[int] = None, end_ts: Optional[
finally:
conn.close()

total_sessions = stats["total_sessions"]
session_window = {
"limit": WEB_REPORT_SESSION_LIMIT,
"returned": len(sessions_data),
"total": total_sessions,
"truncated": total_sessions > len(sessions_data),
}
Comment thread
greptile-apps[bot] marked this conversation as resolved.

@bitflicker64 bitflicker64 Jul 18, 2026

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good metadata shape. Confirm the embedded report template actually reads session_window (banner like “Showing 1000 of 4500 sessions”). If not, either wire it in this PR or open a follow-up so this field isn’t unused JSON.

@floze-the-genius floze-the-genius Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in df859cc. The embedded report now renders an aria-live status banner when session_window.truncated is true (for example, “Showing 1000 of 4500 sessions”), and tests/test_web.py asserts both the banner wiring and truncated metadata. Focused web suite passes locally.

Comment on lines +248 to +254

@greptile-apps greptile-apps Bot Jul 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 truncated and total are wrong when a date filter is active

stats["total_sessions"] comes from analyze_all(db) on line 23, which counts every session in the database with no date filter. When start_ts/end_ts are supplied and the matching window holds fewer than WEB_REPORT_SESSION_LIMIT sessions, total_sessions will be larger than len(sessions_data) even though no truncation occurred — so truncated is True and total shows the full-DB count instead of the in-range count. Concretely: 600 sessions in DB, 50 match the date filter → UI shows "Showing 50 of 600 sessions" even though all 50 were returned. A separate COUNT(*) query that applies the same WHERE conditions as the session fetch is needed to populate total accurately before computing truncated.


return {
"stats": stats,
"projects": projects_data,
"sessions": sessions_data,
"session_window": session_window,
"highlights": highlights_data,
"daily_activity": daily_activity
}
Expand Down Expand Up @@ -468,6 +480,18 @@ def generate_and_open_report(
color: var(--text-secondary);
font-size: 0.9rem;
}}
.session-window-notice {{
display: none;
margin: -16px 0 28px;
padding: 12px 16px;
border-left: 3px solid var(--accent-color);
background: var(--panel-bg);
color: var(--text-secondary);
font-size: 0.9rem;
}}
.session-window-notice.visible {{
display: block;
}}

/* Glassmorphism panel */
.panel {{
Expand Down Expand Up @@ -839,6 +863,8 @@ def generate_and_open_report(
<div style="margin-top: 2px;">Last Ingested: <span id="last-ingested-date">-</span></div>
</div>
</header>

<div id="session-window-notice" class="session-window-notice" role="status" aria-live="polite"></div>

<div class="kpi-grid">
<div class="panel kpi-card">
Expand Down Expand Up @@ -1167,6 +1193,13 @@ def generate_and_open_report(
document.getElementById('kpi-commands').textContent = reportData.stats.total_commands;
document.getElementById('kpi-projects').textContent = reportData.stats.total_projects;
document.getElementById('kpi-streak').textContent = `${{reportData.stats.streak}} Days`;

const sessionWindow = reportData.session_window;
if (sessionWindow?.truncated) {{
const notice = document.getElementById('session-window-notice');
notice.textContent = `Showing ${{sessionWindow.returned}} of ${{sessionWindow.total}} sessions.`;
notice.classList.add('visible');
}}

const searchInput = document.getElementById('search-bar');
searchInput.addEventListener('input', (e) => {{
Expand Down
73 changes: 63 additions & 10 deletions tests/test_web.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from termstory.cli import app
from termstory.database import Database
from termstory.models import Project, Session, Command
from termstory.web import get_web_data, generate_and_open_report
from termstory.web import WEB_REPORT_SESSION_LIMIT, get_web_data, generate_and_open_report

def test_get_web_data_empty_db(tmp_path):
db_file = tmp_path / "empty.db"
Expand All @@ -28,6 +28,12 @@ def test_get_web_data_empty_db(tmp_path):
assert data["stats"]["streak"] == 0
assert len(data["projects"]) == 0
assert len(data["sessions"]) == 0
assert data["session_window"] == {
"limit": WEB_REPORT_SESSION_LIMIT,
"returned": 0,
"total": 0,
"truncated": False,
}
assert len(data["highlights"]) == 0


Expand Down Expand Up @@ -152,6 +158,44 @@ def test_get_web_data_populated_db(tmp_path):
assert data["highlights"][0]["project_name"] == "Project A"
assert data["highlights"][0]["ai_summary"] == "Started project A"


def test_get_web_data_does_not_truncate_unfiltered_sessions_at_30(tmp_path):
db = Database(str(tmp_path / "sessions.db"))
db.init_db()

now = int(datetime(2026, 6, 14, 12, 0, 0).timestamp())
project = Project(
id=1,
name="Project A",
path="/path/to/a",
first_seen=now,
last_seen=now,
session_count=31,
total_time=31,
)
sessions = [
Session(
id=index + 1,
start_time=now + index,
end_time=now + index + 1,
duration_seconds=1,
project_id=1,
)
for index in range(31)
]
db.save_data([project], sessions, [])

data = get_web_data(db)

assert len(data["sessions"]) == 31
assert data["session_window"] == {
"limit": WEB_REPORT_SESSION_LIMIT,
"returned": 31,
"total": 31,
"truncated": False,
}


def test_generate_and_open_report(tmp_path, monkeypatch):
db_file = tmp_path / "report.db"
db = Database(str(db_file))
Expand Down Expand Up @@ -183,6 +227,9 @@ def test_generate_and_open_report(tmp_path, monkeypatch):

assert "<title>TermStory Web Report</title>" in html
assert "const reportData = " in html
assert 'id="session-window-notice"' in html
assert "if (sessionWindow?.truncated)" in html
assert "Showing ${sessionWindow.returned} of ${sessionWindow.total} sessions." in html

def test_cli_web_subcommand(tmp_path, monkeypatch):
db_file = tmp_path / "cli_web.db"
Expand Down Expand Up @@ -268,13 +315,13 @@ def test_swarm_audit_fixes(tmp_path, monkeypatch):
db = Database(str(db_file))
db.init_db()

# We will insert more than 1000 sessions (e.g. 1005 sessions) to check uncapped override
session_count = WEB_REPORT_SESSION_LIMIT + 5
now = int(datetime(2026, 6, 14, 12, 0, 0).timestamp())

projects = [Project(id=1, name="Project A", path="/path/to/a", first_seen=now, last_seen=now, session_count=1005, total_time=10050)]
projects = [Project(id=1, name="Project A", path="/path/to/a", first_seen=now, last_seen=now, session_count=session_count, total_time=session_count * 10)]
sessions = []
commands = []
for i in range(1005):
for i in range(session_count):
s_id = i + 1
s_time = now - i * 10 # spread out in time
sessions.append(Session(id=s_id, start_time=s_time, end_time=s_time + 5, duration_seconds=5, project_id=1))
Expand All @@ -288,17 +335,24 @@ def test_swarm_audit_fixes(tmp_path, monkeypatch):
start_ts = now - 20000
data = get_web_data(db, start_ts=start_ts)

# Verify KPI stats override does NOT cap at 1000
assert data["stats"]["total_sessions"] == 1005
assert data["stats"]["total_commands"] == 1005
# Verify KPI stats override does not cap at the report session limit.
assert data["stats"]["total_sessions"] == session_count
assert data["stats"]["total_commands"] == session_count
assert data["stats"]["total_projects"] == 1
assert len(data["sessions"]) == WEB_REPORT_SESSION_LIMIT
assert data["session_window"] == {
"limit": WEB_REPORT_SESSION_LIMIT,
"returned": WEB_REPORT_SESSION_LIMIT,
"total": session_count,
"truncated": True,
}

# Verify daily activity heatmap calculations work and are populated
today_str = datetime.fromtimestamp(now).strftime("%Y-%m-%d")
assert today_str in data["daily_activity"]
# The sum of commands across all days in heatmap should be 1005
# The sum of commands across all days in heatmap should include every session.
total_heatmap_commands = sum(day["commands"] for day in data["daily_activity"].values())
assert total_heatmap_commands == 1005
assert total_heatmap_commands == session_count

# Test custom template with const reportData = ... and backslashes
template_file = tmp_path / "custom_template.html"
Expand All @@ -321,4 +375,3 @@ def test_swarm_audit_fixes(tmp_path, monkeypatch):
assert "const reportData =" in html
# Check if our backslash command exists intact in the JSON
assert "backslash \\\\" in html

Loading