diff --git a/CHANGELOG.md b/CHANGELOG.md index 55d3424..3c4c649 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,7 +16,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - The `docs` job's `cd docs && make html` had no `docs/` directory to build — added a minimal Sphinx scaffold (`docs/conf.py`, `docs/index.rst`, `docs/Makefile`) so the job builds real output instead of failing on a missing path. - `mypy source/` (run with no config) failed with 16 pre-existing type errors across `DateUtils.py`, `Chat.py`, `KeywordForwardDialog.py`, and `ForwardDialog.py` — untyped-import noise from `telethon`, implicit-`Optional` parameter defaults, a `tzinfo | None` attribute access, a return-type mismatch on `Chat.scan_wanted_user`, and false positives from InquirerPy's `execute_async()` stubs being (incorrectly) typed as returning `None`. Fixed the real type issues and added `[tool.mypy]` config (`ignore_missing_imports`, and `follow_imports = "skip"` for `InquirerPy.*`) so the check reflects actual bugs instead of stub noise. - `ruff format --check source/ tests/` (enforced by CI) failed on 13 files that had never been run through `ruff format` (only `black`, via pre-commit, had touched them, and the two formatters disagree on some edge cases). Reformatted the whole tree so the CI check and the pre-commit hook agree. - +- `web/app.py`'s home page crashed with a 500 error (`TypeError: cannot use 'tuple' as a dict key`) on the pinned Starlette version — `templates.TemplateResponse("index.html", {"request": request})` uses the old, removed calling convention. Fixed to the current signature (`TemplateResponse(request, "index.html")`). +- `python-dotenv` was a declared dependency that nothing ever called — `cp .env.example .env` had no effect on a plain `python web/app.py` run, only under `docker-compose` (which substitutes `.env` itself). `web/app.py` now calls `load_dotenv()` on startup so the documented `.env` workflow actually works locally; real environment variables still take precedence. +- The web dashboard's own frontend (`web/templates/index.html`) called every `/api/*` endpoint with a bare `fetch()`, so the `X-API-Key` requirement added below broke the built-in UI (every button and the initial status load returned 401). Added an `apiFetch()` wrapper that prompts for the key once, sends it on every request, and re-prompts if the server rejects it. +- README instructions in all three install paths said to "edit `.env` with your Telegram API credentials," but `.env` only ever held web-dashboard settings (`API_KEY`, `WEB_HOST`, `WEB_PORT`) — Telegram credentials are entered interactively and stored in `resources/credentials.json`. Corrected each install path, and added a note that the web dashboard/Docker `web` service needs a session created by running `python main.py` once first. +- README had a `## Changelog Policy` section sitting between two subsections of `## Usage`, which made the `### Web API` heading that followed it read as a child of the changelog policy. Moved the changelog policy under `## Contributing` and restored `### Web API` as a subsection of `## Usage`, plus added a note there (and a `curl` example) that every `/api/*` route now requires the `X-API-Key` header. - **History/reply mapping was broken for every queued forward.** `MessageForwardService.forward_message`/`forward_album` returned the *source* message immediately when queuing a send (since the real send happens later, asynchronously), and `Forward._forward_message`/`_forward_album` used that placeholder to record history mappings — meaning `(source_chat, source_id) -> (source_chat, source_id)` was stored instead of `(source_chat, source_id) -> (dest_chat, dest_id)`. Since `Telegram` always constructs a real queue, this hit every forward, silently breaking reply-threading. Fixed by passing an `on_sent` callback into the queue job that fires with the real destination message once it's actually sent, and updating history from that callback instead of the return value. Added regression tests covering the queued path. - `requirements.txt` was missing `fastapi`, `uvicorn`, `jinja2`, `pydantic`, `python-dotenv`, `loguru`, and `aiofiles` (present in `pyproject.toml` but not `requirements.txt`), so `pip install -r requirements.txt` could not run `web/app.py`. Also dropped the obsolete `typing` PyPI backport, which can break stdlib `typing` on Python 3.10+. - `web/app.py`'s startup handler checked for `TELEGRAM_API_ID`/`TELEGRAM_API_HASH`/`TELEGRAM_PHONE` env vars that the app never reads or sets (credentials come from `resources/credentials.json`) and then did nothing (`pass`), so `telegram_client` was never initialized and every `/api/*` route reported "not initialized". It now reuses the session created by the CLI (`python main.py`) via `Credentials.get_all()` + `Telegram.create()`. diff --git a/README.md b/README.md index 1351d8f..dcda663 100644 --- a/README.md +++ b/README.md @@ -63,19 +63,17 @@ Choose one of the following installation methods: pip install -e ".[dev]" ``` -4. **Set up environment variables:** - - ```bash - cp .env.example .env - # Edit .env with your Telegram API credentials - ``` - -5. **Run the bot:** +4. **Run the bot:** ```bash python main.py ``` + You'll be prompted for your Telegram API credentials on first run; they're + saved to `resources/credentials.json` for reuse (no `.env` file needed for + the CLI — that's only required if you also run the web dashboard, see + [Configuration](#configuration)). + #### Option 2: Docker Installation (Recommended for Production) 1. **Clone the repository:** @@ -89,10 +87,24 @@ Choose one of the following installation methods: ```bash cp .env.example .env - # Edit .env with your Telegram API credentials + # Edit .env and set API_KEY (required by the web service, see Configuration below) ``` -3. **Run with Docker Compose:** +3. **Authenticate once, interactively, before running detached:** + + The `telegram-forwarder` container runs `python main.py`, which prompts + for your Telegram API credentials and login code on first use. Run it + attached the first time so you can answer the prompts: + + ```bash + docker-compose run --rm telegram-forwarder + ``` + + This saves a session under `./sessions` and credentials under + `./resources`, which are mounted as volumes — subsequent runs (including + the `web` service) reuse that session without prompting again. + +4. **Run with Docker Compose:** ```bash docker-compose up --build @@ -104,13 +116,13 @@ Choose one of the following installation methods: docker-compose up -d --build ``` -4. **View logs:** +5. **View logs:** ```bash docker-compose logs -f telegram-forwarder ``` -5. **Stop the bot:** +6. **Stop the bot:** ```bash docker-compose down @@ -119,7 +131,7 @@ Choose one of the following installation methods: **Docker Notes:** - Configuration files are stored in Docker volumes for persistence -- Environment variables are loaded from the `.env` file +- The web service's `API_KEY` is loaded from the `.env` file; `docker-compose up` fails fast with a clear error if it isn't set - The bot runs as a non-root user for security #### Option 3: Web Interface (FastAPI Dashboard) @@ -144,14 +156,22 @@ Choose one of the following installation methods: pip install -e ".[web]" ``` -4. **Set up environment variables:** +4. **Authenticate an account first, via the CLI:** + + The web dashboard never prompts for a Telegram login itself — it reuses + the session created by the CLI. Run `python main.py` once, answer the + credential/login prompts, then exit; this saves a session under + `./sessions` and credentials under `./resources`. + +5. **Set up environment variables:** ```bash cp .env.example .env - # Edit .env with your Telegram API credentials + # Edit .env and set API_KEY (the dashboard refuses to serve API + # requests without it — see Configuration below) ``` -5. **Run the web dashboard:** +6. **Run the web dashboard:** ```bash python web/app.py @@ -163,9 +183,11 @@ Choose one of the following installation methods: python -m web.app ``` -6. **Open your browser:** +7. **Open your browser:** - Navigate to `http://localhost:8000` to access the web interface. + Navigate to `http://localhost:8000` to access the web interface. The + page will prompt you once for the `API_KEY` value and remember it in + the browser's local storage for subsequent requests. **Web Interface Features:** @@ -251,17 +273,15 @@ Use the `Keyword Search + Forward` menu option to: Matching messages are forwarded in chronological order. -## Changelog Policy - -`CHANGELOG.md` is maintained as a running log for every update. Each release entry should include: - -- `Added`/`Changed`/`Fixed` details -- Explicit behavior delta when relevant (before vs after) -- Release date and semantic version - ### Web API -The web interface provides a REST API for programmatic access: +The web interface provides a REST API for programmatic access. Every +`/api/*` route requires an `X-API-Key` header matching the server's +`API_KEY` environment variable (see [Configuration](#configuration)): + +```bash +curl -H "X-API-Key: $API_KEY" http://localhost:8000/api/status +``` #### Status & Monitoring @@ -327,6 +347,11 @@ telegram-forwarder-bot-v2/ │ ├── static/ │ ├── templates/ │ └── README.md +├── docs/ +│ ├── conf.py +│ ├── index.rst +│ └── Makefile +├── tests/ ├── .env.example ├── .gitignore ├── .pre-commit-config.yaml @@ -338,6 +363,26 @@ telegram-forwarder-bot-v2/ └── README.md ``` +### Running Tests and Checks + +Install the dev tools, then run the same checks CI runs: + +```bash +pip install -e ".[dev]" +ruff check source/ tests/ +ruff format --check source/ tests/ +mypy source/ +pytest tests/ --cov=source +``` + +### Building Documentation + +```bash +pip install -e ".[docs]" +cd docs +make html # output in docs/_build/html/ +``` + ## Architecture ### Core Components @@ -370,10 +415,18 @@ telegram-forwarder-bot-v2/ - Update documentation for API changes - Use conventional commit messages +### Changelog Policy + +`CHANGELOG.md` is maintained as a running log for every update. Each release entry should include: + +- `Added`/`Changed`/`Fixed` details +- Explicit behavior delta when relevant (before vs after) +- Release date and semantic version + ## Security -- Never commit API credentials or sensitive data -- Use environment variables for configuration +- Never commit API credentials or sensitive data (`resources/credentials.json` and `sessions/` are gitignored) +- The web dashboard requires an `API_KEY` and refuses to serve `/api/*` requests without one; it binds to `127.0.0.1` by default - Regularly update dependencies for security patches - Follow principle of least privilege diff --git a/web/app.py b/web/app.py index c81d68b..3899ddb 100644 --- a/web/app.py +++ b/web/app.py @@ -11,6 +11,7 @@ from typing import Dict, List, Optional +from dotenv import load_dotenv from fastapi import Depends, FastAPI, Header, HTTPException from fastapi.staticfiles import StaticFiles from fastapi.templating import Jinja2Templates @@ -22,6 +23,12 @@ from source.model.Credentials import Credentials from source.model.ForwardConfig import ForwardConfig +# Docker Compose substitutes .env into the container's real environment, but +# a plain `python web/app.py` run never reads .env on its own. Load it here +# so the .env file created via `cp .env.example .env` actually takes effect; +# real environment variables (e.g. set by docker-compose) still win. +load_dotenv() + # The dashboard controls a real Telegram account (listing chats, forwarding # messages), so every API route requires a shared-secret API key. Refuse to # serve requests if the operator hasn't set one, rather than defaulting to @@ -132,7 +139,7 @@ async def lifespan(app: FastAPI): @app.get("/", response_class=HTMLResponse) async def home(request: Request): """Serve the main web interface.""" - return templates.TemplateResponse("index.html", {"request": request}) + return templates.TemplateResponse(request, "index.html") @app.get( diff --git a/web/templates/index.html b/web/templates/index.html index f93d573..f49b060 100644 --- a/web/templates/index.html +++ b/web/templates/index.html @@ -150,9 +150,31 @@

Available Chats

setTimeout(() => alert.classList.add('hidden'), 5000); } + // Every /api/* route requires the X-API-Key header set on the server + // via the API_KEY environment variable. Ask for it once and remember + // it in this browser; re-prompt if the server rejects it. + function getApiKey() { + let key = localStorage.getItem('tfb_api_key'); + if (!key) { + key = window.prompt('Enter the dashboard API key (set via the API_KEY environment variable on the server):') || ''; + localStorage.setItem('tfb_api_key', key); + } + return key; + } + + async function apiFetch(url, options = {}) { + const headers = Object.assign({}, options.headers, { 'X-API-Key': getApiKey() }); + const response = await fetch(url, Object.assign({}, options, { headers })); + if (response.status === 401) { + localStorage.removeItem('tfb_api_key'); + showAlert('Invalid API key. Reload the page and try again.', 'error'); + } + return response; + } + async function refreshStatus() { try { - const response = await fetch('/api/status'); + const response = await apiFetch('/api/status'); const data = await response.json(); document.getElementById('bot-status').textContent = data.status; document.getElementById('active-forwards').textContent = data.active_forwards; @@ -178,7 +200,7 @@

Available Chats

}; try { - const response = await fetch('/api/keyword-forward', { + const response = await apiFetch('/api/keyword-forward', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(payload) @@ -197,7 +219,7 @@

Available Chats

async function loadForwards() { try { - const response = await fetch('/api/forwards'); + const response = await apiFetch('/api/forwards'); const forwards = await response.json(); const tbody = document.getElementById('forwards-body'); @@ -224,7 +246,7 @@

Available Chats

async function loadChats() { try { - const response = await fetch('/api/chats'); + const response = await apiFetch('/api/chats'); const chats = await response.json(); const tbody = document.getElementById('chats-body'); @@ -243,7 +265,7 @@

Available Chats

async function startForwarding() { try { - const response = await fetch('/api/start-forwarding', { method: 'POST' }); + const response = await apiFetch('/api/start-forwarding', { method: 'POST' }); const result = await response.json(); showAlert(result.message); refreshStatus(); @@ -254,7 +276,7 @@

Available Chats

async function stopForwarding() { try { - const response = await fetch('/api/stop-forwarding', { method: 'POST' }); + const response = await apiFetch('/api/stop-forwarding', { method: 'POST' }); const result = await response.json(); showAlert(result.message); refreshStatus(); @@ -273,7 +295,7 @@

Available Chats

}; try { - const response = await fetch('/api/forwards', { + const response = await apiFetch('/api/forwards', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(data) @@ -290,7 +312,7 @@

Available Chats

async function toggleForward(sourceId) { try { - const response = await fetch(`/api/forwards/${sourceId}/toggle`, { method: 'POST' }); + const response = await apiFetch(`/api/forwards/${sourceId}/toggle`, { method: 'POST' }); const result = await response.json(); showAlert(result.message); loadForwards(); @@ -305,7 +327,7 @@

Available Chats

} try { - const response = await fetch(`/api/forwards/${sourceId}`, { method: 'DELETE' }); + const response = await apiFetch(`/api/forwards/${sourceId}`, { method: 'DELETE' }); const result = await response.json(); showAlert(result.message); loadForwards();