Fix media wall reliability

AGENT_DEPLOY.md

@@ -78,7 +78,7 @@ Notes:
 - `MTEAM_PROACTIVE_ENABLED=false` disables system-owned M-Team background access (country preload, FREE refresh loop, media-wall refresh loop, and PILOT guarded cache refresh) while preserving user-initiated RADAR/search/download endpoints.
 - `PANEL_COLLECT_INTERVAL` defaults to `60` seconds, controlling PANEL history collection frequency.
 - `FREE_REFRESH_FAILURE_BACKOFF_SECONDS` defaults to `7200` seconds; on a FREE refresh failure it keeps the old cache and retries after cooldown.
-- `MEDIA_WALL_REFRESH_INTERVAL` defaults to `21600` seconds (6 hours), do not set it below `21600`; this is the per-source refresh interval, and `latest / movies / series / hot` rotate on a stagger, refreshing roughly one source every 90 minutes by default.
+- `MEDIA_WALL_REFRESH_INTERVAL` defaults to `21600` seconds (6 hours), do not set it below `21600`; this is the per-source steady-state interval. Missing `latest / movies / series / hot` sources bootstrap no more than 90 minutes apart, then resume the configured stagger.
 - `MEDIA_WALL_STARTUP_DELAY` defaults to `420` seconds, used to stagger away from SONAR's first refresh after container start.
 - `MEDIA_WALL_REFRESH_FAILURE_BACKOFF_SECONDS` defaults to `7200` seconds; on a media-wall source refresh failure it keeps the old cache and retries after cooldown.
 - `MEDIA_WALL_DOUBAN_POSTER_FETCHES` defaults to `3`, reading Douban subject pages at low frequency only when M-Team metadata lacks a poster; do not raise it to a large value just to chase full poster coverage.

CHANGELOG.md

@@ -1,5 +1,14 @@
 ## [Unreleased]
 
+### Home Media Wall Reliability
+
+- Fixed regional rail conflicts so explicit country/language text overrides numeric hints, cached cards with a clear conflicting region are rerouted, and numeric country IDs use M-Team's runtime mapping instead of hardcoded assumptions.
+- Treat missing, empty, or overdue sources as stale, preserve the previous source on invalid empty responses, and cap missing-source bootstrap spacing at 90 minutes even when the steady-state refresh interval is much longer.
+- Prioritize never-enriched media within the metadata budget, deduplicate across every normalized Douban/IMDb identity, prefer release-name years, reject known Douban placeholder posters, and keep old films out of `高质量新片新剧`.
+- Fixed zero-rating fallback, stale detail sheets, permanent poster failures, error/empty states, rail scrolling, empty-rail layout, source diagnostics, and media-wall polling on the frontend.
+- Project internal source caches out before poster rewriting, bound poster proxy responses to 10 MiB with asynchronous DNS lookup, proxy Amazon posters, and add cache headers for the media response and hashed Next.js assets.
+- Exposed media metadata TTL/budget in `/api/status` and report stale media walls or unavailable country mappings through runtime warnings.
+
 ### Freeleech Auto-Delete Safety
 
 - RADAR（`雷达下载`）下载是用户主动搜索想看的资源，免费监控不再把它当作"免费下车"对象自动删除：`雷达下载` 现在是一票否决标签，即使同时带 `PILOT`/`声呐做种`（qB 按 hash 合并标签时会出现）也不会被自动删；纯 `声呐做种`/`PILOT` 仍按原逻辑清理。

README.md

@@ -188,11 +188,11 @@ If you use PILOT auto-download, also confirm:
 | `API_DELAY` | No | M-Team API request interval, clamped to 3-10 seconds; default is 6 seconds, below 3 seconds tends to trigger dynamic rate limiting | `6` |
 | `MTEAM_PROACTIVE_ENABLED` | No | Whether system-owned background jobs may call M-Team; set `false` for a manual-only instance while keeping user-initiated RADAR/search/download available | `true` |
 | `PANEL_COLLECT_INTERVAL` | No | PANEL data collection interval, in seconds | `60` |
-| `MEDIA_WALL_REFRESH_INTERVAL` | No | HOME media-wall per-source background refresh interval, in seconds; defaults to 6 hours, not allowed below 21600; the four sources rotate on a stagger | `21600` |
+| `MEDIA_WALL_REFRESH_INTERVAL` | No | HOME media-wall per-source background refresh interval, in seconds; defaults to 6 hours, not allowed below 21600; missing sources bootstrap no more than 90 minutes apart before normal staggering resumes | `21600` |
 | `MEDIA_WALL_REFRESH_FAILURE_BACKOFF_SECONDS` | No | Backoff after a HOME media-wall source refresh failure, in seconds; keeps the old cache and retries after cooldown | `7200` |
 | `MEDIA_WALL_STARTUP_DELAY` | No | Delay before the media wall's first refresh after container start, to avoid overlapping SONAR's first refresh | `420` |
 | `MEDIA_WALL_METADATA_TTL` | No | Cache TTL for M-Team media metadata such as posters, year, and summary, in seconds | `604800` |
-| `MEDIA_WALL_MAX_METADATA_FETCHES` | No | Metadata backfill budget per full rotation cycle; a single source refresh uses about 1/4 of the budget | `40` |
+| `MEDIA_WALL_MAX_METADATA_FETCHES` | No | Metadata backfill budget per full rotation cycle; a single source refresh uses about 1/4 and prioritizes media never enriched before refreshing old entries | `40` |
 | `MEDIA_WALL_DOUBAN_POSTER_FETCHES` | No | Max number of Douban-page posters fetched at low frequency per media-wall source refresh; used only when M-Team metadata lacks a poster | `3` |
 | `PUSHPLUS_TOKEN` | No | PushPlus WeChat push Token | - |
 | `QBITTORRENT_URL` | No | qBittorrent Web UI URL; use a pinned qB image tag on NAS until intentionally upgrading | - |

README.zh-CN.md

@@ -188,11 +188,11 @@ curl http://localhost:5050/api/status
 | `API_DELAY` | 否 | M-Team API 请求间隔，限制 3-10 秒，默认 6 秒；低于 3 秒容易触发动态限流 | `6` |
 | `MTEAM_PROACTIVE_ENABLED` | 否 | 是否允许系统后台主动访问 M-Team；手动专用实例可设为 `false`，仍保留用户主动 RADAR 搜索/下载 | `true` |
 | `PANEL_COLLECT_INTERVAL` | 否 | PANEL 数据采集间隔，单位秒 | `60` |
-| `MEDIA_WALL_REFRESH_INTERVAL` | 否 | HOME 媒体墙每个 source 的后台刷新间隔，单位秒；默认 6 小时，不允许低于 21600；四个 source 会自动错峰轮转 | `21600` |
+| `MEDIA_WALL_REFRESH_INTERVAL` | 否 | HOME 媒体墙每个 source 的后台刷新间隔，单位秒；默认 6 小时，不允许低于 21600；缺失 source 会以不超过 90 分钟的间隔先补齐，再恢复正常错峰 | `21600` |
 | `MEDIA_WALL_REFRESH_FAILURE_BACKOFF_SECONDS` | 否 | HOME 媒体墙 source 刷新失败退避时间，单位秒；失败时保留旧缓存，冷却后再试 | `7200` |
 | `MEDIA_WALL_STARTUP_DELAY` | 否 | 容器启动后媒体墙首刷延迟，避免和 SONAR 首轮刷新叠加 | `420` |
 | `MEDIA_WALL_METADATA_TTL` | 否 | 海报、年份、简介等 M-Team 媒体元数据缓存时间，单位秒 | `604800` |
-| `MEDIA_WALL_MAX_METADATA_FETCHES` | 否 | 媒体墙完整轮转周期的元数据补充预算；单个 source 刷新会使用约 1/4 预算 | `40` |
+| `MEDIA_WALL_MAX_METADATA_FETCHES` | 否 | 媒体墙完整轮转周期的元数据补充预算；单个 source 使用约 1/4，并优先补从未获取过元数据的媒体 | `40` |
 | `MEDIA_WALL_DOUBAN_POSTER_FETCHES` | 否 | 每个媒体墙 source 刷新时最多低频抓取的豆瓣页面海报数；只在 M-Team 元数据缺海报时使用 | `3` |
 | `PUSHPLUS_TOKEN` | 否 | PushPlus 微信推送 Token | - |
 | `QBITTORRENT_URL` | 否 | qBittorrent Web UI 地址；NAS 建议固定 qB 镜像 tag，明确升级时再改 | - |

app/main.py

@@ -23,6 +23,8 @@
 from app.config import BEIJING_TZ
 from app.config import BUILD_COMMIT
 from app.config import MEDIA_WALL_DOUBAN_POSTER_FETCHES
+from app.config import MEDIA_WALL_MAX_METADATA_FETCHES
+from app.config import MEDIA_WALL_METADATA_TTL
 from app.config import MEDIA_WALL_REFRESH_INTERVAL
 from app.config import MEDIA_WALL_STARTUP_DELAY
 from app.config import MTEAM_PROACTIVE_ENABLED
@@ -59,6 +61,7 @@
 from app.routes.torrents import api_torrents
 from app.services.http_client import http_client
 from app.services.mteam_api import mt_client
+from app.services.media_wall import MISSING_SOURCE_STAGGER_MAX_SECONDS
 from app.services.media_wall import SOURCE_REFRESH_ORDER
 from app.services.media_wall import media_wall_service
 from app.services.panel_collector import get_panel_collector_status
@@ -140,6 +143,7 @@ def build_status_warnings(
     cache_status: dict,
     panel_collector_status: dict,
     dependencies: dict,
+    media_wall_snapshot: Optional[dict] = None,
 ) -> List[str]:
     warnings = []
     if cache_status.get("stale"):
@@ -159,6 +163,14 @@ def build_status_warnings(
         warnings.append("qbittorrent_unhealthy")
     if MTEAM_PROACTIVE_ENABLED and not dependencies.get("mteam", {}).get("ok", False):
         warnings.append("mteam_unhealthy")
+    if MTEAM_PROACTIVE_ENABLED and (media_wall_snapshot or {}).get("stale"):
+        warnings.append("media_wall_stale")
+    if (
+        MTEAM_PROACTIVE_ENABLED
+        and dependencies.get("mteam", {}).get("ok", False)
+        and not state.COUNTRY_LABELS
+    ):
+        warnings.append("country_labels_unavailable")
     return warnings
 
 
@@ -305,6 +317,11 @@ async def lifespan(app: FastAPI):
 async def add_security_headers(request: Request, call_next):
     """Add security headers to all responses"""
     response = await call_next(request)
+    if (
+        request.url.path.startswith("/_next/static/")
+        and response.status_code in {200, 304}
+    ):
+        response.headers["Cache-Control"] = "public, max-age=31536000, immutable"
     # Prevent clickjacking
     response.headers["X-Frame-Options"] = "DENY"
     # Prevent MIME sniffing
@@ -438,6 +455,7 @@ async def api_status():
     cache_status = build_cache_status()
     panel_collector_status = get_panel_collector_status()
     dependencies = runtime_status.as_dict()
+    media_wall_snapshot = media_wall_service.get_snapshot()
 
     status_payload = {
         "status": "ok",
@@ -454,6 +472,9 @@ async def api_status():
             "media_wall_refresh_interval_seconds": MEDIA_WALL_REFRESH_INTERVAL,
             "media_wall_startup_delay_seconds": MEDIA_WALL_STARTUP_DELAY,
             "media_wall_source_stagger_seconds": MEDIA_WALL_REFRESH_INTERVAL // len(SOURCE_REFRESH_ORDER),
+            "media_wall_missing_source_stagger_max_seconds": MISSING_SOURCE_STAGGER_MAX_SECONDS,
+            "media_wall_metadata_ttl_seconds": MEDIA_WALL_METADATA_TTL,
+            "media_wall_max_metadata_fetches": MEDIA_WALL_MAX_METADATA_FETCHES,
             "media_wall_douban_poster_fetches": MEDIA_WALL_DOUBAN_POSTER_FETCHES,
             "api_delay_seconds": API_DELAY,
             "qbittorrent_configured": bool(QBITTORRENT_URL and QBITTORRENT_USER and QBITTORRENT_PASSWORD),
@@ -465,6 +486,7 @@ async def api_status():
             cache_status,
             panel_collector_status,
             dependencies,
+            media_wall_snapshot,
         ),
     }
     return JSONResponse(status_payload)

app/models.py

@@ -258,6 +258,9 @@ class RuntimeConfigStatus(BaseModel):
     media_wall_refresh_interval_seconds: int = 0
     media_wall_startup_delay_seconds: int = 0
     media_wall_source_stagger_seconds: int = 0
+    media_wall_missing_source_stagger_max_seconds: int = 0
+    media_wall_metadata_ttl_seconds: int = 0
+    media_wall_max_metadata_fetches: int = 0
     media_wall_douban_poster_fetches: int = 0
     api_delay_seconds: float
     qbittorrent_configured: bool

app/routes/home.py

@@ -1,5 +1,6 @@
 """Read-only Home media wall API."""
 
+import asyncio
 import ipaddress
 import socket
 from typing import cast
@@ -19,6 +20,18 @@
 router = APIRouter()
 
 POSTER_PROXY_TIMEOUT_SECONDS = 10.0
+POSTER_MAX_BYTES = 10 * 1024 * 1024
+POSTER_PROXY_CONCURRENCY = 4
+POSTER_ALLOWED_CONTENT_TYPES = frozenset(
+    {
+        "image/avif",
+        "image/gif",
+        "image/jpeg",
+        "image/png",
+        "image/webp",
+    }
+)
+_poster_proxy_semaphore = asyncio.Semaphore(POSTER_PROXY_CONCURRENCY)
 
 _BLOCKED_IP_NETWORKS = tuple(
     ipaddress.ip_network(network)
@@ -35,6 +48,7 @@
     )
 )
 POSTER_CACHE_CONTROL = "public, max-age=86400"
+MEDIA_WALL_CACHE_CONTROL = "private, max-age=60"
 
 
 @router.get("/api/home/poster", include_in_schema=False)
@@ -50,7 +64,7 @@ async def get_home_poster(u: str):
     if hostname is None:
         raise HTTPException(status_code=400, detail="Invalid poster URL")
     try:
-        resolved_ips = _resolve_host_ips(hostname)
+        resolved_ips = await asyncio.to_thread(_resolve_host_ips, hostname)
     except Exception as exc:
         raise HTTPException(status_code=502, detail="Bad Gateway") from exc
     if not resolved_ips:
@@ -63,27 +77,46 @@ async def get_home_poster(u: str):
         headers["Referer"] = "https://movie.douban.com/"
 
     try:
-        client = await get_http_client()
-        response = await client.get(
-            u,
-            headers=headers,
-            follow_redirects=False,
-            timeout=POSTER_PROXY_TIMEOUT_SECONDS,
-        )
+        async with _poster_proxy_semaphore:
+            client = await get_http_client()
+            async with client.stream(
+                "GET",
+                u,
+                headers=headers,
+                follow_redirects=False,
+                timeout=POSTER_PROXY_TIMEOUT_SECONDS,
+            ) as response:
+                if 300 <= response.status_code < 400:
+                    raise HTTPException(status_code=502, detail="Bad Gateway")
+                if not 200 <= response.status_code < 300:
+                    raise HTTPException(status_code=502, detail="Bad Gateway")
+                try:
+                    content_length = int(response.headers.get("content-length", "0"))
+                except ValueError:
+                    content_length = 0
+                if content_length > POSTER_MAX_BYTES:
+                    raise HTTPException(status_code=502, detail="Bad Gateway")
+                content_type = (
+                    response.headers.get("content-type", "")
+                    .split(";", 1)[0]
+                    .strip()
+                    .lower()
+                )
+                if content_type not in POSTER_ALLOWED_CONTENT_TYPES:
+                    raise HTTPException(status_code=502, detail="Bad Gateway")
+
+                content = bytearray()
+                async for chunk in response.aiter_bytes():
+                    if len(content) + len(chunk) > POSTER_MAX_BYTES:
+                        raise HTTPException(status_code=502, detail="Bad Gateway")
+                    content.extend(chunk)
+    except HTTPException:
+        raise
     except Exception as exc:
         raise HTTPException(status_code=502, detail="Bad Gateway") from exc
-    if 300 <= response.status_code < 400:
-        raise HTTPException(status_code=502, detail="Bad Gateway")
-    if not 200 <= response.status_code < 300:
-        raise HTTPException(status_code=502, detail="Bad Gateway")
-    content_type = (
-        response.headers.get("content-type", "").split(";", 1)[0].strip().lower()
-    )
-    if not content_type.startswith("image/"):
-        raise HTTPException(status_code=502, detail="Bad Gateway")
 
     return Response(
-        content=response.content,
+        content=memoryview(content),
         media_type=content_type,
         headers={"Cache-Control": POSTER_CACHE_CONTROL},
     )
@@ -111,6 +144,10 @@ def _is_allowlisted_poster_host_for_suffix(hostname: str, suffix: str) -> bool:
 
 
 @router.get("/api/home/media-wall", response_model=MediaWallResponse)
-async def get_home_media_wall():
+async def get_home_media_wall(response: Response):
     """Return the cached Home media wall snapshot without refreshing M-Team."""
-    return proxy_media_wall_posters(media_wall_service.get_snapshot())
+    response.headers["Cache-Control"] = MEDIA_WALL_CACHE_CONTROL
+    payload = MediaWallResponse.model_validate(
+        media_wall_service.get_snapshot()
+    ).model_dump()
+    return proxy_media_wall_posters(payload)

app/routes/radar.py

@@ -59,7 +59,7 @@ async def api_filter_options():
         }
     else:
         # If no country data loaded, don't include countries field
-        # Frontend will use its fallback data
+        # Frontend hides country choices until the authoritative mapping is available.
         return FILTER_OPTIONS