Skip to content

SPEC.md

Latest commit

 

History

History
143 lines (116 loc) · 7.43 KB

SPEC.md

File metadata and controls

143 lines (116 loc) · 7.43 KB
Raw

Model Info

1. Document Overview

This page serves as the operational dashboard for model health checks in the production environment. It is used to present the model load status, the input feature structure, and the feature weights with their positive or negative influence, delivering an integrated visual presentation for model deployment verification, algorithm iteration optimisation and business operations reference

2. Functional Requirements

FR-01: Page Access and Loading

  • ID: FR-01
  • Title: Page Entry Point and Load Status
  • Description: After the user selects "Model Info" in the sidebar SelectBox, the page renders all modules in order; each module requests the API independently, and the failure of a single module does not block the other modules
  • Acceptance Criteria:
    1. The sidebar navigation items include "Model Info"
    2. If the /model/info request fails, only the top 3 cards show an error, and the Feature Importance module can still load independently, and vice versa
    3. The request timeout threshold is ≥ 30 seconds

FR-02: Model Metadata Overview Cards (Top Three Metrics)

  • ID: FR-02
  • Title: Model Metadata Overview Cards
  • Description: Three Metric cards are displayed side by side at the top of the page, so that the model identity can be judged quickly within 5 seconds
  • Fields:
Field Source (/model/info response) Meaning Teaching Note
Model Type model_type The class name of the model object (for example Pipeline) If it is not Pipeline, this suggests that the deployment form does not meet production standards
Feature Count feature_count (preferred) or raw_feature_count (fallback) The raw feature dimensionality (before one-hot) Used for version comparison: if the previous version = 58 and the new version = 32, the feature engineering chain has very likely broken
Version model_version The semantic version number Used for subsequent model version management
  • Acceptance Criteria:
    1. The three cards are displayed side by side on the same row
    2. feature_count is displayed as an integer; when it is None or empty it falls back to 0
    3. If the API fails, a red error box is displayed in the card area (without raising an exception that interrupts the page)

FR-03: Load Status Module

  • ID: FR-03
  • Title: Model Load Status
  • Description: Displays the status field, whose value range is loaded / not_loaded
  • Acceptance Criteria:
    1. When loaded, a neutral/success style is used
    2. When not_loaded, it must be highlighted (red background or warning icon) and must prompt the user to check the backend startup log

FR-04: Sample Feature List

  • ID: FR-04
  • Title: Sample Features
  • Description: Displays the first 10 raw feature names from the features or raw_features list, output verbatim in JSON format
  • Acceptance Criteria:
    1. Displayed as collapsible/expandable JSON (st.json)
    2. When empty, [] is displayed rather than blank

FR-05: Feature Importance Table

  • ID: FR-05
  • Title: Feature Importance Table
  • Description: Calls the /metrics/features endpoint and displays each item in a table
  • Required Columns:
    1. feature – The feature name (the one-hot name produced by the preprocessor, carrying a num__ or cat__ prefix)
    2. importance – The absolute value of the coefficient/importance, used for sorting
    3. coefficient – The raw signed coefficient, used when discussing direction
  • Default Sorting: Descending by importance
  • Acceptance Criteria:
    1. The table width adapts to the container
    2. The row count is capped at 20 (Top-20)
    3. When the feature_importance key is missing or empty, the text of the message field is displayed

FR-06: Feature Importance Bar Chart

  • ID: FR-06
  • Title: Feature Importance Bar Chart (Top 20)
  • Description: Displays a horizontal bar chart below the table, visually presenting the relative importance of the Top-20
  • Plotting Rules:
    • X axis: importance
    • Y axis: feature (lower importance at the bottom, higher at the top)
    • Bar orientation: horizontal
  • Acceptance Criteria:
    1. The chart is rendered only when the table is not empty
    2. The chart width adapts to the container (use_container_width=True)
    3. The chart title contains the wording "Top 20"

FR-07: API Error Handling

  • ID: FR-07
  • Title: Graceful API Error Handling
  • Description: A failure of either API must not raise a Python exception stack trace
  • Acceptance Criteria:
    1. Error message format: Failed to fetch {xxx}: {error_reason}, where error_reason includes but is not limited to:
      • API Error: {status_code} (for example 500, 501, 503)
      • API server is offline
      • Request timeout
      • Error: {exception_msg}
    2. The error box must be the native Streamlit st.error (a red rounded box)

3. Non-Functional Requirements

NFR-01: Performance

  • /model/info P95 response time ≤ 200 ms (reads the metadata held in memory only, no disk IO)
  • /metrics/features P95 response time ≤ 500 ms (uses the top_coefficients cached in metadata by preference; the pipeline must not be recomputed online)

NFR-02: Observability

  • The backend must write a log at INFO level for every /metrics/features request, and the content must include the number of records returned
  • If a fallback path is taken (fallback 1→2→3), the fallback trigger point must record the reason at WARNING level

NFR-03: Robustness

  • A missing field of any kind in metadata must not raise an exception; a fallback must be taken:
    • Missing feature_count → use len(raw_feature_cols)
    • Missing features → use raw_features
    • Missing top_coefficients → attempt to obtain it from the Pipeline in real time; if that also fails → return an empty list together with a message
  • Old and new metadata versions must be compatible (a missing new field must not cause a 500)

NFR-04: Security

  • Feature names are business data but are not sensitive, so they may be displayed in plain text
  • Under no circumstances may the response expose local absolute paths or environment variables

4. API Contract

4.1 GET /model/info

  • Purpose: Returns the model identity and feature dimensionality metadata
  • Response Shape :
Key Type Required Description
model_type string/null Yes The Python class name
feature_count integer Yes (FR-02) Used by the frontend card
raw_feature_count integer Yes Retained for backward compatibility
model_version string Yes The semantic version
features array[string] Yes (FR-04) Used by Sample Features, length 0~10
raw_features array[string] Yes A synonymous fallback for features
metrics object Yes For example {"auc": 0.51, "accuracy": 0.60}, for future extension
status string Yes (FR-03) loaded / not_loaded

4.2 GET /metrics/features

  • Purpose: Returns the Top-N feature importance
  • Response Shape:
Key Type Required Description
feature_importance array[object] Yes Each item contains the three keys feature, importance, coefficient
count integer Yes The actual number of records returned
message string Yes Provides a readable hint when empty (for example "No feature importance available")