Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions ENVIRONMENT_VARIABLES_AND_SECRETS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,10 @@ Use `"*"` as the sole event entry to subscribe to all events from a contract.

| Variable | Default | Required | Description |
|---|---|---|---|
| `EVENTS_API_PORT` | `8787` | No | Port the listener HTTP server binds to. |
| `EVENTS_API_CORS_ORIGIN` | `http://localhost:5173` | No | Allowed CORS origin. Set to your dashboard URL in production. Avoid `*`. |
| `EVENTS_API_PORT` | `8787` | No | Port the listener HTTP server binds to (1–65535). |
| `EVENTS_API_CORS_ORIGIN` | `http://localhost:5173` | No | Allowed CORS origin(s). Explicit URI or comma-separated list in Production/Staging (e.g. `https://dashboard.notifychain.io`). Wildcard `*` is only allowed in local Development/Test environments. |

For a complete reference table across all environments, see [docs/ENVIRONMENT_MATRIX.md](docs/ENVIRONMENT_MATRIX.md).

### 2.5 Database

Expand Down
83 changes: 83 additions & 0 deletions docs/ENVIRONMENT_MATRIX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Environment Configuration Matrix

> Single reference matrix documenting all supported environment variables across Notify-Chain, their purpose, required/optional status, default values, sensitivity levels, and applicable environments.

---

## 1. Overview & Sensitivity Classification

- **Public / Non-sensitive (`Public`)**: Safe to commit in example files or pass in plain environment configs.
- **Internal / Configurable (`Internal`)**: Specific to deployment infrastructure; non-secret but environment-dependent.
- **Sensitive / Secret (`Secret`)**: Sensitive credentials, API keys, private webhooks, and cryptographic keys. **Must never be committed or logged**.

---

## 2. Master Configuration Matrix

| Environment Variable | Type | Status | Default Value | Applicable Environments | Sensitive? | Description / Purpose |
|----------------------|------|--------|---------------|-------------------------|------------|-----------------------|
| `CONTRACT_ADDRESSES` | JSON Array | **Required** | *(None)* | All (Dev, Staging, Prod) | No | JSON array of Soroban contract addresses and event names to monitor. Must contain at least 1 entry. |
| `STELLAR_NETWORK` | string | Optional | `testnet` | All | No | Target Stellar network name (`testnet`, `public`, `futurenet`, `standalone`). |
| `STELLAR_RPC_URL` | string (URL) | Optional | `https://soroban-testnet.stellar.org:443` | All | No | Endpoint URL for the Soroban RPC provider. |
| `STELLAR_NETWORK_PASSPHRASE` | string | Optional | `Test SDF Network ; September 2015` | All | No | Stellar network passphrase corresponding to the target chain network. |
| `EVENTS_API_PORT` | integer | Optional | `8787` | All | No | HTTP port for listener API endpoints (`/health`, `/api/events`, `/api/schedule`, etc.). |
| `EVENTS_API_CORS_ORIGIN` | string | Optional | `http://localhost:5173` | All | No | Allowed CORS origin(s). Explicit URL in Staging/Prod; wildcard `*` allowed only in Dev/Test. |
| `DATABASE_PATH` | string | Optional | `./data/notifications.db` | All | No | SQLite database filepath for notifications, cursor persistence, and history. |
| `POLL_INTERVAL_MS` | integer | Optional | `30000` | All | No | Frequency in milliseconds for polling on-chain events via Soroban RPC (min: 1000). |
| `MAX_RECONNECT_ATTEMPTS` | integer | Optional | `5` | All | No | Maximum consecutive connection retry attempts before entering degraded status. |
| `RECONNECT_DELAY_MS` | integer | Optional | `5000` | All | No | Initial delay in ms before retrying dropped RPC connections. |
| `LOG_LEVEL` | string | Optional | `info` | All | No | Logging verbosity (`error`, `warn`, `info`, `http`, `verbose`, `debug`, `silly`). |
| `NODE_ENV` | string | Optional | `development` | All | No | Runtime environment (`development`, `test`, `staging`, `production`). Enables structured JSON logs in production. |
| `DISCORD_WEBHOOK_URL` | string (URL) | Conditional | *(None)* | All | **Yes** (Secret) | Discord webhook endpoint URL for notification delivery. Required if `DISCORD_WEBHOOK_ID` is set. |
| `DISCORD_WEBHOOK_ID` | string | Conditional | *(None)* | All | **Yes** (Secret) | Unique Discord webhook identifier. Required if `DISCORD_WEBHOOK_URL` is set. |
| `DISCORD_RETRY_COUNT` | integer | Optional | `5` (uses `RETRY_MAX_RETRIES`) | All | No | Maximum delivery attempts for Discord webhook notifications. |
| `DISCORD_BACKOFF_BASE_SECONDS` | integer | Optional | `5` | All | No | Exponential backoff base delay in seconds between failed Discord notification retries. |
| `NOTIFICATION_DEDUPLICATION_WINDOW_MS` | integer | Optional | `60000` | All | No | Time window in milliseconds within which duplicate outgoing messages are suppressed. |
| `NOTIFICATION_DEDUPLICATION_MAX_SIZE` | integer | Optional | `10000` | All | No | Maximum entries maintained in memory for message deduplication. |
| `WEBHOOK_SECRETS` | JSON Array | Optional | `[]` | All | **Yes** (Secret) | Array of `{ id: string, secret: string }` pairs used to generate and verify HMAC signatures. |
| `API_KEYS` | JSON Array | Optional | `[]` | All | **Yes** (Secret) | Array of `{ key: string, name?: string }` objects authorized for protected API operations. |
| `PAYLOAD_INTEGRITY_SECRET` | string | Optional | *(None)* | Staging, Prod | **Yes** (Secret) | Secret key used to compute and verify HMAC-SHA256 checksums over persisted notification payloads. |
| `SCHEDULER_ENABLED` | boolean | Optional | `true` | All | No | Enable background job scheduler for delayed/scheduled notifications. |
| `SCHEDULER_POLL_INTERVAL_MS` | integer | Optional | `10000` | All | No | Frequency in milliseconds for polling pending scheduled notifications. |
| `SCHEDULER_LOCK_TIMEOUT_MS` | integer | Optional | `60000` | All | No | Maximum lock duration in ms for processing workers before releasing claims. |
| `SCHEDULER_BATCH_SIZE` | integer | Optional | `10` | All | No | Maximum number of notifications processed in a single scheduler loop iteration. |
| `SCHEDULER_TIMING_BUFFER_MS` | integer | Optional | `60000` | All | No | Advance lookahead window in ms for querying upcoming due notifications. |
| `RETRY_SCHEDULER_ENABLED` | boolean | Optional | `true` | All | No | Enable automatic retry scheduler for failed deliveries. |
| `RETRY_SCHEDULER_POLL_INTERVAL_MS` | integer | Optional | `15000` | All | No | Frequency in ms to check for failed notifications eligible for retry. |
| `RETRY_BASE_DELAY_MS` | integer | Optional | `5000` | All | No | Base delay in ms used for exponential backoff calculations. |
| `RETRY_MULTIPLIER` | integer | Optional | `2` | All | No | Multiplier applied to consecutive retry attempt backoff intervals. |
| `RETRY_MAX_DELAY_MS` | integer | Optional | `3600000` (1 hour) | All | No | Upper bound cap on exponential backoff delays. |
| `RETRY_JITTER` | boolean | Optional | `true` | All | No | Adds randomized jitter to retry intervals to prevent thundering herd spikes. |
| `RATE_LIMIT_ENABLED` | boolean | Optional | `true` | All | No | Enable rate-limiting middleware on the events API server. |
| `RATE_LIMIT_WINDOW_MS` | integer | Optional | `60000` (1 min) | All | No | Sliding window duration in ms for tracking client request quotas. |
| `RATE_LIMIT_MAX_REQUESTS` | integer | Optional | `60` | All | No | Allowed request count per window before returning HTTP 429 Too Many Requests. |
| `RATE_LIMIT_CLIENT_OVERRIDES` | JSON Object | Optional | `{}` | All | No | Per-client IP/key custom rate limits: `{"<key>": {"maxRequests": 100, "windowMs": 60000}}`. |
| `ANALYTICS_ENABLED` | boolean | Optional | `true` | All | No | Enable metrics collection and analytics aggregation engine. |
| `ANALYTICS_MAX_RECORDS` | integer | Optional | `10000` | All | No | In-memory capacity of the circular analytics event buffer. |
| `ANALYTICS_MAX_BUCKETS` | integer | Optional | `168` | All | No | Maximum hourly time-series buckets maintained in memory. |
| `ANALYTICS_BUCKET_SIZE_MS` | integer | Optional | `3600000` (1 hour) | All | No | Duration of individual analytics aggregation buckets. |
| `ANALYTICS_PERSIST_INTERVAL_MS`| integer | Optional | `300000` (5 mins) | All | No | Frequency of analytics snapshot persistence to disk. |
| `ANALYTICS_SNAPSHOT_RETENTION_DAYS`| integer | Optional| `30` | All | No | Number of days to retain historical metrics snapshots before pruning. |
| `CLEANUP_INTERVAL_MS` | integer | Optional | `3600000` (1 hour) | All | No | Frequency of database housekeeping and expired event pruning tasks. |
| `NOTIFICATION_RETENTION_MS` | integer | Optional | `604800000` (7 days) | All | No | Retention period for completed and failed notification records. |
| `EVENT_RETENTION_MS` | integer | Optional | `86400000` (24 hours) | All | No | Retention period for ingested on-chain events in the active registry. |
| `EXECUTION_LOG_RETENTION_MS`| integer | Optional | `7776000000` (90 days)| All | No | Retention period for detailed notification execution attempt logs. |

---

## 3. Environment-Specific Profiles

### Development (`NODE_ENV=development`)
- Uses local defaults: `EVENTS_API_CORS_ORIGIN="http://localhost:5173"` or `*`.
- `LOG_LEVEL="debug"` or `"info"`.
- Uses Soroban `testnet` RPC.

### Staging (`NODE_ENV=staging`)
- Explicit CORS origins (wildcard `*` rejected).
- Secret injection via secret store (`DISCORD_WEBHOOK_URL`, `WEBHOOK_SECRETS`, `PAYLOAD_INTEGRITY_SECRET`).
- Polling intervals tuned to staging cluster load.

### Production (`NODE_ENV=production`)
- Structured JSON logging enabled automatically.
- Strict schema and CORS validation enforced on startup.
- All secrets injected via production secret manager / KMS.
17 changes: 12 additions & 5 deletions docs/LISTENER-CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,18 @@ For secrets-handling guidance (what must not be committed), also see [ENVIRONMEN
2. [How configuration is loaded](#how-configuration-is-loaded)
3. [Required vs optional](#required-vs-optional)
4. [Configuration reference](#configuration-reference)
5. [Examples](#examples)
6. [Environment differences](#environment-differences)
7. [Recommended values (operational guidance)](#recommended-values-operational-guidance)
8. [Troubleshooting](#troubleshooting)
9. [Related documentation](#related-documentation)
5. [Environment configuration matrix](#environment-configuration-matrix)
6. [Examples](#examples)
7. [Environment differences](#environment-differences)
8. [Recommended values (operational guidance)](#recommended-values-operational-guidance)
9. [Troubleshooting](#troubleshooting)
10. [Related documentation](#related-documentation)

---

## Environment configuration matrix

For a single consolidated master table specifying all environment variables, their types, required/optional status, defaults, sensitivity classifications, and environment applicability, consult [docs/ENVIRONMENT_MATRIX.md](ENVIRONMENT_MATRIX.md).

---

Expand Down
99 changes: 99 additions & 0 deletions listener/src/config-schema.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
import {
ConfigurationSchemaValidator,
ConfigSchema,
APP_CONFIG_SCHEMA,
} from './config-schema';

describe('Configuration Schema Validation (#694)', () => {
const sampleValidConfig = {
stellarNetwork: 'testnet',
stellarRpcUrl: 'https://soroban-testnet.stellar.org:443',
stellarNetworkPassphrase: 'Test SDF Network ; September 2015',
pollIntervalMs: 30000,
maxReconnectAttempts: 5,
reconnectDelayMs: 5000,
eventsApiPort: 8787,
eventsApiCorsOrigin: 'http://localhost:5173',
contractAddresses: [{ address: 'CABC', events: ['*'] }],
scheduler: {
enabled: true,
pollIntervalMs: 10000,
lockTimeoutMs: 60000,
batchSize: 10,
timingBufferMs: 60000,
},
rateLimit: {
enabled: true,
windowMs: 60000,
maxRequests: 60,
},
analytics: {
enabled: true,
maxRecords: 10000,
maxBuckets: 168,
bucketSizeMs: 3600000,
persistIntervalMs: 300000,
snapshotRetentionDays: 30,
},
cleanup: {
intervalMs: 3600000,
notificationRetentionMs: 604800000,
rateLimitEventRetentionMs: 86400000,
eventRetentionMs: 86400000,
executionLogRetentionMs: 7776000000,
},
};

it('passes valid configuration against full application schema', () => {
const errors = ConfigurationSchemaValidator.validate(sampleValidConfig, APP_CONFIG_SCHEMA);
expect(errors).toHaveLength(0);
});

it('validates required fields and reports field path', () => {
const invalidConfig = { ...sampleValidConfig, stellarRpcUrl: undefined };
const errors = ConfigurationSchemaValidator.validate(invalidConfig, APP_CONFIG_SCHEMA);
expect(errors.length).toBeGreaterThan(0);
expect(errors.some((e) => e.field === 'stellarRpcUrl' && e.message.includes('missing'))).toBe(true);
});

it('validates explicit types', () => {
const invalidConfig = { ...sampleValidConfig, pollIntervalMs: '30000' as any };
const errors = ConfigurationSchemaValidator.validate(invalidConfig, APP_CONFIG_SCHEMA);
expect(errors.some((e) => e.field === 'pollIntervalMs' && e.message.includes('must be of type number'))).toBe(true);
});

it('validates numeric minimum and maximum bounds', () => {
const invalidConfig = {
...sampleValidConfig,
eventsApiPort: 70000,
pollIntervalMs: 50,
};
const errors = ConfigurationSchemaValidator.validate(invalidConfig, APP_CONFIG_SCHEMA);
expect(errors.some((e) => e.field === 'eventsApiPort' && e.message.includes('exceeds maximum 65535'))).toBe(true);
expect(errors.some((e) => e.field === 'pollIntervalMs' && e.message.includes('less than minimum 1000'))).toBe(true);
});

it('validates enumerated values', () => {
const invalidConfig = { ...sampleValidConfig, stellarNetwork: 'invalidnet' };
const errors = ConfigurationSchemaValidator.validate(invalidConfig, APP_CONFIG_SCHEMA);
expect(errors.some((e) => e.field === 'stellarNetwork' && e.message.includes('Allowed values'))).toBe(true);
});

it('validates nested schema fields', () => {
const invalidConfig = {
...sampleValidConfig,
scheduler: {
...sampleValidConfig.scheduler,
pollIntervalMs: 100, // below 1000 min
},
};
const errors = ConfigurationSchemaValidator.validate(invalidConfig, APP_CONFIG_SCHEMA);
expect(errors.some((e) => e.field === 'scheduler.pollIntervalMs')).toBe(true);
});

it('validates custom regex patterns', () => {
const invalidConfig = { ...sampleValidConfig, stellarRpcUrl: 'ftp://not-http.stellar.org' };
const errors = ConfigurationSchemaValidator.validate(invalidConfig, APP_CONFIG_SCHEMA);
expect(errors.some((e) => e.field === 'stellarRpcUrl' && e.message.includes('pattern'))).toBe(true);
});
});
Loading