Add PDU mode for concatenated multi-segment SMS

CLAUDE.md

@@ -1,182 +1,112 @@
 # CLAUDE.md
 
+Guidance for Claude Code when working in this repository.
+
 ## Project Overview
 
-ECHO (Enhanced Cellular Handling Operations) is the OASIS modem daemon for the SIM7600G-H 4G modem. It owns the serial port, handles all AT command traffic, publishes telemetry and events via MQTT, and receives commands from DAWN.
+ECHO (Enhanced Cellular Handling Operations) is the OASIS modem daemon for the SIM7600G-H 4G modem. It owns the serial port, handles all AT command traffic, publishes telemetry and events via MQTT, and receives commands from DAWN. Template: STAT (system telemetry daemon).
+
+See @ARCHITECTURE.md for subsystem details and @README.md for deployment context.
 
-Part of The OASIS Project. Template: STAT (system telemetry daemon).
+## Critical Rules — Always Follow
 
-## Building
+- **NEVER delete files.** Tell the developer which files to delete.
+- **NEVER run `git add`, `git commit`, or `git push`.** Suggest the command and message; let the developer run it.
+- **Feedback before implementation.** Provide analysis, trade-offs, and a recommendation *first*. Wait for explicit confirmation ("go ahead", "do it", "yes") before coding.
+- **Format before committing.** Every change must pass `./format_code.sh --check`. Pre-commit hook enforces this.
+- **GPL header on every new `.c`/`.h`.** Template in @CODING_STYLE_GUIDE.md.
+- **Design doc commit policy**: commit design docs only when they describe shipped or in-flight code. Docs for planned-but-unstarted work stay untracked.
+
+## Build & Test
 
 ```bash
-# Configure and build
+# Build
 cmake -B build -DCMAKE_BUILD_TYPE=Debug
 make -C build -j8
 
-# Run tests
+# Run tests (Unity framework, runs without hardware)
 ctest --test-dir build --output-on-failure
 
-# Run individual test
-./build/tests/test_sms
+# Format
+./format_code.sh                 # fix
+./format_code.sh --check         # CI mode
+./format_code.sh --changed       # only changed files
 ```
 
-### Dependencies
-- `libmosquitto` — MQTT client library
-- `json-c` — JSON construction and parsing
-- `pthread` — threading (system)
-- Unity (vendored in `tests/unity/`) — unit test framework
-
-## Code Formatting
-
-**MANDATORY**: All code MUST be formatted before committing.
-
-```bash
-# Format all code (run from repository root)
-./format_code.sh
-
-# Check formatting without modifying files
-./format_code.sh --check
+- Dependencies: `libmosquitto`, `json-c`, `pthread`, Unity (vendored).
+- Requires `clang-format-14` for format checks.
+- Pre-commit hook: `./install-git-hooks.sh` (one-time).
 
-# Format only changed files (fast)
-./format_code.sh --changed
-```
+## Code Standards
 
-Requires `clang-format-14`. Install: `sudo apt-get install clang-format-14`
+Full standards in @CODING_STYLE_GUIDE.md. Critical gotchas specific to ECHO:
 
-### Git Hooks
-Install the pre-commit hook to automatically check formatting:
-```bash
-./install-git-hooks.sh
-```
+- **Return codes**: `SUCCESS` (0) / `FAILURE` (1) — never negative. Specific error codes > 1.
+- **Logging**: use `OLOG_INFO` / `OLOG_WARNING` / `OLOG_ERROR` (ECHO's convention).
+- **Naming**: `snake_case` functions/vars, `UPPER_CASE` constants, `_t` suffix on types.
+- **Memory**: prefer static allocation; null-check after malloc; `free(ptr); ptr = NULL;`.
+- **Functions**: soft target < 50 lines, inputs first / outputs last.
 
-## Architecture
+## Threading (hard constraints)
 
-### Threading Model
+Three threads — know which one you're in:
 
-Three threads:
-- **Main thread**: Command queue drain, telemetry polling (10s), heartbeat (30s)
-- **URC reader thread**: Blocking serial reads, line parsing, URC classification, AT response delivery via condvar
-- **Mosquitto thread**: Network I/O, message callbacks (queues commands to main thread)
+- **Main thread**: command-queue drain, telemetry polling (10s), heartbeat (30s).
+- **URC reader thread**: blocking serial reads, line parsing, URC classification, AT response delivery via condvar.
+- **Mosquitto thread**: network I/O, message callbacks (queues commands to main thread).
 
-### Key Design Decisions
+**Never call `at_command_send()` from the URC reader thread.** Use the command queue. Doing so deadlocks the condvar.
 
-- **Single serial reader**: URC reader owns ALL reads. Main thread only writes AT commands.
-- **Command queue**: MQTT commands queued to lock-free SPSC ring buffer, drained by main thread. Prevents blocking mosquitto's event loop.
-- **Deferred CMTI**: SMS reads queued from URC thread to main thread to avoid condvar deadlock.
-- **Atomic call state**: `__atomic` builtins for `g_call_state` (3 threads access it).
+- Single serial reader: URC reader owns **all** reads. Main thread only writes AT commands.
+- CMTI handling: SMS reads queued from URC → main thread to avoid deadlock.
+- Call state: `__atomic` builtins on `g_call_state` (3 threads touch it).
 
-### AT Command Types
+## AT Command Types
 
 | Type | Function | Behavior |
 |------|----------|----------|
 | Sync | `at_command_send()` | Block on condvar until OK/ERROR/timeout |
 | Async | `at_command_send_async()` | Write and return; result comes as URC |
 | SMS | `at_command_send_sms()` | Two-phase: wait for `>` prompt, then body+Ctrl-Z |
 
-### MQTT Topics
+## MQTT Topics
 
 | Topic | Dir | Content |
 |-------|-----|---------|
 | `echo/telemetry` | out | Signal, network, call state (every 10s) |
 | `echo/events` | out | Incoming call, SMS, call ended |
-| `echo/response` | out | Command responses with request_id |
+| `echo/response` | out | Command responses with `request_id` |
 | `echo/status` | out | Online/offline (LWT) |
 | `echo/cmd` | in | Commands from DAWN |
 
-All messages conform to OCP v1.3.
-
-## Coding Standards
-
-Follow `CODING_STYLE_GUIDE.md` strictly:
-
-**Naming**: `snake_case` functions/variables, `UPPER_CASE` constants, `_t` suffix on types.
-
-**Error Handling**: Return 0 on success. Always check return values. Log with `OLOG_ERROR()`.
-
-**Memory**: Prefer static allocation. Minimize malloc. Free and NULL.
-
-**File Headers**: GPL license block required on all `.c` and `.h` files (see CODING_STYLE_GUIDE.md).
-
-**Functions**: Soft target < 50 lines. Inputs first, outputs last.
-
-**Threading**: Never call `at_command_send()` from the URC reader thread. Use the command queue.
-
-## Important Files
-
-**Source modules:**
-- `src/oasis-echo.c` — Main entry, command queue, URC event dispatch, MQTT command processor
-- `src/at_command.c` — Serial I/O with flock, sync/async/SMS AT commands, terminator parsing
-- `src/urc_handler.c` — URC reader thread, classification, RING+CLIP merge
-- `src/modem.c` — Init sequence, signal polling, telemetry builder, echo cancellation
-- `src/mqtt_comms.c` — MQTT lifecycle, json-c JSON builders, command parser
-- `src/sms.c` — Phone number validation, SMS body sanitization, CLIP sanitization
-- `src/logging.c` — Logging (copied from STAT)
-
-**Headers:**
-- `include/echo.h` — Global types, config struct, call/reg/SIM enums, rate bucket
-- `include/at_command.h` — AT context, response, pending state types
-- `include/urc_handler.h` — URC event types, callback, context
-- `include/modem.h` — Modem init, polling, telemetry builder
-- `include/mqtt_comms.h` — MQTT topics, publish/subscribe/parse API
-- `include/sms.h` — Validation and sanitization API
-
-**Configuration:**
-- `config/echo.conf` — MQTT credentials, serial port, rate limits (systemd EnvironmentFile)
-- `config/oasis-echo.service` — systemd service unit
-- `config/sim7600-rndis.service` — RNDIS data path boot service
-- `scripts/sim7600-rndis-up.sh` — RNDIS activation script
-
-**Tooling:**
-- `.clang-format` — clang-format-14 config (matches DAWN)
-- `format_code.sh` — Format all code (adapted from DAWN)
-- `pre-commit.hook` — Git pre-commit formatting check
-- `install-git-hooks.sh` — Hook installer
-- `.github/workflows/ci.yml` — CI: format-check + build + tests
-
-## Testing
-
-Unity framework (vendored in `tests/unity/`, MIT license). Four test modules:
-
-| Test | Assertions | What it covers |
-|------|-----------|---------------|
-| `test_at_command` | 14 | Response terminator parsing, status strings |
-| `test_sms` | 24 | Phone number validation, body sanitization, CLIP sanitization |
-| `test_urc_handler` | 22 | URC classification, RING+CLIP merge, VOICE CALL URCs |
-| `test_mqtt_messages` | 16 | Telemetry/event/response JSON, command parsing |
-
-Tests link against specific source files (not the full daemon binary), so they run without hardware or an MQTT broker.
-
-```bash
-# Build and run all tests
-cmake -B build -DCMAKE_BUILD_TYPE=Debug && make -C build -j8
-ctest --test-dir build --output-on-failure
-```
+All messages conform to OCP v1.4 (`ocp_get_timestamp_ms()` for ms timestamps, `msg_type` field on every message).
 
 ## SIM7600 Hardware Notes
 
 Discoveries from live hardware testing:
 
-- Modem kept in default UCS2 charset — `AT+CSMP=17,167,0,8` sets DCS=8 to tell the network body is UCS2-encoded. Enables full Unicode/emoji SMS.
-- Phone numbers and SMS bodies are UCS2 hex-encoded for `AT+CMGS` and decoded from `AT+CMGR` responses. CLIP and ATD use plain ASCII.
-- `AT+CPMS="ME","ME","ME"` required — default SMS read storage is "SR" (status reports)
-- `AT+CHUP` for hangup instead of `ATH` — works reliably in all call states
-- `AT+CECM=1` only works during active calls — sent per-call, not at init
-- `VOICE CALL: BEGIN` / `VOICE CALL: END` are SIM7600-specific URCs (not standard `CONNECT`)
-- Modem sends `VOICE CALL: END` + `NO CARRIER` back-to-back — duplicate suppressed in event handler
+- Modem kept in default UCS2 charset — `AT+CSMP=17,167,0,8` sets DCS=8. Enables full Unicode/emoji SMS.
+- Phone numbers and SMS bodies are UCS2 hex-encoded for `AT+CMGS` and decoded from `AT+CMGR`. CLIP and ATD use plain ASCII.
+- `AT+CPMS="ME","ME","ME"` required — default SMS read storage is "SR" (status reports).
+- `AT+CHUP` for hangup, not `ATH` — works reliably in all call states.
+- `AT+CECM=1` only works during active calls — sent per-call, not at init.
+- `VOICE CALL: BEGIN` / `VOICE CALL: END` are SIM7600-specific URCs (not standard `CONNECT`).
+- Modem sends `VOICE CALL: END` + `NO CARRIER` back-to-back — duplicate suppressed in event handler.
+- Current firmware (`LE20B04SIM7600G22`) does **not** include MMS AT commands. See `~/code/The-OASIS-Project/dawn/docs/UNIFIED_IMAGE_STORE_DESIGN.md` §Phase 4 for unblock paths.
 
 ## Development Lifecycle
 
-1. **Implement**: Build and format check after each chunk: `make -C build -j8` + `./format_code.sh --check`
-2. **Test**: Run `ctest --test-dir build --output-on-failure`
-3. **Review**: Run review agents on the diff (architecture-reviewer, embedded-efficiency-reviewer, security-auditor)
-4. **Manual test**: Verify on live hardware if touching AT commands, URC handling, or MQTT
-5. **Format**: `./format_code.sh`
-6. **Commit**: Provide `git add` + commit message to developer (never run git commands directly)
+1. **Implement** — build + format check after each chunk: `make -C build -j8` + `./format_code.sh --check`.
+2. **Test** — `ctest --test-dir build --output-on-failure`.
+3. **Review** — run review agents on the diff (architecture-reviewer, embedded-efficiency-reviewer, security-auditor).
+4. **Manual test** — verify on live hardware if touching AT commands, URC handling, or MQTT.
+5. **Format** — `./format_code.sh` one final time.
+6. **Commit** — provide `git add` + commit message; **developer runs git commands**.
 
-## Design Document
+## Design Documents
 
-Single source of truth: `~/code/The-OASIS-Project/dawn/docs/PHONE_SMS_DESIGN.md`
+Phone/SMS integration design: `~/code/The-OASIS-Project/dawn/docs/PHONE_SMS_DESIGN.md`.
 
 ## License
 
-GPLv3 or later. All source files include GPL header block.
+GPLv3 or later. Every new source file includes the GPL header block.

CMakeLists.txt

@@ -34,7 +34,10 @@ set(SOURCES
    src/modem.c
    src/mqtt_comms.c
    src/oasis-echo.c
+   src/pdu.c
    src/sms.c
+   src/sms_io.c
+   src/sms_reassembly.c
    src/urc_handler.c
 )
 
@@ -93,4 +96,16 @@ if(BUILD_TESTS)
                          ${MOSQUITTO_LIBRARIES})
    target_include_directories(test_mqtt_messages PRIVATE include ${JSONC_INCLUDE_DIRS})
    add_test(NAME test_mqtt_messages COMMAND test_mqtt_messages)
+
+   # test_pdu — PDU encode/decode + malformed-input rejection
+   add_executable(test_pdu tests/test_pdu.c src/pdu.c)
+   target_link_libraries(test_pdu unity echo_logging pthread)
+   target_include_directories(test_pdu PRIVATE include)
+   add_test(NAME test_pdu COMMAND test_pdu)
+
+   # test_sms_reassembly — LRU, per-sender cap, duplicate handling
+   add_executable(test_sms_reassembly tests/test_sms_reassembly.c src/sms_reassembly.c)
+   target_link_libraries(test_sms_reassembly unity echo_logging)
+   target_include_directories(test_sms_reassembly PRIVATE include)
+   add_test(NAME test_sms_reassembly COMMAND test_sms_reassembly)
 endif()

include/at_command.h

@@ -130,6 +130,28 @@ at_status_t at_command_send_sms(at_context_t *ctx,
                                 const char *body,
                                 at_response_t *response);
 
+/**
+ * @brief Send one PDU-mode SMS segment.
+ *
+ * Two-phase AT+CMGS with the `<octets>` argument, then hex PDU + Ctrl-Z:
+ *   Phase 1: AT+CMGS=<tpdu_octets>\r  →  wait for '>' prompt.
+ *   Phase 2: <pdu_hex>\x1A             →  wait for +CMGS / OK / ERROR.
+ *
+ * The hex string is validated for the hex alphabet before transmission —
+ * an accidental non-hex byte sent in this mode triggers CMS ERROR 305 at
+ * best and unpredictable modem state at worst.
+ *
+ * @param ctx         AT context.
+ * @param tpdu_octets Length argument for AT+CMGS (TPDU only, not SMSC prefix).
+ * @param pdu_hex     Full hex payload, including the "00" SMSC-default prefix.
+ * @param response    Output response.
+ * @return AT_OK or a failure status.
+ */
+at_status_t at_command_send_pdu(at_context_t *ctx,
+                                int tpdu_octets,
+                                const char *pdu_hex,
+                                at_response_t *response);
+
 /**
  * @brief Write raw bytes to the serial port (thread-safe).
  * @return Number of bytes written, or -1 on error.

include/echo.h

@@ -40,12 +40,27 @@
 #define ECHO_DEFAULT_TELEMETRY_S 10
 #define ECHO_DEFAULT_RATE_CALLS_H 5
 #define ECHO_DEFAULT_RATE_SMS_H 20
+/* Per-segment rate bucket for PDU mode. A single concat SMS can burn up to
+ * PDU_MAX_SEGMENTS airtime units, so we budget this separately from the
+ * per-message rate so a chatty user can't exhaust the network quota. */
+#define ECHO_DEFAULT_RATE_SEGMENTS_H 200
+/* Inter-segment pacing. T-Mobile + SIM7600 can wedge on back-to-back
+ * concat sends without a breather. 150ms is gentle on both. */
+#define ECHO_DEFAULT_SEGMENT_DELAY_MS 150
 
 /* AT command limits */
 #define AT_RESPONSE_MAX 4096    /* large enough for UCS2 hex SMS bodies */
 #define AT_TIMEOUT_DEFAULT 2000 /* ms */
 #define AT_TIMEOUT_SMS 60000    /* ms — AT+CMGS waits for network */
 #define AT_TIMEOUT_DIAL 5000    /* ms — ATD returns quickly, result comes as URC */
+/* Inbound SMS storage ops (CMGR read, CMGD delete) hit local modem memory
+ * and normally return <100ms. A short timeout here matters because a
+ * multi-segment SMS fires one CMTI per segment; at the 2s default, a 10-
+ * segment message could block the main-thread command-queue drain for up
+ * to 40s. 500ms caps that at ~10s and still leaves margin over real-world
+ * modem latency. On timeout we fall back to logging + CMGD-fire-forget so
+ * the inbox doesn't fill. */
+#define AT_TIMEOUT_SMS_STORAGE 500
 
 /* SMS limits */
 #define SMS_BODY_MAX 800
@@ -107,26 +122,39 @@ typedef struct {
    int telemetry_interval_s;
    int rate_limit_calls_per_hour;
    int rate_limit_sms_per_hour;
+   int rate_limit_segments_per_hour;
+   int inter_segment_delay_ms;
+   bool pdu_mode; /* true = PDU (AT+CMGF=0), false = legacy text mode */
    bool service_mode;
 } echo_config_t;
 
-/* Rate limiter bucket */
+/* Leaky-bucket rate limiter. Fills at `max_per_hour / 3600` tokens/sec up to
+ * a `max_per_hour` ceiling; a `take_n()` spends N tokens atomically.
+ *
+ * Replaces an earlier ring-buffer design that silently capped active count
+ * at 64 — the old limiter never rejected anything when `max_per_hour > 64`.
+ * The counter form is correct at any configured limit and O(1) per call. */
 typedef struct {
-   int64_t timestamps[64]; /* ring buffer of event timestamps (epoch seconds) */
-   int head;               /* next write position */
-   int count;              /* events in current window */
-   int max_per_hour;       /* configured limit */
+   double tokens;           /* current token balance (fractional) */
+   int64_t last_refill_sec; /* wall clock seconds of last refill */
+   int max_per_hour;        /* bucket ceiling + refill rate input */
 } rate_bucket_t;
 
 /**
- * @brief Initialize a rate limiter bucket.
+ * @brief Initialize a rate limiter bucket, pre-filled to capacity.
  */
 void rate_bucket_init(rate_bucket_t *bucket, int max_per_hour);
 
 /**
- * @brief Check if an action is allowed and record it if so.
+ * @brief Try to consume one token; record it if available.
  * @return true if allowed, false if rate limited.
  */
 bool rate_bucket_allow(rate_bucket_t *bucket);
 
+/**
+ * @brief Try to consume `n` tokens atomically (no partial debit on failure).
+ * @return true if all N allowed, false if insufficient balance.
+ */
+bool rate_bucket_take_n(rate_bucket_t *bucket, int n);
+
 #endif /* ECHO_H */

include/modem.h

@@ -32,14 +32,17 @@
 /**
  * @brief Run the modem initialization sequence.
  *
- * Sends: AT, ATE0, AT+CMEE=2, AT+CLIP=1, AT+CMGF=1,
- * AT+CNMI=2,1,0,0,0, AT+CREG=1, AT+CSDVC=1, AT+CLVL=3,
- * AT+CECM=1, AT+CSQ, AT+COPS?
+ * Sends: AT, ATE0, AT+CMEE=2, AT+CLIP=1, AT+CMGF=0 (PDU) or 1 (text),
+ * AT+CNMI=2,1,0,0,0, AT+CREG=1, AT+CSDVC=1, AT+CLVL=3, AT+CSQ, AT+COPS?
  *
- * @param at AT context with open serial port.
+ * AT+CSMP (SMS parameters) is only sent in text mode — PDU mode carries the
+ * DCS per-frame so the init-time default is irrelevant.
+ *
+ * @param at       AT context with open serial port.
+ * @param pdu_mode true → AT+CMGF=0 (PDU), false → AT+CMGF=1 (text).
  * @return 0 on success (basic AT works), -1 on failure.
  */
-int modem_init(at_context_t *at);
+int modem_init(at_context_t *at, bool pdu_mode);
 
 /**
  * @brief Poll signal strength (AT+CSQ).