From 3b5cd326cdb8d932471ccd4ea46be71389911ec2 Mon Sep 17 00:00:00 2001 From: Leynos Date: Fri, 10 Oct 2025 19:48:17 +0100 Subject: [PATCH 1/3] Update documentation-style-guide.md --- docs/documentation-style-guide.md | 53 +++++++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/docs/documentation-style-guide.md b/docs/documentation-style-guide.md index b77a43786..129f048db 100644 --- a/docs/documentation-style-guide.md +++ b/docs/documentation-style-guide.md @@ -6,14 +6,25 @@ these rules to keep the documentation clear and consistent for developers. ## Spelling - Use British English based on the - [Oxford English Dictionary](https://public.oed.com/) (en-oxendict). + [Oxford English Dictionary](https://public.oed.com/) (en-GB-oxendict): + - suffix -ize in words like _realize_ and _organization_ instead of + -ise endings, + - suffix ‑lyse in words not traced to the Greek ‑izo, ‑izein suffixes, + such as _analyse_, _paralyse_ and _catalyse_, + - suffix -our in words such as _colour_, _behaviour_ and _neighbour_, + - suffix -re in words such as _calibre_, _centre_ and fibre, + - double "l" in words such as _cancelled_, _counsellor_ and _cruellest_, + - maintain the "e" in words such as _likeable_, _liveable_ and _rateable_, + - suffix -ogue in words such as _analogue_ and _catalogue_, + - and so forth. - The word **"outwith"** is acceptable. - Keep US spelling when used in an API, for example `color`. - The project licence file is spelled `LICENSE` for community consistency. ## Punctuation and grammar -- Use the Oxford comma: "ships, planes, and hovercraft". +- Use the Oxford comma: "ships, planes, and hovercraft" where it aids + comprehension. - Company names are treated as collective nouns: "Lille Industries are expanding". @@ -108,4 +119,42 @@ flowchart TD C --> D[Merge] ``` +## Roadmap Task Writing Guidelines + +When documenting development roadmap items, write them so that they are +achievable, measurable, and structured. This ensures the roadmap functions as a +practical planning tool rather than a vague wishlist. Do not commit to +timeframes in the roadmap. Development effort should be roughly consistent from +task to task. + +### Principles for Roadmap Tasks + +- Define outcomes, not intentions: Phrase tasks in terms of the capability + delivered (e.g. “Implement role-based access control for API endpoints”), not + aspirations like “Improve security”. +- Quantify completion criteria: Attach measurable finish lines (e.g. “90% + test coverage for new modules”, “response times under 200ms”, “all endpoints + migrated”). +- Break into atomic increments: Ensure tasks can be completed in weeks, not + quarters. Large goals should be decomposed into clear, deliverable units. +- Tie to dependencies and sequencing: Document prerequisites so tasks can be + scheduled realistically (e.g. “Introduce central logging service” before “Add + error dashboards”). +- Bound scope explicitly: Note both in-scope and out-of-scope elements (e.g. + “Build analytics dashboard (excluding churn prediction)”). + +### Hierarchy of Scope + +Roadmaps should be expressed in three layers of scope to maintain clarity and +navigability: + +- Phases (strategic milestones) – Broad outcome-driven stages that represent + significant capability shifts. Why the work matters. +- Steps (epics / workstreams) – Mid-sized clusters of related tasks grouped + under a phase. What will be built. +- Tasks (execution units) – Small, measurable pieces of work with clear + acceptance criteria. How it gets done. + +______________________________________________________________________ + [^markdownlint]: A linter that enforces consistent Markdown formatting. From 1d8d54e219e1e693a558d7b5d89267be2b25b1fc Mon Sep 17 00:00:00 2001 From: Leynos Date: Sat, 11 Oct 2025 10:08:59 +0100 Subject: [PATCH 2/3] Add checklists to pagination roadmap tasks --- docs/keyset-pagination-roadmap.md | 160 ++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 docs/keyset-pagination-roadmap.md diff --git a/docs/keyset-pagination-roadmap.md b/docs/keyset-pagination-roadmap.md new file mode 100644 index 000000000..bf0312c4e --- /dev/null +++ b/docs/keyset-pagination-roadmap.md @@ -0,0 +1,160 @@ +# Roadmap: keyset pagination crate for Wildside backend + +## Phase 1 – Establish pagination core + +### Step 1.1 – Define cursor primitives and encoding +- **Tasks** + - [ ] Implement `Direction` enum and `Cursor` struct covering forward and + backward traversal semantics. + - [ ] Provide base64url JSON encoding/decoding helpers with explicit error + handling and round-trip tests across representative key payloads. +- **Outcome** + - Library consumers can construct, encode, and decode opaque cursor tokens + safely, enabling consistent pagination state transfer. +- **Dependencies** + - None. + +### Step 1.2 – Model pagination keys and traits +- **Tasks** + - [ ] Define marker trait or helper utilities that bind endpoint-specific key + structs to cursor generation while keeping the crate generic. + - [ ] Supply example implementations for composite keys, including `(created_at, + id)` for users, highlighting index expectations in documentation. +- **Outcome** + - Endpoints can declare stable ordering keys with clear integration guidance + and compiler enforcement. +- **Dependencies** + - Completion of Step 1.1 to reuse cursor types. + +### Step 1.3 – Craft response envelope types +- **Tasks** + - [ ] Implement `PaginationLinks` and `Paginated` structs matching the design + envelope, including `self_`, `next`, and `prev` link fields. + - [ ] Derive serialization and OpenAPI schema traits, ensuring optional links are + omitted when unavailable and the `limit` guardrail is documented. +- **Outcome** + - Shared response container guarantees consistent pagination metadata across + endpoints. +- **Dependencies** + - Step 1.1 for cursor reuse in link composition tests. + +## Phase 2 – Request handling utilities + +### Step 2.1 – Build query parameter extractor +- **Tasks** + - [ ] Introduce `PageParams` with optional `cursor` and `limit` values, plus + helper methods enforcing defaults, maximums, and explicit validation errors. + - [ ] Add exhaustive unit tests for parameter parsing edge cases (missing limit, + non-numeric values, oversized requests). +- **Outcome** + - Actix handlers receive validated pagination inputs with predictable + defaults. +- **Dependencies** + - Step 1.1 to decode cursors during validation. + +### Step 2.2 – Provide pagination context helpers +- **Tasks** + - [ ] Offer builder or utility functions that encapsulate limit selection, + directional intent, and cursor decoding to reduce boilerplate inside + handlers. + - [ ] Document expected handler workflow, including how to compute boundaries and + propagate cursors into link URLs. +- **Outcome** + - Crate consumers follow a consistent integration sequence with minimal + repeated logic. +- **Dependencies** + - Steps 1.1 and 2.1 for cursor and parameter primitives. + +## Phase 3 – Database integration patterns + +### Step 3.1 – Implement Diesel query adapters +- **Tasks** + - [ ] Supply helper functions or macros to apply lexicographic filters for + `Next`/`Prev` cursors against Diesel query builders, targeting composite key + comparisons. + - [ ] Provide guidance for both synchronous and `diesel_async` contexts with + example queries that fetch `limit + 1` results. +- **Outcome** + - Query construction becomes predictable and less error-prone when applying + cursor-based filtering. +- **Dependencies** + - Steps 1.1, 1.2, and 2.2 to understand key semantics and handler context. + +### Step 3.2 – Package boundary evaluation utilities +- **Tasks** + - [ ] Implement helper routines that accept fetched records, trim the extra item, + determine presence of additional pages, and generate direction-appropriate + cursors. + - [ ] Support both forward and backward navigation paths with deterministic unit + tests covering off-by-one and empty-page scenarios. +- **Outcome** + - Handlers can transform raw query results into paginated responses without + duplicating edge-case logic. +- **Dependencies** + - Steps 1.1 through 3.1 for cursor structures and query patterns. + +## Phase 4 – Actix-web integration and ergonomics + +### Step 4.1 – Compose HTTP response builders +- **Tasks** + - [ ] Provide utility functions that assemble `Paginated` responses, including + URL construction for `self`, `next`, and `prev` links while preserving + explicit `limit` selections. + - [ ] Offer extension traits or helper methods to combine request context, cursor + strings, and route metadata. +- **Outcome** + - Actix endpoints can emit complete hypermedia envelopes with minimal + boilerplate and consistent link formatting. +- **Dependencies** + - Steps 1.3, 2.2, and 3.2 for link metadata and boundary evaluation. + +### Step 4.2 – Update `/api/users` handler as reference implementation +- **Tasks** + - [ ] Refactor the existing users listing endpoint to consume the crate utilities + end-to-end, ensuring compatibility with async Diesel queries and existing + response schema expectations. + - [ ] Capture integration tests validating cursor navigation across forward and + backward flows, including limit enforcement. +- **Outcome** + - Demonstrated end-to-end adoption validates the crate and serves as a living + example for other endpoints. +- **Dependencies** + - Completion of Phases 1 through 3 to supply the supporting APIs. + +## Phase 5 – Quality, tooling, and documentation + +### Step 5.1 – Harden error handling and observability +- **Tasks** + - [ ] Standardise error types returned by the crate, mapping decoding, validation, + and database misuse issues to actionable messages. + - [ ] Emit structured logs or tracing spans for pagination failures, ensuring + privacy by excluding cursor contents. +- **Outcome** + - Operational visibility improves while keeping cursor tokens opaque and + secure. +- **Dependencies** + - Steps 1.1 through 4.1 for context around error surfaces. + +### Step 5.2 – Author documentation and migration guidance +- **Tasks** + - [ ] Produce crate-level README and API docs summarising configuration, usage + examples, and extension points for other models (POIs, routes, etc.). + - [ ] Draft migration checklist for future endpoints, covering index requirements + and testing expectations. +- **Outcome** + - Teams adopting the crate have authoritative guidance and know how to verify + their integrations. +- **Dependencies** + - Completion of functional work to document accurately. + +### Step 5.3 – Finalise release readiness +- **Tasks** + - [ ] Ensure lint, format, and test automation covers the crate via Makefile + targets, updating CI configurations if required. + - [ ] Add changelog entry describing the new pagination capabilities and their + scope limitations (no total counts, opaque cursors only). +- **Outcome** + - Pagination crate is production-ready with tooling support and transparent + release notes. +- **Dependencies** + - Prior roadmap steps to deliver the implementation and tests. From f40967ab3af38c9c34f92ae4f831ae24f1546467 Mon Sep 17 00:00:00 2001 From: Leynos Date: Sat, 11 Oct 2025 13:10:47 +0100 Subject: [PATCH 3/3] Document pagination roadmap test coverage --- docs/keyset-pagination-roadmap.md | 137 ++++++++++++++++++++++-------- 1 file changed, 100 insertions(+), 37 deletions(-) diff --git a/docs/keyset-pagination-roadmap.md b/docs/keyset-pagination-roadmap.md index bf0312c4e..f35261068 100644 --- a/docs/keyset-pagination-roadmap.md +++ b/docs/keyset-pagination-roadmap.md @@ -6,8 +6,13 @@ - **Tasks** - [ ] Implement `Direction` enum and `Cursor` struct covering forward and backward traversal semantics. - - [ ] Provide base64url JSON encoding/decoding helpers with explicit error - handling and round-trip tests across representative key payloads. + - [ ] Provide base64url JSON encoding and decoding helpers with explicit + error handling across representative key payloads. +- **Testing** + - [ ] Unit: Round-trip encode and decode sample payloads, inject malformed + inputs, and assert precise error variants. + - [ ] Behavioural: Exercise a thin Actix endpoint that echoes cursors through + request and response cycles to prove stable token handling. - **Outcome** - Library consumers can construct, encode, and decode opaque cursor tokens safely, enabling consistent pagination state transfer. @@ -16,10 +21,16 @@ ### Step 1.2 – Model pagination keys and traits - **Tasks** - - [ ] Define marker trait or helper utilities that bind endpoint-specific key + - [ ] Define marker traits or helper utilities that bind endpoint-specific key structs to cursor generation while keeping the crate generic. - - [ ] Supply example implementations for composite keys, including `(created_at, - id)` for users, highlighting index expectations in documentation. + - [ ] Supply example implementations for composite keys, including + `(created_at, id)` for users, highlighting index expectations in + documentation. +- **Testing** + - [ ] Unit: Validate trait blanket implementations and ensure compile-time + guarantees for required ordering fields. + - [ ] Behavioural: Demonstrate trait usage inside a sample handler pipeline + that constructs cursors for multiple key types. - **Outcome** - Endpoints can declare stable ordering keys with clear integration guidance and compiler enforcement. @@ -28,10 +39,15 @@ ### Step 1.3 – Craft response envelope types - **Tasks** - - [ ] Implement `PaginationLinks` and `Paginated` structs matching the design - envelope, including `self_`, `next`, and `prev` link fields. - - [ ] Derive serialization and OpenAPI schema traits, ensuring optional links are - omitted when unavailable and the `limit` guardrail is documented. + - [ ] Implement `PaginationLinks` and `Paginated` structs matching the + design envelope, including `self_`, `next`, and `prev` link fields. + - [ ] Derive serialization and OpenAPI schema traits, ensuring optional links + are omitted when unavailable and the `limit` guardrail is documented. +- **Testing** + - [ ] Unit: Snapshot serialize representative payloads and assert link + omission rules under varying cursor availability. + - [ ] Behavioural: Exercise generated envelopes through a mock Actix response + to verify OpenAPI documentation and runtime fields align. - **Outcome** - Shared response container guarantees consistent pagination metadata across endpoints. @@ -43,9 +59,15 @@ ### Step 2.1 – Build query parameter extractor - **Tasks** - [ ] Introduce `PageParams` with optional `cursor` and `limit` values, plus - helper methods enforcing defaults, maximums, and explicit validation errors. - - [ ] Add exhaustive unit tests for parameter parsing edge cases (missing limit, - non-numeric values, oversized requests). + helper methods enforcing defaults, maximums, and explicit validation + errors. + - [ ] Document parser failure modes so handlers surface actionable guidance to + clients. +- **Testing** + - [ ] Unit: Cover parsing edge cases, including missing limits, non-numeric + values, oversized requests, and double cursor submission. + - [ ] Behavioural: Drive the extractor through Actix integration tests to + confirm HTTP 400 responses and error payload structure. - **Outcome** - Actix handlers receive validated pagination inputs with predictable defaults. @@ -57,8 +79,13 @@ - [ ] Offer builder or utility functions that encapsulate limit selection, directional intent, and cursor decoding to reduce boilerplate inside handlers. - - [ ] Document expected handler workflow, including how to compute boundaries and - propagate cursors into link URLs. + - [ ] Document expected handler workflow, including how to compute boundaries + and propagate cursors into link URLs. +- **Testing** + - [ ] Unit: Validate helper logic for forward and backward navigation, + including limit coercion and cursor canonicalization. + - [ ] Behavioural: Exercise helpers inside a mock handler to confirm the + integration sequence and error propagation. - **Outcome** - Crate consumers follow a consistent integration sequence with minimal repeated logic. @@ -70,10 +97,15 @@ ### Step 3.1 – Implement Diesel query adapters - **Tasks** - [ ] Supply helper functions or macros to apply lexicographic filters for - `Next`/`Prev` cursors against Diesel query builders, targeting composite key - comparisons. + `Next` and `Prev` cursors against Diesel query builders, targeting + composite key comparisons. - [ ] Provide guidance for both synchronous and `diesel_async` contexts with example queries that fetch `limit + 1` results. +- **Testing** + - [ ] Unit: Validate generated predicates for single and multi-column keys, + including SQL rendering assertions. + - [ ] Behavioural: Run Diesel-backed integration tests that paginate over + seeded datasets in both synchronous and async executors. - **Outcome** - Query construction becomes predictable and less error-prone when applying cursor-based filtering. @@ -82,11 +114,16 @@ ### Step 3.2 – Package boundary evaluation utilities - **Tasks** - - [ ] Implement helper routines that accept fetched records, trim the extra item, - determine presence of additional pages, and generate direction-appropriate - cursors. - - [ ] Support both forward and backward navigation paths with deterministic unit - tests covering off-by-one and empty-page scenarios. + - [ ] Implement helper routines that accept fetched records, trim the extra + item, determine presence of additional pages, and generate + direction-appropriate cursors. + - [ ] Support both forward and backward navigation paths with deterministic + handling of off-by-one and empty-page scenarios. +- **Testing** + - [ ] Unit: Cover boundary trimming, cursor derivation, and empty result sets + to guard against regressions. + - [ ] Behavioural: Validate helpers end-to-end by wiring them to Diesel query + adapters inside scenario-driven integration tests. - **Outcome** - Handlers can transform raw query results into paginated responses without duplicating edge-case logic. @@ -97,11 +134,16 @@ ### Step 4.1 – Compose HTTP response builders - **Tasks** - - [ ] Provide utility functions that assemble `Paginated` responses, including - URL construction for `self`, `next`, and `prev` links while preserving + - [ ] Provide utility functions that assemble `Paginated` responses, + including URL construction for `self`, `next`, and `prev` links while explicit `limit` selections. - - [ ] Offer extension traits or helper methods to combine request context, cursor - strings, and route metadata. + - [ ] Offer extension traits or helper methods to combine request context, + cursor strings, and route metadata. +- **Testing** + - [ ] Unit: Assert link formatting, limit propagation, and omission of + optional links under boundary conditions. + - [ ] Behavioural: Run Actix response tests verifying generated envelopes + integrate with router metadata and produce valid JSON. - **Outcome** - Actix endpoints can emit complete hypermedia envelopes with minimal boilerplate and consistent link formatting. @@ -110,11 +152,16 @@ ### Step 4.2 – Update `/api/users` handler as reference implementation - **Tasks** - - [ ] Refactor the existing users listing endpoint to consume the crate utilities - end-to-end, ensuring compatibility with async Diesel queries and existing - response schema expectations. - - [ ] Capture integration tests validating cursor navigation across forward and - backward flows, including limit enforcement. + - [ ] Refactor the existing users listing endpoint to consume the crate + utilities end-to-end, ensuring compatibility with async Diesel queries + and existing response schema expectations. + - [ ] Capture integration tests validating cursor navigation across forward + and backward flows, including limit enforcement. +- **Testing** + - [ ] Unit: Cover handler-specific fallbacks, including default limits and + empty dataset responses using lightweight state doubles. + - [ ] Behavioural: Execute end-to-end HTTP tests demonstrating cursor cycling + and verifying regression coverage for prior bugs. - **Outcome** - Demonstrated end-to-end adoption validates the crate and serves as a living example for other endpoints. @@ -125,10 +172,15 @@ ### Step 5.1 – Harden error handling and observability - **Tasks** - - [ ] Standardise error types returned by the crate, mapping decoding, validation, - and database misuse issues to actionable messages. + - [ ] Standardize error types returned by the crate, mapping decoding, + validation, and database misuse issues to actionable messages. - [ ] Emit structured logs or tracing spans for pagination failures, ensuring privacy by excluding cursor contents. +- **Testing** + - [ ] Unit: Assert each error case maps to the correct variant and telemetry + payload without leaking sensitive cursor data. + - [ ] Behavioural: Reproduce historical failure scenarios in integration tests + and demonstrate log or span output captures the fix. - **Outcome** - Operational visibility improves while keeping cursor tokens opaque and secure. @@ -137,22 +189,33 @@ ### Step 5.2 – Author documentation and migration guidance - **Tasks** - - [ ] Produce crate-level README and API docs summarising configuration, usage - examples, and extension points for other models (POIs, routes, etc.). - - [ ] Draft migration checklist for future endpoints, covering index requirements - and testing expectations. + - [ ] Produce crate-level README and API docs summarizing configuration, + usage examples, and extension points for other models (POIs, routes, + etc.). + - [ ] Draft migration checklist for future endpoints, covering index + requirements and testing expectations. +- **Testing** + - [ ] Unit: Run `cargo doc` and markdown linting to ensure examples compile + and documentation references resolve. + - [ ] Behavioural: Conduct a dry-run onboarding workshop with the `/api/users` + team and capture follow-up fixes from their feedback. - **Outcome** - Teams adopting the crate have authoritative guidance and know how to verify their integrations. - **Dependencies** - Completion of functional work to document accurately. -### Step 5.3 – Finalise release readiness +### Step 5.3 – Finalize release readiness - **Tasks** - [ ] Ensure lint, format, and test automation covers the crate via Makefile targets, updating CI configurations if required. - [ ] Add changelog entry describing the new pagination capabilities and their scope limitations (no total counts, opaque cursors only). +- **Testing** + - [ ] Unit: Execute crate-specific lint, format, and test targets locally to + prove automation coverage. + - [ ] Behavioural: Run the CI pipeline in a staging environment and confirm it + gates merges on the new pagination checks. - **Outcome** - Pagination crate is production-ready with tooling support and transparent release notes.