diff --git a/TESTING_COVERAGE.md b/TESTING_COVERAGE.md index dbae936d47..6d2d376406 100644 --- a/TESTING_COVERAGE.md +++ b/TESTING_COVERAGE.md @@ -13,6 +13,24 @@ the companion proxy branch has 39 passing tests including CORS preflight and exposed headers. Live OAuth on the browser path still requires deployment of the companion proxy CORS change and is not yet verified. +`browser/e2e/tests/devonian-issue-sync.spec.mts` exercises tenant-secret entry, +proxy consent, direct HTTP writes and local OPFS storage for two-way issue +creation, comments, close/reopen and reload without duplicate resources. Its +stateful HTTP mock isolates repositories and consumes/rotates connection codes; +it does not substitute the in-page sample transport. + +The browser-only Devonian issue tracker demo has focused tests under +`integrations/github-issues/devonian`: real Devonian lenses with deterministic +connectors exercise bidirectional issue/comment creation and edits, close/reopen, +distinct identical resources, conflicts, missing records and restart/replay. +Transport fixtures cover pagination, label preservation, scoped comment links, +rotating connection codes and refusal to resend uncertain writes. The native +OPFS browser flow was manually verified for creation and comments on both sides, +closing from Atomic, reopening from the sample GitHub side and reloading without +duplicate issues/comments. Live proxy OAuth, +GitHub writes and a guided uncertain-write recovery UI remain unverified/unbuilt; +proxy v40 CORS and browser OAuth are verified, but its GitHub credential returns 404 for the private sandbox. + What is tested, at which layer, and — the part that matters — **what is not**. This exists because the protocol is far better tested than the glue around it, @@ -1066,3 +1084,15 @@ cancelled on teardown. Old plugin-name grants are deliberately not migrated. commit. It failed with the server signer before `Resource::destroy_as` was used; installation deletion must use the same selected identity as create/update. LocalThought: Rust handler tests cover connection binding, request signing, duplicate-page rejection, typed paginated previews, and Calendar UTC date-range validation. Live Calendar OAuth, bounded fetch, review/apply and event table display were verified against proxy v39 (54 records). + +Google Calendar one-way projection: `integrations/localthought/calendar.test.ts` +covers all-day/timed start dates, offset boundaries, exclusive end preservation, +feature notes (including WASM-normalized field names), cancellations without +start data, invalid active events, namespace isolation and repeat import/local +field preservation. `browser/e2e/tests/google-calendar-import.spec.mts` uses the +shared HTTP mock integration-proxy with a paginated Google Calendar, tenant +secret entry and OAuth consent. It covers browser WASM fetching, local +schema/proposal/apply, Calendar display, provider updates, OPFS reload and +stable identities while AtomicServer HTTP/WebSockets are unavailable. Missing +rows in a bounded snapshot are retained, not interpreted as deletions. +Live-provider browser OAuth verification remains separate from this fixture test. diff --git a/browser/data-browser/src/chunks/DevonianDemo/DEVONIAN-LICENSE b/browser/data-browser/src/chunks/DevonianDemo/DEVONIAN-LICENSE new file mode 100644 index 0000000000..8dada3edaf --- /dev/null +++ b/browser/data-browser/src/chunks/DevonianDemo/DEVONIAN-LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/browser/data-browser/src/chunks/DevonianDemo/demo.d.mts b/browser/data-browser/src/chunks/DevonianDemo/demo.d.mts new file mode 100644 index 0000000000..8055f7e558 --- /dev/null +++ b/browser/data-browser/src/chunks/DevonianDemo/demo.d.mts @@ -0,0 +1,47 @@ +import type { Store } from '@tomic/lib'; +export interface Demo { + key: string; + state: { + options: { sample: boolean }; + config: { connection: { table: string } }; + fixture: { + issues?: Array<{ number: number; title: string; state: string }>; + comments?: Array<{ id: number; body: string; issue_url: string }>; + }; + }; +} +export interface Row { + id: string; + value: { title: string; body: string; status: string }; + comments: Array<{ id: string; value: { body: string } }>; +} +export function openDemo( + store: Store, + options: { + sample: boolean; + repository: string; + proxy: string; + }, +): Promise; +export function syncDemo(store: Store, demo: Demo): Promise; +export function demoRows(store: Store, demo: Demo): Promise; +export function editAtomic( + store: Store, + demo: Demo, + command: string, + id?: string, + text?: string, +): Promise; +export function editFixture( + demo: Demo, + command: string, + number?: number, + text?: string, +): Promise; + +export function connectDemo( + store: Store, + options: { repository: string; proxy: string }, + secret: string, +): Promise; +export function resumeDemo(store: Store): Promise; diff --git a/browser/data-browser/src/chunks/DevonianDemo/demo.mjs b/browser/data-browser/src/chunks/DevonianDemo/demo.mjs new file mode 100644 index 0000000000..65615d2b21 --- /dev/null +++ b/browser/data-browser/src/chunks/DevonianDemo/demo.mjs @@ -0,0 +1,297 @@ +// @wc-ignore-file +import { BrowserIntegrations } from '../../../../../integrations/localthought/browser'; +import { endpoint } from '../../../../../integrations/github-issues/adapter'; +import { get, set } from 'idb-keyval'; +import { core, server, dataBrowser, Datatype, enableLoro } from '@tomic/lib'; +import * as devonian from './devonian.js'; +import { ensureAgentForDemo } from '../Demo/guestAgent'; +import { buildTableFromSpec } from '../TablePage/createTableFromSpec'; +import { Bridge } from '../../../../../integrations/github-issues/devonian/bridge.mjs'; +import { + AtomicPort, + GitHubPort, +} from '../../../../../integrations/github-issues/devonian/ports.mjs'; +import { + fixtureTransport, + proxyTransport, +} from '../../../../../integrations/github-issues/devonian/proxy.mjs'; + +export async function openDemo(store, options) { + await enableLoro(); + await ensureAgentForDemo(store); + await store.waitForClientDb(10000); + const db = store.getClientDb(); + if (!db || !(await db.waitForReady())) + throw new Error('Enable the browser database on the Sync page first.'); + const repository = options.sample ? 'demo/issues' : options.repository; + endpoint(repository); + const key = `devonian-demo:${JSON.stringify([store.getAgent().subject, repository, options.sample ? 'sample' : new URL(options.proxy).origin])}`; + return navigator.locks.request(key, async () => { + let state = await get(key); + if (!state) { + const actor = store.getAgent().subject; + const drive = await store.newResource({ + noParent: true, + isA: [server.classes.drive], + propVals: { + [core.properties.name]: options.sample + ? 'Devonian sample tracker' + : `GitHub: ${repository}`, + [core.properties.read]: [actor], + [core.properties.write]: [actor], + }, + }); + store.registerLocalOnlyDrive(drive.subject); + await drive.save(); + await store.createDefaultOntology(drive); + const table = await buildTableFromSpec( + store, + { + name: 'Issue Tracker', + rowName: 'Issue', + columns: [ + { name: 'Description', type: 'markdown' }, + { + name: 'Status', + type: 'select', + options: ['Todo', 'Doing', 'Done'], + }, + { name: 'GitHub issue number', type: 'number' }, + ], + views: [ + { + name: 'Board', + kind: 'kanban', + groupByColumn: 'Status', + default: true, + }, + { name: 'All issues', kind: 'table' }, + ], + }, + { + parent: drive.subject, + driveSubject: drive.subject, + addToOntology: async () => {}, + }, + ); + const provenance = await store.newResource({ + parent: drive.subject, + isA: [core.classes.property], + propVals: { + [core.properties.name]: 'GitHub source', + [core.properties.shortname]: 'github-source', + [core.properties.description]: + 'Original author, identity and timestamps from GitHub.', + [core.properties.datatype]: Datatype.JSON, + }, + }); + await provenance.save(); + const folder = await store.newResource({ + parent: drive.subject, + isA: [dataBrowser.classes.folder], + propVals: { [core.properties.name]: 'Comments' }, + }); + await folder.save(); + await drive.set(dataBrowser.properties.commentsFolder, folder.subject); + await drive.save(); + const config = { + connection: { + repository, + drive: drive.subject, + table: table.tableSubject, + rowClass: table.classSubject, + body: table.columns.Description, + status: table.columns.Status, + number: table.columns['GitHub issue number'], + tags: table.tags.Status, + }, + commentsFolder: folder.subject, + provenance: provenance.subject, + }; + state = { + config, + options: { sample: options.sample, proxy: options.proxy, repository }, + base: `https://atomicdata.dev/devonian-bridges/${crypto.randomUUID()}`, + fixture: {}, + journal: {}, + }; + await db.flush(); + await set(key, state); + } + store.registerLocalOnlyDrive(state.config.connection.drive); + store.setDrive(state.config.connection.drive); + return { key, state }; + }); +} + +export async function syncDemo(store, demo) { + return navigator.locks.request(demo.key, async () => { + const state = await get(demo.key); + const save = () => set(demo.key, state); + const call = state.options.sample + ? fixtureTransport(state.fixture, save) + : proxyTransport({ + url: state.options.proxy, + repository: state.options.repository, + journal: state.journal, + save, + dispatch: (path, init) => + client(state.options.proxy).request( + state.config.connection.drive, + store.getAgent().subject, + state.connection, + 'github-issues', + path, + init, + ), + }); + const bridge = new Bridge({ + devonian, + local: new AtomicPort(store, state.config), + remote: new GitHubPort(null, state.config.connection, call), + base: state.base, + snapshot: state.bridge, + save: async snapshot => { + await store.getClientDb().flush(); + state.bridge = snapshot; + await save(); + }, + }); + await bridge.sync(); + demo.state = state; + return Object.keys(bridge.records).length; + }); +} + +export async function demoRows(store, demo) { + const local = new AtomicPort(store, demo.state.config); + const issues = await local.list('issue'); + for (const row of issues) + row.comments = await local.list('comment:ui', { issueId: row.id }); + return issues; +} + +export async function editAtomic(store, demo, command, id, text) { + const port = new AtomicPort(store, demo.state.config); + if (command === 'create') + await port.create( + 'issue', + { title: text, body: '', status: 'Todo' }, + crypto.randomUUID(), + ); + else if (command === 'comment') + await port.create( + 'comment:ui', + { body: text }, + crypto.randomUUID(), + undefined, + { issueId: id }, + ); + else { + const row = await port.get('issue', id); + await port.update('issue', id, { + ...row.value, + status: row.value.status === 'Done' ? 'Todo' : 'Done', + }); + } + await store.getClientDb().flush(); +} + +export async function editFixture(demo, command, number, text) { + return navigator.locks.request(demo.key, async () => { + const state = await get(demo.key); + const call = fixtureTransport(state.fixture, () => set(demo.key, state)); + if (command === 'create') + await call( + 'create_issue', + { title: text, body: '' }, + crypto.randomUUID(), + ); + else if (command === 'comment') + await call('create_comment', { number, body: text }, crypto.randomUUID()); + else { + const issue = state.fixture.issues.find(r => r.number === number); + await call( + 'update_issue', + { + number, + title: issue.title, + body: issue.body, + state: issue.state === 'open' ? 'closed' : 'open', + }, + crypto.randomUUID(), + ); + } + demo.state = state; + }); +} + +const handoffKey = 'devonian-browser-handoff'; +const resumeKey = 'devonian-browser-resume'; +const client = origin => + new BrowserIntegrations( + localStorage, + async () => { + throw new Error('Devonian uses its own resource lenses'); + }, + origin, + ); +export async function connectDemo(store, options, secret) { + const demo = await openDemo(store, { ...options, sample: false }); + const result = await client(options.proxy).start( + demo.state.config.connection.drive, + store.getAgent().subject, + 'github-issues', + `${location.origin}/app/devonian-demo`, + secret, + ); + sessionStorage.setItem( + handoffKey, + JSON.stringify({ key: demo.key, state: result.state }), + ); + location.assign(result.url); +} +export async function resumeDemo(store) { + const url = new URL(location.href); + const callbackCode = url.searchParams.get('connection_code'); + const callbackState = url.searchParams.get('integration_state'); + const handoff = JSON.parse(sessionStorage.getItem(handoffKey) ?? 'null'); + // Save the validated handoff before removing credentials from the URL, so a + // reload while OPFS opens cannot abandon the completed proxy consent. + if (callbackCode || callbackState) { + history.replaceState(null, '', url.pathname); + if (!handoff || callbackState !== handoff.state || !callbackCode) + throw new Error('Invalid connection callback state'); + handoff.code = callbackCode; + sessionStorage.setItem(handoffKey, JSON.stringify(handoff)); + } + const code = handoff?.code; + const stateId = handoff?.state; + const key = code ? handoff.key : sessionStorage.getItem(resumeKey); + if (!key) { + if (callbackCode || callbackState) + throw new Error('Missing browser connection handoff'); + return; + } + const saved = await get(key); + if (!saved) throw new Error('Missing local tracker'); + const demo = await openDemo(store, saved.options); + if (demo.key !== key) throw new Error('Connection belongs to another agent'); + if (code) { + if (!handoff.finished) { + client(saved.options.proxy).finish( + saved.config.connection.drive, + store.getAgent().subject, + stateId, + code, + ); + handoff.finished = true; + sessionStorage.setItem(handoffKey, JSON.stringify(handoff)); + } + demo.state.connection = stateId; + await set(key, demo.state); + sessionStorage.removeItem(handoffKey); + sessionStorage.setItem(resumeKey, key); + } + return demo; +} diff --git a/browser/data-browser/src/chunks/DevonianDemo/devonian.js b/browser/data-browser/src/chunks/DevonianDemo/devonian.js new file mode 100644 index 0000000000..ea269a9670 --- /dev/null +++ b/browser/data-browser/src/chunks/DevonianDemo/devonian.js @@ -0,0 +1,328 @@ +// Devonian native resource API, e11104f78ebd151a361171ca0b21489b25e1e2c8. Apache-2.0; see DEVONIAN-LICENSE. + +// ../../localthought/devonian/src/atomic/Resource.ts +import { Datatype, validateDatatype } from "@tomic/lib"; +var IS_A = "https://atomicdata.dev/properties/isA"; +function assertSubject(value) { + if (typeof value !== "string" || /\s/.test(value)) + throw new Error("Expected an absolute URL without whitespace"); + const url = new URL(value); + if (!["http:", "https:"].includes(url.protocol)) { + throw new Error(`Expected an HTTP(S) URL: ${value}`); + } +} +var AtomicSchema = class { + properties = /* @__PURE__ */ new Map([ + [IS_A, Datatype.RESOURCEARRAY] + ]); + property(subject, datatype) { + assertSubject(subject); + if (!Object.values(Datatype).includes(datatype) || datatype === Datatype.UNKNOWN) { + throw new Error(`Unsupported datatype: ${datatype}`); + } + const existing = this.properties.get(subject); + if (existing && existing !== datatype) + throw new Error(`Conflicting property: ${subject}`); + this.properties.set(subject, datatype); + return this; + } + validate(resource) { + assertSubject(resource["@id"]); + this.validateProperties(resource, true); + } + validateProperties(resource, named) { + if (Object.getPrototypeOf(resource) !== Object.prototype) + throw new Error("Expected a plain JSON object"); + for (const [property, value] of Object.entries(resource)) { + if (property === "@id" && named) continue; + assertSubject(property); + const datatype = this.properties.get(property); + if (!datatype) throw new Error(`Unknown property: ${property}`); + if (typeof value === "number" && !Number.isFinite(value)) + throw new Error("Expected a finite number"); + if (value === void 0 || value === null) + throw new Error(`Missing value for ${property}`); + if (datatype === Datatype.ATOMIC_URL) { + this.validateLink(value); + } else if (datatype === Datatype.RESOURCEARRAY) { + if (!Array.isArray(value)) + throw new Error(`Expected a resource array: ${property}`); + for (const link of value) this.validateLink(link); + } else { + if ([Datatype.FLOAT, Datatype.INTEGER, Datatype.TIMESTAMP].includes( + datatype + ) && typeof value !== "number") + throw new Error("Expected a number"); + if (datatype === Datatype.BOOLEAN && typeof value !== "boolean") + throw new Error("Expected a boolean"); + validateDatatype(value, datatype); + } + } + } + validateLink(value) { + if (typeof value === "string") assertSubject(value); + else if (value && typeof value === "object" && !Array.isArray(value)) + this.validateProperties(value, false); + else throw new Error("Expected a resource URL or nested resource"); + } +}; + +// ../../localthought/devonian/src/atomic/Store.ts +var AtomicStore = class _AtomicStore { + constructor(schema) { + this.schema = schema; + } + schema; + resources = /* @__PURE__ */ new Map(); + get(subject) { + const resource = this.resources.get(subject); + return resource && structuredClone(resource); + } + put(resource) { + this.schema.validate(resource); + this.resources.set(resource["@id"], structuredClone(resource)); + } + /** Preserve omitted properties; removals must be listed in unset. */ + patch(subject, patch) { + assertSubject(subject); + if (Object.hasOwn(patch.set ?? {}, "@id") || patch.unset?.includes("@id")) { + throw new Error("A patch cannot change resource identity"); + } + const resource = { + ...this.get(subject) ?? { "@id": subject }, + ...patch.set + }; + for (const property of patch.unset ?? []) { + assertSubject(property); + delete resource[property]; + } + this.put(resource); + return structuredClone(resource); + } + /** Synchronous local transaction. Async work must finish before entering this callback. */ + transaction(operation) { + const previous = this.resources; + this.resources = new Map(previous); + try { + operation(); + } catch (error) { + this.resources = previous; + throw error; + } + } + /** Apply a resource projection as one local transaction. */ + apply(changes) { + const staged = new _AtomicStore(this.schema); + staged.resources = new Map(this.resources); + for (const change of changes) staged.patch(change.subject, change.patch); + this.resources = staged.resources; + } + delete(subject) { + this.resources.delete(subject); + } + /** Optional class filtering; callers can apply additional predicates to the returned copies. */ + all(classSubject) { + return [...this.resources.values()].filter((resource) => { + const classes = resource["https://atomicdata.dev/properties/isA"]; + return !classSubject || Array.isArray(classes) && classes.includes(classSubject); + }).map((resource) => structuredClone(resource)); + } + toJSONAD() { + return JSON.stringify(this.all()); + } + /** Validate the complete snapshot before replacing state. Import does not emit writes. */ + loadJSONAD(json) { + const parsed = JSON.parse(json); + const resources = Array.isArray(parsed) ? parsed : [parsed]; + const next = /* @__PURE__ */ new Map(); + for (const item of resources) { + if (!item || typeof item !== "object" || Array.isArray(item) || typeof item["@id"] !== "string") { + throw new Error("Expected a named JSON-AD resource"); + } + const resource = item; + this.schema.validate(resource); + if (next.has(resource["@id"])) + throw new Error(`Duplicate subject: ${resource["@id"]}`); + next.set(resource["@id"], structuredClone(resource)); + } + this.resources = next; + } +}; + +// ../../localthought/devonian/src/atomic/IdentityMap.ts +var VOCAB = "https://raw.githubusercontent.com/localthought/devonian/main/vocab/"; +var identity = { + class: `${VOCAB}Identity.json`, + scope: `${VOCAB}scope.json`, + entity: `${VOCAB}entity.json`, + localId: `${VOCAB}localId.json`, + idType: `${VOCAB}idType.json`, + resource: `${VOCAB}resource.json` +}; +var AtomicIdentityMap = class { + constructor(store, baseURL) { + this.store = store; + assertSubject(baseURL); + const url = new URL(baseURL); + if (url.search || url.hash) + throw new Error("Identity base URL cannot contain a query or fragment"); + this.base = baseURL.replace(/\/$/, ""); + store.schema.property(identity.scope, Datatype.ATOMIC_URL).property(identity.entity, Datatype.STRING).property(identity.localId, Datatype.STRING).property(identity.idType, Datatype.STRING).property(identity.resource, Datatype.ATOMIC_URL); + } + store; + base; + key(scope, id) { + assertSubject(scope.scope); + if (!scope.entity || typeof id !== "string" && typeof id !== "number" || typeof id === "number" && !Number.isSafeInteger(id) || id === "") { + throw new Error( + "Expected an entity and a nonempty string or safe integer external ID" + ); + } + return encodeURIComponent( + JSON.stringify([scope.scope, scope.entity, typeof id, id]) + ); + } + subjectFor(scope, id) { + return this.lookup(scope, id) ?? `${this.base}/resources/${this.key(scope, id)}`; + } + lookup(scope, id) { + this.key(scope, id); + return this.find(scope).find( + (resource) => resource[identity.localId] === String(id) && resource[identity.idType] === typeof id + )?.[identity.resource]; + } + externalId(scope, subject) { + const resource = this.find(scope).find( + (resource2) => resource2[identity.resource] === subject + ); + if (!resource) return void 0; + return resource[identity.idType] === "number" ? Number(resource[identity.localId]) : resource[identity.localId]; + } + bind(scope, id, subject) { + const key = this.key(scope, id); + assertSubject(subject); + const existing = this.lookup(scope, id); + const reverse = this.externalId(scope, subject); + if (existing !== void 0 && existing !== subject || reverse !== void 0 && reverse !== id) { + throw new Error("Conflicting identity mapping"); + } + if (existing === subject && reverse === id) return; + this.store.put({ + "@id": `${this.base}/identities/${key}`, + [IS_A]: [identity.class], + [identity.scope]: scope.scope, + [identity.entity]: scope.entity, + [identity.localId]: String(id), + [identity.idType]: typeof id, + [identity.resource]: subject + }); + } + find(scope) { + const resources = this.store.all(identity.class).filter( + (resource) => resource[identity.scope] === scope.scope && resource[identity.entity] === scope.entity + ); + const ids = /* @__PURE__ */ new Set(); + const subjects = /* @__PURE__ */ new Set(); + for (const resource of resources) { + const id = resource[identity.localId]; + const type = resource[identity.idType]; + const subject = resource[identity.resource]; + if (typeof id !== "string" || !id || !["string", "number"].includes(String(type)) || typeof subject !== "string" || type === "number" && (!Number.isSafeInteger(Number(id)) || String(Number(id)) !== id)) { + throw new Error("Invalid identity mapping"); + } + const key = JSON.stringify([type, id]); + if (ids.has(key) || subjects.has(subject)) + throw new Error("Conflicting identity mappings in snapshot"); + ids.add(key); + subjects.add(subject); + } + return resources; + } +}; + +// ../../localthought/devonian/src/atomic/Lens.ts +var AtomicLens = class { + constructor(options) { + this.options = options; + assertSubject(options.scope); + if (!options.entity) throw new Error("A lens requires an entity type"); + if (options.identities.store !== options.store) { + throw new Error("Lens and identity map must use the same store"); + } + } + options; + pending = Promise.resolve(); + /** Handle a webhook or fetched record. Never writes back to the connector. */ + ingest(record) { + return this.enqueue(async () => { + const { connector, identities, store, read } = this.options; + const id = connector.id(record); + const subject = identities.subjectFor(this.options, id); + const patch = await read(record, subject); + store.transaction(() => { + store.apply([...patch.related ?? [], { subject, patch }]); + for (const mapping of patch.identities ?? []) + identities.bind(mapping.scope, mapping.id, mapping.subject); + identities.bind(this.options, id, subject); + }); + return subject; + }); + } + /** Publish the latest native state; failures reject and may be retried. */ + publish(subject) { + return this.enqueue(async () => { + const { connector, identities, store, write } = this.options; + const resource = store.get(subject); + if (!resource) throw new Error(`Unknown resource: ${subject}`); + const id = identities.externalId(this.options, subject); + if (id !== void 0) { + const previous = await connector.get(id); + await connector.update(id, await write(resource, previous)); + return id; + } + const key = JSON.stringify([ + this.options.scope, + this.options.entity, + subject + ]); + const created = await connector.create( + await write(resource, void 0), + key + ); + const createdId = connector.id(created); + identities.bind(this.options, createdId, subject); + return createdId; + }); + } + /** Delete native state after an external deletion. Retain identity for replay/recreation. */ + ingestDelete(id) { + return this.enqueue(async () => { + const subject = this.options.identities.lookup(this.options, id); + if (subject) this.options.store.delete(subject); + }); + } + /** Delete externally first, so a failed request leaves native state available for retry. */ + delete(subject) { + return this.enqueue(async () => { + const { connector, identities, store } = this.options; + const id = identities.externalId(this.options, subject); + if (id !== void 0) await connector.delete(id); + store.delete(subject); + }); + } + enqueue(operation) { + const result = this.pending.then(operation); + this.pending = result.catch(() => void 0); + return result; + } +}; +export { + AtomicIdentityMap, + AtomicLens, + AtomicSchema, + AtomicStore, + Datatype, + IS_A, + assertSubject, + identity +}; diff --git a/browser/data-browser/src/chunks/PluginRuns/ConnectLocalThought.tsx b/browser/data-browser/src/chunks/PluginRuns/ConnectLocalThought.tsx index 4be4f39df2..1448434a53 100644 --- a/browser/data-browser/src/chunks/PluginRuns/ConnectLocalThought.tsx +++ b/browser/data-browser/src/chunks/PluginRuns/ConnectLocalThought.tsx @@ -32,6 +32,10 @@ import { import type { Config } from '../../../../../integrations/localthought/plugin'; import { localImportVerdict } from './localImportVerdict'; import source from '../../../../../integrations/localthought/plugin.js?raw'; +import { + calendarProjection, + calendarFields, +} from '../../../../../integrations/localthought/calendar'; export function ConnectLocalThought({ drive, @@ -130,12 +134,13 @@ export function ConnectLocalThought({ setError(''); try { - const fetched = await proxyRequest(store, 'fetch', { + const response = await proxyRequest(store, 'fetch', { drive, connection: connection.connection, constants, ...(platform === 'google-calendar' ? { calendarRange } : {}), }); + const fetched = calendarProjection(response); if (fetched.platform !== platform) throw new Error('Imported platform did not match this connection'); const schemaStore = localSchemaStore(store); @@ -195,13 +200,47 @@ export function ConnectLocalThought({ [dataBrowser.properties.viewColumns]: columns, }, }); + const calendar = + platform === 'google-calendar' && term.shortname === 'event' + ? await ensureInstallationResource(store, drive, { + parent: destination.subject, + localId: `${identity}:calendar:${term.shortname}`, + isA: [dataBrowser.classes.view], + propVals: { + [core.properties.name]: tableName, + [dataBrowser.properties.viewKind]: 'calendar', + [dataBrowser.properties.viewGroupBy]: + properties[calendarFields.day], + [dataBrowser.properties.viewColumns]: columns, + }, + }) + : undefined; + const existingViews = destination.get( + dataBrowser.properties.tableViews, + ) as string[] | undefined; await destination.set(dataBrowser.properties.tableViews, [ - view.subject, + ...new Set([ + ...(existingViews ?? []), + view.subject, + ...(calendar ? [calendar.subject] : []), + ]), ]); - await destination.set( + const currentDefault = destination.get( dataBrowser.properties.tableDefaultView, - view.subject, ); + + if ( + !currentDefault || + (calendar && + !existingViews?.includes(calendar.subject) && + currentDefault === view.subject) + ) { + await destination.set( + dataBrowser.properties.tableDefaultView, + calendar?.subject ?? view.subject, + ); + } + await destination.save(); destinations[term.shortname] = { table: destination.subject, rowClass }; } diff --git a/browser/data-browser/src/locales/de.po b/browser/data-browser/src/locales/de.po index e3ca21bfa9..ff10ed5033 100644 --- a/browser/data-browser/src/locales/de.po +++ b/browser/data-browser/src/locales/de.po @@ -4704,6 +4704,7 @@ msgstr "" msgid "Syncing…" msgstr "" +#: src/routes/DevonianDemoRoute.tsx #: src/routes/SyncRoute.tsx msgid "Sync now" msgstr "" @@ -6287,6 +6288,7 @@ msgid "Keep an encrypted copy of this workspace in {0}. It is sealed on this dev msgstr "" #: src/components/Vault/VaultPanel.tsx +#: src/routes/DevonianDemoRoute.tsx msgid "Working…" msgstr "" @@ -9661,9 +9663,123 @@ msgid "Events before (UTC)" msgstr "" #: src/chunks/PluginRuns/ConnectLocalThought.tsx +#: src/routes/DevonianDemoRoute.tsx msgid "LocalThought tenant secret" msgstr "" #: src/chunks/PluginRuns/ConnectLocalThought.tsx msgid "The tenant secret is used in this tab. Connection credentials stay in this browser." msgstr "" + +#. 0: count +#: src/routes/DevonianDemoRoute.tsx +msgid "Synchronized {0} issues and comments." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Integration proxy URL" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "GitHub repository (owner/repo)" +msgstr "" + +#~ msgid "Connection code" +#~ msgstr "" + +#~ msgid "The proxy must allow this app’s origin through CORS and expose X-Connection-Code. Use a fresh GitHub connection code from the proxy’s OAuth flow." +#~ msgstr "" + +#~ msgid "Open live tracker" +#~ msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect a real GitHub repository" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Devonian issue tracker demo" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sync issues and comments in both directions. Devonian runs in this browser; the Atomic tracker is stored on this device." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Try sample data" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample mode: the GitHub side below is a browser fixture. No requests are sent to GitHub." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Live mode: Sync now creates and updates real GitHub issues and comments. Keep this tab open to sync." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Open Atomic kanban" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "New issue title or comment" +msgstr "" + +#~ msgid "Done" +#~ msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Reopen Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Close Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Add Atomic comment" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Atomic tracker" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Atomic issues" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Reopen sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Close sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Add sample GitHub comment" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Create sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Create Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample GitHub tracker" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample GitHub issues" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect through LocalThought. The tenant secret is used in this tab to start the connection and is never saved." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect GitHub tracker" +msgstr "" diff --git a/browser/data-browser/src/locales/en.po b/browser/data-browser/src/locales/en.po index 3d23fe2d7d..06bff9a6f1 100644 --- a/browser/data-browser/src/locales/en.po +++ b/browser/data-browser/src/locales/en.po @@ -4715,6 +4715,7 @@ msgstr "{0}% used" msgid "Syncing…" msgstr "Syncing…" +#: src/routes/DevonianDemoRoute.tsx #: src/routes/SyncRoute.tsx msgid "Sync now" msgstr "Sync now" @@ -6298,6 +6299,7 @@ msgid "Keep an encrypted copy of this workspace in {0}. It is sealed on this dev msgstr "Keep an encrypted copy of this workspace in {0}. It is sealed on this device, so we store it without being able to read it." #: src/components/Vault/VaultPanel.tsx +#: src/routes/DevonianDemoRoute.tsx msgid "Working…" msgstr "Working…" @@ -9700,9 +9702,123 @@ msgid "Events before (UTC)" msgstr "Events before (UTC)" #: src/chunks/PluginRuns/ConnectLocalThought.tsx +#: src/routes/DevonianDemoRoute.tsx msgid "LocalThought tenant secret" msgstr "LocalThought tenant secret" #: src/chunks/PluginRuns/ConnectLocalThought.tsx msgid "The tenant secret is used in this tab. Connection credentials stay in this browser." msgstr "The tenant secret is used in this tab. Connection credentials stay in this browser." + +#. 0: count +#: src/routes/DevonianDemoRoute.tsx +msgid "Synchronized {0} issues and comments." +msgstr "Synchronized {0} issues and comments." + +#: src/routes/DevonianDemoRoute.tsx +msgid "Integration proxy URL" +msgstr "Integration proxy URL" + +#: src/routes/DevonianDemoRoute.tsx +msgid "GitHub repository (owner/repo)" +msgstr "GitHub repository (owner/repo)" + +#~ msgid "Connection code" +#~ msgstr "Connection code" + +#~ msgid "The proxy must allow this app’s origin through CORS and expose X-Connection-Code. Use a fresh GitHub connection code from the proxy’s OAuth flow." +#~ msgstr "The proxy must allow this app’s origin through CORS and expose X-Connection-Code. Use a fresh GitHub connection code from the proxy’s OAuth flow." + +#~ msgid "Open live tracker" +#~ msgstr "Open live tracker" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect a real GitHub repository" +msgstr "Connect a real GitHub repository" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Devonian issue tracker demo" +msgstr "Devonian issue tracker demo" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sync issues and comments in both directions. Devonian runs in this browser; the Atomic tracker is stored on this device." +msgstr "Sync issues and comments in both directions. Devonian runs in this browser; the Atomic tracker is stored on this device." + +#: src/routes/DevonianDemoRoute.tsx +msgid "Try sample data" +msgstr "Try sample data" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample mode: the GitHub side below is a browser fixture. No requests are sent to GitHub." +msgstr "Sample mode: the GitHub side below is a browser fixture. No requests are sent to GitHub." + +#: src/routes/DevonianDemoRoute.tsx +msgid "Live mode: Sync now creates and updates real GitHub issues and comments. Keep this tab open to sync." +msgstr "Live mode: Sync now creates and updates real GitHub issues and comments. Keep this tab open to sync." + +#: src/routes/DevonianDemoRoute.tsx +msgid "Open Atomic kanban" +msgstr "Open Atomic kanban" + +#: src/routes/DevonianDemoRoute.tsx +msgid "New issue title or comment" +msgstr "New issue title or comment" + +#~ msgid "Done" +#~ msgstr "Done" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Reopen Atomic issue" +msgstr "Reopen Atomic issue" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Close Atomic issue" +msgstr "Close Atomic issue" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Add Atomic comment" +msgstr "Add Atomic comment" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Atomic tracker" +msgstr "Atomic tracker" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Atomic issues" +msgstr "Atomic issues" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Reopen sample GitHub issue" +msgstr "Reopen sample GitHub issue" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Close sample GitHub issue" +msgstr "Close sample GitHub issue" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Add sample GitHub comment" +msgstr "Add sample GitHub comment" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Create sample GitHub issue" +msgstr "Create sample GitHub issue" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Create Atomic issue" +msgstr "Create Atomic issue" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample GitHub tracker" +msgstr "Sample GitHub tracker" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample GitHub issues" +msgstr "Sample GitHub issues" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect through LocalThought. The tenant secret is used in this tab to start the connection and is never saved." +msgstr "Connect through LocalThought. The tenant secret is used in this tab to start the connection and is never saved." + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect GitHub tracker" +msgstr "Connect GitHub tracker" diff --git a/browser/data-browser/src/locales/es.po b/browser/data-browser/src/locales/es.po index 0e1c244ed0..c21040df5e 100644 --- a/browser/data-browser/src/locales/es.po +++ b/browser/data-browser/src/locales/es.po @@ -4704,6 +4704,7 @@ msgstr "" msgid "Syncing…" msgstr "" +#: src/routes/DevonianDemoRoute.tsx #: src/routes/SyncRoute.tsx msgid "Sync now" msgstr "" @@ -6287,6 +6288,7 @@ msgid "Keep an encrypted copy of this workspace in {0}. It is sealed on this dev msgstr "" #: src/components/Vault/VaultPanel.tsx +#: src/routes/DevonianDemoRoute.tsx msgid "Working…" msgstr "" @@ -9661,9 +9663,123 @@ msgid "Events before (UTC)" msgstr "" #: src/chunks/PluginRuns/ConnectLocalThought.tsx +#: src/routes/DevonianDemoRoute.tsx msgid "LocalThought tenant secret" msgstr "" #: src/chunks/PluginRuns/ConnectLocalThought.tsx msgid "The tenant secret is used in this tab. Connection credentials stay in this browser." msgstr "" + +#. 0: count +#: src/routes/DevonianDemoRoute.tsx +msgid "Synchronized {0} issues and comments." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Integration proxy URL" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "GitHub repository (owner/repo)" +msgstr "" + +#~ msgid "Connection code" +#~ msgstr "" + +#~ msgid "The proxy must allow this app’s origin through CORS and expose X-Connection-Code. Use a fresh GitHub connection code from the proxy’s OAuth flow." +#~ msgstr "" + +#~ msgid "Open live tracker" +#~ msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect a real GitHub repository" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Devonian issue tracker demo" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sync issues and comments in both directions. Devonian runs in this browser; the Atomic tracker is stored on this device." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Try sample data" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample mode: the GitHub side below is a browser fixture. No requests are sent to GitHub." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Live mode: Sync now creates and updates real GitHub issues and comments. Keep this tab open to sync." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Open Atomic kanban" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "New issue title or comment" +msgstr "" + +#~ msgid "Done" +#~ msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Reopen Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Close Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Add Atomic comment" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Atomic tracker" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Atomic issues" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Reopen sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Close sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Add sample GitHub comment" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Create sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Create Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample GitHub tracker" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample GitHub issues" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect through LocalThought. The tenant secret is used in this tab to start the connection and is never saved." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect GitHub tracker" +msgstr "" diff --git a/browser/data-browser/src/locales/fr.po b/browser/data-browser/src/locales/fr.po index b74aceab3e..46b33e9914 100644 --- a/browser/data-browser/src/locales/fr.po +++ b/browser/data-browser/src/locales/fr.po @@ -4704,6 +4704,7 @@ msgstr "" msgid "Syncing…" msgstr "" +#: src/routes/DevonianDemoRoute.tsx #: src/routes/SyncRoute.tsx msgid "Sync now" msgstr "" @@ -6287,6 +6288,7 @@ msgid "Keep an encrypted copy of this workspace in {0}. It is sealed on this dev msgstr "" #: src/components/Vault/VaultPanel.tsx +#: src/routes/DevonianDemoRoute.tsx msgid "Working…" msgstr "" @@ -9661,9 +9663,123 @@ msgid "Events before (UTC)" msgstr "" #: src/chunks/PluginRuns/ConnectLocalThought.tsx +#: src/routes/DevonianDemoRoute.tsx msgid "LocalThought tenant secret" msgstr "" #: src/chunks/PluginRuns/ConnectLocalThought.tsx msgid "The tenant secret is used in this tab. Connection credentials stay in this browser." msgstr "" + +#. 0: count +#: src/routes/DevonianDemoRoute.tsx +msgid "Synchronized {0} issues and comments." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Integration proxy URL" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "GitHub repository (owner/repo)" +msgstr "" + +#~ msgid "Connection code" +#~ msgstr "" + +#~ msgid "The proxy must allow this app’s origin through CORS and expose X-Connection-Code. Use a fresh GitHub connection code from the proxy’s OAuth flow." +#~ msgstr "" + +#~ msgid "Open live tracker" +#~ msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect a real GitHub repository" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Devonian issue tracker demo" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sync issues and comments in both directions. Devonian runs in this browser; the Atomic tracker is stored on this device." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Try sample data" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample mode: the GitHub side below is a browser fixture. No requests are sent to GitHub." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Live mode: Sync now creates and updates real GitHub issues and comments. Keep this tab open to sync." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Open Atomic kanban" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "New issue title or comment" +msgstr "" + +#~ msgid "Done" +#~ msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Reopen Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Close Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Add Atomic comment" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Atomic tracker" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Atomic issues" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Reopen sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Close sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Add sample GitHub comment" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Create sample GitHub issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Create Atomic issue" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample GitHub tracker" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Sample GitHub issues" +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect through LocalThought. The tenant secret is used in this tab to start the connection and is never saved." +msgstr "" + +#: src/routes/DevonianDemoRoute.tsx +msgid "Connect GitHub tracker" +msgstr "" diff --git a/browser/data-browser/src/routes/DevonianDemoRoute.tsx b/browser/data-browser/src/routes/DevonianDemoRoute.tsx new file mode 100644 index 0000000000..f961500c59 --- /dev/null +++ b/browser/data-browser/src/routes/DevonianDemoRoute.tsx @@ -0,0 +1,275 @@ +import { createLazyRoute } from '@tanstack/react-router'; +import { useEffect, useRef, useState } from 'react'; +import { useStore } from '@tomic/react'; +import { Main } from '@components/Main'; +import { ContainerWide } from '@components/Containers'; +import { Column, Row as Horizontal } from '@components/Row'; +import { Card } from '@components/Card'; +import { Button } from '@components/Button'; +import { AtomicLink } from '@components/AtomicLink'; +import Field from '@components/forms/Field'; +import { Input, ErrMessage } from '@components/forms/InputStyles'; +import { + openDemo, + connectDemo, + resumeDemo, + syncDemo, + demoRows, + editAtomic, + editFixture, + type Demo, + type Row, +} from '../chunks/DevonianDemo/demo.mjs'; + +function DevonianDemo() { + const store = useStore(); + const [demo, setDemo] = useState(); + const [rows, setRows] = useState([]); + const [proxy, setProxy] = useState('https://localthought.io'); + const [repository, setRepository] = useState(''); + const [secret, setSecret] = useState(''); + const [busy, setBusy] = useState(false); + const [error, setError] = useState(''); + const [status, setStatus] = useState(''); + const [text, setText] = useState(''); + const run = async (work: () => Promise) => { + setBusy(true); + setError(''); + try { + const current = await work(); + if (current) { + setDemo({ ...current }); + setRows(await demoRows(store, current)); + } + } catch (reason) { + setError(reason instanceof Error ? reason.message : String(reason)); + } finally { + setBusy(false); + } + }; + const resumed = useRef(false); + useEffect(() => { + if (resumed.current) return; + resumed.current = true; + void run(() => resumeDemo(store)); + }, [store]); + const start = (sample: boolean) => + run(async () => { + if (!sample) { + const credential = secret; + setSecret(''); + await connectDemo(store, { repository, proxy }, credential); + return; + } + const opened = await openDemo(store, { sample, repository, proxy }); + if (sample) await syncDemo(store, opened); + return opened; + }); + const sync = () => + run(async () => { + if (!demo) return; + const count = await syncDemo(store, demo); + setStatus(`Synchronized ${count} issues and comments.`); + return demo; + }); + const edit = ( + side: 'atomic' | 'fixture', + command: string, + id?: string | number, + ) => + run(async () => { + if (!demo) return; + if (side === 'atomic') + await editAtomic(store, demo, command, id as string, text); + else await editFixture(demo, command, id as number, text); + setText(''); + return demo; + }); + return ( +
+ + +

Devonian issue tracker demo

+

+ Sync issues and comments in both directions. Devonian runs in this + browser; the Atomic tracker is stored on this device. +

+ {!demo && ( + <> + +
+ Connect a real GitHub repository + +

+ Connect through LocalThought. The tenant secret is used in + this tab to start the connection and is never saved. +

+ + setProxy(e.target.value)} + /> + + + setRepository(e.target.value)} + /> + + + setSecret(e.target.value)} + /> + + + +
+ + )} + {demo && ( + <> +

+ {demo.state.options.sample + ? 'Sample mode: the GitHub side below is a browser fixture. No requests are sent to GitHub.' + : 'Live mode: Sync now creates and updates real GitHub issues and comments. Keep this tab open to sync.'} +

+ + + + Open Atomic kanban + + + + setText(e.target.value)} + /> + + + + {demo.state.options.sample && ( + + )} + +
+

Atomic tracker

+ + {rows.map(row => ( + + + + {row.value.title} + + {row.value.status} + + + + + {row.comments.map(comment => ( +

{comment.value.body}

+ ))} + + + ))} + +
+ {demo.state.options.sample && ( +
+

Sample GitHub tracker

+ + {demo.state.fixture.issues?.map(issue => ( + + + + #{issue.number} {issue.title} + + {issue.state} + + + + + {demo.state.fixture.comments + ?.filter(c => + c.issue_url.endsWith(`/${issue.number}`), + ) + .map(c => ( +

{c.body}

+ ))} + + + ))} + +
+ )} + + )} +

{status}

+ {error && {error}} + + +
+ ); +} +export const devonianDemoRouteLazy = createLazyRoute('/app/devonian-demo')({ + component: DevonianDemo, +}); diff --git a/browser/data-browser/src/routes/Router.tsx b/browser/data-browser/src/routes/Router.tsx index 5c79967682..176a64ffcb 100644 --- a/browser/data-browser/src/routes/Router.tsx +++ b/browser/data-browser/src/routes/Router.tsx @@ -42,6 +42,13 @@ const DemoRoute = createRoute({ path: pathNames.demo, }).lazy(() => import('./DemoRoute').then(mod => mod.demoRouteLazy)); +const DevonianDemoRoute = createRoute({ + getParentRoute: () => appRoute, + path: '/devonian-demo', +}).lazy(() => + import('./DevonianDemoRoute').then(mod => mod.devonianDemoRouteLazy), +); + const PruneTestsRoute = createRoute({ getParentRoute: () => appRoute, path: pathNames.pruneTests, @@ -91,6 +98,7 @@ const routeTree = rootRoute.addChildren({ SandboxRoute, DevDriveRoute, DemoRoute, + DevonianDemoRoute, InviteRoute, LinkOpenRouter, }), diff --git a/browser/e2e/tests/devonian-issue-sync.spec.mts b/browser/e2e/tests/devonian-issue-sync.spec.mts new file mode 100644 index 0000000000..b79b8e97fa --- /dev/null +++ b/browser/e2e/tests/devonian-issue-sync.spec.mts @@ -0,0 +1,182 @@ +import { test, expect } from '@playwright/test'; +import { + mockProxy, + tenantSecret, +} from '../../../integrations/localthought/mock-proxy.mjs'; +const FRONTEND_URL = process.env.FRONTEND_URL ?? 'http://localhost:6747'; +const SERVER_URL = process.env.SERVER_URL ?? 'http://localhost:9883'; + +test.use({ serviceWorkers: 'block' }); + +// This is a public fixture secret, never an environment/live credential. +// Exercise the real form, tenant challenge, consent, rotating HTTP transport and OPFS. +test('Devonian syncs issue creation, state and comments both ways through the browser proxy', async ({ + page, +}) => { + test.setTimeout(180_000); + const proxy = mockProxy({ frontendOrigin: new URL(FRONTEND_URL).origin }); + await new Promise(resolve => proxy.listen(0, '127.0.0.1', resolve)); + const address = proxy.address() as { port: number }; + const proxyOrigin = `http://127.0.0.1:${address.port}`; + const repository = 'demo/two-way'; + const remote = proxy.github; + const initial = remote.createIssue(repository, { + title: 'Created on GitHub', + }); + remote.createComment(repository, initial.number, { + body: 'Initial GitHub comment', + }); + // No AtomicServer WebSocket writes (or Vite HMR reloads) during this journey. + await page.routeWebSocket('**/*', socket => socket.close()); + const forbidden: string[] = []; + const requests: string[] = []; + page.on('request', req => { + if (req.url().startsWith(`${proxyOrigin}/proxy/`)) + requests.push(req.method()); + }); + // Keep static app assets available even when frontend and AtomicServer share an origin. + await page.route('**/*', route => { + const url = new URL(route.request().url()); + if (/^\/(integration-proxy|plugin-run|commit)(\/|$)/.test(url.pathname)) { + forbidden.push(url.pathname); + return route.abort(); + } + if ( + url.origin === new URL(SERVER_URL).origin && + url.origin !== new URL(FRONTEND_URL).origin + ) + return route.abort(); + return route.continue(); + }); + try { + await page.goto(`${FRONTEND_URL}/app/dev-drive`); + await page.waitForURL(/app\/show\?subject=/, { timeout: 60000 }); + await page.goto(`${FRONTEND_URL}/app/devonian-demo`); + await page + .getByText('Connect a real GitHub repository', { exact: true }) + .click(); + await page.getByLabel('Integration proxy URL').fill(proxyOrigin); + await page.getByLabel('GitHub repository (owner/repo)').fill(repository); + await page.getByLabel('LocalThought tenant secret').fill(tenantSecret); + await page + .getByRole('button', { name: 'Connect GitHub tracker', exact: true }) + .click(); + await page + .getByRole('button', { name: 'Connect test account', exact: true }) + .click(); + const sync = async () => { + const button = page.getByRole('button', { + name: 'Sync now', + exact: true, + }); + await button.click(); + await expect(button).toBeEnabled(); + await expect(page.getByRole('alert')).toHaveCount(0); + }; + await expect( + page.getByRole('button', { name: 'Sync now', exact: true }), + ).toBeVisible(); + expect(page.url()).not.toContain('connection_code'); + expect( + await page.evaluate(() => + JSON.stringify({ ...localStorage, ...sessionStorage }), + ), + ).not.toContain(tenantSecret); + await sync(); + const issue = (title: string) => + page + .getByTestId('atomic-issue') + .filter({ has: page.getByRole('link', { name: title, exact: true }) }); + await expect(issue('Created on GitHub')).toContainText( + 'Initial GitHub comment', + ); + + const text = page.getByLabel('New issue title or comment'); + await text.fill('Created in Atomic'); + await page + .getByRole('button', { name: 'Create Atomic issue', exact: true }) + .click(); + await expect(issue('Created in Atomic')).toBeVisible(); + await sync(); + const created = remote + .snapshot(repository) + .issues.find(i => i.title === 'Created in Atomic'); + expect(created).toBeDefined(); + expect(remote.snapshot(repository).issues).toHaveLength(2); + + await text.fill('Comment from Atomic'); + await issue('Created in Atomic') + .getByRole('button', { name: 'Add Atomic comment', exact: true }) + .click(); + await issue('Created in Atomic') + .getByRole('button', { name: 'Close Atomic issue', exact: true }) + .click(); + await sync(); + expect( + remote.snapshot(repository).issues.find(i => i.number === created!.number) + ?.state, + ).toBe('closed'); + expect( + remote + .snapshot(repository) + .comments.filter(c => c.issue_url.endsWith(`/${created!.number}`)) + .map(c => c.body), + ).toEqual(['Comment from Atomic']); + + remote.updateIssue(repository, created!.number, { state: 'open' }); + remote.createComment(repository, created!.number, { + body: 'Reply from GitHub', + }); + remote.updateIssue(repository, initial.number, { state: 'closed' }); + await sync(); + await expect( + issue('Created in Atomic').getByRole('button', { + name: 'Close Atomic issue', + exact: true, + }), + ).toBeVisible(); + await expect(issue('Created in Atomic')).toContainText('Reply from GitHub'); + await expect( + issue('Created on GitHub').getByRole('button', { + name: 'Reopen Atomic issue', + exact: true, + }), + ).toBeVisible(); + await issue('Created on GitHub') + .getByRole('button', { name: 'Reopen Atomic issue', exact: true }) + .click(); + await sync(); + expect( + remote.snapshot(repository).issues.find(i => i.number === initial.number) + ?.state, + ).toBe('open'); + + const before = remote.snapshot(repository); + const subjects = await page + .getByTestId('atomic-issue') + .getByRole('link') + .evaluateAll(links => + links.map(link => link.getAttribute('href')).sort(), + ); + await page.reload(); + await sync(); + await expect(page.getByTestId('atomic-issue')).toHaveCount(2); + await expect(issue('Created in Atomic')).toContainText('Reply from GitHub'); + expect( + await page + .getByTestId('atomic-issue') + .getByRole('link') + .evaluateAll(links => + links.map(link => link.getAttribute('href')).sort(), + ), + ).toEqual(subjects); + expect(remote.snapshot(repository)).toEqual(before); + expect(requests).toEqual(expect.arrayContaining(['GET', 'POST', 'PATCH'])); + expect(forbidden).toEqual([]); + } finally { + proxy.closeAllConnections(); + await new Promise((resolve, reject) => + proxy.close(error => (error ? reject(error) : resolve())), + ); + } +}); diff --git a/browser/e2e/tests/google-calendar-import.spec.mts b/browser/e2e/tests/google-calendar-import.spec.mts new file mode 100644 index 0000000000..22d00c5e81 --- /dev/null +++ b/browser/e2e/tests/google-calendar-import.spec.mts @@ -0,0 +1,169 @@ +import { test, expect } from '@playwright/test'; +import { + mockProxy, + tenantSecret, +} from '../../../integrations/localthought/mock-proxy.mjs'; +const FRONTEND_URL = process.env.FRONTEND_URL ?? 'http://localhost:6747'; +const SERVER_URL = process.env.SERVER_URL ?? 'http://localhost:9883'; +test.use({ serviceWorkers: 'block' }); + +// Reuse the browser transport and HTTP mock from the Devonian integration. +// The tenant secret is a public fixture. No real provider credentials are used. +test('Calendar imports, refreshes and persists through the browser with AtomicServer unavailable', async ({ + page, +}) => { + test.setTimeout(180_000); + const proxy = mockProxy({ frontendOrigin: new URL(FRONTEND_URL).origin }); + await new Promise(resolve => proxy.listen(0, '127.0.0.1', resolve)); + const port = (proxy.address() as { port: number }).port; + const proxyOrigin = `http://127.0.0.1:${port}`; + const configuredProxy = + process.env.VITE_INTEGRATION_PROXY_URL || 'https://localthought.io'; + await page.routeWebSocket('**/*', socket => socket.close()); + const forbidden: string[] = []; + const providerMethods: string[] = []; + // Forward the configured proxy to this test's isolated HTTP fixture. All + // tenant proof, consent, code rotation, pagination and WASM code remain real. + await page.route('**/*', async route => { + const request = route.request(); + const url = new URL(request.url()); + + if (/^\/(integration-proxy|plugin-run|commit)(\/|$)/.test(url.pathname)) { + forbidden.push(url.pathname); + + return route.abort(); + } + + if (url.origin === configuredProxy) { + if (url.pathname.startsWith('/proxy/')) + providerMethods.push(request.method()); + const response = await route.fetch({ + url: `${proxyOrigin}${url.pathname}${url.search}`, + maxRedirects: 0, + }); + + return route.fulfill({ response }); + } + + if ( + url.origin === new URL(SERVER_URL).origin && + url.origin !== new URL(FRONTEND_URL).origin + ) + return route.abort(); + + return route.continue(); + }); + + try { + await page.goto(`${FRONTEND_URL}/app/dev-drive`); + await page.waitForURL(/app\/show\?subject=/, { timeout: 60000 }); + + const setup = async () => { + await page + .getByRole('link', { name: 'Integrations', exact: true }) + .click(); + await page + .locator('[data-integration="google-calendar"]') + .getByRole('button', { name: 'Set up connection' }) + .click(); + }; + + await setup(); + await page.getByLabel('LocalThought tenant secret').fill(tenantSecret); + await page + .getByRole('button', { name: 'Install and connect', exact: true }) + .click(); + await page + .getByRole('button', { name: 'Connect test account', exact: true }) + .click(); + await expect( + page.getByRole('button', { name: 'Fetch and preview', exact: true }), + ).toBeVisible(); + expect(page.url()).not.toContain('connection_code'); + expect( + await page.evaluate(() => + JSON.stringify({ ...localStorage, ...sessionStorage }), + ), + ).not.toContain(tenantSecret); + + const apply = async (count: number) => { + await page + .getByRole('button', { name: 'Fetch and preview', exact: true }) + .click(); + await page + .getByRole('button', { name: new RegExp(`^Apply ${count} changes?$`) }) + .click(); + await page + .getByRole('link', { name: 'Open imported records', exact: true }) + .click(); + await expect(page.getByTestId('calendar-view')).toBeVisible(); + }; + + await apply(2); + await expect(page.getByTestId('calendar-event')).toHaveCount(2); + await page.getByText('Calendar timed fixture', { exact: true }).click(); + await expect(page.getByRole('dialog').last()).toContainText( + 'Recurring event', + ); + await expect(page.getByRole('dialog').last()).toContainText( + 'Attendees and RSVP', + ); + await page.keyboard.press('Escape'); + const original = await page.evaluate(async () => { + const store = window.store!; + const table = new URL(location.href).searchParams.get('subject')!; + const result = await store.queryLocalDb({ + drive: store.getDrive()!, + property: 'https://atomicdata.dev/properties/parent', + value: table, + limit: 100, + }); + const subjects = result!.subjects.sort(); + const row = await store.getResource(subjects[0]); + await row.set( + 'https://atomicdata.dev/properties/description', + 'Atomic-only notes', + ); + await row.save(); + await store.getClientDb()!.flush(); + + return { subjects, table, annotated: row.subject }; + }); + await page.reload(); + await expect(page.getByTestId('calendar-event')).toHaveCount(2); + proxy.calendar.events[1].summary = 'Calendar refreshed fixture'; + await setup(); + await apply(1); + await expect( + page.getByText('Calendar refreshed fixture', { exact: true }), + ).toBeVisible(); + await expect(page.getByTestId('calendar-event')).toHaveCount(2); + const restored = await page.evaluate(async saved => { + const store = window.store!; + const result = await store.queryLocalDb({ + drive: store.getDrive()!, + property: 'https://atomicdata.dev/properties/parent', + value: saved.table, + limit: 100, + }); + + return { + subjects: result!.subjects.sort(), + note: (await store.getResource(saved.annotated)).get( + 'https://atomicdata.dev/properties/description', + ), + }; + }, original); + expect(restored.subjects).toEqual(original.subjects); + expect(restored.note).toBe('Atomic-only notes'); + expect(providerMethods).toEqual(['GET', 'GET', 'GET', 'GET']); + expect( + proxy.calendar.requests.filter(r => r.query.pageToken === 'second'), + ).toHaveLength(2); + expect(forbidden).toEqual([]); + } finally { + await new Promise((resolve, reject) => + proxy.close(error => (error ? reject(error) : resolve())), + ); + } +}); diff --git a/integrations/github-issues/README.md b/integrations/github-issues/README.md index f265fa5f0c..e0f80d1254 100644 --- a/integrations/github-issues/README.md +++ b/integrations/github-issues/README.md @@ -1,3 +1,10 @@ +## Browser-only Devonian demo + +Open `/app/devonian-demo` for a browser-side issue/comment sync demo using +Devonian and this integration's mappings. It uses a local-only Atomic drive and +direct integration-proxy requests, with an explicit sample mode. See +[setup, source and the current live proxy CORS limitation](devonian/README.md). + ## Connect in the app Open **Integrations**, enter `owner/repository` and a repository-scoped GitHub diff --git a/integrations/github-issues/adapter.test.ts b/integrations/github-issues/adapter.test.ts index e60052a966..854422b60d 100644 --- a/integrations/github-issues/adapter.test.ts +++ b/integrations/github-issues/adapter.test.ts @@ -3,6 +3,7 @@ import { preview, project, manifest, type Issue } from './adapter.js'; import { validateManifest } from '../../browser/lib/src/plugin-manifest.js'; import { readFile } from 'node:fs/promises'; import { execFileSync } from 'node:child_process'; +import { existsSync } from 'node:fs'; const issue = (number: number): Issue => ({ number, title: `Issue ${number}`, @@ -18,7 +19,9 @@ describe('GitHub package', () => { }); it('ships exactly the artifact tested by the sandbox', async () => { const built = execFileSync( - './browser/node_modules/.bin/esbuild', + existsSync('./browser/node_modules/.bin/esbuild') + ? './browser/node_modules/.bin/esbuild' + : './browser/node_modules/.pnpm/node_modules/.bin/esbuild', [ 'integrations/github-issues/plugin.ts', '--bundle', diff --git a/integrations/github-issues/devonian/README.md b/integrations/github-issues/devonian/README.md new file mode 100644 index 0000000000..56c895d925 --- /dev/null +++ b/integrations/github-issues/devonian/README.md @@ -0,0 +1,129 @@ +# Browser issue tracker demo + +Open `/app/devonian-demo` in the Atomic web app. Choose **Try sample data**, +create issues or comments on either side, close/reopen an issue, and press +**Sync now**. **Open Atomic kanban** opens the real native tracker, whose issue +pages also have the normal Comments panel. Reload and choose the same mode to +resume saved resources and mappings without importing duplicates. + +The JavaScript runs in the browser. Devonian's native resource lenses run there; +Atomic resources live in a local-only drive in the browser's WASM/OPFS database. +The intermediate graph, scoped identity mappings, baselines and request journal +live in IndexedDB. No Node runtime, custom AtomicServer endpoint, server-side +plugin executor, tenant secret on AtomicServer or server worker is used. Enable +the browser database if it has been disabled. + +## Live GitHub + +Expand **Connect a real GitHub repository**, enter the integration-proxy origin, +`owner/repo`, and your **LocalThought tenant secret**, then choose **Connect GitHub +tracker**. The demo uses #1401's shared `BrowserIntegrations` client to sign the +tenant challenge in the browser, navigate to proxy consent, and bind the return +to this agent and local drive. The secret is cleared from the form and never +persisted. The callback code is removed from the address bar immediately. + +After consent, **Sync now** authorizes two-way writes. The shared transport +serializes requests with Web Locks, consumes each code before dispatch, and +persists rotated credentials in browser localStorage, outside the Atomic graph. +Reloading the tab resumes the local tracker and connection. Use a dedicated +demo repository; writes use the connected GitHub account. + +The browser calls `/proxy/github-issues/repos/{owner}/{repo}/issues...` directly. +The proxy instance must answer unauthenticated OPTIONS preflights, allow the +app's origin and GET/POST/PATCH/DELETE with Authorization and Content-Type, +expose `X-Connection-Code`, preserve query parameters, and allow issue/comment +and label operations in its catalog. CORS headers must cover error responses too. + +**Live verification (2026-09-09):** Heroku release v40 (`bc02f13f`) now +answers browser preflights and exposes `X-Connection-Code`. Tenant challenge, +GitHub OAuth and return to the browser demo succeeded with AtomicServer +unavailable. The first authenticated issue-list request for the private +`ontola/atomic-github-sync-sandbox` returned 404, although the independent +GitHub CLI credential can access it. Repository access for the proxy credential +must be resolved before live two-way writes can be verified. The disposable +CLI-created issue #17 was closed; no issue/comment writes occurred through the proxy. + +## Mapping + +`GitHubPort` reuses `github-issues/adapter.ts`'s projection and request builder. +`tracker-actions.ts` adds scoped issue/comment request construction for the +browser transport. The installed server plugin's actions remain unchanged. + +- Title/body map to the native row's name and Markdown description. +- Open maps to Todo; open with `atomic:doing` maps to Doing; closed maps to Done. + Other labels are preserved. Provision `atomic:doing` before using Doing. +- A comment is a native Message whose `about` points to its issue and whose + `parent` is the drive's Comments folder. Body edits synchronize both ways. +- GitHub identity, author and original timestamps are retained in a separate + provenance property. The importing Atomic agent is distinct from the original + author. GitHub writes use the connected account's authorship. +- The demo creates drive-local task properties so setup works offline. The + intermediate Devonian graph uses the shared task vocabulary and maps native + properties explicitly. Atomic DIDs are scoped external IDs because Devonian's + current native subjects must be HTTP(S) URLs. +- Explicit issue numbers can bind existing rows; matching text never does. + Comments have their own scoped IDs; identical comments remain distinct. + +## Persistence and limits + +A Web Lock serializes syncs. Three-way reconciliation preserves independent +field edits and stops on same-field conflicts. Reloads resume saved operations +before discovering new records. Ingest never automatically publishes an echo. +Missing records are conflicts; neither side is deleted. + +The proxy does not provide idempotent GitHub creation. Writes are journaled +before sending. Successful receipts can be replayed; an uncertain/lost response +stops without resending. An operator must inspect and reconcile that operation; +there is no recovery wizard yet. Do not clear IndexedDB to retry a create. +Concurrent edits during a multi-request status transition may also require +reconciliation. Pre-write reads and verification do not make local/provider +writes one transaction. Sync is explicit while the tab is open, with a +10,000-record scan cap. Clearing browser site data loses the demo and its state. + +## Source and verification + +- `browser/data-browser/src/chunks/DevonianDemo/demo.mjs`: browser entry script. +- `bridge.mjs`: Devonian lenses and checkpointed reconciliation. +- `ports.mjs`: native Atomic and GitHub projections/transports. +- `proxy.mjs`: rotating-code transport and labelled sample fixture. +- `browser/data-browser/src/routes/DevonianDemoRoute.tsx`: demo controls. + +The committed `DevonianDemo/devonian.js` bundle contains only the native resource +API from Devonian main `e11104f78ebd151a361171ca0b21489b25e1e2c8`, with its +Apache-2.0 license alongside. Regenerate at development time: + +```sh +DEVONIAN_PATH=/path/to/devonian node integrations/github-issues/devonian/build.mjs +browser/node_modules/.bin/vitest run --config integrations/github-issues/devonian/vitest.config.mjs +browser/node_modules/.bin/vitest run --config integrations/github-issues/vitest.config.ts +cd browser/data-browser && pnpm typecheck +``` + +Tests use real Devonian lenses and deterministic connectors: creation, identical +content with distinct identities, independent edits, close/reopen, comments, +conflicts, missing resources, replay after lost receipts, label preservation, +pagination, scoped URLs and serialized code rotation. The browser sample flow +was manually verified with native OPFS resources, issue creation on both sides, +comments both ways, closing from Atomic and reopening from the GitHub fixture, +then reloading and resuming the same three issues and two comments without duplication. + +## Playwright two-way regression + +`browser/e2e/tests/devonian-issue-sync.spec.mts` starts an isolated HTTP integration +proxy with stateful, repository-scoped GitHub issue/comment endpoints. It fills +the tenant-secret form with the **public mock secret** (no real credentials), +completes consent, and exercises the live transport mode rather than sample mode. +It verifies creation and comments in both directions, close/reopen in both +directions, matching parent issues, and reload without duplicate resources or +provider writes. Legacy server integration endpoints, commit POSTs and all WebSockets are blocked. +The local run uses an unavailable AtomicServer port to verify browser-only storage. + +With Vite and the built browser/WASM packages available: + +```sh +cd browser/e2e +FRONTEND_URL=http://localhost:6747 playwright test tests/devonian-issue-sync.spec.mts --project chromium +``` + +The test is in the full E2E suite, without a smoke tag. The mock is local test +infrastructure only; the runtime demo uses the real integration-proxy protocol. diff --git a/integrations/github-issues/devonian/bridge.mjs b/integrations/github-issues/devonian/bridge.mjs new file mode 100644 index 0000000000..3f90390a4d --- /dev/null +++ b/integrations/github-issues/devonian/bridge.mjs @@ -0,0 +1,243 @@ +import { reconcileRecord } from '../../../browser/lib/src/plugin-reconcile.js'; + +const p = { + title: 'https://atomicdata.dev/properties/name', + body: 'https://atomicdata.dev/task/v1/body', + status: 'https://atomicdata.dev/task/v1/status', +}; +const tag = 'https://atomicdata.dev/task/v1/'; +const equal = (a, b) => JSON.stringify(a) === JSON.stringify(b); +const copy = value => structuredClone(value); + +/** Single-writer, checkpointed reconciliation. Ports own transport and durable writes. */ +export class Bridge { + constructor({ devonian, local, remote, base, snapshot, save }) { + this.api = devonian; + this.local = local; + this.remote = remote; + this.save = save; + const { AtomicSchema, AtomicStore, AtomicIdentityMap, Datatype } = devonian; + this.store = new AtomicStore( + new AtomicSchema() + .property(p.title, Datatype.STRING) + .property(p.body, Datatype.MARKDOWN) + .property(p.status, Datatype.RESOURCEARRAY), + ); + this.identities = new AtomicIdentityMap(this.store, base); + this.binding = { base, local: local.scope, remote: remote.scope }; + if (snapshot && !equal(snapshot.binding, this.binding)) + throw new Error('State belongs to another connection'); + if (snapshot?.graph) this.store.loadJSONAD(snapshot.graph); + this.records = copy(snapshot?.records ?? {}); + } + + scope(side, entity) { + return { scope: this[side].scope, entity }; + } + id(side, entity, subject) { + return this.identities.externalId(this.scope(side, entity), subject); + } + context(side, entity) { + if (entity === 'issue') return {}; + const parent = entity.slice('comment:'.length); + const issueId = this.id(side, 'issue', parent); + if (issueId === undefined) + throw new Error('Issue must be mapped before comments'); + return { issueId }; + } + async checkpoint() { + await this.save({ + version: 1, + binding: this.binding, + graph: this.store.toJSONAD(), + records: copy(this.records), + }); + } + properties(value) { + return { + [p.body]: value.body, + ...(value.title === undefined + ? {} + : { + [p.title]: value.title, + [p.status]: [`${tag}${value.status.toLowerCase()}`], + }), + }; + } + value(resource, entity) { + if (entity !== 'issue') return { body: resource[p.body] }; + const status = { + [`${tag}todo`]: 'Todo', + [`${tag}doing`]: 'Doing', + [`${tag}done`]: 'Done', + }[resource[p.status]?.[0]]; + if (!status || resource[p.status].length !== 1) + throw new Error('Unsupported task status'); + return { title: resource[p.title], body: resource[p.body], status }; + } + lens(side, entity, operation, metadata) { + const port = this[side], + context = this.context(side, entity); + return new this.api.AtomicLens({ + store: this.store, + identities: this.identities, + ...this.scope(side, entity), + connector: { + id: row => row.id, + get: id => port.get(entity, id, context), + create: (row, key) => + port.create(entity, row.value, key, metadata, context), + update: (id, row) => + port.update( + entity, + id, + row.value, + `${operation}:${side}`, + metadata, + context, + ), + delete: async () => { + throw new Error('Deletion is outside issue sync'); + }, + }, + read: row => ({ set: this.properties(row.value) }), + write: (resource, previous) => ({ + ...previous, + value: this.value(resource, entity), + }), + }); + } + + async sync() { + // Resume saved operations BEFORE discovering their newly-created counterparts. + for (const [subject, record] of Object.entries(this.records)) { + if (record.pending) await this.finish(subject, record); + } + await this.syncEntity('issue'); + for (const [subject, record] of Object.entries(this.records)) { + if (record.entity === 'issue') + await this.syncEntity(`comment:${subject}`); + } + } + + async syncEntity(entity) { + const lists = {}; + for (const side of ['remote', 'local']) { + const rows = await this[side].list(entity, this.context(side, entity)); + lists[side] = new Map(); + for (const row of rows) { + if (lists[side].has(row.id)) + throw new Error('Duplicate external identity'); + lists[side].set(row.id, row); + } + } + // An existing pilot's explicit issue-number column is an identity, never a title match. + for (const row of lists.local.values()) { + if (row.remoteId === undefined) continue; + const scope = this.scope('remote', entity); + const subject = this.identities.subjectFor(scope, row.remoteId); + this.identities.bind(scope, row.remoteId, subject); + this.identities.bind(this.scope('local', entity), row.id, subject); + this.records[subject] ??= { entity }; + } + for (const side of ['remote', 'local']) { + for (const row of lists[side].values()) { + let subject = this.identities.lookup(this.scope(side, entity), row.id); + if (!subject) subject = await this.lens(side, entity).ingest(row); + this.records[subject] ??= { entity }; + } + } + await this.checkpoint(); + for (const [subject, record] of Object.entries(this.records)) { + if (record.entity !== entity) continue; + const rows = {}; + for (const side of ['local', 'remote']) { + const id = this.id(side, entity, subject); + rows[side] = id === undefined ? undefined : lists[side].get(id); + if (id !== undefined && !rows[side]) + throw new Error(`Missing ${side} record: ${subject}`); + } + const decision = reconcileRecord( + record.baseline, + rows.local?.value, + rows.remote?.value, + ); + if (decision.conflicts.length) + throw new Error( + `Conflict on ${subject}: ${decision.conflicts.map(c => c.property).join(', ')}`, + ); + const desired = { + ...(rows.remote?.value ?? rows.local?.value), + ...decision.remote, + }; + const metadata = rows.remote?.metadata; + if ( + rows.local && + rows.remote && + equal(rows.local.value, rows.remote.value) && + equal(rows.local.metadata, metadata) + ) { + record.baseline = copy(desired); + await this.checkpoint(); + continue; + } + record.pending = { + operation: crypto.randomUUID(), + local: rows.local?.value, + remote: rows.remote?.value, + desired, + metadata, + }; + this.store.patch(subject, { set: this.properties(desired) }); + await this.checkpoint(); + await this.finish(subject, record); + } + } + + async finish(subject, record) { + const { pending, entity } = record; + // A retry accepts only the original observation or this operation's exact result. + for (const side of ['local', 'remote']) { + const id = this.id(side, entity, subject); + if (id === undefined) continue; + const row = await this[side].get(entity, id, this.context(side, entity)); + if ( + !equal(row.value, pending[side]) && + !equal(row.value, pending.desired) + ) + throw new Error(`Conflict during saved operation on ${subject}`); + } + for (const side of ['remote', 'local']) { + const id = this.id(side, entity, subject); + const row = + id === undefined + ? undefined + : await this[side].get(entity, id, this.context(side, entity)); + if ( + !row || + !equal(row.value, pending.desired) || + (side === 'local' && !equal(row.metadata, pending.metadata)) + ) { + await this.lens( + side, + entity, + pending.operation, + pending.metadata, + ).publish(subject); + await this.checkpoint(); + } + } + for (const side of ['local', 'remote']) { + const row = await this[side].get( + entity, + this.id(side, entity, subject), + this.context(side, entity), + ); + if (!equal(row.value, pending.desired)) + throw new Error(`Concurrent edit after write on ${subject}`); + } + record.baseline = copy(pending.desired); + delete record.pending; + await this.checkpoint(); + } +} diff --git a/integrations/github-issues/devonian/bridge.test.mjs b/integrations/github-issues/devonian/bridge.test.mjs new file mode 100644 index 0000000000..672a7b1030 --- /dev/null +++ b/integrations/github-issues/devonian/bridge.test.mjs @@ -0,0 +1,133 @@ +import { expect, it } from 'vitest'; +import * as devonian from 'devonian'; +import { Bridge } from './bridge.mjs'; + +function fixture(snapshot) { + let saved = snapshot; + const makePort = scope => ({ + scope, + rows: new Map(), + receipts: new Map(), + writes: 0, + lose: false, + async list(entity) { + return [...this.rows.values()].filter(r => r.entity === entity); + }, + async get(entity, id) { + const row = this.rows.get(id); + if (!row || row.entity !== entity) throw new Error('Missing record'); + return structuredClone(row); + }, + async create(entity, value, key, metadata) { + if (this.receipts.has(key)) return this.receipts.get(key); + const id = this.rows.size + 1; + const row = { id, entity, value: structuredClone(value), metadata }; + this.rows.set(id, row); + this.receipts.set(key, row); + this.writes++; + if (this.lose) { + this.lose = false; + throw new Error('Lost response'); + } + return row; + }, + async update(entity, id, value) { + const row = await this.get(entity, id); + this.rows.set(id, { ...row, value: structuredClone(value) }); + this.writes++; + }, + }); + const local = makePort('https://atomic.example/bridge'); + const remote = makePort('https://github.com/acme/repo'); + const open = () => + new Bridge({ + devonian, + local, + remote, + snapshot: saved, + base: 'https://bridge.example/sync', + save: async s => { + saved = structuredClone(s); + }, + }); + return { local, remote, open, saved: () => saved }; +} +const issue = (id, title = 'Same title') => ({ + id, + entity: 'issue', + value: { title, body: '', status: 'Todo' }, +}); + +it('syncs creation both ways without deduplicating equal content, then no-ops after restart', async () => { + const f = fixture(); + f.local.rows.set(1, issue(1)); + f.remote.rows.set(1, issue(1)); + await f.open().sync(); + expect(f.local.rows.size).toBe(2); + expect(f.remote.rows.size).toBe(2); + const writes = f.local.writes + f.remote.writes; + await f.open().sync(); + expect(f.local.writes + f.remote.writes).toBe(writes); +}); + +it('merges independent title/body edits and propagates close/reopen', async () => { + const f = fixture(); + f.remote.rows.set(1, issue(1)); + await f.open().sync(); + f.local.rows.get(1).value.title = 'Local title'; + f.remote.rows.get(1).value.body = 'Remote body'; + f.local.rows.get(1).value.status = 'Done'; + await f.open().sync(); + expect(f.remote.rows.get(1).value).toEqual({ + title: 'Local title', + body: 'Remote body', + status: 'Done', + }); + f.remote.rows.get(1).value.status = 'Todo'; + await f.open().sync(); + expect(f.local.rows.get(1).value.status).toBe('Todo'); +}); + +it('syncs comments and edits both ways, preserving source metadata', async () => { + const f = fixture(); + f.remote.rows.set(1, issue(1)); + await f.open().sync(); + const parent = Object.keys(f.saved().records)[0]; + const entity = `comment:${parent}`; + f.remote.rows.set(2, { + id: 2, + entity, + value: { body: 'GitHub comment' }, + metadata: { author: 'octocat' }, + }); + f.local.rows.set(2, { id: 2, entity, value: { body: 'Atomic comment' } }); + await f.open().sync(); + expect(f.local.rows.get(3).metadata).toEqual({ author: 'octocat' }); + expect(f.remote.rows.get(3).value.body).toBe('Atomic comment'); + f.local.rows.get(3).value.body = 'Edited'; + await f.open().sync(); + expect(f.remote.rows.get(2).value.body).toBe('Edited'); +}); + +it('stops on same-field conflicts and missing records without deleting either side', async () => { + const f = fixture(); + f.remote.rows.set(1, issue(1)); + await f.open().sync(); + f.local.rows.get(1).value.title = 'A'; + f.remote.rows.get(1).value.title = 'B'; + await expect(f.open().sync()).rejects.toThrow('Conflict'); + expect(f.local.rows.get(1).value.title).toBe('A'); + f.remote.rows.delete(1); + await expect(f.open().sync()).rejects.toThrow('Missing'); + expect(f.local.rows.size).toBe(1); +}); + +it('reuses the same create identity after a lost receipt and restart', async () => { + const f = fixture(); + f.local.rows.set(1, issue(1)); + f.remote.lose = true; + await expect(f.open().sync()).rejects.toThrow('Lost response'); + await f.open().sync(); + expect(f.remote.rows.size).toBe(1); + expect(f.remote.writes).toBe(1); +}); diff --git a/integrations/github-issues/devonian/build.mjs b/integrations/github-issues/devonian/build.mjs new file mode 100644 index 0000000000..9a6c4c2051 --- /dev/null +++ b/integrations/github-issues/devonian/build.mjs @@ -0,0 +1,36 @@ +/** Development-time bundle; the demo itself runs entirely in the browser. */ +import { createRequire } from 'node:module'; +import { resolve, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +const root = resolve(dirname(fileURLToPath(import.meta.url)), '../../..'); +const require = createRequire(import.meta.url); +const { build } = require( + require.resolve('esbuild', { + paths: [ + resolve(root, 'browser'), + resolve(root, 'browser/node_modules/.pnpm'), + ], + }), +); +if (!process.env.DEVONIAN_PATH) + throw new Error('Set DEVONIAN_PATH to Devonian main'); +await build({ + stdin: { + contents: ['Resource', 'Store', 'IdentityMap', 'Lens'] + .map(name => `export * from './src/atomic/${name}.js';`) + .join('\n'), + resolveDir: resolve(process.env.DEVONIAN_PATH), + loader: 'ts', + }, + bundle: true, + platform: 'browser', + format: 'esm', + external: ['@tomic/lib'], + banner: { + js: '// Devonian native resource API, e11104f78ebd151a361171ca0b21489b25e1e2c8. Apache-2.0; see DEVONIAN-LICENSE.', + }, + outfile: resolve( + root, + 'browser/data-browser/src/chunks/DevonianDemo/devonian.js', + ), +}); diff --git a/integrations/github-issues/devonian/ports.mjs b/integrations/github-issues/devonian/ports.mjs new file mode 100644 index 0000000000..ca5733d489 --- /dev/null +++ b/integrations/github-issues/devonian/ports.mjs @@ -0,0 +1,322 @@ +import { project } from '../adapter.js'; +import { core, dataBrowser } from '../../../browser/lib/src/index.js'; + +export const digest = async value => + Array.from( + new Uint8Array( + await crypto.subtle.digest('SHA-256', new TextEncoder().encode(value)), + ), + byte => byte.toString(16).padStart(2, '0'), + ).join(''); +const equal = (a, b) => JSON.stringify(a) === JSON.stringify(b); + +/** Reuses the GitHub integration mapping with an injected browser proxy transport. */ +export class GitHubPort { + constructor(store, connection, call) { + this.store = store; + this.connection = connection; + this.scope = `https://github.com/${connection.repository}`; + if (!call) + throw new Error('A browser integration-proxy transport is required'); + this.call = call; + } + async request(action, args, key) { + const receipt = await this.call( + action, + args, + await digest(key ?? `${action}:${JSON.stringify(args)}`), + ); + if (receipt.status < 200 || receipt.status >= 300) + throw new Error(`GitHub ${action} returned ${receipt.status}`); + return receipt.body ? JSON.parse(receipt.body) : undefined; + } + row(entity, raw, context = {}) { + if (entity === 'issue') { + if (raw.pull_request) throw new Error('Pull requests are excluded'); + return { + id: raw.number, + value: project(raw), + metadata: this.metadata(raw, { number: raw.number }), + }; + } + if ( + !Number.isSafeInteger(raw.id) || + raw.id <= 0 || + typeof raw.body !== 'string' + ) + throw new Error('Invalid GitHub comment'); + const expected = `https://api.github.com/repos/${this.connection.repository}/issues/${context.issueId}`; + if (raw.issue_url !== expected) + throw new Error('Comment belongs to another issue'); + return { + id: raw.id, + value: { body: raw.body }, + metadata: this.metadata(raw, { commentId: raw.id }), + }; + } + metadata(raw, identity) { + return { + ...identity, + ...(raw.user?.login ? { author: raw.user.login } : {}), + ...(raw.html_url ? { url: raw.html_url } : {}), + ...(raw.created_at ? { createdAt: raw.created_at } : {}), + ...(raw.updated_at ? { updatedAt: raw.updated_at } : {}), + }; + } + async list(entity, context = {}) { + const result = []; + for (let page = 1; page <= 100; page++) { + const raw = await this.request( + entity === 'issue' ? 'list_issues' : 'list_comments', + { + page, + ...(entity === 'issue' ? {} : { number: context.issueId }), + }, + ); + if (!Array.isArray(raw)) throw new Error('Invalid GitHub page'); + for (const r of raw) { + if (entity === 'issue' && 'pull_request' in r) continue; + result.push(this.row(entity, r, context)); + } + if (raw.length < 100) return result; + } + throw new Error('More than 10,000 records in one scan'); + } + async get(entity, id, context) { + const raw = await this.request( + entity === 'issue' ? 'get_issue' : 'get_comment', + entity === 'issue' ? { number: id } : { id }, + ); + const row = this.row(entity, raw, context); + if (row.id !== id) throw new Error('Provider returned another identity'); + return row; + } + async create(entity, value, key, metadata, context) { + const raw = await this.request( + entity === 'issue' ? 'create_issue' : 'create_comment', + entity === 'issue' + ? { title: value.title, body: value.body } + : { number: context.issueId, body: value.body }, + `${key}:create`, + ); + const row = this.row(entity, raw, context); + if (entity === 'issue' && value.status !== 'Todo') + await this.update( + entity, + row.id, + value, + `${key}:initial-state`, + metadata, + context, + ); + return { ...row, value }; + } + async update(entity, id, value, key, _metadata, context) { + if (entity !== 'issue') { + await this.get(entity, id, context); // validate issue membership before writing + await this.request( + 'update_comment', + { id, body: value.body }, + `${key}:body`, + ); + return; + } + if (!['Todo', 'Doing', 'Done'].includes(value.status)) + throw new Error('Unsupported task status'); + const current = await this.get(entity, id, context); + await this.request( + 'update_issue', + { + number: id, + title: value.title, + body: value.body, + state: value.status === 'Done' ? 'closed' : 'open', + }, + `${key}:fields`, + ); + // Only this workflow label is managed. Closing preserves the label like the pilot. + if (value.status === 'Doing' && current.value.status !== 'Doing') { + await this.request('add_doing_label', { number: id }, `${key}:doing`); + } else if (value.status === 'Todo') { + // Closed issues may still carry the doing label, so inspect raw labels. + const raw = await this.request('get_issue', { number: id }); + if ( + raw.labels.some( + l => + (typeof l === 'string' ? l : l.name).toLowerCase() === + 'atomic:doing', + ) + ) { + await this.request('remove_doing_label', { number: id }, `${key}:todo`); + } + } + } +} + +/** Signed SDK writes build on the fetched Loro state, preserving unmanaged fields. */ +export class AtomicPort { + constructor(store, config) { + this.store = store; + this.config = config; + this.connection = config.connection; + this.scope = `https://atomicdata.dev/devonian-local/${encodeURIComponent(this.connection.table)}`; + } + async subjects(property, value) { + const result = await this.store.queryLocalDb({ + drive: this.connection.drive, + property, + value, + limit: 10001, + }); + if ( + !result || + result.count > 10000 || + result.subjects.length !== result.count + ) + throw new Error('Incomplete local database query'); + return result.subjects; + } + async findByLocalId(parent, localId) { + const rows = await Promise.all( + (await this.subjects(core.properties.localId, localId)).map(id => + this.resource(id), + ), + ); + const matches = rows.filter(r => r.get(core.properties.parent) === parent); + if (matches.length > 1) + throw new Error('Duplicate Atomic creation identity'); + return matches[0]; + } + async resource(subject) { + const r = await this.store.getResource(subject); + if (r.error) throw r.error; + r.getLoroDoc(); + return r; + } + row(entity, r, context = {}) { + const c = this.connection; + const classes = r.get(core.properties.isA) ?? []; + if (entity === 'issue') { + if ( + !classes.includes(c.rowClass) || + r.get(core.properties.parent) !== c.table + ) + throw new Error('Resource is not a row of this issue tracker'); + } else if ( + !classes.includes(dataBrowser.classes.message) || + r.get(dataBrowser.properties.about) !== context.issueId + ) + throw new Error('Message belongs to another issue'); + let metadata = r.get(this.config.provenance); + if (typeof metadata === 'string') metadata = JSON.parse(metadata); + if (entity !== 'issue') + return { + id: r.subject, + value: { body: r.get(core.properties.description) ?? '' }, + ...(metadata ? { metadata } : {}), + }; + const statuses = r.get(c.status) ?? [c.tags.Todo]; + const status = Object.keys(c.tags).find(s => c.tags[s] === statuses[0]); + if (statuses.length !== 1 || !status) + throw new Error( + 'Choose exactly one Todo/Doing/Done status; Blocked is unmapped', + ); + const title = r.get(core.properties.name), + body = r.get(c.body) ?? ''; + if (typeof title !== 'string' || !title.trim() || typeof body !== 'string') + throw new Error('Invalid Atomic issue'); + const remoteId = r.get(c.number); + if ( + remoteId !== undefined && + (!Number.isSafeInteger(remoteId) || remoteId <= 0) + ) + throw new Error('Invalid GitHub issue number'); + return { + id: r.subject, + remoteId, + value: { title, body, status }, + ...(metadata ? { metadata } : {}), + }; + } + async list(entity, context = {}) { + const ids = await this.subjects( + entity === 'issue' + ? core.properties.parent + : dataBrowser.properties.about, + entity === 'issue' ? this.connection.table : context.issueId, + ); + const rows = []; + for (const id of ids) { + const resource = await this.resource(id); + if ( + !(resource.get(core.properties.isA) ?? []).includes( + entity === 'issue' + ? this.connection.rowClass + : dataBrowser.classes.message, + ) + ) + continue; + rows.push(this.row(entity, resource, context)); + } + return rows; + } + async get(entity, id, context) { + return this.row(entity, await this.resource(id), context); + } + values(entity, value, metadata, context) { + const c = this.connection; + return { + ...(entity === 'issue' + ? { + [core.properties.name]: value.title, + [c.body]: value.body, + [c.status]: [c.tags[value.status]], + ...(metadata?.number ? { [c.number]: metadata.number } : {}), + } + : { + [core.properties.description]: value.body, + [dataBrowser.properties.about]: context.issueId, + }), + ...(metadata ? { [this.config.provenance]: metadata } : {}), + }; + } + async create(entity, value, key, metadata, context) { + const parent = + entity === 'issue' ? this.connection.table : this.config.commentsFolder; + const localId = `devonian:${await digest(key)}`; + const existing = await this.findByLocalId(parent, localId); + if (existing) { + const row = await this.get(entity, existing.subject, context); + if (!equal(row.value, value)) + throw new Error( + 'Recovered Atomic create was edited; reconcile before retry', + ); + return row; + } + const r = await this.store.newResource({ + parent, + isA: [ + entity === 'issue' + ? this.connection.rowClass + : dataBrowser.classes.message, + ], + propVals: { + ...this.values(entity, value, metadata, context), + [core.properties.localId]: localId, + }, + }); + if ((await r.save()) === 'offline') + throw new Error('AtomicServer disconnected'); + return this.get(entity, r.subject, context); + } + async update(entity, id, value, _key, metadata, context) { + const r = await this.resource(id); + this.row(entity, r, context); + for (const [p, v] of Object.entries( + this.values(entity, value, metadata, context), + )) + await r.set(p, v); + if ((await r.save()) === 'offline') + throw new Error('AtomicServer disconnected'); + } +} diff --git a/integrations/github-issues/devonian/ports.test.mjs b/integrations/github-issues/devonian/ports.test.mjs new file mode 100644 index 0000000000..25590e5ec6 --- /dev/null +++ b/integrations/github-issues/devonian/ports.test.mjs @@ -0,0 +1,237 @@ +import { expect, it } from 'vitest'; +import { GitHubPort, AtomicPort } from './ports.mjs'; +import { + trackerAction, + trackerActions, + trackerOperations, +} from '../tracker-actions.js'; +import { manifest } from '../adapter.js'; +import { validateManifest } from '../../../browser/lib/src/plugin-manifest.js'; + +it('declares and prepares scoped issue/comment actions with validated arguments', () => { + const original = manifest('owner/repo'); + const m = validateManifest({ + ...original, + actions: [...original.actions, ...trackerActions], + operations: [ + ...original.operations, + ...trackerOperations('https://api.github.com/repos/owner/repo/issues'), + ], + }); + for (const [action, args] of [ + ['list_issues', { page: 1 }], + ['list_comments', { number: 5, page: 2 }], + ['create_comment', { number: 5, body: 'Hello' }], + ['get_comment', { id: 9 }], + ['update_comment', { id: 9, body: 'Edit' }], + ['update_issue', { number: 5, title: 'Title', body: '', state: 'closed' }], + ['add_doing_label', { number: 5 }], + ['remove_doing_label', { number: 5 }], + ]) { + const intent = trackerAction('owner/repo', action, args); + expect( + intent.url.startsWith('https://api.github.com/repos/owner/repo/issues'), + ).toBe(true); + expect(m.actions.find(a => a.name === action).operation).toBe( + intent.operation, + ); + expect( + m.operations.some( + o => o.id === intent.operation && o.method === intent.method, + ), + ).toBe(true); + } + expect(() => + trackerAction('owner/repo', 'get_comment', { id: '../escape' }), + ).toThrow(); + expect(() => + trackerAction('owner/repo', 'list_issues', { page: 101 }), + ).toThrow(); + expect(() => + trackerAction('owner/repo', 'create_comment', { number: 1, body: '' }), + ).toThrow(); +}); + +function github() { + const issues = new Map([ + [ + 1, + { number: 1, title: 'First', body: '', state: 'open', labels: ['bug'] }, + ], + ]); + const comments = new Map(); + const receipts = new Map(); + const writes = []; + const port = new GitHubPort( + null, + { repository: 'owner/repo' }, + async (action, args, id) => { + const isWrite = /^(create|update|add|remove)_/.test(action); + if (isWrite && receipts.has(id)) return receipts.get(id); + let value; + if (isWrite) writes.push(action); + switch (action) { + case 'get_issue': + value = issues.get(args.number); + break; + case 'list_issues': + value = [...issues.values()]; + break; + case 'create_issue': + value = { + number: issues.size + 1, + ...args, + state: 'open', + labels: [], + }; + issues.set(value.number, value); + break; + case 'update_issue': + value = issues.get(args.number); + Object.assign(value, args); + break; + case 'add_doing_label': + value = issues.get(args.number); + value.labels.push('atomic:doing'); + break; + case 'remove_doing_label': + value = issues.get(args.number); + value.labels = value.labels.filter(l => l !== 'atomic:doing'); + break; + case 'create_comment': + value = { + id: comments.size + 1, + body: args.body, + issue_url: `https://api.github.com/repos/owner/repo/issues/${args.number}`, + }; + comments.set(value.id, value); + break; + case 'get_comment': + value = comments.get(args.id); + break; + case 'list_comments': + value = [...comments.values()]; + break; + case 'update_comment': + value = comments.get(args.id); + value.body = args.body; + break; + default: + throw new Error(action); + } + const receipt = { status: 200, body: JSON.stringify(value) }; + if (isWrite) receipts.set(id, receipt); + return receipt; + }, + ); + return { port, issues, comments, writes }; +} + +it('creates closed/doing issues and reopens while preserving unrelated labels', async () => { + const { port, issues } = github(); + const value = { title: 'Created closed', body: 'Markdown', status: 'Done' }; + const row = await port.create('issue', value, 'stable-create'); + expect(issues.get(row.id).state).toBe('closed'); + await port.create('issue', value, 'stable-create'); + expect(issues.size).toBe(2); + await port.update('issue', 1, { ...value, status: 'Doing' }, 'doing'); + expect(issues.get(1).labels).toEqual(['bug', 'atomic:doing']); + await port.update('issue', 1, value, 'close'); + await port.update('issue', 1, { ...value, status: 'Todo' }, 'reopen'); + expect(issues.get(1).state).toBe('open'); + expect(issues.get(1).labels).toEqual(['bug']); +}); + +it('creates/edits comments with replay-safe IDs and rejects another issue’s comment', async () => { + const { port, comments } = github(); + const context = { issueId: 1 }; + await port.create( + 'comment:x', + { body: 'Hi' }, + 'comment-create', + undefined, + context, + ); + await port.create( + 'comment:x', + { body: 'Hi' }, + 'comment-create', + undefined, + context, + ); + expect(comments.size).toBe(1); + await port.update( + 'comment:x', + 1, + { body: 'Edited' }, + 'comment-edit', + undefined, + context, + ); + expect(comments.get(1).body).toBe('Edited'); + await expect(port.get('comment:x', 1, { issueId: 2 })).rejects.toThrow( + 'another issue', + ); +}); + +it('fails on provider errors and scans all pages without importing pull requests', async () => { + const f = github(); + const row = f.issues.get(1); + let calls = 0; + const port = new GitHubPort(null, { repository: 'owner/repo' }, async () => ({ + status: 200, + body: JSON.stringify( + ++calls === 1 + ? Array.from({ length: 100 }, (_, i) => ({ + ...row, + number: i + 1, + ...(i === 0 ? { pull_request: {} } : {}), + })) + : [], + ), + })); + expect(await port.list('issue')).toHaveLength(99); + expect(calls).toBe(2); + port.call = async () => ({ status: 429, body: '{}' }); + await expect(port.list('issue')).rejects.toThrow('429'); +}); + +it('Atomic creates reuse native localId after a lost acknowledgement', async () => { + const stored = new Map(); + let lose = true; + const store = { + getServerUrl: () => 'https://atomic.example', + findByLocalId: async (_drive, parent, key) => + [...stored.values()].find(r => r.parent === parent && r.key === key), + newResource: async ({ parent, propVals }) => { + const r = { + subject: `did:ad:${stored.size + 1}`, + parent, + key: propVals['https://atomicdata.dev/properties/localId'], + async save() { + stored.set(r.subject, r); + if (lose) { + lose = false; + throw new Error('Lost acknowledgement'); + } + }, + }; + return r; + }, + }; + const port = new AtomicPort(store, { + connection: { + table: 'did:ad:table', + drive: 'did:ad:drive', + tags: { Todo: 'https://atomicdata.dev/task/v1/todo' }, + }, + }); + const value = { title: 'Same', body: '', status: 'Todo' }; + port.get = async (_entity, id) => ({ id, value }); + port.findByLocalId = (parent, key) => store.findByLocalId('', parent, key); + await expect(port.create('issue', value, 'key')).rejects.toThrow( + 'Lost acknowledgement', + ); + expect((await port.create('issue', value, 'key')).id).toBe('did:ad:1'); + expect(stored.size).toBe(1); +}); diff --git a/integrations/github-issues/devonian/proxy.mjs b/integrations/github-issues/devonian/proxy.mjs new file mode 100644 index 0000000000..e997b41311 --- /dev/null +++ b/integrations/github-issues/devonian/proxy.mjs @@ -0,0 +1,199 @@ +import { endpoint, request } from '../adapter.js'; +import { trackerAction } from '../tracker-actions.js'; + +/** Direct browser transport. Codes stay in tab storage, never in Atomic resources. */ +export function proxyTransport({ + url, + repository, + getCode, + setCode, + journal, + save, + fetcher = fetch, + dispatch, +}) { + const origin = new URL(url); + if ( + origin.protocol !== 'https:' && + !( + origin.protocol === 'http:' && + ['localhost', '127.0.0.1'].includes(origin.hostname) + ) + ) + throw new Error('Use an HTTPS proxy or loopback HTTP'); + const root = endpoint(repository); + let pending = Promise.resolve(); + return (action, args, id) => { + const operation = async () => { + if ( + action === 'get_issue' && + (!Number.isSafeInteger(args.number) || args.number <= 0) + ) + throw new Error('Invalid issue number'); + if ( + action === 'create_issue' && + (typeof args.title !== 'string' || + !args.title.trim() || + args.title.length > 256 || + (args.body !== undefined && typeof args.body !== 'string')) + ) + throw new Error('Invalid issue title or body'); + const intent = + trackerAction(repository, action, args) ?? + (action === 'get_issue' + ? request('get', 'GET', `${root}/${args.number}`, id) + : action === 'create_issue' + ? request('create', 'POST', root, id, args) + : undefined); + if (!intent) throw new Error('Unsupported GitHub action'); + const writes = intent.method !== 'GET'; + const signature = JSON.stringify({ action, args }); + const old = journal[id]; + if (writes && old) { + if (old.signature !== signature) + throw new Error('Operation identity reused with different arguments'); + if (old.receipt) return old.receipt; + throw new Error( + `Uncertain GitHub write (${action}). Inspect its outcome before retrying; it will not be resent.`, + ); + } + if (writes) { + journal[id] = { signature }; + await save(); + } + const target = new URL(intent.url); + let receipt; + let next = true; + if (dispatch) { + receipt = await dispatch(`${target.pathname}${target.search}`, { + method: intent.method, + ...(intent.body ? { body: intent.body } : {}), + }).catch(() => { + throw new Error( + 'Proxy request failed. Check CORS and reconnect; an uncertain write will not be resent.', + ); + }); + } else { + const code = getCode(); + if (!code) + throw new Error( + 'Connect to the proxy or supply a fresh connection code', + ); + setCode(''); + const destination = new URL( + `/proxy/github-issues${target.pathname}${target.search}`, + origin, + ); + const response = await fetcher(destination.href, { + method: intent.method, + redirect: 'error', + credentials: 'omit', + headers: { + Authorization: `Bearer ${code}`, + Accept: 'application/vnd.github+json', + 'Content-Type': 'application/json', + }, + ...(intent.body ? { body: intent.body } : {}), + }).catch(() => { + throw new Error( + 'Proxy request failed. Check CORS and reconnect; an uncertain write will not be resent.', + ); + }); + next = response.headers.get('X-Connection-Code'); + if (next) setCode(next); + receipt = { status: response.status, body: await response.text() }; + } + if (writes && receipt.status >= 200 && receipt.status < 300) { + journal[id].receipt = receipt; + await save(); + } + if (!next) + throw new Error( + 'Proxy did not expose X-Connection-Code. Enable CORS exposure and reconnect.', + ); + return receipt; + }; + const next = pending.then(operation); + pending = next.catch(() => {}); + return next; + }; +} + +/** An explicitly labelled GitHub fixture; no network or real GitHub mutations. */ +export function fixtureTransport(state, save) { + state.issues ??= [ + { + number: 1, + title: 'Welcome from GitHub', + body: 'Edit either tracker, then sync again.', + state: 'open', + labels: ['demo'], + }, + ]; + state.comments ??= []; + state.receipts ??= {}; + return async (action, args, id) => { + if (state.receipts[id]) return state.receipts[id]; + let value; + const issue = state.issues.find(r => r.number === args.number); + const comment = state.comments.find(r => r.id === args.id); + switch (action) { + case 'list_issues': + value = state.issues.slice((args.page - 1) * 100, args.page * 100); + break; + case 'get_issue': + value = issue; + break; + case 'create_issue': + value = { + number: Math.max(0, ...state.issues.map(r => r.number)) + 1, + ...args, + state: 'open', + labels: [], + }; + state.issues.push(value); + break; + case 'update_issue': + value = Object.assign(issue, args); + break; + case 'add_doing_label': + issue.labels = [...new Set([...issue.labels, 'atomic:doing'])]; + value = issue; + break; + case 'remove_doing_label': + issue.labels = issue.labels.filter(l => l !== 'atomic:doing'); + value = issue; + break; + case 'list_comments': + value = state.comments + .filter(c => c.issue_url.endsWith(`/${args.number}`)) + .slice((args.page - 1) * 100, args.page * 100); + break; + case 'get_comment': + value = comment; + break; + case 'create_comment': + value = { + id: Math.max(0, ...state.comments.map(r => r.id)) + 1, + body: args.body, + issue_url: `https://api.github.com/repos/demo/issues/issues/${args.number}`, + user: { login: 'demo-user' }, + }; + state.comments.push(value); + break; + case 'update_comment': + value = Object.assign(comment, { body: args.body }); + break; + default: + throw new Error(`Unsupported fixture action: ${action}`); + } + const receipt = { + status: value ? 200 : 404, + body: JSON.stringify(value ?? {}), + }; + if (!action.startsWith('get_') && !action.startsWith('list_')) + state.receipts[id] = receipt; + await save(); + return receipt; + }; +} diff --git a/integrations/github-issues/devonian/proxy.test.mjs b/integrations/github-issues/devonian/proxy.test.mjs new file mode 100644 index 0000000000..48bf983161 --- /dev/null +++ b/integrations/github-issues/devonian/proxy.test.mjs @@ -0,0 +1,100 @@ +import { expect, it } from 'vitest'; +import { proxyTransport } from './proxy.mjs'; + +it('serializes rotating codes, preserves query strings and checkpoints successful writes', async () => { + let code = 'first'; + const seen = []; + const journal = {}; + let saves = 0; + const call = proxyTransport({ + url: 'https://proxy.example', + repository: 'owner/repo', + journal, + getCode: () => code, + setCode: c => { + code = c; + }, + save: async () => { + saves++; + }, + fetcher: async (url, opts) => { + seen.push({ url, ...opts }); + return new Response('{"id":1}', { + headers: { 'X-Connection-Code': `next-${seen.length}` }, + }); + }, + }); + await Promise.all([ + call('list_issues', { page: 2 }, 'read'), + call('create_comment', { number: 1, body: 'Hello' }, 'write'), + ]); + expect(seen[0].url).toContain( + '/proxy/github-issues/repos/owner/repo/issues?state=all&per_page=100&page=2', + ); + expect(seen.map(r => r.headers.Authorization)).toEqual([ + 'Bearer first', + 'Bearer next-1', + ]); + expect(code).toBe('next-2'); + expect(saves).toBe(2); + await call('create_comment', { number: 1, body: 'Hello' }, 'write'); + expect(seen).toHaveLength(2); + await expect( + call('create_comment', { number: 1, body: 'Changed' }, 'write'), + ).rejects.toThrow('different arguments'); +}); + +it('never retries an uncertain create after a lost response or restart', async () => { + const journal = {}; + let writes = 0; + let code = 'first'; + const options = { + url: 'https://proxy.example', + repository: 'owner/repo', + journal, + getCode: () => code, + setCode: c => { + code = c; + }, + save: async () => {}, + fetcher: async () => { + writes++; + throw new Error('network'); + }, + }; + await expect( + proxyTransport(options)('create_issue', { title: 'Title' }, 'create'), + ).rejects.toThrow('Proxy request failed'); + code = 'reconnected'; + await expect( + proxyTransport(options)('create_issue', { title: 'Title' }, 'create'), + ).rejects.toThrow('Uncertain GitHub write'); + expect(writes).toBe(1); +}); + +it('retains rotated codes on provider errors and identifies missing exposed headers', async () => { + let code = 'first'; + const options = { + url: 'https://proxy.example', + repository: 'owner/repo', + journal: {}, + getCode: () => code, + setCode: c => { + code = c; + }, + save: async () => {}, + fetcher: async () => + new Response('{}', { + status: 403, + headers: { 'X-Connection-Code': 'rotated' }, + }), + }; + expect( + (await proxyTransport(options)('get_issue', { number: 1 }, 'read')).status, + ).toBe(403); + expect(code).toBe('rotated'); + options.fetcher = async () => new Response('{}'); + await expect( + proxyTransport(options)('get_issue', { number: 1 }, 'read'), + ).rejects.toThrow('expose X-Connection-Code'); +}); diff --git a/integrations/github-issues/devonian/vitest.config.mjs b/integrations/github-issues/devonian/vitest.config.mjs new file mode 100644 index 0000000000..1b8f18ff79 --- /dev/null +++ b/integrations/github-issues/devonian/vitest.config.mjs @@ -0,0 +1,14 @@ +import { resolve } from 'node:path'; + +export default { + resolve: { + alias: { + devonian: process.env.DEVONIAN_PATH + ? resolve(process.env.DEVONIAN_PATH, 'build/src/main.js') + : resolve('browser/data-browser/src/chunks/DevonianDemo/devonian.js'), + '@tomic/lib': resolve('browser/lib/src/index.ts'), + vitest: resolve('browser/node_modules/vitest/dist/index.js'), + }, + }, + test: { include: ['integrations/github-issues/devonian/*.test.*'] }, +}; diff --git a/integrations/github-issues/tracker-actions.ts b/integrations/github-issues/tracker-actions.ts new file mode 100644 index 0000000000..47e9096c14 --- /dev/null +++ b/integrations/github-issues/tracker-actions.ts @@ -0,0 +1,190 @@ +/** Additional repository-scoped actions used by the Devonian tracker bridge. */ +import { endpoint, request } from './adapter.js'; + +const integer = { + type: 'integer' as const, + description: 'Positive identifier or page number', +}; +const string = { + type: 'string' as const, + description: 'Issue or comment field value', +}; +const definitions = [ + ['list_issues', 'List issues', 'list', { page: integer }, ['page']], + [ + 'update_issue', + 'Update an issue', + 'update', + { number: integer, title: string, body: string, state: string }, + ['number', 'title', 'body', 'state'], + ], + [ + 'add_doing_label', + 'Mark an issue as doing', + 'doing-add', + { number: integer }, + ['number'], + ], + [ + 'remove_doing_label', + 'Remove the doing label', + 'doing-remove', + { number: integer }, + ['number'], + ], + [ + 'list_comments', + 'List issue comments', + 'comments-list', + { number: integer, page: integer }, + ['number', 'page'], + ], + [ + 'get_comment', + 'Get an issue comment', + 'comments-get', + { id: integer }, + ['id'], + ], + [ + 'create_comment', + 'Create an issue comment', + 'comments-create', + { number: integer, body: string }, + ['number', 'body'], + ], + [ + 'update_comment', + 'Update an issue comment', + 'comments-update', + { id: integer, body: string }, + ['id', 'body'], + ], +] as const; + +export const trackerActions = definitions.map( + ([name, title, operation, properties, required]) => ({ + name, + title, + description: `${title} in this connected repository. Writes require approval.`, + operation, + inputSchema: { + type: 'object', + properties, + required: [...required], + additionalProperties: false, + }, + }), +); + +export function trackerOperations(root: string) { + return [ + { + id: 'comments-list', + method: 'GET', + url: `${root}/{number}/comments`, + effect: 'read', + }, + { + id: 'comments-get', + method: 'GET', + url: `${root}/comments/{id}`, + effect: 'read', + }, + { + id: 'comments-create', + method: 'POST', + url: `${root}/{number}/comments`, + effect: 'write', + }, + { + id: 'comments-update', + method: 'PATCH', + url: `${root}/comments/{id}`, + effect: 'write', + }, + ]; +} + +export function trackerAction( + repository: string, + action: string, + args: Record, +) { + const definition = definitions.find(d => d[0] === action); + if (!definition) return undefined; + const [, , operation, properties, required] = definition; + for (const key of required) { + if (!(key in args)) throw new Error(`Missing ${key}`); + } + for (const [key, value] of Object.entries(args)) { + if (!(key in properties)) throw new Error(`Unexpected ${key}`); + if (['number', 'id', 'page'].includes(key)) { + if ( + !Number.isSafeInteger(value) || + Number(value) <= 0 || + (key === 'page' && Number(value) > 100) + ) + throw new Error(`Invalid ${key}`); + } else if (typeof value !== 'string') throw new Error(`Invalid ${key}`); + } + const root = endpoint(repository); + switch (action) { + case 'list_issues': + return request( + operation, + 'GET', + `${root}?state=all&per_page=100&page=${args.page}&sort=created&direction=asc`, + 'action', + ); + case 'list_comments': + return request( + operation, + 'GET', + `${root}/${args.number}/comments?per_page=100&page=${args.page}`, + 'action', + ); + case 'get_comment': + return request(operation, 'GET', `${root}/comments/${args.id}`, 'action'); + case 'create_comment': + case 'update_comment': + if (!(args.body as string).trim()) + throw new Error('Comment body cannot be empty'); + return request( + operation, + action === 'create_comment' ? 'POST' : 'PATCH', + action === 'create_comment' + ? `${root}/${args.number}/comments` + : `${root}/comments/${args.id}`, + 'action', + { body: args.body }, + ); + case 'add_doing_label': + return request( + operation, + 'POST', + `${root}/${args.number}/labels`, + 'action', + { + labels: ['atomic:doing'], + }, + ); + case 'remove_doing_label': + return request( + operation, + 'DELETE', + `${root}/${args.number}/labels/atomic%3Adoing`, + 'action', + ); + case 'update_issue': + if (!['open', 'closed'].includes(String(args.state))) + throw new Error('Invalid issue state'); + if (!(args.title as string).trim() || (args.title as string).length > 256) + throw new Error('Invalid issue title'); + return request(operation, 'PATCH', `${root}/${args.number}`, 'action', { + title: args.title, + body: args.body, + state: args.state, + }); + } +} diff --git a/integrations/localthought/README.md b/integrations/localthought/README.md index 9ea97f7efc..3674472724 100644 --- a/integrations/localthought/README.md +++ b/integrations/localthought/README.md @@ -82,3 +82,53 @@ An unbounded fetch successfully traversed multiple pages but exceeded the 5,000-record preview limit; the UI now defaults to the next 30 days. Date bounds and recurrence expansion are passed to Syncables as collection query settings. The importer remains a manual snapshot, not a background sync. + +## Calendar view + +Google event imports now install a Calendar view alongside the source table. +The projected date uses the day in Google's supplied start offset (or the +unchanged all-day date), so mixed all-day/timed events share one DATE column. +The original Start and End objects retain timezones and exclusive end values. +The month view shows each event on its start day; it does not draw duration or +multi-day spans. Additional notes identify recurring events, attendees, +reminders and conferencing when those fields are returned by the catalog. +Recurring instances are expanded by the existing bounded provider fetch. + +Use **Fetch and preview** again to refresh, then review and apply changes. +Existing source identities and import baselines prevent duplicates and preserve +Atomic-only fields and local edits. Cancellation records are retained with a +note; absence from a bounded fetch never deletes an Atomic resource. Google +may omit cancelled events from list results, so this is not a deletion feed. +Removed optional provider fields are not cleared by the shared snapshot importer. +This first version deliberately remains manual and one-way. No OAuth write +scope is requested and edits in Atomic do not update Google. + +`calendar.test.ts` exercises mixed dates, offset boundaries, exclusive ends, +recurrence/attendee notes, cancellations, malformed starts, cross-calendar +identity and repeated imports with private local fields. + +## Browser-only Calendar regression + +`browser/e2e/tests/google-calendar-import.spec.mts` starts the shared mock +integration-proxy from #1399 with a synthetic Google Calendar. The test enters +the public fixture tenant secret, completes consent, and exercises real browser +credential rotation, WASM pagination, local schema installation, proposal review, +OPFS application, and Calendar rendering. It refreshes changed provider data +and checks that native identities and Atomic-only notes survive reload. +AtomicServer HTTP and all WebSockets are blocked throughout; only GET requests +are permitted for provider data. The configured LocalThought origin is forwarded +to the isolated HTTP fixture, so no live provider credentials or data are used. + +Run with a dev frontend built from this branch and its matching WASM bundle: + +```sh +FRONTEND_URL=http://127.0.0.1:6747 SERVER_URL=http://127.0.0.1:19999 \ + browser/e2e/node_modules/.bin/playwright test \ + --config browser/e2e/playwright.config.ts \ + browser/e2e/tests/google-calendar-import.spec.mts --project chromium +``` + +If the frontend uses `VITE_INTEGRATION_PROXY_URL`, pass the same value to the +test process. The test forwards that origin to its own fixture. Live Google +OAuth on the browser path still depends on the proxy CORS deployment described +above; this fixture test does not claim live-provider verification. diff --git a/integrations/localthought/browser.test.ts b/integrations/localthought/browser.test.ts index d7194196bf..87a8081208 100644 --- a/integrations/localthought/browser.test.ts +++ b/integrations/localthought/browser.test.ts @@ -124,3 +124,37 @@ it('calls the browser fetch function without binding it to the client', async () const client = new BrowserIntegrations(storage, async () => engine, origin); expect(await client.catalog()).toEqual(['pets']); }); + +it('supports the demo callback and write credentials without using the import engine', async () => { + const { client, http, values } = setup(); + const { state } = await client.start( + 'drive', + 'actor', + 'pets', + 'https://atomic.example/app/devonian-demo', + secret, + ); + client.finish('drive', 'actor', state, 'first'); + http.mockImplementation(async (_url, init) => { + expect(JSON.parse([...values.values()][0]).code).toBeUndefined(); + expect(init?.method).toBe('POST'); + expect(init?.body).toBe('{"title":"new"}'); + return new Response('{"id":1}', { + status: 201, + headers: { 'X-Connection-Code': 'next' }, + }); + }); + await expect( + client.request('drive', 'actor', state, 'github-issues', '/issues'), + ).rejects.toThrow('another platform'); + await expect( + client.request('drive', 'actor', state, 'pets', '//evil.example'), + ).rejects.toThrow('Invalid proxy path'); + expect( + await client.request('drive', 'actor', state, 'pets', '/issues', { + method: 'POST', + body: '{"title":"new"}', + }), + ).toEqual({ status: 201, body: '{"id":1}' }); + expect(JSON.parse([...values.values()][0]).code).toBe('next'); +}); diff --git a/integrations/localthought/browser.ts b/integrations/localthought/browser.ts index 9ed1248fbc..94350c048f 100644 --- a/integrations/localthought/browser.ts +++ b/integrations/localthought/browser.ts @@ -142,7 +142,9 @@ export class BrowserIntegrations { const callback = new URL(returnUrl); if ( callback.origin !== location.origin || - callback.pathname !== '/app/integrations' || + !['/app/integrations', '/app/devonian-demo'].includes( + callback.pathname, + ) || callback.search || callback.hash ) @@ -196,6 +198,72 @@ export class BrowserIntegrations { ); return { connection: state, platform: c.platform }; } + /** Shared rotating-code transport for browser-owned writes as well as reads. */ + async request( + drive: string, + actor: string, + id: string, + platform: string, + path: string, + init: { method?: string; body?: string } = {}, + ): Promise<{ status: number; body: string }> { + if (!navigator.locks) + throw new Error('This browser needs Web Locks for integrations'); + if (!path.startsWith('/') || path.startsWith('//') || /[\\\\#]/.test(path)) + throw new Error('Invalid proxy path'); + const destination = new URL(`/proxy/${platform}${path}`, this.origin); + if (!destination.pathname.startsWith(`/proxy/${platform}/`)) + throw new Error('Invalid proxy path'); + return navigator.locks.request(key + id, async () => { + const c = this.connection(id, drive, actor); + if (c.platform !== platform) + throw new Error('Connection belongs to another platform'); + const response = await this.send( + id, + drive, + actor, + path, + init, + AbortSignal.timeout(30000), + ); + return { status: response.status, body: await limitedText(response) }; + }); + } + private async send( + id: string, + drive: string, + actor: string, + path: string, + init: { method?: string; body?: string }, + signal: AbortSignal, + ) { + const current = this.connection(id, drive, actor); + if (!current.ready || !current.code) + throw new Error('Reconnect before retrying an uncertain request'); + const code = current.code; + delete current.code; + this.storage.setItem(key + id, JSON.stringify(current)); + const response = await this.http( + `${this.origin}/proxy/${current.platform}${path}`, + { + ...init, + headers: { + Authorization: `Bearer ${code}`, + 'Content-Type': 'application/json', + }, + credentials: 'omit', + redirect: 'error', + signal, + }, + ); + const next = response.headers.get('x-connection-code'); + if (!next) + throw new Error( + 'Proxy did not expose a rotated code; reconnect and check CORS', + ); + this.storage.setItem(key + id, JSON.stringify({ ...current, code: next })); + return response; + } async fetchRecords( drive: string, actor: string, @@ -227,32 +295,14 @@ export class BrowserIntegrations { throw new Error('Pagination left the catalog API origin'); if (++requests > 200) throw new Error('Import exceeds 200 requests; narrow its scope'); - const current = this.connection(id, drive, actor); - if (!current.code) - throw new Error('Reconnect before retrying an uncertain request'); - const code = current.code; - delete current.code; - // Consume durably BEFORE dispatch; a lost response must never replay a code. - this.storage.setItem(key + id, JSON.stringify(current)); - const response = await this.http( - `${this.origin}/proxy/${c.platform}${target.pathname}${target.search}`, - { - headers: { Authorization: `Bearer ${code}` }, - credentials: 'omit', - redirect: 'error', - signal, - }, + const response = await this.send( + id, + drive, + actor, + `${target.pathname}${target.search}`, + {}, + signal, ); - const next = response.headers.get('x-connection-code'); - if (next) - this.storage.setItem( - key + id, - JSON.stringify({ ...current, code: next }), - ); - else - throw new Error( - 'Proxy did not expose a rotated code; reconnect and check CORS', - ); const headers = Object.fromEntries( [...response.headers].filter( ([name]) => name !== 'x-connection-code', diff --git a/integrations/localthought/calendar.test.ts b/integrations/localthought/calendar.test.ts new file mode 100644 index 0000000000..1e14884ff4 --- /dev/null +++ b/integrations/localthought/calendar.test.ts @@ -0,0 +1,151 @@ +import { expect, it } from 'vitest'; +import { calendarProjection, calendarFields as fields } from './calendar'; +import { Datatype } from '../../browser/lib/src/index'; +import { run } from './plugin'; +import type { FetchedPlatform } from './schema'; +const fixture = (): FetchedPlatform => ({ + platform: 'google-calendar', + ontology: { + description: '', + terms: [ + { + path: 'event', + kind: 'class', + shortname: 'event', + description: '', + datatype: Datatype.STRING, + requires: [], + recommends: [], + }, + ], + }, + records: [ + { + resource: 'event', + namespace: 'calendar-a', + id: '1', + name: 'Meeting', + values: { + start: { + dateTime: '2026-09-10T00:30:00+02:00', + timeZone: 'Europe/Amsterdam', + }, + end: { dateTime: '2026-09-10T01:30:00+02:00' }, + attendees: [{ email: 'test@example.com' }], + 'recurring-event-id': 'series', + reminders: { useDefault: true }, + }, + }, + ], +}); +it('projects timed events without shifting their day and retains provider structures', () => { + const input = fixture(); + const output = calendarProjection(input); + expect(output.records[0].values).toMatchObject({ + ...input.records[0].values, + [fields.day]: '2026-09-10', + [fields.allDay]: false, + }); + expect(output.records[0].values[fields.notes]).toContain('Recurring'); + expect(output.records[0].values[fields.notes]).toContain('Attendees'); + expect(input.records[0].values[fields.day]).toBeUndefined(); + expect( + output.ontology.terms.find(t => t.shortname === fields.day)?.datatype, + ).toBe(Datatype.DATE); +}); +it('supports all-day dates and preserves exclusive multi-day ends', () => { + const input = fixture(); + input.records[0].values = { + start: { date: '2026-09-10' }, + end: { date: '2026-09-13' }, + }; + expect(calendarProjection(input).records[0].values).toMatchObject({ + [fields.day]: '2026-09-10', + [fields.allDay]: true, + end: { date: '2026-09-13' }, + }); +}); +it('retains cancelled events even when Google omits start', () => { + const input = fixture(); + input.records[0].values = { status: 'cancelled' }; + expect(calendarProjection(input).records[0].values[fields.notes]).toContain( + 'Cancelled', + ); +}); +it('fails the projection on malformed active events instead of silently losing rows', () => { + for (const start of [ + { date: '2026-02-30' }, + { date: '2026-09-10garbage' }, + {}, + { dateTime: '2026-09-10T12:00:00' }, + ]) { + const input = fixture(); + input.records[0].values = { start }; + expect(() => calendarProjection(input)).toThrow(); + } +}); +it('leaves other platforms untouched', () => { + const input = fixture(); + input.platform = 'pets'; + expect(calendarProjection(input)).toBe(input); +}); +it("repeated imports reconcile IDs, preserve local fields and don't delete out-of-window events", () => { + const data = calendarProjection(fixture()); + const properties = Object.fromEntries( + Object.keys(data.records[0].values).map(k => [ + k, + `https://example.com/${k}`, + ]), + ); + const config = { + platform: data.platform, + destinations: { + event: { table: 'did:ad:table', rowClass: 'did:ad:event' }, + }, + properties, + records: data.records, + }; + const first = run({ config, query: () => [], read: () => ({}) }).intents[0]; + if (first.op !== 'create') throw new Error('Expected create'); + const saved = { + ...first.set, + 'https://atomicdata.dev/properties/parent': 'did:ad:table', + 'https://atomicdata.dev/properties/isA': ['did:ad:event'], + 'https://example.com/private': 'my notes', + }; + const host = { + config, + query: (p: string, v: string) => (saved[p] === v ? ['did:ad:row'] : []), + read: () => saved, + }; + expect(run(host).intents).toHaveLength(0); + expect( + run({ ...host, config: { ...config, records: [] } }).intents, + ).toHaveLength(0); + const changed = calendarProjection(fixture()); + changed.records[0].name = 'Updated meeting'; + const result = run({ + ...host, + config: { ...config, records: changed.records }, + }); + expect(result.intents[0]).toMatchObject({ + op: 'set', + subject: 'did:ad:row', + set: { 'https://atomicdata.dev/properties/name': 'Updated meeting' }, + }); + expect(JSON.stringify(result.intents)).not.toContain('my notes'); + const secondCalendar = { ...data.records[0], namespace: 'calendar-b' }; + expect( + run({ ...host, config: { ...config, records: [secondCalendar] } }) + .intents[0].op, + ).toBe('create'); +}); +it('recognizes the lowercase field names produced by the WASM ontology', () => { + const input = fixture(); + delete input.records[0].values['recurring-event-id']; + input.records[0].values.recurringeventid = 'series'; + input.records[0].values.conferencedata = { conferenceId: 'meeting' }; + const notes = calendarProjection(input).records[0].values[fields.notes]; + expect(notes).toContain('Recurring event'); + expect(notes).toContain('Conferencing'); +}); diff --git a/integrations/localthought/calendar.ts b/integrations/localthought/calendar.ts new file mode 100644 index 0000000000..f9706e8792 --- /dev/null +++ b/integrations/localthought/calendar.ts @@ -0,0 +1,146 @@ +// @wc-ignore-file +import { Datatype } from '../../browser/lib/src/index.js'; +import type { JSONValue } from '../../browser/lib/src/value.js'; +import type { FetchedPlatform, Term } from './schema.js'; + +export const calendarFields = { + day: 'atomic-calendar-day', + allDay: 'atomic-calendar-all-day', + notes: 'atomic-calendar-notes', +}; + +/** An additional projection, never a replacement for the provider's fields. + * One DATE column supports both all-day dates and timed events in a single view. + * Timed events use the date in Google's supplied offset; raw start/end retain + * the instant, zone and exclusive-end semantics for future richer rendering. + */ +export function calendarProjection(fetched: FetchedPlatform): FetchedPlatform { + if (fetched.platform !== 'google-calendar') return fetched; + const event = fetched.ontology.terms.find( + t => t.kind === 'class' && t.shortname === 'event', + ); + if (!event) return fetched; + const definitions: [string, Datatype, string][] = [ + [ + calendarFields.day, + Datatype.DATE, + "Start date in the event's supplied offset; all-day dates stay unchanged.", + ], + [ + calendarFields.allDay, + Datatype.BOOLEAN, + 'Whether Google represents this as an all-day event.', + ], + [ + calendarFields.notes, + Datatype.STRING, + 'Display limitations and Google-only features. Edits here remain local.', + ], + ]; + const terms: Term[] = definitions.map( + ([shortname, datatype, description]) => ({ + path: `urn:atomic:google-calendar:${shortname}`, + kind: 'property', + shortname, + datatype, + description, + requires: [], + recommends: [], + }), + ); + if ( + fetched.ontology.terms.some(t => + terms.some(extra => extra.shortname === t.shortname), + ) + ) + throw new Error( + 'Calendar projection property collides with provider ontology', + ); + return { + ...fetched, + ontology: { + ...fetched.ontology, + terms: [ + ...fetched.ontology.terms.map(t => + t === event + ? { + ...t, + recommends: [ + ...t.recommends, + ...terms.map(extra => extra.path), + ], + } + : t, + ), + ...terms, + ], + }, + records: fetched.records.map(row => { + if (row.resource !== 'event') return row; + const start = object(row.values.start); + const end = object(row.values.end); + const date = start.date ?? start.dateTime; + const cancelled = row.values.status === 'cancelled'; + if ( + !cancelled && + (typeof date !== 'string' || !validDay(date.slice(0, 10))) + ) + throw new Error(`Calendar event ${row.id} has no valid start date`); + if ( + start.date !== undefined && + (typeof start.date !== 'string' || !validDay(start.date)) + ) + throw new Error(`Calendar event ${row.id} has an invalid all-day date`); + if ( + start.dateTime !== undefined && + (typeof start.dateTime !== 'string' || + !/^\d{4}-\d{2}-\d{2}T.*(?:Z|[+-]\d{2}:\d{2})$/.test(start.dateTime) || + !Number.isFinite(Date.parse(start.dateTime))) + ) + throw new Error( + `Calendar event ${row.id} has no offset-qualified start time`, + ); + const notes = ['One-way import: edits stay in Atomic']; + if (cancelled) notes.push('Cancelled in Google; retained in Atomic'); + if ( + row.values['recurring-event-id'] || + row.values.recurringEventId || + row.values.recurringeventid || + row.values.recurrence + ) + notes.push('Recurring event: manage the series in Google'); + if (Array.isArray(row.values.attendees) && row.values.attendees.length) + notes.push('Attendees and RSVP: manage in Google'); + if (row.values.reminders) notes.push('Reminders: manage in Google'); + if ( + row.values['conference-data'] || + row.values.conferenceData || + row.values.conferencedata + ) + notes.push('Conferencing: manage in Google'); + if (end.date || end.dateTime) + notes.push('Duration retained in End; shown on start day only'); + const values: Record = { + ...row.values, + [calendarFields.notes]: notes.join('. '), + }; + if (typeof date === 'string' && validDay(date.slice(0, 10))) { + values[calendarFields.day] = date.slice(0, 10); + values[calendarFields.allDay] = typeof start.date === 'string'; + } + return { ...row, values }; + }), + }; +} +function object(value: JSONValue | undefined): Record { + return value && typeof value === 'object' && !Array.isArray(value) + ? value + : {}; +} +function validDay(value: string): boolean { + if (!/^\d{4}-\d{2}-\d{2}$/.test(value)) return false; + const date = new Date(`${value}T00:00:00Z`); + return ( + Number.isFinite(date.getTime()) && date.toISOString().slice(0, 10) === value + ); +} diff --git a/integrations/localthought/mock-calendar.mjs b/integrations/localthought/mock-calendar.mjs new file mode 100644 index 0000000000..13ba1e576a --- /dev/null +++ b/integrations/localthought/mock-calendar.mjs @@ -0,0 +1,159 @@ +/** Synthetic Google Calendar fixtures. No real calendar data or provider writes. */ +const string = { type: 'string' }; +const dateTime = { + type: 'object', + properties: { + date: { ...string, format: 'date' }, + dateTime: { ...string, format: 'date-time' }, + timeZone: string, + }, +}; +export const calendarDocument = { + openapi: '3.0.3', + info: { title: 'Synthetic Calendar', version: 'v3' }, + servers: [{ url: 'https://www.googleapis.com/calendar/v3' }], + paths: { + '/calendars/{calendarId}/events': { + get: { + parameters: [ + { name: 'calendarId', in: 'path', required: true, schema: string }, + ...['pageToken', 'timeMin', 'timeMax'].map(name => ({ + name, + in: 'query', + schema: string, + })), + { name: 'singleEvents', in: 'query', schema: { type: 'boolean' } }, + ], + 'x-pagination': [{ scheme: 'pageToken' }], + responses: { + 200: { + description: 'Events', + content: { + 'application/json': { + schema: { + type: 'object', + properties: { + items: { + type: 'array', + items: { $ref: '#/components/schemas/event' }, + }, + nextPageToken: string, + }, + }, + }, + }, + }, + }, + }, + }, + }, + components: { + paginationSchemes: { + pageToken: { + type: 'pageToken', + request: { queryParameters: { pageToken: { role: 'pageToken' } } }, + response: { bodyFields: { nextPageToken: { role: 'nextPageToken' } } }, + }, + }, + schemas: { + event: { + type: 'object', + properties: { + id: string, + summary: string, + status: string, + start: dateTime, + end: dateTime, + recurringEventId: string, + htmlLink: string, + attendees: { + type: 'array', + items: { + type: 'object', + properties: { email: string, responseStatus: string }, + }, + }, + }, + }, + }, + crudResources: { + event: { + schema: { $ref: '#/components/schemas/event' }, + identity: { + urlTemplate: '/calendars/{calendarId}/events/{eventId}', + bindings: { eventId: { field: 'id' } }, + }, + collections: { + events: { urlTemplate: '/calendars/{calendarId}/events' }, + }, + }, + }, + }, +}; +export function calendarFixture(day = new Date().toISOString().slice(0, 10)) { + const tomorrow = new Date(Date.parse(`${day}T00:00:00Z`) + 86400000) + .toISOString() + .slice(0, 10); + const events = [ + { + id: 'all-day', + summary: 'Calendar all-day fixture', + status: 'confirmed', + start: { date: day }, + end: { date: tomorrow }, + }, + { + id: 'timed', + summary: 'Calendar timed fixture', + status: 'confirmed', + start: { + dateTime: `${day}T00:30:00+02:00`, + timeZone: 'Europe/Amsterdam', + }, + end: { dateTime: `${day}T01:30:00+02:00`, timeZone: 'Europe/Amsterdam' }, + recurringEventId: 'series', + attendees: [ + { email: 'synthetic@example.com', responseStatus: 'accepted' }, + ], + }, + ]; + const requests = []; + return { + events, + requests, + request(method, url) { + requests.push({ + method, + path: url.pathname, + query: Object.fromEntries(url.searchParams), + }); + if (method !== 'GET') return { status: 405, body: {} }; + if ( + !/^\/proxy\/google-calendar\/calendar\/v3\/calendars\/[^/]+\/events$/.test( + url.pathname, + ) + ) + return { status: 404, body: {} }; + // Require the production adapter to send its safety bounds and recurrence expansion. + if ( + !url.searchParams.has('timeMin') || + !url.searchParams.has('timeMax') || + url.searchParams.get('singleEvents') !== 'true' + ) + return { + status: 400, + body: { error: 'Missing bounded recurrence query' }, + }; + const token = url.searchParams.get('pageToken'); + if (token && token !== 'second') return { status: 400, body: {} }; + return { + status: 200, + body: structuredClone( + token + ? { items: events.slice(1) } + : { items: events.slice(0, 1), nextPageToken: 'second' }, + ), + }; + }, + }; +} diff --git a/integrations/localthought/mock-github.mjs b/integrations/localthought/mock-github.mjs new file mode 100644 index 0000000000..629a1e28e1 --- /dev/null +++ b/integrations/localthought/mock-github.mjs @@ -0,0 +1,108 @@ +/** Stateful provider fixture, shared by the HTTP proxy and its test-side driver. */ +export function githubTracker() { + const repositories = new Map(); + const repo = name => { + if (!repositories.has(name)) + repositories.set(name, { issues: [], comments: [] }); + return repositories.get(name); + }; + let id = 0; + const now = () => new Date().toISOString(); + const api = { + snapshot: name => structuredClone(repo(name)), + createIssue(name, input) { + const state = repo(name); + const number = state.issues.length + 1; + const issue = { + id: ++id, + number, + title: input.title, + body: input.body ?? '', + state: 'open', + labels: [], + url: `https://api.github.com/repos/${name}/issues/${number}`, + html_url: `https://github.com/${name}/issues/${number}`, + user: { login: 'mock-user' }, + created_at: now(), + updated_at: now(), + }; + state.issues.push(issue); + return structuredClone(issue); + }, + updateIssue(name, number, input) { + const issue = repo(name).issues.find(i => i.number === number); + if (!issue) return; + for (const field of ['title', 'body', 'state', 'labels']) + if (input[field] !== undefined) issue[field] = input[field]; + issue.updated_at = now(); + return structuredClone(issue); + }, + createComment(name, number, input) { + if (!repo(name).issues.some(i => i.number === number)) return; + const comment = { + id: ++id, + body: input.body, + issue_url: `https://api.github.com/repos/${name}/issues/${number}`, + user: { login: 'mock-commenter' }, + created_at: now(), + updated_at: now(), + }; + repo(name).comments.push(comment); + return structuredClone(comment); + }, + request(method, url, input = {}) { + const match = url.pathname.match( + /^\/proxy\/github-issues\/repos\/([^/]+\/[^/]+)\/issues(?:\/(.*))?$/, + ); + if (!match) return { status: 404, body: {} }; + const [, name, tail = ''] = match; + const state = repo(name); + const page = Number(url.searchParams.get('page') ?? 1); + const size = Number(url.searchParams.get('per_page') ?? 100); + const paginate = rows => rows.slice((page - 1) * size, page * size); + let value; + if (!tail) { + if (method === 'GET') value = paginate(state.issues); + if (method === 'POST') value = api.createIssue(name, input); + } else if (/^comments\/\d+$/.test(tail)) { + const comment = state.comments.find( + c => c.id === Number(tail.split('/')[1]), + ); + if (method === 'GET') value = comment; + if (method === 'PATCH' && comment) + value = Object.assign(comment, { + body: input.body, + updated_at: now(), + }); + } else { + const [numberText, resource, label] = tail.split('/'); + const number = Number(numberText); + const issue = state.issues.find(i => i.number === number); + if (issue && !resource) { + if (method === 'GET') value = issue; + if (method === 'PATCH') value = api.updateIssue(name, number, input); + } else if (issue && resource === 'comments') { + if (method === 'GET') + value = paginate( + state.comments.filter(c => c.issue_url === issue.url), + ); + if (method === 'POST') value = api.createComment(name, number, input); + } else if (issue && resource === 'labels') { + if (method === 'POST') + value = issue.labels = [ + ...new Set([...issue.labels, ...input.labels]), + ]; + if (method === 'DELETE') + value = issue.labels = issue.labels.filter( + l => l !== decodeURIComponent(label), + ); + } + } + return { + status: value === undefined ? 404 : method === 'POST' ? 201 : 200, + body: structuredClone(value ?? {}), + }; + }, + }; + return api; +} diff --git a/integrations/localthought/mock-proxy.d.mts b/integrations/localthought/mock-proxy.d.mts new file mode 100644 index 0000000000..72bdfcb1bd --- /dev/null +++ b/integrations/localthought/mock-proxy.d.mts @@ -0,0 +1,49 @@ +import type { Server } from 'node:http'; +interface Issue { + id: number; + number: number; + title: string; + body: string; + state: string; + labels: string[]; +} +interface Comment { + id: number; + body: string; + issue_url: string; +} +interface GitHubTracker { + snapshot(repository: string): { issues: Issue[]; comments: Comment[] }; + createIssue( + repository: string, + input: { title: string; body?: string }, + ): Issue; + updateIssue( + repository: string, + number: number, + input: Partial, + ): Issue | undefined; + createComment( + repository: string, + number: number, + input: { body: string }, + ): Comment | undefined; +} +interface CalendarFixture { + events: Array<{ + id: string; + summary: string; + status: string; + start: { date?: string; dateTime?: string; timeZone?: string }; + end: { date?: string; dateTime?: string; timeZone?: string }; + }>; + requests: Array<{ + method: string; + path: string; + query: Record; + }>; +} +export const tenantSecret: string; +export function mockProxy(options?: { + frontendOrigin?: string; +}): Server & { github: GitHubTracker; calendar: CalendarFixture }; diff --git a/integrations/localthought/mock-proxy.mjs b/integrations/localthought/mock-proxy.mjs index 650c9aee31..445f8d1749 100644 --- a/integrations/localthought/mock-proxy.mjs +++ b/integrations/localthought/mock-proxy.mjs @@ -1,54 +1,169 @@ /** Local-only integration-proxy fixture. Never deploy this service. */ +import { calendarDocument, calendarFixture } from './mock-calendar.mjs'; +import { githubTracker } from './mock-github.mjs'; import { readFileSync } from 'node:fs'; import { createServer } from 'node:http'; import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto'; import { pathToFileURL } from 'node:url'; export const tenantSecret = 'bW9jay10ZW5hbnQ.mock-signature'; -const sign = value => createHmac('sha256', tenantSecret).update(value).digest('base64url'); -const equal = (a, b) => typeof a === 'string' && a.length === b.length && timingSafeEqual(Buffer.from(a), Buffer.from(b)); -const pets = ['Rex', 'Whiskers', 'Tweety', 'Nibbles', 'Bubbles'].map((name, i) => ({ id: i + 1, name, species: ['Dog', 'Cat', 'Bird', 'Rabbit', 'Fish'][i], age: i + 1, vaccinated: i % 2 === 0, weight: i + 0.5, updated_at: '2026-09-09T00:00:00Z' })); -export function mockProxy() { +const sign = value => + createHmac('sha256', tenantSecret).update(value).digest('base64url'); +const equal = (a, b) => + typeof a === 'string' && + a.length === b.length && + timingSafeEqual(Buffer.from(a), Buffer.from(b)); +const pets = ['Rex', 'Whiskers', 'Tweety', 'Nibbles', 'Bubbles'].map( + (name, i) => ({ + id: i + 1, + name, + species: ['Dog', 'Cat', 'Bird', 'Rabbit', 'Fish'][i], + age: i + 1, + vaccinated: i % 2 === 0, + weight: i + 0.5, + updated_at: '2026-09-09T00:00:00Z', + }), +); +export function mockProxy({ + frontendOrigin = process.env.MOCK_FRONTEND_ORIGIN ?? 'http://localhost:6747', +} = {}) { + const github = githubTracker(); + const calendar = calendarFixture(); const codes = new Map(); const challenges = new Set(); - const issueCode = platform => { const code = randomBytes(32).toString('base64url'); codes.set(code, platform); return code; }; - return createServer((req, res) => { + const issueCode = platform => { + const code = randomBytes(32).toString('base64url'); + codes.set(code, platform); + return code; + }; + const server = createServer(async (req, res) => { res.setHeader('Access-Control-Allow-Origin', '*'); - res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS'); - res.setHeader('Access-Control-Allow-Headers', 'Authorization, Content-Type'); + res.setHeader( + 'Access-Control-Allow-Methods', + 'GET, POST, PATCH, DELETE, OPTIONS', + ); + res.setHeader( + 'Access-Control-Allow-Headers', + 'Authorization, Content-Type', + ); res.setHeader('Access-Control-Expose-Headers', 'X-Connection-Code, Link'); - if (req.method === 'OPTIONS') { res.writeHead(204); return res.end(); } + if (req.method === 'OPTIONS') { + res.writeHead(204); + return res.end(); + } const url = new URL(req.url, 'http://localhost'); - const json = (status, value, headers = {}) => { res.writeHead(status, { 'Content-Type': 'application/json', ...headers }); res.end(JSON.stringify(value)); }; - if (url.pathname === '/catalog') return json(200, ['github-issues', 'google-calendar', 'pets']); - if (url.pathname === '/catalog/pets.yaml') { res.writeHead(200, { 'Content-Type': 'application/yaml' }); return res.end(readFileSync(new URL('./mock-document.json', import.meta.url))); } - if (url.pathname === '/session') { const challenge = randomBytes(32).toString('base64url'); challenges.add(challenge); return json(200, { ts: Math.floor(Date.now() / 1000), nonce: randomBytes(16).toString('hex'), challenge }); } + const json = (status, value, headers = {}) => { + res.writeHead(status, { 'Content-Type': 'application/json', ...headers }); + res.end(JSON.stringify(value)); + }; + if (url.pathname === '/catalog') + return json(200, ['github-issues', 'google-calendar', 'pets']); + if (url.pathname === '/catalog/pets.yaml') { + res.writeHead(200, { 'Content-Type': 'application/yaml' }); + return res.end( + readFileSync(new URL('./mock-document.json', import.meta.url)), + ); + } + if (url.pathname === '/catalog/google-calendar.yaml') + return json(200, calendarDocument); + if (url.pathname === '/session') { + const challenge = randomBytes(32).toString('base64url'); + challenges.add(challenge); + return json(200, { + ts: Math.floor(Date.now() / 1000), + nonce: randomBytes(16).toString('hex'), + challenge, + }); + } if (url.pathname === '/connect') { const p = url.searchParams; const challenge = p.get('challenge'); - if (!challenges.has(challenge) || !equal(p.get('response'), sign(challenge)) || !equal(p.get('user_id_sig'), sign(p.get('user_id') ?? '')) || p.get('tenant_id') !== 'mock-tenant') return json(401, { error: 'Invalid tenant proof' }); + if ( + !challenges.has(challenge) || + !equal(p.get('response'), sign(challenge)) || + !equal(p.get('user_id_sig'), sign(p.get('user_id') ?? '')) || + p.get('tenant_id') !== 'mock-tenant' + ) + return json(401, { error: 'Invalid tenant proof' }); const redirect = new URL(p.get('redirect_uri')); - if (redirect.origin !== (process.env.MOCK_FRONTEND_ORIGIN ?? 'http://localhost:6747') || redirect.pathname !== '/app/integrations') return json(400, { error: 'Invalid callback' }); + if ( + redirect.origin !== frontendOrigin || + !['/app/integrations', '/app/devonian-demo'].includes(redirect.pathname) + ) + return json(400, { error: 'Invalid callback' }); const platform = p.get('platform'); - if (!['github-issues', 'google-calendar', 'pets'].includes(platform)) return json(400, { error: 'Invalid platform' }); - if (req.method === 'GET') { res.writeHead(200, { 'Content-Type': 'text/html' }); return res.end('

Mock integration proxy

Connect your test account.

'); } + if (!['github-issues', 'google-calendar', 'pets'].includes(platform)) + return json(400, { error: 'Invalid platform' }); + if (req.method === 'GET') { + res.writeHead(200, { 'Content-Type': 'text/html' }); + return res.end( + '

Mock integration proxy

Connect your test account.

', + ); + } if (req.method !== 'POST') return json(405, {}); challenges.delete(challenge); redirect.searchParams.set('connection_code', issueCode(platform)); - res.writeHead(303, { Location: redirect.href }); return res.end(); + res.writeHead(303, { Location: redirect.href }); + return res.end(); } if (url.pathname.startsWith('/proxy/')) { const code = req.headers.authorization?.replace(/^Bearer /, ''); const platform = codes.get(code); - if (!platform) return json(401, { error: 'Invalid or consumed connection code' }); + if (!platform) + return json(401, { error: 'Invalid or consumed connection code' }); codes.delete(code); - if (req.method !== 'GET' || !url.pathname.startsWith(`/proxy/${platform}/`)) return json(403, {}); - const data = platform === 'pets' ? (url.searchParams.get('page') === '2' ? pets.slice(2) : pets.slice(0, 2)) : platform === 'google-calendar' ? { items: [{ id: 'event-1', summary: 'Team meeting' }] } : [{ id: 42, title: 'Fetched GitHub issue', number: 42, state: 'open' }]; - return json(200, data, { 'X-Connection-Code': issueCode(platform), ...(platform === 'pets' && !url.searchParams.has('page') ? { Link: '; rel="next"' } : {}) }); + const headers = { 'X-Connection-Code': issueCode(platform) }; + if (!url.pathname.startsWith(`/proxy/${platform}/`)) + return json(403, {}, headers); + if (platform === 'github-issues') { + try { + let body = ''; + for await (const chunk of req) { + body += chunk; + if (body.length > 1024 * 1024) return json(413, {}, headers); + } + const result = github.request( + req.method, + url, + body ? JSON.parse(body) : {}, + ); + return json(result.status, result.body, headers); + } catch { + return json(400, { error: 'Invalid request body' }, headers); + } + } + if (platform === 'google-calendar') { + const result = calendar.request(req.method, url); + return json(result.status, result.body, headers); + } + if (req.method !== 'GET') return json(403, {}, headers); + const data = + platform === 'pets' + ? url.searchParams.get('page') === '2' + ? pets.slice(2) + : pets.slice(0, 2) + : { items: [{ id: 'event-1', summary: 'Team meeting' }] }; + return json(200, data, { + ...headers, + ...(platform === 'pets' && !url.searchParams.has('page') + ? { Link: '; rel="next"' } + : {}), + }); } json(404, {}); }); + server.github = github; + server.calendar = calendar; + return server; } -if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { - mockProxy().listen(Number(process.env.MOCK_PROXY_PORT ?? 19090), process.env.MOCK_PROXY_HOST ?? '127.0.0.1', () => console.log('Mock integration proxy listening on http://127.0.0.1:19090')); +if ( + process.argv[1] && + import.meta.url === pathToFileURL(process.argv[1]).href +) { + mockProxy().listen( + Number(process.env.MOCK_PROXY_PORT ?? 19090), + process.env.MOCK_PROXY_HOST ?? '127.0.0.1', + () => + console.log('Mock integration proxy listening on http://127.0.0.1:19090'), + ); } diff --git a/integrations/localthought/tsconfig.json b/integrations/localthought/tsconfig.json index e412e3c473..8a41739785 100644 --- a/integrations/localthought/tsconfig.json +++ b/integrations/localthought/tsconfig.json @@ -8,5 +8,5 @@ "types": ["node"], "noEmit": true }, - "include": ["data.ts", "schema.ts", "plugin.ts", "browser.ts"] + "include": ["calendar.ts", "schema.ts", "plugin.ts", "browser.ts"] } diff --git a/planning/devonian-issue-sync.md b/planning/devonian-issue-sync.md new file mode 100644 index 0000000000..657a8ac58d --- /dev/null +++ b/planning/devonian-issue-sync.md @@ -0,0 +1,21 @@ +# Devonian issue tracker sync + +- [x] Verify Devonian main (`e11104f`) and AtomicServer PR #1394 (`a087a7ca7`). +- [x] Inspect native lenses, GitHub integration actions, comments and test coverage. +- [x] Add regression tests for bidirectional issues/comments, conflict handling and restart recovery. +- [x] Build a browser demo using Devonian, local-only Atomic storage and direct integration-proxy requests. +- [x] Verify 13 focused tests, 7 existing integration tests and frontend typecheck; document setup/limits and update coverage. +- [x] Verify native browser creation/comments both ways, close/reopen and reload without duplicates. +- [x] Verify deployed v40 CORS preflight/exposed headers and browser GitHub OAuth. +- [ ] Verify live two-way writes: proxy credential returns 404 for the private sandbox; resolve repository access first. + +Use Devonian HTTP subjects for the intermediate graph and scoped external mappings +for Atomic DIDs. Persist graph, mappings and request receipts in IndexedDB. Serialize +rotating connection codes; refuse to retry uncertain writes. Tenant authentication and rotating credentials use #1401’s shared BrowserIntegrations client. +No Node runtime, AtomicServer plugin endpoint, tenant secret on AtomicServer, or server scheduler. +Live proxy CORS and OAuth now work; private-sandbox access is the remaining live blocker. +Missing records are conflicts, not deletion requests. + +- [x] Rebase onto #1401 and reuse browser tenant challenge, callback and rotating transport. +- [x] Expand HTTP mock for stateful GitHub issues/comments and add Playwright two-way sync coverage. +- [x] Run 13 sync tests, 11 LocalThought browser tests, 7 existing GitHub integration tests, browser E2E and typechecks.