-
Notifications
You must be signed in to change notification settings - Fork 6
[ENG-1856] Store source identity metadata for Roam imported nodes #1161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| import { describe, expect, it } from "vitest"; | ||
| import type { CrossAppNode } from "@repo/database/crossAppNodeContract"; | ||
| import type { json } from "~/utils/getBlockProps"; | ||
| import { DISCOURSE_GRAPH_PROP_NAME } from "~/utils/createReifiedBlock"; | ||
| import { | ||
| IMPORTED_FROM_PROP_KEY, | ||
| parseImportedSourceIdentity, | ||
| toImportedSourceIdentity, | ||
| type ImportedSourceIdentity, | ||
| } from "~/utils/importedSourceIdentity"; | ||
|
|
||
| const identity: ImportedSourceIdentity = { | ||
| sourceApp: "obsidian", | ||
| sourceSpaceId: "obsidian:9a8b7c6d5e4f3210", | ||
| sourceNodeId: "0192f1a0-7b3c-7e2a-9f10-1a2b3c4d5e6f", | ||
| sourceNodeRid: | ||
| "orn:obsidian.note:9a8b7c6d5e4f3210/0192f1a0-7b3c-7e2a-9f10-1a2b3c4d5e6f", | ||
| sourceTitle: "EVD - REM sleep and recall", | ||
| sourceModifiedAt: "2026-06-14T10:30:00.000Z", | ||
| }; | ||
|
|
||
| const propsWithImportedFrom = ( | ||
| importedFrom: json, | ||
| siblingProps: Record<string, json> = {}, | ||
| ): Record<string, json> => ({ | ||
| [DISCOURSE_GRAPH_PROP_NAME]: { | ||
| ...siblingProps, | ||
| [IMPORTED_FROM_PROP_KEY]: importedFrom, | ||
| }, | ||
| }); | ||
|
|
||
| describe("toImportedSourceIdentity", () => { | ||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What this covers: the two PURE functions — toImportedSourceIdentity (CrossAppNode → stored shape, using Run from apps/roam: |
||
| it("maps the six identity fields from a cross-app node, using direct content as title", () => { | ||
| const node: CrossAppNode = { | ||
| sourceApp: "obsidian", | ||
| sourceSpaceId: identity.sourceSpaceId, | ||
| sourceSpaceName: "Research Vault", | ||
| sourceNodeId: identity.sourceNodeId, | ||
| sourceNodeRid: identity.sourceNodeRid, | ||
| nodeType: { sourceNodeTypeId: "evd-7c1f9a2b", label: "Evidence" }, | ||
| content: { | ||
| direct: { value: identity.sourceTitle }, | ||
| full: { format: "text/markdown", value: "# body\n" }, | ||
| }, | ||
| sourceModifiedAt: identity.sourceModifiedAt, | ||
| }; | ||
|
|
||
| const result = toImportedSourceIdentity(node); | ||
|
|
||
| expect(result).toEqual(identity); | ||
| expect(Object.keys(result).sort()).toEqual( | ||
| [ | ||
| "sourceApp", | ||
| "sourceModifiedAt", | ||
| "sourceNodeId", | ||
| "sourceNodeRid", | ||
| "sourceSpaceId", | ||
| "sourceTitle", | ||
| ].sort(), | ||
| ); | ||
| }); | ||
| }); | ||
|
|
||
| describe("parseImportedSourceIdentity", () => { | ||
| it("round-trips a stored identity", () => { | ||
| expect( | ||
| parseImportedSourceIdentity(propsWithImportedFrom(identity)), | ||
| ).toEqual(identity); | ||
| }); | ||
|
|
||
| it("ignores sibling discourse-graph props", () => { | ||
| const props = propsWithImportedFrom(identity, { | ||
| "relation-migration": { abc123: 1718000000000 }, | ||
| }); | ||
| expect(parseImportedSourceIdentity(props)).toEqual(identity); | ||
| }); | ||
|
|
||
| it("returns undefined when there are no discourse-graph props", () => { | ||
| expect(parseImportedSourceIdentity({})).toBeUndefined(); | ||
| }); | ||
|
|
||
| it("returns undefined when discourse-graph has no importedFrom", () => { | ||
| expect( | ||
| parseImportedSourceIdentity({ | ||
| [DISCOURSE_GRAPH_PROP_NAME]: { "relation-migration": {} }, | ||
| }), | ||
| ).toBeUndefined(); | ||
| }); | ||
|
|
||
| it("returns undefined when a required field is missing", () => { | ||
| const withoutRid = { | ||
| sourceApp: identity.sourceApp, | ||
| sourceSpaceId: identity.sourceSpaceId, | ||
| sourceNodeId: identity.sourceNodeId, | ||
| sourceTitle: identity.sourceTitle, | ||
| sourceModifiedAt: identity.sourceModifiedAt, | ||
| }; | ||
| expect( | ||
| parseImportedSourceIdentity(propsWithImportedFrom(withoutRid)), | ||
| ).toBeUndefined(); | ||
| }); | ||
|
|
||
| it("returns undefined when a field has the wrong type", () => { | ||
| expect( | ||
| parseImportedSourceIdentity( | ||
| propsWithImportedFrom({ ...identity, sourceModifiedAt: 1718000000000 }), | ||
| ), | ||
| ).toBeUndefined(); | ||
| }); | ||
|
|
||
| it("returns undefined for an unknown sourceApp", () => { | ||
| expect( | ||
| parseImportedSourceIdentity( | ||
| propsWithImportedFrom({ ...identity, sourceApp: "notion" }), | ||
| ), | ||
| ).toBeUndefined(); | ||
| }); | ||
| }); | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| import type { CrossAppNode } from "@repo/database/crossAppNodeContract"; | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: Should the example live in the test folder? |
||
| import { spaceUriAndLocalIdToRid } from "@repo/database/lib/rid"; | ||
| import { | ||
| toImportedSourceIdentity, | ||
| type ImportedSourceIdentity, | ||
| } from "./importedSourceIdentity"; | ||
|
|
||
| /** | ||
| * Example source identity persisted on a Roam page when an Obsidian-origin node is | ||
| * imported, type-checked against the cross-app node contract. Not used at runtime. | ||
| */ | ||
|
|
||
| const OBSIDIAN_SOURCE_SPACE_ID = "obsidian:9a8b7c6d5e4f3210"; | ||
| const OBSIDIAN_SOURCE_NODE_ID = "0192f1a0-7b3c-7e2a-9f10-1a2b3c4d5e6f"; | ||
|
|
||
| const obsidianOriginNode: CrossAppNode = { | ||
| sourceApp: "obsidian", | ||
| sourceSpaceId: OBSIDIAN_SOURCE_SPACE_ID, | ||
| sourceSpaceName: "Research Vault", | ||
| sourceNodeId: OBSIDIAN_SOURCE_NODE_ID, | ||
| sourceNodeRid: spaceUriAndLocalIdToRid( | ||
| OBSIDIAN_SOURCE_SPACE_ID, | ||
| OBSIDIAN_SOURCE_NODE_ID, | ||
| "note", | ||
| ), | ||
| nodeType: { sourceNodeTypeId: "evd-7c1f9a2b", label: "Evidence" }, | ||
| content: { | ||
| direct: { value: "EVD - REM sleep and recall" }, | ||
| full: { | ||
| format: "text/markdown", | ||
| value: "# REM sleep correlates with recall\n", | ||
| }, | ||
| }, | ||
| sourceModifiedAt: "2026-06-14T10:30:00.000Z", | ||
| }; | ||
|
|
||
| export const obsidianImportedIdentityExample: ImportedSourceIdentity = | ||
| toImportedSourceIdentity(obsidianOriginNode); | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| import type { CrossAppNode } from "@repo/database/crossAppNodeContract"; | ||
| import getBlockProps, { type json } from "./getBlockProps"; | ||
| import setBlockProps from "./setBlockProps"; | ||
| import { DISCOURSE_GRAPH_PROP_NAME } from "./createReifiedBlock"; | ||
|
|
||
| export type ImportedSourceIdentity = { | ||
| sourceApp: CrossAppNode["sourceApp"]; | ||
| sourceSpaceId: string; | ||
| sourceNodeId: string; | ||
| sourceNodeRid: string; | ||
| sourceTitle: string; | ||
| sourceModifiedAt: string; | ||
| }; | ||
|
|
||
| export const IMPORTED_FROM_PROP_KEY = "importedFrom"; | ||
|
|
||
| const isJsonObject = (value: json): value is { [key: string]: json } => | ||
| typeof value === "object" && value !== null && !Array.isArray(value); | ||
|
|
||
| const isSourceApp = (value: json): value is CrossAppNode["sourceApp"] => | ||
| value === "roam" || value === "obsidian"; | ||
|
|
||
| export const toImportedSourceIdentity = ( | ||
| node: CrossAppNode, | ||
| ): ImportedSourceIdentity => ({ | ||
| sourceApp: node.sourceApp, | ||
| sourceSpaceId: node.sourceSpaceId, | ||
| sourceNodeId: node.sourceNodeId, | ||
| sourceNodeRid: node.sourceNodeRid, | ||
| sourceTitle: node.content.direct.value, | ||
| sourceModifiedAt: node.sourceModifiedAt, | ||
| }); | ||
|
|
||
| export const parseImportedSourceIdentity = ( | ||
| props: Record<string, json>, | ||
| ): ImportedSourceIdentity | undefined => { | ||
| const dgData = props[DISCOURSE_GRAPH_PROP_NAME]; | ||
| if (!isJsonObject(dgData)) return undefined; | ||
| const imported = dgData[IMPORTED_FROM_PROP_KEY]; | ||
| if (!isJsonObject(imported)) return undefined; | ||
| const { | ||
| sourceApp, | ||
| sourceSpaceId, | ||
| sourceNodeId, | ||
| sourceNodeRid, | ||
| sourceTitle, | ||
| sourceModifiedAt, | ||
| } = imported; | ||
| if (!isSourceApp(sourceApp)) return undefined; | ||
| if ( | ||
| typeof sourceSpaceId !== "string" || | ||
| typeof sourceNodeId !== "string" || | ||
| typeof sourceNodeRid !== "string" || | ||
| typeof sourceTitle !== "string" || | ||
| typeof sourceModifiedAt !== "string" | ||
| ) | ||
| return undefined; | ||
| return { | ||
| sourceApp, | ||
| sourceSpaceId, | ||
| sourceNodeId, | ||
| sourceNodeRid, | ||
| sourceTitle, | ||
| sourceModifiedAt, | ||
| }; | ||
| }; | ||
|
|
||
| export const readImportedSourceIdentity = ( | ||
| pageUid: string, | ||
| ): ImportedSourceIdentity | undefined => | ||
| parseImportedSourceIdentity(getBlockProps(pageUid)); | ||
|
|
||
| export const writeImportedSourceIdentity = ( | ||
| pageUid: string, | ||
| identity: ImportedSourceIdentity, | ||
| ): void => { | ||
| const existing = getBlockProps(pageUid)[DISCOURSE_GRAPH_PROP_NAME]; | ||
| const dgData: Record<string, json> = isJsonObject(existing) ? existing : {}; | ||
| dgData[IMPORTED_FROM_PROP_KEY] = { | ||
| sourceApp: identity.sourceApp, | ||
| sourceSpaceId: identity.sourceSpaceId, | ||
| sourceNodeId: identity.sourceNodeId, | ||
| sourceNodeRid: identity.sourceNodeRid, | ||
| sourceTitle: identity.sourceTitle, | ||
| sourceModifiedAt: identity.sourceModifiedAt, | ||
| }; | ||
| setBlockProps(pageUid, { [DISCOURSE_GRAPH_PROP_NAME]: dgData }); | ||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Merged into the shared |
||
| }; | ||
|
|
||
| export const findImportedNodeUidByRid = async ( | ||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Duplicate-prevention entry point: before importing, look up by sourceNodeRid (the canonical cross-app key — it alone identifies a node) and reuse the page if found. Warns instead of throwing on >1 match so a bad data state degrades gracefully. |
||
| sourceNodeRid: string, | ||
| ): Promise<string | null> => { | ||
| const query = `[:find ?uid | ||
| :in $ ?rid | ||
| :where | ||
| [?page :block/uid ?uid] | ||
| [?page :block/props ?props] | ||
| [(get ?props :${DISCOURSE_GRAPH_PROP_NAME}) ?dgData] | ||
| [(get ?dgData :${IMPORTED_FROM_PROP_KEY}) ?imported] | ||
| [(get ?imported :sourceNodeRid) ?rid]]`; | ||
| const result = (await window.roamAlphaAPI.data.async.q( | ||
| query, | ||
| sourceNodeRid, | ||
| )) as [string][]; | ||
| if (result.length > 1) { | ||
| console.warn( | ||
| `findImportedNodeUidByRid: ${result.length} pages share sourceNodeRid '${sourceNodeRid}'`, | ||
| ); | ||
| } | ||
| return result.length > 0 ? result[0][0] : null; | ||
| }; | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm puzzled that the structure contains both the sourceNodeRid on the one hand, and the set of sourceSpaceId, sourceNodeId, and sourceApp on the other. The former is a compressed representation of all the latter, and we have a utility function to recover them. So this is very redundant. I would only keep the Rid.
If you compare to the obsidian implementation, OTH, we also have authorId (which I understand we may punt; we have not thought enough about multiple authors.)