Skip to content

[ENG-1852] Keep Roam shared node content fresh#1147

Open
sid597 wants to merge 4 commits into
mainfrom
eng-1852-keep-roam-shared-node-content-fresh-in-the-sync-loop
Open

[ENG-1852] Keep Roam shared node content fresh#1147
sid597 wants to merge 4 commits into
mainfrom
eng-1852-keep-roam-shared-node-content-fresh-in-the-sync-loop

Conversation

@sid597

@sid597 sid597 commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

https://www.loom.com/share/2dcd282d7a0f4a878c8780cbca56c820

Summary

  • Extends the existing Roam Supabase sync loop to refresh already-shared source nodes.
  • Detects shared source pages whose Roam page content changed via :page/edit-time or title-level :edit/time.
  • Uploads refreshed full markdown content without running embeddings, while keeping direct/direct_and_description embedding updates on the existing changed-node path.
  • Carries the matched node type through the shared freshness query so required schema metadata stays current.
  • Adds the compact full-markdown validation artifact as a typed example.

Manual demo validation:

  • Used local space 105833 and shared Roam page UID tzmTaBmqT ([[CLM]] - cat -).
  • Backfill case: confirmed the page already had ResourceAccess and Content.variant = direct, but no Content.variant = full; ran the existing Roam Supabase sync path; confirmed a new full row was created with Roam markdown
    content.
  • Freshness case: after full existed, edited the Roam page body; ran the existing Roam Supabase sync path again; confirmed the existing full row updated with the new page content.
  • No re-share/publish action was performed for either update.

@vercel

vercel Bot commented Jun 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
discourse-graph Skipped Skipped Jul 9, 2026 6:51am

Request Review

@supabase

supabase Bot commented Jun 22, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project zytfjzqyijgagqxrzbmz because there are no changes detected in packages/database/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@linear-code

linear-code Bot commented Jun 22, 2026

Copy link
Copy Markdown

ENG-1852

@sid597 sid597 changed the base branch from main to eng-1848-add-roam-full-markdown-content-variant-for-shared-nodes June 22, 2026 16:13
@sid597 sid597 marked this pull request as ready for review June 28, 2026 16:35
@sid597 sid597 changed the base branch from eng-1848-add-roam-full-markdown-content-variant-for-shared-nodes to main June 28, 2026 16:36
@graphite-app

graphite-app Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

PR size/scope check

This PR is over our review-size guideline.

  • Recommended: ~200 lines changed
  • Acceptable limit: up to 400 lines when well-scoped/self-contained
  • Preferred file count: fewer than 5 files

Please split this into smaller PRs unless there is a clear reason the changes need to land together.

If keeping it as one PR, please add a brief justification covering:

  • What single problem this PR solves
  • Why the files/changes are coupled

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 554884cf58

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/roam/src/utils/syncDgNodesToSupabase.ts
Comment thread apps/roam/src/utils/syncDgNodesToSupabase.ts

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment thread apps/roam/src/utils/syncDgNodesToSupabase.ts
@sid597 sid597 force-pushed the eng-1852-keep-roam-shared-node-content-fresh-in-the-sync-loop branch from 554884c to 116e32c Compare June 28, 2026 16:50
@sid597 sid597 force-pushed the eng-1852-keep-roam-shared-node-content-fresh-in-the-sync-loop branch from 116e32c to ad939d4 Compare July 2, 2026 18:59
@sid597 sid597 requested review from maparent and mdroidian July 3, 2026 10:58
Comment thread apps/roam/src/utils/convertRoamNodeToFullContent.simple.example.ts Outdated
import type { LocalContentDataInput } from "@repo/database/inputTypes";
import { contentTypes } from "@repo/content-model";

export type RoamFullContentNode = {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the intent to replace this with CrossAppContent eventually?

Comment thread apps/roam/src/utils/syncDgNodesToSupabase.ts
Comment thread apps/roam/src/utils/syncDgNodesToSupabase.ts Outdated

const sinceMs = since ?? DEFAULT_SYNC_TIME;
const query = `[
:find ?node-title ?uid ?nodeCreateTime ?nodeEditTime ?pageEditTime ?author_local_id

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be wrong, but I am a bit puzzled that this query uses only node, which seems to bind to the page and not to sub-nodes. I'd expect to find ?page, ?node, a [?node :block/page ?page] clause. Are we sure that nodeEditTime and pageEditTime are ever distinct in the query as written?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pageEditTime gets updated when any block on the page changes whereas the nodeEditTime is tracking the title of the page. So when we combine both we get the info if this page as a whole entity has changed since last time.

Comment thread apps/roam/src/utils/convertRoamNodeToFullContent.simple.example.ts Outdated
direct: {
localId: node.source_local_id,
value: title,
},

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maparent @mdroidian crossAppNode requires direct to be present but crossAppNodeToDbContent converter explicitly requires us to pass the variant type, so should we update the crossAppNode.content an or case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants