Skip to content

[DAPS-1516] Default Repository Type to Globus - #1602

Closed
AronPerez wants to merge 2 commits into
feat-DAPS-1515-endpoint-integrationfrom
feat-DAPS-1516-default-repo-type
Closed

AronPerez wants to merge 2 commits into
feat-DAPS-1515-endpoint-integrationfrom
feat-DAPS-1516-default-repo-type

Conversation

@AronPerez

@AronPerez AronPerez commented Aug 17, 2025 •

Copy link
Copy Markdown
Contributor

Ticket

DAPS-1516

Description

  • Added lazy migration in RepositoryOps.find() to set missing types to "globus".
  • Updated RepositoryOps.update() to add type if missing.
  • Modified repo_router.js create endpoint to set type for new repos.
  • Updated Repo class for metadata-only repos without path.
  • Added tests for lazy migration.

How Has This Been Tested?

operations.test.js & repo.test.js

Manually

  1. Created repositories without type field to simulate legacy data
  2. Verified find operations add default type
  3. Confirmed update operations preserve/add type field
  4. Tested backward compatibility with existing repository operations
docker-compose -f compose/local/compose.yml exec -T database arangosh --server.password lol
db._useDatabase("sdms");
  db.repo.save({
      _key: "test-legacy-manual",
      title: "Manual Test Legacy Repo",
      capacity: 1000000,
      pub_key: "test-key",
      address: "tcp://localhost:5555",
      endpoint: "test-endpoint",
      path: "/data/repos/test-legacy-manual",
      domain: "example.com"
  });
  const repo = db.repo.document("test-legacy-manual");
  print("Before find - has type field?", repo.hasOwnProperty("type")); // Should be false
  const ops = require("/path/to/foxx/api/repository/operations").RepositoryOps;
  const result = ops.find("test-legacy-manual");
  print("After find - type:", result.value.type); // Should be "globus"
  const updateResult = ops.update(result.value, { title: "Updated Title" });
  const updated = db.repo.document("test-legacy-manual");
  print("After update - type field:", updated.type); // Should be "globus"

Artifacts (if appropriate):

See CI

@sourcery-ai

sourcery-ai Bot commented Aug 17, 2025 •

Copy link
Copy Markdown
Contributor

Reviewer's Guide

This PR implements lazy migration of the repository type to default to Globus—injecting missing type fields in RepositoryOps.find and update, setting the default on new repositories in the router, extends the Repo class for metadata-only repos without a path, and adds comprehensive tests for these behaviors.

Sequence diagram for creating a new repository with default type

sequenceDiagram
    participant User
    participant repo_router.js
    participant g_db.repo
    User->>repo_router.js: Create new repository
    repo_router.js->>repo_router.js: Set type to GLOBUS
    repo_router.js->>g_db.repo: Save repository
    g_db.repo-->>repo_router.js: Return new repository
    repo_router.js-->>User: Respond with repository
Loading

Class diagram for updated RepositoryOps and Repo class

classDiagram
    class RepositoryOps {
        +find(repoId)
        +update(repository, updates)
    }
    class RepositoryType {
        <<enum>>
        GLOBUS
        // ...other types
    }
    class Repo {
        +type
        +path
        // ...other attributes
    }
    RepositoryOps --> RepositoryType
    Repo --> RepositoryType
Loading

Flow diagram for lazy migration of repository type

flowchart TD
    A[RepositoryOps.find or update called] --> B{Does repository have type?}
    B -- Yes --> C[Return repository]
    B -- No --> D[Set type to GLOBUS]
    D --> C
Loading

File-Level Changes

Change Details Files
Default repository type to Globus
  • Import RepositoryType constant
  • Add default type assignment in update when missing
  • Assign default type in find for legacy repos
  • Set Globus type on repo creation endpoint
core/database/foxx/api/repository/operations.js
core/database/foxx/api/repo_router.js
Support metadata-only repositories
  • Extend Repo class to handle repos lacking a path
  • Return UNKNOWN pathType for metadata-only repos
core/database/foxx/api/repository/repo.js
core/database/foxx/tests/repo.test.js
Add tests for lazy migration behavior
  • Validate default type on find for repos without type
  • Ensure existing type is preserved during update
  • Verify type field is added during update if missing
  • Handle repo ID input with and without 'repo/' prefix
core/database/foxx/tests/operations.test.js

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • In repo_router.js you reference RepositoryType.GLOBUS but never import RepositoryType—add the required import from your types module to avoid a ReferenceError.
  • Rather than unconditionally setting obj.type to Globus for all new repositories, only default it when no type is provided to avoid overwriting explicit type inputs.
  • The find() migration sets type on the returned object but doesn’t persist it back to the database—consider whether you need to write the default type to the stored document for a proper migration.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In repo_router.js you reference RepositoryType.GLOBUS but never import RepositoryType—add the required import from your types module to avoid a ReferenceError.
- Rather than unconditionally setting obj.type to Globus for all new repositories, only default it when no type is provided to avoid overwriting explicit type inputs.
- The find() migration sets type on the returned object but doesn’t persist it back to the database—consider whether you need to write the default type to the stored document for a proper migration.

## Individual Comments

### Comment 1
<location> `core/database/foxx/tests/operations.test.js:8` </location>
<code_context>
+const { RepositoryType } = require("../api/repository/types");
+const g_db = require("@arangodb").db;
+
+describe("RepositoryOps lazy migration", function () {
+    beforeEach(function () {
+        g_db.repo.truncate();
+    });
+
+    it("should default type to 'globus' when finding repo without type field", function () {
</code_context>

<issue_to_address>
Missing test for error handling in RepositoryOps.find and update.

Please add tests for scenarios where RepositoryOps.find and RepositoryOps.update encounter missing repositories or database errors to ensure robust error handling.
</issue_to_address>

### Comment 2
<location> `core/database/foxx/tests/operations.test.js:52` </location>
<code_context>
+        expect(updateResult.value.title).to.equal("Updated Title");
+    });
+
+    it("should add type field during update if missing", function () {
+        const legacyRepo = g_db.repo.save({
+            _key: "test-migration",
+            title: "Repository for Migration",
+            capacity: 500000,
+        });
+
+        const beforeUpdate = g_db.repo.document("test-migration");
+        expect(beforeUpdate.type).to.be.undefined;
+
+        const findResult = RepositoryOps.find("test-migration");
+        const updateResult = RepositoryOps.update(findResult.value, {
+            capacity: 600000,
+        });
+
+        expect(updateResult.ok).to.be.true;
+
+        const afterUpdate = g_db.repo.document("test-migration");
+        expect(afterUpdate.type).to.equal(RepositoryType.GLOBUS);
+        expect(afterUpdate.capacity).to.equal(600000);
+    });
+
</code_context>

<issue_to_address>
Consider adding a test for explicit type update.

Please add a test where the update sets the type explicitly to ensure the logic correctly preserves the provided type value.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
    it("should add type field during update if missing", function () {
        const legacyRepo = g_db.repo.save({
            _key: "test-migration",
            title: "Repository for Migration",
            capacity: 500000,
        });

        const beforeUpdate = g_db.repo.document("test-migration");
        expect(beforeUpdate.type).to.be.undefined;

        const findResult = RepositoryOps.find("test-migration");
        const updateResult = RepositoryOps.update(findResult.value, {
            capacity: 600000,
        });

        expect(updateResult.ok).to.be.true;

        const afterUpdate = g_db.repo.document("test-migration");
        expect(afterUpdate.type).to.equal(RepositoryType.GLOBUS);
        expect(afterUpdate.capacity).to.equal(600000);
    });
=======
    it("should add type field during update if missing", function () {
        const legacyRepo = g_db.repo.save({
            _key: "test-migration",
            title: "Repository for Migration",
            capacity: 500000,
        });

        const beforeUpdate = g_db.repo.document("test-migration");
        expect(beforeUpdate.type).to.be.undefined;

        const findResult = RepositoryOps.find("test-migration");
        const updateResult = RepositoryOps.update(findResult.value, {
            capacity: 600000,
        });

        expect(updateResult.ok).to.be.true;

        const afterUpdate = g_db.repo.document("test-migration");
        expect(afterUpdate.type).to.equal(RepositoryType.GLOBUS);
        expect(afterUpdate.capacity).to.equal(600000);
    });

    it("should preserve explicit type value during update", function () {
        const legacyRepo = g_db.repo.save({
            _key: "test-explicit-type",
            title: "Repository for Explicit Type",
            capacity: 100000,
        });

        const beforeUpdate = g_db.repo.document("test-explicit-type");
        expect(beforeUpdate.type).to.be.undefined;

        const findResult = RepositoryOps.find("test-explicit-type");
        const updateResult = RepositoryOps.update(findResult.value, {
            type: RepositoryType.S3,
            capacity: 200000,
        });

        expect(updateResult.ok).to.be.true;

        const afterUpdate = g_db.repo.document("test-explicit-type");
        expect(afterUpdate.type).to.equal(RepositoryType.S3);
        expect(afterUpdate.capacity).to.equal(200000);
    });
>>>>>>> REPLACE

</suggested_fix>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +8 to +11
describe("RepositoryOps lazy migration", function () {
beforeEach(function () {
g_db.repo.truncate();
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Missing test for error handling in RepositoryOps.find and update.

Please add tests for scenarios where RepositoryOps.find and RepositoryOps.update encounter missing repositories or database errors to ensure robust error handling.

Comment on lines +52 to +72
it("should add type field during update if missing", function () {
const legacyRepo = g_db.repo.save({
_key: "test-migration",
title: "Repository for Migration",
capacity: 500000,
});

const beforeUpdate = g_db.repo.document("test-migration");
expect(beforeUpdate.type).to.be.undefined;

const findResult = RepositoryOps.find("test-migration");
const updateResult = RepositoryOps.update(findResult.value, {
capacity: 600000,
});

expect(updateResult.ok).to.be.true;

const afterUpdate = g_db.repo.document("test-migration");
expect(afterUpdate.type).to.equal(RepositoryType.GLOBUS);
expect(afterUpdate.capacity).to.equal(600000);
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Consider adding a test for explicit type update.

Please add a test where the update sets the type explicitly to ensure the logic correctly preserves the provided type value.

Suggested change
it("should add type field during update if missing", function () {
const legacyRepo = g_db.repo.save({
_key: "test-migration",
title: "Repository for Migration",
capacity: 500000,
});
const beforeUpdate = g_db.repo.document("test-migration");
expect(beforeUpdate.type).to.be.undefined;
const findResult = RepositoryOps.find("test-migration");
const updateResult = RepositoryOps.update(findResult.value, {
capacity: 600000,
});
expect(updateResult.ok).to.be.true;
const afterUpdate = g_db.repo.document("test-migration");
expect(afterUpdate.type).to.equal(RepositoryType.GLOBUS);
expect(afterUpdate.capacity).to.equal(600000);
});
it("should add type field during update if missing", function () {
const legacyRepo = g_db.repo.save({
_key: "test-migration",
title: "Repository for Migration",
capacity: 500000,
});
const beforeUpdate = g_db.repo.document("test-migration");
expect(beforeUpdate.type).to.be.undefined;
const findResult = RepositoryOps.find("test-migration");
const updateResult = RepositoryOps.update(findResult.value, {
capacity: 600000,
});
expect(updateResult.ok).to.be.true;
const afterUpdate = g_db.repo.document("test-migration");
expect(afterUpdate.type).to.equal(RepositoryType.GLOBUS);
expect(afterUpdate.capacity).to.equal(600000);
});
it("should preserve explicit type value during update", function () {
const legacyRepo = g_db.repo.save({
_key: "test-explicit-type",
title: "Repository for Explicit Type",
capacity: 100000,
});
const beforeUpdate = g_db.repo.document("test-explicit-type");
expect(beforeUpdate.type).to.be.undefined;
const findResult = RepositoryOps.find("test-explicit-type");
const updateResult = RepositoryOps.update(findResult.value, {
type: RepositoryType.S3,
capacity: 200000,
});
expect(updateResult.ok).to.be.true;
const afterUpdate = g_db.repo.document("test-explicit-type");
expect(afterUpdate.type).to.equal(RepositoryType.S3);
expect(afterUpdate.capacity).to.equal(200000);
});

@AronPerez
AronPerez force-pushed the feat-DAPS-1515-endpoint-integration branch from 4f6f23a to cde5516 Compare August 18, 2025 16:57
@AronPerez
AronPerez force-pushed the feat-DAPS-1516-default-repo-type branch from c7e29d8 to 39ca1d1 Compare August 18, 2025 16:57
@AronPerez AronPerez linked an issue Aug 21, 2025 that may be closed by this pull request
8 tasks
@AronPerez
AronPerez force-pushed the feat-DAPS-1515-endpoint-integration branch from cde5516 to db5f941 Compare August 26, 2025 17:13
@AronPerez
AronPerez force-pushed the feat-DAPS-1516-default-repo-type branch 2 times, most recently from 27ec97e to 9233377 Compare August 26, 2025 17:50
@AronPerez
AronPerez force-pushed the feat-DAPS-1515-endpoint-integration branch from 7007e52 to 5e9fe8f Compare August 27, 2025 15:50
@AronPerez
AronPerez force-pushed the feat-DAPS-1516-default-repo-type branch from 9233377 to 120b760 Compare August 27, 2025 15:50
@AronPerez
AronPerez force-pushed the feat-DAPS-1515-endpoint-integration branch from f4f6690 to 50be6a3 Compare August 28, 2025 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Core, Arango] Database Migration: Add Repository Type Field to Database Schema

3 participants