List all items of a given Entity (scan-like)

[zirkelc]

Hi,

all my entities have tenantId as part of the pk, but now I have the use case to list all Tenants. That means I don't have any param I could use for the query. So it's basically a scan operation without any params. Nonetheless, I was wondering if there is any way to make this more performant.

I was thinking I could create a 5th index gsi5pk-gsi5sk-index on the entity name __edb_e__? I assume I would have to create a local attribute that stores an additional copy of __edb_e__ in order to use it as composite index.

Is there another option I'm overseeing here?

[zirkelc]

It was even easier than I thought: #210

[zirkelc]

@tywalch sorry to ping you directly, but I'd like to hear your opinion on this:

Is there any obvious disadvantage when configuring the primary index like this. with an empty pk and the sk making up the full key:

indexes: {
  task: {
    pk: {
      field: "pk",
      composite: [],
    },
    sk: {
      field: "sk",
      composite: ["task", "project", "employee"],
    },
  }
},

// vs.

indexes: {
  task: {
    pk: {
      field: "pk",
      composite: ["task"],
    },
    sk: {
      field: "sk",
      composite: ["project", "employee"],
    },
  }
},

The latter options offers 4 access patterns:

Task.get({ task: "task", project: "project", employee: "employee" }).go();
Task.query.task({ task: "task" }).go();
Task.query.task({ task: "task", project: "project" }).go();
Task.query.task({ task: "task", project: "project", employee: "employee" }).go();

However, the first option offers an additional access pattern to read all Tasks:

Task.get({ task: "task", project: "project", employee: "employee" }).go();
Task.query.task({ task: "task" }).go();
Task.query.task({ task: "task", project: "project" }).go();
Task.query.task({ task: "task", project: "project", employee: "employee" }).go();

Task.query.task({}).go();

Here are the two Playgrounds to test ts:
Without pk

With pk

In combination with scope, the first option looks like better option, right?

[phawxby]

@zirkelc did you come up with a solution for this that works with records that are already present in the table? I also came to the conclusion that moving the id from the PK to the SK does make it easier to query out all items of a specific type, but has the disadvantage that it needs a new field to avoid breaking things, that field would not be populated for existing items

[zirkelc]

Migrating existing items is always a pain. I did it once for an entity for the exact purpose of moving the PK attributes to the SK to be able to query all items. Since changing the PK is not possible, I had to delete and create the item again. I used the raw DynamoDB client (without ElectroDB) and formatted the key manually.

Alternatively, you could duplicate your entity in an old and new version. Then run a script that uses the old entity to query and delete the old items. Then, use the new entity to create each item again. Afterwards you can delete the old entity altogether.

[lukemorton]

Useful, thanks for sharing. Worth adding this pattern to the docs I think!