Failing to return some nested data within content types

[IanBoyte]

We have a somewhat complex component library with fairly heavy levels of nesting. We're in active development and have recently seen fields not being returned by queries using populate=all. Components that previously returned their full nested dataset are not

Reviewing the plugin readme, you state it builds a standard Strapi populate query via middleware. Is it possible that a sufficiently complex component structure might be hitting the url limits?

[faessler]

Hello @IanBoyte 👋 Thx for your bug report.

Url limits should not apply here because it creates a populate query inside the node env and doesn't pass them through any url.

There was another bug which is fixed now (#17) that might be related to yours. 🤔 Could you try if v1.2.5 fixes your issue?

If not can you provide more details? What kind of field (relations, components, dz, string, etc.)? How does your component library schema look like? Or could you provide some kind of reproduction steps? 🙏

[IanBoyte]

Thanks, I've swapped the plugin back and am having my team test.

[IanBoyte]

@faessler Just realized part of the issue I was having was from copying the example usage:

module.exports = {
  "populate-all": {
    enabled: true,
    config: {
      cache: true,
      realtions: true,
    },
  },
};

relations is misspelled.

[faessler]

Oh my 🤦🏼‍♂️ Thanks for the report. Gonna fix the docs today.

Also just a fyi the cache: boolean config option hasn‘t been released yet. I need to write some tests.

[github-actions]

🎉 This issue has been resolved in version 1.4.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

EDIT: Only the misspelling was fixed here. It's not confirmed yet that the initial bug report has been resolved.

[IanBoyte]

Unfortunately, during testing, it looks like some of our more deeply nested components still aren't being returned in the data response. All we're getting back is that component id. I did try both with and without the caching config set (in case it's implemented in some way).

[IanBoyte]

Some more detail on our schemas. We build with a lot of reuse with an atomic structure. The example I'm seeing the issue on presently is similar to the following:

Our relevant content structures:
Content type: page
Module: contentBlock
Molecule: commonModuleHeader
Molecule: buttonGroup
Atom: button
Atom: internalLink

Our content schema would be similar to the following:

page
> dynamicZone
...
  > contentBlock
    > commonModuleHeader
      > buttonGroup
        > button
          > internalLink
            > relationship to another page
...

What we would see returned, for example, is

{ page
 ...
 contentBlock {
    commonModuleHeader {
      buttonGroup {
        id: 123
      }
    }
  }
...
}

[faessler]

Thanks for the schema! I rebuilt it in the sandbox. Unlike you I got all the data I'd expect. 🤔

plugin	schema	permissions	collection view	document view	result

Followup questions:

• Do you have the correct permissions set?
• Is the page you linked in draft and thus cannot be loaded? (add &status=draft)
• Do other components work? Do other pages work?
• Which version of Strapi are you using?

[IanBoyte]

"@strapi/strapi": "^5.29.0"

All pages being tested are published.

We're using a full access token and I've checked all permissions.

I suspect part of the issue might be due to the size/complexity of our full schema. We didn't start noticing issues until fairly late in development.

Other components and content types have similar issues. We've seen nested assets not having the entire asset object being returned, we've tested the buttons where they're used in standalone areas, and the button data might return, but not the nested page_link.

Ex. we use this masthead component on all pages, and on some content types, the image isn't being returned.

"masthead": {
      "id": 148,
      "masthead_type": "basic",
      "subhead": null,
      "description": null
    },

vs

"masthead": {
      "id": 209,
      "masthead_type": "basic",
      "subhead": null,
      "description": null,
      "media": {
        "id": 1031,
        "type": "image",
        "image": {
          ... // all asset data being returned here
        },
        "video": null
      },
      "logo": null,
      "title": {
        "id": 1625,
        "Text": "Module Test 3",
        "Tag": "h2"
      },
      "buttons": null
    },

[IanBoyte]

If it's helpful, let me know and I can provide a link to a zip of the full schema.

[zhawzid-ottj]

The problem for some cases (sorts of nested containers scenarios)
is the recently added "prevent infinite loop" check.

You can also implement a max depth guard like this:

export const getPopulateQuery = (
  modelUid: UID.Schema,
  parentsModelUids: UID.Schema[] = [],
  depth: number = 0
): { populate: object | true } | undefined => {
...

and add depth+1 to every recursive call like this (example):

query.populate[fieldName] = getPopulateQuery(
  attribute.component,
  parentsModelUids,
  depth + 1
);

then tweak this:

// prevent infinite loop
//    if (parentsModelUids.includes(modelUid)) {
if (depth > 5) {
  strapi.log.debug(
    `[populate-all] max depth detected skipping population: ${modelUid}`
  );
  return { populate: {} };
} else {
  parentsModelUids.push(modelUid);
}

This isn't too efficient but it enables some nice scenarios which aren't possible with the current implementation where every entiy can only occur once, otherwise nested entities won't be populated.

I would let this switch in the config by two separates settings, where you have to enable either one or the other way.

Cheers