Skip to content

cdktf get: parallelism issue with providers #92

@egonbraun

Description

@egonbraun

Expected Behavior

I am running CDKTF_LOG_LEVEL=debug cdktn get --force on my project inside the CI environment and I expected the command to succeed on downloading the providers and build them.

cdktf.json

{
  "language": "python",
  "app": "poetry run cdktf_app",
  "projectId": "807c6f23-a23f-4ae1-9c20-2aa3ae2c9b55",
  "codeMakerOutput": "imports",
  "terraformProviders": [
    "hashicorp/tfe@~> 0.76",
    "gitlabhq/gitlab@~> 18.10",
    "hashicorp/aws@~> 6.39"
  ]
}

Actual Behavior

Instead I got this error message.

Error: non-zero exit code null
    at ChildProcess.<anonymous> (/home/nonroot/.local/share/pnpm/global/5/.pnpm/cdktn-cli@0.22.0_ink@3.2.0_react@17.0.2__react@17.0.2/node_modules/cdktn-cli/bundle/bin/cmds/handlers.js:102:34391)
    at Object.onceWrapper (node:events:634:26)
    at ChildProcess.emit (node:events:519:28)
    at ChildProcess.emit (node:domain:489:12)
    at maybeClose (node:internal/child_process:1101:16)
    at Socket.<anonymous> (node:internal/child_process:456:11)
    at Socket.emit (node:events:519:28)
    at Socket.emit (node:domain:489:12)
    at Pipe.<anonymous> (node:net:346:12) {
  stderr: '!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n' +
    '!!                                                                                                     !!\n' +
    '!!  The 5.8 release line of jsii has reached End-of-Support.                                           !!\n' +
    '!!  We strongly recommend you upgrade to the current release line (5.9) at your earliest convenience.  !!\n' +
    '!!                                                                                                     !!\n' +
    '!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n'
}
⠧ downloading and generating modules and providers...
Error: non-zero exit code null

One thing to note is that when running locally it also complains about jsii reaching end of support, however the command succeeds.

Steps to Reproduce

Run the following command using the example cdktf.json file:

export CDKTF_LOG_LEVEL=debug
export CI=true
cdktn get --force

Versions

cdktn debug
language: python
cdktn-cli: 0.22.0
node: v22.22.0
cdktn: 0.22.0
cdktf: 0.21.0
constructs: 10.4.5
jsii: 1.127.0
terraform: 1.14.5
arch: x64
os: linux 4.14.355-280.710.amzn2.x86_64
python: Python 3.12.13
pip: pip 26.0.1 from /home/nonroot/.local/lib/python3.12/site-packages/pip (python 3.12)
pipenv: pipenv, version 2026.5.1
providers
tfe@> 0.76 (LOCAL)
terraform provider version: null
gitlabhq/gitlab@
> 18.10 (LOCAL)
terraform provider version: null
aws@~> 6.39 (LOCAL)
terraform provider version: null

Migration Context

I used cdktf and saw the same issue there

Upstream cdktf Issue

No response

Providers

────────────────┬──────────────┬─────┬─────┬─────────┬───────────┬─────────────┐
──── ─── ─ ─ ── ── ───
Provider Name │ Provider │ CDKT│ CDKT│ │ Package │ Package │
Version ConstraintName Version
────────────────┼──────────────┼─────┼─────┼─────────┼───────────┼─────────────┤
──── ─── ─ ─ ── ── ───
tfe │ │ │ │ ~> 0.76 │ │ │
────────────────┼──────────────┼─────┼─────┼─────────┼───────────┼─────────────┤
──── ─── ─ ─ ── ── ───
gitlabhq/gitlab│ │ │ │ ~> 18.10│ │ │
────────────────┼──────────────┼─────┼─────┼─────────┼───────────┼─────────────┤
──── ─── ─ ─ ── ── ───
aws │ │ │ │ ~> 6.39 │ │ │
────────────────┴──────────────┴─────┴─────┴─────────┴───────────┴─────────────┘
──── ─── ─ ─ ── ── ───

Gist

You can use this Dockerfile to reproduce the error on arm64 and x64 machines.

Dockerfile

Possible Solutions

No response

Workarounds

You can temporarily remove the aws provider from cdktf.json and run cdktf get. This will download and build the other providers correctly. You can add aws back to cdktf.json and run cdktf get and now it will work.

Note that you should not remove the gen or imports directory between the runs.

This pointed me to being a concurrency issue so I tried to run cdktf get --parallelism 1 and it works.

Anything Else?

Even on debug mode the only message displayed on screen is the jsii end of support message. No other relevant messages are printed.

The error only occurs when these providers are together in the cdktf.json file:

"hashicorp/tfe@~> 0.76",
"hashicorp/aws@~> 6.39"

The exit code of the command is 1.

We are running on python:3.12-trixie docker image.

References

No response

Help Wanted

  • I'm interested in contributing a fix myself

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions