Skip to content

[FIX] Repair local Jupyter Conda permissions - #492

Open
KilianTrunk wants to merge 4 commits into
awslabs:mainfrom
Ortecha:chore/lexical-graph-dev-environment
Open

KilianTrunk wants to merge 4 commits into
awslabs:mainfrom
Ortecha:chore/lexical-graph-dev-environment

Conversation

@KilianTrunk

@KilianTrunk KilianTrunk commented Aug 20, 2026

Copy link
Copy Markdown

Description

This PR isolates the local Jupyter image permission fix requested during the review of #405.

After the root-level Conda and pip installation step, it repairs the Conda installation permissions before the image switches back to the jovyan runtime user. This prevents later package installation and notebook development workflows from failing with permission errors under /opt/conda.

This PR contains no Compose project rename, volume migration, graph-operation changes, SPARQL implementation, documentation, notebooks, .dockerignore, or Dockerfile.hub changes.

Changes

  • Run fix-permissions "${CONDA_DIR}" after the root-level Conda and pip setup.
  • Keep the existing Compose project name (local-dev).
  • Keep the existing services, ports, bind mounts, named volumes, notebook command, and dependency services unchanged.

Problem

The image performs Conda and pip installation as root, then runs as jovyan. Without repairing the Conda installation permissions, later package installation can fail with permission errors under /opt/conda.

Compatibility

No Docker Compose project rename or volume migration is included. Existing local-dev networks, volumes, container names, and host ports remain unchanged.

Testing

  • Unit tests added/updated (not applicable to this Docker-only change)
  • Integration tests added (not applicable)
  • Existing tests pass (pytest) — not run because no Python source changed
  • Docker Compose configuration validation: docker compose -f examples/lexical-graph-local-dev/docker/docker-compose-dev.yml config --quiet
  • git diff --check: passed
  • Python syntax/bytecode compilation check: passed

Checklist

  • Code follows existing style and conventions
  • No core graph or SPARQL functionality changed
  • No Compose project or volume migration introduced
  • No breaking changes

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@acarbonetto

Copy link
Copy Markdown
Collaborator

Thank you for this. I will take a look at this asap.

plotly

# LlamaIndex readers (hard imports in lexical-graph source)
USER root

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this necessary? I didn't think pip install required root access. We should keep the USER root call before the apt-get calls instead.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might make more sense in the follow-up PR when we do other installations. I just quite understand the problem with the current build.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's necessary, I can reproduce the failure from current main branch and it fails with:

ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/opt/conda/lib/python3.11/site-packages/backports/zstd'

Yeah pip install itself does not need the root, but the problem is that the earlier root-level Conda/pip step leaves files under /opt/conda that jovyan can't update.

The first version of this PR worked around that by installing the readers as root and repairing the permissions afterwards however I now changed it so that the permissions are repaired immediately after the root-level Conda/pip step instead.

@@ -1,4 +1,4 @@
name: local-dev
name: graphrag-toolkit-rdf-dev

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can cause migration issues. Can you pin the names of the volumes too?

Update lines 58...:

volumes:
  # Pin explicit volume names so a Compose project rename does not orphan
  # existing data. With an explicit name Compose uses it verbatim (no project
  # prefix), so these always map to the same on-disk volumes regardless of the
  # top-level `name:`.
  neo4j_local_data_dev:
    name: local-dev_neo4j_local_data_dev
  neo4j_local_logs_dev:
    name: local-dev_neo4j_local_logs_dev
  pgvector_local_data_dev:
    name: local-dev_pgvector_local_data_dev

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Andrew, I have added the names for all three volumes and also I tested the rename path with an existing volume from the old project, and the renamed Compose project mounted the same volume instead of creating a new empty one 🙌

@mykola-pereyma mykola-pereyma left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix-permissions "${CONDA_DIR}" change and the volume pinning look good. One request on the Compose project rename — see the inline note on the name: line.

@@ -1,4 +1,4 @@
name: local-dev
name: graphrag-toolkit-rdf-dev

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphrag-toolkit-rdf-dev is out of step here — this stack runs neo4j-local + pgvector-local (not an RDF store), the SPARQL package that motivates "rdf" isn't in main yet, and it breaks the existing naming set (local-standard, local-dev, hybrid-standard, hybrid-dev).

Could we keep this PR as the permissions chore and leave the project as local-dev? The rdf naming would then land with the SPARQL store PR, where it matches reality.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! Thank you for your time!

@KilianTrunk
KilianTrunk force-pushed the chore/lexical-graph-dev-environment branch from bfe6860 to 605f091 Compare September 16, 2026 13:17
@KilianTrunk KilianTrunk changed the title [CHORE] Fix local Jupyter permissions and rename the Compose project [FIX] Repair local Jupyter Conda permissions Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants