[flytekit]: Nix-based ImageSpec builder wrapping makePythonProject#56
Open
Carlos-Marques wants to merge 1 commit intomasterfrom
Open
[flytekit]: Nix-based ImageSpec builder wrapping makePythonProject#56Carlos-Marques wants to merge 1 commit intomasterfrom
Carlos-Marques wants to merge 1 commit intomasterfrom
Conversation
Co-Authored-By: carlos <carlosmarques.personal@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
Superseding this with an Exa-specific implementation in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
Building Flyte task container images currently requires Docker or depot. Projects that already use Nix and
makePythonProject(viauv2nix+nix2container) have no way to leverage that infrastructure through Flyte'sImageSpecabstraction — they must maintain a parallel build path.This PR adds a
NixImageSpecBuilderso that any project with auv.lockcan build OCI images through Nix, using the sameImageSpecinterface that Flyte tasks already consume.What changes were proposed in this pull request?
New file:
flytekit/image_spec/nix_builder.pyNixImageSpecBuilder— anImageSpecBuildersubclass that:ImageSpeconly contains parameters compatible with a Nix build (rejects packages, apt_packages, conda, cuda, etc.)_copy_lock_files_into_contextfromdefault_builderto handle local-package rewritingflake.nix, uses it as-isflake.nixthat wrapsmakePythonProject(configured viaFLYTEKIT_NIX_PYTHON_FLAKEenv var)nix build(local) ornix run … copyTo(push to ECR) with cross-compilation supportnix_image_spec(requirements, **kwargs)— convenience factory that returns anImageSpecpre-configured withbuilder="nix"andnix=TrueModified:
flytekit/image_spec/__init__.pybuilder="nix"Usage:
How was this patch tested?
Unit tests in
tests/flytekit/unit/core/image_spec/test_nix_builder.py(14 tests):nix buildfor local builds, mockednix run … copyTofor ECR push, verifies correct command constructionflake.nixcontains correctmakePythonProjectcall, python version, env vars, entrypointFLYTEKIT_NIX_PYTHON_FLAKEenv varnix_image_specsets correct defaultsCheck all the applicable boxes
Link to Devin session: https://app.devin.ai/sessions/ccc5697639074693932b0b7aed586a19
Requested by: @Carlos-Marques