diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2a7d652794..0f1e7d0241 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -13,14 +13,19 @@ To build Earthly from source, you need the same requirements as Earthly. We reco To build Earthly from source for your target system, use - Linux and WSL + ```bash ./earthly +for-linux ``` + - Mac + ```bash ./earthly +for-darwin ``` + - Mac with M1 chip + ```bash ./earthly +for-darwin-m1 ``` @@ -37,17 +42,20 @@ The buildkitd image is tagged with your current branch name and also the built b For development purposes, you may use the built `earthly` binary to rebuild itself. It's usually faster than switching between the built binary and the prerelease binary because it avoids constant buildkitd restarts. After the first initial build, you'll end up using: - - Linux and WSL + ```bash ./build/linux/amd64/earthly +for-linux ``` + - Mac + ```bash ./build/darwin/amd64/earthly +for-darwin ``` - Mac with M1 chip + ```bash ./build/darwin/amd64/earthly +for-darwin-m1 ``` @@ -56,13 +64,13 @@ For development purposes, you may use the built `earthly` binary to rebuild itse To use the [delve debugger](https://github.com/go-delve/delve) with the earthly binary, you need to disable optimizations in the 'go build' command. This is done using the -GO_GCFLAGS arg: -``` +```sh ./earthly +for-own -GO_GCFLAGS='all=-N -l' ``` From there, you may use `dlv exec` against the binary, using `--` to separate dlv args from earthly args: -``` +```sh dlv exec ./build/own/earthly -- +base Type 'help' for list of commands. @@ -111,12 +119,12 @@ It is also possible to run tests without credentials. But running all of them, o If you don't want to specify these directly on the CLI, or don't want to type these each time, it's possible to store them in [.arg and .secret files](https://docs.earthly.dev/docs/earthly-command#build-args) instead. Here is a template to get you started: -```shell +```sh # .arg file DOCKERHUB_AUTH=true ``` -```shell +```sh # .secret file DOCKERHUB_USER= DOCKERHUB_PASS= @@ -153,13 +161,13 @@ To use a mirror that requires authentication, you can run: You can alternatively store these settings in the `.arg` and `.secret` files: -```shell +```sh # .arg file DOCKERHUB_MIRROR=: DOCKERHUB_MIRROR_AUTH=true ``` -```shell +```sh # .secret file DOCKERHUB_MIRROR_USER= DOCKERHUB_MIRROR_PASS= @@ -176,10 +184,9 @@ If you have access to `earthly-technologies/core`, you can make use of the inter which will use the credentials which are stored in earthly's [cloud-hosted secrets](https://docs.earthly.dev/earthly-cloud/cloud-secrets). - ## Updates to buildkit or fsutil -Earthly is built against a fork of [buildkit](https://github.com/earthly/buildkit) and [fsutil](https://github.com/earthly/fsutil). +Earthly is built against a fork of [buildkit](https://github.com/EarthBuild/buildkit) and [fsutil](https://github.com/EarthBuild/fsutil). To work with changes to this fork, you can use `earthly +for-linux --BUILDKIT_PROJECT=../buildkit`. This will use the local directory `../buildkit` for the buildkit code, when using buildkit in both `go.mod` and when building the buildkitd image. @@ -202,7 +209,6 @@ To update earthly's reference to buildkit, you may run `earthly +update-buildkit Updates to fsutil must first be vendored into buildkit, then updated under `go.mod`; additional docs and scripts exist in the buildkit repo. - ## Running buildkit under debug mode Buildkit's scheduler has a debug mode, which can be enabled with the following `~/.earthly/config.yml`[^dir] config: @@ -216,7 +222,7 @@ then run `earthly --debug +target`. This will produce scheduler debug messages such as -``` +```text time="2022-10-27T18:18:06Z" level=debug msg="<< unpark [eyJzbCI6eyJmaWxlIjoiRWFydGhmaWxlIiwic3RhcnRMaW5lIjoyMSwic3RhcnRDb2x1bW4iOjQsImVuZExpbmUiOjIxLCJlbmRDb2x1bW4iOjI1fSwidGlkIjoiOTEyMWZkNzYtYjI5MS00YmQyLTg2MGUtNTZhYzJjZDVhMmY3IiwidG5tIjoiK3NsZWVwIiwicGx0IjoibGludXgvYW1kNjQifQ==] RUN --no-cache sleep 123\n" time="2022-10-27T18:18:06Z" level=debug msg="> creating jzaxegge8eh5hjqe33jybv2ml [/bin/sh -c EARTHLY_LOCALLY=false PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /usr/bin/earth_debugger /bin/sh -c 'sleep 123']" span="[eyJzbCI6eyJmaWxlIjoiRWFydGhmaWxlIiwic3RhcnRMaW5lIjoyMSwic3RhcnRDb2x1bW4iOjQsImVuZExpbmUiOjIxLCJlbmRDb2x1bW4iOjI1fSwidGlkIjoiOTEyMWZkNzYtYjI5MS00YmQyLTg2MGUtNTZhYzJjZDVhMmY3IiwidG5tIjoiK3NsZWVwIiwicGx0IjoibGludXgvYW1kNjQifQ==] RUN --no-cache sleep 123" ``` diff --git a/docs-internals/README.md b/docs-internals/README.md index 30920ac12c..a0b139f98b 100644 --- a/docs-internals/README.md +++ b/docs-internals/README.md @@ -32,7 +32,7 @@ the [buildkit/docs/dev](https://github.com/moby/buildkit/tree/master/docs/dev) s | **pullping** | Once the build function returns (passing a set of LLB references back to buildkit), the BuildKit server will execute the commands, and call the earthlyoutputs exporter, which will call back to the client (Earthly), which will be received by the pullping handler. This will cause earthly to perform a `docker pull ...` against the embedded registry | | **dockertar** | The legacy approach for exporting images from BuildKit to the host via a `tar` file; we try to use pullping instead, since it only pulls the needed layers | | **logbus** | An interface for writing output to both stdout and the web-based log viewer under cloud.earthly.dev | -| **earthlyoutputs** | A custom buildkit exporter (within the [earthly/buildkit fork](https://github.com/earthly/buildkit/tree/earthly-main/exporter/earthlyoutputs)), which is used to send images back to earthly | +| **earthlyoutputs** | A custom buildkit exporter (within the [EarthBuild/buildkit fork](https://github.com/EarthBuild/buildkit/tree/main/exporter/earthlyoutputs)), which is used to send images back to earthly | | **embedded registry** | A [docker registry](https://github.com/distribution/distribution) which runs within the earthly-buildkitd container, used in combination with earthlyoutputs and the pullping callback; also referred to as "local registry" | ## Guides