Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,19 @@ To build Earthly from source, you need the same requirements as Earthly. We reco
To build Earthly from source for your target system, use

- Linux and WSL

```bash
./earthly +for-linux
```

- Mac

```bash
./earthly +for-darwin
```

- Mac with M1 chip

```bash
./earthly +for-darwin-m1
```
Expand All @@ -37,17 +42,20 @@ The buildkitd image is tagged with your current branch name and also the built b

For development purposes, you may use the built `earthly` binary to rebuild itself. It's usually faster than switching between the built binary and the prerelease binary because it avoids constant buildkitd restarts. After the first initial build, you'll end up using:


- Linux and WSL

```bash
./build/linux/amd64/earthly +for-linux
```

- Mac

```bash
./build/darwin/amd64/earthly +for-darwin
```

- Mac with M1 chip

```bash
./build/darwin/amd64/earthly +for-darwin-m1
```
Expand All @@ -56,13 +64,13 @@ For development purposes, you may use the built `earthly` binary to rebuild itse

To use the [delve debugger](https://github.com/go-delve/delve) with the earthly binary, you need to disable optimizations in the 'go build' command. This is done using the -GO_GCFLAGS arg:

```
```sh
./earthly +for-own -GO_GCFLAGS='all=-N -l'
```

From there, you may use `dlv exec` against the binary, using `--` to separate dlv args from earthly args:

```
```sh
dlv exec ./build/own/earthly -- +base
Type 'help' for list of commands.

Expand Down Expand Up @@ -111,12 +119,12 @@ It is also possible to run tests without credentials. But running all of them, o
If you don't want to specify these directly on the CLI, or don't want to type these each time, it's possible to store them in [.arg and .secret files](https://docs.earthly.dev/docs/earthly-command#build-args) instead.
Here is a template to get you started:

```shell
```sh
# .arg file
DOCKERHUB_AUTH=true
```

```shell
```sh
# .secret file
DOCKERHUB_USER=<my-docker-username>
DOCKERHUB_PASS=<my-docker-password-or-token>
Expand Down Expand Up @@ -153,13 +161,13 @@ To use a mirror that requires authentication, you can run:

You can alternatively store these settings in the `.arg` and `.secret` files:

```shell
```sh
# .arg file
DOCKERHUB_MIRROR=<ip-address-or-hostname>:<port>
DOCKERHUB_MIRROR_AUTH=true
```

```shell
```sh
# .secret file
DOCKERHUB_MIRROR_USER=<my-mirror-username>
DOCKERHUB_MIRROR_PASS=<my-mirror-password>
Expand All @@ -176,10 +184,9 @@ If you have access to `earthly-technologies/core`, you can make use of the inter

which will use the credentials which are stored in earthly's [cloud-hosted secrets](https://docs.earthly.dev/earthly-cloud/cloud-secrets).


## Updates to buildkit or fsutil

Earthly is built against a fork of [buildkit](https://github.com/earthly/buildkit) and [fsutil](https://github.com/earthly/fsutil).
Earthly is built against a fork of [buildkit](https://github.com/EarthBuild/buildkit) and [fsutil](https://github.com/EarthBuild/fsutil).

To work with changes to this fork, you can use `earthly +for-linux --BUILDKIT_PROJECT=../buildkit`. This will use the local directory `../buildkit` for the buildkit code, when using buildkit in both `go.mod` and when building the buildkitd image.

Expand All @@ -202,7 +209,6 @@ To update earthly's reference to buildkit, you may run `earthly +update-buildkit

Updates to fsutil must first be vendored into buildkit, then updated under `go.mod`; additional docs and scripts exist in the buildkit repo.


## Running buildkit under debug mode

Buildkit's scheduler has a debug mode, which can be enabled with the following `~/.earthly/config.yml`[^dir] config:
Expand All @@ -216,7 +222,7 @@ then run `earthly --debug +target`.

This will produce scheduler debug messages such as

```
```text
time="2022-10-27T18:18:06Z" level=debug msg="<< unpark [eyJzbCI6eyJmaWxlIjoiRWFydGhmaWxlIiwic3RhcnRMaW5lIjoyMSwic3RhcnRDb2x1bW4iOjQsImVuZExpbmUiOjIxLCJlbmRDb2x1bW4iOjI1fSwidGlkIjoiOTEyMWZkNzYtYjI5MS00YmQyLTg2MGUtNTZhYzJjZDVhMmY3IiwidG5tIjoiK3NsZWVwIiwicGx0IjoibGludXgvYW1kNjQifQ==] RUN --no-cache sleep 123\n"
time="2022-10-27T18:18:06Z" level=debug msg="> creating jzaxegge8eh5hjqe33jybv2ml [/bin/sh -c EARTHLY_LOCALLY=false PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /usr/bin/earth_debugger /bin/sh -c 'sleep 123']" span="[eyJzbCI6eyJmaWxlIjoiRWFydGhmaWxlIiwic3RhcnRMaW5lIjoyMSwic3RhcnRDb2x1bW4iOjQsImVuZExpbmUiOjIxLCJlbmRDb2x1bW4iOjI1fSwidGlkIjoiOTEyMWZkNzYtYjI5MS00YmQyLTg2MGUtNTZhYzJjZDVhMmY3IiwidG5tIjoiK3NsZWVwIiwicGx0IjoibGludXgvYW1kNjQifQ==] RUN --no-cache sleep 123"
```
Expand Down
2 changes: 1 addition & 1 deletion docs-internals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ the [buildkit/docs/dev](https://github.com/moby/buildkit/tree/master/docs/dev) s
| **pullping** | Once the build function returns (passing a set of LLB references back to buildkit), the BuildKit server will execute the commands, and call the earthlyoutputs exporter, which will call back to the client (Earthly), which will be received by the pullping handler. This will cause earthly to perform a `docker pull ...` against the embedded registry |
| **dockertar** | The legacy approach for exporting images from BuildKit to the host via a `tar` file; we try to use pullping instead, since it only pulls the needed layers |
| **logbus** | An interface for writing output to both stdout and the web-based log viewer under cloud.earthly.dev |
| **earthlyoutputs** | A custom buildkit exporter (within the [earthly/buildkit fork](https://github.com/earthly/buildkit/tree/earthly-main/exporter/earthlyoutputs)), which is used to send images back to earthly |
| **earthlyoutputs** | A custom buildkit exporter (within the [EarthBuild/buildkit fork](https://github.com/EarthBuild/buildkit/tree/main/exporter/earthlyoutputs)), which is used to send images back to earthly |
| **embedded registry** | A [docker registry](https://github.com/distribution/distribution) which runs within the earthly-buildkitd container, used in combination with earthlyoutputs and the pullping callback; also referred to as "local registry" |

## Guides
Expand Down
Loading