Skip to content

Commit 769e2f1

Browse files
committed
docs: Update release README examples to 55.2.0 and add SVN download steps
- Use 55.2.0 (and branch-55, 55.1.0) in all examples. - Replace the manual version edit and `cargo check -p datafusion` with sed commands and `cargo update --workspace`, plus a check that no reference to the old version is left. - Add `svn ls` checks and commands to `svn export`, verify and unpack the RC and release tarballs.
1 parent f9b7f34 commit 769e2f1

1 file changed

Lines changed: 105 additions & 41 deletions

File tree

‎dev/release/README.md‎

Lines changed: 105 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,18 @@ Follow the instructions in the header of the KEYS file to append your key. Here
6969
svn commit KEYS -m "Add key for John Doe"
7070
```
7171

72+
### Check SVN access
73+
74+
Check that you can read the `dev` and `release` areas of the Apache distribution SVN server:
75+
76+
```shell
77+
svn ls https://dist.apache.org/repos/dist/dev/datafusion
78+
svn ls https://dist.apache.org/repos/dist/release/datafusion
79+
```
80+
81+
The commands list the existing release candidates and releases. The scripts used below commit
82+
to these areas and will ask for your ASF credentials.
83+
7284
## Release Process: Step by Step
7385

7486
As part of the Apache governance model, official releases consist of signed
@@ -79,34 +91,47 @@ We then publish the code in the approved artifacts to crates.io.
7991

8092
First create a new release branch from `main` in the `apache` repository.
8193

82-
For example, to create the `branch-50` branch for the `50.x.y` release series:
94+
For example, to create the `branch-55` branch for the `55.x.y` release series:
8395

8496
```shell
8597
git fetch apache # make sure we are up to date
8698
git checkout apache/main # checkout current latest development branch
87-
git checkout -b branch-50 # create local branch
88-
git push -u apache branch-50 # push branch to apache remote
99+
git checkout -b branch-55 # create local branch
100+
git push -u apache branch-55 # push branch to apache remote
89101
```
90102

91103
### 2. Prepare PR to Update the Release Version
92104

93-
Manually update the DataFusion version in the root `Cargo.toml` to
94-
reflect the new release version. Ensure `Cargo.lock` is updated accordingly by
95-
running:
96-
97-
```shell
98-
cargo check -p datafusion
99-
```
100-
101-
Within the user documentation there are references to the current version number.
102-
Update these to the current version. At the time of this writing we need to manually
103-
update the following files
105+
Update the DataFusion version in the root `Cargo.toml` (the workspace version and the
106+
versions of the internal crates) and in `Cargo.lock` to reflect the new release version.
107+
Within the user documentation there are also references to the current version number.
108+
At the time of this writing we need to update the following files:
104109

105110
- `docs/source/download.md`
106111
- `docs/source/user-guide/configs.md`
107112
- `docs/source/user-guide/crate-configuration.md`
108113
- `docs/source/user-guide/example-usage.md`
109114

115+
For example, to update the version from `55.1.0` to `55.2.0`:
116+
117+
```shell
118+
# version and lockfile
119+
sed -i '' 's/"55\.1\.0"/"55.2.0"/g' Cargo.toml
120+
sed -i '' 's/55\.1\.0/55.2.0/' docs/source/download.md docs/source/user-guide/{configs,crate-configuration,example-usage}.md
121+
cargo update --workspace
122+
```
123+
124+
`sed -i ''` is the macOS (BSD) syntax, use `sed -i` with GNU `sed`. `cargo update --workspace`
125+
only refreshes the workspace crates in `Cargo.lock`, so it does not compile anything.
126+
127+
Review the diff of `Cargo.toml`: only the workspace `version` and the internal `datafusion*`
128+
entries should change. Then check that no reference to the old version is left. The following
129+
command should print nothing:
130+
131+
```shell
132+
git grep -nF '55.1.0' -- . ':!dev/changelog'
133+
```
134+
110135
Then commit the changes and create a PR targeting the release branch `branch-N`.
111136

112137
```shell
@@ -118,20 +143,20 @@ git commit -a -m 'Update version'
118143
To protect a release candidate branch from accidental merges, create PR against `main`:
119144

120145
```shell
121-
git fetch apache && git checkout -b protect_branch_50
122-
./dev/release/add-branch-protection.sh 50
146+
git fetch apache && git checkout -b protect_branch_55
147+
./dev/release/add-branch-protection.sh 55
123148
```
124149

125150
The script will modify `.asf.yaml` and add following block:
126151

127152
```yaml
128-
branch-50:
153+
branch-55:
129154
required_pull_request_reviews:
130155
required_approving_review_count: 1
131156
```
132157
133158
- Commit changes
134-
- Push to `origin/protect_branch_50`
159+
- Push to `origin/protect_branch_55`
135160
- Create a PR against `main`.
136161
- Merge to `main`.
137162
- Notify community in Discord/Slack that release branch is created
@@ -146,7 +171,7 @@ Backports are important and sometimes unexpected, so please proceed to next rele
146171
### 5. Prepare PR to Update Changelog
147172

148173
Update the changelog in `dev/changelog/`. Each release has its
149-
own file, such as `dev/changelog/50.0.0.md`, which should include all changes
174+
own file, such as `dev/changelog/55.2.0.md`, which should include all changes
150175
since the previous release.
151176

152177
The changelog is generated using a Python script, which requires a GitHub
@@ -160,7 +185,7 @@ uv sync
160185
To generate the changelog, set the `GITHUB_TOKEN` environment variable and run
161186
`./dev/release/generate-changelog.py` with two commit IDs or tags followed by
162187
the release version. For example, to generate a changelog of all changes
163-
between the `50.3.0` tag and `branch-51` for release `51.0.0`:
188+
between the `55.1.0` tag and `branch-55` for release `55.2.0`:
164189

165190
> [!NOTE]
166191
>
@@ -173,7 +198,7 @@ between the `50.3.0` tag and `branch-51` for release `51.0.0`:
173198

174199
```shell
175200
export GITHUB_TOKEN=<your-token-here>
176-
uv run ./dev/release/generate-changelog.py 50.3.0 branch-51 51.0.0 > dev/changelog/51.0.0.md
201+
uv run ./dev/release/generate-changelog.py 55.1.0 branch-55 55.2.0 > dev/changelog/55.2.0.md
177202
```
178203

179204
This script creates a changelog from GitHub PRs based on the labels associated with them as well as looking for
@@ -182,7 +207,7 @@ titles starting with `feat:`, `fix:`, or `docs:`.
182207
Once the changelog is generated, run `prettier` to format the document:
183208

184209
```shell
185-
prettier -w dev/changelog/51.0.0.md
210+
prettier -w dev/changelog/55.2.0.md
186211
```
187212

188213
Then commit the changes and create a PR targeting the release branch.
@@ -209,17 +234,17 @@ Pick numbers in sequential order, with `1` for `rc1`, `2` for `rc2`, etc.
209234

210235
While the official release artifacts are signed tarballs and zip files, we also
211236
tag the commit it was created from for convenience and code archaeology. Release tags
212-
look like `50.3.0`, and release candidate tags look like `50.3.0-rc1`. See [the list of existing
237+
look like `55.2.0`, and release candidate tags look like `55.2.0-rc1`. See [the list of existing
213238
tags].
214239

215240
[the list of existing tags]: https://github.com/apache/datafusion/tags
216241

217-
Create and push the RC tag, for example, to create the `50.3.0-rc1` tag from `branch-50`, use:
242+
Create and push the RC tag, for example, to create the `55.2.0-rc1` tag from `branch-55`, use:
218243

219244
```shell
220245
git fetch apache
221-
git tag 50.3.0-rc1 apache/branch-50
222-
git push apache 50.3.0-rc1
246+
git tag 55.2.0-rc1 apache/branch-55
247+
git push apache 55.2.0-rc1
223248
```
224249

225250
Please make sure the format is correct, tools like Homebrew listens for tags and in case of malformed tags users would be notified for non-existent version
@@ -228,10 +253,10 @@ Please make sure the format is correct, tools like Homebrew listens for tags and
228253

229254
Run the `create-tarball.sh` script with the `<version>` tag and `<rc>` number you determined in previous steps:
230255

231-
For example, to create the `50.3.0-rc1` artifacts:
256+
For example, to create the `55.2.0-rc1` artifacts:
232257

233258
```shell
234-
GITHUB_TOKEN=<TOKEN> ./dev/release/create-tarball.sh 50.3.0 1
259+
GITHUB_TOKEN=<TOKEN> ./dev/release/create-tarball.sh 55.2.0 1
235260
```
236261

237262
The `create-tarball.sh` script
@@ -243,6 +268,14 @@ The `create-tarball.sh` script
243268
2. Provides you an email template to
244269
send to `dev@datafusion.apache.org` for release voting.
245270

271+
Check that the artifacts were uploaded to SVN:
272+
273+
```shell
274+
svn ls https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-55.2.0-rc1/
275+
```
276+
277+
It should list `apache-datafusion-55.2.0.tar.gz` together with its `.asc`, `.sha256` and `.sha512` files.
278+
246279
### 7. Vote on Release Candidate Artifacts
247280

248281
Send the email output from the script to `dev@datafusion.apache.org`.
@@ -257,7 +290,21 @@ review the release candidate.
257290
`dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification process. Run it like this:
258291

259292
```shell
260-
./dev/release/verify-release-candidate.sh 50.3.0 1
293+
./dev/release/verify-release-candidate.sh 55.2.0 1
294+
```
295+
296+
To download and inspect the artifacts of a release candidate yourself, export them from SVN,
297+
check the signature and the checksums, and unpack the tarball:
298+
299+
```shell
300+
svn export https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-55.2.0-rc1
301+
cd apache-datafusion-55.2.0-rc1
302+
svn export https://dist.apache.org/repos/dist/dev/datafusion/KEYS
303+
gpg --import KEYS
304+
gpg --verify apache-datafusion-55.2.0.tar.gz.asc apache-datafusion-55.2.0.tar.gz
305+
shasum -a 256 -c apache-datafusion-55.2.0.tar.gz.sha256
306+
shasum -a 512 -c apache-datafusion-55.2.0.tar.gz.sha512
307+
tar xzf apache-datafusion-55.2.0.tar.gz
261308
```
262309

263310
#### If Changes Are Requested
@@ -282,11 +329,17 @@ with the release verification.
282329
NOTE: steps in this section can only be done by PMC members after release is approved.
283330

284331
Move artifacts to the release location in SVN, e.g.
285-
https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.3.0/, using
332+
https://dist.apache.org/repos/dist/release/datafusion/datafusion-55.2.0/, using
286333
the `release-tarball.sh` script:
287334

288335
```shell
289-
./dev/release/release-tarball.sh 50.3.0 1
336+
./dev/release/release-tarball.sh 55.2.0 1
337+
```
338+
339+
Check that the artifacts are in the release location:
340+
341+
```shell
342+
svn ls https://dist.apache.org/repos/dist/release/datafusion/datafusion-55.2.0/
290343
```
291344

292345
Congratulations! The release is now official!
@@ -296,9 +349,9 @@ Congratulations! The release is now official!
296349
Tag the same release candidate commit with the final release tag
297350

298351
```shell
299-
git checkout 50.3.0-rc1
300-
git tag 50.3.0
301-
git push apache 50.3.0
352+
git checkout 55.2.0-rc1
353+
git tag 55.2.0
354+
git push apache 55.2.0
302355
```
303356

304357
### 10. Publish on Crates.io
@@ -315,10 +368,21 @@ instructions](https://doc.rust-lang.org/cargo/reference/publishing.html) to
315368
create an account and login to crates.io before asking to be added as an owner
316369
to all DataFusion crates.
317370

318-
Download and unpack the official release tarball
371+
Download the official release tarball from SVN, check its signature and checksum, and unpack it:
372+
373+
```shell
374+
svn export https://dist.apache.org/repos/dist/release/datafusion/datafusion-55.2.0
375+
cd datafusion-55.2.0
376+
svn export https://dist.apache.org/repos/dist/release/datafusion/KEYS
377+
gpg --import KEYS
378+
gpg --verify apache-datafusion-55.2.0.tar.gz.asc apache-datafusion-55.2.0.tar.gz
379+
shasum -a 512 -c apache-datafusion-55.2.0.tar.gz.sha512
380+
tar xzf apache-datafusion-55.2.0.tar.gz
381+
cd apache-datafusion-55.2.0
382+
```
319383

320384
Verify that the Cargo.toml in the tarball contains the correct version
321-
(e.g. `version = "50.3.0"`) and then publish the crates by running the following commands
385+
(e.g. `version = "55.2.0"`) and then publish the crates by running the following commands
322386

323387
```shell
324388
(cd datafusion/common && cargo publish)
@@ -370,10 +434,10 @@ If it happens crates.io fails with wrong dependency message like below, just rer
370434
error: failed to prepare local package for uploading
371435
372436
Caused by:
373-
failed to select a version for the requirement `datafusion-proto = "^53.1.0"`
374-
candidate versions found which didn't match: 53.0.0, 52.5.0, 52.4.0, ...
437+
failed to select a version for the requirement `datafusion-proto = "^55.2.0"`
438+
candidate versions found which didn't match: 55.1.0, 55.0.0, 54.1.0, ...
375439
location searched: crates.io index
376-
required by package `datafusion-ffi v53.1.0 (/private/tmp/apache-datafusion-53.1.0/datafusion/ffi)`
440+
required by package `datafusion-ffi v55.2.0 (/private/tmp/apache-datafusion-55.2.0/datafusion/ffi)`
377441
```
378442

379443
### Publish datafusion-cli on Homebrew
@@ -423,7 +487,7 @@ svn ls https://dist.apache.org/repos/dist/dev/datafusion
423487
To delete a release candidate:
424488

425489
```shell
426-
svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-50.0.0-rc1/
490+
svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-55.2.0-rc1/
427491
```
428492

429493
#### Delete old releases from `release` SVN
@@ -440,5 +504,5 @@ svn ls https://dist.apache.org/repos/dist/release/datafusion
440504
To delete a release:
441505

442506
```shell
443-
svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.0.0
507+
svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-55.1.0
444508
```

0 commit comments

Comments
 (0)