Repository navigation
Conversation
|
I am not an authenticated Apache committer. How does this know that it should not publish my builds? Is it because I have not provisioned a Gradle Enterprise access key for ge.apache.org as instructed in https://docs.gradle.com/enterprise/maven-extension/#authenticating_with_gradle_enterprise, or does this anyway ask the server whether I am authenticated? |
|
It is the former: because you have no access key, it will not attempt to authenticate. The unauthenticated contributor won't see anything different happen in their build. |
|
What useful information would GE give to the developer ? |
martin-g
left a comment
martin-g
left a comment
There was a problem hiding this comment.
Also I think there is no need to upload data for the non-Java workflows, i.e. the interop jobs.
|
Related apache.org links:
|
| <extension> | ||
| <groupId>com.gradle</groupId> | ||
| <artifactId>gradle-enterprise-maven-extension</artifactId> | ||
| <version>1.19.2</version> | ||
| </extension> |
There was a problem hiding this comment.
What are the license terms for gradle-enterprise-maven-extension, when invoked in an Avro build without an access key?
There was a problem hiding this comment.
https://mvnrepository.com/artifact/com.gradle/gradle-enterprise-maven-extension/1.19.2 points to Gradle Terms of Service, which says it does not apply to "Develocity software", and it's not clear to me whether the extension is included in that.
(This PR adds common-custom-user-data-maven-extension as well, but https://mvnrepository.com/artifact/com.gradle/common-custom-user-data-maven-extension/1.12.4 points to Apache-2.0.)
There was a problem hiding this comment.
Would this PR cause new legal responsibilities for a person who is not an Apache committer but makes a pull request to this repository, triggering a GitHub Action that accesses the Develocity server? That action seems to be "contributing code", and the following definition in Develocity Software License Agreement would then cover the person:
“User” means a person who accesses, uses, interacts with or directs the Software in the performance of its functions and/or contributing code to a source repository that has a build that is connected with Develocity to collect build data, use the Develocity build cache or use the Develocity testing agent. For the avoidance of doubt, any person who accesses, uses or interacts with the Software for oversight or governance purposes (i.e. internal audit, external audit, or internal security risk personnel) shall not be deemed to be a “User” under this Agreement for purposes of Seat count.
Would such a "User" then also be an "end user" as referenced in clause 3.2 ("any end user of the Software shall be subject to and bound by the terms of this Agreement")?
There was a problem hiding this comment.
I don't think a pull request in itself == 'committing code'. That would happen when a committer, who is already covered by their ICLA, commits the pull request
There was a problem hiding this comment.
The license agreement references "contributing code", not "committing code".
There was a problem hiding this comment.
This is a good question. I believe there is a particular license in place for usage with ASF. Let me track that information down.
There was a problem hiding this comment.
Hey there ... I was pointed to this issue as something to review and help to resolve. As the Officer in charge of Infrastructure, I'll get an FAQ developed for our communities. As @clayburn notes: there is a specific agreement that I signed with Gradle, Inc for our usage of Develocity, which is not the same as their general/public terms.
We'll get this sorted, and come back with more information.
3a8569a to
a4685cc
Compare
Done, although I left if for the Java one. If it doesn't make sense there either, let me know.
Good question. Most that we work with do not like having to parse through logs, but beyond that:
By aggregating all of this data, you can start to see trends as well through some of the performance, failure, and test dashboards. One popular use case is identifying flaky tests so that they can be acted on (for example). Perhaps those I tagged in the PR will have more direct ideas on how this can help Avro based on what they saw. I created a build scan of the Avro project over here if you are interested in seeing the specifics for this project. |
|
Wow -- thanks for this PR, it's really educational for me! I'm currently still on a vacation after the Community over Code conference, but I'm looking forward to looking at this next week ❤️ |
RyanSkraba
left a comment
RyanSkraba
left a comment
There was a problem hiding this comment.
Hey -- I've reviewed this PR and it's consistent with the instructions in the INFRA onboarding. Taking a look at the example build: it looks pretty nice, and even just having the timings for the tasks and tests is a pretty compelling feature.
I don't see any harm in enabling this, but I'll reach out on the dev@ mailing list regardless before merging, so people can see and be reassured that nothing changes for local development or on their own forks.
As a foundation-wide service, I'm assuming this was vetted as safe to merge, but I'll also wait to hear back from @gstein! (Thanks for reading so carefully @KalleOlaviNiemitalo).
| <enabled>false</enabled> | ||
| </local> | ||
| <remote> | ||
| <enabled>false</enabled> |
There was a problem hiding this comment.
My comment is probably out of scope for this PR but I am curious @clayburn
If we enable remote caching, will it improve our build speed even if we use maven?
There was a problem hiding this comment.
Yes, although there are some ASF Infra side changes we want to make before supporting remote caching for projects in order to make sure that projects cannot read cache entries from other projects.
It may also be the case that there are some behaviors in the project that invalidate cache entries frequently. After this PR is accepted, we will put in time to investigating these things and offering a PR to fix them if they exist.
|
I am +1 with this, the changes seem not intrusive so we can 'easily' roll them back if we decide so and the advantages are there. (Ryan just for my own reference where can we find the INFRA instructions for this tool). |
| <testLogging>true</testLogging> | ||
| </capture> | ||
| <backgroundBuildScanUpload>#{isFalse(env['GITHUB_ACTIONS'])}</backgroundBuildScanUpload> | ||
| <publish>ALWAYS</publish> |
There was a problem hiding this comment.
Will there be a way to 'easily' see/find the URL of the published build in the run CI jobs? I was expecting some change on the github actions file for this.
There was a problem hiding this comment.
We have this for gradle builds via the gradle-build-action. It creates a job summary with links to the build scans. For example, you can see them in this GitHub Actions build.
We don't yet have this for Maven builds, although it is in our backlog to make something like this for Maven.
Add the Develocity Maven extension and Common Custom User Data extension so that Maven builds publish a Build Scan to the ASF Develocity instance when, and only when, the build is authenticated with an access key. Builds without a key, including fork pull requests and contributors' local builds, are unaffected. The Java-focused workflows export DEVELOCITY_ACCESS_KEY from the secret of the same name. Interop jobs in the other language workflows are left out. The Develocity build cache stays disabled, since the build already uses the Apache Maven Build Cache Extension.
clayburn
changed the title
582d8fc to
1ec1845
Compare
|
@RyanSkraba, thanks for the chat today! As discussed, I've brought this up to date for develocity.apache.org. Two things when you have a moment:
The PR is safe to merge before that: without the secret, builds just don't publish. |
7 participants
What is the purpose of the change
This change, for AVRO-3882, publishes a Build Scan from Avro's Java CI builds, and from local builds by committers who have set up an access key, to develocity.apache.org. The ASF hosts that Develocity instance for its projects, and every Apache project can use it at no cost. Builds without a key publish nothing and behave exactly as they do today. That covers fork pull requests and contributors' local builds.
I've reworked this PR since it was first opened in 2023. The instance has moved from ge.apache.org to develocity.apache.org, and the Gradle Enterprise extension has been renamed Develocity.
.mvn/extensions.xmladdsdevelocity-maven-extension2.6.0 andcommon-custom-user-data-maven-extension2.4.0 next to the existingmaven-build-cache-extension..mvn/develocity.xmlsets the server andprojectIdavro. It publishes only when the build is authenticated, uploads in the background except on GitHub Actions, and obfuscates IP addresses.DEVELOCITY_ACCESS_KEYis exported from the secret of the same name intest-lang-java,rat,spotlessandjava-publish-snapshot. As @martin-g suggested, the interop jobs in the other language workflows are left out..mvn/.develocity/is gitignored.Once this is in, Avro gets dashboards across its whole build history, build and test failure analytics, and flaky test detection (example from Kafka).
Before merging: ASF Infra has to grant
apache/avroaccess to the sharedDEVELOCITY_ACCESS_KEYActions secret. A PMC member requests this with an INFRA Jira ticket, as described in infra.apache.org/gradle.html. Until Infra grants it, the workflows get an empty key and simply don't publish, so merging first does no harm. Scans start appearing as soon as access is granted.Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
I checked it locally with
mvn -B -ntp validate -pl :avro-toplevelon Maven 3.10.0 withGITHUB_ACTIONS=trueset:Documentation