-
Notifications
You must be signed in to change notification settings - Fork 3
Run-BuildbotWorker improvements #474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -74,6 +74,34 @@ Make sure the amount of disk space is set in the storage-opts setting of the dae | |
| "storage-opts": [ "size={0}GB" ]' -f $minDiskSpace); | ||
| } | ||
|
|
||
| # Make a request to aws.amazon.com to prevent certificate errors when interacting with S3 | ||
| Write-Host 'Make a HTTP request to aws.amazon.com to prevent S3 certificate errors'; | ||
| $commandletArgs = @{ | ||
| UseBasicParsing = $true; | ||
| Method = 'Head' | ||
| }; | ||
| $urls = @( | ||
| 'https://aws.amazon.com' | ||
| ); | ||
|
|
||
| if (Test-Path env:HTTPS_PROXY) { | ||
| $commandletArgs['Proxy'] = $env:HTTPS_PROXY; | ||
| } | ||
|
|
||
| foreach ($url in $urls) { | ||
| $commandletArgs['Uri'] = $url; | ||
| Write-Host ('Requesting {0}' -f $url); | ||
| Invoke-WebRequest @commandletArgs; | ||
| } | ||
|
Comment on lines
+77
to
+95
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hey @iangrunert so during development of these images I added some extensibility under the assumption that maybe there's some weird environment specific stuff needed for the host. In this particular case the above only seemed to be needed on a Windows Server host. It wasn't needed when running a Win 10/11 host. So that was a script done over in https://github.com/WebKitForWindows/buildbot-scripts . Longer explanation at https://github.com/WebKitForWindows/buildbot-scripts/tree/main?tab=readme-ov-file#worker-container-administration-scripts So this portion I'd probably be a no on.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I haven't built an automation around this, so end up manually run the copy / expand and restart the container afterwards. If you feel strongly about it though - I'll remove this and write a script to work around it. Is anyone else actually using Run-BuildbotWorker.ps1 other than the Windows bots? If we're maintaining extra abstraction for a hypothetical situation where someone runs a buildbot on Win 10 / 11 instead of Server, when this request shouldn't break them anyway - it feels a little silly to generate additional automation work for myself.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are you using https://github.com/WebKitForWindows/buildbot-scripts at all? The idea is that your Our infrastructure had a bunch of fun things to deal with like proxies that has us running a bunch more in that scripts directory. The idea was also about iterating since its simple to change the contents in that scripts directory vs doing a
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah I'm using it on each host, but only for the Invoke-WebRequests script. |
||
|
|
||
| # Move the git bash msys perl out of the way, so strawberry perl is on the path | ||
| Write-Host "Rename git bash msys perl, so strawberry perl is picked up first on PATH" | ||
| bash -c "test -f /usr/bin/perl && mv /usr/bin/perl /usr/bin/perl-msys" | ||
|
Comment on lines
+97
to
+99
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Feel like this should be in a docker file proper. |
||
|
|
||
| # Clear ccache on restart | ||
| Write-Host "Clearing ccache" | ||
| ccache.exe -C | ||
|
Comment on lines
+101
to
+103
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same with this. I'd put it in the |
||
|
|
||
| # Initialize the Visual Studio environment | ||
| Write-Host 'Initializing Visual Studio environment'; | ||
| Initialize-VSEnvironment -Architecture 'amd64' -Path (Get-VSBuildTools2022VCVarsAllPath); | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you seeing cases where the docker daemon is not up and running? Or is it because you're trashing a bunch of stuff on disk that might cause problems with it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, the build has been flaky failing on the
docker system prune -a -fcommand as it couldn't connect to the Docker daemon.