Conversation
iangrunert
force-pushed
the
ig/buildbot-worker-fixes
branch
from
76b139c to
0ae70d0
Compare
Automating a couple of manual steps: * Making a HTTP request to aws.amazon.com to prevent cert errors * Move perl to perl-msys so strawberry perl is on PATH for build * Clear ccache on restart Also wait for Docker to start at the beginning of the github actions build to work around actions/runner-images#13729
iangrunert
force-pushed
the
ig/buildbot-worker-fixes
branch
from
0ae70d0 to
3ddb695
Compare
| # Make a request to aws.amazon.com to prevent certificate errors when interacting with S3 | ||
| Write-Host 'Make a HTTP request to aws.amazon.com to prevent S3 certificate errors'; | ||
| $commandletArgs = @{ | ||
| UseBasicParsing = $true; | ||
| Method = 'Head' | ||
| }; | ||
| $urls = @( | ||
| 'https://aws.amazon.com' | ||
| ); | ||
|
|
||
| if (Test-Path env:HTTPS_PROXY) { | ||
| $commandletArgs['Proxy'] = $env:HTTPS_PROXY; | ||
| } | ||
|
|
||
| foreach ($url in $urls) { | ||
| $commandletArgs['Uri'] = $url; | ||
| Write-Host ('Requesting {0}' -f $url); | ||
| Invoke-WebRequest @commandletArgs; | ||
| } |
There was a problem hiding this comment.
Hey @iangrunert so during development of these images I added some extensibility under the assumption that maybe there's some weird environment specific stuff needed for the host. In this particular case the above only seemed to be needed on a Windows Server host. It wasn't needed when running a Win 10/11 host. So that was a script done over in https://github.com/WebKitForWindows/buildbot-scripts .
Longer explanation at https://github.com/WebKitForWindows/buildbot-scripts/tree/main?tab=readme-ov-file#worker-container-administration-scripts
So this portion I'd probably be a no on.
There was a problem hiding this comment.
I haven't built an automation around this, so end up manually run the copy / expand and restart the container afterwards. If you feel strongly about it though - I'll remove this and write a script to work around it.
Is anyone else actually using Run-BuildbotWorker.ps1 other than the Windows bots? If we're maintaining extra abstraction for a hypothetical situation where someone runs a buildbot on Win 10 / 11 instead of Server, when this request shouldn't break them anyway - it feels a little silly to generate additional automation work for myself.
There was a problem hiding this comment.
Are you using https://github.com/WebKitForWindows/buildbot-scripts at all? The idea is that your docker run includes -v <path-to-scripts>:C:/Scripts and then anything specific to your environment can run.
Our infrastructure had a bunch of fun things to deal with like proxies that has us running a bunch more in that scripts directory.
The idea was also about iterating since its simple to change the contents in that scripts directory vs doing a docker build.
There was a problem hiding this comment.
Yeah I'm using it on each host, but only for the Invoke-WebRequests script.
| # Clear ccache on restart | ||
| Write-Host "Clearing ccache" | ||
| ccache.exe -C |
There was a problem hiding this comment.
Same with this. I'd put it in the Scripts directory since any ccache clearing logic could go in there.
| # Move the git bash msys perl out of the way, so strawberry perl is on the path | ||
| Write-Host "Rename git bash msys perl, so strawberry perl is picked up first on PATH" | ||
| bash -c "test -f /usr/bin/perl && mv /usr/bin/perl /usr/bin/perl-msys" |
There was a problem hiding this comment.
Feel like this should be in a docker file proper.
| - name: Wait for Docker | ||
| run: | | ||
| Write-Host "Waiting for Docker to be ready..." | ||
| for ($i = 1; $i -le 60; $i++) { | ||
| $output = docker info 2>&1 | ||
| if ($LASTEXITCODE -eq 0) { | ||
| Write-Host "Docker is ready!" | ||
| break | ||
| } | ||
| if ($i -eq 1) { | ||
| Write-Host "Docker not ready. Checking for Docker services..." | ||
| $dockerServices = Get-Service | Where-Object { $_.Name -like '*docker*' -or $_.DisplayName -like '*docker*' } | ||
| if ($dockerServices) { | ||
| Write-Host "Available Docker services:" | ||
| $dockerServices | Format-Table Name, Status, DisplayName -AutoSize | ||
| $dockerServices | Where-Object { $_.Status -ne 'Running' } | ForEach-Object { | ||
| Write-Host "Starting service: $($_.Name)" | ||
| Start-Service $_.Name -ErrorAction SilentlyContinue | ||
| } | ||
| } else { | ||
| Write-Error "No Docker services found." | ||
| exit 1 | ||
| } | ||
| } | ||
| if ($i % 10 -eq 0) { | ||
| Write-Host "Attempt $i/60 - Still waiting... Error: $output" | ||
| } else { | ||
| Write-Host "Attempt $i/60" | ||
| } | ||
| if ($i -eq 60) { | ||
| Write-Error "Docker failed to start after 60 attempts" | ||
| exit 1 | ||
| } | ||
| Start-Sleep -Seconds 2 | ||
| } |
There was a problem hiding this comment.
Are you seeing cases where the docker daemon is not up and running? Or is it because you're trashing a bunch of stuff on disk that might cause problems with it?
There was a problem hiding this comment.
Yep, the build has been flaky failing on the docker system prune -a -f command as it couldn't connect to the Docker daemon.
2 participants
Automating a couple of manual steps: