[BUG] AWS SigV4 signature broken by Guzzle middlewares abd OpenTelemetry extension

[kidager]

What is the bug?

The AWS SigV4 signature can get corrupted.

When using the SigningClientDecorator, the AWS signature is calculated before the sendRequest method, which seems to be a bit early for some clients.

Guzzle, for example, have middlewares and handlers that can potentially add new headers, these middlewares are executed inside the sendRequest method, which is called after signing the request by the SigningClientDecorator, which will lead to a signature mismatch.

How can one reproduce the bug?

• Use the GuzzleClientFactory to create a client
• In the options, have a middleware that adds a new header to the request
• Run $client->info() or any other command
• You should see an error stating that there was a signature mismatch

# The private data in the error have been redacted

The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
'GET
/

host:aws-host.aos.us-east-1.on.aws
new-header:<header-value>
x-amz-content-sha256:<hash>
x-amz-date:<date>
x-amz-security-token:<token>

host;new-header;x-amz-content-sha256;x-amz-date;x-amz-security-token
<hash>'

The String-to-Sign should have been
'AWS4-HMAC-SHA256
20251119T154521Z
20251119/us-east-1/es/aws4_request
39d0b2f58483c7d31e0b1ae5b11c831201821d2ac51f7c2850aae1f32e0bd590'

What is the expected behavior?

The AWS signature should include the newly added header in the calculation

What is your host/environment?

ECS on AWS, using the php:8.3-fpm base image

Do you have any screenshots?

Do you have any additional context?

This also happen when having the open-telemetry package installed and enabled, since they also inject some w3c tracing headers just before sending the request. This is how we discovered the issue, we first thought it was due to open-telemetry but after debugging and narrowing it down, we figured out it was due to the wrapping of the http client inside an aws signing decorator.

[kidager]

To by pass this bug, I've had to add the AWS signature as a guzzle middleware and ignore the SigningClientDecorator.

# Laravel specific code that can be adapted if needed
$this->app->singleton(
    OpenSearchClient::class,
    static function (Container $app) {
        $options = [
            'base_uri' => config('opensearch.endpoint'),
            'verify' => config('opensearch.ssl-verification'),
            'timeout' => config('opensearch.timeout'),
        ];

        if (true === config('opensearch.aws-auth')) {
            $options['middleware'] = [
                'aws-sign-request' => static fn (callable $handler) => new AwsSignRequestMiddleware($handler),
            ];
        }

        return (new GuzzleClientFactory())->create($options);
    }
);

<?php

# AwsSignRequestMiddleware: Mostly burrowed from the SigningClientDecorator

declare(strict_types=1);

namespace App\Extensions\Guzzle\Middlewares;

use Aws\Credentials\CredentialProvider;
use Aws\Credentials\Credentials;
use Aws\Exception\CredentialsException;
use Aws\Signature\SignatureInterface;
use Aws\Signature\SignatureV4;
use GuzzleHttp\Promise\PromiseInterface;
use Illuminate\Support\Facades\Log;
use Psr\Http\Message\RequestInterface;

class AwsSignRequestMiddleware
{
    public const array ALLOWED_SERVICES = ['es', 'aoss'];

    protected ?SignatureInterface $signer = null;

    /**
     * @var callable(RequestInterface, array<string, mixed>): PromiseInterface
     */
    private $nextHandler;

    /**
     * @param callable(RequestInterface, array<string, mixed>): PromiseInterface $nextHandler Next handler to invoke.
     */
    public function __construct(callable $nextHandler)
    {
        $this->nextHandler = $nextHandler;
    }

    /**
     * @param array<string, mixed> $options
     */
    public function __invoke(RequestInterface $request, array $options): PromiseInterface
    {
        $awsOptions = $options['aws_auth'] ?? $options;
        unset($options['aws_auth']);

        // Get the credentials.
        $provider = CredentialProvider::defaultProvider();
        $promise = $provider();
        try {
            $credentials = $promise->wait();
        } catch (CredentialsException $e) {
            Log::error('Failed to get AWS credentials: @message', ['@message' => $e->getMessage()]);
            $credentials = new Credentials('', '');
        }

        $request = $request->withHeader('x-amz-content-sha256', hash('sha256', (string) $request->getBody()));
        $request = $this->getSigner($awsOptions)->signRequest($request, $credentials);

        return ($this->nextHandler)($request, $options);
    }

    /**
     * Gets the request signer.
     *
     * @param array<string,string> $options The aws options.
     */
    protected function getSigner(array $options): SignatureInterface
    {
        if ($this->signer) {
            return $this->signer;
        }

        $region = $options['region'] ?? config('settings.aws_default_region');

        if (!isset($region)) {
            throw new \InvalidArgumentException('The region option is required.');
        }

        $service = $options['service'] ?? 'es';

        if (!in_array($service, self::ALLOWED_SERVICES, true)) {
            throw new \InvalidArgumentException('The service option must be either "es" or "aoss".');
        }

        return new SignatureV4($service, $region, $options);
    }
}

[andrross]

Catch All Triage - 1 2 3