diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4c9db4181..cd6d81930 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,7 +17,7 @@ jobs: runs-on: macos-15 timeout-minutes: 30 env: - AUTHORIZATION_TEST_SUITES: 'RefreshableHLSAuthorizationTests|RefreshableSubtitleAuthorizationTests' + AUTHORIZATION_TEST_SUITES: 'RefreshableHLSAuthorizationTests|RefreshableSubtitleAuthorizationTests|LiveTrustEvaluatorTests' steps: - uses: actions/checkout@v4 @@ -43,8 +43,10 @@ jobs: id: swift-tests run: swift test --skip "$AUTHORIZATION_TEST_SUITES" - # These suites assert short authorization deadlines. Run them in a separate process + # These suites use short authorization deadlines. Run them in a separate process # so blocking work in the broader suite cannot starve their async resolvers. + # LiveTrustEvaluatorTests holds the redirect-scope tests, which have those deadlines, and + # the live TLS handshake tests, which share its serialized hold on the global evaluator. # The shared filter partitions all tests without changing their deadlines or parallelism. - name: Authorization deadline regressions if: ${{ !cancelled() && steps.swift-tests.outcome != 'skipped' }} diff --git a/CHANGELOG.md b/CHANGELOG.md index 15a8cc03b..9387976a5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,12 +12,14 @@ the public-API contract. ### Changed -- CI runs the HLS and subtitle/resource authorization suites in a separate test process so unrelated blocking tests cannot delay their resolvers. All tests and existing deadlines remain enforced. +- CI runs the HLS, subtitle/resource authorization and live trust-evaluator suites in a separate test process so unrelated blocking tests cannot delay their resolvers. All tests and existing deadlines remain enforced. +- Once a redirect chain has reached HTTPS, the engine drops credential headers (`Authorization`, `Cookie`, Emby/Jellyfin tokens) from every later HTTP hop, including headers returned by `HTTPRequestAuthorization`. A provider that authorizes every URL can no longer send its credentials in cleartext after a downgrade. The HTTP hop still runs, so anonymous redirects keep working, and an origin the host configured as HTTP still receives provider credentials. ### Added - `LoadOptions.objectAudioRendering` keeps the height channels and object positioning of TrueHD Atmos by rendering its objects into a speaker bed; it is a lossy conversion, not passthrough of the original stream. With `.apac(SpatialSpeakerLayout)` a TrueHD track FFmpeg marks as Atmos is decoded (beds, objects and their metadata), rendered into the chosen 5.1.2 to 9.1.6 speaker bed and delivered as Apple Positional Audio (`CODECS="apac.31.LL"`), which tvOS sends to an Atmos receiver as Dolby MAT. Lossy (320 kbps per bed channel) in place of the lossless 7.1 channel presentation, so it is opt-in; requires OS 26 and falls back to `audioBridgeMode` otherwise. See [formats.md › TrueHD Atmos (object rendering)](docs/formats.md#truehd-atmos-object-rendering). - `aetherctl serve --atmos-bed ` and `serve --audio-index ` serve a TrueHD Atmos track through that path. +- Live HTTPS-to-HTTP redirect tests cover provider refusal, anonymous redirects, static-header stripping and the downgrade credential filter. - `ExternalSubtitleTrack.httpRequestAuthorization` supplies refreshable headers for primary/secondary sidecars and native subtitle stores without changing registered track IDs or rendition mappings. Authorized container decoding retains AVIO streaming and range access. - `HTTPRequestAuthorization.data(from:maximumBytes:)` fetches raw auxiliary resources such as font bundles with a caller-supplied byte limit and a whole-transfer deadline, reusing the relay's redirect, authorization, retry, cancellation and TLS policy. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b341ab27a..dacf79664 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -17,14 +17,18 @@ swift build swift test ``` -CI runs `RefreshableHLSAuthorizationTests` and `RefreshableSubtitleAuthorizationTests` -in a separate process because their short deadlines require responsive async resolvers. -Blocking work elsewhere in the suite can delay those resolvers on smaller runners. +CI runs `RefreshableHLSAuthorizationTests`, `RefreshableSubtitleAuthorizationTests` and +`LiveTrustEvaluatorTests` in a separate process because their short authorization deadlines require +responsive async resolvers. Blocking work elsewhere in the suite can delay those resolvers on smaller +runners. `LiveTrustEvaluatorTests` is the serialized parent of every live suite that sets the +process-global `EngineTLS.serverTrustEvaluator`. Nest any new suite that sets it there; the +redirect-scope tests carry the deadlines, and the TLS handshake tests run with them because they share +that global. The two commands below cover the entire test suite, keeping the existing deadlines and parallel execution within each group: ```bash -AUTHORIZATION_TEST_SUITES='RefreshableHLSAuthorizationTests|RefreshableSubtitleAuthorizationTests' +AUTHORIZATION_TEST_SUITES='RefreshableHLSAuthorizationTests|RefreshableSubtitleAuthorizationTests|LiveTrustEvaluatorTests' swift test --skip "$AUTHORIZATION_TEST_SUITES" swift test --skip-build --filter "$AUTHORIZATION_TEST_SUITES" ``` diff --git a/Sources/AetherEngine/Demuxer/RedirectHeaderPolicy.swift b/Sources/AetherEngine/Demuxer/RedirectHeaderPolicy.swift index 1992e53ea..185326261 100644 --- a/Sources/AetherEngine/Demuxer/RedirectHeaderPolicy.swift +++ b/Sources/AetherEngine/Demuxer/RedirectHeaderPolicy.swift @@ -25,7 +25,11 @@ enum RedirectHeaderPolicy { if credentialsAllowed(from: originalURL, to: redirectURL) { return extraHeaders } - return extraHeaders.filter { !credentialHeaders.contains($0.key.lowercased()) } + return withoutCredentials(extraHeaders) + } + + static func withoutCredentials(_ headers: [String: String]) -> [String: String] { + headers.filter { !credentialHeaders.contains($0.key.lowercased()) } } /// Builds the request actually handed back to URLSession on redirect: re-applies the diff --git a/Sources/AetherEngine/Network/EngineTLS.swift b/Sources/AetherEngine/Network/EngineTLS.swift index 35a477bcc..2bc6d017d 100644 --- a/Sources/AetherEngine/Network/EngineTLS.swift +++ b/Sources/AetherEngine/Network/EngineTLS.swift @@ -40,15 +40,17 @@ public enum EngineTLS { /// Single disposition shared by the session-level delegate and the /// per-task delegates in AVIOReader. Anything other than a server-trust /// challenge the host accepted is left to default handling, so client - /// certificates and HTTP auth behave exactly as before. + /// certificates and HTTP auth behave exactly as before. `evaluator` defaults to the host's + /// answer; unit tests pass their own rather than writing the process global. static func resolve( _ challenge: URLAuthenticationChallenge, + evaluator: (@Sendable (URLProtectionSpace) -> Bool)? = EngineTLS.serverTrustEvaluator, completionHandler: (URLSession.AuthChallengeDisposition, URLCredential?) -> Void ) { guard challenge.protectionSpace.authenticationMethod == NSURLAuthenticationMethodServerTrust, - let evaluator = serverTrustEvaluator, + let evaluator, evaluator(challenge.protectionSpace), let trust = challenge.protectionSpace.serverTrust else { diff --git a/Sources/AetherEngine/Network/HLSOriginRelay.swift b/Sources/AetherEngine/Network/HLSOriginRelay.swift index 2e136c689..534adabdf 100644 --- a/Sources/AetherEngine/Network/HLSOriginRelay.swift +++ b/Sources/AetherEngine/Network/HLSOriginRelay.swift @@ -393,11 +393,20 @@ final class HLSOriginRelay: @unchecked Sendable { var retryHeaders: [String: String]? var challenged = false var redirects = 0 + // Once a chain has reached TLS, no credential crosses a later cleartext hop, whatever the + // provider answers for it. The hop itself still runs, so anonymous redirects keep working. + var reachedTLS = Self.isTLS(origin) + func downgradeSafe(_ headers: [String: String], to destination: URL) -> [String: String] { + reachedTLS && !Self.isTLS(destination) ? RedirectHeaderPolicy.withoutCredentials(headers) : headers + } while true { let resolved: [String: String] do { if let retryHeaders { resolved = retryHeaders } - else { resolved = try authorize(url, rejectedHeaders: nil, fallback: staticHeaders) } + else { + let answer = try authorize(url, rejectedHeaders: nil, fallback: staticHeaders) + resolved = downgradeSafe(answer, to: url) + } } catch { reportRequestFailure(); return .failed } retryHeaders = nil var request = URLRequest(url: url) @@ -434,11 +443,13 @@ final class HLSOriginRelay: @unchecked Sendable { staticHeaders = RedirectHeaderPolicy.headersToReplay( extraHeaders: staticHeaders, originalURL: url, redirectURL: destination) url = destination + reachedTLS = reachedTLS || Self.isTLS(destination) case .challenge(let response, let sentHeaders): challenged = true let respondingURL = response.url ?? url - guard let fresh = try? authorize(respondingURL, rejectedHeaders: sentHeaders, fallback: [:]), - Self.authorizationValue(fresh) != Self.authorizationValue(sentHeaders) else { + let fresh = (try? authorize(respondingURL, rejectedHeaders: sentHeaders, fallback: [:])) + .map { downgradeSafe($0, to: respondingURL) } + guard let fresh, Self.authorizationValue(fresh) != Self.authorizationValue(sentHeaders) else { reportRequestFailure() return .held(Fetched(url: respondingURL, status: 401, body: Data(), contentType: nil, contentRange: nil)) } @@ -452,6 +463,8 @@ final class HLSOriginRelay: @unchecked Sendable { "range", "host", "content-length", "transfer-encoding", "connection", "trailer", "te", "upgrade" ] + private static func isTLS(_ url: URL) -> Bool { url.scheme?.lowercased() == "https" } + private static func authorizationValue(_ headers: [String: String]) -> String? { headers.first { $0.key.caseInsensitiveCompare("Authorization") == .orderedSame }?.value } diff --git a/Sources/AetherEngine/Network/HTTPRequestAuthorization.swift b/Sources/AetherEngine/Network/HTTPRequestAuthorization.swift index 3355796e0..22d5fb1e6 100644 --- a/Sources/AetherEngine/Network/HTTPRequestAuthorization.swift +++ b/Sources/AetherEngine/Network/HTTPRequestAuthorization.swift @@ -8,6 +8,10 @@ import Foundation /// Direct media AVIO and live ingest still use their existing static headers. /// The resolver must independently validate every URL, including redirects and playlist-discovered /// origins. Discovery grants no credential authority. Credentials must never be placed in URLs. +/// Include scheme, host and effective port in that scope. Redirects are authorized afresh. Throw to +/// refuse a destination, or return no credentials to allow an anonymous request. Once a chain has +/// reached HTTPS, credential headers are dropped from every later HTTP hop, resolver output included; +/// a chain that starts on HTTP sends what the resolver returns. /// /// The engine owns Range, Host, and HTTP framing. A nil rejected-header dictionary asks for a new /// request. A nonnil dictionary is the actual request headers rejected by one HTTP 401; returning a diff --git a/Tests/AetherEngineTests/EngineTLSHandshakeTests.swift b/Tests/AetherEngineTests/EngineTLSHandshakeTests.swift index 4b99336b9..734b168e6 100644 --- a/Tests/AetherEngineTests/EngineTLSHandshakeTests.swift +++ b/Tests/AetherEngineTests/EngineTLSHandshakeTests.swift @@ -11,230 +11,245 @@ @testable import AetherEngine - /// Everything that sets `EngineTLS.serverTrustEvaluator` lives in this one - /// suite. The evaluator is process global and suites otherwise run in - /// parallel, so a second suite setting it would decide what this one is - /// testing. - /// Every request below goes through `HLSLocalServer`, which is why these clients carry no deadline -/// of their own worth reading: the hang catcher is the `.timeLimit` trait, and 150 s is only there -/// because a URL request must name something. The 15 s they used to carry reported the server as -/// dead twice on CI (`NSURLErrorTimedOut`) while it was merely waiting for a thread. -@Suite("EngineTLS live handshake against a self-signed origin", .serialized, .timeLimit(.minutes(3))) - struct EngineTLSHandshakeTests { - - /// Lives here rather than beside the resolver tests because reading the - /// evaluator is as much a claim on the process global as writing it, - /// and a suite that only reads still races the ones that write. - @Test("No evaluator is set by default") - func defaultsToNoEvaluator() { - #expect(EngineTLS.serverTrustEvaluator == nil) - } - - @Test("No evaluator: the handshake is refused and no request reaches the origin") - func refusedByDefault() async throws { - let server = try #require(await SelfSignedTLSOrigin()) - defer { server.stop() } - + /// Every live suite that sets the process-global `EngineTLS.serverTrustEvaluator` nests here. + /// `.serialized` applies to nested suites too, so no two of their tests overlap on that global. + /// `EngineTLSTests` passes its evaluator in and never writes the global. + @Suite(.serialized, .timeLimit(.minutes(3))) + enum LiveTrustEvaluatorTests { + /// Runs `body` with `evaluator` installed, then restores the previous answer. + static func withEvaluator( + _ evaluator: (@Sendable (URLProtectionSpace) -> Bool)?, + _ body: () async throws -> T + ) async rethrows -> T { let previous = EngineTLS.serverTrustEvaluator defer { EngineTLS.serverTrustEvaluator = previous } - EngineTLS.serverTrustEvaluator = nil - - let reader = AVIOReader( - url: URL(string: "https://127.0.0.1:\(server.port)/movie.bin")!, - chunkRequestTimeout: 5, chunkMaxRetries: 1) - defer { reader.markClosed(); reader.close() } - let refusal = Self.openFailure(of: reader) - - try await Task.sleep(for: .seconds(1)) - #expect(server.requestsServed == 0, - "a request crossed a handshake that system trust should have refused") - #expect(Self.isTrustRefusal(refusal), - "open failed as \(String(describing: refusal)), not a trust refusal") + EngineTLS.serverTrustEvaluator = evaluator + return try await body() } + } - @Test("Accepted for this origin: the same server serves the reader") - func acceptedWhenOptedIn() async throws { - let server = try #require(await SelfSignedTLSOrigin()) - defer { server.stop() } - - let previous = EngineTLS.serverTrustEvaluator - defer { EngineTLS.serverTrustEvaluator = previous } - EngineTLS.serverTrustEvaluator = { _ in true } - - let reader = AVIOReader( - url: URL(string: "https://127.0.0.1:\(server.port)/movie.bin")!, - chunkRequestTimeout: 10, chunkMaxRetries: 2) - defer { reader.markClosed(); reader.close() } - try reader.open() - - let sliceCap = 64 * 1024 - let buf = UnsafeMutablePointer.allocate(capacity: sliceCap) - defer { buf.deallocate() } - var got = 0 - let deadline = Date().addingTimeInterval(20) - while got < sliceCap && Date() < deadline { - let n = reader.read(into: buf, size: Int32(sliceCap - got)) - if n <= 0 { break } - got += Int(n) + extension LiveTrustEvaluatorTests { + /// Requests through `HLSLocalServer` carry no client deadline of their own worth reading: + /// the hang catcher is the parent's `.timeLimit` trait, and 150 s is only there because a + /// URL request must name something. The 15 s they used to carry reported the server as dead + /// twice on CI (`NSURLErrorTimedOut`) while it was merely waiting for a thread. + @Suite("EngineTLS live handshake against a self-signed origin") + struct EngineTLSHandshakeTests { + + /// Lives here rather than beside the resolver tests because reading the + /// evaluator is as much a claim on the process global as writing it, + /// and a suite that only reads still races the ones that write. + @Test("No evaluator is set by default") + func defaultsToNoEvaluator() { + #expect(EngineTLS.serverTrustEvaluator == nil) } - #expect(got == sliceCap, "delivered \(got) of \(sliceCap) bytes") - #expect(buf[0] == 0xA7) - #expect(server.requestsServed > 0) - } - - @Test("An evaluator that answers for another host leaves this one refused") - func refusedForAnOriginTheHostDidNotAccept() async throws { - let server = try #require(await SelfSignedTLSOrigin()) - defer { server.stop() } - - // The case a process-wide flag cannot express: a host holding a LAN - // address behind a private certificate and a WAN address with a real - // one, accepting the first without quietly relaxing the second. - let previous = EngineTLS.serverTrustEvaluator - defer { EngineTLS.serverTrustEvaluator = previous } - EngineTLS.serverTrustEvaluator = { $0.host == "media.example" } - - let reader = AVIOReader( - url: URL(string: "https://127.0.0.1:\(server.port)/movie.bin")!, - chunkRequestTimeout: 5, chunkMaxRetries: 1) - defer { reader.markClosed(); reader.close() } - let refusal = Self.openFailure(of: reader) - - try await Task.sleep(for: .seconds(1)) - #expect(server.requestsServed == 0, - "a request crossed a handshake the evaluator did not accept") - #expect(Self.isTrustRefusal(refusal), - "open failed as \(String(describing: refusal)), not a trust refusal") - } - - @Test("Through the relay: a client that never sees the certificate gets the stream") - func relayServesThroughUntrustedOrigin() async throws { - let origin = try #require(await SelfSignedHLSOrigin()) - defer { origin.stop() } + @Test("No evaluator: the handshake is refused and no request reaches the origin") + func refusedByDefault() async throws { + let server = try #require(await SelfSignedTLSOrigin()) + defer { server.stop() } + + let previous = EngineTLS.serverTrustEvaluator + defer { EngineTLS.serverTrustEvaluator = previous } + EngineTLS.serverTrustEvaluator = nil + + let reader = AVIOReader( + url: URL(string: "https://127.0.0.1:\(server.port)/movie.bin")!, + chunkRequestTimeout: 5, chunkMaxRetries: 1) + defer { reader.markClosed(); reader.close() } + let refusal = Self.openFailure(of: reader) + + try await Task.sleep(for: .seconds(1)) + #expect(server.requestsServed == 0, + "a request crossed a handshake that system trust should have refused") + #expect(Self.isTrustRefusal(refusal), + "open failed as \(String(describing: refusal)), not a trust refusal") + } - let previous = EngineTLS.serverTrustEvaluator - defer { EngineTLS.serverTrustEvaluator = previous } - EngineTLS.serverTrustEvaluator = { _ in true } - - let server = try Self.relayServer() - defer { server.stop(); server.relay?.stop() } - - let master = URL(string: "https://127.0.0.1:\(origin.port)/master.m3u8")! - let entry = try #require(server.relayURL(for: master)) - - let playlist = try await Self.text(of: entry) - #expect(playlist.contains("#EXT-X-STREAM-INF")) - let variant = try #require( - playlist.components(separatedBy: "\n").first { $0.hasPrefix("http://127.0.0.1:") }) - - let media = try await Self.text(of: try #require(URL(string: variant))) - #expect(media.contains("#EXTINF")) - let segmentLine = try #require( - media.components(separatedBy: "\n").first { - $0.hasPrefix("http://127.0.0.1:") && !$0.contains("m3u8") - }) - - var segmentRequest = URLRequest(url: try #require(URL(string: segmentLine))) - segmentRequest.timeoutInterval = 150 - let (bytes, response) = try await URLSession.shared.data(for: segmentRequest) - #expect((response as? HTTPURLResponse)?.statusCode == 200) - #expect(bytes.count == 4096, "served \(bytes.count) segment bytes") - #expect(bytes.first == 0x47, "not an MPEG-TS sync byte") - } + @Test("Accepted for this origin: the same server serves the reader") + func acceptedWhenOptedIn() async throws { + let server = try #require(await SelfSignedTLSOrigin()) + defer { server.stop() } - @Test("Through the relay: no evaluator refuses to launder an untrusted origin") - func relayRefusesWhenNotOptedIn() async throws { - let origin = try #require(await SelfSignedHLSOrigin()) - defer { origin.stop() } + let previous = EngineTLS.serverTrustEvaluator + defer { EngineTLS.serverTrustEvaluator = previous } + EngineTLS.serverTrustEvaluator = { _ in true } - let previous = EngineTLS.serverTrustEvaluator - defer { EngineTLS.serverTrustEvaluator = previous } - EngineTLS.serverTrustEvaluator = nil + let reader = AVIOReader( + url: URL(string: "https://127.0.0.1:\(server.port)/movie.bin")!, + chunkRequestTimeout: 10, chunkMaxRetries: 2) + defer { reader.markClosed(); reader.close() } + try reader.open() - let server = try Self.relayServer() - defer { server.stop(); server.relay?.stop() } + let sliceCap = 64 * 1024 + let buf = UnsafeMutablePointer.allocate(capacity: sliceCap) + defer { buf.deallocate() } + var got = 0 + let deadline = Date().addingTimeInterval(20) + while got < sliceCap && Date() < deadline { + let n = reader.read(into: buf, size: Int32(sliceCap - got)) + if n <= 0 { break } + got += Int(n) + } + #expect(got == sliceCap, "delivered \(got) of \(sliceCap) bytes") + #expect(buf[0] == 0xA7) + #expect(server.requestsServed > 0) + } - let master = URL(string: "https://127.0.0.1:\(origin.port)/master.m3u8")! - let entry = try #require(server.relayURL(for: master)) - var request = URLRequest(url: entry) - request.timeoutInterval = 150 - let (_, response) = try await URLSession.shared.data(for: request) - #expect((response as? HTTPURLResponse)?.statusCode == 502, - "the upstream handshake should have failed system trust") - } + @Test("An evaluator that answers for another host leaves this one refused") + func refusedForAnOriginTheHostDidNotAccept() async throws { + let server = try #require(await SelfSignedTLSOrigin()) + defer { server.stop() } + + // The case a process-wide flag cannot express: a host holding a LAN + // address behind a private certificate and a WAN address with a real + // one, accepting the first without quietly relaxing the second. + let previous = EngineTLS.serverTrustEvaluator + defer { EngineTLS.serverTrustEvaluator = previous } + EngineTLS.serverTrustEvaluator = { $0.host == "media.example" } + + let reader = AVIOReader( + url: URL(string: "https://127.0.0.1:\(server.port)/movie.bin")!, + chunkRequestTimeout: 5, chunkMaxRetries: 1) + defer { reader.markClosed(); reader.close() } + let refusal = Self.openFailure(of: reader) + + try await Task.sleep(for: .seconds(1)) + #expect(server.requestsServed == 0, + "a request crossed a handshake the evaluator did not accept") + #expect(Self.isTrustRefusal(refusal), + "open failed as \(String(describing: refusal)), not a trust refusal") + } - @Test("Through the relay: an origin the evaluator declines is not laundered either") - func relayRefusesAnOriginTheEvaluatorDeclines() async throws { - let origin = try #require(await SelfSignedHLSOrigin()) - defer { origin.stop() } + @Test("Through the relay: a client that never sees the certificate gets the stream") + func relayServesThroughUntrustedOrigin() async throws { + let origin = try #require(await SelfSignedHLSOrigin()) + defer { origin.stop() } + + let previous = EngineTLS.serverTrustEvaluator + defer { EngineTLS.serverTrustEvaluator = previous } + EngineTLS.serverTrustEvaluator = { _ in true } + + let server = try Self.relayServer() + defer { server.stop(); server.relay?.stop() } + + let master = URL(string: "https://127.0.0.1:\(origin.port)/master.m3u8")! + let entry = try #require(server.relayURL(for: master)) + + let playlist = try await Self.text(of: entry) + #expect(playlist.contains("#EXT-X-STREAM-INF")) + let variant = try #require( + playlist.components(separatedBy: "\n").first { $0.hasPrefix("http://127.0.0.1:") }) + + let media = try await Self.text(of: try #require(URL(string: variant))) + #expect(media.contains("#EXTINF")) + let segmentLine = try #require( + media.components(separatedBy: "\n").first { + $0.hasPrefix("http://127.0.0.1:") && !$0.contains("m3u8") + }) + + var segmentRequest = URLRequest(url: try #require(URL(string: segmentLine))) + segmentRequest.timeoutInterval = 150 + let (bytes, response) = try await URLSession.shared.data(for: segmentRequest) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + #expect(bytes.count == 4096, "served \(bytes.count) segment bytes") + #expect(bytes.first == 0x47, "not an MPEG-TS sync byte") + } - // The relay is mounted for every https origin once an evaluator - // exists, so the per-origin answer has to hold at the handshake it - // makes on the player's behalf. - let previous = EngineTLS.serverTrustEvaluator - defer { EngineTLS.serverTrustEvaluator = previous } - EngineTLS.serverTrustEvaluator = { $0.host == "media.example" } + @Test("Through the relay: no evaluator refuses to launder an untrusted origin") + func relayRefusesWhenNotOptedIn() async throws { + let origin = try #require(await SelfSignedHLSOrigin()) + defer { origin.stop() } - let server = try Self.relayServer() - defer { server.stop(); server.relay?.stop() } + let previous = EngineTLS.serverTrustEvaluator + defer { EngineTLS.serverTrustEvaluator = previous } + EngineTLS.serverTrustEvaluator = nil - let master = URL(string: "https://127.0.0.1:\(origin.port)/master.m3u8")! - let entry = try #require(server.relayURL(for: master)) + let server = try Self.relayServer() + defer { server.stop(); server.relay?.stop() } - var request = URLRequest(url: entry) - request.timeoutInterval = 150 - let (_, response) = try await URLSession.shared.data(for: request) - #expect((response as? HTTPURLResponse)?.statusCode == 502, - "an origin the evaluator declined was served anyway") - } + let master = URL(string: "https://127.0.0.1:\(origin.port)/master.m3u8")! + let entry = try #require(server.relayURL(for: master)) - @Test("A self-signed origin is what the relay is mounted for") - func trustProbeNamesTheSelfSignedOrigin() async throws { - let origin = try #require(await SelfSignedHLSOrigin()) - defer { origin.stop() } + var request = URLRequest(url: entry) + request.timeoutInterval = 150 + let (_, response) = try await URLSession.shared.data(for: request) + #expect((response as? HTTPURLResponse)?.statusCode == 502, + "the upstream handshake should have failed system trust") + } - // The probe asks the system, not the evaluator, so an answer already given here must not - // change what it reads: what is being measured is whether AVPlayer could reach the origin - // unaided, and AVPlayer never sees the evaluator. - let previous = EngineTLS.serverTrustEvaluator - defer { EngineTLS.serverTrustEvaluator = previous } - EngineTLS.serverTrustEvaluator = { _ in true } + @Test("Through the relay: an origin the evaluator declines is not laundered either") + func relayRefusesAnOriginTheEvaluatorDeclines() async throws { + let origin = try #require(await SelfSignedHLSOrigin()) + defer { origin.stop() } + + // The relay is mounted for every https origin once an evaluator + // exists, so the per-origin answer has to hold at the handshake it + // makes on the player's behalf. + let previous = EngineTLS.serverTrustEvaluator + defer { EngineTLS.serverTrustEvaluator = previous } + EngineTLS.serverTrustEvaluator = { $0.host == "media.example" } + + let server = try Self.relayServer() + defer { server.stop(); server.relay?.stop() } + + let master = URL(string: "https://127.0.0.1:\(origin.port)/master.m3u8")! + let entry = try #require(server.relayURL(for: master)) + + var request = URLRequest(url: entry) + request.timeoutInterval = 150 + let (_, response) = try await URLSession.shared.data(for: request) + #expect((response as? HTTPURLResponse)?.statusCode == 502, + "an origin the evaluator declined was served anyway") + } - let refused = await HLSOriginRelay.systemTrustRefuses( - URL(string: "https://127.0.0.1:\(origin.port)/master.m3u8")!) - #expect(refused, "the origin the relay exists for was read as one AVPlayer could reach") - } + @Test("A self-signed origin is what the relay is mounted for") + func trustProbeNamesTheSelfSignedOrigin() async throws { + let origin = try #require(await SelfSignedHLSOrigin()) + defer { origin.stop() } + + // The probe asks the system, not the evaluator, so an answer already given here must not + // change what it reads: what is being measured is whether AVPlayer could reach the origin + // unaided, and AVPlayer never sees the evaluator. + let previous = EngineTLS.serverTrustEvaluator + defer { EngineTLS.serverTrustEvaluator = previous } + EngineTLS.serverTrustEvaluator = { _ in true } + + let refused = await HLSOriginRelay.systemTrustRefuses( + URL(string: "https://127.0.0.1:\(origin.port)/master.m3u8")!) + #expect(refused, "the origin the relay exists for was read as one AVPlayer could reach") + } - private static func relayServer() throws -> HLSLocalServer { - let server = HLSLocalServer(relay: HLSOriginRelay()) - try server.start() - return server - } + private static func relayServer() throws -> HLSLocalServer { + let server = HLSLocalServer(relay: HLSOriginRelay()) + try server.start() + return server + } - private static func text(of url: URL) async throws -> String { - var request = URLRequest(url: url) - request.timeoutInterval = 150 - let (data, response) = try await URLSession.shared.data(for: request) - #expect((response as? HTTPURLResponse)?.statusCode == 200) - return String(decoding: data, as: UTF8.self) - } + private static func text(of url: URL) async throws -> String { + var request = URLRequest(url: url) + request.timeoutInterval = 150 + let (data, response) = try await URLSession.shared.data(for: request) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + return String(decoding: data, as: UTF8.self) + } - /// A refused handshake reaches the host as a typed failure rather than as unreadable media, - /// so the refusal arms assert the classification and not only that no byte was served. - private static func openFailure(of reader: AVIOReader) -> Error? { - do { - try reader.open() - return nil - } catch { - return error + /// A refused handshake reaches the host as a typed failure rather than as unreadable media, + /// so the refusal arms assert the classification and not only that no byte was served. + private static func openFailure(of reader: AVIOReader) -> Error? { + do { + try reader.open() + return nil + } catch { + return error + } } - } - private static func isTrustRefusal(_ error: Error?) -> Bool { - guard case .transportSecurityFailed = error as? AVIOReaderError else { return false } - return true + private static func isTrustRefusal(_ error: Error?) -> Bool { + guard case .transportSecurityFailed = error as? AVIOReaderError else { return false } + return true + } } } @@ -244,9 +259,9 @@ /// the refusal observable: a client that distrusts the certificate never /// gets a request line onto the wire. final class SelfSignedTLSOrigin { - let port: UInt16 - private let process: Process - private let workDir: URL + var port: UInt16 { launched.port } + private var workDir: URL { launched.workDir } + private let launched: PythonOrigin.Launched var requestsServed: Int { let log = workDir.appendingPathComponent("requests.log") @@ -259,15 +274,10 @@ prefix: "aether-tls-origin", script: Self.serverPy, files: ["cert.pem": Self.certPEM, "key.pem": Self.keyPEM]) else { return nil } - process = launched.process - port = launched.port - workDir = launched.workDir + self.launched = launched } - func stop() { - process.terminate() - try? FileManager.default.removeItem(at: workDir) - } + func stop() { launched.stop() } private static let serverPy = """ import http.server, os, re, ssl, sys diff --git a/Tests/AetherEngineTests/EngineTLSTests.swift b/Tests/AetherEngineTests/EngineTLSTests.swift index 4b1aa921f..a649dca8d 100644 --- a/Tests/AetherEngineTests/EngineTLSTests.swift +++ b/Tests/AetherEngineTests/EngineTLSTests.swift @@ -6,8 +6,9 @@ import Testing // Self-signed and private-CA media servers fail URLSession's system trust, // which the FFmpeg stack never enforced, so hosts need an explicit opt-in. // The resolver must stay on default handling for everything except a -// server-trust challenge the host answered for. -@Suite("EngineTLS trust resolution", .serialized) +// server-trust challenge the host answered for. The evaluator is passed in, so this suite never +// touches the process-global `EngineTLS.serverTrustEvaluator` the live suites set. +@Suite("EngineTLS trust resolution") struct EngineTLSTests { private final class RecordingSender: NSObject, URLAuthenticationChallengeSender { @@ -32,12 +33,8 @@ struct EngineTLSTests { method: String, host: String = "server.example" ) -> URLSession.AuthChallengeDisposition { - let previous = EngineTLS.serverTrustEvaluator - defer { EngineTLS.serverTrustEvaluator = previous } - EngineTLS.serverTrustEvaluator = evaluator - var got: URLSession.AuthChallengeDisposition? - EngineTLS.resolve(challenge(method: method, host: host)) { disposition, _ in + EngineTLS.resolve(challenge(method: method, host: host), evaluator: evaluator) { disposition, _ in got = disposition } return got ?? .performDefaultHandling diff --git a/Tests/AetherEngineTests/HLSOriginRelayTests.swift b/Tests/AetherEngineTests/HLSOriginRelayTests.swift index 6ebc9027e..f870d5479 100644 --- a/Tests/AetherEngineTests/HLSOriginRelayTests.swift +++ b/Tests/AetherEngineTests/HLSOriginRelayTests.swift @@ -1,6 +1,7 @@ // Addressing and rewriting, none of which asks the trust evaluator. The live // proof that a client which never sees the certificate still gets the stream -// lives with the other tests that set an evaluator, in EngineTLSHandshakeTests. +// lives with the other tests that set an evaluator, in EngineTLSHandshakeTests. Redirect +// authorization across an HTTPS-to-HTTP downgrade is in RedirectAuthorizationScopeTests. import Foundation import Testing @@ -384,9 +385,8 @@ struct HLSOriginRelayAddressingTests { static let sliceBytes = 64 * 1024 static func totalBytes(slices: Int) -> Int { sliceBytes * slices } - let port: UInt16 - private let process: Process - private let workDir: URL + var port: UInt16 { launched.port } + private let launched: PythonOrigin.Launched init?(slices: Int = 8, pauseSeconds: Double = 0.05, declaresLength: Bool = true) async { guard let launched = await PythonOrigin.launch( @@ -394,15 +394,10 @@ struct HLSOriginRelayAddressingTests { script: Self.serverPy(slices: slices, pauseSeconds: pauseSeconds, declaresLength: declaresLength)) else { return nil } - process = launched.process - port = launched.port - workDir = launched.workDir + self.launched = launched } - func stop() { - process.terminate() - try? FileManager.default.removeItem(at: workDir) - } + func stop() { launched.stop() } private static func serverPy(slices: Int, pauseSeconds: Double, declaresLength: Bool) -> String @@ -451,9 +446,9 @@ struct HLSOriginRelayAddressingTests { /// Loopback HTTP origin that answers Range requests exactly as asked and records the last /// one it saw, which is what makes "forwarded verbatim" observable rather than asserted. final class RangeEchoOrigin { - let port: UInt16 - private let process: Process - private let workDir: URL + var port: UInt16 { launched.port } + private var workDir: URL { launched.workDir } + private let launched: PythonOrigin.Launched var lastRange: String? { let log = workDir.appendingPathComponent("range.log") @@ -465,15 +460,10 @@ struct HLSOriginRelayAddressingTests { guard let launched = await PythonOrigin.launch( prefix: "aether-range-origin", script: Self.serverPy(status: status)) else { return nil } - process = launched.process - port = launched.port - workDir = launched.workDir + self.launched = launched } - func stop() { - process.terminate() - try? FileManager.default.removeItem(at: workDir) - } + func stop() { launched.stop() } private static func serverPy(status: Int) -> String { """ @@ -520,24 +510,18 @@ struct HLSOriginRelayAddressingTests { final class SelfSignedHLSOrigin { - let port: UInt16 - private let process: Process - private let workDir: URL + var port: UInt16 { launched.port } + private let launched: PythonOrigin.Launched init?() async { guard let launched = await PythonOrigin.launch( prefix: "aether-hls-origin", script: Self.serverPy, files: ["cert.pem": SelfSignedTLSOrigin.certPEM, "key.pem": SelfSignedTLSOrigin.keyPEM]) else { return nil } - process = launched.process - port = launched.port - workDir = launched.workDir + self.launched = launched } - func stop() { - process.terminate() - try? FileManager.default.removeItem(at: workDir) - } + func stop() { launched.stop() } private static let serverPy = """ import http.server, ssl diff --git a/Tests/AetherEngineTests/RedirectAuthorizationScopeTests.swift b/Tests/AetherEngineTests/RedirectAuthorizationScopeTests.swift new file mode 100644 index 000000000..e886cd8a5 --- /dev/null +++ b/Tests/AetherEngineTests/RedirectAuthorizationScopeTests.swift @@ -0,0 +1,198 @@ +// Real HTTPS-to-HTTP redirects through `HLSOriginRelay`, with synthetic credentials only. The TLS +// leg needs the self-signed loopback certificate accepted, which is a write to the process-global +// trust evaluator, so the suite nests under `LiveTrustEvaluatorTests`. `Process` is macOS-only. +#if os(macOS) + + import Foundation + import Testing + + @testable import AetherEngine + + extension LiveTrustEvaluatorTests { + /// PR #2 review: once a request chain has reached HTTPS, no credential header may cross a + /// later cleartext hop, whether the static headers or the provider supplied it. + @Suite("Redirect authorization scope across an HTTPS-to-HTTP downgrade") + struct RedirectAuthorizationScopeTests { + + private static let bearer = "Bearer synthetic-test-only" + + @Test("A provider's credentials are dropped from the HTTP hop; other headers still go") + func providerCredentialsDroppedOnDowngrade() async throws { + try await Self.withDowngradeOrigin { origin in + let asked = AskedURLs() + let provider = HTTPRequestAuthorization { url, _ in + asked.append(url) + return ["Authorization": Self.bearer, "X-Test-Client": "aether"] + } + let body = try await provider.data(from: origin.url, maximumBytes: 1024) + #expect(String(decoding: body, as: UTF8.self) == "redirect-body") + #expect(asked.schemes == ["https", "http"]) + let requests = try origin.requests() + #expect(requests.map(\.scheme) == ["https", "http"]) + #expect(requests.map(\.authorization) == [Self.bearer, nil]) + #expect(requests.map(\.client) == ["aether", "aether"]) + } + } + + @Test("A provider that throws for the HTTP destination stops the chain before it is sent") + func providerRefusalStopsDowngradedRequest() async throws { + try await Self.withDowngradeOrigin { origin in + let asked = AskedURLs() + let provider = HTTPRequestAuthorization { url, _ in + asked.append(url) + guard url.scheme == "https" else { throw URLError(.userAuthenticationRequired) } + return ["Authorization": Self.bearer] + } + await #expect(throws: URLError(.badServerResponse)) { + _ = try await provider.data(from: origin.url, maximumBytes: 1024) + } + // The provider was asked about the HTTP destination, so the HTTP server seeing + // nothing is its refusal and not an earlier failure. + #expect(asked.schemes == ["https", "http"]) + let requests = try origin.requests() + #expect(requests.map(\.scheme) == ["https"]) + #expect(requests.map(\.authorization) == [Self.bearer]) + } + } + + @Test("Static credentials are stripped from the HTTP redirect request") + func staticCredentialsStrippedOnDowngrade() async throws { + try await Self.withDowngradeOrigin { origin in + let relay = HLSOriginRelay() + defer { relay.stop() } + let (body, finalURL) = try await relay.fetchPlaylist( + origin.url, headers: ["Authorization": Self.bearer]) + #expect(body == "redirect-body") + #expect(finalURL.scheme == "http") + let requests = try origin.requests() + #expect(requests.map(\.scheme) == ["https", "http"]) + #expect(requests.map(\.authorization) == [Self.bearer, nil]) + } + } + + @Test("A provider can explicitly allow an anonymous HTTP redirect destination") + func providerAllowsAnonymousDowngradedDestination() async throws { + try await Self.withDowngradeOrigin { origin in + let provider = HTTPRequestAuthorization { url, _ in + url.scheme == "https" ? ["Authorization": Self.bearer] : [:] + } + let body = try await provider.data(from: origin.url, maximumBytes: 1024) + #expect(String(decoding: body, as: UTF8.self) == "redirect-body") + let requests = try origin.requests() + #expect(requests.map(\.scheme) == ["https", "http"]) + #expect(requests.map(\.authorization) == [Self.bearer, nil], + "the redirect must use the new provider result, not replay the old bearer") + } + } + + @Test("An HTTP origin the host chose still receives the provider's credentials") + func providerCredentialsReachAConfiguredHTTPOrigin() async throws { + try await Self.withDowngradeOrigin { origin in + let provider = HTTPRequestAuthorization { _, _ in ["Authorization": Self.bearer] } + let body = try await provider.data(from: origin.plainURL, maximumBytes: 1024) + #expect(String(decoding: body, as: UTF8.self) == "redirect-body") + let requests = try origin.requests() + #expect(requests.map(\.scheme) == ["http"]) + #expect(requests.map(\.authorization) == [Self.bearer]) + } + } + + private static func withDowngradeOrigin( + _ body: (TLSDowngradeOrigin) async throws -> Void + ) async throws { + let origin = try await TLSDowngradeOrigin() + defer { origin.stop() } + try await LiveTrustEvaluatorTests.withEvaluator({ $0.host == "127.0.0.1" }) { + try await body(origin) + } + } + + /// The resolver is `@Sendable` and runs off the test's task. + private final class AskedURLs: @unchecked Sendable { + private let lock = NSLock() + private var urls: [URL] = [] + + func append(_ url: URL) { lock.withLock { urls.append(url) } } + var schemes: [String?] { lock.withLock { urls.map(\.scheme) } } + } + } + } + + /// Paired loopback origins record headers before responding, so awaiting the transfer also + /// makes the request log observable without a sleep. No traffic leaves this machine. + private final class TLSDowngradeOrigin { + struct Request: Decodable { + let scheme: String + let authorization: String? + let client: String? + } + + /// The HTTPS origin, which redirects to `plainURL`. + let url: URL + /// The HTTP origin, which answers with the body. + let plainURL: URL + private let launched: PythonOrigin.Launched + + /// Every request either origin received, in order. Empty when none arrived. + func requests() throws -> [Request] { + let log = launched.workDir.appendingPathComponent("requests.jsonl") + guard FileManager.default.fileExists(atPath: log.path) else { return [] } + return try String(decoding: Data(contentsOf: log), as: UTF8.self).split(separator: "\n").map { + try JSONDecoder().decode(Request.self, from: Data($0.utf8)) + } + } + + init() async throws { + launched = try #require(await PythonOrigin.launch( + prefix: "aether-tls-downgrade", script: Self.serverPy, + files: ["cert.pem": SelfSignedTLSOrigin.certPEM, + "key.pem": SelfSignedTLSOrigin.keyPEM])) + let plainPort = try String( + contentsOf: launched.workDir.appendingPathComponent("plain.port"), encoding: .utf8) + url = try #require(URL(string: "https://127.0.0.1:\(launched.port)/start")) + plainURL = try #require(URL(string: "http://127.0.0.1:\(plainPort)/end")) + } + + func stop() { launched.stop() } + + private static let serverPy = """ + import http.server, json, ssl, threading + + lock = threading.Lock() + + class Handler(http.server.BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + def log_message(self, *args): pass + def do_GET(self): + secure = isinstance(self.connection, ssl.SSLSocket) + with lock: + with open("requests.jsonl", "a") as f: + f.write(json.dumps({"scheme": "https" if secure else "http", + "authorization": self.headers.get("Authorization"), + "client": self.headers.get("X-Test-Client")}) + "\\n") + if secure: + self.send_response(302) + self.send_header("Location", f"http://127.0.0.1:{plain.server_address[1]}/end") + self.send_header("Content-Length", "0") + self.end_headers() + else: + body = b"redirect-body" + self.send_response(200) + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + plain = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler) + threading.Thread(target=plain.serve_forever, daemon=True).start() + with open("plain.port", "w") as f: + f.write(str(plain.server_address[1])) + secure = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler) + context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + context.load_cert_chain("cert.pem", "key.pem") + secure.socket = context.wrap_socket(secure.socket, server_side=True) + print("READY", secure.server_address[1], flush=True) + secure.serve_forever() + """ + } + +#endif diff --git a/Tests/AetherEngineTests/Support/PythonOrigin.swift b/Tests/AetherEngineTests/Support/PythonOrigin.swift index f91dd2144..e147c6184 100644 --- a/Tests/AetherEngineTests/Support/PythonOrigin.swift +++ b/Tests/AetherEngineTests/Support/PythonOrigin.swift @@ -32,6 +32,11 @@ enum PythonOrigin { let process: Process let port: UInt16 let workDir: URL + + func stop() { + process.terminate() + try? FileManager.default.removeItem(at: workDir) + } } /// Writes `files` and `script` into a scratch directory, runs the script with the system diff --git a/docs/api.md b/docs/api.md index 3e54cc297..4d04f8d47 100644 --- a/docs/api.md +++ b/docs/api.md @@ -148,6 +148,24 @@ valid; wait for refresh when a credential has expired or was rejected. Never pla URLs. The engine owns Range, routing and HTTP framing headers. Authorization waits are bounded; stopping the load cancels pending work and ignores late resolver results. +**Redirect credential scope includes the scheme.** Validate the destination's scheme, host and +effective port before obtaining credentials, as well as any session/path restrictions the host +requires. Checking only the hostname allows an HTTPS-to-HTTP downgrade on the same host. An +explicitly configured HTTP server is a separate host policy decision; it does not authorize an +HTTPS session to downgrade. + +Each redirect asks the resolver for a new set of headers. The engine does not replay the previous +provider result. Throw to reject an out-of-scope destination before its request is sent; refusal +does not fall back to static headers. Returning `[:]` instead permits a request without application +headers, so use it only when anonymous access to that destination is intended. + +Once a request chain has reached HTTPS, the engine drops credential headers (`Authorization`, +`Proxy-Authorization`, `Cookie` and the Emby/Jellyfin token headers) from every later HTTP hop, +whether they came from static headers or from the resolver. Other headers are still sent, and the +hop itself still runs, so an anonymous HTTP redirect target keeps working. This is a backstop, not a +substitute for scope checks: credentials in a custom header are not recognized, and a chain that +starts on HTTP sends whatever the resolver returns. + `LoadOptions.httpRequestAuthorization` covers native HLS media and its master/variant playlist preparation. Direct media AVIO, live ingest and audio taps retain static headers.