Skip to content

Document the egress traffic supported for GA #1749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
haiyanmeng wants to merge 1 commit into
base: main
Choose a base branch
from
+35 −0
Open

Document the egress traffic supported for GA #1749

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ We provide several sample applications demonstrating Agent Substrate's capabilit
* [Integration Repositories](docs/integration-repos.md): Where integrations live, how their repositories are named, and how fixes flow back to core.
* [Observability Guide](docs/observability.md): Guide to actor logging, metrics, and distributed tracing.
* [Authentication Guide](docs/authentication.md): Configure trusted JWT providers and human credentials.
* [Egress Traffic](docs/egress-traffic.md): Which protocols an Actor may reach the outside world with and which are blocked.
* [Enabling man-in-the-middle (MITM) interception for Actor Egress policy](docs/egress-trust-bundle.md): Egress policies such as header injection depend on MITM interception of Actor traffic. This guide explains how an Actor should be configured to enable interception.
* [Request Parking](docs/request-parking.md): How the router parks requests through transient worker-pool saturation.
* [Rolling Upgrade Runbook](docs/upgrade.md): Upgrade a running substrate node by node without losing actor state.
Expand Down
34 changes: 34 additions & 0 deletions docs/egress-traffic.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
This document specifies the supported egress traffic for [GA](https://github.com/agent-substrate/substrate/milestone/3).

Actor TCP egress (except DNS traffic on port 53) is redirected to atunnel,
which opens a CONNECT tunnel to the egress gateway. Egress gateway applies policy.
Comment on lines +3 to +4

@LiorLieberman Lior Lieberman (LiorLieberman) Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with #1689 all will be redirected to atunnel, maybe we removed that?

@haiyanmeng haiyanmeng Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1689 does not redirect DNS-over-TCP traffic to atunnel.


DNS-over-TCP, UDP and other traffic is filtered by nftables and never reaches the gateway.

@LiorLieberman Lior Lieberman (LiorLieberman) Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are also going to atunnel with #1689. they never reach the gateway though and are exempt from egresspolicy

@haiyanmeng haiyanmeng Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1689 only route TCP (except DNS-over-TCP) to atunnel.


## TCP

| Port | Traffic | Behavior | Path | What the actor sees when refused |
| :---- | :---- | :---- | :---- | :---- |
| any | HTTP(S) 1.1 / 2 | Supported with policy controls | atunnel -> egress gateway -> origin | `403 Forbidden` |
| any | WebSocket | Blocked | n/a | `403 Forbidden` |
| any | Standard HTTP(S) CONNECT (forward-proxy tunnel) | Blocked | n/a | `403 Forbidden` |
| 53 | DNS | Allowed via netfilter rules | nftables -> node-configured DNS | n/a |

@LiorLieberman Lior Lieberman (LiorLieberman) Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should reflect this after #1689 is merged

@haiyanmeng haiyanmeng Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this line aligns with #1689 currently

| any | Any other TCP | Blocked | n/a | The connection is accepted and then closed with no bytes returned. There is no status code. atunnel logs the failure. |

@bowei Bowei Du (bowei) Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should send an RST by setting SO_LINGER to 0 on close vs having a normal FIN close.

@haiyanmeng haiyanmeng Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keith Mattix II (@keithmattix) , yanavlasov, WDYT?

@yanavlasov yanavlasov Sep 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit complicated. Actor's connection is inside the CONNECT tunnel. Egress Gateway can close the CONNECT abortively. For H/2 it is closed with RTS_STREAM and for H/1 TCP connection is closed abortively with RST. However the iptables interceptor that initiated CONNECT needs to translate these events into abortive close for the TCP connection from actor. I think Envoy already has the functionality it needs, but I'm not 100% sure. The atunnel client will need to have support for this as well.

We could work toward making this be the behavior for GA.


## UDP


| Port | Traffic | Behavior | Path | What the actor sees when refused |
| :---- | :---- | :---- | :---- | :---- |
| 53 | DNS | Allowed via netfilter rules | nftables -> node-configured DNS | n/a |
| any other | Any other UDP | Blocked | n/a | Packets are dropped, not rejected: no ICMP port-unreachable is sent, so the client hangs until its own timeout. |
Comment on lines +23 to +24

@LiorLieberman Lior Lieberman (LiorLieberman) Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should update this to reflect the state after #1689
cc: Benjamin Elder (@BenTheElder)

@haiyanmeng haiyanmeng Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this line aligns with #1689 currently


## Other transports

Everything that is neither TCP nor UDP is blocked. Packets are dropped, not rejected: no ICMP port-unreachable is sent, so the client hangs until its own timeout.

@LiorLieberman Lior Lieberman (LiorLieberman) Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

either we say, future version can look into ICMP rejection messages for verbosity or we drop ICMP from here


## Requesting support

If you would like Substrate to support egress traffic that is blocked above, please
[file an issue](https://github.com/agent-substrate/substrate/issues/new)
describing your use case.
Loading