explore: Demonstrate and test retrieval through Kubo by Peeja · Pull Request #304 · storacha/guppy

Peeja · 2026-01-22T17:13:09Z

This doesn't necessarily all need to be merged, although it probably wouldn't hurt to. We do need the changes to the gateway server there to make this possible.

A little breakdown here:

We need to implement the Delegated Routing spec (or at least the Content Routing API) to make the connection. We point Kubo at the routing endpoint, and that points all requests to the Gupway gateway.
The gateway needs to serve all of its content over TLS, or else Kubo will ignore it. It doesn't say anything, it just treats non-TLS HTTP as a protocol it doesn't know and skips it. So, we need to construct a phony cert for localhost and tell Kubo not to verify it. That's safe, because we know we control both ends of the conversation, on the same box.
Unfortunately, that setting not to verify TLS doesn't apply to the routing requests. They'll fail with a made up certificate. Fortunately, the routing system is happy to make its requests over plain HTTP if that's where you point it. So, annoyingly, we have to run two separate servers in the gateway process: one for the gateway proper using TLS, and one for the routing API without TLS. The latter is disabled unless you explicitly set a port for it to run on.

alanshaw

Love this ❤️

We should defintely remove the TLS stuff before merging.

alanshaw · 2026-01-22T17:39:59Z

cmd/gateway/serve.go

+			r.HidePort = true
+			r.Use(requestLogger(log))
+			r.Use(middleware.Recover())
+			r.GET("/routing/v1/providers/*", func(c echo.Context) error {


Why not just add this route to the existing echo server? Oh for the TLS hoop jump?

Because we need to run one with TLS and one without.

alanshaw · 2026-01-22T17:40:39Z

cmd/gateway/serve.go

 	cobra.CheckErr(viper.BindPFlag("gateway.port", serveCmd.Flags().Lookup("port")))

+	serveCmd.Flags().Int("routing-port", routingPort, "Port for delegated routing server (HTTP, no TLS). Set to 0 to disable.")
+	cobra.CheckErr(viper.BindPFlag("gateway.routing-port", serveCmd.Flags().Lookup("routing-port")))


No dash allowed in TOML keys (like JS).

Suggested change

cobra.CheckErr(viper.BindPFlag("gateway.routing-port", serveCmd.Flags().Lookup("routing-port")))

cobra.CheckErr(viper.BindPFlag("gateway.routing_port", serveCmd.Flags().Lookup("routing-port")))

alanshaw · 2026-01-22T17:41:00Z

cmd/gateway/serve.go

 	cobra.CheckErr(viper.BindPFlag("gateway.log_level", serveCmd.Flags().Lookup("log-level")))

+	serveCmd.Flags().String("tls-cert", "", "Path to TLS certificate file (enables HTTPS)")
+	cobra.CheckErr(viper.BindPFlag("gateway.tls-cert", serveCmd.Flags().Lookup("tls-cert")))


Suggested change

cobra.CheckErr(viper.BindPFlag("gateway.tls-cert", serveCmd.Flags().Lookup("tls-cert")))

cobra.CheckErr(viper.BindPFlag("gateway.tls_cert", serveCmd.Flags().Lookup("tls-cert")))

alanshaw · 2026-01-22T17:41:08Z

cmd/gateway/serve.go

+	cobra.CheckErr(viper.BindPFlag("gateway.tls-cert", serveCmd.Flags().Lookup("tls-cert")))
+
+	serveCmd.Flags().String("tls-key", "", "Path to TLS key file (enables HTTPS)")
+	cobra.CheckErr(viper.BindPFlag("gateway.tls-key", serveCmd.Flags().Lookup("tls-key")))


Suggested change

cobra.CheckErr(viper.BindPFlag("gateway.tls-key", serveCmd.Flags().Lookup("tls-key")))

cobra.CheckErr(viper.BindPFlag("gateway.tls_key", serveCmd.Flags().Lookup("tls-key")))

alanshaw · 2026-01-22T17:41:29Z

pkg/config/gateway.go

 	// Port is the port to run the gateway on.
 	Port int `mapstructure:"port" flag:"port" toml:"port"`
+	// TlsCert is the path to the TLS certificate file. If empty, TLS is disabled.
+	TlsCert string `mapstructure:"tls-cert" flag:"tls-cert" toml:"tls-cert"`


Suggested change

TlsCert string `mapstructure:"tls-cert" flag:"tls-cert" toml:"tls-cert"`

TlsCert string `mapstructure:"tls_cert" flag:"tls-cert" toml:"tls_cert"`

alanshaw · 2026-01-22T17:41:38Z

pkg/config/gateway.go

+	// TlsCert is the path to the TLS certificate file. If empty, TLS is disabled.
+	TlsCert string `mapstructure:"tls-cert" flag:"tls-cert" toml:"tls-cert"`
+	// TlsKey is the path to the TLS key file. If empty, TLS is disabled.
+	TlsKey string `mapstructure:"tls-key" flag:"tls-key" toml:"tls-key"`


Suggested change

TlsKey string `mapstructure:"tls-key" flag:"tls-key" toml:"tls-key"`

TlsKey string `mapstructure:"tls_key" flag:"tls-key" toml:"tls_key"`

alanshaw · 2026-01-22T17:41:47Z

pkg/config/gateway.go

+	TlsKey string `mapstructure:"tls-key" flag:"tls-key" toml:"tls-key"`
+	// RoutingPort is the port to use for delegated routing requests. Use 0 to
+	// disable delegated routing.
+	RoutingPort int `mapstructure:"routing-port" flag:"routing-port" toml:"routing-port"`


Suggested change

RoutingPort int `mapstructure:"routing-port" flag:"routing-port" toml:"routing-port"`

RoutingPort int `mapstructure:"routing_port" flag:"routing-port" toml:"routing_port"`

Peeja · 2026-01-22T19:13:53Z

We should defintely remove the TLS stuff before merging.

Not sure what you mean. How can it work without the TLS stuff?

alanshaw · 2026-01-23T10:09:12Z

We should defintely remove the TLS stuff before merging.

Not sure what you mean. How can it work without the TLS stuff?

In my experience TLS termination is almost always done with a reverse proxy. i.e. run an nginx to terminate the SSL and have it proxy to the service wherever it is running.

This is why I queried about the 2 echo servers and I assumed it was for local testing.

Personally I would add the route to the existing echo server and caveat that for delegated routing to work the gateway needs to be deployed with an SSL cert.

alanshaw · 2026-01-23T10:19:57Z

cmd/gateway/serve.go

+							"Schema": "peer",
+							"Protocols": ["transport-ipfs-gateway-http"],
+							"ID": "k51qzi5uqu5dj26lryc36mobgexftham120h0nu7o4ig6xu56y4h8wdvc4le6t",
+							"Addrs": ["/dns4/localhost/tcp/%d/tls/http"]


For this you'll need to use X-Forwarded-Host as /dns/${X-Forwarded-Host}/tls/http otherwise similar to what you're already doing except don't specify TLS - /dns/localhost/tcp/%d/http

Wait, is that why requests weren't working without TLS - because you had /tls in the multiaddr here?

Well, sort of. It's because the /tls has to be there, or it won't recognize it as an address type it can use. It only looks for /tls/http addresses. If there's a plain /http, it'll ignore it.

Peeja · 2026-01-27T17:15:48Z

In my experience TLS termination is almost always done with a reverse proxy. i.e. run an nginx to terminate the SSL and have it proxy to the service wherever it is running.

Oh, sure. That's what I'd expect for a "real" server. Here, the intention is to use it purely as a local connection between your local guppy and a local Kubo. This isn't a setup for running a Bitswap service over Guppy (which would be what you're describing, which is not far from this), this is a setup for retrieving directly from Forge to your own machine, but with incremental. resumable retrieval. This is the setup where you have UCAN auth to retrieve, and no one is running a gateway and paying for traffic running through it. Really, it's a hack until we have true resumable retrieval inside Guppy itself.

I think we want to support both, so since this is a hacky setup already, maybe we should require a wrapping TLS server like nginx anyhow. Nginx isn't hard to run, and we can ship a script that sets everything up for you as long as you have Kubo and nginx installed.

feat: Demonstrate and test retrieval through Kubo

b420935

Peeja requested review from alanshaw, hannahhoward and volmedo as code owners January 22, 2026 17:13

alanshaw reviewed Jan 22, 2026

View reviewed changes

alanshaw reviewed Jan 23, 2026

View reviewed changes

Peeja mentioned this pull request Jan 27, 2026

Guppy can offer delegated routing to Kubo to retrieve #313

Closed

	cobra.CheckErr(viper.BindPFlag("gateway.routing-port", serveCmd.Flags().Lookup("routing-port")))
	cobra.CheckErr(viper.BindPFlag("gateway.routing_port", serveCmd.Flags().Lookup("routing-port")))

	TlsCert string `mapstructure:"tls-cert" flag:"tls-cert" toml:"tls-cert"`
	TlsCert string `mapstructure:"tls_cert" flag:"tls-cert" toml:"tls_cert"`

	TlsKey string `mapstructure:"tls-key" flag:"tls-key" toml:"tls-key"`
	TlsKey string `mapstructure:"tls_key" flag:"tls-key" toml:"tls_key"`

	RoutingPort int `mapstructure:"routing-port" flag:"routing-port" toml:"routing-port"`
	RoutingPort int `mapstructure:"routing_port" flag:"routing-port" toml:"routing_port"`

Conversation

Peeja commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alanshaw left a comment

Choose a reason for hiding this comment

Uh oh!

alanshaw Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Peeja commented Jan 22, 2026

Uh oh!

alanshaw commented Jan 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Peeja commented Jan 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Peeja commented Jan 22, 2026 •

edited

Loading

alanshaw Jan 22, 2026 •

edited

Loading