Testers Needed: Multi-Site Support

[tvancott42]

I'm going to have Multi-Site support ready for testing very soon. Please reply to this thread if you're interested in testing it on your personal sites or want to trial it for your MSP.

I'll follow up with details on the infrastructure needed, but basically there's an agent service that gets deployed per-site on an arm64/amd64 box of your choice that also serves as an optional proxy to local UniFi Network Console access and Device SSH, along with proxying ONT / CM / cellular modem monitoring traffic, so you do not need central VPN tunneling to your sites from the Network Optimizer Server.

The per-site agent of course enables Network performance monitoring LAN and WAN via SNMP and the configurable latency / loss targets, along with the ISP Health and all other Monitoring features. It also serves as a local speed test server and initiator for that site.

To test refer to the latest v2.0.0-beta release notes: https://github.com/Ozark-Connect/NetworkOptimizer/releases/

Send me an email for a trial license if you want to test on more than 3 sites. tj@ozarkconnect.net

[Liosnel]

I only have one site, so..... Unless if you have a scenario where I still can help, not sure how I can... I Could spin up a second controller in a VM and put it on a different subnet, if that helps?

[tvancott42]

Much appreciated! But I have some basic multi-site testing going across a few sites I manage currently.

I'd love to have people test the multi-site branch on their single sites though, thoroughly, every feature if possible, to report back and ensure I didn't regress the majority use case of single site.

Are you set up to build from source or can you? I will let people know here when that branch is ready for testing.

When you do go to test, make sure to take a full export backup first though. Just in case. It's rolling forward cleanly for me, but just want to make sure I don't hose anybody while this is still in beta.

[tvancott42]

@Liosnel https://github.com/Ozark-Connect/NetworkOptimizer/releases/tag/v2.0.0-beta.2

[Liosnel]

Just upgraded to beta 2!
Anything you want tested specifically? Any need for me to build from another source?

[tvancott42]

Just upgraded to beta 2! Anything you want tested specifically? Any need for me to build from another source?

Nice, that's great! Just stay on the latest beta releases and report back to me if you notice any regressions from a single-site perspective.

What platform / arch are you deployed on?

If you want to stay on the bleeding edge, you can build from release/2.0-multi-site but you may catch some in-flight enhancements and fixes. I plan to cut v2.0 beta releases once or twice a day as I find things.

Thank you, I really appreciate it.

[Liosnel]

Nice, that's great! Just stay on the latest beta releases and report back to me if you notice any regressions from a single-site perspective.

Will do!!

What platform / arch are you deployed on?

Unraid Unleashed, via the included (basic) Docker implementation. Plan on rebuilding in a Compose stack soon. Maybe tonight...

If you want to stay on the bleeding edge, you can build from release/2.0-multi-site but you may catch some in-flight enhancements and fixes. I plan to cut v2.0 beta releases once or twice a day as I find things.

I'm fine with the daily updates, hit me up if you need anything specifically!

Thank you, I really appreciate it.

And I like tinkering and your software!!!

[suburbazine]

I'm interested. I'd actually sent you an email a couple of weeks ago inquiring as to licensing and this was one of the features requested. I have a lot of sites I'd like to monitor.

[tvancott42]

I'm interested. I'd actually sent you an email a couple of weeks ago inquiring as to licensing and this was one of the features requested. I have a lot of sites I'd like to monitor.

Hey there! Can you double-check your spam folder? I followed up on a few inquiries a couple weeks ago, but I think unfortunately my self-hosted biz email has a poor sender score.

tj@ozarkconnect.net just to hit me up again so I can reach back out via my personal email in case I'm getting spam'd. Thanks!

edit: also my Discord @tjvc42 if you want to add me on there to DM me

[suburbazine]

It didn't make it to me or my mail server. I'll reach out on my personal.

[tvancott42]

It didn't make it to me or my mail server. I'll reach out on my personal.

@suburbazine sent you a reply last night, this time from my gmail. Your original email indeed never hit my inbox :( just found it in my spam box.

I'm very flexible on how this should look for the early-adopter MSPs, so let me know what would work for you.

[tvancott42]

Removing a test site

There's no delete-site button yet, so if you're cycling test sites up and down, here's the quick manual cleanup. Use the Site ID shown in the app.

First, list your sites to grab the Site ID (the Slug column):

sqlite3 -header -column <data>/network_optimizer.db "SELECT Slug, Name, IsDefault FROM Sites ORDER BY SortOrder;"

Then delete that site's registry row:

sqlite3 <data>/network_optimizer.db "DELETE FROM Sites WHERE Slug='your-site-id' AND IsDefault=0;"

Finally, restart Network Optimizer so it drops the site from memory.

That's all you need. Everything below is totally optional cleanup of leftover data that's otherwise just orphaned and ignored:

• Delete the site's database folder: rm -rf <data>/sites/your-site-id
• If you use InfluxDB, delete its two buckets: your-site-id-<primary> and your-site-id-<longterm> (the Site-ID-prefixed versions of your bucket names)

<data> is your data directory: /app/data for Docker (run the sqlite command via docker exec, and the folder lives under your mapped data volume), or ~/Library/Application Support/NetworkOptimizer on Mac native. The default site can't be removed this way since its data lives in the main database rather than a per-site folder.

[tvancott42]

Please reach out for a trial license key if you want to test on more than 3 sites. Just dropped licensing support in beta.8

tj@ozarkconnect.net

[tvancott42]

If anybody wants to try out the on-gateway agent capabilities, with even speed testing being available on UDM-Beast, EFG, and EF-Core gateways, let me know. I'm testing it out right now on a UCG-Fiber and the standard dotnet self-contained agent binary is working great with only 3% added CPU burden and only ~50 MB extra RAM usage. I should have it ready w/ a beta release probably next week.

And, also, if anybody has Raspberry-Pi-grade hardware that wants to try out the agent on, please let me know and I can give you a trial license for > 3 sites if you need it.

[tvancott42]

On-gateway agent on UDR7 also validated, plenty of headroom so long as the lower-class gateways aren't also running Protect. And even if they are, there's just enough with Ubiquiti's recent optimization of Suricata (in EA). CPU usage goes up single-digits on both the UCG-Fiber and UDR7 from my testing.

Sites that want local speed testing (very useful of course) need a full agent still since I don't want to open that can-o-worms yet of hosting that on-gateway (but will be working on it for UDM-Beast, EFG, EF-Core, UXG-Enterprise class gateways as they can totally host iperf3 and the nginx-based web speed test server).

WAN speed testing via the gateway still works great w/ the Go binary alongside the Agent.

[suburbazine]

I'm unable to get any of the gateway agents to "tunnel home". They show as "online" in the multi-site manager but once I go to "Connect" they say the agents are waiting to connect. I've deleted/reinstalled the agents many times and they can definitely see the host Network Monitor server and it can technically see them. Just missing a functional link.

Network Monitor running in Debian 13 VPS with FQDN, so far a CGF and a UDM-Pro for agents.

[tvancott42]

Hmm definitely something small in infrastructure.

The agent will refuse to connect to an http endpoint. Has to be TLS / HTTPS. First thing to verify.

The forwarding infrastructure may also not be complete on the Network Optimizer side in your reverse proxy. The latest docs and examples explain what is needed there to map the gRPC endpoints. See here if you're running Traefik: https://github.com/Ozark-Connect/NetworkOptimizer-Proxy#multi-site-on-site-agents-optional

Do you want to hit me up on discord if that doesn't solve it I'll totally walk you through it? @tjvc42 on there

[suburbazine]

Discord would be good. Sent a request.

[noumenon272]

I’ve been trying to get this to work and have been unsuccessful at getting the agent to connect. I’ve tried setting up both an agent remotely and on the same LAN without success. looking through the logs on the server side, it says listening on 8042, but I don’t see any mention of it listening on 443? I am using macvlan networking in my docker setup on my server, so perhaps that is part of the problem?

[suburbazine]

Are you able to connect with just an API key and the local address?

[noumenon272]

I was able to connect to the remote gateway (I ran the agent on the gateway), the console connected, but the agent never would. it just said waiting for agent. I got wifi metrics, but ssh, etc wouldn’t work.

[suburbazine]

You are setting an https domain name for the agent return path, correct? With valid TLS?

[noumenon272]

yes. i’m using a ddns domain that points to my IP. i have 443 port forwarding turned on. this is also why i tested it on my local network to make sure it wasn’t a firewall issue and I can’t connect that way either. I also tried just using the IP address, but no luck. the agent log shows that it fails on the 443 connection.

[suburbazine]

Are you sure your ISP is allowing 443 inbound if you're hosting locally? Some ISPs reserve service hosting for business accounts.

[noumenon272]

no, i’m not sure. i have AT&T fiber, they’ve never blocked anything else.
all that being said, I can’t connect over my local network with a local IP either.

[tvancott42]

AT&T fiber doesn't tend to block HTTPS. I have sites I manage that self host stuff with no issues.When you say you can't connect with a local IP locally what do you mean exactly?Is this an additional site you're setting up to monitor or the main site with a separate agent. I'll be honest the latter case I still haven't tested but will be sorting that out soon. Additional sites connect in seconds when the HTTP connectivity is available.What does your port mapping set up look like, does it have pre existing IP list restrictions to Cloudflare or anything like that?You can totally push this through Cloudflare Tunnel if you need to. I have it all set up to be fine with that infrastructure.On Aug 2, 2026, at 6:19 AM, noumenon272 ***@***.***> wrote: no, i’m not sure. i have AT&T fiber, they’ve never blocked anything else. all that being said, I can’t connect over my local network with a local IP either. —Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today! You are receiving this because you authored the thread.Message ID: ***@***.***>

[noumenon272]

I have a NUC that I am planning to deploy at one of my remote locations. I put the agent on it, just to see if I could get it to connect over my LAN. I do not currently have it associated with a gateway. This is from the agent logs:

Enrolling with central server...
Unhandled exception. System.Net.Http.HttpRequestException: Connection refused (192.168.60.30:443)
 ---> System.Net.Sockets.SocketException (111): Connection refused
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---

this is the startup from the server side:

Binding to all interfaces (*:8042, IPv4 + IPv6)
[13:13:29 INF] Applying database migrations...
[13:13:29 INF] Database migrations complete
[13:13:32 INF] Applied migrations to 2 site database(s)
[13:13:32 INF] Database journal mode: WAL (filesystem: ext4)
[13:13:32 INF] External speed test servers configured: 1 (https://192.168.1.20:3005)
[13:13:32 WRN] GeoLite2-City.mmdb not found at /app/data/GeoLite2-City.mmdb. Geo enrichment will be unavailable. Download from https://dev.maxmind.com/geoip/geolite2-free-geolocation-data
[13:13:32 WRN] GeoLite2-ASN.mmdb not found at /app/data/GeoLite2-ASN.mmdb. ASN enrichment will be unavailable
[13:13:33 INF] Loaded 39859 OUI entries from cache
[13:13:33 INF] Admin authentication enabled using database-stored password
[13:13:34 INF] iperf3 server mode is disabled. Enable via Iperf3Server:Enabled=true
[13:13:34 INF] Digest service started
[13:13:34 INF] Alert processing service started
[13:13:34 INF] ScheduleService started
[13:13:34 INF] Threat collection service starting
[13:13:34 WRN] The query uses the 'First'/'FirstOrDefault' operator without 'OrderBy' and filter operators. This may lead to unpredictable results.
[13:13:34 INF] Started WAN data usage tracking for site cabin
[13:13:34 INF] Loaded saved UniFi configuration for https://192.168.1.20:11443
[13:13:34 INF] Started WAN data usage tracking for site delta-quadrant
[13:13:34 WRN] Overriding HTTP_PORTS '8042' and HTTPS_PORTS ''. Binding to values defined by URLS instead 'http://*:8042'.

this is from my docker compose file:

network-optimizer:
   image: ghcr.io/ozark-connect/network-optimizer:latest
   container_name: network-optimizer
   restart: unless-stopped
   networks:
     vlan60:
       ipv4_address: 192.168.60.30
   environment:
     - TZ=${TZ}
     - BIND_LOCALHOST_ONLY=${BIND_LOCALHOST_ONLY:-false}
     # Host IP/name for path analysis and CORS (required for client speed tests)
     - HOST_IP=${HOST_IP:-}
     - HOST_NAME=${HOST_NAME:-}
     # Reverse proxy hostname (e.g., optimizer.example.com) - overrides HOST_NAME for API URLs
     - REVERSE_PROXIED_HOST_NAME=${REVERSE_PROXIED_HOST_NAME:-}
     - TRUSTED_PROXIES=${TRUSTED_PROXIES:-}
     - TRUSTED_PROXY_HOPS=${TRUSTED_PROXY_HOPS:-}
     - TRUST_CF_CONNECTING_IP=${TRUST_CF_CONNECTING_IP:-}
     # OpenSpeedTest configuration (for UI display and CORS)
     - OPENSPEEDTEST_PORT=${OPENSPEEDTEST_PORT:-3005}
     - OPENSPEEDTEST_HOST=${OPENSPEEDTEST_HOST:-}
     - OPENSPEEDTEST_HTTPS=${OPENSPEEDTEST_HTTPS:-false}
     - OPENSPEEDTEST_HTTPS_PORT=${OPENSPEEDTEST_HTTPS_PORT:-443}
     # iperf3 server mode - enable to accept client-initiated speed tests on port 5201
     - Iperf3Server__Enabled=${IPERF3_SERVER_ENABLED:-false}
     # Logging (Trace, Debug, Information, Warning, Error, Critical)
     - Logging__LogLevel__Default=${LOG_LEVEL:-Information}
     - Logging__LogLevel__NetworkOptimizer=${APP_LOG_LEVEL:-Information}
   volumes:
     - ${USERDIR}/docker/network-optimizer/data:/app/data
     - ${USERDIR}/docker/network-optimizer/ssh-keys:/app/ssh-keys
     - ${USERDIR}/docker/network-optimizer/logs:/app/logs
   healthcheck:
     test: ["CMD", "curl", "-f", "http://localhost:8042/api/health"]
     interval: 30s
     timeout: 10s
     retries: 3
     start_period: 120s

[tvancott42]

Hey @noumenon272 I had a chance to read over everything again at my keys now that I got this release knocked out.

Couple things, so NO Server doesn't bind or serve TLS/HTTPS by default for its web/API connection, and it's not meant to. I defer to a reverse proxy like Caddy, Traefik, Nginx+Certbot to handle that TLS termination and certification management for people, usually assuming they may want to work that into existing infrastructure or stand it up for the first time as an umbrella for stuff they want to self host.

The TLS connection is just a security thing, and I've not given a way to override it, and that's by design. If you were to try to get the agent to connect to NO Server on 8042 / non HTTPS, it would fail and reject the connection. The requirement is because not everybody will have it riding over site-to-site VPN, and some deployments will selectively expose the NO Server gRPC and public API endpoints via public Internet. So HTTP would be a no-go.

The gRPC endpoint does, however, bind to TLS and it uses a self-signed cert, but if you dig through the agent deployment docs, it covers that that's just so that the path between the reverse proxy and NO Server remain end-to-end encrypted as sensitive site traffic and a direct tunnel to the site's gateway and other UniFi devices runs over that tunnel.

Anyway, background aside, if you want to go with Traefik as your TLS termination/reverse proxy/certificate management server, here's a good starter for it, and it even contains config blocks you can include for the Multi-Site features: https://github.com/Ozark-Connect/NetworkOptimizer-Proxy

If you need anything directly, please DM me on Discord at @tjvc42, happy to help out.

[noumenon272]

thanks for your help. your documentation is excellent. I finally got it working. it took me a while to figure out that cloudflare blocks gRPC by default.