Automatic device discovery

[szymonmasternak]

Is there a way to get automatic device discovery working under Docker? What are the limitations currently that prohibit this feature from working?

[szymonmasternak]

Also I believe UDP Port 10001 is required as per UniFi's official documents.

https://help.ui.com/hc/en-us/articles/218506997-Required-Ports-Reference

Port 10003 is enabled instead.

unifi-os-server/docker-compose.yaml

Line 27 in 83cbbb9

- 10003:10003/udp

[lemker]

Device discovery should be working (at least the last time I added a new device it was discovered automatically).
The documentation you referenced is not updated for OS Server, it is referencing Network which uses 10001/udp. OS Server uses 10003/udp, 3478/udp, and 8080/tcp for device discovery, adoption, and communication.

You can check the workflow that test runs the installation for the ports it uses:

run --detach --pids-limit 65536 --systemd=always --cap-add=NET_RAW --cap-add=NET_ADMIN --name uosserver --restart unless-stopped --health-cmd curl --fail http://127.0.0.1/api/ping || exit 1 --health-interval 60s --health-timeout 5s --health-retries 3 --network pasta:--ns-ifname,eth0,--map-host-loopback,203.0.113.113,--dns-forward,203.0.113.113 --add-host host.docker.internal:203.0.113.113 --add-host host.containers.internal:203.0.113.113 --dns 203.0.113.113 -e UOS_UUID=94d634b2-8c92-5b08-af04-12312f8a2ec5 -e UOS_SERVER_VERSION=5.0.6 -e FIRMWARE_PLATFORM=linux-x64 -v uosserver_persistent:/persistent -v uosserver_var_log:/var/log -v uosserver_data:/data -v uosserver_srv:/srv -v uosserver_var_lib_unifi:/var/lib/unifi -v uosserver_var_lib_mongodb:/var/lib/mongodb -v uosserver_etc_rabbitmq_ssl:/etc/rabbitmq/ssl -p 11443:443 -p 5005:5005 -p 9543:9543 -p 6789:6789 -p 8080:8080 -p 8443:8443 -p 8444:8444 -p 3478:3478/udp -p 5514:5514/udp -p 10003:10003/udp -p 11084:11084 -p 5671:5671 -p 8880:8880 -p 8881:8881 -p 8882:8882 docker.io/library/uosserver:0.0.54

[tthoma24]

@lemker can we reopen this for further investigation? I don't believe this is correct in its entirety. Though you are right that 10003 is used by Unifi OS Server, I suspect that there may be an interaction here that's undocumented. In particular, the way Unifi appears to have setup the discovery application is that there is a discovery application running on the Linux host in addition to the actual podman container.

udp        0      0 0.0.0.0:10003           0.0.0.0:*                           14639/pasta
udp        0      0 0.0.0.0:5514            0.0.0.0:*                           14639/pasta
udp        0      0 0.0.0.0:3478            0.0.0.0:*                           14639/pasta


tcp        0      0 127.0.0.1:41174         127.0.0.1:11084         ESTABLISHED 15246/uosserver-ser
tcp        0      0 127.0.0.1:11084         127.0.0.1:41174         ESTABLISHED 14639/pasta
tcp        0      0 0.0.0.0:5671            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:9543            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:11443           0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:5005            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:11084           0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:8882            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:8881            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:8880            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:6789            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:8443            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 0.0.0.0:8444            0.0.0.0:*               LISTEN      14639/pasta
tcp        0      0 127.0.0.1:11002         0.0.0.0:*               LISTEN      15254/discovery
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      14639/pasta

I suspect it may actually be listening on a different port and translating or forwarding packets into the container, as evidenced by the netstat output from above, and this may be why we're seeing the behavior with mesh/wireless adoption in #33.

[iMonZ]

Is there a way to get automatic device discovery working under Docker? What are the limitations currently that prohibit this feature from working?

I have the exact issue. Automatic device discovery is not working since I switched to this Docker container.
I had to set "set-inform" every time I performed a restart. Those devices that are in adoption mode, I am completely unable to use right now.

[lemker]

@tthoma24 this issue is about device discovery, which should be working for wired and wireless. Let's move discussion about wireless adoption to the issue you linked #33.

@iMonZ please post your Docker Compose and any relevant information, including container logs.

[iMonZ]

@tthoma24 this issue is about device discovery, which should be working for wired and wireless. Let's move discussion about wireless adoption to the issue you linked #33.

@iMonZ please post your Docker Compose and any relevant information, including container logs.

Can I send you the logs privately? There are a lot of sensitive things I can't censor (maybe those are important)
But apparently L2 discovery is not properly working.
It now works with not only setting the set inform address but also setting a 43 entry into the dhcp
https://help.ui.com/hc/en-us/articles/204909754-Remote-Adoption-Layer-3
Maybe it's a problem with the Docker bridge?
The docker compose file is unedited from the one of that repo except the console ip address (which is set to the unifi os vm of course) and the mounts for the config.
Setting the container to host mode made it crash because the container doesn’t work without the bridge.

[lemker]

Upon further investigation, device discovery is indeed not quite working correctly, see #33 (comment)
@iMonZ having to manually set the inform address might be a limitation for now.

[warst]

If you’re seeing Server Reject with UnsatisfiedLinkError in logs, check /run and /tmp are not mounted noexec.

[bmartino1]

I believe this is 1900 ssdp that assists with auto discovery.

[kevin-king]

I just migrated from jacobalberty/unifi. On jacobalberty/unifi, device adoption worked as you'd expect after setting the Inform Host Override as per the docs.

I did straight backup and restore in the new UnifiOS container. All my access points came over and adoped fine, but once I removed one of them it would not longer show up in the UniFi Devices list, let alone adopt.

Running set-inform http://<raw_ip_of_uos_host>:8080/inform in an ssh session to the AP worked and made it show up in UOS GUI, but this is clunky especially when it might be hard to know the IP of a new AP (without some tooling).

So, I reset the AP to try to find a better auto-discovery fix. I tried things in this thread with no luck like:

• Inform Host Override
• Setting hostname: unifi in docker
• Setting UOS_SYSTEM_IP: unifi in docker
• Setting UOS_SYSTEM_IP: <raw_ip_of_host> in docker
• Changing the docker container name to unifi
• Exposing 10001:10001/udp in addition to 10003:10003/udp in docker
• Exposing 1900:1900 in docker

I then ssh'ed into the access point that is not longer showing up and ran info:

Model:       U7-Pro-Wall                                        
Version:     8.4.6.18068                                        
MAC Address: 6c:63:f8:27:27:5d                                  
IP Address:  192.168.1.82                                       
Hostname:    U7-Pro-Wall                                        
Uptime:      214 seconds                                        
NTP:         Synchronized                                       
                                                                
Status:      Unable to resolve (http://unifi:8080/inform) 

then nslookup unifi

Server:         192.168.1.2                                     
Address:        192.168.1.2:53
** server can't find unifi.localdomain: NXDOMAIN                                                                       
** server can't find unifi.localdomain: NXDOMAIN

So I added a DNS Rewrite unifi.localdomain->192.168.1.2 in adguard. Then, my APs all started showing up and adopting normally. I considered using extra_hosts in docker, but realized it's the APs themselves that are the issue (not UOS) and so you need something at the network level. Worth noting that a DNS Rewrite unifi->192.168.1.2 didn't work (probably obvious to some people why, but it caught me off guard). This configuration works with all of my extra configuration attempts in the bulleted list removed.

Still seems there's still some underlying difference between this image and jacobalberty/unifi, but figured I'd share an approach that worked for me to not need to ssh into my APs.

[bmartino1]

I just migrated from jacobalberty/unifi. On jacobalberty/unifi, device adoption worked as you'd expect after setting the Inform Host Override as per the docs.

I did straight backup and restore in the new UnifiOS container. All my access points came over and adoped fine, but once I removed one of them it would not longer show up in the UniFi Devices list, let alone adopt.

Running set-inform http://<raw_ip_of_uos_host>:8080/inform in an ssh session to the AP worked and made it show up in UOS GUI, but this is clunky especially when it might be hard to know the IP of a new AP (without some tooling).

So, I reset the AP to try to find a better auto-discovery fix. I tried things in this thread with no luck like:

• Inform Host Override
• Setting hostname: unifi in docker
• Setting UOS_SYSTEM_IP: unifi in docker
• Setting UOS_SYSTEM_IP: <raw_ip_of_host> in docker
• Changing the docker container name to unifi
• Exposing 10001:10001/udp in addition to 10003:10003/udp in docker
• Exposing 1900:1900 in docker

I then ssh'ed into the access point that is not longer showing up and ran info:

Model: U7-Pro-Wall
Version: 8.4.6.18068
MAC Address: 6c:63:f8:27:27:5d
IP Address: 192.168.1.82
Hostname: U7-Pro-Wall
Uptime: 214 seconds
NTP: Synchronized

Status: Unable to resolve (http://unifi:8080/inform)
then nslookup unifi

Server: 192.168.1.2
Address: 192.168.1.2:53
** server can't find unifi.localdomain: NXDOMAIN
** server can't find unifi.localdomain: NXDOMAIN
So I added a DNS Rewrite unifi.localdomain->192.168.1.2 in adguard. Then, my APs all started showing up and adopting normally. I considered using extra_hosts in docker, but realized it's the APs themselves that are the issue (not UOS) and so you need something at the network level. Worth noting that a DNS Rewrite unifi->192.168.1.2 didn't work (probably obvious to some people why, but it caught me off guard). This configuration works with all of my extra configuration attempts in the bulleted list removed.

Still seems there's still some underlying difference between this image and jacobalberty/unifi, but figured I'd share an approach that worked for me to not need to ssh into my APs.

what is you docker run or compose file ???

example updated compose thats workign for me on other sytems...

services:
  unifi-os-server:
    container_name: unifi-os-server
    image: ghcr.io/lemker/unifi-os-server:v1.0.0
    hostname: unifi
    network_mode: bridge
    restart: unless-stopped
    pids_limit: 2048
    environment:
      TZ: America/Chicago
      HOST_CONTAINERNAME: unifi-os-server
      UOS_SYSTEM_IP: unifi
    ports:
      - "11443:443/tcp"
      - "5005:5005/tcp"
      - "9543:9543/tcp"
      - "6789:6789/tcp"
      - "8080:8080/tcp"
      - "8443:8443/tcp"
      - "8444:8444/tcp"
      - "1900:1900/udp"
      - "3478:3478/udp"
      - "5514:5514/udp"
      - "10001:10001/udp"
      - "10003:10003/udp"
      - "11084:11084/tcp"
      - "5671:5671/tcp"
      - "8880:8880/tcp"
      - "8881:8881/tcp"
      - "8882:8882/tcp"
    volumes:
      - /mnt/user/appdata/unifi-os-server/persistent:/persistent:rw
      - /mnt/user/appdata/unifi-os-server/var-log:/var/log:rw
      - /mnt/user/appdata/unifi-os-server/data:/data:rw
      - /mnt/user/appdata/unifi-os-server/srv:/srv:rw
      - /mnt/user/appdata/unifi-os-server/var-lib-unifi:/var/lib/unifi:rw
      - /mnt/user/appdata/unifi-os-server/var-lib-mongodb:/var/lib/mongodb:rw
      - /mnt/user/appdata/unifi-os-server/etc-rabbitmq-ssl:/etc/rabbitmq/ssl:rw
      - /sys/fs/cgroup:/sys/fs/cgroup:rw
    cgroupns: host
    cap_add:
      - NET_RAW
      - NET_ADMIN
    tmpfs:
      - /run:exec
      - /run/lock
      - /tmp:exec
      - /var/lib/journal
      - /var/opt/unifi/tmp:size=64m

are you able to docker exec into the container?

For the UniFi service inside a systemd-based host or container, use:

systemctl status unifi

If that service name does not exist, check likely related services:

systemctl --type=service | grep -i unifi

Or for UniFi OS Server pieces:

systemctl --type=service | grep -Ei 'unifi|ubnt|mongo|postgres|rabbitmq'

did you set a dns name for unifi on the network outside of teh docker?

please docker exec and run

set-inform http://$UOS_SYSTEM_IP:8080/inform

where UOS_SYSTEM_IP is the p of the host / continer dependng on docker netowrk assiinged to container...

set-inform http://192.168.1.10:8080/inform

[kevin-king]

please docker exec and run
set-inform http://$UOS_SYSTEM_IP:8080/inform

set-inform is an access point command, not a UOS docker command, no?

are you able to docker exec into the container?

yes

did you set a dns name for unifi on the network outside of teh docker?

yes, as I said its a DNS rewrite in adguard (basically the DNS server for the sake of this conversation).

I bootstrapped my docker-compose file from here, which looks like a version newer than the one you posted @bmartino1. https://github.com/lemker/unifi-os-server/blob/main/docker-compose.yaml

I removed/commented out some pieces like hostname, UOS_SYSTEM_IP: unifi, etc. to rule out false positive configurations. As I said the post above, I tried some configurations in your docker compose with no luck, but with local DNS rewrite many of them don't seem necessary.

  unifi:
    container_name: unifi
    image: ghcr.io/lemker/unifi-os-server:latest
    restart: unless-stopped
    cgroup: host
    cap_add:
      - NET_RAW
      - NET_ADMIN
    tmpfs:
      - /run:exec
      - /run/lock
      - /tmp:exec
      - /var/lib/journal
      - /var/opt/unifi/tmp:size=64m
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:rw
      - .../unifios/persistent:/persistent
      - .../unifios/var-log:/var/log
      - .../unifios/data:/data
      - .../unifios/srv:/srv
      - .../unifios/var-lib-unifi:/var/lib/unifi
      - .../unifios/var-lib-mongodb:/var/lib/mongodb
      - .../unifios/etc-rabbitmq-ssl:/etc/rabbitmq/ssl
    ports:
      - 11443:443
      - 8080:8080
      - 3478:3478/udp
      - 10003:10003/udp

Also, are you sure about HOST_CONTAINERNAME in your config? That looks specific to the linuxserver version of UnifiOS.