Use mdns.Browser over VPN connection

[manuelnucci]

After reading the documentation, I'm still unable to understand how service discovery works.

In my case, I have two machines connected through a Wireguard VPN with containerized applications running in both of them. One of them is a Node app based on Alpine and running this library (mdns).

# Client
version: "3.7"

services:
  wireguard:
    container_name: wireguard
    image: linuxserver/wireguard:version-v1.0.20200827
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/Argentina/Buenos_Aires
    networks:
      - client
    ports:
      - 51820:51820/udp
      - 631:631
      - 5353:5353/udp # As I understand, mDNS/DNS-SD work over this port and protocol
    volumes:
      - /lib/modules:/lib/modules
      - ./wireguard/config:/config
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

  cups:
    container_name: cups
    image: ydkn/cups
    environment:
      ADMIN_PASSWORD: password
    network_mode: service:wireguard
    volumes:
      - /var/run/dbus:/var/run/dbus
      - /dev/bus/usb:/dev/bus/usb
      - ./cups:/etc/cups
    depends_on:
      - wireguard
    restart: unless-stopped

networks:
  client:
    name: client
    driver: bridge

# Server
version: "3.7"

services:
  wireguard:
    container_name: wireguard
    image: linuxserver/wireguard:version-v1.0.20200827
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/Argentina/Buenos_Aires
      - SERVERURL=XXX.XXX.XXX.XXX #optional
      - SERVERPORT=51820 #optional
      - PEERS=raspberrypi #optional
      - PEERDNS=auto #optional
      - INTERNAL_SUBNET=10.13.13.0 #optional
      - ALLOWEDIPS=0.0.0.0/0 #optional
    networks:
      - web
    ports:
      - 51820:51820/udp
      - 80:${APP_PORT}
    volumes:
      - /lib/modules:/lib/modules
      - ./wireguard/config:/config
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

  app:
    container_name: app
    image: manuelnucci/mdns
    network_mode: service:wireguard
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup
      - /var/run/dbus:/var/run/dbus
      - ./files:/home/node/app/files
    depends_on:
      - wireguard
    restart: unless-stopped

networks:
  web:
    name: web
    driver: bridge

The problem appears after successfully starting avahi-daemon in the container where the app is running. For now, the app consists of this simple file:

// index.js
const mdns = require('mdns');

this.browser = mdns.createBrowser(mdns.tcp('ipp'), { networkInterface: 'wg0' });

this.browser.on('serviceUp', (service) => {
  console.log(service);
});

this.browser.on('serviceDown', ({ name }) => {
  console.log(`Removed printer: ${name}`);
});

this.browser.on('error', (error) => {
  console.log(error);
});

this.browser.start();

console.log('Browser started')

Here, wg0 is the network interface which both containers wireguard and app share in the server.

However, I would expect to receive any package over multicast in the client, which successfully connected to the VPN and is in the same network, as seen here:

// Server, 'app' container
/home/node/app # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN qlen 1000
    link/[65534]
    inet 10.13.13.1/32 scope global wg0
       valid_lft forever preferred_lft forever
42: eth0@if43: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
    link/ether 02:42:ac:14:00:02 brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.2/16 brd 172.20.255.255 scope global eth0
       valid_lft forever preferred_lft forever

// Client, 'wireguard' container
root@473fd0d1896f:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 10.13.13.2/32 scope global wg0
       valid_lft forever preferred_lft forever
13: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:15:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.21.0.2/16 brd 172.21.255.255 scope global eth0
       valid_lft forever preferred_lft forever

The logs from the app are:

Service `hwdrivers' needs non existent service `dev'
 * Caching service dependencies ... [ ok ]
Runlevel: sysinit
Dynamic Runlevel: hotplugged
Dynamic Runlevel: needed/wanted
Dynamic Runlevel: manual
 * Starting System Message Bus ... [ ok ]
 * Starting avahi-daemon ... [ ok ]
Runlevel: sysinit
Dynamic Runlevel: hotplugged
Dynamic Runlevel: needed/wanted
 dbus                                                  [  started 00:00:16 (0) ]
 avahi-daemon                                                      [  started  ]
Dynamic Runlevel: manual
*** WARNING *** The program 'node' uses the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see <http://0pointer.de/blog/projects/avahi-compat.html>
*** WARNING *** The program 'node' called 'DNSServiceRegister()' which is not supported (or only supported partially) in the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see <http://0pointer.de/blog/projects/avahi-compat.html>
Browser started

I'd expect the app to discover a printer I've previously set in the cups container in the client, but after starting a tcpdump in the client, there're no packages sent to this machine.

Some of the solutions I've already tried are:

• Set MULTICAST in wg0 network interface (ifconfig wg0 multicast). Somehow breaks completely the app:

*** WARNING *** The program 'node' uses the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see <http://0pointer.de/blog/projects/avahi-compat.html>
*** WARNING *** The program 'node' called 'DNSServiceRegister()' which is not supported (or only supported partially) in the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see <http://0pointer.de/blog/projects/avahi-compat.html>
/home/node/app/node_modules/mdns/lib/browser.js:88
  dns_sd.DNSServiceBrowse(self.serviceRef, flags, ifaceIdx, '' + requested_type,
         ^

Error: dns service error: unknown
    at new Browser (/home/node/app/node_modules/mdns/lib/browser.js:88:10)
    at Object.create [as createBrowser] (/home/node/app/node_modules/mdns/lib/browser.js:116:10)
    at Object.<anonymous> (/home/node/app/index.js:3:21)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at internal/main/run_main_module.js:17:47 {
  errorCode: -65537
}

• Verify if packages can be sent over the interface. For that I've used the tool mcsender using the following command: mcsender -iwg0 10.13.13.2:<PORT>, with PORT = 51820 | 631 | 5353. All of them send back a successful response.

So, all in all, the questions that still remain are:

• Can mdns library send multicast packages over a VPN connection?
• The fact that the network in the server is 10.13.13.1/32 could be confusing the library?
• Can the avahi-daemon running in the server (inside app) send multicast packages over the network?

I've been working on this for several days and don't know what else to do. Any help is really appreciated.

[manuelnucci]

I would also like to clarify that unlike the approach made in #244, I've mounted the dbus system bus socket and started the avahi-daemon inside the container, because I need the service to see the interface wg0 and make any multicast request through it.

[agnat]

Hi @manuelnucci,

node_mdns probably isn't at fault here. Most VPNs simply don't support this type of service discovery (out of the box). See

• https://www.reddit.com/r/WireGuard/comments/f5ebi6/bonjour_homekit_vpn/
• https://serverfault.com/questions/445680/how-can-bonjour-be-setup-to-function-over-a-vpn-connection-using-mac-os-x-moun
• https://superuser.com/questions/1490332/time-machine-detection-over-vpn-via-dns-sd

... for example. Just google for "vpn dnssd" "wireguard vpn bonjour" etc. Also, to exclude node_mdns as a source of bugs replace your node script with a simple avahi-browse command for now. Get it to work on the command line first and then switch back to node_mdns.

[manuelnucci]

Hi @agnat! Yes, I should've made the test you suggest before any attempt to use the library. As you said, it's not clear if WG supports multicast.

Hopefully I will find the answer this week and will be able to use this library or not. Thanks for all the info.