Mastodon

This blog has joined the Fediverse

TL;DR

My blog noteblok.net has joined the Fediverse. You can follow my posts via this new handle: stv0g@noteblok.net.

This has been made possible by the WordPress ActivityPub Plugin. With the ActivityPub plugin installed, the WordPress blog functions as a federated profile, along with profiles for each author. For example, my blog-wide profile can be found at @blog@noteblok.net. Authors like myself, on the other hand, would have their individual profiles at @stv0g@noteblok.net.

The integration allows following the blog from your own Fediverse platform and account like Mastodon. I return you can also react and comment to my blog posts via simply replying with your existing Fediverse account.

Fritz-DNS – An authoritative DNS server for AVM FRITZ!Box routers

AVM FRITZ!Box Cable 6591

In my home network, I am using an AVM FRITZ!Box Cable 6690. It handles DHCP, DNS, Wifi and recently also interfaces my home network via WireGuard to my servers.

Just like the venerable Dnsmasq AVM’s FRITZ!OS uses hostnames learned from its DHCP leases and makes them resolvable via its internal DNS server.

Unfortunately, this feature in FRITZ!OS has some limitations:

  1. The name of the DNS Zone is hard coded to fritz.box and can not be adjusted. Hence, the resolvable names have the following schema: myhostname.fritz.box
  2. The internal DNS server only supports recursive DNS looks. It does not act as an authoritative DNS server. Hence the local zone can not be delegated.
  3. AXFR zone transfers are not supported.

My solution to these shortcomings is Fritz-DNS which:

  • is a small tool written in the Go programming language.
  • is a small authoritative DNS server which serves A/AAAA resource records for local hosts connected to an AVM Fritz Box home WiFi router.
  • can be used in a hidden master configuration as it supports AXFR zone transfers.
  • uses the custom extension (X_AVM-DE_GetHostListPath) of the TR-064 Hosts SOAP-API as documented here to retrieve a list of local hosts.
  • supports the generation of AAAA (IPv6) resource records based on the hosts MAC addresses using 64-Bit Extended Unique Identifier (EUI-64) and a configured unique local address (ULA) prefix.
  • does not yet support PTR resource records (to be implemented…)
  • is licensed under the Apache 2.0 license

You can find Fritz-DNS at GitHub: https://github.com/stv0g/fritz-dns

Here is a small figure illustrating the interaction of Fritz-DNS with the Fritz!Box and other DNS servers / clients:

CLI Usage

Usage of fritz-dns
  -ipv6-ula-prefix string
    	Fritz Box IPv6 ULA Prefix (default "fd00::/64")
  -pass string
    	FritzBox password
  -port int
    	Listen port (default 53)
  -soa-expire duration
    	SOA expire value (default 744h0m0s)
  -soa-mbox string
    	SOA mailbox value
  -soa-minttl duration
    	SOA minimum TTL value (default 1h0m0s)
  -soa-ns string
    	Authorative DNS server for the zone
  -soa-refresh duration
    	SOA refresh value (default 2h0m0s)
  -soa-retry duration
    	SOA retry value (default 1h0m0s)
  -ttl duration
    	default TTL values for records (default 5m0s)
  -url string
    	FritzBox URL (default "http://fritz.box/")
  -user string
    	FritzBox username (default "admin")
  -zone string
    	DNS Zone (default "fritz.box.")

Aachen Transparent!

Ich möchte Stadtpolitik in Aachen für alle verständlich machen. Mein aktuellstes Projekt aachen-transparent.de ermöglicht es, die öffentlichen Informationen aus dem städtischen Ratsinformationssystem modern und benutzerfreundlich aufzubereiten. Dazu habe ich das bereits existieren Open-Source Projekt Meine-Stadt-Transparent erweitert und für die Bedürfnisse in Aachen angepasst.

Screenshot von aachen-transparent.de

Aachen Transparent ist ein Projekt, dass ich ehrenamtlich im Rahmen des Open Data Labs Aachen ins Leben gerufen habe. Es versucht einige der Unzulänglichkeiten des Ratsinformationssystems der Stadt Aachen zu umgehen. Dazu nutzt es dessen öffentliche OParl Schnittstelle um die dort hinterlegten Informationen über eine moderne Oberfläche zugänglich zu machen.

Dies ist besonders deshalb von Interesse, da das Aachener Ratsinformationssystem leider keine Indexierung von Suchmaschinen wie Google zulässt und Bürger daher bisher nur die recht beschränkte Suchfunktion der AllRis Software nutzen konnten. Aachen Transparent unterstützt und fördert explizit die Indizierung aller Anträge, Dokumente, Tagesordnungspunkte und Beschlüsse durch gängige Suchmaschinen und stellt auch eine eigene Volltextsuche zu Verfügung.

Funktionalität

Durch Meine Stadt Transparent erfahren Bürgerinnen und Bürger, wer für sie im Stadtrat sitzt, wann der Stadtrat tagt und welche Themen besprochen werden. Da alle Dokumente räumlich auf einer Karte verortet werden, sind leicht jene Themen erkennbar, die sich auf das unmittelbare Lebensumfeld beziehen.

Großen Wert habe ich auf eine intuitive und flexible Suche gelegt: Mit einer zentralen Suche lässt sich die gesamte Seite – Sitzungsvorlagen, Tagesordnungen, uvm. – im Volltext durchsuchen, genauso wie gewählte Stadtratsmitglieder oder Ausschüsse. Um auch dauerhaft auf dem Laufenden zu bleiben, bietet „Aachen Transparent“ verschiedene Anschlusspunkte: Nutzer können sich E-Mail-Benachrichtigungen anlegen, die über neue Dokumente zu abonnierten Suchbegriffen informiere. Termine und Terminreihen lassen sich per iCal in eigene Kalendersysteme einbetten, die aktuellen Dokumente lassen sich auch im eigenen RSS-Reader lesen.

Aachen Transparent ist kein isoliertes System, sondern greift auf das existierende Ratsinformationssysteme der Stadt Aachen zurück, um von dort Daten automatisch zu beziehen und aufzubereiten. Die Verwaltung von Dokumenten erfolgt weiterhin über ein klassisches Ratsinformationssystem, zu dem Aachen Transparent eine moderne Oberfläche bietet.

Im Überblick

  • Volltextsuche durch alle Dokumente
  • Texterkennung in gescannten Dokumenten
  • Georeferenzierung von Dokumenten
  • iCal Kalender für Integration in Outlook, Google und andere Kalender
  • RSS Feeds für Ausschüsse und Personen
  • Abonnierbare Suchen mit E-Mail Benachrichtigungen und RSS Feeds

Geo-referenzierte Dokumente

Interessant ist auch die Verortung von Dokumenten und Anträgen. Aachen Transparent durchsucht alle indizierten Text auf Straßennamen im Aachener Stadtgebiet und nutzt diese zur Georeferenzierung. Wie im folgenden Bild zu sehen kann dadurch eine Karte generiert werden, welche einen Überblick über die aktuellen Themen in der Stadtpolitik gibt. Diese Georeferenzierung ist zudem auch über die Suchfunktion nutzbar, sodass z.B. eine Suche auf einen Stadtteil oder Straße eingeschränkt werden kann.

Verortung von Anträgen

Informationen zu Rats- und Ausschussmitgliedern

Desweiteren stellt Aachen Transparent eine Übersicht aller aktuellen Rats und Ausschussmitglieder bereit, die deren Mitgliedschaft in Ausschüssen zusammengefasst.

Personen Übersichtsseite

Offene Schnittstellen

Aachen Transparent unterstützt auch bereits existierende offene Schnittstellen um Daten aus dem Ratsinformationssystem auch über andere Kanäle zu verarbeiten.

So unterstützen wir z.B einen iCal Export des es ermöglicht Sitzungskalender in andere Kalendersoftware (z.B Outlook oder Android Kalender) zu integrieren und diese kontinuierlich zu synchronisieren.

Neue Anträge und Beschlusssachen können zudem über RSS Feeds abonniert werden. Interessant ist dabei auch die Möglichkeit individualisierte Feeds basierend auf Suchanfragen zu erstellen.

Häufige Fragen

Wer steckt hinter Aachen Transparent?

Aachen Transparent ist Angebot das auf ehrenamtlicher Basis von mir im Umfeld des Open Data Labs Aachen betrieben wird. Es ist daher kein offizielles Angebot der Stadt Aachen.

Kann ich Aachen Transparent auch in meiner Stadt/Gemeinde einsetzen?

Ja! Aachen Transparent basiert auf der Open Source Software Meine Stadt Transparent, die gemeinschaftlich von mehreren Open Data Labs und interessierten Privatpersonen entwickelt wird. Daher ist es prinzipiell auch möglich eine separate Instanz der Software für ihre Gemeinde aufzusetzen.

Aktuelle Statistiken aus Aachen

  • Dateien: 11.628
  • Sitzungen: 3.439
  • Gremien: 165
  • Vorlagen: 18.367
  • Personen: 1.370

(Stand: 2. August 2022)

A highly available WireGuard VPN setup

WireGuard is a communication protocol and free and open-source software that implements encrypted virtual private networks (VPNs), and was designed with the goals of ease of use, high speed performance, and low attack surface.

My Journey to WireGuard

I’ve been using it in my home lab setup since about 2020. When, in the end of 2021, it was finally merged into the Linux mainline with release 5.9, I started to replace my former Tinc-VPN setup with it. Tinc-VPN is another great open source VPN solution. Unfortunately, its development has stalled over the last years which motivated me to look for alternatives. In contrast to WireGuard, Tinc runs as a user-space daemon and uses tun/tap devices which adds a significant processing overhead. Like WireGuard, it is also using UDP for tunneling data, but falls back to TCP in situations where direct datagram connectivity is not feasible. Another big advantage of Tinc is its ability to form a mesh of nodes and to route traffic within it when direct P2P connections are not possible due to firewall restrictions. At the same time, this mesh is also used for facilitating direct connections by signaling endpoint addresses of NATed hosts.

Tinc’s mesh capability.

This mesh functionality made Tinc quite robust against the failure of single nodes as usually we could route traffic via other paths.

Highly Available WireGuard server setup

To counteract this shortcoming, this blog post will present a highly available WireGuard setup using the Virtual Router Redundancy Protocol (VRRP) as implemented by the keepalived daemon.

That said, it is worth noting that this setup does will not bring back some of the beloved features of Tinc. Both meshing, the peer and and endpoint discovery features of Tinc are currently and will never be supported by WireGuard. Jason A. Donenfeld the author of WireGuard focused the design of WireGuard on simplicity, performance and auditability. Hence advanced features like the ones mentioned will only be available to WireGuard by additional agents / daemons which control and configure WireGuard for you. Examples for such are Tailscale, Netmaker and Netbird.

The setup presented in this post is a so called active / standby configuration consisting of two almost equal configured Linux servers running both WireGuard and the keepalived daemon. As the name suggest only one of those two servers will by actively handling WireGuard tunneling traffic while the other one stands by for the event of a failure or maintenance of the active node.

VRRP Wireguard Setup

Requirements

Before get started some requirements for the setup:

  • 2 Servers running Linux 5.9 or newer.
  • A working Wireguard configuration.
  • A local L2 network segment two which both servers are connected.
  • Upstream connectivity without NATing via gateway connected to the network segment (usually provided by your internet or hosting provider).
  • An unused address to be used as Virtual IP (VIP) which roamed between the two servers by VRRP.

An important point is here the assumption that we are running both servers in the same switched network segment as this is a requirement for VRRP. We are also assuming that the upstream gateway performs no NATing. This guide covers only IPv6 addressing. However all steps can be also adapted or repeated for a dual stack or IPv4-only setup.

Detailed steps

Here are some of the specifics for my setup which need to be adapted by you:

  • Server Key (same use by both servers)
    • Private: YIEDx+A2ONo5+uv3mrk/p7ileL3T5QQ8hleQK0IYEEI=
    • Public: XGubrkGtuECdvoykKeUiNMigk2onfLCPfEo9Im+vmx8=
  • Peer Key (In this example we only have a single peer)
    • Private: OIbpWVIVVBOtWfwkmXkFRN7Q/jBdfYtsGt7j97aHx1Q=
    • Public: 3NGl6gTOGs6ai0RE91VmVFgF+N4gw1EBG11KOeiKJAg=
  • Public Server Subnet: 2001:DB8:1::/64
    • Gateway: 2001:DB8:1::1
    • Virtual IP: 2001:DB8:1::2
    • Server A: 2001:DB8:1:::3
    • Server B: 2001:DB8:1::4
  • WireGuard Tunnel Subnet: 2001:DB8:2::1/64
    • Server: 2001:DB8:2::1 (same used by both servers)
    • Peer: 2001:DB8:2::2
  • Interface names
    • Wireguard: wg1
    • Upstream: eno1

1. Prepare servers

We start of preparing the two servers by installing WireGuard and keepalived:

sudo apt install keepalived wireguard-tools iproute2

Next we configure a WireGuard interface on both servers using exactly the same configuration file at /etc/wireguard/wg1.conf:

[Interface]
Address = 2001:DB8:2::1/64
PrivateKey = YIEDx+A2ONo5+uv3mrk/p7ileL3T5QQ8hleQK0IYEEI=
ListenPort = 51800

[Peer]
PublicKey = 3NGl6gTOGs6ai0RE91VmVFgF+N4gw1EBG11KOeiKJAg=
AllowedIPs = 2001:DB8:2::2/128
PersistentKeepalive = 25

Similarly, a reciprocal configuration file is needed on the client side which skip here for brevity. Before proceeding, we activate the interface on both servers:

systemctl enable --now wg-quick@wg1

wg show wg1 # Check if interface is up

2. Configuring Keepalived

Create a configuration file for keepalived at /etc/keepalived/keepalived.conf

global_defs {
    enable_script_security
    script_user root
}

# Check if the server the WireGuard interface configured
vrrp_script check_wg {
    script "/usr/bin/wg show wg1"
    user root
}

vrrp_instance wg_v6 {
    interface eno1
    virtual_router_id 52
    notify /usr/local/bin/keepalived-wg.sh

    state BACKUP # use BACKUP for Server B
    priority 99 # use 100 for Server B

    virtual_ipaddress {
	2001:DB8:1::1/64
    }

    track_script {
        check_wg
    }
}

Create a notification script for keepalived at /usr/local/bin/keepalived-wg.sh

#!/bin/bash

TYPE=$1
NAME=$2
STATE=$3
PRIO=$4

WGIF=wg1

case ${STATE} in
	MASTER)
		ip link set up dev ${WGIF}
		;;

	BACKUP|FAULT|STOP|DELETED)
		ip link set down dev ${WGIF}
		;;

	*)
		echo "unknown state"
		exit 1
esac

Now start the keepalived daemon:

chmod +x /usr/local/bin/keepalived-wg.sh
systemctl enable --now keepalived

4. Testing the fail over

In our configuration, Server A has a higher VRRP priority and as such will be preferred if both servers are healthy. To test our setup, we simply bring down the WireGuard interface on Server A and observe how the VIP gets moved to Server B. From the WireGuard peers perspective not much changes. In fact no connections will be dropped during the fail-over. Internally, the clients WireGuard interface renegotiate the handshake. However, that step is actually not observable by the user.

Run the following commands on Server A while alongside test the connectivity from the client side through the tunnel via ping -i0.2 2001:DB8:2::1:

# Check that keepalived has moved the VIP to interface eno1
ip addr show dev eno1

# Bring down the Wireguard interface
wg-quick down wg1

# Keepalived should now have moved the VIP to Server B
ip addr show dev eno1

Going further

In my personal network, I operate a Interior Gateway Protocol (IGP) to dynamically route traffic within and also towards other networks. Common IGPs are OSPF, ISIS or BGP. In my specific case, both Servers A & B run the Bird2 routing daemon with interior and exterior BGP sessions.

So how does the WireGuard HA setup interoperates with my interior routing? Quite well actually. As my notify script (keepalive-wg.sh) will automatically bring up/down the interface, the routes attached to the interface will be picked up by Bird’s direct protocol.

I am also planning to extend my WireGuard agent ɯice to support the synchronization of WireGuard interface configurations between multiple servers.

Background

Surprisingly, the setup works by using Keepalived and does not require any iptables or nftables magic to rewrite source IP addresses. I’ve seen some people mentioning that SNAT/DNAT would be required to convince WireGuard to use the virtual IP instead of the server addresses. However, in my experience this was not necessary.

Another concern has been that the backup Wireguard interface still might attempt to establish a handshake with its peers. This would quite certainly interfere with the handshakes originated by the current master server. However, also this has not been proven to be the case. I assume the fact that our notify script brings down the WireGuard interface on the backup server causes them to cease all communication with its peers.

SSH Access for Netgear’s Nighthawk M5 Mobile LTE/Router

In my previous post, I demonstrated how to gain root access by enabling a Telnet daemon via the routers AT-over-TCP interface. In this post I will close this gasping security hole by replacing the Telnet with a Secure Shell (SSH) daemon. Netgear’s firmware does not ship with a SSH daemon itself. So we first build a statically linked Dropbear instead of the rather heavy OpenSSH daemon.

Building Dropbear SSH

I’ve build a statically linked version of Dropbear using a Debian-based Docker image as it allows us to use the packaged cross-compiler toolchains by Debian:

Dockerfile
FROM debian:bullseye

RUN apt-get update && \
    apt-get -y install \
    	wget tar bzip2 build-essential \
	gcc-arm-linux-gnueabihf \
	binutils-arm-linux-gnueabihf

RUN wget https://matt.ucc.asn.au/dropbear/releases/dropbear-2022.82.tar.bz2
RUN tar xvf dropbear-2022.82.tar.bz2

WORKDIR /dropbear-2022.82

ENV CC=arm-linux-gnueabihf-gcc
ENV CFLAGS="-DDROPBEAR_SVR_PASSWORD_AUTH=0"

RUN ./configure --host=arm-linux-gnueabhf \
	--disable-zlib \
	--disable-shadow \
	--disable-syslog \
	--disable-lastlog \
	--enable-static
RUN make PROGRAMS="dropbear scp" MULTI=1

RUN arm-linux-gnueabihf-strip dropbearmulti

With this Dockerfile we can build the image, create a temporary container and copy the resulting binary from the image to your local folder:

docker build -t dropbear .

id=$(docker create dropbear)
docker cp ${id}:/dropbear-2022.82/dropbearmulti ./
docker rm ${id}

Installing Dropbear

Now that we have a statically linked version of the SSH daemon, we will need to copy it to our target. I accomplished this by using netcat (nc):

On the target

mkdir -p /data/mod/bin
pushd /data/mod/bin

nc -l -p 1234 > dropbearmulti
chmod +x dropbearmulti

ln -s dropbearmulti dropbear
ln -s dropbearmulti scp

On the machine which build Dropbear

nc <ip-of-target> 1234 < dropbearmulti

This is followed by installing a SystemD service which start the SSH daemon on system boot:

cat > /etc/systemd/system/dropbear.service <<EOF
[Unit]
Description=Dropbear SSH server
After=network.target

[Service]
Type=forking
ExecStart=/data/mod/bin/dropbear -R
PIDFile=/var/run/dropbear.pid

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable --now dropbear.service

Before you will be able to connect to the target, you will need to install an authorized_keys file. Password login is not supported.

mkdir -p /home/root/.ssh
cat > /home/root/.ssh/authorized_keys <<EOF
ssh-rsa AAAAB3NzaC1...GaoxPrQ== # replace by your SSH key
EOF

All that remains is a quick test:

ssh root@<ip-of-target>

Disable SSH daemon

In order to restore the security of the device we must also disable the Telnet daemon. There are in principle two options to achieve this:

  • Reverse the steps from my first blog post via the AT-over-TCP interface.
  • Use the iptables firewall to block access to the Telnet port

I’ve decided to go for the second option:

cat > /etc/systemd/system/block-telnet.service <<EOF
[Unit]
Description=Block Telnet access
After=network.target

[Service]
Type=simple
ExecStart=/usr/sbin/iptables -I INPUT -p tcp --dport telnet -j DROP
ExecStop=/usr/sbin/iptables -D INPUT -p tcp --dport telnet -j DROP

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable --now block-telnet.service