Albert Wu's Blog

Category: Self-hosting

  • Dockerizing everything

    7–11 minutes

    The New Year

    For a long time I’ve always wanted to self host my own services for the sake of data sovereignty. However, with my current internet plan, my upload speeds are a paltry 50mbps. I needed something with better speeds and within reasonable costs to host the services that I want publicly available. In 2026, I changed my reverse proxy solution and decided to start to document my adventure.

    The Background

    I’ve done a bit of self hosting and I’ve focused heavily on data retention, sorting, and backups. I’ve ran with Proxmox with Open Media Vault on top with some flavor of ZFS as my file system as my main storage. I’ve learned how to script, push and pull data, and manage disk failures. But so far I’ve only really just self hosted my SMB share akin to OneDrive. I only access it within my LAN or via my VPN. I’m too much of a chicken to publicly expose any massive file services to the internet… as is well cautioned by many other self hosters.

    The Initial Setup

    So I finally got myself a tiny instance in the cloud that’s 1 core/1gb ram/40gb SSD. This was enough for me to at least tryout some sort of reverse proxy. I spun up Tailscale and crated a DMZ within my home network. The overall concept is that if my reverse proxy were to ever get compromised, they would at least just make it to my DMZ with no way into my network. The data hosted within my DMZ are secondary data. My real data is backed up into my main LAN servers.

    First Test

    Using my VPS in the cloud, I created a Tailscale network between it and my DMZ network. For example the VPS ip is 100.100.100.2 and my DMZ server is 100.100.100.3. I ran Immich within docker and setup Caddy as the reverse proxy for my VPS.

    Immich docker-compose snippet

    services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - '2283:2283'

    Note the '2283:2283' portion of the code. This will be important later.

    As of now, the docker-compose exposes the Immich instance to all available interfaces. For this it’s fine since I can turn off the VPN and do maintenance on the service via the LAN IP. This allows for some internal segregation and allows me to use a kill switch if something were to get compromised.

    Additionally, my Immich is behind an OAuth authentication from Google. I’ll get that guide setup later. I’ll link it here.

    Setting up the VPS

    The VPS side is easy as it is only to function as a reverse proxy. I install and authenticate my Tailscale network, and install Caddy. That’s it! Next is to point my DNS records to my VPS public facing IP. Once I’ve made sure it has propagated properly, then I can setup Caddy.

    Caddy file

    Caddy is extremely simple to use. One Caddyfile and one DNS challenge later, your reverse proxy now has a valid SSL cert ready to go. (These domains are not in use btw)

    immich.albertwu.dev {
        reverse_proxy 100.100.100.3:2283
}

photos.albertwu.dev {
        reverse_proxy /_app/* 100.100.100.3:2283
        reverse_proxy /share/* 100.100.100.3:2283
        reverse_proxy /api/* 100.100.100.3:2283
        respond /auth/* "Access denied" 403
        respond / "Access denied" 403 {
                close
        }
}

    I deploy this Caddyfile by typing in

    $ caddy run

    You can also do Caddy start, but this doesn’t run it in the background and once you exit, you’ll stop Caddy.

    As you noticed, 100.100.100.3 is the DMZ VPN IP which means that this Caddy is reverse proxying my DMZ server.

    From line 5 downwards are my configs for my publicly facing photos. These are the photos that I share to the public mostly for friends and family. As you can see, I’ve blocked authentication through this domain. This is not quite a full firewall, but at least prevents users from trying login with the wrong domain.

    Testing

    I’ve ran this on a mini PC with a Ryzen 7 7840HS with 32GB of ram, 2TB system NVMe and a 4TB data NVMe. Theoretically, giving me about 4TB of “Google photos” cloud storage. Unfortunately, with the anemic upload speeds, when I request photos that are server only, the thumbnails populate like molasses… and forget about querying videos that are on the server. It did not pass the wife test and I was somewhat defeated.

    A New Plan

    Since my plan was limited by my internet connection and all tiers of the plans are limited to 50mbps, I had to outsource my so called internet.

    The Magical VPS

    In the world filled with subscriptions, let’s add another to the pile. This subscription will hopefully allow me to be a little more in control of my own data.

    Enter Netcup. During one of their Easter sales, I snagged an ARM64 VPS with 10vCores, 16GB ram, 1TB SSD, and 2.5GbE for only €10. This checked a lot of the boxes I needed.

    Deploying and setting up the Docker Stack From Scratch

    So here’s the grand finale of putting everything together.

    I’m going to try to list of everything you need.

    • Prerequisites
      • Domain Name
      • VPS and Publicly accessible IP address
    • Hardware
      • x86 or ARM64 architecture
      • Management device

    First we’ll secure and setup the VPS. Run all of your scripts, security settings, and setup your keys. Then install and authenticate Tailscale. Your Tailscale is the only way for you to manage your server. After that, you’d setup your iptable (I’m lazy so I setup UFW)

         To                         Action      From
     --                         ------      ----
[ 1] Anywhere on tailscale0     ALLOW IN    Anywhere
[ 2] 41641/udp                  ALLOW IN    Anywhere
[ 3] 22/tcp                     ALLOW IN    Anywhere
[ 4] 80/tcp                     ALLOW IN    Anywhere
[ 5] 443/tcp                    ALLOW IN    Anywhere
[ 6] Anywhere (v6) on tailscale0 ALLOW IN    Anywhere (v6)
[ 7] 41641/udp (v6)             ALLOW IN    Anywhere (v6)
[ 8] 22/tcp (v6)                ALLOW IN    Anywhere (v6)
[ 9] 80/tcp (v6)                ALLOW IN    Anywhere (v6)
[10] 443/tcp (v6)               ALLOW IN    Anywhere (v6)

    This snippet is just an example. For instance, I’d recommending moving your SSH port to a non-standard port. You can also chose to only allow connections through your VPN, but if you get locked out, you’ll need to access your admin console. You must have port 80/tcp and 443/tcp open for Nginx Proxy Manager to work. Finally 41641/udp is used for Tailscale. Note that tailscale0 is allowed from anywhere to anywhere.

    Next identify your tailscale0 ip address and make a note of that. For this example it is $Tailscale0_IP

    Now to deploy the docker instance. Identify your mount points and keep a good note of them.

    Installing Docker

    Here’s a quick snippit of setting up docker (without snap)

    # Add the official Docker GPG key
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the official Docker repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install Docker
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

# Verify version
docker version

    First I setup portainer. It’s a web based GUI to manage your docker stacks. I love it, but hate how sometimes it breaks things. Good thing is that a broken stack rarely impacts another stack. So portainer going bad just means I’ll need to manage via the CLI.

    Also please note the following ports 9000:9000 and 9443:9443 see how there are no IP addresses before it? This is very dangerous as I could expose my portainer instance to the whole wide world. (Which I just realized when I moved from LTS to STS a few weeks back)

    Modify 9000:9000 and 9443:9443 to your $Tailscale0_IP address. This will only allow portainer to be accessible via your Tailscale network.

    # Creating the data volume for Portainer
docker volume create portainer_data

# Creating Portainer Image
docker run -d -p $Tailscale0_IP:9000:9000 -p $Tailscale0_IP:9443:9443 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:sts

# Checking if it's running
docker ps

    Please note that this version is the portainer-ce:sts version which can break items… but this means nothing as the lts also breaks items. (Example: when portainer was not compatible with Docker 29) My grievances mostly end there, but such is the life of open source.

    You can swap between the two versions by changing sts (Short term Support – 1.5 month support) to lts (Long term Support – 1 year support) and visa versa.

    Next stack is the Nginx Proxy Manager.

    services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: always
    ports:
      # These ports are in format <host-port>:<container-port>
      - '$VPS_IP:80:80' # Public HTTP Port
      - '$VPS_IP:443:443' # Public HTTPS Port
      - '$Tailscale0_IP:81:81' # Admin Web Port

    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt

    I do the same thing here, $Tailscale0_IP:81 is the only way to manage NPM.

    Once that container deploys, I go to $Tailscale0_IP:81 and setup my NPM instance.

    Create a username and password and we go into our services.

    Setting up services

    In this example I’m going to set up Immich and Reitti. Both of which benefit greatly from being publicly accessible.

    Make sure you bind the correct IP address and note the container’s IP address.

    I’m going to add another post for reverse proxying properly since I don’t want this post to get so long it’s hard to follow.

  • BirdNET-PI 2024-2026

    3–5 minutes

    I’ve gotten into birding about 2022-2023 time frame. I found an app called Merlin Bird ID that allowed anyone with a smartphone to listen to birds and the app will identify the birds based on their spectrogram patterns.

    I try to keep a bird feeder full for the local backyard birds and always wanted to try to identify the birds in the neighborhood even when I’m not there. I looked around for something like Bird ID that can run 24/7 without me being there.

    Enter BirdNET-Pi

    BirdNET-Pi uses a RasberryPi 5 along with a microphone to record and identify birds 24/7. At the time, I only had a Pi 3B so I had to use the lite version of the OS/Software. The repository is open source and available here.

    Version 1

    With a High Endurance SD card and a Yeti Snowball microphone, I started the project. I put it outside under an overhang to protect it from the elements and had it listen to all of the feathered friends flying around the area. This project lasted about 1 year before the SQL database corrupted and froze the device. I rescued the data and bought a Pi5 with an NVMe drive so that I can reduce the chances of data corruption.

    Version 2

    Armed with a new Pi5 with a NVMe drive and using the additional RAM as a cache to further reduce wear on the storage, I restored the backup from the older Pi3B and imported everything onto the new Pi5. I also changed the match sensitivity from 89% to 91% since I felt like I was getting a few too many false positives. I had to shut down this project in the summer of 2026 since I was about to move to a new place.

    Results

    This data was captured in the South East corner of Rheinland-Pfalz near the edge of the Pfälzerwald . Please note that the data is still considered raw and does not account for nearly impossible birds. I do live in the city so audio anomalies can still happen as there are people, kids, dogs, cats, and other animals walking nearby.

    Overview of the data

    As you can see, there are two little blips in the data. One around early September 2024 when the pi failed to reboot properly, and another one around May 2025 when the database corrupted.

    The left bar chart is the frequency using 15 minutes sampling of the birds near the device. The time chart on the top right shows the activity of the birds during the day. Finally on the bottom, there is a timeline along with activity of the birds. This 2 year project includes 53,025 unique samples that have been analyzed. I’ll go into deeper analysis for the top 10 birds.

    The Birds

    Eurasian Blackbird (Turdus merula)

    This lovely bird will usually perch on top of a tall tree or building in the evenings and seem to sing about it’s day. They are plentiful and numerous through out the city (and continent).

    Common Swift (Apus apus)

    The shrill screams of these fast flying birds indicate the start of summer. They show up from Africa around mid to late May as the weather gets warmer. In the evenings, they from screaming parties as they talk to each other about their days soaring and diving.

    Eurasian Blue Tit (Cyanistes caeruleus)

    This wide spread bird always have the very distinctive raspy chatter calls. This bird is the most popular visitor of the seed balls that I put out. Its beautiful blue and yellow plumage is a welcomed sight since they are usually here year round.

    Eurasian Magpie (Pica pica)

    These dapper black and white birds are in the Corvidae family. They are always seem so mischievous. They love the meal worms that I put out.

    European Robin (Erithacus rubecula)

    These birds seem to not shy away from humans. They are very small, yet have such a loud and distinct song. They are know as the Christmas bird in Germany.

    Great Tit (Parus major)

    A neighbor of the blue tit, it has the very distinctive high low high low songs. These also swarm the seed balls.

    Redwing (Turdus iliacus)

    These ones are a little rarer and I don’t hear these one as much. They’re quite and need a good ear to seek their calls out. Unfortunately, I did not get any good recordings of these birds.

    European Goldfinch (Carduelis carduelis)

    These birds have a very flippy and complex song. They also have bold colors and are mostly active around Spring to Summer.

    Common Wood-Pigeon (Columba palumbus)

    I usually hear these in the mid morning of warmer days. The coos of this pigeon is reminiscent of the Mourning Doves that I used to wake up to.

    Black Redstart (Phoenicurus ochruros)

    These little birds have a shrill raspy song that coincides of the start of warmer weather. They also perch on tall trees and roofs to call out to the world.

    Top 25 birds

    Detail of the time charts