Migrating from Legacy Networking to systemd-networkd on Ubuntu 24.04

2025-11-25T00:00:00+00:00

During a recent Ubuntu server upgrade, I migrated a production system from the legacy ifupdown networking stack to the modern systemd-networkd service. This migration was part of preparing several remote production servers for a smooth upgrade to Ubuntu 24.04, which relies heavily on systemd-managed components.

In this post, I’ll walk through the full migration process, how we automated it safely across multiple remote servers, and the lessons learned from implementing rollback and watchdog mechanisms for zero-downtime transitions.

Why migrate to systemd-networkd?

Ubuntu 24.04 uses systemd-networkd as the default backend for network management. The traditional ifupdown scripts (/etc/network/interfaces) are no longer the recommended way to configure networking. Some of the modern features enabled by systemd-networkd include:

Predictable network interface naming (e.g., ens3, eth0, etc.)
Built-in support for bridges, VLANs, and bonds
Faster boot times with asynchronous network handling
Unified configuration under /etc/systemd/network

Migrating now also avoids future compatibility issues and takes advantage of systemd’s integrated logging and management.

Step 1: Check the current setup

Before the migration, I confirmed the system was still using ifupdown:

ls /etc/network/interfaces
cat /etc/network/interfaces

Output showed legacy configuration similar to:

auto eth0
iface eth0 inet static
    address 10.42.41.1
    netmask 255.255.0.0

Step 2: Install and enable systemd-networkd

Since systemd-networkd wasn’t already active, I installed and enabled it:

sudo apt install systemd-networkd systemd-resolved -y
sudo systemctl enable systemd-networkd
sudo systemctl enable systemd-resolved

Then, I stopped ifupdown to prevent conflicts.

Step 3: Create a network configuration file

Each interface now has its own .network file under /etc/systemd/network/.

DHCP-based interface:

# /etc/systemd/network/10-eth0.network
[Match]
Name=eth0

[Network]
DHCP=yes

Static interface:

# /etc/systemd/network/20-eth1.network
[Match]
Name=eth1

[Network]
Address=10.42.41.1/16
Gateway=10.42.0.1
DNS=8.8.8.8
DNS=1.1.1.1

Step 4: Configure systemd-resolved

This step ensures your system’s DNS resolver works correctly with systemd-networkd.

systemd-resolved listens locally on 127.0.0.53 and handles DNS resolution, caching, and per-interface settings.

Link your system’s /etc/resolv.conf to use it:

sudo ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf

Then to verify:

resolvectl status

If we skip this, the system might lose DNS resolution after reboot. Applications like apt, curl, and ping may fail to resolve hostnames even though interfaces are up.

Restart and verify

Restart both services:

sudo systemctl restart systemd-networkd
sudo systemctl restart systemd-resolved

Check active links:

networkctl list

Detailed view for one interface:

networkctl status eth0

Sample output:

● 2: eth0
       Type: ether
      State: routable (configured)
     Address: 10.42.41.1/16
     Gateway: 10.42.0.1
     DNS: 8.8.8.8

Test:

ping -c 3 8.8.8.8
ping -c 3 google.com

Step 6: Reboot and confirm

Reboot once to ensure everything persists. After the reboot,

systemctl status systemd-networkd
systemctl status systemd-resolved

and confirm interface and DNS are still functional.

Handling remote migration and downtime risks

Performing this migration remotely on production systems without physical console access required special care. I prepared several layers of protection to prevent accidental lockouts.

Console or IPMI verification: Ensured each node had out-of-band access in case SSH connectivity was lost.

Timed rollback safety script: A background process automatically reverted to legacy networking if the new configuration didn’t come up within a few minutes.

#!/bin/bash
sleep 300
if ! ping -c1 8.8.8.8 &>/dev/null; then
    systemctl disable systemd-networkd
    systemctl enable networking
    reboot
fi

Automating the migration and rollback process

To streamline upgrades across multiple servers, I created automation scripts that handle migration, rollback, and ongoing monitoring.

migrate_to_systemd_networkd.sh

This script performs a complete, logged migration with built-in validation and rollback. It checks for valid .network files, disables legacy services, enables systemd-networkd, tests connectivity, and if all retries fail, restores the old configuration automatically.

Key features

Validates configuration syntax using systemd-analyze verify
Masks conflicting services (ifup@eth*, NetworkManager)
Performs up to three connectivity tests
Restarts SSH and dependent services
Schedules a controlled reboot or user-confirmed one
Rolls back cleanly if connectivity fails

upgrade_watchdog.sh

During do-release-upgrade, networking may restart multiple times. This watchdog script runs in the background to ensure systemd-networkd remains active and services like SSH come back automatically.

Executes every 10 minutes
Verifies /etc/systemd/network/*.network exists and services are running as expected if it fails, this script restarts the service
Restarts systemd-networkd, ssh, and dependent services

Together, both scripts allowed us to perform fully remote OS upgrades with zero manual downtime and consistent recovery paths.

Conclusion

Migrating to systemd-networkd modernizes Ubuntu servers and simplifies long-term maintenance. With the automation scripts and safety mechanisms in place, we successfully migrated multiple remote servers with no downtime and automatic rollback protection.

This upgrade not only prepares infrastructure for Ubuntu 24.04 but also integrates cleanly with our automation and monitoring systems, ensuring reliable networking for years ahead.

Making a Loading Spinner with tkinter

2024-03-05T00:00:00+00:00

When you need a loading spinner, you really need a loading spinner. Interested in putting something on the screen without installing a pile of dependencies, I reached deep into the toolbox of the Python standard library, dug around a bit, and pulled out the tkinter module.

The tkinter module is an interface to the venerable Tcl/Tk GUI toolkit, a cross-platform suite for creating user interfaces in the style of whatever operating system you run it on. It’s the only built-in GUI toolkit in Python, but there are many worthy alternatives available (see the end of the post for a list).

Here I’ll demonstrate how to make a loading spinnner with tkinter on Ubuntu 22.04. It should work on any platform that runs Python, with some variations when setting up the system for it.

Prerequisites

My vision for the loading spinner is some spinning dots and a logo, since this is such a convenient branding opportunity. To accomplish this we’ll be extending tkinter with Pillow’s ImageTk capability, which can load a PNG with transparency.

To produce that PNG with transparency, first we may need to rasterize an SVG file, because wise designers work in vectors. This is made trivial by Inkscape, a free and complete vector graphics tool:

$ inkscape logo.svg -o logo.png

With the logo in hand, we can move on to setting up our dependencies. Ubuntu’s python3 distribution doesn’t include tkinter by default, so we’ll need to install it explicitly, along with Pillow’s separate ImageTk support:

$ sudo apt install python3-tk python3-pil.imagetk

This may occupy up to 75MB, if Python was already installed. This was still the smallest apt footprint of all of the Python GUI libraries in consideration. Pygame was also a strong contender.

Code

Let’s start with something simple: putting the logo on the screen.

#!/usr/bin/env python3

from PIL import Image, ImageTk
from tkinter import BOTH, Canvas, Tk


# Desired dimensions of our window.
WIDTH, HEIGHT = 500, 500

if __name__ == "__main__":
    # Create the root window object.
    root = Tk()
    # Create a canvas for drawing our graphics.
    canvas = Canvas(root, width=WIDTH, height=HEIGHT, background="black")
    # Fill the entire window with the canvas.
    canvas.pack(fill=BOTH, expand=1)

    # Load the logo PNG with regular PIL.
    logo_img = Image.open("logo.png")
    # Convert the logo to an ImageTk PhotoImage.
    logo_pi = ImageTk.PhotoImage(logo_img)
    # Add our logo image to the canvas.
    canvas.create_image(
        WIDTH / 2,
        HEIGHT / 2,
        image=logo_pi,
    )

    # Run the tkinter main loop.
    root.mainloop()

This puts the logo in the center of a window, but the logo may be too large or small. Let’s scale it according to the window dimensions, let’s say to about ⅔ of the width so we have some room for spinning dots:

#!/usr/bin/env python3

from PIL import Image, ImageTk
from tkinter import BOTH, Canvas, Tk


# Desired dimensions of our window.
WIDTH, HEIGHT = 500, 500

if __name__ == "__main__":
    # Create the root window object.
    root = Tk()
    # Create a canvas for drawing our graphics.
    canvas = Canvas(root, width=WIDTH, height=HEIGHT, background="black")
    # Fill the entire window with the canvas.
    canvas.pack(fill=BOTH, expand=1)

    # Load the logo PNG with regular PIL.
    logo_img = Image.open("logo.png")
    # Resize the logo to about 2/3 the window width.
    scaled_w = round(WIDTH * 0.6)
    scaled_h = round(scaled_w / (logo_img.width / logo_img.height))
    logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
    # Convert the logo to an ImageTk PhotoImage.
    logo_pi = ImageTk.PhotoImage(logo_img)
    # Add our logo image to the canvas.
    canvas.create_image(
        WIDTH / 2,
        HEIGHT / 2,
        image=logo_pi,
    )

    # Run the tkinter main loop.
    root.mainloop()

That’s better. Now let’s add the promised spinning dots. We’ll draw some ovals on the canvas and modify our main loop to animate them:

#!/usr/bin/env python3

import math
import time

from PIL import Image, ImageTk
from tkinter import BOTH, Canvas, Tk


# Desired dimensions of our window.
WIDTH, HEIGHT = 500, 500
# Coordinates of the center.
CENTER_X, CENTER_Y = WIDTH / 2, HEIGHT / 2
# How many spinning dots we want.
NUM_DOTS = 8

if __name__ == "__main__":
    # Create the root window object.
    root = Tk()
    # Create a canvas for drawing our graphics.
    canvas = Canvas(root, width=WIDTH, height=HEIGHT, background="black")
    # Fill the entire window with the canvas.
    canvas.pack(fill=BOTH, expand=1)

    # Load the logo PNG with regular PIL.
    logo_img = Image.open("logo.png")
    # Resize the logo to about 2/3 the window width.
    scaled_w = round(WIDTH * 0.6)
    scaled_h = round(scaled_w / (logo_img.width / logo_img.height))
    logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
    # Convert the logo to an ImageTk PhotoImage.
    logo_pi = ImageTk.PhotoImage(logo_img)
    # Add our logo image to the canvas.
    canvas.create_image(
        CENTER_X,
        CENTER_Y,
        image=logo_pi,
    )

    # Radius in pixels of a single dot.
    dot_radius = WIDTH * 0.05
    # Radius of the ring of dots from the center of the window.
    dots_radius = WIDTH / 2 - dot_radius * 2

    # Helper function to calculate dot position on each update.
    def get_dot_coords(n: int, t: float):
        """Get the x0, y0, x1, y1 coords of dot at index 'n' at time 't'."""
        angle = (n / NUM_DOTS) * math.pi * 2 + t
        x = math.cos(angle) * dots_radius + CENTER_X
        y = math.sin(angle) * dots_radius + CENTER_Y
        return x - dot_radius, y - dot_radius, x + dot_radius, y + dot_radius

    # Create all the dots.
    t0 = time.monotonic()
    for n in range(NUM_DOTS):
        coords = get_dot_coords(n, t0)
        canvas.create_oval(
            *coords,
            fill="#888888",
            width=0,  # Border width.
            tags=f"dot_{n}",
        )

    # Set up a custom main loop to animate the moving dots.
    while True:
        # Check the time of this update.
        t = time.monotonic()
        for n in range(NUM_DOTS):
            # Get the desired coords for this dot at this time.
            coords = get_dot_coords(n, t)
            # Move the dot on the canvas, finding it by its tag.
            canvas.coords(
                f"dot_{n}",
                *coords,
            )
        # Call the required tkinter update function.
        root.update()
        # Attempt to stabilize the timing of this loop by targeting 60Hz.
        while t0 < t:
            t0 += 1 / 60
        time.sleep(t0 - t)

You may notice that the dots don’t look all that great. There’s no anti-aliasing when drawing shape primitives in tkinter, so the edges look jagged compared to our well-scaled logo image. One hack is to layer slightly larger and dimmer shapes under each object, which you might do like so:

#!/usr/bin/env python3

import math
import time

from PIL import Image, ImageTk
from tkinter import BOTH, Canvas, Tk


# Desired dimensions of our window.
WIDTH, HEIGHT = 500, 500
# Coordinates of the center.
CENTER_X, CENTER_Y = WIDTH / 2, HEIGHT / 2
# How many spinning dots we want.
NUM_DOTS = 8
# Colors for each layer of fake anti-aliasing around each dot.
# Must be in order from back to front.
COLORS = ["#888888", "#BBBBBB", "#FFFFFF"]

if __name__ == "__main__":
    # Create the root window object.
    root = Tk()
    # Create a canvas for drawing our graphics.
    canvas = Canvas(root, width=WIDTH, height=HEIGHT, background="black")
    # Fill the entire window with the canvas.
    canvas.pack(fill=BOTH, expand=1)

    # Load the logo PNG with regular PIL.
    logo_img = Image.open("logo.png")
    # Resize the logo to about 2/3 the window width.
    scaled_w = round(WIDTH * 0.6)
    scaled_h = round(scaled_w / (logo_img.width / logo_img.height))
    logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
    # Convert the logo to an ImageTk PhotoImage.
    logo_pi = ImageTk.PhotoImage(logo_img)
    # Add our logo image to the canvas.
    canvas.create_image(
        CENTER_X,
        CENTER_Y,
        image=logo_pi,
    )

    # Radius in pixels of a single dot.
    dot_radius = WIDTH * 0.05
    # Radius of the ring of dots from the center of the window.
    dots_radius = WIDTH / 2 - dot_radius * 2

    # Helper function to calculate dot position on each update.
    def get_dot_coords(n: int, t: float, c: int):
        """Get the x0, y0, x1, y1 coords of dot at index 'n' at time 't'.
        Inflate the radius by color index 'c'."""
        angle = (n / NUM_DOTS) * math.pi * 2 + t
        x = math.cos(angle) * dots_radius + CENTER_X
        y = math.sin(angle) * dots_radius + CENTER_Y
        # Invert the color index and add to the radius.
        radius = dot_radius + (len(COLORS) - c) * 0.75
        #radius = dot_radius + c
        return x - radius, y - radius, x + radius, y + radius

    # Create all the dots.
    t0 = time.monotonic()
    for c, color in enumerate(COLORS):
        for n in range(NUM_DOTS):
            coords = get_dot_coords(n, t0, c)
            canvas.create_oval(
                *coords,
                fill=color,
                width=0,  # Border width.
                tags=f"dot_{c}_{n}",
            )

    # Set up a custom main loop to animate the moving dots.
    while True:
        # Check the time of this update.
        t = time.monotonic()
        for c, color in enumerate(COLORS):
            for n in range(NUM_DOTS):
                # Get the desired coords for this dot at this time.
                coords = get_dot_coords(n, t, c)
                # Move the dot on the canvas, finding it by its tag.
                canvas.coords(
                    f"dot_{c}_{n}",
                    *coords,
                )
        # Call the required tkinter update function.
        root.update()
        # Attempt to stabilize the timing of this loop by targeting 60Hz.
        while t0 < t:
            t0 += 1 / 60
        time.sleep(t0 - t)

The fake anti-aliasing was a fun exercise, but for this use case you’ll probably get better-looking results out of scaling a PNG asset like we did the logo.

Resources

If you’re interested in learning more about tkinter, see also:

Other Python GUI/graphics toolkits you might consider:

Pygame
Pyglet
PySide6 (official Qt binding)
PyQt (unofficial Qt binding)
wxPython

Ubuntu Touch on a Galaxy S7 & a Pixel 3a

2023-08-29T00:00:00+00:00

I have a shoebox in my closet I call my “phone graveyard.” At times I’ve had five or more old phones in there, in various states of decay—some have fully shattered screens, some broken USB ports, etc. But some still have quite usable hardware, their main drawback being how slowly they run modern versions of Android or iOS, and the lack of support for modern features.

It has always troubled me to have such incredible devices, with way more computing power than, say, a Raspberry Pi, gather dust because of software limitations. Even if I don’t use an old phone for daily use any more, what if I could use it as a DNS server (like a Pi-hole), or as a camera or media player?

When I heard about Ubuntu Touch, it seemed like the perfect OS to bring back some long-term functionality to these old devices. Originally created by Canonical, it was soon abandoned but revived by UBports, who started community development in 2015. They actively maintain the OS for around 80 devices, including two I have in my phone graveyard: a Samsung Galaxy S7 and a Google Pixel 3a.

Installation

Installing Ubuntu Touch is quite straightforward; the UBports installer does most of the heavy lifting. See the UBports website for instructions.

For the Pixel 3a, I had to flash an older version of the stock OS before running the UBports installer. The Ubuntu Touch device page for the Pixel 3a specifies which version to flash under “Preparatory Step,” but this seems to be in a different place for each device. Read the device-specific instructions carefully before installing.

Usability

The native app landscape for Ubuntu Touch is, as you would expect, pretty limited. You won’t be able to play most DRM content, and while there are web apps for Uber, Discord, and others, they tend to be various levels of buggy. You can see the apps, including helpful usability reports, at open-store.io.

There are plenty of bugs I encountered while briefly testing, like having no volume indicator when using volume keys and not seeing the Libertine installation which should’ve been there, making it hard to install programs with the terminal emulator.

Audio playback had fairly frequent interruptions, at least on my Pixel 3a, making it not viable as a music player. However, for audiobooks or podcasts this probably wouldn’t be a big deal.

However, most of the base functionality for web browsing, photos, and media playback seemed to work well.

The inclusion of a fully-featured terminal emulator expands the possibilities greatly, allowing your old phone to act in place of a Raspberry Pi for projects like a Pi-hole.

The system is, by default, in read-only mode, requiring containers (the intent is to run the custom container system, Libertine) to run most apps. However, you can change it to read-write for a more traditional Linux system. For more info on terminal capability, see the UBports blog.

While I wouldn’t want to run Ubuntu Touch as my only smartphone OS, it is an exciting option for secondary or old devices, especially if you want to contribute to open-source projects and help out the community!

GStreamer Nvenc for Ubuntu 20.04

2021-04-22T00:00:00+00:00

Image by Martin Adams on Unsplash

GStreamer is a library for creating media-handling components. Using GStreamer you can screencast your desktop, transcode a live stream, or write a media player application for your kiosk.

Video encoding is expensive, even with AMD’s current lineup making it more palatable. Recent Nvidia Quadro and GeForce video cards include dedicated H.264 encoding and decoding hardware as a set of discrete components alongside the GPU. The hardware is used in the popular Shadowplay toolkit on Windows and available to developers through the Nvidia Video SDK on Linux.

GStreamer includes elements as part of the “GStreamer Bad” plugin set that leverages the SDK without having to get your hands too dirty. The plugins are not included with gst-plugins-bad in apt, and must be compiled with supporting libraries from Nvidia. Previously this required registering with Nvidia and downloading the Nvidia Video SDK, but Ubuntu recently added apt packages providing them, a big help for automation.

Environment

CUDA

The nvenc and nvdec plugins depend on CUDA 11. The apt version is too old. I’ve found the runfile to be the most reliable installation method. Deselect the nvidia drivers when using the runfile if using the distro-maintained driver.

Plugin

Install prerequisites from apt:

$ apt install nvidia-driver-460 libnvidia-encode-460 libnvidia-decode-460 libdrm-dev

Clone gst-plugins-bad source matching distro version:

$ git clone --single-branch -b 1.16.2 git://anongit.freedesktop.org/gstreamer/gst-plugins-bad
$ cd gst-plugins-bad

Compile and install plugins:

$ ./autogen.sh --with-cuda-prefix="/usr/local/cuda"
$ cd sys/nvenc
$ make
$ cp .libs/libgstnvenc.so /usr/lib/x86_64-linux-gnu/gstreamer-1.0/
$ cd ../nvdec
$ make
$ cp .libs/libgstnvdec.so /usr/lib/x86_64-linux-gnu/gstreamer-1.0/

Clear GStreamer cache and check for dependency issues using gst-inspect:

$ rm -r ~/.cache/gstreamer-1.0
$ gst-inspect-1.0 | grep 'nvenc\|nvdec'
nvenc:  nvh264enc: NVENC H.264 Video Encoder
nvenc:  nvh265enc: NVENC HEVC Video Encoder
nvdec:  nvdec: NVDEC video decoder

Benchmark

Here is an example pipeline using the standard CPU-based H.264 encoder to encode 10000 frames at 320x240:

$ gst-launch-1.0 videotestsrc num-buffers=10000 ! x264enc ! h264parse ! mp4mux ! filesink location=vid1.mp4

On my modest machine, this took around 9.6 seconds and 400% CPU.

Running the same pipeline with the nvenc element:

$ gst-launch-1.0 videotestsrc num-buffers=10000 ! nvh264enc ! h264parse ! mp4mux ! filesink location=vid2.mp4

About 2.3 seconds with 100% CPU.

Alternatives

The apt-supported version of these plugins is limited to H.264 and 4K pixels in either dimension. Features have been fleshed out upstream. Elements for the Nvidia Tegra line of mobile processors provide more features, but the required hardware probably isn’t included with your workstation.

FFmpeg also provides hardware accelerated elements, including nvcodec supported H.264 and HEVC encoders and decoders, out of the box on Ubuntu.

Installing Ubuntu 18.04 to a different partition from an existing Ubuntu installation

2020-04-06T00:00:00+00:00

Photo by Patryk Grądys on Unsplash

Our Liquid Galaxy systems are running on Ubuntu 14.04 LTS (Trusty). We decided to upgrade them to Ubuntu 18.04 LTS (Bionic) since Ubuntu 14.04 LTS reached its end of life on April 30, 2019.

Upgrading from Ubuntu 14.04 LTS

The recommended way to upgrade from Ubuntu 14.04 LTS is to first upgrade to 16.04 LTS, then to 18.04 LTS, which will continue to receive support until April 2023:

14.04 LTS → 16.04 LTS → 18.04 LTS

Ubuntu has LTS → LTS upgrades, allowing you to skip intermediate non-LTS releases, but we can’t skip intermediate LTS releases; we have to go via 16.04, unless we want to do a fresh install of 18.04 LTS.

For a little more longevity, we decided to do a fresh install of Ubuntu 18.04 LTS. Not only is this release supported into 2023 but it will offer a direct upgrade route to Ubuntu 20.04 LTS when it’s released in April 2020.

Installing Clean Ubuntu 18.04 LTS from Ubuntu 14.04 LTS

Install debootstrap

The debootstrap utility installs a very minimal Debian system. Debootstrap will install a Debian-based OS into a sub-directory. You don’t need an installation CD for this. However, you need to have access to the corresponding Linux distribution repository (e.g. Debian or Ubuntu).

apt-get update
apt-get -y install debootstrap

Creating a new root partition

Create a logical volume with size 12G and format the filesystem to ext4:

lvcreate -L12G -n ROOT_VG/ROOT_VOLUME
mkfs.ext4 /dev/ROOT_VG/ROOT_VOLUME

Mounting the new root partition

Mount the partition at /mnt/root18. This will be the root (/) of your new system.

mkdir -p /mnt/root18
mount /dev/ROOT_VG/ROOT_VOLUME /mnt/root18

Bootstrapping the new root partition

Debootstrap can download the necessary files directly from the repository. You can substitute any Ubuntu archive mirror for ports.ubuntu.com/ubuntu-ports in the command example below. Mirrors are listed here.

Replace $ARCH below with your architecture: amd64, arm64, armhf, i386, powerpc, ppc64el, or s390x.

debootstrap --arch $ARCH $DISTRO $ROOT_MOUNTPOINT
debootstrap --arch amd64 bionic /mnt/root18

Installing fstab

This just changes the root (/) partition path in the new installation while keeping the /boot partition intact. For example, /dev/mapper/headVG-root / → /dev/mapper/headVG-root18 /. Since device names are not guaranteed to be the same after rebooting or when a new device is connected, we use UUIDs (Universally Unique Identifiers) to refer to partitions in fstab. We don’t need to use UUIDs for logical volumes since their device names won’t change.

OLD_ROOT_PATH="$(awk '$2 == "/" { print $1 }' /etc/fstab)"
sed "s:^${OLD_ROOT_PATH}\s:/dev/mapper/headVG-root18 :" /etc/fstab > /mnt/root18/etc/fstab

Mounting things in the new root partition

Bind /dev to the new location, then mount /sys, /proc, and /dev/pts from your host system to the target system.

mount --bind /dev /mnt/root18/dev
mount -t sysfs none /mnt/root18/sys
mount -t proc none /mnt/root18/proc
mount -t devpts none /mnt/root18/dev/pts

Configuring apt

Debootstrap will have created a very basic /mnt/root18/etc/apt/sources.list that will allow installing additional packages. However, I suggest that you add some additional sources, such as the following, for source packages and security updates:

echo "deb http://us.archive.ubuntu.com/ubuntu bionic main universe
deb-src http://us.archive.ubuntu.com/ubuntu bionic main universe
deb http://security.ubuntu.com/ubuntu bionic-security main universe
deb-src http://security.ubuntu.com/ubuntu bionic-security main universe" > /mnt/root18/etc/apt/sources.list

Make sure to run apt update with chroot after you have made changes to the target system sources list.

Now we’ve got a real Ubuntu system, if a rather small one, on disk. chroot into it to set up the base configurations.

LANG=C.UTF-8 chroot /mnt/root18 /bin/bash

Installing required packages and running chef-client

As we are maintaining most of the Liquid Galaxy configuration and packages with Chef, we need to install chef-client, configure it on the new target system, and run chef-client to complete the setup.

Copy the chef configuration and persistent net udev rules into place:

cp -a /etc/chef /mnt/root18/etc/
cp /etc/udev/rules.d/70-persistent-net.rules /mnt/root18/etc/udev/rules.d/

Install and run chef-client and let it set up our user login:

cat <<EOF | chroot /mnt/root18
apt-get update && apt-get install -y curl wget
curl -L https://omnitruck.chef.io/install.sh | bash -s -- -v 12.5.1
chef-client -E production_trusty -o 'recipe[users]'
EOF

Next, chroot and install the required packages:

cat <<EOF | chroot /mnt/root18
mount /boot
apt-get update && apt-get install -y --no-install-recommends linux-image-generic lvm2 openssh-server ifupdown net-tools
locale-gen en_US.UTF-8
EOF

Set Ubuntu 14.04 to boot default

Create file 42_custom_template_trusty:

#!/bin/sh
# Entry for trusty system
menuentry 'TRUSTY' --class liquid --class gnu-linux --class gnu --class os {
        # Skipped lines
        linux   /trusty/vmlinuz-$trusty_kernel_version root=/dev/mapper/headVG-root ro nomodeset biosdevname=0 modprobe.blacklist=gma500_gfx quiet
        initrd  /trusty/initrd.img-$trusty_kernel_version
}

Create file 42_custom_template_bionic:

#!/bin/sh
# Entry for bionic system
menuentry 'BIONIC' --class liquid --class gnu-linux --class gnu --class os {
        # Skipped lines
        linux   /vmlinuz-$bionic_kernel_version-generic root=/dev/mapper/headVG-root18 ro nomodeset net.ifnames=0 biosdevname=0 modprobe.blacklist=gma500_gfx quiet
        initrd  /initrd.img-$bionic_kernel_version-generic
}

On the Current system (Trusty):

Back up the current Trusty kernel files into /boot/trusty and create a custom menu entry configuration for Ubuntu 14.04 on 42_custom_trusty. Update /etc/default/grub to set Ubuntu 14.04 as the default menu entry and run update-grub to apply it to the current system. This will be used as a fail-safe method to run Trusty again if there is a problem with the new installation.

mkdir -vp /boot/trusty
cp -v /boot/*-generic /boot/trusty/

envsubst '${trusty_kernel_version}' < 42_custom_template_trusty > /etc/grub.d/42_custom_template_trusty
chmod +x /etc/grub.d/42_custom_template_trusty

sed -i 's/GRUB_DEFAULT=.*/GRUB_DEFAULT="TRUSTY"/' /etc/default/grub
update-grub

On the target system (Bionic):

Create the custom menu entry for Ubuntu 14.04 and Ubuntu 18.04 on the target system.

mkdir -p /mnt/root18/etc/grub.d
envsubst '${trusty_kernel_version}' < 42_custom_template_trusty > /mnt/root18/etc/grub.d/42_custom_template_trusty
envsubst '${bionic_kernel_version}' < 42_custom_template_bionic > /mnt/root18/etc/grub.d/42_custom_template_bionic
chmod +x /mnt/root18/etc/grub.d/42_custom_template_{trusty,bionic}

chroot into the target system and update /etc/default/grub to set Ubuntu 14.04 as the default menu entry and run update-grub. This will also update the GRUB configuration to boot Ubuntu 14.04 as default and update the 0th menu entry to Ubuntu 18.04 (Bionic).

cat <<EOF | chroot /mnt/root18
sed -i 's/GRUB_DEFAULT=.*/GRUB_DEFAULT="TRUSTY"/' /etc/default/grub
update-grub
EOF

Boot into Bionic

To boot into Ubuntu 18.04 (Bionic), reboot the system after grub-reboot bionic and test if the bionic system is working as expected.

$ grub-reboot bionic
$ reboot

Reboot and test our new 0th GRUB entry:

$ grub-reboot 0
$ reboot

A normal reboot returns to Ubuntu 14.04 (Trusty) since the default menu entry is still set to Ubuntu 14.04 (Trusty).

Set Ubuntu 18.04 to boot default

To set our new Ubuntu 18.04 installation as the default menu entry, change GRUB_DEFAULT to 0 in /etc/default/grub and run update-grub to apply it. The next reboot will boot into Ubuntu 18.04.

sed -i 's/GRUB_DEFAULT=.*/GRUB_DEFAULT=0/' /etc/default/grub
update-grub

Congratulations! You now have a freshly installed Ubuntu 18.04 system.

How to set up a local development environment for WordPress from scratch

2019-08-07T00:00:00+00:00

I recently got pulled into a project for a client who wanted to have a new WordPress website developed for them. I started by setting up a development environment with the niceties that I’m used to from my other application development work. That is, a development server, interactive debugging, linting, and a good editor.

Another thing that I wanted was not to have to deal with LAMP or WAMP or XAMPP or any of that. I wanted a clean, from scratch installation where I knew and controlled everything that was there. I’ve got nothing against those packages, but I think that, by setting up everything manually, I’d be able to better learn the technology as I would know exactly how everything is set up under the hood. The shortcuts could come later.

Luckily for me, there aren’t many pieces when it comes to setting up a basic, running development environment for WordPress. You only need three things: 1. MySQL, 2. PHP, and 3. WordPress itself. I also wanted a few other goodies and we’ll get there.

Let’s go through the steps that I took to set all of this up:

1. Set up PHP

In Ubuntu, installing PHP is easy enough. Just run the following command:

sudo apt-get install php

After that’s done, run php -v to validate that it was successfully installed. It should result in something like this:

PHP 7.2.19-0ubuntu0.18.04.1 (cli) (built: Jun  4 2019 14:48:12) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.19-0ubuntu0.18.04.1, Copyright (c) 1999-2018, by Zend Technologies

There’s one particular PHP extension that we’re going to need. Let’s install it with:

sudo apt-get install php-mysql

The php-mysql extension is necessary for our PHP installation to interact with MySQL. That’s all that’s needed to run WordPress as far as PHP is concerned.

2. Set up MySQL

WordPress uses MySQL for all of its data storage concerns. So, let’s install it. Again, in Ubuntu, installing and setting up MySQL is super easy. First, we need to run this command:

sudo apt-get install mysql-server

This will install both the MySQL database engine and a command-line client for us to connect to it and do some initial configuration. We now need to log into our freshly installed MySQL instance. But first, make sure that it’s running with:

sudo service mysql start

Now that our instance is running, log into it as root with:

sudo mysql -u root

This will open up the MySQL command-line client where we can do some initial configuration to support WordPress.

We now need to create a new MySQL user that will be used by WordPress to log into the database. You can do so with a command like this from within the MySQL CLI client:

CREATE USER 'wordpress_user'@'localhost' IDENTIFIED BY 'password';

Obviously, choose a username and password that work for you. I like to keep things simple and obvious so that’s what I use. Also obviously, use a strong, unique password in a production environment.

Now, create a new database that will be used by WordPress with:

CREATE DATABASE wordpress_dev;

Again, feel free to choose a name that suits your needs.

Now, we need to allow the user that we created a few steps ago to access and control that new database. Since this is only a dev environment, let’s just give our WordPress user access to everything. This can be done with:

GRANT ALL ON *.* TO 'wordpress_user'@'localhost'

With that, we’re done with MySQL, we can close the CLI client with the exit command.

3. Set up WordPress

Now that we have our prerequisites ready, we can proceed to setting up the actual WordPress site. It turns out, this is pretty easy as well. Get a new directory ready and let’s get started.

First, we need to download the package of WordPress files from the official site. In Ubuntu, this can be done with this command:

wget https://wordpress.org/latest.tar.gz

This will result in a new latest.tar.gz file being created in your directory. Now, extract it with:

tar xvzf latest.tar.gz

Then, enter the new wordpress directory that gets created as a result of the last operation. Explore the wordpress directory and you should be able to see a bunch of wp-* files and directories. These are all the files that WordPress needs to run.

Before running WordPress though, we need to configure it so that it uses the MySQL database that we just created. We do this by specifying that configuration in a wp-config.php file. This file does not exist yet, but we have a wp-config-sample.php file that we can use as a template. Create the new wp-config.php file based on wp-config-sample.php with the following command:

cp wp-config-sample.php wp-config.php

Now, in the new wp-config.php file, put in your MySQL database information. Starting around line 23, it should look like this:

/** The name of the database for WordPress */
define( 'DB_NAME', 'wordpress_dev' );

/** MySQL database username */
define( 'DB_USER', 'wordpress_user' );

/** MySQL database password */
define( 'DB_PASSWORD', 'password' );

/** MySQL hostname */
define( 'DB_HOST', 'localhost' );

/** Database Charset to use in creating database tables. */
define( 'DB_CHARSET', 'utf8' );

/** The Database Collate type. Don't change this if in doubt. */
define( 'DB_COLLATE', '' );

4. Run WordPress

Now we’re finally ready to actually run WordPress. WordPress, as a web application, needs a web server like Apache to run. We don’t have Apache though; what we have is PHP’s built-in development web server. From within our wordpress directory, we can fire up the built-in web server with:

php -S localhost:3000

Now it’s just a matter of navigating to the localhost:3000/wp-admin/install.php page in your browser of choice. This page should show up on screen:

Just follow the steps within the wizard at install.php and your new development WordPress site will be ready to go in no time.

Bonus 1: Set up a linter: PHP Code Sniffer

Prerequisites: Composer and the php-xml extension

You can install Composer by following these instructions. In Ubuntu, installing the php-xml extension can be done with:

sudo apt-get install php-xml

Now, to set up the PHP Code Sniffer linter, just follow these steps, from your wordpress directory:

Install PHP Code Sniffer with composer require --dev squizlabs/php_codesniffer.
Download the WordPress PHP Code Sniffer standards with composer require --dev wp-coding-standards/wpcs.
Install it into phpcs with: vendor/bin/phpcs --config-set installed_paths vendor/wp-coding-standards/wpcs.
Sniff something with vendor/bin/phpcs --standard=WordPress index.php.

If all goes well, you should be able to see reports like this (which is exaggerated for demonstration purposes; your default index.php file will not show as many warnings):

FILE: /home/kevin/projects/random/wordpress/index.php
-----------------------------------------------------------------------------------------
FOUND 5 ERRORS AND 1 WARNING AFFECTING 2 LINES
-----------------------------------------------------------------------------------------
 14 | ERROR   | [x] Expected 1 spaces after opening bracket; 0 found
 14 | ERROR   | [x] Expected 1 spaces before closing bracket; 0 found
 17 | WARNING | [x] "require" is a statement not a function; no parentheses are required
 17 | ERROR   | [x] Expected 1 spaces after opening bracket; 0 found
 17 | ERROR   | [x] Expected 1 spaces before closing bracket; 0 found
 17 | ERROR   | [x] Concat operator must be surrounded by a single space
-----------------------------------------------------------------------------------------
PHPCBF CAN FIX THE 6 MARKED SNIFF VIOLATIONS AUTOMATICALLY
-----------------------------------------------------------------------------------------

Time: 103ms; Memory: 10MB

Bonus 2: Setup interactive debugging with VS Code

Prerequisites: VS Code, XDebug and the PHP Debug VS Code extension

For writing PHP, my editor of choice is VS Code. You can get the editor from the official download site. VS Code has a huge extensions ecosystem where you can find almost anything. Naturally, there’s a debugger for PHP. It’s aptly called PHP Debug. See the instructions on how to set it up. The PHP Debug extension works on top of XDebug, a debugger for PHP. Luckily for us, PHP Debug’s instructions page includes all the details on how to install and set up both itself and XDebug.

If you followed the instructions, you should now have a new .vscode/launch.json file within your wordpress directory with the following contents:

{
   "version": "0.2.0",
   "configurations": [
       {
           "name": "Listen for XDebug",
           "type": "php",
           "request": "launch",
           "port": 9000
       }
   ]
}

This is what’s called a launch configuration in VS Code speak. This tells VS Code’s debugger all the info it needs to attach to a running PHP process. To see it in action, fire up the built-in web development server with:

php -S localhost:3000

Then, click the Start Debugging button. That’s the green triangle icon near the top of the screen, when you select the Debug sidebar in VS Code. It should have the Listen for XDebug option selected.

Now it’s just a matter of putting a breakpoint anywhere within the source code and request the page from your browser. You can set a breakpoint by clicking right next to the line number indicator, within any file. Here, I’ve put a breakpoint on line 14 in index.php:

When the code execution hits the breakpoint, it should stop right there and allow you to inspect variables and the like, just like any other debugger.

And that’s all for now! Hopefully this little write-up can help you jumpstart your next WordPress development project.

Install Tested Packages on Production Server

2015-09-07T00:00:00+00:00

One of our customers has us to do scheduled monthly OS updates following a specific rollout process. First week of the month, we will update the test server and wait for a week to confirm that everything looks as expected in the application; then next week we apply the very same updates to the production servers.

Since not long ago we used to use aptitude to perform system updates. While doing the update on the test server, we also executed aptitude on production servers to “freeze” the same packages and version to be updated on following week. That helped to ensure that only tested packages would have been updated on the production servers afterward.

Since using aptitude in that way wasn’t particularly efficient, we decided to use directly apt-get to stick with our standard server update process. We still wanted to keep our test-production synced updated process cause software updates released between the test and the production server update are untested in the customer specific environment. Thus we needed to find a method to filter out the unneeded packages for the production server update.

In order to do so we have developed a shell script that automates the process and maintain the package version in test and production in sync during OS updates. I’ll explain the processes involved used on both the test and the production servers.

Test Server Update Process:

Update the repository index
Get the list of installed packages and version before update
Complete the dist-upgrade
Get the list of installed packages and version after update
Compare the packages before and after the update and generate a diff output which has the information about the packages installed, updated and removed

Example:

root@selva_test:~# ./auto-tested-server-update.sh -t test

    ##########################################################
    ############ Auto Tested Server Update Script ############
    ##########################################################

Packages index list and server update will be completed here...

Files were generated in default directory. after_* and before_* contains the installed packages list before and after server update. diff_* contains the changes of the packages in the system.

root@selva_test:~# ls tmp/
after_update_selva_test.2015-09-02.txt  before_update_selva_test.2015-09-02.txt  diff_server_update_selva_test.2015-09-02.txt

# head -5 tmp/before_update_selva_test.2015-09-02.txt 
accountsservice=0.6.15-2ubuntu9.6
acl=2.2.51-5ubuntu1
acpi-support=0.140.1
acpid=1:2.0.10-1ubuntu3
activity-log-manager-common=0.9.4-0ubuntu3.2

# head -5 tmp/after_update_selva_test.2015-09-02.txt 
accountsservice=0.6.15-2ubuntu9.6
acl=2.2.51-5ubuntu1
acpi-support=0.140.1
acpid=1:2.0.10-1ubuntu3
activity-log-manager-common=0.9.4-0ubuntu3.2

# cat tmp/diff_server_update_selva_test.2015-09-02.txt 
             > git=1:1.7.9.5-1ubuntu0.1
             > git-man=1:1.7.9.5-1ubuntu0.1
metacity=1:2.34.1-1ubuntu11          <
mysql-client=5.43-0ubuntu0.12.04.1         | mysql-client=5.5.43-0ubuntu0.12.04.1
ubuntu-desktop=1.267.1           <
unity-2d=5.14.0-0ubuntu1          <
             > wget=1.13.4-2ubuntu1

Production Server Update Process:

Get the diff output file (which has information about packages installed, updated and removed packages) from the test server
Collect the packages which will be changed on server update
Process the packages to categorize under install, upgrade and remove
Filter the packages from test server installed, upgraded and removed packages
Perform the install/upgrade/remove actions based on the filtered tested packages

Example:

root@selva_prod:~# ./auto-tested-server-update.sh -t prod -S selva_test -U root

    ##########################################################
    ############ Auto Tested Server Update Script ############
    ##########################################################

*** Production Server Update Process ***

      remote_user: root
      remote_server: selva_prod
      remote_path: /root/tmp
    
1./root/tmp/diff_server_update_selva_test.2015-09-02.txt
Enter a number to choose file: 1
INFO: You have selected the file: diff_server_update_selva_test.2015-09-02.txt
INFO: scp root@selva_test:/root/tmp/diff_server_update_selva_test.2015-09-02.txt /root/tmp
diff_server_update_selva_test.2015-09-02.txt                       100%  763     0.8KB/s   00:00  
  
INFO: Fetching the list of packages need to be changed...!

*********************** Installation *************************
INFO: No packages to install
**************************************************************

************************* Upgrade ****************************
apt-get install mysql-client=5.5.43-0ubuntu0.12.04.1
Reading package lists... Done
Building dependency tree
etc...
**************************************************************

************************** Removal ***************************
apt-get remove zenity=3.4.0-0ubuntu4 zenity-common=3.4.0-0ubuntu4
Reading package lists... Done
Building dependency tree
etc...
**************************************************************

As showed in the sample output above, the script will then run through different phases in which it will install/upgrade/remove all the needed packages in the target production system.

In this way we ensure that all packages tested in the test environment will be replayed on the production server to minimize the chances of introducing bugs and maximize application stability during production system updates.

The Git repository below contains the script and a few additional information to use it.

https://github.com/selvait90/auto-tested-server-update

Ubuntu upgrade gotchas

2014-08-16T00:00:00+00:00

African penguins by Nick Ares

I recently upgraded my main laptop to Ubuntu 14.04, and had to solve a few issues along the way. Ubuntu is probably the most popular Linux distribution. Although it is never my first choice (that would be FreeBSD or Red Hat), Ubuntu is superb at working “out of the box”, so I often end up using it, as the other distributions all have issues.

Ubuntu 14.04.1 is a major “LTS” version, where LTS is “long term support”. The download page states that 14.04 (aka “Trusty Tahr”) comes with “five years of security and maintenance updates, guaranteed.” Alas, the page fails to mention the release date, which was July 24, 2014. When a new version of Ubuntu comes out, the OS will keep nagging you until you upgrade. I finally found a block of time in which I could survive without my laptop, and started the upgrade process. It took a little longer than I thought it would, but went smoothly except for one issue:

Issue 1: xscreensaver

During the install, the following warning appeared:

“One or more running instances of xscreensaver or xlockmore have been detected on this system. Because of incompatible library changes, the upgrade of the GNU libc library will leave you unable to authenticate to these programs. You should arrange for these programs to be restarted or stopped before continuing this upgrade, to avoid locking your users out of their current sessions.”

First, this is a terrible message. I’m sure it has caused lots of confusion, as most users probably do not know what what xscreensaver and xlockmore are. Is it so hard for the installer to tell which one is in use? Why in the world can the installer not simply stop these programs itself?! The solution was simple enough: in a terminal, I ran:

pgrep -l screensaver
pkill screensaver
pgrep -l screensaver

The first command was to see if I had any programs running with “screensaver” in their name (I did: xscreensaver). As it was the only program that matched, it was safe to run the second command, which stopped xscreensaver. Finally, I re-ran the pgrep to make sure it was stopped and gone. Then I did the same thing with the string “lockmore” (which found no matches, as I expected). Once xscreensaver was turned off, I told the upgrade to continue, and had no more problems until after Ubuntu 14.04 was installed and running. The first post-install problem appeared after I suspended the computer and brought it back to life—no wireless network!

Issue 2: no wireless after suspend

Once suspended and revived, the wireless would simply not work. Everything looked normal: networking was enabled, wifi hotspots were detected, but a connection could simply not be made. After going through bug reports online and verifying the sanity of the output of commands such as “nmcli nm” and “lshw -C network”, I found a solution. This was the hardest issue to solve, as it had no intuitive solution, nothing definitive online, and was full of red herrings. What worked for me was to remove the suspension of the iwlwifi module. I commented out the line from /etc/pm/config.d/modules, in case I ever need it again, so the file now looks like this:

# SUSPEND_MODULES=“iwlwifi”

Once that was commented out, everything worked fine. I tested by doing sudo pm-suspend from the command-line, and then bringing the computer back up and watching it automatically reconnect to my local wifi.

Issue 3: color diffs in Git

I use the command-line a lot, and a day never goes by without heavy use of Git as well. On running a git diff in the new Ubuntu version, I was surprised to see a bunch of escape codes instead of the usual pretty colors I was used to:

ESC[1mdiff --git a/t/03dbmethod.t b/t/03dbmethod.tESC[m
ESC[1mindex 108e0c5..ffcab48 100644ESC[m
ESC[1m--- a/t/03dbmethod.tESC[m
ESC[1m+++ b/t/03dbmethod.tESC[m
ESC[36m@@ -26,7 +26,7 @@ESC[m ESC[mmy $dbh = connect_database();ESC[m
 if (! $dbh) {ESC[m
    plan skip_all => 'Connection to database failed, cannot continue testing';ESC[m
 }ESC[m
ESC[31m-plan tests => 543;ESC[m
ESC[32m+ESC[mESC[32mplan tests => 545;ESC[m

After poking around with terminal settings and the like, a coworker suggested I simply tell git to use an intelligent pager with the command git config --global core.pager "less -r". The output immediately improved:

diff --git a/t/03dbmethod.t b/t/03dbmethod.t
index 108e0c5..ffcab48 100644
--- a/t/03dbmethod.t
+++ b/t/03dbmethod.t
@@ -26,7 +26,7 @@ my $dbh = connect_database();
 if (! $dbh) {
    plan skip_all => 'Connection to database failed, cannot continue testing';
 }
-plan tests => 543;
+plan tests => 545;

Thanks, Josh Williams! The above fix worked perfectly.

Issue 4: cannot select text in emacs

The top three programs I use every day are ssh, git, and emacs. While trying (post-upgrade) to reply to an email inside mutt, I found that I could not select text in emacs using ctrl-space. This is a critical problem, as this is an extremely important feature to lose in emacs. This problem was pretty easy to track down. The program “ibus” was intercepting all ctrl-space calls for its own purpose. I have no idea why ctrl-space was chosen, being used by emacs since before Ubuntu was even born (the technical term for this is “crappy default”). Fixing it requires visiting the ibus-setup program. You can reach it via the system menu by going to Settings Manager, then scroll down to the “Other” section and find “Keyboard Input Methods”. Or you can simply run ibus-setup from your terminal (no sudo needed).

The ibus-setup window

However you get there, you will see a section labelled “Keyboard Shortcuts”. There you will see a “Next input method:” text box, with the inside of it containing space. Aha! Click on the three-dot button to the right of it, and change it to something more sensible. I decided to simply add an “Alt”, such that going to the next input method will require Ctrl-Alt-Space rather than Ctrl-Space. To make that change, just select the “Alt” checkbox, click “Apply”, click “Ok”, and verify that the text box now says space.

So far, those are the only issues I have encountered using Ubuntu 14.04. Hopefully this post is useful to someone running into the same problems. Perhaps I will need to refer back to it in a few years(?) when I upgrade Ubuntu again! :)

Estimating overlayfs File Space Usage on Ubuntu 12.04 LiveCD

2013-04-24T00:00:00+00:00

End Point’s Liquid Galaxy platform is a cluster of computers distributing the rendering load of Google Earth and other visualization applications. Each of these “display nodes” boots the same disk image distributed over the local network. Our disk image is based on the Ubuntu 12.04 LiveCD, and uses the same overlayfs to combine a read-only ISO with a writeable ramdisk. This Union mount uses Copy-on-write to copy files from the read-only “lower” filesystem to the virtual “upper” filesystem whenever those files or directories are opened for writing.

We often allocate 4GB of system memory to the ramdisk containing the “upper” filesystem. This allows 4GB of changed files on the / filesystem, most of which are often Google Earth cache files. But sometimes the ramdisk fills up with other changes and it’s difficult to track down which files have changed unexpectedly.

The df command properly displays total usage for the overlayfs “upper” filesystem mounted at /.

$ df -h
Filesystem              Size  Used Avail Use% Mounted on
/cow                    3.9G  2.2G  1.8G  55% /
/dev/loop0              833M  833M     0 100% /cdrom
/dev/loop1              808M  808M     0 100% /rofs

But how can we identify which files are consuming that space? Because the root device has been pivoted by the casper scripts at boot, the /cow device is not readily available. We often use the du tool to estimate disk usage, but in this case it cannot tell the difference between files in the “upper” ramdisk and the “lower” read-only filesystem. To find the files filling our /cow device, we need a way to enumerate only the files in the “upper” filesystem, and then estimate only their disk usage.

The mount command shows the / filesystem is type “overlayfs”.

$ mount
/cow on / type overlayfs (rw)
/dev/loop0 on /cdrom type iso9660 (ro,noatime)
/dev/loop1 on /rofs type squashfs (ro,noatime)

The find command does indicate that most directories exist in the filesystem of type “overlayfs”, and most unmodified files are on the “lower” filesystem, in this case “squashfs”.

$ sudo find / -printf '%F\t%D\t%p\n' | head -n 7 ### fstype, st_dev, filename
overlayfs 17 /
overlayfs 17 /bin
squashfs 1793 /bin/bash
squashfs 1793 /bin/bunzip2
squashfs 1793 /bin/bzcat
squashfs 1793 /bin/bzcmp
squashfs 1793 /bin/bzdiff

However, the modified files are reported to be on an “unknown” filesystem on a device 16. These are the files that have been copied to the “upper” filesystem upon writing.

$ find /home/lg/ -printf '%F\t%D\t%p\n' | head -n 33
overlayfs 17 /home/lg/
squashfs 1793 /home/lg/.Xresources
unknown  16 /home/lg/.bash_history
squashfs 1793 /home/lg/.bash_logout
squashfs 1793 /home/lg/.bashrc
overlayfs 17 /home/lg/.config
overlayfs 17 /home/lg/.config/Google
unknown  16 /home/lg/.config/Google/GoogleEarthPlus.conf
unknown  16 /home/lg/.config/Google/GECommonSettings.conf

I couldn’t quickly discern how find is identifying the filesystem type, but it can use the -fstype test to reliably identify files that have been modified and copied. (Unfortunately find does not have a test for device number so if you have more than one overlayfs filesystem this solution may not work for you.)

Now that we have a reliable list of which files have been written to and copied, we can see which are consuming the most disk space by piping that list to du. We’ll pass it a null-terminated list of files to accommodate any special characters, then we’ll sort the output to identify the largest disk space hogs.

$ sudo find / -fstype unknown -print0 | du --files0-from=- --total | sort -nr | head -n 4
2214228 total
38600 /var/cache/apt/pkgcache.bin
38576 /var/cache/apt/srcpkgcache.bin
17060 /var/log/daemon.log

We included a total in this output, and notice that total is exactly the 2.2GB indicated by the df output above, so I believe this is measuring what we intend.

Of course five hundred 2MB Google Earth cache files consume more space than a single 38MB apt cache file, so we’d like to list the directories whose files are consuming the most ramdisk space. Unfortunately giving find and du depth arguments won’t work: the “unknown” filesystem doesn’t have any directories that we can see. We’ll have to parse the output, and that’s left as an exercise for the reader for now.

I just realized I could’ve simply looked for files modified after boot-time and gotten very similar results, but that’s not nearly as fun. There may be a way to mount only the “upper” filesystem, but I was disappointed by the lack of documentation around overlayfs, which will likely be included in the mainline 3.10 Linux kernel.

Ubuntu Dual Monitor Setup on Dell XPS

2012-10-01T00:00:00+00:00

Over the weekend, I received a new desktop (Dell XPS 8500 with NVIDIA graphics card) and troubleshot dual monitor setup on Ubuntu. Because I spent quite a while googling for results, I thought I’d write up a quick summary of what did and didn’t work.

One monitor was connected via HDMI and the other through DVI (with a VGA to DVI adaptor provided with the computer). When I started up the computer in Windows, both monitors were recognized immediately. Besides configuring the positioning of the monitors, Windows was good to go. But when I installed Ubuntu, the DVI/VGA monitor was recognized with incorrect resolution and the monitor connected via HDMI was not recognized at all. I tried switching the unrecognized monitor to a VGA/DVI connection, and it worked great by itself, so I concluded that it wasn’t an issue with a driver for the HDMI-connected monitor.

Many of the Google results I came across pointed to troubleshooting with xrandr, but any xrandr commands produced a “Failed to get size of gamma for output default.” error and any progress beyond that was shut down. Another set of Google results pointed to using “nvidia-detector”, but there weren’t any follow-up tips or pointers on that when it returned nothing. And many Google results were all over the place or were on forums and were unanswered.

Finally, I came across a couple of articles that suggested that Ubuntu didn’t have the proprietary nvidia driver and to install it with the command nvidia-current. After installing this, and restarting my X session, both monitors were working with correct resolution. I finished up by resetting the positioning of the monitors, adjusting the primary display, and saving to the xorg (X Configuration) file. This was probably the fastest I’ve figured out dual monitor setup (it’s always an adventure), most likely because of improved Linux/Ubuntu support on various machines over time.

nvidia-settings display settings.

Using Different PostgreSQL Versions at The Same Time.

2012-08-20T00:00:00+00:00

When I work for multiple clients on multiple different projects, I usually need a bunch of different stuff on my machine. One of the things I need is having multiple PostgreSQL versions installed.

I use Ubuntu 12.04. Installing PostgreSQL there is quite easy. Currently there are available two versions out of the box: 8.4 and 9.1. To install them I used the following command:

~$ sudo apt-get install postgresql-9.1 postgresql-8.4 postgresql-client-common

Now I have the above two versions installed.

Starting the database is also very easy:

~$ sudo service postgresql restart
 * Restarting PostgreSQL 8.4 database server   [ OK ]
 * Restarting PostgreSQL 9.1 database server   [ OK ]

The problem I had for a very long time was using the proper psql version. Both database installed their own programs like pg_dump and psql. Normally you can use pg_dump from the higher version PostgreSQL, however using different psql versions can be dangerous because psql uses a lot of queries which dig deep into the PostgreSQL internal tables for getting information about the database. Those internals sometimes change from one database version to another, so the best solution is to use the psql from the PostgreSQL installation you want to connect to.

The solution to this problem turned out to be quite simple. There is a pg_wrapper program which can take care of the different versions. It is enough to provide information about the PostgreSQL version you want to connect to and it will automatically choose the correct psql version.

Below you can see the results of using psql –version command which prints the psql version. As you can see there are different psql versions chosen according to the –cluster parameter.

~$ psql --cluster 8.4/main --version
psql (PostgreSQL) 8.4.11
contains support for command-line editing
~$ psql --cluster 9.1/main --version
psql (PostgreSQL) 9.1.4
contains support for command-line editing

You can find more information in the program manual using man pg_wrapper or at pg_wrapper manual

Problem with Cisco VPN on Ubuntu 12.04

2012-05-07T00:00:00+00:00

A couple of days ago I had to change my notebook. I installed Ubuntu 12.04 on the new one, while on the previous one there was Ubuntu 11.10. There were no problems with copying all the files from the old to the new machine, including GPG and SSH keys. Everything went smoothly and I could connect to all the machines I needed.

The only problem was with VPN. While working for one of our clients, I need to connect to their VPN. On the old machine I did that through the Network Manager. Nothing easier, I went to the Network Manager, chose the Export option and saved all the settings to a file. I copied the file to the new computer and loaded it into the Network Manager.

The file loaded correctly. I could switch the VPN on. It said everything works. But in fact it didn’t. The message was “VPN is connected”, I could switch it on and off, but I couldn’t access any of the client’s resources available from my previous notebook.

The first thing I checked was the content of /etc/resolv.conf on both computers. The file without connecting to VPN looked like this on both computers:

$ cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 127.0.0.1

When I connected to the VPN the files on both computers were quite different. For example on my new computer (and Ubuntu 12.04) the content of the file looked like this:

$ cat /etc/resolv.conf
# Generated by NetworkManager
domain something.net
search something.net
nameserver 127.0.0.1

I changed the data a little bit of course, so the domain names and IP addresses (except for 127.0.0.1) are not real.

On my old computer the resolv.conf file had a lot more entries, however I thought the above file should work as well. The problem was still the same: I couldn’t connect to the client’s resources.

The client is using the CISCO VPN, so I had to install network-manager-vpnc. This is just a plugin for network-manager which uses the vpnc program internally. I thought that maybe the plugin was doing something wrong.

I checked the plugin versions. Yes, they really differ. I started thinking about using the program without the Network Manager.

It turned out to be very simple to use. I need just a config file. The file is really simple:

IPSec gateway   something.net
IPSec ID        something.id
IPSec secret    somethingpass
Xauth username  mylogin
Xauth password  mypass

I keep all my local scripts in ~/bin (which can also be accessed as /home/szymon/bin). The directory ~/bin is added to the PATH environment variable. This way I can access all the scripts placed there in the console without providing the whole path. I did it by adding the following line at the end of my local ~/.bashrc file.

PATH=$PATH:$HOME/bin

To keep the things together I saved the config file at the same location ~/bin/vpn.conf.

Now I can connect to the VPN using:

$ sudo vpnc-connect /home/szymon/bin/vpn.conf

I can also stop the VPN using:

$ sudo vpnc-disconnect

To automate it a little bit I created a simple script stored at ~/bin/vpn:

#!/usr/bin/env bash

case "$1" in

start)
  sudo vpnc-connect /home/szymon/bin/vpn.conf
  ;;
stop)
  sudo vpnc-disconnect
  ;;
status)
  ps uaxf | grep vpnc-connect | grep -v grep
  ;;
restart)
  sudo vpnc-disconnect
  sudo vpnc-connect /home/szymon/bin/vpn.conf
  ;;
*)
  echo "Usage: vpn (start|stop|status|restart)"
  exit 1
  ;;

esac

This way I can simply write:

$ vpn start
[sudo] password for szymon:
VPNC started in background (pid: 13771)...

I noticed that now the /etc/resolv.conf file contains different entries than when using the Network Manager plugin:

$ cat /etc/resolv.conf
#@VPNC_GENERATED@ -- this file is generated by vpnc
# and will be overwritten by vpnc
# as long as the above mark is intact
# Generated by NetworkManager
nameserver 1.2.3.4
nameserver 1.2.3.4
search something.net

I can also disconnect from the VPN with simple command:

$ vpn stop
Terminating vpnc daemon (pid: 13771)

I’m using this script for a couple of days and I don’t have any problems with the CISCO VPN. It seems like the vpnc program in Ubuntu 12.04 is OK, however there is something wrong with the Network Manager plugin for vpnc.

Easy creating ramdisk on Ubuntu

2012-04-16T00:00:00+00:00

Hard drives are extremely slow compared to RAM. Sometimes it is useful to use a small amount of RAM as a drive.

However, there are some drawbacks to this solution. All the files will be gone when you reboot your computer, so in fact it is suitable only for storing some temporary files—those which are generated during some process and are not useful later.

I will mount the ramdisk in my local directory. I use Ubuntu 11.10, my user name is ‘szymon’, and my home directory is ‘/home/szymon’.

I create the directory for mounting the ramdisk in my home dir:

mkdir /home/szymon/ramdisk

When creating the ramdisk, I have a couple of possibilities:

ramdisk—there are sixteen standard block devices at /dev/ram* (from /dev/ram0 to /dev/ram15) which can be used for storing ram data. I can format it with any of the filesystems I want, but usually this is too much complication
ramfs—a virtual filesystem stored in ram. It can grow dynamically, and in fact it can use all available ram, which could be dangerous.
tmpfs—another virtual filesystem stored in ram, but because it has a fixed size, it cannot grow like ramfs.

I want to have a ramdisk that won’t be able to use all of my ram, and I want to keep it as simple as possible; therefore, I will use tmpfs.

The following command will mount a simple ramdisk in my new local directory.

$ sudo mount -t tmpfs -o size=512M,mode=777 tmpfs
/home/szymon/ramdisk

I can even unmount it with:

$ sudo umount /home/szymon/ramdisk

Let’s check if the ramdisk is really there. I can do so in a couple of ways.

I can use df -h to check the size of the mounted device:

$ df -h | grep szymon
tmpfs                 512M     0  512M   0% /home/szymon/ramdisk

I can also use mount to report on the mounted devices:

$ mount | grep ramdisk
tmpfs on /home/szymon/ramdisk type tmpfs (rw,size=512M,mode=777)

There is one more thing to do—make the ramdisk load automatically at machine start. This can be done by adding the following line into /etc/fstab:

tmpfs    /home/szymon/ramdisk    tmpfs    rw,size=512M,mode=777 0    0

Two things to be aware of:

Data stored in ramdisk will be removed at the machine restart. Creating a script for saving the files to hard drive at machine shutdown won’t persist the data during a machine crash or reset.
The computer won’t be able to boot normally if the entry in the fstab file has any errors in it. If you do make such an error, you can always boot the computer in recovery mode, so you have the root console and can fix the fstab file.

Linux unshare -m for per-process private filesystem mount points

2012-01-27T00:00:00+00:00

Private mount points with unshare

Linux offers some pretty interesting features that are either new, borrowed, obscure, experimental, or any combination of those qualities. One such feature that is interesting is the unshare() function, which the unshare(2) man page says “allows a process to disassociate parts of its execution context that are currently being shared with other processes. Part of the execution context, such as the mount namespace, is shared implicitly when a new process is created using fork(2) or vfork(2)”.

I’m going to talk here about one option to unshare: per-process private filesystem mount points, also described as mount namespaces. This Linux kernel feature has been around for a few years and is easily accessible in the userland command unshare(1) in util-linux-ng 2.17 or newer (which is now simply util-linux again without the “ng” distinction because the fork took over mainline development).

Running unshare -m gives the calling process a private copy of its mount namespace, and also unshares file system attributes so that it no longer shares its root directory, current directory, or umask attributes with any other process.

Yes, completely private mount points for each process. Isn’t that interesting and strange?

A demonstration

Here’s a demonstration on an Ubuntu 11.04 system. In one terminal:

% su -
Password:
# unshare -m /bin/bash
# secret_dir=`mktemp -d --tmpdir=/tmp`
# echo $secret_dir
/tmp/tmp.75xu4BfiCw
# mount -n -o size=1m -t tmpfs tmpfs $secret_dir
# df -hT
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/mapper/auge-root
              ext4    451G  355G   74G  83% /

There’s no system-wide sign of /tmp/tmp.* there thanks to mount -n which hides it. But it can be seen process-private here:

# grep /tmp /proc/mounts
tmpfs /tmp/tmp.75xu4BfiCw tmpfs rw,relatime,size=1024k 0 0
# cd $secret_dir
# ls -lFa
total 36
drwxrwxrwt  2 root root    40 2011-11-03 22:10 ./
drwxrwxrwt 21 root root 36864 2011-11-03 22:10 ../
# touch play-file
# mkdir play-dir
# ls -lFa
total 36
drwxrwxrwt  3 root root    80 2011-11-03 22:10 ./
drwxrwxrwt 21 root root 36864 2011-11-03 22:10 ../
drwxr-xr-x  2 root root    40 2011-11-03 22:10 play-dir/
-rw-r--r--  1 root root     0 2011-11-03 22:10 play-file

Afterward, in another terminal, and thus a separate process with no visibility into the above-shown terminal process’s private mount points:

% su -
Password:
# grep /tmp /proc/mounts
# cd /tmp/tmp.75xu4BfiCw
# ls -lFa
total 40
drwx------  2 root root  4096 2011-11-03 22:10 ./
drwxrwxrwt 21 root root 36864 2011-11-03 22:18 ../

It’s all secret!

Use cases

This feature makes it possible for us to create a private temporary filesystem that even other root-owned processes cannot see or browse through, raising the bar considerably for a naive attacker to get access to sensitive files or even see that they exist, at least when they’re not currently open and visible to e.g. lsof.

Of course a sophisticated attacker would presumably have a tool to troll through kernel memory looking for what they need. As always, assume that a sophisticated attacker who has access to the machine will sooner or later have anything they really want from it. But we’d might as well make it a challenge.

Another possible use of this feature is to have a process unmount a filesystem privately, perhaps to reduce the exposure of other files on a system to a running daemon if it is compromised.

/etc/mtab vs. /proc/mounts

Experimenting with this feature also drew my attention to differences in how popular Linux distributions expose mount points. There are actually traditionally two places that the list of mounts is stored on a Linux system.

First, the classic Unix /etc/mtab, which is in essence a materialized view. It is the reason that on the Ubuntu 11.04 example above we see the private mount point everywhere on the system, but it reported different disk sizes. The existence of the mount point was global in /etc/mtab but the sizes are determined dynamically and differ based on process’s view into the mount points themselves. The mount -n option tells mount to not put the new mount point into /etc/mtab. And this is what the df(1) command refers to. How repulsive that a file in the normally read-only /etc is written to so nonchalantly!

Second, the Linux-specific /proc/mounts, which is real-time, exact, and accurate, and can appear differently to each process. The mount invocation can’t hide anything from /proc/mounts. This is what you would think is the only place to look for mounts, but /etc/mtab is still used some places.

Ubuntu 11.04 still has both, with a separate /etc/mtab. Fedora 16 has done away with /etc/mtab entirely and made it merely a symlink to /proc/mounts, which makes sense, but that is a newer convention and leads to the surprising difference here.

Linux distributions and unshare

The unshare userland command in util-linux(-ng) comes with RHEL 6, Debian 6, Ubuntu 11.04, and Fedora 16, but not on the very common RHEL 5 or CentOS 5. Because we needed it on RHEL 5, I made a simple package that contains only the unshare(1) command and peacefully coexists with the older stock RHEL 5 util-linux. It’s called util-linux-unshare and here are the RPM downloads for RHEL 5:

x86_64: util-linux-unshare-2.20.1-3.ep.x86_64.rpm
i386: util-linux-unshare-2.20.1-3.ep.i386.rpm
SRPM: util-linux-unshare-2.20.1-3.ep.src.rpm

I hope you’ve found this as interesting as I did!

MySQL replication monitoring on Ubuntu 10.04 with Nagios and NRPE

2012-01-21T00:00:00+00:00

If you’re using MySQL replication, then you’re probably counting on it for some fairly important need. Monitoring via something like Nagios is generally considered a best practice. This article assumes you’ve already got your Nagios server setup and your intention is to add a Ubuntu 10.04 NRPE client. This article also assumes the Ubuntu 10.04 NRPE client is your MySQL replication master, not the slave. The OS of the slave does not matter.

Getting the Nagios NRPE client setup on Ubuntu 10.04

At first it wasn’t clear what packages would be appropriate packages to install. I was initially misled by the naming of the nrpe package, but I found the correct packages to be:

sudo apt-get install nagios-nrpe-server nagios-plugins

The NRPE configuration is stored in /etc/nagios/nrpe.cfg, while the plugins are installed in /usr/lib/nagios/plugins/ (or lib64). The installation of this package will also create a user nagios which does not have login permissions. After the packages are installed the first step is to make sure that /etc/nagios/nrpe.cfg has some basic configuration.

Make sure you note the server port (defaults to 5666) and open it on any firewalls you have running. (I got hung up because I forgot I have both a software and hardware firewall running!) Also make sure the server_address directive is commented out; you wouldn’t want to only listen locally in this situation. I recommend limiting incoming hosts by using your firewall of choice.

Choosing what NRPE commands you want to support

Further down in the configuration, you’ll see lines like command[check_users]=/usr/lib/nagios/plugins/check_users -w 5 -c 10. These are the commands you plan to offer the Nagios server to monitor. Review the contents of /usr/lib/nagios/plugins/ to see what’s available and feel free to add what you feel is appropriate. Well designed plugins should give you a usage if you execute them from the command line. Otherwise, you may need to open your favorite editor and dig in!

After verifying you’ve got your NRPE configuration completed and made sure to open the appropriate ports on your firewall(s), let’s restart the NRPE service:

service nagios-nrpe-server restart

This would also be an appropriate time to confirm that the nagios-nrpe-server service is configured to start on boot. I prefer the chkconfig package to help with this task, so if you don’t already have it installed:

sudo apt-get install chkconfig
chkconfig | grep nrpe

# You should see...
nagios-nrpe-server     on

# If you don't...
chkconfig nagios-nrpe-server on

Pre flight check - running check_nrpe

Before going any further, log into your Nagios server and run check_nrpe and make sure you can execute at least one of the commands you chose to support in nrpe.cfg. This way, if there are any issues, it is obvious now, while we’ve not started modifying your Nagios server configuration. The location of your check_nrpe binary may vary, but the syntax is the same:

check_nrpe -H host_of_new_nrpe_client -c command_name

If your command output something useful and expected, your on the right track. A common error you might see: Connection refused by host. Here’s a quick checklist:

Did you start the nagios-nrpe-server service?
Run netstat -lunt on the NRPE client to make sure the service is listening on the right address and ports.
Did you open the appropriate ports on all your firewall(s)?
Is there NAT translation which needs configuration?

Adding the check_mysql_replication plugin

There is a lot of noise out there on Google for Nagios plugins which offer MySQL replication monitoring. I wrote the following one using ideas pulled from several existing plugins. It is designed to run on the MySQL master server, check the master’s log position and then compare it to the slave’s log position. If there is a difference in position, the alert is considered Critical. Additionally, it checks the slave’s reported status, and if it is not “Waiting for master to send event”, the alert is also considered critical. You can find the source for the plugin at my Github account under the project check_mysql_replication. Pull that source down into your plugins directory (/usr/lib/nagios/plugins/ (or lib64)) and make sure the permissions match the other plugins.

With the plugin now in place, add a command to your nrpe.cfg.

command[check_mysql_replication]=sudo /usr/lib/nagios/plugins/check_mysql_replication.sh -H <slave_host_address></slave_host_address>

At this point you may be saying, WAIT! How will the user running this command (nagios) have login credentials to the MySQL server? Thankfully we can create a home directory for that nagios user, and add a .my.cnf configuration with the appropriate credentials.

usermod -d /home/nagios nagios #set home directory
mkdir /home/nagios
chmod 755 /home/nagios
chown nagios:nagios /home/nagios

# create /home/nagios/.my.cnf with your preferred editor with the following:
[client]
user=example_replication_username
password=replication_password

chmod 600 /home/nagios/.my.cnf
chown nagios:nagios /home/nagios/.my.cnf

This would again be an appropriate place to run a pre flight check and run the check_nrpe from your Nagios server to make sure this configuration works as expected. But first we need to add this command to the sudoer’s file.

nagios ALL= NOPASSWD: /usr/lib/nagios/plugins/check_mysql_replication.sh

Wrapping Up

At this point, you should run another check_nrpe command from your server and see the replication monitoring report. If not, go back and check these steps carefully. There are lots of gotchas and permissions and file ownership are easily overlooked. With this in place, just add the NRPE client using the existing templates you have for your Nagios servers and make sure the monitoring is reporting as expected.

Building Xpdf on Ubuntu

2011-08-31T00:00:00+00:00

It may happen that you need to use Xpdf, even though it no longer ships with Ubuntu and is considered … outdated? buggy? insecure? In any case, it still renders some PDFs that Poppler-based viewers such as Evince don’t, or allows some troublesome PDFs to print as fonts and line art instead of a rasterized mess.

Here’s how I built and installed xpdf 3.02 on Ubuntu 11.04 (Natty Narwhal) x86_64:

sudo apt-get install libfreetype6-dev libmotif-dev
wget ftp://ftp.foolabs.com/pub/xpdf/xpdf-3.02.tar.gz  # now 3.03 is current
tar xzpf xpdf-3.02.tar.gz
cd xpdf-3.02
./configure --with-freetype2-library=/usr/lib/x86_64-linux-gnu \
    --with-freetype2-includes=/usr/include/freetype2 \
    --with-Xm-library=/usr/lib \
    --with-Xm-includes=/usr/include/Xm
make
# see lots of warnings!
sudo make install

That’s it. Not as nice as the old native Debian/Ubuntu packages, but gets the job done.

Migrating from Legacy Networking to systemd-networkd on Ubuntu 24.04

Why migrate to systemd-networkd?

Step 1: Check the current setup

Step 2: Install and enable systemd-networkd

Step 3: Create a network configuration file

Step 4: Configure systemd-resolved

Restart and verify

Step 6: Reboot and confirm

Handling remote migration and downtime risks

Automating the migration and rollback process

migrate_to_systemd_networkd.sh

upgrade_watchdog.sh

Conclusion

Making a Loading Spinner with tkinter

Prerequisites

Code

Resources

Ubuntu Touch on a Galaxy S7 & a Pixel 3a

Installation

Usability

GStreamer Nvenc for Ubuntu 20.04

Environment

CUDA

Plugin

Benchmark

Alternatives

Installing Ubuntu 18.04 to a different partition from an existing Ubuntu installation

Upgrading from Ubuntu 14.04 LTS

Installing Clean Ubuntu 18.04 LTS from Ubuntu 14.04 LTS

Install debootstrap

Creating a new root partition

Mounting the new root partition

Bootstrapping the new root partition

Installing fstab

Mounting things in the new root partition

Configuring apt

Installing required packages and running chef-client

Set Ubuntu 14.04 to boot default

On the Current system (Trusty):

On the target system (Bionic):

Boot into Bionic

Set Ubuntu 18.04 to boot default

How to set up a local development environment for WordPress from scratch

1. Set up PHP

2. Set up MySQL

3. Set up WordPress

4. Run WordPress

Bonus 1: Set up a linter: PHP Code Sniffer

Bonus 2: Setup interactive debugging with VS Code

Install Tested Packages on Production Server

Test Server Update Process:

Example:

Production Server Update Process:

Example:

Ubuntu upgrade gotchas

Issue 1: xscreensaver

Issue 2: no wireless after suspend

Issue 3: color diffs in Git

Issue 4: cannot select text in emacs

Estimating overlayfs File Space Usage on Ubuntu 12.04 LiveCD

Ubuntu Dual Monitor Setup on Dell XPS

Using Different PostgreSQL Versions at The Same Time.

Problem with Cisco VPN on Ubuntu 12.04

Easy creating ramdisk on Ubuntu

Linux unshare -m for per-process private filesystem mount points

Private mount points with unshare

A demonstration

Use cases

/etc/mtab vs. /proc/mounts

Linux distributions and unshare

Further reading

MySQL replication monitoring on Ubuntu 10.04 with Nagios and NRPE

Getting the Nagios NRPE client setup on Ubuntu 10.04

Choosing what NRPE commands you want to support

Pre flight check - running check_nrpe

Adding the check_mysql_replication plugin

Wrapping Up

Building Xpdf on Ubuntu