Making a Loading Spinner with tkinter

2024-03-05T00:00:00+00:00

When you need a loading spinner, you really need a loading spinner. Interested in putting something on the screen without installing a pile of dependencies, I reached deep into the toolbox of the Python standard library, dug around a bit, and pulled out the tkinter module.

The tkinter module is an interface to the venerable Tcl/Tk GUI toolkit, a cross-platform suite for creating user interfaces in the style of whatever operating system you run it on. It’s the only built-in GUI toolkit in Python, but there are many worthy alternatives available (see the end of the post for a list).

Here I’ll demonstrate how to make a loading spinnner with tkinter on Ubuntu 22.04. It should work on any platform that runs Python, with some variations when setting up the system for it.

Prerequisites

My vision for the loading spinner is some spinning dots and a logo, since this is such a convenient branding opportunity. To accomplish this we’ll be extending tkinter with Pillow’s ImageTk capability, which can load a PNG with transparency.

To produce that PNG with transparency, first we may need to rasterize an SVG file, because wise designers work in vectors. This is made trivial by Inkscape, a free and complete vector graphics tool:

$ inkscape logo.svg -o logo.png

With the logo in hand, we can move on to setting up our dependencies. Ubuntu’s python3 distribution doesn’t include tkinter by default, so we’ll need to install it explicitly, along with Pillow’s separate ImageTk support:

$ sudo apt install python3-tk python3-pil.imagetk

This may occupy up to 75MB, if Python was already installed. This was still the smallest apt footprint of all of the Python GUI libraries in consideration. Pygame was also a strong contender.

Code

Let’s start with something simple: putting the logo on the screen.

#!/usr/bin/env python3

from PIL import Image, ImageTk
from tkinter import BOTH, Canvas, Tk


# Desired dimensions of our window.
WIDTH, HEIGHT = 500, 500

if __name__ == "__main__":
    # Create the root window object.
    root = Tk()
    # Create a canvas for drawing our graphics.
    canvas = Canvas(root, width=WIDTH, height=HEIGHT, background="black")
    # Fill the entire window with the canvas.
    canvas.pack(fill=BOTH, expand=1)

    # Load the logo PNG with regular PIL.
    logo_img = Image.open("logo.png")
    # Convert the logo to an ImageTk PhotoImage.
    logo_pi = ImageTk.PhotoImage(logo_img)
    # Add our logo image to the canvas.
    canvas.create_image(
        WIDTH / 2,
        HEIGHT / 2,
        image=logo_pi,
    )

    # Run the tkinter main loop.
    root.mainloop()

This puts the logo in the center of a window, but the logo may be too large or small. Let’s scale it according to the window dimensions, let’s say to about ⅔ of the width so we have some room for spinning dots:

#!/usr/bin/env python3

from PIL import Image, ImageTk
from tkinter import BOTH, Canvas, Tk


# Desired dimensions of our window.
WIDTH, HEIGHT = 500, 500

if __name__ == "__main__":
    # Create the root window object.
    root = Tk()
    # Create a canvas for drawing our graphics.
    canvas = Canvas(root, width=WIDTH, height=HEIGHT, background="black")
    # Fill the entire window with the canvas.
    canvas.pack(fill=BOTH, expand=1)

    # Load the logo PNG with regular PIL.
    logo_img = Image.open("logo.png")
    # Resize the logo to about 2/3 the window width.
    scaled_w = round(WIDTH * 0.6)
    scaled_h = round(scaled_w / (logo_img.width / logo_img.height))
    logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
    # Convert the logo to an ImageTk PhotoImage.
    logo_pi = ImageTk.PhotoImage(logo_img)
    # Add our logo image to the canvas.
    canvas.create_image(
        WIDTH / 2,
        HEIGHT / 2,
        image=logo_pi,
    )

    # Run the tkinter main loop.
    root.mainloop()

That’s better. Now let’s add the promised spinning dots. We’ll draw some ovals on the canvas and modify our main loop to animate them:

#!/usr/bin/env python3

import math
import time

from PIL import Image, ImageTk
from tkinter import BOTH, Canvas, Tk


# Desired dimensions of our window.
WIDTH, HEIGHT = 500, 500
# Coordinates of the center.
CENTER_X, CENTER_Y = WIDTH / 2, HEIGHT / 2
# How many spinning dots we want.
NUM_DOTS = 8

if __name__ == "__main__":
    # Create the root window object.
    root = Tk()
    # Create a canvas for drawing our graphics.
    canvas = Canvas(root, width=WIDTH, height=HEIGHT, background="black")
    # Fill the entire window with the canvas.
    canvas.pack(fill=BOTH, expand=1)

    # Load the logo PNG with regular PIL.
    logo_img = Image.open("logo.png")
    # Resize the logo to about 2/3 the window width.
    scaled_w = round(WIDTH * 0.6)
    scaled_h = round(scaled_w / (logo_img.width / logo_img.height))
    logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
    # Convert the logo to an ImageTk PhotoImage.
    logo_pi = ImageTk.PhotoImage(logo_img)
    # Add our logo image to the canvas.
    canvas.create_image(
        CENTER_X,
        CENTER_Y,
        image=logo_pi,
    )

    # Radius in pixels of a single dot.
    dot_radius = WIDTH * 0.05
    # Radius of the ring of dots from the center of the window.
    dots_radius = WIDTH / 2 - dot_radius * 2

    # Helper function to calculate dot position on each update.
    def get_dot_coords(n: int, t: float):
        """Get the x0, y0, x1, y1 coords of dot at index 'n' at time 't'."""
        angle = (n / NUM_DOTS) * math.pi * 2 + t
        x = math.cos(angle) * dots_radius + CENTER_X
        y = math.sin(angle) * dots_radius + CENTER_Y
        return x - dot_radius, y - dot_radius, x + dot_radius, y + dot_radius

    # Create all the dots.
    t0 = time.monotonic()
    for n in range(NUM_DOTS):
        coords = get_dot_coords(n, t0)
        canvas.create_oval(
            *coords,
            fill="#888888",
            width=0,  # Border width.
            tags=f"dot_{n}",
        )

    # Set up a custom main loop to animate the moving dots.
    while True:
        # Check the time of this update.
        t = time.monotonic()
        for n in range(NUM_DOTS):
            # Get the desired coords for this dot at this time.
            coords = get_dot_coords(n, t)
            # Move the dot on the canvas, finding it by its tag.
            canvas.coords(
                f"dot_{n}",
                *coords,
            )
        # Call the required tkinter update function.
        root.update()
        # Attempt to stabilize the timing of this loop by targeting 60Hz.
        while t0 < t:
            t0 += 1 / 60
        time.sleep(t0 - t)

You may notice that the dots don’t look all that great. There’s no anti-aliasing when drawing shape primitives in tkinter, so the edges look jagged compared to our well-scaled logo image. One hack is to layer slightly larger and dimmer shapes under each object, which you might do like so:

#!/usr/bin/env python3

import math
import time

from PIL import Image, ImageTk
from tkinter import BOTH, Canvas, Tk


# Desired dimensions of our window.
WIDTH, HEIGHT = 500, 500
# Coordinates of the center.
CENTER_X, CENTER_Y = WIDTH / 2, HEIGHT / 2
# How many spinning dots we want.
NUM_DOTS = 8
# Colors for each layer of fake anti-aliasing around each dot.
# Must be in order from back to front.
COLORS = ["#888888", "#BBBBBB", "#FFFFFF"]

if __name__ == "__main__":
    # Create the root window object.
    root = Tk()
    # Create a canvas for drawing our graphics.
    canvas = Canvas(root, width=WIDTH, height=HEIGHT, background="black")
    # Fill the entire window with the canvas.
    canvas.pack(fill=BOTH, expand=1)

    # Load the logo PNG with regular PIL.
    logo_img = Image.open("logo.png")
    # Resize the logo to about 2/3 the window width.
    scaled_w = round(WIDTH * 0.6)
    scaled_h = round(scaled_w / (logo_img.width / logo_img.height))
    logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
    # Convert the logo to an ImageTk PhotoImage.
    logo_pi = ImageTk.PhotoImage(logo_img)
    # Add our logo image to the canvas.
    canvas.create_image(
        CENTER_X,
        CENTER_Y,
        image=logo_pi,
    )

    # Radius in pixels of a single dot.
    dot_radius = WIDTH * 0.05
    # Radius of the ring of dots from the center of the window.
    dots_radius = WIDTH / 2 - dot_radius * 2

    # Helper function to calculate dot position on each update.
    def get_dot_coords(n: int, t: float, c: int):
        """Get the x0, y0, x1, y1 coords of dot at index 'n' at time 't'.
        Inflate the radius by color index 'c'."""
        angle = (n / NUM_DOTS) * math.pi * 2 + t
        x = math.cos(angle) * dots_radius + CENTER_X
        y = math.sin(angle) * dots_radius + CENTER_Y
        # Invert the color index and add to the radius.
        radius = dot_radius + (len(COLORS) - c) * 0.75
        #radius = dot_radius + c
        return x - radius, y - radius, x + radius, y + radius

    # Create all the dots.
    t0 = time.monotonic()
    for c, color in enumerate(COLORS):
        for n in range(NUM_DOTS):
            coords = get_dot_coords(n, t0, c)
            canvas.create_oval(
                *coords,
                fill=color,
                width=0,  # Border width.
                tags=f"dot_{c}_{n}",
            )

    # Set up a custom main loop to animate the moving dots.
    while True:
        # Check the time of this update.
        t = time.monotonic()
        for c, color in enumerate(COLORS):
            for n in range(NUM_DOTS):
                # Get the desired coords for this dot at this time.
                coords = get_dot_coords(n, t, c)
                # Move the dot on the canvas, finding it by its tag.
                canvas.coords(
                    f"dot_{c}_{n}",
                    *coords,
                )
        # Call the required tkinter update function.
        root.update()
        # Attempt to stabilize the timing of this loop by targeting 60Hz.
        while t0 < t:
            t0 += 1 / 60
        time.sleep(t0 - t)

The fake anti-aliasing was a fun exercise, but for this use case you’ll probably get better-looking results out of scaling a PNG asset like we did the logo.

Resources

If you’re interested in learning more about tkinter, see also:

Other Python GUI/graphics toolkits you might consider:

Pygame
Pyglet
PySide6 (official Qt binding)
PyQt (unofficial Qt binding)
wxPython

Automating reading the screen and interacting with GUI programs on X Window System

2022-03-10T00:00:00+00:00

A while back, Google Earth made some changes to the layer select menu in the sidebar, which broke a program that toggles the 3D imagery on VisionPort systems. These run the X Window System (also known as X11, or just X) on Ubuntu Linux.

In looking for a workaround, I found that shell scripts can fully interact with GUI apps using the xdotool, xwd, and convert commands. This script would send a series of keystrokes to open the sidebar, navigate the layers menu, and toggle the box for the 3D buildings layer.

Changing the series of keystrokes to match the new number of layers should have fixed the issue, but there was more to this script. The next part of the script would take a screenshot, crop the checkbox, and compare it to saved files of other cropped boxes. Fixing this part of the script required correcting the positions of the captures and replacing the reference files with ones that pictured the updated Google Earth checkbox states.

Here I will explain how the script works and how we changed it so that it no longer needs these reference files and ultimately runs faster.

Overview of how the script works

xwd takes a screenshot of a window on the screen. convert transforms the pixel data in the image into lines of text with location and color that we can easily search and read with grep, sed, or similar. xdotool interacts with GUI windows. It can find, focus, and send keystrokes and mouse commands, among other things.

Preparing

To illustrate, let’s make a simple case of looking for a button and clicking on it from the terminal. We will skip making a script for this example as most of this can be accomplished on a single command line with xdotool. The commands here are for the Ubuntu operating system and may be a little different on other systems.

If you would like to try it as you read along, you will need an image editor to quickly look up pixel positions and colors. GIMP works great and is pictured in this example.

You will also need to install some packages. On a terminal:

sudo apt install xdotool x11-apps imagemagick

Working through an example

1. To know which window to interact with, xdotool needs to know the window’s name. So let’s open a browser and navigate to endpointdev.com and then note the page title in the browser tab, “Secure Business Solutions”:

2. On the terminal:

xdotool search "secure business solutions"

Defaulting to search window name, class, and classname
69206019

This reminds us that it can search for windows in many ways and returns a window ID, in this example 69206019. Search results also get stored on the “window stack” and can be referenced as %1, %2, and so on.

3. Use this ID in the next terminal command. If you have more than one ID, you may need to refine your search:

xwd -id 69206019 -out endpointdev.xwd

xwd creates a screenshot of the window that we want to see and saves it to the filename passed with the -out argument.

4. Then run a convert command:

convert endpointdev.xwd endpointdev.txt

By converting the image to text we can use any text tools like grep, cut, diff, or sed to find colors and coordinates on the images. We just need to know how to read it. The first few lines of the endpointdev.txt file look like this:

# ImageMagick pixel enumeration: 1156,638,65535,srgb
0,0: (11565,11565,11565)  #2D2D2D  srgb(45,45,45)
1,0: (10537,10537,10537)  #292929  grey16
2,0: (4369,4369,4369)  #111111  srgb(17,17,17)
3,0: (0,0,0)  #000000  black
4,0: (0,0,0)  #000000  black
5,0: (0,0,0)  #000000  black

Each line represents a pixel:

The first two numbers are the x and y positions,
in parentheses are the decimal color values per RGB channel,
after # is the HTML RGB color value which we will be using in our example,
and last the srgb code or name for the color.

We can use any of these color codes for matching. To decode these color codes visually we use an image editor.

5. Open the endpointdev.xwd file that we created in an image editor. Here we will use GIMP:

6. Select the zoom tool (see the mouse pointer in the screenshot above to know which that is). Use it to draw a box around the VisionPort button or something else that we want our script to “see” and zoom in on it:

7. Next, use the color picker tool (check the mouse pointer in the screenshot above for this tool) and click on the circle, or any other spot you want to pick for your script:

8. Notice by the pointer above how the color picker box changed colors. Now clicking on this box brings up a new window:

Here we can find the HTML notation of the color we selected: 00ffcc (see the pointer in the screenshot above). We’ll call this “VisionPort button green”.

In my experience even things that look like the same color can have a unique pixel caused by a different shade on an edge or something else. Our old script checked a 25×25 pixel image to guarantee what it had was a check mark, but finding one of these colors unique enough to use as a key ensures that the check mark is next to it, so we can check the state on a single pixel too.

If one pixel is not unique enough we can look for a pattern of pixels and still store them as variables on the script to avoid saving and comparing files.

9. With this HTML hex RGB color code we go back to the terminal and search our file for it. Use capital letters for the color or else grep -i to make it case-insensitive:

grep 00FFCC endpointdev.txt

This will return a list of pixels with this color, such as:

827,107: (0,65535,52428)  #00FFCC  srgb(0,255,204)
839,107: (0,65535,52428)  #00FFCC  srgb(0,255,204)
853,107: (0,65535,52428)  #00FFCC  srgb(0,255,204)
878,107: (0,65535,52428)  #00FFCC  srgb(0,255,204)
883,107: (0,65535,52428)  #00FFCC  srgb(0,255,204)

10. For our purpose we can use any of those, so let’s try the first result. In the terminal run:

xdotool windowfocus 69206019 mousemove 827 107 click 1

Your browser should then follow the link to visionport.com and load that website’s home page in your browser:

Did it work for you too? 🎉

That is all we need to build a script that interacts with GUI apps. If you would like to practice, take another screenshot and use xdotool to look for and click the “contact” button. 😁

How it works in a little more detail

The last xdotool command chain’s steps are:

windowfocus 69206019 selects the ID that we searched for before,
mousemove moves the mouse pointer to the specified absolute coordinates, in this case the first pixel that matched the VisionPort button green color in the endpointdev.txt file,
click 1 sends a mouse left click at the mouse’s current position.

Updating our script to use this instead of the series of keystrokes took some investigation and testing but made it faster in the end.

One of the issues to overcome was that the VisionPort custom layers capture xdotool’s window and mouse commands, so to target a specific window on the VisionPort, we have to offset the coordinates of our commands according to the position of the window on the screen.

Chaining xdotool actions

Here is another example of chaining several xdotool actions in a single command:

xdotool search "secure business solutions" windowfocus key Alt+F4

That should close the browser window on most systems. If any of these commands failed, try replacing windowfocus with windowactivate; some xdotool commands depend on system support. Check xdotool’s manual for more options.

Command chain steps:

search <name>: Finds windows with that name, and stores the results in the “window stack” memory.
windowfocus: Selects a window. Defaults to the first window from xdotool’s window stack; no need to supply the window ID if following a search.
key Alt+F4: Sends the Alt+F4 keypress to the selected window. The + here means at the same time. Use spaces to separate a list of keys. By default there is a delay between keystrokes, and that is configurable.

Use the standard Unix sleep <seconds> in between other xdotool commands that may need delays.

xdotool takes a list of commands naturally, which can even be stored and read directly from text files. They can also be executed directly by making the file executable (with chmod +x) and setting the shebang line (first line of the file) to:

#!/usr/bin/xdotool

Such executable scripts are commonly named using a .xdo extension.

Fine-tuning

While we can convert the whole screenshot to text and search on it like we did in the example above, the files are large, so cropping a smaller section before converting to text is recommended.

We can crop a section, save it to a file and compare it to another file, like our script did. We can also crop an image to 1 pixel wide across a menu bar if that’s what we’re aiming for, or a column of icons in our case, to find all the button positions without much overhead. Doing this instead of only checking the area hard-coded for a check box gave our script the ability to find the check box based on the icon next to it when changing positions due to other tabs expanding on the menu.

It is also possible to use the multiprocessing features of shell scripts: When working on multiple window screenshots or crops, we can have them all run at the same time using the & operator at the end of the commands, then use the wait command to let them all finish before performing the text searches. These are very fast on small cropped files.

Learning more

The xdotool documentation has information on other ways to identify and interact with windows and is a great place to go to learn more.

Optimizing media delivery with Cloudinary

2022-03-01T00:00:00+00:00

I remember how we needed to deal with different image formats and sizes years ago: From using the WordPress-style approach of automatically saving different resolutions on the server when uploading a picture, to using a PHP script to resize or crop images on the fly and return the result as a response to the frontend. Of course, many of those approaches were expensive, and not fully optimized for different browsers or device sizes.

With those experiences in mind, it was a nice surprise for me to discover Cloudinary when working on a new project a couple of months ago. It’s basically a cloud service that saves and delivers media content with a lot of transformations and management options for us to use. There is a free version with a usage limit: Up to 25K transformations or 25 GB of storage/bandwidth, which should be enough for most non-enterprise websites. The cheapest paid service is $99 per month.

Here’s a list of the image features we used on that project. I know they offer many other things that can be used as well, but I think this is a good start for anyone who hasn’t used this service yet:

Resizing and cropping

When you make a request for an image, you can instruct the Cloudinary API to retrieve it with a given size, which will trigger a transformation on their end before delivering the content. You can also use a cropping method: fill, fill with padding, scale down, etc.

Gravity position

When we specify a gravity position to crop an image, the service will keep the area of the image we decide to use as the focal point. We can choose a corner (for example, top left), but also—and this is probably one of the most interesting capabilities on this service—we can specify “special positions”: By using machine learning, we can instruct Cloudinary to use face detection, or even focus on other objects, like an animal or a flower in the picture.

Automatic format

Another cool feature is the automatic format, which will use your request headers to find the most efficient picture format for your browser type and version. For example, if the browser supports it, Cloudinary will return the image in WebP format, which is generally more efficient than standard JPEG, as End Point CTO Jon Jensen demonstrates on his recent blog post.

Automatic format in action: Returning a WebP image in Chrome

Other features

There are many other options for us to choose when querying their API, like setting up a default placeholder when we don’t have an image, applying color transformations, removing red eyes, among other things. The Transformation reference page on their documentation section is a great resource.

NuxtJS integration

The project I mentioned above was a NuxtJS application with a Node.js backend. And since there’s a NuxtJS module for Cloudinary, it made sense to use it instead of building the queries to the API from scratch.

The component works great, except for one bug that we found that didn’t allow us to fully use their image component with server-side rendering enabled. Between that drawback and some issues trying to use the lazy loading setting, we ended up creating a Vue component ourselves that used a standard image tag instead. But we still used their component to generate most of the API calls and render the results.

Below is an example of using the Cloudinary Image component on a Vue template:

<template>
  <div>
    <cld-image
      :public-id="publicId"
      width="200"
      height="200"
      crop="fill"
      gravity="auto:subject"
      radius="max"
      fetchFormat="auto"
      quality="auto"
      alt="An image example with Cloudinary"
    />
  </div>
</template>

Alternatives

Of course, Cloudinary is not the only image processing and CDN company out there: There are other companies offering similar services, like Cloudflare images, Cloudimage, or imagekit.io.

Do you know any other good alternatives, or have you used any other Cloudinary feature that is not listed here? Feel free to add a comment below!

Image compression: WebP presets, HEIC, AVIF, JPEG XL

2022-02-15T00:00:00+00:00

How time flies. Eight years ago I wrote the blog post WebP images experiment on End Point website to describe and demonstrate how the WebP image format can store an equivalent-quality image in a much smaller file size than the older JPEG, PNG, and GIF formats.

My WebP examples there were 17–23% of the JPEGs they came from, or about 5–6× smaller. While experimenting with higher levels of compression, I found that WebP tends to leave less-noticeable artifacts than JPEG does.

The main drawback at the time was that, among major browsers, only Chrome and Opera supported WebP, and back then, Chrome was far less popular than it is now.

Can I use it?

Since Apple’s iOS 14 and macOS 11 (Big Sur) became available in late 2020, the WebP image format now works in all the currently supported major operating systems and browsers: Linux, Windows, and macOS, running Chromium, Chrome, Brave, Edge, Opera, Firefox, and Safari. You can see the specifics at the ever-useful site “Can I use”.

It only took about 10 years! 😁

So for you who are hosting websites, all your site visitors can see WebP images and animations except those vanishing few using Internet Explorer (now long past its end of support by Microsoft and dangerous to use), and people (or their organizations) who intentionally do not allow their older browsers and operating systems to be updated.

That means you may want to continue using JPEG, PNG, and/or GIF images in places crucial for rendering your main site features, for people to be able to understand the main things your site is trying to communicate.

But for any images that are less essential and primarily making it prettier, or where you don’t mind suggesting that users of old browsers upgrade to see them, WebP can now be your default image format.

How do I create WebP images?

Mobile phones and digital cameras typically save JPEG, HEIF, or raw (uncompressed) images. Some stock photography collections offer WebP downloads, but many still use JPEG.

So in many cases you won’t be starting with a WebP image, and you’ll convert some other image to WebP, likely after cropping, scaling, and other adjustments.

GIMP (GNU Image Manipulation Program) supports WebP images since about 2017, and Adobe Photoshop does not natively but can use the free WebPShop plugin.

The oldest way to convert images to WebP, and still very useful for batch processing or fine-tuning, is Google’s WebP converter “cwebp”.

cwebp settings

With the power “cwebp” offers comes some complexity, but it is mostly harmless.

Run this to see its many options:

cwebp -longhelp

Or read the same thing in its online documentation.

There among other things you will see the -z option which activates preset features for lossless encoding, with an integer level chosen from 0 to 9 where 0 is fastest but compresses less and 9 is slowest but compresses better. Use this to replace PNG files when you want no degradation of the image at all.

The documentation also shows the useful -preset and -hint options for lossy compression similar to what JPEG does, but better:

-preset <string> ....... preset setting, one of:
                          default, photo, picture,
                          drawing, icon, text

-hint <string> ......... specify image characteristics hint,
                         one of: photo, picture or graph

The meaning of a few of those terms, especially “photo” and “picture”, was not clear to me and not defined elsewhere in the documentation that I could see.

To find that out I had to make a quick trip into the source code, and there are comments explaining each option’s use case a bit:

The comments for the -preset option:

WEBP_PRESET_PICTURE,  // digital picture, like portrait, inner shot
WEBP_PRESET_PHOTO,    // outdoor photograph, with natural lighting
WEBP_PRESET_DRAWING,  // hand or line drawing, with high-contrast details
WEBP_PRESET_ICON,     // small-sized colorful images
WEBP_PRESET_TEXT      // text-like

And the comments for the -hint option:

WEBP_HINT_PICTURE,    // digital picture, like portrait, inner shot
WEBP_HINT_PHOTO,      // outdoor photograph, with natural lighting
WEBP_HINT_GRAPH,      // Discrete tone image (graph, map-tile etc).

So in short, “cwebp” considers a “picture” to be indoors and close-up, while “photo” is outdoors and more likely with more distant focus. That’s good to know.

Batch conversion

With that in mind, I can convert a pile of screenshots that have been collecting on my computer to refer to later. One kind of screenshots I sometimes take is of video meetings with mostly indoor views of people. I will use the “picture” preset for those.

A simple bash script works well to process many images in a row:

for infile in Screenshot*.png
do
    echo $infile
    base=$(basename "$infile" .png)
    cwebp -preset picture -v "$infile" -o "$base".webp
done

If you have many images to convert to WebP and want to do several at the same time to get done faster, you can use GNU parallel.

My screenshots when converted from PNG to WebP consistently take about 3% of the original space, 33–35× smaller! And the quality looks about the same. Amazing.

These screenshots are 2880×1800 pixels, mostly of Google Meet low-bandwidth video streams. The originals of these screenshots don’t look particularly good to begin with, with some blurriness. But exactly because of this, there is no reason for me to keep a larger high-quality original here. The much smaller WebP is fine.

Competitors to WebP

Other newer image formats have also been in the works for years, chasing some of the same goals. Should we skip WebP and use one of them instead?

HEIC

The HEIC (High-Efficiency Image Container) subset of the HEIF (High Efficiency Image File Format) standard uses High Efficiency Video Coding (HEVC, H.265) for storing images with a .heic suffix.

Compared to JPEG, HEIC offers the nice advantages of smaller file sizes for the same quality level (roughly half the size of an equivalent JPEG), and animation support (to replace GIF).

On the downside, HEIC is encumbered by patents that limit its use for major commercial projects, even on devices licensed for consumer use. It is also slower to encode/decode. And, concerning for archivists, HEIC shows severe visual damage to the entire image if part of the file is corrupted. In contrast, with corrupted JPEG files the visual damage is typically localized to particular smaller square regions, not the entire image.

HEIC has been used in Apple’s operating systems since the 2017 release of the iPhone 7 and iOS 11 and macOS 10.13 (High Sierra). Support was later added to Windows 10, Android 9, and Ubuntu 20.04.

As of this writing, no major browsers support HEIC natively, not even Apple’s own Safari.

So for now, HEIC is primarily used by Apple to store photos more efficiently on its mobile devices.

AVIF

The AVIF (AV1 Image File Format) competes with HEIC and, confusingly, uses the same HEIF container file format that HEIC does. That confusion is reduced in practice by its use of the separate file extension .avif.

AVIF is supported in current Chrome, Firefox, and Opera. Support was added to WebKit in 2021, but it still has not made its way into Safari. It also works in newer VLC, GIMP, Windows, Android, etc.

Netflix has a very detailed blog post comparing AVIF to JPEG and showing AVIF’s many advantages.

JPEG XL

A semi-compatible successor to JPEG has long been in the works, and JPEG XL seems likely to eventually fill that role.

Whereas the other new image formats mentioned above usually lose some quality when recompressing JPEG and other images that were already lossy-compressed, according to the Joint Photographic Experts Group (JPEG):

Existing JPEG files can be losslessly transcoded to JPEG XL, significantly reducing their size.

It has been reported that JPEG XL is expected to become available in its final standard form in 2022, and support is already available in preliminary form in some software (see the Wikipedia JPEG XL article).

That of course means that for now, no major browsers support JPEG XL natively.

Use WebP now

So all the other new options are not yet usable for general web images. WebP is the current obvious choice, whether you want a lossy replacement for JPEG photos, a lossless replacement for PNG images, or a replacement for GIF animations.

Our developers have set up automatic server-side app conversion of high-quality PNG originals to WebP or JPEG on the fly, with the image size dependent on the browser viewport size. And we have worked with Cloudinary, Cloudflare, and other CDNs to use their image conversion services. We are available to help with your projects too.

Reference

The Mozilla Development Network (MDN) has excellent documentation of web image format details, filename suffixes, and support in the major browsers in its Image file type and format guide.

Fun with Rational Numbers

2022-02-11T00:00:00+00:00

So, you say you adored learning long division in elementary school? Well here we are revisiting that wonderful activity we have missed so much: fabulous magical moments like dividing 1.0 by 7. The especially tedious exercise of dragging over one more of the endless 0’s and seeing how many 7’s would fit into that woefully large remainder. Soon the delight mounts as you desperately hope the next digit will be the last and you discover that beautiful 1 as a remainder.

Undoubtedly you have wondered, like I have, what would happen if you divided 1.0 by 19? Would it repeat after only 8 digits? How many digits would it really take before it repeats?

That’s exactly how I stumbled into the wonderful world of rational numbers. There I was, wondering and wandering along the gloomy Belgian beach — the tide way out as far as you could see — in the winter wind, and I couldn’t stop thinking how long it would take for that mantissa to repeat. I had actually already tried 1.0 over 19. Finally, on that miserable day, I got the courage to divide 1 by 23.

Fortunately for me, I wasn’t alone, and I was able to enlist, coax, oblige, implore the help of baffled little Susan, my daughter who was 7 years old at the time. Her job would be to write out the digits in the sand as I discovered them, yelling them out to her, getting farther and farther away from her as the calculation got longer and longer. Finally, shrieking in the wind with delight as I hit 1 as the remainder, Susan seemed confused but amused.

Well, it was so much fun I decided to share my excitement and enthusiasm while at the same time bringing attention to these often overlooked treasure chests of digits.

After my satisfaction in coming up with the answer, the pleasure of which clearly eluded my daughter who was overjoyed that this exercise was over, I decided to write a computer program to discover how these numbers might turn out as the values became larger.

Numbers as music

As a composer, I thought it might be nice to portray the results in a sound representation. My first attempt was to use a fundamental sine wave and have each digit represent the number of overtones in the resulting sound. So the fundamental stayed constant, while the overtones quivered above not unlike Mongolian, Sardinian, or Tibetan Buddhist monk throat singing.

The digit 0 became the lonely fundamental, the digit 1 became the first harmonic an octave above, 2 was an octave and a fifth, and so on, until 9 the 10th harmonic. For timing I added varying amounts of delay between the sound events proportional to the digits calculated.

The first incarnation of my computer program was in 1999 in the programming language C, creating a sound wave file as an output. It created an interesting harmonic whistling kind of wind chime not unlike serrated plastic tubes that you swing around your head — the faster you swing, the higher the harmonics generated.

Later, I thought of applying the repeating nature of the calculated series of numbers to create a periodic waveform itself, instead of using sine waves. Seems like an obvious path to explore, doesn’t it? I still have to try that sound experiment some day.

Visualizing numbers

However, before continuing with sound, I did create a simple visual representation in JavaScript by making gradient-filled colored rectangles overlaid by arcs and line segments.

Let’s look at my program showing results for some interesting numbers.

For example, here is a screenshot of what the reciprocal of 7 (1 divided by 7) looks like in my little program:

1/7

1/13

1/523

A demo

Click here to try it out for yourself!

And make sure your sound is turned up a bit.

Two types of sounds are generated: one, by ‘subtractive’ synthesis, using filtered white noise, and the second, by just using a sampled plucked string sound. The first one, whistle-like, starts with white noise bandpass filtered with the center frequency set at the notes of a piano and the bandwidth or Q of each filter set at one semitone. This produces airy, whispery, wind-like flute sounds.

Repeating digits

By the way, it turns out that:

1/19 produces 18 digits before repeating
1/23 produces 22
1/29 ⇒ 28
1/263 ⇒ 262 repeating digits, etc.

Nice pattern, right? Seems like you can safely say that a reciprocal will never have a mantissa with more digits than the number itself.

The Web Audio API components allow you to construct ‘circuits’ like you would with wires on an analog synthesizer, plugging outputs of one component into another, by ‘connecting’ modules.
Olivia Jack’s work:
- PIXELSYNTH is a very nice version of this concept.
- Her main website has other very nice visual coding experiments.

Liquid Galaxy at 2017 BOMA International Annual Conference & Expo

2017-07-03T00:00:00+00:00

We just returned from Nashville after bringing Liquid Galaxy to the 2017 BOMA International Annual Conference & Expo. Commercial real estate has always been a seamless fit for Liquid Galaxy due to the system’s ability to showcase real estate and property data in an interactive and panoramic setting. Within real estate, Liquid Galaxy was first used by CBRE and has since been adopted by Hilton, JLL, and Prologis to name a few.

In preparation for BOMA (Building Owners and Managers Association), we prepared sample commercial real estate content on our content management system to be displayed on the Liquid Galaxy. This included the creation of content about Hudson Yards, the new development being built in lower Manhattan.

The content that was created demonstrates how brokers and directors at commercial real estate companies can tell an immersive, panoramic, and interactive story to their potential clients and investors. Future developments can be shown in developing areas, and with the content management system you can create stories that include videos, images, and 3D models of these future developments. The following video demonstrates this well:

We were able to effectively showcase our ability to incorporate 3D models and mapping layers at BOMA through the use of Google Earth, Cesium, ArcGIS, Unity, and Sketchfab. We were also able to pull data and develop content for neighboring booths and visitors, demonstrating what an easy and data-agnostic platform Liquid Galaxy can be.

We’re very excited about the increasing traction we have in the real estate industry, and hope our involvement with BOMA will take that to the next level. If you’d like to learn more about Liquid Galaxy, please visit our website.

Updating Firefox and the Black Screen

2014-10-16T00:00:00+00:00

If you are updating your Firefox installation for Windows and you get a puzzling black screen of doom, here’s a handy tip: disable graphics acceleration.

The symptoms here are that after you upgrade Firefox to version 33, the browser will launch into a black screen, possibly with a black dialog box (it’s asking if you want to choose Firefox to be your default browser). Close this as you won’t be able to do much with it.

Launch Firefox by holding down the SHIFT key and clicking on the Firefox icon. It will ask if you want to reset Firefox (Nope!) or launch in Safe mode (Yes).

Once you get to that point, click the “Open menu” icon (three horizonal bars, probably at the far right of your toolbar). Choose “Preferences”, “Advanced”, and uncheck “Use hardware acceleration when available”.

Close Firefox, relaunch as normal, and you should be AOK. You can try re-enabling graphics acceleration if and when your graphics driver is updated.

Reference: here.

Point Clouds on the Liquid Galaxy

2014-07-31T00:00:00+00:00

Image by Stoermerjp, unmodified (CC BY-SA 3.0)

The Liquid Galaxy began as a system to display geographic data through Google Earth, but it has expanded quickly as a display platform for other types of information. We’ve used Liquid Galaxies for panoramic images and video, three dimensional models of all sorts, as well as time-lapse renderings of weather, infrastructure, and economic data. Now we’ve added support for a new type of data, the point cloud.

“Point cloud” is simply the common term for a data set consisting of individual points, often in three-dimensional space, and frequently very large, containing thousands or millions of entries. Points in a cloud can include not just coordinate data, but other information as well, and because this sort of data can come from many different fields, the possible variations are endless. For instance, the terrain features visible in Google Earth began life as point clouds, the output of an aerial scanning device such as a LIDAR scanner. These scanners sweep their field of view rapidly, scanning millions of points to determine their location and any other interesting measurements—color, for instance, or temperature—and with that data create a point cloud. Smaller scale hardware scanners have made their way into modern life, scanning rooms and buildings, or complex objects. A few years ago, the band Radiohead collaborated with Google to use the 3-D scanning techniques to film a music video, and published the resulting point cloud on Google Code.

Image by Xorx, unmodified (CC BY-SA 3.0)

For the Liquid Galaxy platform, we modified an open source point cloud viewer called Potree to allow one master instance to control several others. Potree is a WebGL application. It runs in a browser, and depends on three.js, a widely used WebGL library. Generally speaking, to convert an application to run on a Liquid Galaxy, the developer must give the software two separate modes: one which allows normal user control and transmits information to a central data bus, and another which receives information from the bus and uses it to create its own modified display. In this case, where the application simply loads a model and lets the user move it around, the “master” mode tracks all camera movements and sends them to the other Liquid Galaxy nodes, which will draw the same point cloud in the same orientation as the master, but with the camera pointing offset to the left or right a certain amount. We’ve dubbed our version lg-potree.

This marks the debut of a simple three.js extension we’ve been working on, which we’ve called lg-three.js, designed to make it easy to adapt three.js applications for the Liquid Galaxy. lg-three.js gives the programmer an interface for capturing and serializing things like camera movements or other scene changes on the master node, and de-serializing and using that data on the other nodes, hopefully without having to modify the original application much. Some applications’ structure doesn’t lend itself well to lg-three, but potree proved fairly straightforward.

With that introduction, please enjoy this demonstration.

Creating a Symbol Web Font

2014-07-17T00:00:00+00:00

Creating a custom font that only includes a few characters can be very useful. I was looking for a good way to display left and right arrows for navigating between clients and also between team members on our site and after doing some research, creating a custom font seemed like a good way that would be small and that would support all kinds of screens and browsers. So, here I’ll show how to create a web font with a few custom characters in it that you can use on your website.

You’ll need to get the free, open source vector graphics editor Inkscape and familiarize yourself with its drawing tools.

To start, open Inkscape and open the SVG font editor by clicking Text -> SVG Font Editor. Under the font column, click “New” and then name your new font.

Now you can start adding characters. Begin by adding as many glyphs as you need and choosing letters for your character to be represented by. Only use characters that you can find on a standard QWERTY keyboard, as FontSquirrel (which we’ll use to convert this to a web font) won’t work with, for instance, Unicode special characters.

Now, for each symbol, draw it using Inkscape’s tools and make sure that its dimensions are roughly 750 pixels high (which will be about the height of an uppercase letter) and that it’s flush with the bottom of the canvas.

When your symbol looks like you want it to, make sure that all of the shapes you used to form it are selected and merge them together with Path -> Union. When you’re done, you should have a single object, your glyph. Now, select your glyph and do Path -> Object to Path.

To add this symbol to your new font, select your object and the corresponding glyph and click “Get curves from selection.”

To test, enter the character you’re using for your symbol in the “Preview Text” area. If it shows your symbol, you’re set. Otherwise, you need to make sure that you merged and converted your object to a path correctly.

After you’ve repeated these steps with every symbol you need, save with Inkscape as an SVG. We need to convert this to a TrueType font, so go to www.freefontconverter.com/ (or any other font converter) and convert to .ttf.

The last thing you need to do before using your font in your webpage is convert it to a webfont. Fortunately, FontSquirrel makes this easy. Go to FontSquirrel’s webfont generator and upload your TrueType font. After the conversion has finished, you’ll get a zipfile with the font in several different webfont formats, and even an HTML page telling you how to use it in a webpage.

Have fun creating custom webfonts!

Supporting Apple Retina displays on the Web

2014-05-27T00:00:00+00:00

Apple’s Retina displays (on Mac desktop & laptop computers, and on iPhones and iPads) have around twice the pixel density of traditional displays. Most recent Android phones and tablets have higher-resolution screens as well.

I was recently given the task of adding support for these higher-resolution displays to our End Point company website. Our imagery had been created prior to Retina displays being commonly used, but even now many web developers still overlook supporting high-resolution screens because it hasn’t been part of the website workflow before, because they aren’t simple to cope with, and since most people don’t notice any lack of sharpness without comparing low & high-resolution images side by side.

Most images which are not designed for Retina displays look blurry on them, like this:

The higher-resolution image is on the left, and the lower-resolution image is on the right.

Now, to solve this problem, you need to serve a larger, higher quality image to Retina displays. There are several different ways to do this. I’ll cover a few ways to do it, and explain how I implemented it for our site.

Retina.js

As I was researching ways to implement support for Retina displays, I found that a popular suggestion is the JavaScript library Retina.js. Retina.js automatically detects Retina screens, and then for each image on the page, it checks the web server for a Retina image version under the same name with @2x before the suffix. For example, when fetching the image background.jpg on a Retina-capable system, it would afterward look for background@2x.jpg and serve that if it’s available.

Retina.js makes it relatively painless to deal with serving Retina images to the correct people, but it has a couple of large problems. First, it fetches and replaces the Retina image after the default image, serving both the normal and Retina images to Retina users, greatly increasing download size and time.

Second, Retina.js does not use the correct image if the browser window is moved from a Retina display to a non-Retina display or vice versa when using multiple monitors. For example, if an image is loaded on a standard 1080p monitor and then the browser is moved to a Retina display, it will show the incorrect, low-res image.

Using CSS for background images

Doesn’t the “modern web” have a way to handle this natively in HTML & CSS? For sites using CSS background images, CSS media queries will do the trick:

@media only screen and (-webkit-min-device-pixel-ratio: 2), (min-resolution: 192dpi) {
  .icon {
    background-image: url(icon@2x.png);
    background-size: 20px 20px;
  }
}

But this method only works with CSS background images, so for our site and a lot of other sites, it will only be useful for a small number of images.

Take a look at this CSS-Tricks page for some excellent examples of Retina (and other higher-res display) support.

Server-side checks for Retina images

A very efficient way to handle all types of images is to have the browser JavaScript set a cookie that tells the web server whether to serve Retina or standard images. That will keep data transfer to a minimum, with a minimum of trickery required in the browser. You’ll still need to create an extra Retina-resolution image for every standard image on the server. And you’ll need to have a dynamic web process run for every image served. The Retina Images open source PHP program shows how to do this.

Why we didn’t use these methods

There is one reason common to all of these methods which made us decide against them: All of them require you to maintain multiple versions of each image. This ends up taking a lot of time and effort. It also means your content distribution network (CDN) or other HTTP caches will have twice as many image files to load and cache, increasing cache misses and data transfer. It also uses more disk space, which isn’t a big problem for the small number of images on our website, but on an ecommerce website with many thousands of images, it adds up quickly.

We would feel compelled to have the separate images if it were necessary if the Retina images were much larger and slow down the browsing experience for non-Retina users for no purpose. But instead we decided on the following solution that we saw others describe.

Serving Retina images to everybody (how we did it)

We read that you can serve Retina images to everyone, but we immediately thought that wouldn’t work out well. We were sure that the Retina images would be several times larger than the normal images, wasting a ton of bandwidth for anyone not using a Retina screen. We were very pleasantly surprised to find out that this wasn’t the case at all.

After testing on a few images, I found I could get Retina images within 2-3 KB of the normal images while keeping the visual fidelity, by dropping the JPEG compression rate. How? Because the images were being displayed at a smaller size than they were, the compression artifacts weren’t nearly as noticeable.

These are the total file sizes for each image on our team page:

Retina  Normal  Filename
 10K    9.3K    adam_spangenthal.jpg
 13K     13K    adam_vollrath.jpg
 12K     11K    benjamin_goldstein.jpg
7.6K    4.2K    bianca_rodrigues.jpg
 14K     13K    brian_buchalter.jpg
 13K     15K    brian_gadoury.jpg
7.5K    8.0K    brian_zenone.jpg
9.8K    6.6K    bryan_berry.jpg
 12K     11K    carl_bailey.jpg
6.9K     15K    dave_jenkins.jpg
 13K     13K    david_christensen.jpg
7.7K     21K    emanuele_calo.jpg
 16K     16K    erika_hamby.jpg
 13K     11K    gerard_drazba.jpg
 14K     14K    greg_davidson.jpg
 14K     12K    greg_sabino_mullane.jpg
 14K     15K    jeff_boes.jpg
 14K     12K    jon_jensen.jpg
 13K     12K    josh_ausborne.jpg
 13K     14K    josh_tolley.jpg
 13K     11K    josh_williams.jpg
8.9K    9.5K    kamil_ciemniewski.jpg
 13K     21K    kent_krenrich.jpg
 15K     12K    kiel_christofferson.jpg
9.9K     11K    kirk_harr.jpg
7.7K     13K    marco_manchego.jpg
 12K     13K    marina_lohova.jpg
 14K     11K    mark_johnson.jpg
7.3K     13K    matt_galvin.jpg
 15K     12K    matt_vollrath.jpg
6.6K     14K    miguel_alatorre.jpg
 13K     14K    mike_farmer.jpg
7.1K     19K    neil_elliott.jpg
9.9K    9.0K    patrick_lewis.jpg
 13K    5.6K    phin_jensen.jpg
 12K     14K    richard_templet.jpg
 12K    9.9K    rick_peltzman.jpg
 14K     13K    ron_phipps.jpg
9.7K     14K    selvakumar_arumugam.jpg
9.3K     15K    spencer_christensen.jpg
 12K     12K    steph_skardal.jpg
 15K     18K    steve_yoman.jpg
6.7K     15K    szymon_guz.jpg
7.5K    6.8K    tim_case.jpg
 15K     21K    tim_christofferson.jpg
9.3K     12K    will_plaut.jpg
 12K     14K    wojciech_ziniewicz.jpg
 12K    9.9K    zed_jensen.jpg

TOTALS

Retina: 549.4K
Normal: 608.8K

This is where I found the biggest, and best, surprise. The cumulative size of the Retina image files was less than that of the original images. So now we have support for Retina displays, making our website look nice on modern screens, while actually using less data transfer. We don’t need JavaScript, cookies, or any extra server-side trickery to do this. And best of all, we don’t have to maintain a separate set of Retina images.

Once you’ve seen the difference in quality on a Retina screen or a new Android phone, you’ll wonder how you ever were able to tolerate the lower-resolution images. And at least for our selection of JPEG images, there’s not even a file size penalty to pay!

Reference reading

A guide for creating a better retina web by Ivo Mynttinen
5 Things I Learned Designing For High-Resolution Retina Displays by Lee Munroe
About Proper Image Delivery on the Web on the Safari Developer Library
Serving Images Efficiently to Displays of Varying Pixel Density on the Safari Developer Library

WebP images experiment on End Point website

2014-01-28T00:00:00+00:00

WebP is an image format for RGB images on the web that supports both lossless (like PNG) and lossy (like JPEG) compression. It was released by Google in September 2010 with open source reference software available under the BSD license, accompanied by a royalty-free public patent license, making it clear that they want it to be widely adopted by any and all without any encumbrances.

Its main attraction is smaller file size at similar quality level. It also supports an alpha channel (transparency) and animation for both lossless and lossy images. Thus it is the first image format that offers the transparency of PNG in lossy images at much smaller file size, and animation only available in the archaic limited-color GIF format.

Comparing quality & size

While considering WebP for an experiment on our own website, we were very impressed by its file size to quality ratio. In our tests it was even better than generally claimed. Here are a few side-by-side examples from our site. You’ll only see the WebP version if your browser supports it:

13,420 bytes JPEG	2776 bytes WebP
14,734 bytes JPEG	3386 bytes WebP

The original PNG images were converted by ImageMagick to JPEG, and by cwebp -q 80 to WebP. I think we probably should increase the WebP quality a bit to keep a little of the facial detail that flattens out, but it’s amazing how good these images look for file sizes that are only 17% and 23% of the JPEG equivalent.

One of our website’s background patterns has transparency, making the PNG format a necessity, but it also has a gradient, which PNG compression is particularly inefficient with. WebP is a major improvement there, at 13% the size of the PNG. The image is large so I won’t show it here, but you can follow the links if you’d like to see it:

337,186 bytes	container-pattern.png
43,270 bytes	container-pattern.webp

Browser support

So, what is the downside? WebP is currently natively supported only in Chrome and Opera among the major browsers, though amazingly, support for other browsers can be added via WebPJS, a JavaScript WebP renderer.

Update! As of 2021, all current major browsers support WebP image rendering.

Why don’t the other browsers add support given the liberal license? Especially Firefox you’d expect to support it. In fact a patch has been pending for years, and a debate about adding support still smolders. Why?

WebP does not yet support progressive rendering, Exif tagging, non-RGB color spaces such as CMYK, and is limited to 16,384 pixels per side. Some Firefox developers feel that it would do the Internet community a disservice to support an image format still under development and cause uncertain levels of support in various clients, so they will not accept WebP in its current state.

Many batch image-processing tools now support WebP, and there is a free Photoshop plug-in for it. Some websites are quietly using it just because of the cost savings due to reduced bandwidth.

For our first experiment serving WebP images from the End Point website, I decided to serve WebP images only to browsers that claim to be able to support it. They advertise that support in this HTTP request header:

Accept: image/webp,*/*;q=0.8

That says explicitly that the browser can render image/webp, so we just need to configure the server to send WebP images. One way to do that is in the application server, by having it send URLs pointing to WebP files.

Let’s plan to have both common format (JPEG or PNG) and WebP files side by side, and then try a way that is transparent to the application and can be enabled or disabled very easily.

Web server rewrites

It’s possible to set up the web server to transparently serve WebP instead of JPEG or PNG if a matching file exists. Based on some examples other people posted, we used this nginx configuration:

set $webp "";
set $img "";
if ($http_accept ~* "image/webp") { set $webp "can"; }
if ($request_filename ~* "(.*)\.(jpe?g|png)$") { set $img $1.webp; }
if (-f $img) { set $webp "$webp-have"; }
if ($webp = "can-have") {
    add_header Vary Accept;
    rewrite "(.*)\.\w+$" $1.webp break;
    break;
}

It’s also good to add to /etc/nginx/mime.types:

image/webp .webp

so that .webp files are served with the correct MIME type instead of the default application/octet-stream, or worse, text/plain with perhaps a bogus character set encoding.

Then we just make sure identically-named .webp files match .png or .jpg files, such as those for our examples above:

-rw-rw-r-- 337186 Nov  6 14:10 container-pattern.png
-rw-rw-r--  43270 Jan 28 08:14 container-pattern.webp
-rw-rw-r--  14734 Nov  6 14:10 josh_williams.jpg
-rw-rw-r--   3386 Jan 28 08:14 josh_williams.webp
-rw-rw-r--  13420 Nov  6 14:10 marina_lohova.jpg
-rw-rw-r--   2776 Jan 28 08:14 marina_lohova.webp

A request for a given $file.png will work as normal in browsers that don’t advertise WebP support, while those that do will instead receive the $file.webp image.

The image is still being requested with a name ending in .jpg or .png, but that’s just a name as far as both browser and server are concerned, and the image type is determined by the MIME type in the HTTP response headers (and/or by looking at the file’s magic numbers). So the browser will have a file called $something.jpg in the DOM and in its cache, but it will actually be a WebP file. That’s ok, but could be confusing to users who save the file for whatever reason and find it isn’t actually the JPEG they were expecting.

301/302 redirect option

One remedy for that is to serve the WebP file via a 301 or 302 redirect instead of transparently in the response, so that the browser knows it’s dealing with a different file named $something.webp. To do that we changed the nginx configuration like this:

rewrite "(.*)\.\w+$" $1.webp permanent;

That adds a little bit of overhead, around 100-200 bytes unless large cookies are sent in the request headers, and another network round-trip or two, though it’s still a win with the reduced file sizes we saw. However, I found that it isn’t even necessary right now due to an interesting behavior in Chrome that may even be intentional to cope with this very situation. (Or it may be a happy accident.)

Chrome image download behavior

Versions of Chrome I tested only send the Accept: image/webp [etc.] request header when fetching images from an HTML page, not when you manually request a single file or asking the browser to save the image from the page by right-clicking or similar. In those cases the Accept header is not sent, so the server doesn’t know the browser supports WebP, so you get the JPEG or PNG you asked for. That was actually a little confusing to hunt down by sniffing the HTTP traffic on the wire, but it may be a nice thing for users as long as WebP is still less-known.

Batch conversion

It’s fun to experiment, but we needed to actually get all the images converted for our website. Surprisingly, even converting from JPEG isn’t too bad, though you need a higher quality setting and the file size will be larger. Still, for best image quality at the smallest file size, we wanted to start with original PNG images, not recompress JPEGs.

To make that easy, we wrote two shell scripts for Linux, bash, and cwebp. We found a few exceptional images that were larger in WebP than in PNG or JPEG, so the script deletes any WebP file that is not smaller, and our nginx configuration will in that case not find a .webp file and will serve the original PNG or JPEG.

Full-page download sizes compared

Here are performance tests run by WebPageTest.org using Chrome 32 on Windows 7 on a simulated cable Internet connection. The total download size difference is most impressive, and on a slower mobile network or with higher latency (greater distance from the server) would affect the download time more.

Page URL	With WebP	Without WebP
Bytes	Time	Bytes	Time
https://www.endpointdev.com/	374 KB	2.9s	850 KB	3.4s
https://www.endpointdev.com/team/	613 KB	3.6s	1308 KB	4.1s

Conclusion

This article is not even close to a comprehensive shootout between WebP and other image types. There are other sites that consider the image format technical details more closely and have well-chosen sample images.

My purpose here was to convert a real website in bulk to WebP without hand-tuning individual images or spending too much time on the project overall, and to see if the overall infrastructure is easy enough to set up, and the download size and speed improved enough to make it worth the trouble, and get real-world experience with it to see if we can recommend it for our clients, and in which situations.

So far it seems worth it, and we plan to continue using WebP on our website. With empty browser caches, visit www.endpointdev.com using Chrome and then one of the browsers that doesn’t support WebP, and see if you notice a speed difference on first load, or any visual difference.

I hope to see WebP further developed and more widely supported.

Kamelopard update—Panoramic camera simulation, and splines have returned

2013-07-16T00:00:00+00:00

A few days ago I pushed Kamelopard version 0.0.12 to RubyGems. This version includes a couple big items. The first of these is a new implementation of the spline code that was removed a while ago. As I mentioned in a previous blog post, this original implementation was built in anticipation of an API that never materialized. The new version is built on the same API discussed in the previous post I mentioned, modified to support multidimensional functions. More information about these splines is available on the wiki for Liquid Galaxy.

The other big new feature is the ability to simulate multiple-camera panoramic images, which we’re calling “multicam”. It has its own wiki page as well, but I wanted to describe it in greater detail, because there’s a fair bit of 3D geometry involved that seemed blog-worthy. First, though, it’s important to understand the goal. In a Liquid Galaxy, each instance of Google Earth displays the view from one virtual “camera”. One display’s camera points exactly where you tell it to point; the others point to one side or the other, based on a few settings in a Google Earth configuration file. When placed side-by-side in the right order, these displays form a single panoramic image. Google Earth itself figures out where these displays’ cameras should point, but for some applications, we wanted to be able to calculate those display angles and position the cameras on our own. For instance, it would sometimes be nice to pre-record a particular tour and play it back on a Liquid Galaxy as a simple video. For a seven-screen galaxy, we’ll need seven different video files, each with the same movements to and from the same geographic locations, but each with a slightly different camera orientation.

Camera orientation is controlled by KML AbstractView elements, of which there are two varieties: Camera, and LookAt. The former tells the camera its position and orientation exactly; the latter describes it in terms of another point. For now, multicam only supports Camera objects, because LookAt is somewhat more complicated.

A Camera object gives the camera’s latitude, longitude, and altitude, and three angles called heading, tilt, and roll. The camera initially points straight down, positioned so that north is up. When orienting the camera, Google Earth first rotates around the Z axis, which extends through the lens of the camera, by the heading amount. Then it rotates around the X axis, which extends through the sides of the camera, by the amount given by the tilt angle. Finally, it rotates around the Z axis again, by the roll amount. The Y axis always points in whatever direction the camera thinks is “up”.

With that background, we’re ready to tackle the problem at hand. We’ll consider our cameras as shown in the drawing. Camera 0 is the master view, positively numbered cameras go off to the master camera’s right, and negatively numbered cameras to the left. Each camera has the same view angle. We want an API that will accept the master camera’s orientation, the number of the camera we’re interested in, and the view angle of each camera, and return the orientation for the camera number we gave. We’ll consider each camera’s orientation in terms of two vectors; the first will tell us what direction the camera is pointing, and the second, the up direction. I’ll call these the “camera” vector and the “up” vector. The first step will be to find these two vectors for the camera we want, and the second will be to translate that back into a Camera object.

Getting the position of these vectors for any given camera is a simple application of rotation matrices. In the Kamelopard code, there’s a function defined for rotating around each of the three axes. Here’s the example for the X axis; the others are all similar:

def self.rot_x(a)
   # Convert the angle to radians
   a = a * Math::PI / 180.0
   return Matrix[[1, 0, 0], [0, Math.cos(a), -1 * Math.sin(a)], [0, Math.sin(a), Math.cos(a)]]
end

So we start with a camera pointed at the ground, and we multiply it by a rotation matrix, so that it points in the direction our camera would, before the heading, tilt, and roll rotations are applied. When we perform those rotations, we’ll end up with a vector pointed in the right direction for our camera. This code does just that.

# The camera vector is [0,0,1] rotated around the Y axis the amount
# of the camera angle
camera = rot_y(cam_angle) * Vector[0,0,1]

# The up vector is the same for all cameras
up = Vector[0,1,0]
matrix = rot_z(heading) * rot_x(tilt) * rot_z(roll)
(h, t, r) = vector_to_camera(matrix * camera, matrix * up)

The last line of this snippet calculates the camera and up vectors, and passes them to the vector_to_camera() function, which takes care of the second step in the process: converting these vectors back into a usable heading, tilt, and roll. Two operations fundamental to linear algebra will become important here. Both take two vectors as input. The dot product returns product of the two vectors’ magnitudes and the cosine of the angle between them. We’ll use it here to find the angle between two vectors. The cross product returns a normal vector, or a vector which is perpendicular to the two input vectors.

First we want to calculate the heading, which we can find by calculating the angle between two planes. The first plane is defined by the camera vector and the Z axis, and the second is the plane of the Y and Z axes. To find the angle between two planes, we find the angle between their normal vectors. The normal vector of the YZ plane is simply the X axis; the normal vector for hte first plane is the cross product of the camera vector and the Z axis. The dot product lets us find the angle between these two vectors, which is our heading.

Tilt is simply the angle between the camera vector and the original Z axis, but roll is a bit harder. To find it, we transform the original Y axis—the original “up” vector—using the heading and tilt calculated previously. We then find the angle between it and the current “up” vector, again using the dot product.

These calculations underlie a simple API, which simply takes a view, for the original camera, the number of the camera we’re interested in getting, and the camera angle or total number of cameras. Like this, for instance:

view_hash = {:latitude => otp[:latitude], :longitude => lo,
    :altitudeMode => otp[:altitudeMode], :altitude => otp[:altitude],
    :roll => otp[:roll], :timestamp => Kamelopard::TimeStamp.new('1921-07-29'),
    :heading => otp[:heading], :tilt => otp[:tilt]}
v = Kamelopard::Multicam.get_camera_view(make_view_from(view_hash), camera, nil, CamCount)

Creating custom button graphics in Android

2013-06-07T00:00:00+00:00

In the Android timesheet app I’m working on, I have a scrollable layout of RadioButtons for the user to pick how much time they’ve spent on a project (see my earlier blog post about it), and for that I use custom button graphics to make it look nice. So, I’m going to show you how to do that with 9-patch PNGs and selector XML.

First, what’s a 9-patch PNG? A 9-patch is a special PNG image where you specify regions that can be stretched to make room for text. Android will automatically resize a 9-patch to best fit whatever contents you give it. The tool you need to create a 9-patch image is included in the Android SDK Tools, so download that if you haven’t already.

More information about 9-patch images can be found here.

Okay! I’ve got custom button graphics (72x72 for HDPI screens), drawn in the Gimp and saved in my project’s res/drawable-hdpi/ folder as button_selected.png and button_unselected.png:

To convert it to a 9-patch, browse to the tools/ directory of the Android SDK and run draw9patch. This will open a window with a graphical editor on the left, and a button preview on the right. The editor window is for specifying which parts of the image will be stretched. The top and left edges show this, and the right and bottom edges show which parts of the image can contain the text you put in the button.

When you’ve finished with draw9patch, save the images in the same place, but with .9.png as the file extension (in this case, res/drawable-hdpi/button_selected.png will be res/drawable-hdpi/button_selected.9.png). Make sure to delete the old images, because Android R (the generated resource class) doesn’t use file extensions, so it can’t tell the difference between our two image types.

Now, let’s try making a button with our custom graphics. Add a Button to your Activity XML, like so:

res/layout/activity_main.xml

 <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
   xmlns:tools="http://schemas.android.com/tools"
   android:layout_width="match_parent"
   android:layout_height="match_parent"
   android:paddingBottom="@dimen/activity_vertical_margin"
   android:paddingLeft="@dimen/activity_horizontal_margin"
   android:paddingRight="@dimen/activity_horizontal_margin"
   android:paddingTop="@dimen/activity_vertical_margin"
   tools:context=".MainActivity" >
   <Button
     android:id="@+id/button1"
     android:layout_width="wrap_content"
     android:layout_height="wrap_content"
     android:background="@drawable/button_selected"
     android:onClick="doStuff"
     android:text="@string/hello_world" />
 </RelativeLayout>

Now, when we run it, it looks like this:

So we have our custom background working, but it’s the same (red) whether or not you’re pushing it. To use different images for different states, we can use selector XML. With ours, we just have two images, so it’s simple:

res/drawable/button_selector.xml

 <?xml version="1.0" encoding="utf-8"?>
 <selector xmlns:android="http://schemas.android.com/apk/res/android" >
   <!-- When selected, use this image -->
   <item android:drawable="@drawable/button_selected"
     android:state_pressed="true" />
   <!-- When not selected, use this image -->
   <item android:drawable="@drawable/button_unselected"
     android:state_pressed="false" />
 </selector>

Now, instead of pointing our Button at an image directly, we can reference the XML instead:

res/layout/activity_main.xml

 <font color="#969696">android:layout_height="wrap_content"</font>
 android:background="@drawable/button_selector"
 <font color="#969696">android:onClick="doStuff"</font>

And when we run it, it looks great (assuming you like bright red)!

This project is on GitHub, if you’d like to download it and try some stuff.

CSS Conf 2013 — When Bootstrap Attacks!

2013-06-03T00:00:00+00:00

I attended the inaugural CSS Conf last week at Amelia Island, Florida. The conference was organized by Nicole Sullivan, Brett Stimmerman, Jonathan Snook, and Paul Irish and put on with help from a host of volunteers. The talks were presented in a single track style on a wide range of CSS-related topics; there was something interesting for everyone working in this space. I really enjoyed the conference, learned lots and had great discussions with a variety of people hacking on interesting things with CSS. In the coming days I will be blogging about some of the talks I attended and sharing what I learned, so stay tuned!

When Bootstrap Attacks

Pamela Fox had the opening slot and spoke about the experiences and challenges she faced when upgrading Bootstrap to V2 in a large web app (Coursera). What she initially thought would be a quick project turned into a month-long “BOOTSTRAPV2ATHON”. Bootstrap styles were used throughout the project in dozens of PHP, CSS and JavaScript files. The fact that Bootstrap uses generic CSS class names like “alert”, “btn”, error etc made it very difficult to grep through the codebase for them. The Bootstrap classes were also used as hooks by the project’s JavaScript application code.

Lessons Learned

Fox offered some tips for developers facing a similar situation. The first of which was to prefix the Bootstrap CSS classes (e.g. .tbs-alert) in order to decouple Bootstrap from the customizations in your project. Some requests have been made to the Bootstrap team on this front but the issue has not been addressed yet. In the meantime, devs can add a task to their build step (e.g. Grunt, the asset pipeline in Rails etc) to automate the addition of prefixes to each of the CSS classes.

Another tip is to avoid using Bootstrap CSS classes directly. Instead, use the “extend” functionality in your preprocessor (Sass, Less, Stylus etc) of choice. For example:

  .ep-btn {
    @extend .btn
      &:hover {
          @extend .btn:hover
      }
  }

This way your project can extend the Bootstrap styles but keep your customizations separate and not closely coupled to the framework.

The same logic should also be applied to the JavaScript in your project. Rather than using the Bootstrap class names as hooks in your JavaScript code, use a prefix (e.g. js-btn) or use HTML5 data attributes. Separating the hooks used for CSS styles from those used in JavaScript is very helpful when upgrading or swapping out a client-side framework like Bootstrap.

Test All Of The Things

Pamela wrapped up the talk by explaining how testing front end code would ease the pain of upgrading a library next time. There are many testing libraries available today which address some of these concerns. She mentioned mocha, Chai, jsdom and Selenium which all look very helpful. In addition to testing front end code she offered up the idea of “diffing your front end” in a visual way. This concept was very interesting to someone who ensures designs are consistent across a wide array of browsers and devices on a daily basis.

Needle is a tool which allows you to do this automatically. Once you develop a test case, you can run Needle to view a visual diff of your CSS changes. I think this is an excellent idea. Pamela also noted that the combination of Firefox screenshots and Kaleidoscope could be used manually in much the same way.

Many thanks to Pamela for sharing this! The slides for this talk can be viewed here and the talk was recorded so the video will also be available sometime soon.

Building Xpdf on Ubuntu

2011-08-31T00:00:00+00:00

It may happen that you need to use Xpdf, even though it no longer ships with Ubuntu and is considered … outdated? buggy? insecure? In any case, it still renders some PDFs that Poppler-based viewers such as Evince don’t, or allows some troublesome PDFs to print as fonts and line art instead of a rasterized mess.

Here’s how I built and installed xpdf 3.02 on Ubuntu 11.04 (Natty Narwhal) x86_64:

sudo apt-get install libfreetype6-dev libmotif-dev
wget ftp://ftp.foolabs.com/pub/xpdf/xpdf-3.02.tar.gz  # now 3.03 is current
tar xzpf xpdf-3.02.tar.gz
cd xpdf-3.02
./configure --with-freetype2-library=/usr/lib/x86_64-linux-gnu \
    --with-freetype2-includes=/usr/include/freetype2 \
    --with-Xm-library=/usr/lib \
    --with-Xm-includes=/usr/include/Xm
make
# see lots of warnings!
sudo make install

That’s it. Not as nice as the old native Debian/Ubuntu packages, but gets the job done.

Paperclip in Spree: Extending Product Image Sizes

2011-06-06T00:00:00+00:00

Spree uses the popular gem Paperclip for assigning images as attachments to products. The basic installation requires you to install the gem, create a migration to store the paperclip-specific fields in your model, add the has_attached_file information to the model with the attachment, add the ability to upload the file, and display the file in a view. In Spree, the Image model has an attached file with the following properties:

class Image < Asset
  ...
  has_attached_file :attachment,
                    :styles => { :mini => '48x48>',
                      :small => '100x100>',
                      :product => '240x240>',
                      :large => '600x600>'
                    },
                    :default_style => :product,
                    :url => "/assets/products/:id/:style/:basename.:extension",
                    :path => ":rails_root/public/assets/products/:id/:style/:basename.:extension"
  ...
end

As you can see, when an admin uploads an image, four image sizes are created: large, product, small, and mini.

Four images are created per product image uploaded in Spree (Note: not to scale).

Last week, I wanted to add several additional sizes to be created upon upload to improve performance. This involved several steps, described below.

Step 1: Extend attachment_definitions

First, I had to override the image attachment styles, with the code shown below. My application is running on Spree 0.11.2 (Rails 2.3.*), so this was added inside the extension activate method, but in Rails 3.0 versions of Spree, this would be added inside the engine’s activate method.

Image.attachment_definitions[:attachment][:styles].merge!(
      :newsize1 => '200x200>',
      :newsize2 => '284x284>'
)

Step 2: Add Image Helper Methods

Spree has the following bit of code in its base_helper.rb, which in theory should create methods for calling each image (mini_image, small_image, product_image, large_image, newsize1_image, newsize2_image):

Image.attachment_definitions[:attachment][:styles].each do |style, v|
    define_method "#{style}_image" do |product, *options|
      options = options.first || {}
      if product.images.empty?
        image_tag "noimage/#{style}.jpg", options
      else
        image = product.images.first
        options.reverse_merge! :alt => image.alt.blank? ? product.name : image.alt
        image_tag image.attachment.url(style), options
      end
    end
  end

But for some reason in this application, perhaps based on order of extension evaluation, this was only applied to the original image sizes. I remedied this by adding the following code to my extension base helper:

  [:newsize1, newsize2].each do |style|
    define_method "#{style}_image" do |product, *options|
      options = options.first || {}
      if product.images.empty?
        image_tag "noimage/#{style}.jpg", options
      else
        image = product.images.first
        options.reverse_merge! :alt => image.alt.blank? ? product.name : image.alt
        image_tag image.attachment.url(style), options
      end 
    end 
  end

Step 3: Create Cropped Images for Existing Images

Finally, instead of requiring all images to be re-uploaded to create the new cropped images, I wrote a quick bash script to generate images with the new sizes. This script was placed inside the RAILS_ROOT/public/assets/products/ directory, where product images are stored. The script iterates through each existing directory and creates cropped images based on the original uploaded image with the ImageMagick command-line tool, which is what Paperclip uses for resizing.

#!/bin/bash

for i in `ls */original/*`
do
    image_name=${i#*original\/}
    dir_name=${i/\/original\/$image_name/}
    mkdir $dir_name/newsize1/ $dir_name/newsize2/
    convert $i -resize '200x200' $dir_name/newsize1/$image_name
    convert $i -resize '284x284' $dir_name/newsize2/$image_name
    echo "created images for $i"
done

Step 4: Update Views

Finally, I added newsize1_image and newsize2_image methods throughout the views, e.g.:

<%= link_to newsize1_image(product), product %>
<%= link_to newsize2_image(taxon.products.first), seo_url(taxon) %>

Conclusion

It would be ideal to remove Step 2 described here by investigating why the image methods are not defined by the Spree core BaseHelper module. It’s possible that this is working as expected on more recent versions of Spree. Other than that violation of the DRY principle, it is a fairly simple process to extend the Paperclip image settings to include additional sizes.

Determining dominant image color

2011-04-21T00:00:00+00:00

This grew out of a misunderstanding of a client’s request, so it never saw the light of day, but I thought it was an interesting problem.

The request was to provide a “color search” of products, i.e., “show me all the orange products”. Finding the specific products was not a challenge since it was just a database query. Instead I was interested in how to choose a “representative image” from among the available images for that product. (And as it turns out, the image filename gave me that information, but let’s assume you don’t have that luxury: how do you tell, from a group of images, which one is “more orange” than the others?)

Of course, this depends on the composition of the image. In this case, I knew that the majority were of solid-color (or two- or three-color at most) products on a white background. The approach that was settled on was to severely pixellate the image into something like 20x20 (arbitrary; this could be very dependent on the images under study, or the graphics library in use). If you also supply a color palette restricted to the colors you are interested in matching (e.g., primary, and secondary colors, plus perhaps black, white, and gray), you would have a roster of the colors represented.

Another approach makes use of the powerful ImageMagick library. There’s a huge list of examples and instructions at https://www.imagemagick.org/Usage/quantize/, but for my purposes this short sample will do:

$ convert Waffle.jpg -scale 1x1\! -format '%[pixel:u]' info:-
rgb(219,166,94)

Here we reduce an image to a single pixel, then report the RGB value of that color. After that it’s just a matter of determining how close this “average” is to your desired color.

(Image|Graphics)Magick trick for monitoring or visualizations

2010-11-03T00:00:00+00:00

It’s a good time for all when we start poking fun at the visual assault of stereotypical PowerPoint presentations. On the other hand, when data is presented in an effective visual format, human brains are able to quickly grasp the ideas involved and pick out important pieces of information, such as “outliers”.

Without getting into a long trumpeting session about the usefulness of data visualization (there are plenty of books on the subject), I’d like to jump directly into a Magick trick or two for creating simple visualizations.

Let’s imagine we’ve got a group of machines serving a particular purpose. Now let’s say I want quick insight into not only the internal activity of all 8 machines, but also what the systems believe they are sending to their displays.

With a little magick (of the ImageMagick or GraphicsMagick variety), we can save ourselves from running “ps” and “free” and from having to be in the same room (or the same country) as the system we’re checking up on.

First, let’s organize some simple output from the system:

$ echo -en "$(hostname) GPID: $( pgrep googleearth-bin ). APPID: $( pgrep -u root -f sbin/apache2 ).\nCRASH: $( ls -1 ${HOME}/.googleearth/crashlogs/ | wc -l ). MEMF: $( awk '/^MemFree/ {print $2$3}' /proc/meminfo )."

Which gives us output something like this:

lg1 - GPID: 5265. APPID: 10452.
CRASH: 3. MEMF: 4646240kB.

Cool, but we want to combine this with the imagery supposedly being currently displayed by X. So, we turn it into an image that we can overlay, like this:

$ echo -en "$(hostname) GPID: $( pgrep googleearth-bin ). APPID: $( pgrep -u root -f sbin/apache2 ).\nCRASH: $( ls -1 ${HOME}/.googleearth/crashlogs/ | wc -l ). MEMF: $( awk '/^MemFree/ {print $2$3}' /proc/meminfo )." | \
convert -pointsize 18 -background '#00000080' -fill white text:- -trim -bordercolor '#00000080' -border 5x5 miff:/tmp/text

This is one long command and might be hard to read, but it is simply using “convert” to turn the text output into a semi-transparent “miff” image for later use. It would be very easy to put the stat collection into a script on each host, but we’re just going with quick and dirty at the moment.

Second, let’s get our little overlay image composited with a screenshot from X:

$ DISPLAY=:0 import -window root miff:- | composite -gravity south -geometry +0+3 miff:/tmp/text miff:- -resize 600 miff:/tmp/$(hostname).miff

So, in a single pipeline we imported a screenshot of the root window, then used “composite” to overlay our semi-transparent stats image and resize the whole thing to be a bit more manageable.

Finally, we want to perform these things across all the systems and be left with something we can quickly glance at to see if there are obvious problems. So, let’s create a quick shell loop and execute our commands via ssh, placing the resize/composite burden on the shoulders of each individual system (be sure to escape variables for remote interpolation!):

#!/bin/bash

#collect data first
for system in `seq 1 8`; do
 ssh user@$system "

echo -en \"\$(hostname) GPID: \$( pgrep googleearth-bin ). APPID: \$( pgrep -u root -f sbin/apache2 ).\nCRASH: \$( ls -1 \${HOME}/.googleearth/crashlogs/ | wc -l ). MEMF: \$( awk '/^MemFree/ {print \$2\$3}' /proc/meminfo )." | \
convert -pointsize 18 -background '#00000080' -fill white text:- -trim -bordercolor '#00000080' -border 5x5 miff:/tmp/text;

DISPLAY=:0 import -window root miff:- | \
composite -gravity south -geometry +0+3 miff:/tmp/text miff:- -resize 600 miff:-" >/tmp/system${system}.miff;

done

#make a montage of the data
montage -monitor -background black -tile 8x1 -geometry +5+0 \
 /tmp/system{6,7,8,1,2,3,4,5}.miff \
 /tmp/system-montage.png && rm -f /tmp/system?.miff

With something so simple, we can quickly view from New York what’s happening on systems installed in California, like so:

JPEG compression: quality or quantity?

2009-12-24T00:00:00+00:00

There are many aspects of JPEG files that are interesting to web site developers, such as:

The optimal trade off between quality and file size for any encoder and uncompressed source image.
Reducing size of an existing JPEG image when the uncompressed source is unavailable, but still finding the same optimal trade-off.
Comparison of different encoders and/or settings for quality at a given file size.

Two essential factors are file size and image quality. Bytes are objectively measurable, but image quality is much more nebulous. What to one person is a perfectly acceptable image is to another a grotesque abomination of artifacts. So the quality factor is subjective. For example, Steph sent me some images to compare compression artifacts. Here is the first one with three different settings in ImageMagick: 95, 50, and 8:

Compare the subtle (or otherwise) differences in the following images (mouseover shows the filesize and compression setting):

Mouseover each image for the file size and ImageMagick compression setting. Additional comparisons are below. Each image can be opened in a separate browser tab for easy A/B comparison. I think many would find the setting of 8 to have too many artifacts, even though it’s 10 times smaller than image compressed at a setting of 95. Some would find the setting of 50 to be an acceptable tradeoff between quality and size, since it sends 3.4 times fewer bytes.

Here is the code I wrote to make the comparison (shell script is great for this stuff):

#!/bin/bash
HTML_OUTFILE=comparison.html
echo '<html>' > $HTML_OUTFILE

write_img_html () {
    size=`du -h --apparent-size $1 | cut -f 1`
    if [ -n "$2" ]; then
       qual="setting: $2"
    fi
    cat <<EOF >>$HTML_OUTFILE
<a href="$1"><img src="$1" title="size: $size $qual"></a>
EOF
}

for name in image1 image2; do
    orig=$name-original.jpg
    resized=$name-300.png

    echo Resizing $orig to 300 on longest side: $resized...
    convert $orig -resize 300x300 $resized
    write_img_html $resized "lossless"

    for quality in 100 95 85 50 20 8 1; do
        echo Creating JPEG quality $quality...
        jpeg=$name-300-q-$quality.jpg
        convert $resized -strip -quality $quality $jpeg
        write_img_html $jpeg $quality
    done
done

Another factor that often comes into play is how artifacts in the image (e.g. aliasing, ringing, noise) combine with JPEG compression artifacts to exacerbate quality problems. So one way to get smaller file sizes is to reduce the other types of artifacts in the image, thereby allowing higher JPEG compression.

The most common source of artifacts is image resizing. If you are resizing the images, I strongly recommend using a program that has a high quality filter. Irfanview and ImageMagick are two good choices.

The ideal situation is this:

Uncompressed source image
Full-resolution if you will be handling the resize
Absent artifacts such as aliasing
Resize performed with good software like ImageMagick
JPEG compression chosen based on subjective quality assessment.

Choosing the trade-off between quality and file size is difficult in part because it varies by image content. Images with lots of small color details (e.g. bright fabric threads; AKA high spatial frequency chroma) stand less compression than images that only have medium sized details that do not have important and minute color information.

One of the settings that is important for small web images is removal of the color space profile (e.g. sRGB). The only time it is needed is when there is a good reason for using non-sRGB JPEG, such as when you are certain that your users will have color managed browsers. Removing it can shave off 5KB or so; software will assume images without profiles have an sRGB profile. It can be removed with the -strip parameter of ImageMagick.

As for choosing the specific compression settings, keep in mind that there are over 30 different types of options/techniques that can be used in compressing the image. Most image programs simplify that to a sliding scale from 0 to 100, 1 to 12, or something else. Keep in mind that even when programs use the same scale (e.g. 0 to 100), they probably have different ideas of what the numbers mean. 95 in one program may be very different than 95 in another.

If bandwidth is not an issue, then I use a setting of 95 on ImageMagick, because in normal images I can’t tell the difference between 95 and 100. But when file size in an important concern, I consider 85 to be the optimal setting. In this image, the difference should be clear, but I generally find that cutting filesize in half is worth it. Below 85, the artifacts are too onerous for my taste.

You don’t often hear about web site visitors’ dissatisfaction with compression artifacts, so you might be tempted to just reduce file sizes even beyond the point where it gets noticable. But I think there is a subliminal effect from the reduced image quality. Visitors may not stop visiting the site immediately, but my gut feeling is it leaves them with a certain impression in their mind or taste in their mouth. I would guess that user testing might result in comments such as “the X web site is not the same high-grade quality as the Y web site”, even if they don’t put it into words as specific as “the compression artifacts make X look uglier than Y”. Even if that pet theory is true, it still has to be balanced against the benefit of faster page loading times.

Ideally, the tradeoff between quality and page loading time would be a choice left to the user. Those who prefer fewer artifacts could set their browser to download larger, less-compressed image files than the default, while users with low bandwidth could set it for more compressed images to get a faster page load at the expense of quality. I could imagine an Apache module and corresponding Firefox add-on some day.

Regarding the situation where you want to reduce the file size of existing JPEGs, my advice is to first try (hard) to get the original source files. You can do better (for any given quality/size tradeoff) from those than you can by just manipulating the existing files. If that’s not possible, then the suboptimal workflows like jpegtran, jpegoptim, and doing a full decompress/recompress are the only alternative.

As far as comparing different encoders, I haven’t really looked into that except to compare ImageMagick and Photoshop, where I (subjectively) determined they both had about similar quality for file size (and vice-versa).

Here are all the comparison images. The file size and ImageMagick quality setting are in the rollover. I suggest opening images in browser tabs for easy A/B comparison.

Making a Loading Spinner with tkinter

Prerequisites

Code

Resources

Automating reading the screen and interacting with GUI programs on X Window System

Overview of how the script works

Preparing

Working through an example

How it works in a little more detail

Chaining xdotool actions

Fine-tuning

Learning more

Optimizing media delivery with Cloudinary

Resizing and cropping

Gravity position

Automatic format

Other features

NuxtJS integration

Alternatives

Image compression: WebP presets, HEIC, AVIF, JPEG XL

Can I use it?

How do I create WebP images?

cwebp settings

Batch conversion

Competitors to WebP

HEIC

AVIF

JPEG XL

Use WebP now

Reference

Fun with Rational Numbers

Numbers as music

Visualizing numbers

1/7

1/13

1/523

A demo

Repeating digits

Related resources

Liquid Galaxy at 2017 BOMA International Annual Conference & Expo

Updating Firefox and the Black Screen

Point Clouds on the Liquid Galaxy

Creating a Symbol Web Font

Supporting Apple Retina displays on the Web

Retina.js

Using CSS for background images

Server-side checks for Retina images

Why we didn’t use these methods

Serving Retina images to everybody (how we did it)

Reference reading

WebP images experiment on End Point website

Comparing quality & size

Browser support

Web server rewrites

301/302 redirect option

Chrome image download behavior

Batch conversion

Full-page download sizes compared

Conclusion

Further reading

Kamelopard update—​Panoramic camera simulation, and splines have returned

Creating custom button graphics in Android

CSS Conf 2013 — When Bootstrap Attacks!

When Bootstrap Attacks

Lessons Learned

Test All Of The Things

Building Xpdf on Ubuntu

Paperclip in Spree: Extending Product Image Sizes

Step 1: Extend attachment_definitions

Step 2: Add Image Helper Methods

Step 3: Create Cropped Images for Existing Images

Step 4: Update Views

Conclusion

Determining dominant image color

(Image|Graphics)Magick trick for monitoring or visualizations

JPEG compression: quality or quantity?

Kamelopard update—Panoramic camera simulation, and splines have returned