End Point Dev blogOngoing observations by End Point Dev peoplehttps://www.endpointdev.com/blog/2024-03-05T00:00:00+00:00End Point DevMaking a Loading Spinner with tkinterhttps://www.endpointdev.com/blog/2024/03/making-a-loading-spinner-with-tkinter/2024-03-05T00:00:00+00:00Matt Vollrath
<p><img src="/blog/2024/03/making-a-loading-spinner-with-tkinter/spiral-stairs.webp" alt="An overhead shot of a carpeted spiral staircase, with spiraling railings on either side. The staircase is cut off at the bottom by a wall, so that only half of the circle of stairs is visible. The stairs are enclosed by a semicircular wall, and lit by sunlight streaming through a window on the left. On the right is a window whose view is filled with green leaves."></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>When you need a loading spinner, you really need a loading spinner. Interested in putting something on the screen without installing a pile of dependencies, I reached deep into the toolbox of the Python standard library, dug around a bit, and pulled out the <a href="https://docs.python.org/3/library/tkinter.html">tkinter</a> module.</p>
<p>The tkinter module is an interface to the venerable Tcl/Tk GUI toolkit, a cross-platform suite for creating user interfaces in the style of whatever operating system you run it on. It’s the only built-in GUI toolkit in Python, but there are many worthy alternatives available (see the end of the post for a list).</p>
<p>Here I’ll demonstrate how to make a loading spinnner with tkinter on Ubuntu 22.04. It should work on any platform that runs Python, with some variations when setting up the system for it.</p>
<h3 id="prerequisites">Prerequisites</h3>
<p>My vision for the loading spinner is some spinning dots and a logo, since this is such a convenient branding opportunity. To accomplish this we’ll be extending tkinter with Pillow’s ImageTk capability, which can load a PNG with transparency.</p>
<p>To produce that PNG with transparency, first we may need to rasterize an SVG file, because wise designers work in vectors. This is made trivial by <a href="https://inkscape.org/">Inkscape</a>, a free and complete vector graphics tool:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ inkscape logo.svg -o logo.png
</code></pre></div><p>With the logo in hand, we can move on to setting up our dependencies. Ubuntu’s python3 distribution doesn’t include tkinter by default, so we’ll need to install it explicitly, along with Pillow’s separate ImageTk support:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ sudo apt install python3-tk python3-pil.imagetk
</code></pre></div><p>This may occupy up to 75MB, if Python was already installed. This was still the smallest apt footprint of all of the Python GUI libraries in consideration. Pygame was also a strong contender.</p>
<h3 id="code">Code</h3>
<p>Let’s start with something simple: putting the logo on the screen.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-python" data-lang="python"><span style="color:#888">#!/usr/bin/env python3</span>
<span style="color:#080;font-weight:bold">from</span> <span style="color:#b06;font-weight:bold">PIL</span> <span style="color:#080;font-weight:bold">import</span> Image, ImageTk
<span style="color:#080;font-weight:bold">from</span> <span style="color:#b06;font-weight:bold">tkinter</span> <span style="color:#080;font-weight:bold">import</span> BOTH, Canvas, Tk
<span style="color:#888"># Desired dimensions of our window.</span>
WIDTH, HEIGHT = <span style="color:#00d;font-weight:bold">500</span>, <span style="color:#00d;font-weight:bold">500</span>
<span style="color:#080;font-weight:bold">if</span> __name__ == <span style="color:#d20;background-color:#fff0f0">"__main__"</span>:
<span style="color:#888"># Create the root window object.</span>
root = Tk()
<span style="color:#888"># Create a canvas for drawing our graphics.</span>
canvas = Canvas(root, width=WIDTH, height=HEIGHT, background=<span style="color:#d20;background-color:#fff0f0">"black"</span>)
<span style="color:#888"># Fill the entire window with the canvas.</span>
canvas.pack(fill=BOTH, expand=<span style="color:#00d;font-weight:bold">1</span>)
<span style="color:#888"># Load the logo PNG with regular PIL.</span>
logo_img = Image.open(<span style="color:#d20;background-color:#fff0f0">"logo.png"</span>)
<span style="color:#888"># Convert the logo to an ImageTk PhotoImage.</span>
logo_pi = ImageTk.PhotoImage(logo_img)
<span style="color:#888"># Add our logo image to the canvas.</span>
canvas.create_image(
WIDTH / <span style="color:#00d;font-weight:bold">2</span>,
HEIGHT / <span style="color:#00d;font-weight:bold">2</span>,
image=logo_pi,
)
<span style="color:#888"># Run the tkinter main loop.</span>
root.mainloop()
</code></pre></div><p>This puts the logo in the center of a window, but the logo may be too large or small. Let’s scale it according to the window dimensions, let’s say to about ⅔ of the width so we have some room for spinning dots:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-python" data-lang="python"><span style="color:#888">#!/usr/bin/env python3</span>
<span style="color:#080;font-weight:bold">from</span> <span style="color:#b06;font-weight:bold">PIL</span> <span style="color:#080;font-weight:bold">import</span> Image, ImageTk
<span style="color:#080;font-weight:bold">from</span> <span style="color:#b06;font-weight:bold">tkinter</span> <span style="color:#080;font-weight:bold">import</span> BOTH, Canvas, Tk
<span style="color:#888"># Desired dimensions of our window.</span>
WIDTH, HEIGHT = <span style="color:#00d;font-weight:bold">500</span>, <span style="color:#00d;font-weight:bold">500</span>
<span style="color:#080;font-weight:bold">if</span> __name__ == <span style="color:#d20;background-color:#fff0f0">"__main__"</span>:
<span style="color:#888"># Create the root window object.</span>
root = Tk()
<span style="color:#888"># Create a canvas for drawing our graphics.</span>
canvas = Canvas(root, width=WIDTH, height=HEIGHT, background=<span style="color:#d20;background-color:#fff0f0">"black"</span>)
<span style="color:#888"># Fill the entire window with the canvas.</span>
canvas.pack(fill=BOTH, expand=<span style="color:#00d;font-weight:bold">1</span>)
<span style="color:#888"># Load the logo PNG with regular PIL.</span>
logo_img = Image.open(<span style="color:#d20;background-color:#fff0f0">"logo.png"</span>)
<span style="color:#888"># Resize the logo to about 2/3 the window width.</span>
scaled_w = <span style="color:#038">round</span>(WIDTH * <span style="color:#00d;font-weight:bold">0.6</span>)
scaled_h = <span style="color:#038">round</span>(scaled_w / (logo_img.width / logo_img.height))
logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
<span style="color:#888"># Convert the logo to an ImageTk PhotoImage.</span>
logo_pi = ImageTk.PhotoImage(logo_img)
<span style="color:#888"># Add our logo image to the canvas.</span>
canvas.create_image(
WIDTH / <span style="color:#00d;font-weight:bold">2</span>,
HEIGHT / <span style="color:#00d;font-weight:bold">2</span>,
image=logo_pi,
)
<span style="color:#888"># Run the tkinter main loop.</span>
root.mainloop()
</code></pre></div><p>That’s better. Now let’s add the promised spinning dots. We’ll draw some ovals on the canvas and modify our main loop to animate them:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-python" data-lang="python"><span style="color:#888">#!/usr/bin/env python3</span>
<span style="color:#080;font-weight:bold">import</span> <span style="color:#b06;font-weight:bold">math</span>
<span style="color:#080;font-weight:bold">import</span> <span style="color:#b06;font-weight:bold">time</span>
<span style="color:#080;font-weight:bold">from</span> <span style="color:#b06;font-weight:bold">PIL</span> <span style="color:#080;font-weight:bold">import</span> Image, ImageTk
<span style="color:#080;font-weight:bold">from</span> <span style="color:#b06;font-weight:bold">tkinter</span> <span style="color:#080;font-weight:bold">import</span> BOTH, Canvas, Tk
<span style="color:#888"># Desired dimensions of our window.</span>
WIDTH, HEIGHT = <span style="color:#00d;font-weight:bold">500</span>, <span style="color:#00d;font-weight:bold">500</span>
<span style="color:#888"># Coordinates of the center.</span>
CENTER_X, CENTER_Y = WIDTH / <span style="color:#00d;font-weight:bold">2</span>, HEIGHT / <span style="color:#00d;font-weight:bold">2</span>
<span style="color:#888"># How many spinning dots we want.</span>
NUM_DOTS = <span style="color:#00d;font-weight:bold">8</span>
<span style="color:#080;font-weight:bold">if</span> __name__ == <span style="color:#d20;background-color:#fff0f0">"__main__"</span>:
<span style="color:#888"># Create the root window object.</span>
root = Tk()
<span style="color:#888"># Create a canvas for drawing our graphics.</span>
canvas = Canvas(root, width=WIDTH, height=HEIGHT, background=<span style="color:#d20;background-color:#fff0f0">"black"</span>)
<span style="color:#888"># Fill the entire window with the canvas.</span>
canvas.pack(fill=BOTH, expand=<span style="color:#00d;font-weight:bold">1</span>)
<span style="color:#888"># Load the logo PNG with regular PIL.</span>
logo_img = Image.open(<span style="color:#d20;background-color:#fff0f0">"logo.png"</span>)
<span style="color:#888"># Resize the logo to about 2/3 the window width.</span>
scaled_w = <span style="color:#038">round</span>(WIDTH * <span style="color:#00d;font-weight:bold">0.6</span>)
scaled_h = <span style="color:#038">round</span>(scaled_w / (logo_img.width / logo_img.height))
logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
<span style="color:#888"># Convert the logo to an ImageTk PhotoImage.</span>
logo_pi = ImageTk.PhotoImage(logo_img)
<span style="color:#888"># Add our logo image to the canvas.</span>
canvas.create_image(
CENTER_X,
CENTER_Y,
image=logo_pi,
)
<span style="color:#888"># Radius in pixels of a single dot.</span>
dot_radius = WIDTH * <span style="color:#00d;font-weight:bold">0.05</span>
<span style="color:#888"># Radius of the ring of dots from the center of the window.</span>
dots_radius = WIDTH / <span style="color:#00d;font-weight:bold">2</span> - dot_radius * <span style="color:#00d;font-weight:bold">2</span>
<span style="color:#888"># Helper function to calculate dot position on each update.</span>
<span style="color:#080;font-weight:bold">def</span> <span style="color:#06b;font-weight:bold">get_dot_coords</span>(n: <span style="color:#038">int</span>, t: <span style="color:#038">float</span>):
<span style="color:#d20;background-color:#fff0f0">"""Get the x0, y0, x1, y1 coords of dot at index 'n' at time 't'."""</span>
angle = (n / NUM_DOTS) * math.pi * <span style="color:#00d;font-weight:bold">2</span> + t
x = math.cos(angle) * dots_radius + CENTER_X
y = math.sin(angle) * dots_radius + CENTER_Y
<span style="color:#080;font-weight:bold">return</span> x - dot_radius, y - dot_radius, x + dot_radius, y + dot_radius
<span style="color:#888"># Create all the dots.</span>
t0 = time.monotonic()
<span style="color:#080;font-weight:bold">for</span> n <span style="color:#080">in</span> <span style="color:#038">range</span>(NUM_DOTS):
coords = get_dot_coords(n, t0)
canvas.create_oval(
*coords,
fill=<span style="color:#d20;background-color:#fff0f0">"#888888"</span>,
width=<span style="color:#00d;font-weight:bold">0</span>, <span style="color:#888"># Border width.</span>
tags=<span style="color:#d20;background-color:#fff0f0">f</span><span style="color:#d20;background-color:#fff0f0">"dot_</span><span style="color:#33b;background-color:#fff0f0">{</span>n<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">"</span>,
)
<span style="color:#888"># Set up a custom main loop to animate the moving dots.</span>
<span style="color:#080;font-weight:bold">while</span> <span style="color:#080;font-weight:bold">True</span>:
<span style="color:#888"># Check the time of this update.</span>
t = time.monotonic()
<span style="color:#080;font-weight:bold">for</span> n <span style="color:#080">in</span> <span style="color:#038">range</span>(NUM_DOTS):
<span style="color:#888"># Get the desired coords for this dot at this time.</span>
coords = get_dot_coords(n, t)
<span style="color:#888"># Move the dot on the canvas, finding it by its tag.</span>
canvas.coords(
<span style="color:#d20;background-color:#fff0f0">f</span><span style="color:#d20;background-color:#fff0f0">"dot_</span><span style="color:#33b;background-color:#fff0f0">{</span>n<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">"</span>,
*coords,
)
<span style="color:#888"># Call the required tkinter update function.</span>
root.update()
<span style="color:#888"># Attempt to stabilize the timing of this loop by targeting 60Hz.</span>
<span style="color:#080;font-weight:bold">while</span> t0 < t:
t0 += <span style="color:#00d;font-weight:bold">1</span> / <span style="color:#00d;font-weight:bold">60</span>
time.sleep(t0 - t)
</code></pre></div><p>You may notice that the dots don’t look all that great. There’s no anti-aliasing when drawing shape primitives in tkinter, so the edges look jagged compared to our well-scaled logo image. One hack is to layer slightly larger and dimmer shapes under each object, which you might do like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-python" data-lang="python"><span style="color:#888">#!/usr/bin/env python3</span>
<span style="color:#080;font-weight:bold">import</span> <span style="color:#b06;font-weight:bold">math</span>
<span style="color:#080;font-weight:bold">import</span> <span style="color:#b06;font-weight:bold">time</span>
<span style="color:#080;font-weight:bold">from</span> <span style="color:#b06;font-weight:bold">PIL</span> <span style="color:#080;font-weight:bold">import</span> Image, ImageTk
<span style="color:#080;font-weight:bold">from</span> <span style="color:#b06;font-weight:bold">tkinter</span> <span style="color:#080;font-weight:bold">import</span> BOTH, Canvas, Tk
<span style="color:#888"># Desired dimensions of our window.</span>
WIDTH, HEIGHT = <span style="color:#00d;font-weight:bold">500</span>, <span style="color:#00d;font-weight:bold">500</span>
<span style="color:#888"># Coordinates of the center.</span>
CENTER_X, CENTER_Y = WIDTH / <span style="color:#00d;font-weight:bold">2</span>, HEIGHT / <span style="color:#00d;font-weight:bold">2</span>
<span style="color:#888"># How many spinning dots we want.</span>
NUM_DOTS = <span style="color:#00d;font-weight:bold">8</span>
<span style="color:#888"># Colors for each layer of fake anti-aliasing around each dot.</span>
<span style="color:#888"># Must be in order from back to front.</span>
COLORS = [<span style="color:#d20;background-color:#fff0f0">"#888888"</span>, <span style="color:#d20;background-color:#fff0f0">"#BBBBBB"</span>, <span style="color:#d20;background-color:#fff0f0">"#FFFFFF"</span>]
<span style="color:#080;font-weight:bold">if</span> __name__ == <span style="color:#d20;background-color:#fff0f0">"__main__"</span>:
<span style="color:#888"># Create the root window object.</span>
root = Tk()
<span style="color:#888"># Create a canvas for drawing our graphics.</span>
canvas = Canvas(root, width=WIDTH, height=HEIGHT, background=<span style="color:#d20;background-color:#fff0f0">"black"</span>)
<span style="color:#888"># Fill the entire window with the canvas.</span>
canvas.pack(fill=BOTH, expand=<span style="color:#00d;font-weight:bold">1</span>)
<span style="color:#888"># Load the logo PNG with regular PIL.</span>
logo_img = Image.open(<span style="color:#d20;background-color:#fff0f0">"logo.png"</span>)
<span style="color:#888"># Resize the logo to about 2/3 the window width.</span>
scaled_w = <span style="color:#038">round</span>(WIDTH * <span style="color:#00d;font-weight:bold">0.6</span>)
scaled_h = <span style="color:#038">round</span>(scaled_w / (logo_img.width / logo_img.height))
logo_img = logo_img.resize((scaled_w, scaled_h), Image.LANCZOS)
<span style="color:#888"># Convert the logo to an ImageTk PhotoImage.</span>
logo_pi = ImageTk.PhotoImage(logo_img)
<span style="color:#888"># Add our logo image to the canvas.</span>
canvas.create_image(
CENTER_X,
CENTER_Y,
image=logo_pi,
)
<span style="color:#888"># Radius in pixels of a single dot.</span>
dot_radius = WIDTH * <span style="color:#00d;font-weight:bold">0.05</span>
<span style="color:#888"># Radius of the ring of dots from the center of the window.</span>
dots_radius = WIDTH / <span style="color:#00d;font-weight:bold">2</span> - dot_radius * <span style="color:#00d;font-weight:bold">2</span>
<span style="color:#888"># Helper function to calculate dot position on each update.</span>
<span style="color:#080;font-weight:bold">def</span> <span style="color:#06b;font-weight:bold">get_dot_coords</span>(n: <span style="color:#038">int</span>, t: <span style="color:#038">float</span>, c: <span style="color:#038">int</span>):
<span style="color:#d20;background-color:#fff0f0">"""Get the x0, y0, x1, y1 coords of dot at index 'n' at time 't'.
</span><span style="color:#d20;background-color:#fff0f0"> Inflate the radius by color index 'c'."""</span>
angle = (n / NUM_DOTS) * math.pi * <span style="color:#00d;font-weight:bold">2</span> + t
x = math.cos(angle) * dots_radius + CENTER_X
y = math.sin(angle) * dots_radius + CENTER_Y
<span style="color:#888"># Invert the color index and add to the radius.</span>
radius = dot_radius + (<span style="color:#038">len</span>(COLORS) - c) * <span style="color:#00d;font-weight:bold">0.75</span>
<span style="color:#888">#radius = dot_radius + c</span>
<span style="color:#080;font-weight:bold">return</span> x - radius, y - radius, x + radius, y + radius
<span style="color:#888"># Create all the dots.</span>
t0 = time.monotonic()
<span style="color:#080;font-weight:bold">for</span> c, color <span style="color:#080">in</span> <span style="color:#038">enumerate</span>(COLORS):
<span style="color:#080;font-weight:bold">for</span> n <span style="color:#080">in</span> <span style="color:#038">range</span>(NUM_DOTS):
coords = get_dot_coords(n, t0, c)
canvas.create_oval(
*coords,
fill=color,
width=<span style="color:#00d;font-weight:bold">0</span>, <span style="color:#888"># Border width.</span>
tags=<span style="color:#d20;background-color:#fff0f0">f</span><span style="color:#d20;background-color:#fff0f0">"dot_</span><span style="color:#33b;background-color:#fff0f0">{</span>c<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">_</span><span style="color:#33b;background-color:#fff0f0">{</span>n<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">"</span>,
)
<span style="color:#888"># Set up a custom main loop to animate the moving dots.</span>
<span style="color:#080;font-weight:bold">while</span> <span style="color:#080;font-weight:bold">True</span>:
<span style="color:#888"># Check the time of this update.</span>
t = time.monotonic()
<span style="color:#080;font-weight:bold">for</span> c, color <span style="color:#080">in</span> <span style="color:#038">enumerate</span>(COLORS):
<span style="color:#080;font-weight:bold">for</span> n <span style="color:#080">in</span> <span style="color:#038">range</span>(NUM_DOTS):
<span style="color:#888"># Get the desired coords for this dot at this time.</span>
coords = get_dot_coords(n, t, c)
<span style="color:#888"># Move the dot on the canvas, finding it by its tag.</span>
canvas.coords(
<span style="color:#d20;background-color:#fff0f0">f</span><span style="color:#d20;background-color:#fff0f0">"dot_</span><span style="color:#33b;background-color:#fff0f0">{</span>c<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">_</span><span style="color:#33b;background-color:#fff0f0">{</span>n<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">"</span>,
*coords,
)
<span style="color:#888"># Call the required tkinter update function.</span>
root.update()
<span style="color:#888"># Attempt to stabilize the timing of this loop by targeting 60Hz.</span>
<span style="color:#080;font-weight:bold">while</span> t0 < t:
t0 += <span style="color:#00d;font-weight:bold">1</span> / <span style="color:#00d;font-weight:bold">60</span>
time.sleep(t0 - t)
</code></pre></div><p>The fake anti-aliasing was a fun exercise, but for this use case you’ll probably get better-looking results out of scaling a PNG asset like we did the logo.</p>
<p><img src="/blog/2024/03/making-a-loading-spinner-with-tkinter/spinner.webp" alt="A screenshot of the loading spinner. In the center is a logo reading “VisionPort”, with the O replaced by a globe with a locator icon in it. Surrounding the logo are animated dots rotating in a circle."></p>
<h3 id="resources">Resources</h3>
<p>If you’re interested in learning more about tkinter, see also:</p>
<ul>
<li><a href="https://docs.python.org/3/library/tkinter.html">Python tkinter docs</a></li>
<li><a href="https://anzeljg.github.io/rin2/book2/2405/docs/tkinter/index.html">The tkinter reference</a></li>
</ul>
<p>Other Python GUI/graphics toolkits you might consider:</p>
<ul>
<li><a href="https://www.pygame.org/news">Pygame</a></li>
<li><a href="https://pyglet.org/">Pyglet</a></li>
<li><a href="https://doc.qt.io/qtforpython-6/">PySide6</a> (official Qt binding)</li>
<li><a href="https://riverbankcomputing.com/software/pyqt/intro">PyQt</a> (unofficial Qt binding)</li>
<li><a href="https://wxpython.org/index.html">wxPython</a></li>
</ul>
Key Takeaways from Practical Object-Oriented Design by Sandi Metzhttps://www.endpointdev.com/blog/2024/02/key-takeaways-from-poodr/2024-02-27T00:00:00+00:00Kevin Campusano
<p><img src="/blog/2024/02/key-takeaways-from-poodr/well-trimmed-trees.webp" alt="View upward at a bright sky, with lines drawn down at an angle toward the left center by a tree trimmed into a rectangular prism and one trimmed into a vague cone. The lens is imperfect, leaving the corner of the square tree in focus while the rest of the green foliage and blue and bright white sky is somewhat distorted by the lens."></p>
<!-- Photo by Seth Jensen, 2022 -->
<p><a href="https://www.poodr.com/">Practical Object-Oriented Design: An Agile Primer Using Ruby</a> by <a href="https://sandimetz.com/">Sandi Metz</a> is one of those books that everybody who writes or aspires to write object-oriented code should read. Indeed, reading through this book will be valuable for seasoned practitioners and inexperienced novices alike. Whether it be to discover new concepts, remember why they are important, or articulate the reasons behind that warm fuzzy feeling you get in your stomach when you read well designed code, POODR can help.</p>
<p>I personally really like this book, and here at End Point it does have quite a following as well; it was the subject of a <a href="/blog/2016/08/ruby-fight-club/">study group that we ran back in 2016</a>. I like to dust it off every now and then to give it a re-read. To help with that, I’ve come up with a list of key takeaways from the book, with some of my own interpretations sprinkled in. For me, it serves as a sort of summary to help commit the book’s contents to memory. I thought I’d share it with you today.</p>
<blockquote>
<p>While this book uses the Ruby language for its examples and discussions, the concepts it describes are equally applicable to any classical object-oriented language. Dynamically typed language users do get a little more mileage than users of statically typed ones. But still, something like 90% of the content is relevant to both. For this summary, I’ve taken care not to tie the discussion to Ruby too much and be as language agnostic as possible. I’ve chosen to write the code examples in C#.</p>
</blockquote>
<blockquote>
<p>Images taken from the book. Code taken from the book’s examples in Ruby and translated to C#.</p>
</blockquote>
<h3 id="chapter-1-object-oriented-design">Chapter 1: Object-Oriented Design</h3>
<h4 id="about-design">About design</h4>
<ol>
<li>Following the principles and patterns of object-oriented design produces code that is both cost-effective and a joy to work with. “Cost effective” means that it doesn’t take more effort than necessary to evolve. To change.</li>
<li>The reason design is important is because software changes. A good design is one that makes software easy to change.</li>
<li>Object-oriented design sees the world as a collection of parts that interact with each other. The parts we call “objects”, and the interactions we call “messages”.</li>
<li>In order to send messages to each other, objects need to have some knowledge about one another. This knowledge produces “dependencies” between them.</li>
<li>Managing these dependencies is a big part of object-oriented design. OOD aims to create dependencies in such a way that they don’t make the process of changing objects too difficult.</li>
<li>Design is important no matter the size of the application. Because both small and large applications change. And change is what design allows. A badly designed small application will eventually become a badly designed large one.</li>
<li>Design is the way in which code is arranged. Good design is an arrangement of code that makes change easy.</li>
<li>Our job as developers is to take our application’s requirements and our knowledge of design principles and use them to produce code that is cost effective today and tomorrow: code that is easy to change.</li>
<li>The idea is not to predict future changes. Instead, we have to accept that change will come and prepare for it. We prepare for it by writing code that leaves our options open for the future.</li>
<li>“The purpose of design is to allow you to design later, and its primary goal is to reduce the cost of change”.</li>
</ol>
<h4 id="failures-of-design">Failures of design</h4>
<ol start="11">
<li><strong>Design fails when you don’t do it</strong>. Code bases that lack in design will eventually evolve into unmaintainable messes, where change becomes increasingly hard even for the most minor new requirements. “Throwing everything away and beginning from scratch” becomes a viable alternative.</li>
<li><strong>Design fails when you overdesign</strong>. Armed with basic design skills, it’s easy to fall into the trap of developing wrong abstractions, applying principles incorrectly, seeing the wrong pattern in the wrong context. Applications that suffer from overdesign become gigantic castles of code full of indirection that become hard to change.</li>
<li><strong>Design fails when you separate it from programming</strong>. Design is best executed as an iterative and incremental process with a quick feedback loop. The design needs to be adjusted frequently, as understanding of the domain changes. Design that is dictated to programmers from on high as part of a “<a href="https://en.wikipedia.org/wiki/Big_design_up_front">Big Up Front Design</a>” is doomed to failure. Such arrangements are not agile enough to be resilient to requirement changes.</li>
</ol>
<h4 id="agile-and-design">Agile and design</h4>
<ol start="14">
<li>Agile development says: “Avoid Big Up Front Design because you can’t know what the application will end up being because requirements always change and they do so often”.</li>
<li>Agile also says: “Good design is essential. Because change comes often, your code needs to be ready to accommodate those changes in an efficient and cost effective way”.</li>
<li>Design is not a “one time at the beginning of the project” type of task. It is an ongoing, iterative and incremental task that happens as requirements change, and as our domain knowledge and skills increase.</li>
</ol>
<h4 id="metrics">Metrics</h4>
<ol start="17">
<li>Bad object-oriented design metrics are an indicator of a bad design. But good metrics are not an indicator of good design. A code base with bad metrics will most likely be hard to change.</li>
<li>The ideal software metric would be “<strong>cost per feature per time interval</strong>”. But this is nigh on impossible to calculate because “cost”, “feature”, and “time interval” are in most cases nebulous concepts.</li>
<li>Still, “cost per feature per time interval” tells us that our goal should always be to write code with a low cost per feature. Making sure that it is also low cost to evolve in the future.</li>
</ol>
<h4 id="technical-debt">Technical debt</h4>
<ol start="20">
<li><strong>Technical debt</strong> is borrowing time from the future. That is, putting out a bad design today in order to release a feature quickly. Tomorrow, when change comes, the bad design will prevent cost effective change. So time will have to be spent refactoring, turning the bad design into a good one. If refactoring doesn’t happen, the debt increases, making future changes more and more expensive.</li>
<li>How much design you do depends on your skill and your time frame. You can’t do so much design that it prevents you from delivering on time. Design is an investment, and for an investment to be good, it needs to return some profit. Good design’s returns are quick and plentiful.</li>
<li>“The trick to getting the most bang for your design buck is to acquire an understanding of the theories of design and to apply them appropriately, at the right time, and in the right amounts”.</li>
</ol>
<h3 id="chapter-2-designing-classes-with-a-single-responsibility">Chapter 2: Designing Classes with a Single Responsibility</h3>
<h4 id="about-classes">About classes</h4>
<ol>
<li><strong>Classes</strong> are the most basic organizational structure of object-oriented systems written with class-based object-oriented languages. As such, that’s the first thing that we focus on when designing such systems.</li>
<li>Your first instinct should be to keep things simple. Use classes to model only the features that your application needs today and make sure that they are easy to change tomorrow.</li>
<li>“Design is more the art of preserving changeability than it is the act of achieving perfection”.</li>
</ol>
<h4 id="write-code-that-is-true">Write code that is TRUE</h4>
<ol start="4">
<li>In order to be easy to change, code should be <strong>TRUE</strong>.</li>
<li><strong>Code should be “Transparent”</strong>. It should be easy to tell what will be the consequences of changing the code.</li>
<li><strong>Code should be “Reasonable”</strong>. The cost of a change should be proportional to the size of the requirement that provoked the change.</li>
<li><strong>Code should be “Usable”</strong>. The code should be easy to reuse in situations other than the one it’s currently being used on.</li>
<li><strong>Code should be “Exemplary”</strong>. The code should exhibit qualities that guide and/or encourage those who change it to keep and replicate those qualities.</li>
<li>The <a href="https://en.wikipedia.org/wiki/Single_responsibility_principle">Single Responsibility Principle</a> is a prerequisite to creating code that is TRUE.</li>
</ol>
<h4 id="single-responsibility">Single Responsibility</h4>
<ol start="10">
<li>A class should have one single responsibility. That means that it needs to do one thing and do it well. It needs to be as small as possible and fulfill a single purpose. It needs to have only one reason to change.</li>
<li>A good way to identify what classes your system needs is to read through your user stories or problem descriptions and identify <strong>nouns</strong>. Those nouns are good candidates for classes.</li>
<li>If there are <strong>verbs</strong> and <strong>properties</strong> associated with those nouns, then it is most likely a class. Because classes represent bundles of both data (attributes) and behavior (methods).</li>
<li>Classes that have many responsibilities are hard to reuse. And because they are not “Usable”, they are not easy to change.</li>
<li>A class that has many responsibilities usually has unrelated code tangled together within it. This makes the class hard to reason about, and hard to reuse.</li>
<li>In order to reuse the behavior defined within a class with multiple tangled responsibilities, you either need to duplicate code or use the entire class, even the parts that you don’t need.</li>
<li>If you use a class that has many reasons to change, when something unrelated to your use case changes, you will still have to change your code. This is unexpected and surprising. This is not “Transparent”.</li>
<li>Depending on classes that have many responsibilities increases the chances of your application breaking.</li>
<li>A good way to determine whether a class has a single responsibility is to try an enunciate what it does in a single sentence. That sentence should be simple, because classes should be simple. Look out for the words “and” and “or” in this sentence, as they point to it having more than one responsibility.</li>
<li>Another good way to determine whether a class has a single responsibility is thinking about every one of its methods as a question that the class would be able to respond to if it were a sentient being. For example “Array, what is your length?”. If the question doesn’t make sense, then maybe the method does not belong in the class.</li>
<li><strong>Cohesion</strong> is the measure of how related to their core purpose the elements of a component are. Classes that have a single responsibility are highly cohesive because all of their parts work towards a single goal.</li>
</ol>
<h4 id="when-to-make-design-decisions">When to make design decisions</h4>
<ol start="21">
<li>Only design as much as you need to for right now. Don’t make design decisions prematurely. Sometimes waiting for more information before committing to a design is the way to go. More information in the future will often reveal a better design.</li>
<li>Consider postponing design decisions when the information available to you today does not point clearly to a good design. Also, when the cost of not doing it now does not considerably make tomorrow’s change more expensive.</li>
<li>On the other hand, consider that a class that’s left in a “bad state” may be reused by others, making their code depend on badly designed code. The application will suffer as a result.</li>
<li>The structure of a class is supposed to reveal your design’s intention to other developers. When it doesn’t, when it is not “Exemplary”, then future maintenance costs increase.</li>
<li>The tension between “improve it now” and “improve it later” is always present. Our job as designers is to strike a good balance “between the needs of the present and the possibilities of the future” so that the costs of change are kept low.</li>
</ol>
<h4 id="techniques-for-writing-code-thats-easy-to-change">Techniques for writing code that’s easy to change</h4>
<ol start="26">
<li><strong>Depend on behavior, not data: hide instance variables</strong>. Instead of accessing instance variables directly, use accessor methods. Even from within the class that owns them. “Always send messages to access data”.</li>
<li>That way you consolidate the knowledge of what the data represents in a single place. That’s the Single Responsibility Principle in action. Changing it becomes easy as a result because instead of changing references to a variable in potentially multiple places; you only need to change the definition of a method, in a single location.</li>
<li><strong>Depend on behavior, not data: hide data structures</strong>. When code depends on or receives as input a complex data structure like a big array or hash, encapsulate the data structure in a class with a clear interface, instead of accessing and manipulating the structure directly.</li>
<li>Directly manipulating complex data structures is a style of coding that tends to propagate. If the incoming data structure changes, even slightly, then a lot of your code also needs to change because it depends on its very particular structure.</li>
<li>Handling complex raw data structures through classes and messages with clear, intention-revealing names demystifies the structure and gives it meaning within the context of the application domain.</li>
</ol>
<blockquote>
<p>For example, instead of this…</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ObscuringReferences</span>
{
<span style="color:#888;font-weight:bold">int</span>[][] data;
<span style="color:#080;font-weight:bold">public</span> ObscuringReferences(<span style="color:#888;font-weight:bold">int</span>[][] data)
{
<span style="color:#080;font-weight:bold">this</span>.data = data;
}
<span style="color:#080;font-weight:bold">public</span> IEnumerable<<span style="color:#888;font-weight:bold">int</span>> GetDiameters()
{
<span style="color:#080;font-weight:bold">return</span> data.Select(cell => cell[<span style="color:#00d;font-weight:bold">0</span>] + (cell[<span style="color:#00d;font-weight:bold">1</span>] * <span style="color:#00d;font-weight:bold">2</span>));
}
}
</code></pre></div><p>…do this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">RevealingReferences</span>
{
IEnumerable<Wheel> Wheels { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> RevealingReferences(<span style="color:#888;font-weight:bold">int</span>[][] data)
{
Wheels = Wheelify(data);
}
<span style="color:#888">// This method is now more readable.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> IEnumerable<<span style="color:#888;font-weight:bold">int</span>> GetDiameters()
{
<span style="color:#080;font-weight:bold">return</span> Wheels.Select(wheel => wheel.Rim + (wheel.Tire * <span style="color:#00d;font-weight:bold">2</span>));
}
<span style="color:#888">// Now everyone can send rim/tire to wheel.
</span><span style="color:#888"></span> record Wheel(<span style="color:#888;font-weight:bold">int</span> Rim, <span style="color:#888;font-weight:bold">int</span> Tire);
<span style="color:#080;font-weight:bold">private</span> IEnumerable<Wheel> Wheelify(<span style="color:#888;font-weight:bold">int</span>[][] data)
{
<span style="color:#080;font-weight:bold">return</span> data.Select(cell => <span style="color:#080;font-weight:bold">new</span> Wheel(cell[<span style="color:#00d;font-weight:bold">0</span>], cell[<span style="color:#00d;font-weight:bold">1</span>]));
}
}
</code></pre></div></blockquote>
<ol start="31">
<li><strong>Enforce single responsibility everywhere: extract extra responsibilities from methods</strong>. Methods should also have a single responsibility. Just like classes, that makes them easy to reuse, change, and understand.</li>
<li>To determine if a method has a single responsibility, the same techniques that work for classes apply. Try to enunciate their purpose in a single sentence, and ask them what they do.</li>
<li>Methods that iterate on items and act upon them too are a common case of multiple responsibilities. Separating iteration and action into two methods is a common refactoring to correct it.</li>
<li>Complex calculations embedded within methods are also good candidates to be separated into other methods. It gives them a name and makes them reusable.</li>
</ol>
<blockquote>
<p>For example, instead of this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">IEnumerable<<span style="color:#888;font-weight:bold">int</span>> GetDiameters()
{
<span style="color:#080;font-weight:bold">return</span> Wheels.Select(wheel => wheel.Rim + (wheel.Tire * <span style="color:#00d;font-weight:bold">2</span>));
}
</code></pre></div><p>…do this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// First - iterate over the collection.
</span><span style="color:#888"></span>IEnumerable<<span style="color:#888;font-weight:bold">int</span>> <span style="color:#00d;font-weight:bold">_</span>GetDiameters()
{
<span style="color:#080;font-weight:bold">return</span> Wheels.Select(wheel => GetDiameter(wheel));
}
<span style="color:#888">// Second - calculate diameter of ONE wheel.
</span><span style="color:#888"></span><span style="color:#888;font-weight:bold">int</span> GetDiameter(Wheel wheel)
{
<span style="color:#080;font-weight:bold">return</span> wheel.Rim + (wheel.Tire * <span style="color:#00d;font-weight:bold">2</span>);
}
</code></pre></div></blockquote>
<ol start="35">
<li>These refactorings are useful and necessary even when you don’t know what the final design will look like. In fact, these refactorings (and good practices like these) will often reveal the final design.</li>
<li>Small methods with a single responsibility have many benefits. They make a class’s details, features, and components more obvious, they are self-documenting, encourage reuse, establish a style of programming that is self-perpetuating, become easy to move to another class when and if that time comes.</li>
<li><strong>Enforce single responsibility everywhere: isolate extra responsibilities in classes</strong>. Move the responsibilities that are extraneous to a class into another class. If you can afford it, create a separate class to hold them. If not, at least encapsulate them in an embedded class so that they are contained and don’t leak.</li>
<li>Embedded classes say: This class only has meaning and value when thought about within the context of this other class.</li>
<li>Embedded classes can be easily promoted to independent classes later if the need arises.</li>
<li>Once all methods inside a class have a single responsibility and are each doing the smallest useful thing, it becomes easier to identify the features that may belong to another class.</li>
<li>The purpose of design is not to produce “perfect” code but instead to produce code that’s “good enough”.</li>
<li>The Single Responsibility Principle “allows change without consequence and reuse without duplication”.</li>
</ol>
<h3 id="chapter-3-managing-dependencies">Chapter 3: Managing Dependencies</h3>
<h4 id="about-dependencies">About dependencies</h4>
<ol>
<li>Objects should have a single responsibility and do the smallest useful thing. This means that, in order to fulfill complex application requirements, objects need to collaborate with one another.</li>
<li>Objects collaborate by knowing things about and sending messages to each other. That creates <strong>dependencies</strong>. A good design manages these dependencies, making sure they don’t couple objects so much that change becomes painful.</li>
<li><strong>Coupling</strong> measures how much interdependency exists between software components. High coupling makes things hard to change.</li>
<li>An object has a dependency when it knows <strong>the name of another class</strong>.</li>
<li>An object has a dependency when it knows <strong>the name of a method from another class</strong>.</li>
<li>An object has a dependency when it knows <strong>the arguments of that method</strong>.</li>
<li>An object has a dependency when it knows <strong>the order of those arguments</strong>.</li>
<li>Dependencies create a situation where if an object changes, then other objects that depend on it may be forced to change as well.</li>
<li>Dependencies couple objects to each other. Highly coupled objects behave as if they were a unit. It becomes hard to reuse one without the other. If not managed properly, dependencies will disseminate and entangle the entire code base.</li>
<li>Some dependencies are unavoidable. So the purpose of design is to reduce and contain dependencies. Ensure objects only have the minimum number of dependencies that they need to do their jobs. Keep the essential ones, remove the rest.</li>
</ol>
<h4 id="techniques-for-writing-loosely-coupled-code">Techniques for writing loosely coupled code</h4>
<ol start="11">
<li><a href="https://en.wikipedia.org/wiki/Dependency_injection"><strong>Dependency injection</strong></a>. It can help eliminate dependencies on concrete classes. Instead of instantiating new objects of another class inside your object, let the object receive it as a constructor or method parameter. The object’s users can do the instantiation.</li>
<li>An object that knows less can often times do more. The knowledge of how to instantiate a particular object limits its potential. Object creation and utilization don’t need to be contained within the same unit of code.</li>
<li>When an object hard-codes the reference to a class, it advertises that it can’t work with any other type. Replace the reference to the concrete class with an abstraction (e.g. an <a href="https://en.wikipedia.org/wiki/Interface_(object-oriented_programming)">interface</a> or <a href="https://en.wikipedia.org/wiki/Duck_typing">duck type</a>). That gives it the flexibility to be able to collaborate with other objects that respond to the same messages.</li>
</ol>
<blockquote>
<p>With dependency injection you can turn this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Gear</span>
{
<span style="color:#888;font-weight:bold">float</span> Ratio { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888;font-weight:bold">int</span> Rim { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888;font-weight:bold">int</span> Tire { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Gear(<span style="color:#888;font-weight:bold">float</span> ratio, <span style="color:#888;font-weight:bold">int</span> rim, <span style="color:#888;font-weight:bold">int</span> tire)
{
Ratio = ratio;
Rim = rim;
Tire = tire;
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">float</span> GetGearInches()
{
<span style="color:#080;font-weight:bold">return</span> Ratio * <span style="color:#080;font-weight:bold">new</span> Wheel(Rim, Tire).GetDiameter();
}
}
</code></pre></div><p>Into this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Gear</span>
{
<span style="color:#888;font-weight:bold">float</span> Ratio { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
Wheel Wheel { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888">// Instead of rim and tire, Gear's constructor now receives a wheel.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> Gear(<span style="color:#888;font-weight:bold">float</span> ratio, Wheel wheel)
{
Ratio = ratio;
Wheel = wheel;
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">float</span> GetGearInches()
{
<span style="color:#888">// Instead of creating a new wheel, it uses the one that's been injected.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> Ratio * Wheel.GetDiameter();
}
}
</code></pre></div></blockquote>
<ol start="14">
<li>Sometimes it is not possible to remove a dependency from an object. In those cases, your best bet is to isolate it.</li>
<li><strong>Isolate dependencies: isolate instance creation</strong>. Sometimes it is not possible to apply dependency injection. In those cases, the next best thing is to encapsulate instantiation of external classes into their own methods.</li>
<li>An instantiation encapsulated in a method allows the code to clearly and explicitly communicate that there’s an extraneous dependency here. One that we’d like to remove but for now cannot. We also protect the class from that dependency propagating through it.</li>
<li><strong>Isolate dependencies: isolate vulnerable external messages</strong>. The same technique can be applied to invocations of methods on external classes. Encapsulate such calls into their own methods in order to contain the dependency.</li>
</ol>
<blockquote>
<p>Here’s what isolating some dependencies could look like. From this…</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888;font-weight:bold">float</span> GetGearInches()
{
<span style="color:#080;font-weight:bold">return</span> Ratio * <span style="color:#080;font-weight:bold">new</span> Wheel(Rim, Tire).GetDiameter();
}
</code></pre></div><p>…to this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888;font-weight:bold">float</span> GetGearInches()
{
<span style="color:#080;font-weight:bold">return</span> Ratio * GetWheelDiameter();
}
<span style="color:#888">// The dependency on the Wheel class is isolated in this method.
</span><span style="color:#888"></span><span style="color:#888;font-weight:bold">int</span> GetWheelDiameter()
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> Wheel(Rim, Tire).GetDiameter();
}
</code></pre></div></blockquote>
<ol start="18">
<li>
<p><strong>Reduce argument dependencies: use keyword arguments</strong>. If your language supports them, keyword arguments are a great way of reducing the dependency incurred via method invocations. They allow the parameters to be sent in any order, which means that changes to the method’s definition and signature are less likely to cause the callers to have to also change. Also, they have a self-documenting property on both caller and callee. (See <a href="https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/named-and-optional-arguments">named arguments in C#</a> and <a href="https://thoughtbot.com/blog/ruby-2-keyword-arguments">keyword arguments in Ruby</a>).</p>
</li>
<li>
<p><strong>Reduce argument dependencies: explicitly define defaults</strong>. If your language has the feature, parameter defaults are a great way of reducing dependencies to method parameters, as establishing a default for a parameter makes it optional.</p>
</li>
<li>
<p><strong>Reduce argument dependencies: isolate multiparameter initialization</strong>. If you don’t own the code for a complex object initialization or method invocation, you can always encapsulate it in an object or method of your own. It can be done with a thin wrapper that exposes an interface that follows the techniques we’ve discussed and that uses the language of your application.</p>
</li>
<li>
<p>Classes should depend on things that change less often than they do.</p>
</li>
<li>
<p>There are three truths about code related to change and dependencies:</p>
<ol>
<li>“Some classes are more likely to change than others”.</li>
<li>“Concrete classes are more likely to change than abstract classes”.</li>
<li>“Changing a class that has many dependents will have many consequences”.</li>
</ol>
</li>
<li>
<p>It’s OK to depend on classes with a low likelihood of change. For example, classes in the built-in library of your language generally will change less than your own custom classes. Framework classes commonly change less often too, but that may not always be the case if you framework is new or rapidly evolving.</p>
</li>
<li>
<p>Abstractions are generally much more stable than concretions. Because of this, <strong>it is safer to depend on abstract classes</strong> (e.g. interfaces and duck types) than on concrete ones.</p>
</li>
<li>
<p>Avoid classes that have lots of dependencies. Any changes done to them produce a ripple effect of changes throughout the code base. Also, because of that, people will go through great lengths to try to avoid changing them.</p>
</li>
<li>
<p>When thinking about dependents and likelihood of change, <strong>the following are harmless</strong>:</p>
<ol>
<li>Classes with few dependents and low likelihood of change (e.g. specialized infrastructure interaction classes).</li>
<li>Classes with few dependents and high likelihood of change (e.g. concrete classes that implement your app’s domain logic).</li>
<li>Classes with many dependents and low likelihood of change (e.g. interfaces and other abstractions).</li>
</ol>
</li>
<li>
<p>When thinking about dependents and likelihood of change, <strong>the following is harmful</strong>: Classes with many dependents and high likelihood of change (e.g. concrete classes that are used throughout the code base).</p>
</li>
</ol>
<p><img src="/blog/2024/02/key-takeaways-from-poodr/likelihood-of-change-versus-number-of-dependents.webp" alt="A graph divided into four cells. The y axis is labeled “dependents”, from Few to Many. The x axis is labeled “Likelihood of requirements change”, from Less to More. The cell with less likelihood of requirements change and many dependents reads “Abstract Zone: Changes are unlikely but, if they occur, will have broad effects.” The cell with less likelihood of requirements change and fewer dependents reads “Neutral Zone: Changes are unlikely and have few side effects.” The cell with more likelihood of requirements change and few dependents reads “Neutral Zone: Changes are likely but they have few side effects.” The cell with more likelihood of requirements change and many dependents reads “Danger Zone: These classes WILL change and the changes will cascade into dependents.""></p>
<h3 id="chapter-4-creating-flexible-interfaces">Chapter 4: Creating Flexible Interfaces</h3>
<h4 id="about-interfaces">About interfaces</h4>
<ol>
<li>The messages that objects pass between each other are a big concern for design. In addition to what objects know (responsibilities), and who they know (dependencies), design cares about how they talk to one another. Messages pass between objects through their interfaces.</li>
<li>In a well designed application, the messages that pass between objects follow a pattern that’s closer to the diagram on the right than that of the diagram on the left:</li>
</ol>
<p><img src="/blog/2024/02/key-takeaways-from-poodr/communication-patterns.webp" alt="On the left, many dots are connected by a complex network of single-direction arrows. Dots have several arrows pointing to and from them. On the right, the same dots are connected by a single arrow to each. Each dot either has one or more arrows pointing away from it, or a single arrow pointing to it."></p>
<ol start="3">
<li>
<p>Objects that expose too much of themselves and know too much about others are hard to reuse and make systems hard to change.</p>
</li>
<li>
<p>Objects that minimize what they expose of themselves and know little about others are easily reusable and changeable.</p>
</li>
<li>
<p>An object’s <strong>public interface</strong> is the set of messages that it responds to. That is, the set of methods that other objects are welcome to invoke. Good design calls for objects with clear and concise public interfaces.</p>
</li>
<li>
<p>Classes should have one responsibility, but they will likely have many methods. Some methods are more general and expose the main features of a class; they are the services that the class offers its callers. These should make up the class’s public interface. Other methods are more specific, serve to support those features, and contain implementation details internal to the class and uninteresting to callers. These make up the class’s private interface.</p>
</li>
<li>
<p><strong>The methods in a class’s public interface</strong>:</p>
<ol>
<li>“Reveal its primary responsibility”</li>
<li>“Are expected to be invoked by others”</li>
<li>“Are not likely to change”</li>
<li>“Are safe for others to depend on”</li>
<li>“Are directly covered by tests”</li>
</ol>
</li>
<li>
<p><strong>The methods in a class’s private interface</strong>:</p>
<ol>
<li>“Handle implementation details”</li>
<li>“Are not expected to be invoked by other objects”</li>
<li>“Are likely to change”</li>
<li>“Are unsafe for others to depend on”</li>
<li>“May not even be referenced in the tests”</li>
</ol>
</li>
<li>
<p>Public methods list the specific features of a class that allow it to fulfill its responsibility. They advertise to the world what the purpose of the class they belong to is.</p>
</li>
<li>
<p>Public methods are stable and are expected to not change often, so others are welcome to depend on them.</p>
</li>
<li>
<p>Private methods are not stable at all. They are hidden from others so nobody should depend on them.</p>
</li>
</ol>
<h4 id="discovering-less-obvious-objects-and-messages">Discovering less obvious objects and messages</h4>
<ol start="12">
<li>When analyzing a problem domain, the nouns in the narrative usually become your first classes. These are called “<strong>domain objects</strong>”.</li>
<li>Don’t let domain objects be your only classes, though. Shift your focus toward messages and away from classes to discover new, less obvious objects that will implement key functionality. Not doing so will risk overloading your domain objects with more responsibility than what they can handle.</li>
<li><a href="https://en.wikipedia.org/wiki/Sequence_diagram">UML sequence diagrams</a> are an excellent way to explore design alternatives. You can use them to draft objects and their interactions, i.e. the messages they send each other. They are quick, low cost, and allow easy communication of ideas between team members. Here’s an online tool that generates them based on plain text: <a href="https://sequencediagram.org/">sequencediagram.org</a>.</li>
<li>The transition from class-based design to message-based design is one that yields more flexible applications. It means asking “I need to send this message, who should respond to it?” instead of “I know I need this class, what should it do?”.</li>
</ol>
<h4 id="focusing-on-what-instead-of-how">Focusing on “what” instead of “how”</h4>
<ol start="16">
<li>Public methods should focus on “what” instead of “how”. That is, they should express what the caller wants instead of how the callee must behave.</li>
<li>If you’re in a situation where an object calls many methods on another in succession, try to refactor so that all the logic is executed as a result of a single message. This consolidation reduces the size of the second object’s public interface, which reduces what callers need to know about it, which reduces dependency, which makes change easier. The caller only knows “what” it needs, not “how” to make the callee deliver.</li>
</ol>
<blockquote>
<p>When focusing on “what” instead of “how”, code like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Trip</span>
{
IEnumerable<Bicycle> Bicycles { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
Mechanic Mechanic { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> Prepare()
{
<span style="color:#080;font-weight:bold">foreach</span> (<span style="color:#888;font-weight:bold">var</span> bike <span style="color:#080;font-weight:bold">in</span> Bicycles)
{
Mechanic.CleanBicyble(bike);
Mechanic.PumpTires(bike);
Mechanic.LubeChain(bike);
Mechanic.CheckBrakes(bike);
}
}
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Mechanic</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> CheckBrakes(Bicycle bike) { }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> CleanBicyble(Bicycle bike) { }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> LubeChain(Bicycle bike) { }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> PumpTires(Bicycle bike) { }
}
</code></pre></div><p>Can be transformed into code like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Trip</span>
{
IEnumerable<Bicycle> Bicycles { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
Mechanic Mechanic { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888">// This method was greatly simplified and its dependencies were reduced.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> Prepare()
{
<span style="color:#080;font-weight:bold">foreach</span> (<span style="color:#888;font-weight:bold">var</span> bike <span style="color:#080;font-weight:bold">in</span> Bicycles)
{
Mechanic.PrepareBicycle(bike);
}
}
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Mechanic</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> PrepareBicycle(Bicycle bike)
{
CleanBicyble(bike);
PumpTires(bike);
LubeChain(bike);
CheckBrakes(bike);
}
<span style="color:#888">// Notice how Mechanic's public interface was reduced considerably.
</span><span style="color:#888"></span> <span style="color:#888">// All of these methods are private now.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> CheckBrakes(Bicycle bike) { }
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> CleanBicyble(Bicycle bike) { }
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> LubeChain(Bicycle bike) { }
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> PumpTires(Bicycle bike) { }
}
</code></pre></div></blockquote>
<h4 id="liberate-objects-from-their-context">Liberate objects from their context</h4>
<ol start="18">
<li>An object’s context is made up of the things that an object knows about others. What other objects it calls and how. What it needs in order to be able to work.</li>
<li>A good design tries to allow objects to work with minimal context. “Objects that have a simple context are easy to use and easy to test; they expect few things from their surroundings. Objects that have a complicated context are hard to use and hard to test; they require complicated setup before they can do anything”.</li>
<li>In order to reduce context, we need to reduce the knowledge that callers have about the other components they call. A simple public interface helps with that. In practical terms that means fewer and less verbose methods, fewer parameters, fewer dependencies.</li>
<li><strong>Objects need to trust their collaborators to do their part</strong>. Focusing on what the caller wants instead of what the callee does is a way of keeping context in check. This allows objects to collaborate with others regardless of what type they are and what they do.</li>
<li>Dependency injection is a mechanism through which objects can collaborate with others when they don’t know their type. A dependency injected via parameter (declared as an interface or duck type) hides its identity from its user.</li>
<li>Naming the methods of this injected dependency from the perspective of the caller reveals a generic interface that offers the features that the caller wants in the vocabulary that it understands. The caller does not need to know what the injected dependency does, only what it needs it to do.</li>
<li>Highly coupled objects with verbose public interfaces say to their collaborators: “I know what I want, and I know how you do it”.</li>
<li>More decoupled objects with concise public interfaces say to their collaborators: “I know what I want, and I know what you do”.</li>
<li>Highly decoupled objects with concise public interface and minimal required context say to their collaborators: “I know what I want, and I trust you to do your part”.</li>
</ol>
<blockquote>
<p>When objects trust their collaborators, and focus on what they want instead of what others do, the previous Trip/Mechanic example can become like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Trip</span>
{
<span style="color:#080;font-weight:bold">public</span> IEnumerable<Bicycle> Bicycles { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
Mechanic Mechanic { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> Prepare()
{
<span style="color:#888">// Trip now fully trusts Mechanic, and doesn't even know what it does.
</span><span style="color:#888"></span> Mechanic.PrepareTrip(<span style="color:#080;font-weight:bold">this</span>);
}
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Mechanic</span>
{
<span style="color:#888">// The knowledge of "how a mechanic prepares a trip" now completely lives
</span><span style="color:#888"></span> <span style="color:#888">// within Mechanic.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> PrepareTrip(Trip trip)
{
<span style="color:#080;font-weight:bold">foreach</span> (<span style="color:#888;font-weight:bold">var</span> bike <span style="color:#080;font-weight:bold">in</span> trip.Bicycles)
{
PrepareBicycle(bike);
}
}
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> PrepareBicycle(Bicycle bike)
{
CleanBicyble(bike);
PumpTires(bike);
LubeChain(bike);
CheckBrakes(bike);
}
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> CheckBrakes(Bicycle bike) { }
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> CleanBicyble(Bicycle bike) { }
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> LubeChain(Bicycle bike) { }
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> PumpTires(Bicycle bike) { }
}
</code></pre></div></blockquote>
<h4 id="rules-of-thumb-for-writing-code-with-good-interfaces">Rules of thumb for writing code with good interfaces</h4>
<ol start="27">
<li>
<p><strong>Create explicit interfaces</strong>. Be intentional and obvious when defining public and private methods. Use your language access modifiers (i.e. public, private, protected, etc). Your public methods should:</p>
<ol>
<li>Be explicitly identified as such</li>
<li>Be more about what than how</li>
<li>Have names that are unlikely to change</li>
<li>Use keyword arguments</li>
</ol>
</li>
<li>
<p><strong>Respect the public interface of others</strong>. Invoke only public methods on the classes that you use. Avoid circumventing their public interfaces and directly calling private members. Depending on the private interface of framework and library classes is like a ticking time bomb. The reason they are private is because they are expected to change or disappear entirely.</p>
</li>
<li>
<p><strong>Minimize context</strong>. Focus on “what” instead of “how” when designing public interfaces. Favor public methods that allow callers to access your classes' functionality without having to know how they do it. Use interface and duck types to name methods from the perspective of and with the vocabulary of the callers.</p>
</li>
<li>
<p>If you have to interact with a class that does not have a clean interface, and you don’t own it or otherwise can’t refactor it so that it does, isolate it. Use the same techniques for dependency isolation mentioned in Chapter 3. Wrap the invocation in a new class or method to contain it and give that wrapper a clean public interface.</p>
</li>
<li>
<p><strong>Follow the law of Demeter</strong>. The law of Demeter states that you shouldn’t chain multiple method calls that navigate across many different types of objects. In other words “talk only to your close neighbors” or “use only one dot”.</p>
</li>
</ol>
<blockquote>
<p>Violations ot the law of Demeter look like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">customer.GetBicycle().Wheel.Rotate();
</code></pre></div></blockquote>
<ol start="32">
<li>Violations of the law of Demeter make for code that’s not TRUE. Changes in the object at the end of the chain ripple through the entire chain. This is unexpected and laborious, making the code neither Transparent nor Reasonable. The class that uses the chain depends on all the objects in the chain, making it non-Usable. These chains are easy to replicate and harm changeability, making the code not Exemplary.</li>
<li>Always evaluate the cost of violating the law of Demeter versus the cost of abiding by it. Method chains that ultimately read an attribute are generally less harmful. Also, method chains on really stable classes like those of your language library or framework have low impact.</li>
<li>Delegation can appear to be a solution to law of Demeter violations. Unfortunately, all it does is remove the evidence that it’s there.</li>
<li>In reality, a violation to the law of Demeter indicates that there’s an object missing or that the public interface of an object is lacking. Find the object, and its interface, by thinking more about messages and less about classes. Define the interface by thinking about what the object needs of its collaborators, and not about how it can get it by itself.</li>
</ol>
<h3 id="chapter-5-reducing-costs-with-duck-typing">Chapter 5: Reducing Costs with Duck Typing</h3>
<h4 id="about-duck-types">About duck types</h4>
<ol>
<li><strong>Duck types</strong> are public interfaces that manifest themselves across classes. When multiple classes accept the same messages (i.e. define methods with the same signature), they are said to be of the same duck type.</li>
<li>“Duck type” is a dynamic language concept. Static languages offer equivalent features via “<a href="https://en.wikipedia.org/wiki/Interface_(object-oriented_programming)">interfaces</a>”.</li>
<li>The difference is that, in statically typed languages, interfaces need to be explicitly defined in code and concrete classes need to explicitly implement them. On the other hand, duck types in dynamically typed languages need only be acknowledged in an abstract sense. They don’t have to be explicitly defined in code.</li>
<li>Duck types and interfaces establish a contract. A set of messages that a given object is expected to be able to receive. A protocol that users of these objects can be confident that they will adhere to.</li>
<li>Just as a class can implement multiple interfaces, an object can be of many duck types. That’s because interfaces and duck types materialize depending on the situation in which objects that implement them are used. The same object can be used in different places for different things. Each user will see the object from their own perspective, and each can expect it conform to a different interface. See the <a href="https://en.wikipedia.org/wiki/Interface_segregation_principle">interface segregation principle</a>.</li>
<li>Moreover, code that depends on an interface or duck type is flexible enough to collaborate with any concrete class that implements that interface or duck type. See the <a href="https://en.wikipedia.org/wiki/Liskov_substitution_principle">Liskov substitution principle</a>.</li>
<li>As we’ve established, it’s better to depend on abstractions than to depend on concretions. So, users of an object should not care what its type is, only what it does. Behavior over identity. In other words, they should focus on messages and public interfaces, instead of specific classes. Duck types and interfaces bring to the forefront what an object does and abstracts away what it is.</li>
<li>Applying duck types to a situation makes the code more abstract and less concrete. Concrete code is easier to understand but harder to extend. Abstract code is harder to understand at the beginning but easier to change. In the long run, abstract code tends to reduce maintenance costs.</li>
<li>When an object invokes methods on multiple objects in order to achieve a single purpose, that’s a situation where a duck type may be helpful. Think from the perspective of the caller and what it needs to come up with a message that makes sense for it to send its many collaborators. That method should be part of the public interface of each of its collaborators. To the caller’s eyes, each of its collaborators belong in the same duck type.</li>
</ol>
<h4 id="writing-code-that-leverages-duck-types">Writing code that leverages duck types</h4>
<ol start="10">
<li>Whenever you see conditional logic that sends a different message depending on the concrete class of a given object, that’s the code telling you that it needs a duck type or interface.</li>
<li>Come up with a message that makes sense from the perspective of the caller and add it to each of the classes expected by your code in the conditional. These classes now share the same public interface, they are of the same duck type. Then remove the conditional and just call that method. Replace conditional logic with <a href="https://en.wikipedia.org/wiki/Polymorphism_(computer_science)">polymorphism</a>.</li>
<li>In general, try to avoid code that explicitly mentions concrete classes or checks for the existence of particular methods in order to determine further behavior. Instead, come up with a shared interface and send those messages to the collaborators regardless of their actual type.</li>
</ol>
<blockquote>
<p>Higly conditional and concrete logic like this…</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Trip</span>
{
<span style="color:#080;font-weight:bold">public</span> IEnumerable<Bicycle> Bicycles { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> IEnumerable<Customer> Customers { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Vehicle Vehicle { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> Prepare(IEnumerable preparers)
{
<span style="color:#080;font-weight:bold">foreach</span> (<span style="color:#888;font-weight:bold">var</span> preparer <span style="color:#080;font-weight:bold">in</span> preparers)
{
<span style="color:#080;font-weight:bold">if</span> (preparer <span style="color:#080;font-weight:bold">is</span> Mechanic)
{
((Mechanic)preparer).PrepareBicycles(Bicycles);
}
<span style="color:#080;font-weight:bold">else</span> <span style="color:#080;font-weight:bold">if</span> (preparer <span style="color:#080;font-weight:bold">is</span> TripCoordinator)
{
((TripCoordinator)preparer).BuyFood(Customers);
}
<span style="color:#080;font-weight:bold">else</span> <span style="color:#080;font-weight:bold">if</span> (preparer <span style="color:#080;font-weight:bold">is</span> Driver)
{
((Driver)preparer).GasUp(Vehicle);
((Driver)preparer).FillWaterTank(Vehicle);
}
}
}
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Mechanic</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> PrepareBicycles(IEnumerable<Bicycle> bicycles) { }
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">TripCoordinator</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> BuyFood(IEnumerable<Customer> customers) { }
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Driver</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> FillWaterTank(Vehicle vehicle) { }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> GasUp(Vehicle vehicle) { }
}
</code></pre></div><p>…can be rewritten to be more abstract if we leverage polymorphism. Like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Trip</span>
{
<span style="color:#080;font-weight:bold">public</span> IEnumerable<Bicycle> Bicycles { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> IEnumerable<Customer> Customers { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Vehicle Vehicle { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888">// This method can be much simpler now, and abstract.
</span><span style="color:#888"></span> <span style="color:#888">// Ready to collaborate with any "TripPreparer".
</span><span style="color:#888"></span> <span style="color:#888">// The conditional logic has been replaced with polymorphism.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> Prepare(IEnumerable<ITripPreparer> preparers)
{
<span style="color:#080;font-weight:bold">foreach</span> (<span style="color:#888;font-weight:bold">var</span> preparer <span style="color:#080;font-weight:bold">in</span> preparers)
{
preparer.PrepareTrip(<span style="color:#080;font-weight:bold">this</span>);
}
}
}
<span style="color:#888">// New interface that all preparers implement. It has a single method.
</span><span style="color:#888">// It greatly simplifies the communication between Trip and its many
</span><span style="color:#888">// possible preparers.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">interface</span> ITripPreparer
{
<span style="color:#080;font-weight:bold">void</span> PrepareTrip(Trip trip);
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Mechanic</span> : ITripPreparer
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> PrepareTrip(Trip trip)
{
PrepareBicycles(trip.Bicycles);
}
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> PrepareBicycles(IEnumerable<Bicycle> bicycles) { }
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">TripCoordinator</span> : ITripPreparer
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> PrepareTrip(Trip trip)
{
BuyFood(trip.Customers);
}
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> BuyFood(IEnumerable<Customer> customers) { }
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Driver</span> : ITripPreparer
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> PrepareTrip(Trip trip)
{
FillWaterTank(trip.Vehicle);
GasUp(trip.Vehicle);
}
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> FillWaterTank(Vehicle vehicle) { }
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> GasUp(Vehicle vehicle) { }
}
</code></pre></div></blockquote>
<ol start="13">
<li>The one exception to this rule is when dealing with exceptionally stable classes. Like those in your language libraries, where the introduction of duck types would mean modifying core language libraries. Explicit type checks against these classes are usually low cost.</li>
<li>Duck types are abstract and less obvious in the code. That’s why they need to be well documented and tested.</li>
</ol>
<h3 id="chapter-6-acquiring-behavior-through-inheritance">Chapter 6: Acquiring Behavior through Inheritance</h3>
<h4 id="about-inheritance">About inheritance</h4>
<ol>
<li><strong>Inheritance</strong> establishes a relationship between two classes where it is said that a subclass B “is a” superclass A. Subclass and superclass are also referred to as derived class and base class, respectively.</li>
<li>Inheritance can be more technically defined as an “automatic message forwarding mechanism”. That is, when a subclass receives a message that it cannot respond to directly (i.e. a method that it does not implement), the language runtime takes care of automatically sending the message to the superclass.</li>
<li>That is, inheritance establishes a hierarchy where superclasses share their code with their subclasses.</li>
<li>Subclasses are meant to be specialized versions of their superclass. Subclasses should be “the same, plus more” when compared to their superclass. They should offer the same public interface.</li>
<li>Inheritance is ideal to solve the problem where you need a series of slightly different classes that share a lot of behavior.</li>
<li>A telltale sign that inheritance needs to be applied is when code contains an “if” statement checking an attribute that contains the “category” of the object, and based on that determines what code to execute. Watch out for this anti-pattern and for variables with names like “type”, “category”, “style” that control branches in behavior.</li>
<li>This is an anti-pattern that reveals that the object knows too much and needs to be broken down into smaller pieces. It increases the costs of change.</li>
</ol>
<blockquote>
<p>Here’s what the anti-pattern looks like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Bicycle</span>
{
<span style="color:#888;font-weight:bold">string</span> Style { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888;font-weight:bold">string</span> TapeColor { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888;font-weight:bold">string</span> FrontShock { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Bicycle(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options)
{
Style = options[<span style="color:#d20;background-color:#fff0f0">"style"</span>];
TapeColor = options[<span style="color:#d20;background-color:#fff0f0">"tape_color"</span>];
FrontShock = options[<span style="color:#d20;background-color:#fff0f0">"front_shock"</span>];
}
<span style="color:#888">// Checking 'style' starts down a slippery slope.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> GetSpares()
{
<span style="color:#080;font-weight:bold">if</span> (Style == <span style="color:#d20;background-color:#fff0f0">"road"</span>)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span>() {
<span style="color:#369"> ["chain"]</span> = <span style="color:#d20;background-color:#fff0f0">"11-speed"</span>,
<span style="color:#369"> ["tire_size"]</span> = <span style="color:#d20;background-color:#fff0f0">"23"</span>,
<span style="color:#369"> ["tape_color"]</span> = TapeColor
};
}
<span style="color:#080;font-weight:bold">else</span> <span style="color:#080;font-weight:bold">if</span> (Style == <span style="color:#d20;background-color:#fff0f0">"mountain"</span>)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span>() {
<span style="color:#369"> ["chain"]</span> = <span style="color:#d20;background-color:#fff0f0">"11-speed"</span>,
<span style="color:#369"> ["tire_size"]</span> = <span style="color:#d20;background-color:#fff0f0">"2.1"</span>,
<span style="color:#369"> ["front_shock"]</span> = FrontShock
};
}
}
}
</code></pre></div></blockquote>
<h4 id="applying-inheritance">Applying inheritance</h4>
<ol start="8">
<li>Be on the lookout for existing classes that may lead you down the wrong path when it comes to inheritance. More often than not, classes that already exist in the code base are not good candidates for extension via inheritance. They are not good candidates to be superclasses.</li>
<li>The path forward is likely to be a new class to serve as the base, and then update your existing class to be a subclass. Then add other subclasses as peers of it.</li>
<li>For inheritance to work, the objects being modeled need to truly share a generalization-specialization relationship.</li>
<li>The superclass needs to define the common behavior that is shared among subclasses. The subclasses define the specializations.</li>
<li>In many cases, the superclass should be <strong>abstract</strong>, meaning that they are not supposed to be instantiated. They represent an incomplete object which only becomes whole when looked at in the context of each of its subclasses.</li>
<li>It almost never makes sense to create a superclass with only a single subclass. In fact, creating an inheritance hierarchy with two subclasses is often risky. You risk coming up with the wrong abstraction.</li>
<li>Three subclasses is the sweet spot to commit to inheritance. Put off the decision to implement inheritance until that point, if you can. That’s the point when there’s enough information available to confidently determine an abstraction that will be useful and cost effective.</li>
<li>If you put off implementing an inheritance hierarchy, then you won’t be able to share code between the highly related classes. That will likely lead to code repetition, which is also costly. Consider what costs more: having the repetition for the time being; or doing nothing and waiting for more information to avoid making the wrong decision.</li>
<li>“Every decision you make includes two costs: one to implement it and another to change it when you discover that you were wrong”.</li>
<li>When refactoring towards an inheritance hierarchy, consider that “promoting” code from the concrete subclass up into the abstract superclass is often easier and less error prone.</li>
<li>Errors in promoting are easy to identify and fix. All that can happen if you miss a promotion is that a subclass that was meant to inherit some behavior won’t have it.</li>
<li>Going in the opposite direction and missing a “demotion” will produce design errors that are harder to spot and have dire consequences if left alone.</li>
<li>That would mean that concrete behavior, which does not apply to all subclasses, stays incorrectly in the abstract base class. That’ll throw a wrench in the works and the whole inheritance hierarchy will be on shaky grounds. The abstract base class won’t be generic enough and subclasses will be tempted to circumvent it.</li>
<li>“The general rule for refactoring into a new inheritance hierarchy is to arrange code so that you can promote abstractions rather than demote concretions”.</li>
<li>The <a href="https://en.wikipedia.org/wiki/Template_method_pattern">template method pattern</a> provides a clean way of defining a common basic algorithm in the superclass and allowing subclasses to supply specializations for it. Superclasses can define an algorithm, and call certain methods in key points within it. These are extension points. These methods can then be implemented by subclasses, letting them control part of the overall logic.</li>
<li>When promoting concrete code to the more abstract superclass, consider using the template method when methods cannot be promoted wholesale, and have to be broken up instead. The parts that can be promoted become the template method; the parts that can’t be become the specialization methods that each concrete subclass implements.</li>
<li>To avoid future bugs, superclasses should provide default implementations for the specialization methods that it expects its subclasses to implement. These can be no-ops or, even better, raise errors. These errors let developers know that extending this inheritance hierarchy requires these methods to be implemented.</li>
<li><a href="https://www.geeksforgeeks.org/ruby-hook-methods/">Hook methods</a> are a slightly lighter iteration of the template method concept that helps alleviate superclass-subclass coupling. Same principle: let the superclass send a message that the subclass implements in order to provide a specialization.</li>
<li>A hook method need not be a part of a common abstract algorithm, hence the slight distinction from the full fledged template method design pattern.</li>
<li>Beware of subclasses explicitly invoking functionality on their superclasses. Languages often offer keywords such as “<a href="https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/base">base</a>” or “<a href="https://www.rubyguides.com/2018/09/ruby-super-keyword/">super</a>” that allow easily sending messages to superclasses. These are dangerous because they couple subclasses with their superclasses. They reveal that subclasses know the general algorithm.</li>
<li>The template method pattern and hook methods invert this dependency, allowing the superclass to call the subclass. Allowing the subclass to provide specializations without knowing too much about the superclass.</li>
</ol>
<blockquote>
<p>Here’s an example of a properly factored inheritance hierarchy:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// The main purpose of the base class is to establish a basic functionality
</span><span style="color:#888">// that can be easily extended by subclasses.
</span><span style="color:#888">// This class cannot be instantiated, as given by the "abstract" modifier.
</span><span style="color:#888">// This means that our application has no need of holding onto pure Bicycle
</span><span style="color:#888">// instances, only the concrete subclasses are to be used.
</span><span style="color:#888">// This abstract class is just a vehicle for sharing code between other classes.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">abstract</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Bicycle</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Size { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Chain { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> TireSize { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888">// Notice how this constructor implements the template method pattern
</span><span style="color:#888"></span> <span style="color:#888">// in order to allow subclasses to make changes to the overall algorithm.
</span><span style="color:#888"></span> <span style="color:#888">// Specifically, it allows them to provide default values for the Chain and
</span><span style="color:#888"></span> <span style="color:#888">// TireSize attributes as well as run any additional logic after
</span><span style="color:#888"></span> <span style="color:#888">// initialization.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> Bicycle(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options)
{
Size = options.GetValueOrDefault(<span style="color:#d20;background-color:#fff0f0">"size"</span>);
Chain = options.GetValueOrDefault(<span style="color:#d20;background-color:#fff0f0">"chain"</span>) ?? GetDefaultChain();
TireSize = options.GetValueOrDefault(<span style="color:#d20;background-color:#fff0f0">"tire_size"</span>) ?? GetDefaultTireSize();
AfterInitialize(options);
}
<span style="color:#888">// This method establishes the basic spares information and allows
</span><span style="color:#888"></span> <span style="color:#888">// subclasses to supply more by calling the GetLocalSpares hook method.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> GetSpares()
{
<span style="color:#888;font-weight:bold">var</span> defaultSpares = <span style="color:#080;font-weight:bold">new</span> Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> {
<span style="color:#369"> ["tire_size"]</span> = <span style="color:#d20;background-color:#fff0f0">"23"</span>,
<span style="color:#369"> ["chain"]</span> = <span style="color:#d20;background-color:#fff0f0">"11-speed"</span>
};
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span>(defaultSpares.Concat(GetLocalSpares()));
}
<span style="color:#888">// This method, which is marked as abstract, has to be implemented by
</span><span style="color:#888"></span> <span style="color:#888">// subclasses. It allows subclasses to contribute specializations.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">abstract</span> <span style="color:#888;font-weight:bold">string</span> GetDefaultTireSize();
<span style="color:#888">// These methods, marked as virtual, all have default implementations of
</span><span style="color:#888"></span> <span style="color:#888">// varying complexity but the important aspect is that they can be
</span><span style="color:#888"></span> <span style="color:#888">// overridden by subclasses. They also allow subclasses to contribute
</span><span style="color:#888"></span> <span style="color:#888">// specializations.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> AfterInitialize(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options) { }
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> GetLocalSpares() => <span style="color:#080;font-weight:bold">new</span>();
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#888;font-weight:bold">string</span> GetDefaultChain() => <span style="color:#d20;background-color:#fff0f0">"11-speed"</span>;
}
<span style="color:#888">// Notice how the following concrete subclasses are very simple.
</span><span style="color:#888">// Extending this kind of code is easy, because creating new subclasses is easy.
</span><span style="color:#888">// They leverage their abstract base class to implement most of their features.
</span><span style="color:#888">// Their job is to supply specializations on the core logic defined in the base
</span><span style="color:#888">// class. To do so, they override the hook methods defined by the base class.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">RoadBike</span> : Bicycle
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> TapeColor { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> RoadBike(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options) : <span style="color:#080;font-weight:bold">base</span>(options) { }
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> AfterInitialize(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options) =>
TapeColor = options.GetValueOrDefault(<span style="color:#d20;background-color:#fff0f0">"tape_color"</span>);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> GetLocalSpares() =>
<span style="color:#080;font-weight:bold">new</span>() { [<span style="color:#d20;background-color:#fff0f0">"tape_color"</span>] = TapeColor };
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#888;font-weight:bold">string</span> GetDefaultTireSize() => <span style="color:#d20;background-color:#fff0f0">"23"</span>;
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">MountainBike</span> : Bicycle
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> FrontShock { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> RearShock { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> MountainBike(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options) : <span style="color:#080;font-weight:bold">base</span>(options) { }
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> AfterInitialize(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options)
{
FrontShock = options.GetValueOrDefault(<span style="color:#d20;background-color:#fff0f0">"front_shock"</span>);
RearShock = options.GetValueOrDefault(<span style="color:#d20;background-color:#fff0f0">"rear_shock"</span>);
}
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> GetLocalSpares() =>
<span style="color:#080;font-weight:bold">new</span>() { [<span style="color:#d20;background-color:#fff0f0">"front_shock"</span>] = FrontShock };
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#888;font-weight:bold">string</span> GetDefaultTireSize() => <span style="color:#d20;background-color:#fff0f0">"2.1"</span>;
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">RecumbentBike</span> : Bicycle
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Flag { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> RecumbentBike(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options) : <span style="color:#080;font-weight:bold">base</span>(options) { }
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> AfterInitialize(Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> options) =>
Flag = options.GetValueOrDefault(<span style="color:#d20;background-color:#fff0f0">"flag"</span>);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> Dictionary<<span style="color:#888;font-weight:bold">string</span>, <span style="color:#888;font-weight:bold">string</span>> GetLocalSpares() =>
<span style="color:#080;font-weight:bold">new</span>() { [<span style="color:#d20;background-color:#fff0f0">"flag"</span>] = Flag };
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#888;font-weight:bold">string</span> GetDefaultChain() => <span style="color:#d20;background-color:#fff0f0">"10-speed"</span>;
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#888;font-weight:bold">string</span> GetDefaultTireSize() => <span style="color:#d20;background-color:#fff0f0">"28"</span>;
}
</code></pre></div></blockquote>
<h3 id="chapter-7-sharing-role-behavior-with-modules">Chapter 7: Sharing Role Behavior with Modules</h3>
<h4 id="about-roles">About roles</h4>
<ol>
<li>Using classical inheritance is always optional. Every situation that calls for inheritance could be solved using another technique that allows for sharing code between otherwise unrelated classes.</li>
<li>The concept of “<strong>roles</strong>” is an alternative to classical inheritance. This concept emerges from the need of multiple classes to be used in a particular context to do the same thing. So they themselves need to share behavior.</li>
<li>Think of a role as an augmented form of a duck type. A group of classes are said to play the same role when they belong in the same duck type or implement the same interface. In addition to exposing the same public interface, roles allow them to share internal behavior.</li>
<li>Roles are ideal for storing responsibilities that are orthogonal to classes. It allows classes that are otherwise unrelated to share behavior. They establish a “behaves like” type of relationship between objects, as opposed to the “is a” relationship that is established with inheritance.</li>
<li>When you include a module into an existing class, all the methods defined in the module become available to the class. This is the same thing that happens when a subclass inherits from a superclass.</li>
<li>Many languages offer native features to allow for this kind of relationship between objects. In Ruby, we use <a href="http://ruby-for-beginners.rubymonstas.org/advanced/modules.html">Modules</a>. Other languages use <a href="https://www.pythontutorial.net/python-oop/python-mixin/">Mixins</a>, <a href="https://www.php.net/manual/en/language.oop5.traits.php">Traits</a>, <a href="https://docs.oracle.com/javase/tutorial/java/IandI/defaultmethods.html">Default Methods</a>. Simply put, all these are ways of bundling together a group of methods that can be easily plugged into existing classes.</li>
</ol>
<h4 id="writing-inheritable-code">Writing inheritable code</h4>
<ol start="7">
<li>The same principles, techniques, and anti patterns that apply to the design of duck types and inheritance hierarchies also apply to roles.</li>
<li>Beware of objects that use a variable with a name like “type”, “category” or “kind” to determine what code to execute. The object is likely concealing (and acting like) two or more types. These would be subclasses or module includers.</li>
<li>When code is checking the type of an object to decide which message to send it, that’s a signal that there’s a missing abstraction. There’s a duck type or role in there that needs to be explicitly brought to light. A public interface needs to be defined for it. If there is a need to share behavior, put it in a module, mixin, etc.</li>
<li>“All of the code in the abstract superclass should apply to every class that inherits it”.</li>
<li>Beware of subclasses or module includers that override a superclass/module method just to raise a “not implemented” error. Chances are that if only part of the superclass applies to it, then it doesn’t belong in the hierarchy. Or maybe the whole hierarchy needs a redesign.</li>
<li>Subclasses agree to the contract specified by their superclasses. They must respond to every message in the superclass’s public interface, accept the same types of inputs and return the same types outputs.</li>
<li>Put in other words, “subtypes must be substitutable for their supertype”. That is, all subclasses should act like their superclass. This means that any code that expects a superclass should be able to work, without change, with all its subclasses as well. That’s the Liskov substitution principle.</li>
<li>Through the concept of <a href="https://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)">variance</a>, subclasses have a slightly higher degree of freedom. Subclasses can accept inputs of more abstract types and can return outputs of more concrete types than those specified in the superclass’s public interface while still being substitutable for their superclass.</li>
<li>For example, given a class “Object” that is a superclass of a class “String”; if a method in a superclass “MyClass” accepts a String parameter, a subclass of MyClass could override that same method and accept Object and still be compliant. Likewise, if a method in a superclass returns an Object result, the subclass could return a String for that same method, and still be compliant. Still be substitutable.</li>
<li>In other words, some code that sends a message with a String parameter can send the same message to a receiver that accepts Object. Likewise, some code that expects an Object as a result of sending a message can send the same message to a receiver that returns a String. Because a String “is an” Object, any code that works with an Object can work with a String, it’ll just treat it like an Object and only access Object’s public interface, which String fully supports.</li>
</ol>
<blockquote>
<p>This is what variance allows:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Superclass</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">virtual</span> Object DoSomething(String parameter) => <span style="color:#080;font-weight:bold">new</span> Object();
}
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Subclass</span> : Superclass
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">override</span> String DoSomething(Object parameter) => <span style="color:#d20;background-color:#fff0f0">"the result"</span>;
}
</code></pre></div><p>Be aware that C# currently does not support variance in parameters, only in return types. So this code would not actually work as it is, since the parameter types are different.</p>
</blockquote>
<ol start="17">
<li>The template method pattern is the ideal technique for creating superclasses that are easy to inherit from. It allows clean separation of an abstract algorithm from the concrete specializations.</li>
<li>Be on the lookout for code that directly invokes superclass behavior with keywords like “super” or “base” etc. Instead, use hook methods to allow subclasses to contribute to parts of the common algorithm.</li>
<li>Favor shallow hierarchies instead of deep ones. Narrow is also preferable to wide. Wide ones are easier to live with as long as they are shallow. But deep and wide ones are a maintenance nightmare. That is, keep the vertical levels of inheritance as low as you can. Hierarchies with many layers of superclasses are hard to understand and maintain.</li>
</ol>
<p><img src="/blog/2024/02/key-takeaways-from-poodr/hierarchies-come-in-different-shapes.webp" alt="Four diagrams show hierarchies of boxes. The top left is labeled “Shallow, Narrow” and has one box on top, with two boxes beneath, each connected to the top one by a line. The bottom left is labeled “Deep, Narrow” and shows the same diagram, but the box on the bottom left has two boxes of its own beneath it, one of which has its own boxes, et cetera, for a total of five layers, each with two boxes. The top right is labeled “Shallow, Wide”, and shows one box on top, with a single line connecting to each of five boxes directly beneath it. The bottom right is labeled “Deep, Wide”, and shows a 5-layer box diagram, with anywhere from 3 to 7 boxes on each level, connected by single lines. Different boxes have their own hierarchy, so boxes on the same level are not always direct siblings."></p>
<h3 id="chapter-8-combining-objects-with-composition">Chapter 8: Combining Objects with Composition</h3>
<h4 id="about-composition">About composition</h4>
<ol>
<li><strong>Composition</strong> establishes a “has a” relationship between objects. It is a technique where you combine simple, independent objects to turn them into larger, more complex ones. Composition combines distinct parts so that the resulting entity is more than the sum of its parts.</li>
<li>In composition, the container object (or composed object) “has a” contained component (or part). Mechanically, this usually means that the container’s class has an attribute which holds a reference to the part object. It’s a dependency that’s usually injected via constructor parameter.</li>
<li>The container communicates with the part via its public interface. The part plays a role, and the container collaborates with it. This means that containers generally interact with their parts through interfaces or duck types.</li>
<li>Composition is generally a cheaper alternative to code sharing when compared to inheritance.</li>
<li>When refactoring from inheritance into composition, the parts sometimes need to share some code, in addition to sharing the same public interface. This is because all possible parts play the same role for the container. In these cases, an inheritance hierarchy of parts may be what’s needed. Or, they could share behavior via modules or mixins.</li>
<li>In technical terms, the technique of composition takes two forms: <strong>aggregation</strong> and, well, <strong>composition</strong>. Under composition, parts don’t have any use or value outside of their container objects. Under aggregation, on the other hand, parts can live on their own.</li>
</ol>
<h4 id="factories">Factories</h4>
<ol start="7">
<li>When creating certain objects becomes complex, encapsulate that complexity into a factory. That way the knowledge is stored in a single place in the code base. In principle, factories are simple: They are objects that create other objects.</li>
<li>An advantage of factories is that they make the process of turning complex data structures into objects easy. You can have a “specification”, stored as pure data in a file or database. The factory can then implement the logic that understands the meaning of the data structure and how to create living objects based on it.</li>
</ol>
<h4 id="deciding-between-inheritance-and-composition">Deciding between inheritance and composition</h4>
<ol start="9">
<li>Inheritance and composition are fundamentally code arrangement techniques where logic is distributed among various objects. With inheritance, objects are organized into a strict hierarchical structure and get automatic message delegation. With composition, objects are independent from each other but messages need to be manually delegated.</li>
<li>In general, “<a href="https://en.wikipedia.org/wiki/Composition_over_inheritance">favor composition over inheritance</a>”. Composition is lower cost (i.e. fewer dependencies) than inheritance in most circumstances where both could be used to solve a problem.</li>
</ol>
<h4 id="pros-and-cons-of-inheritance">Pros and cons of inheritance</h4>
<ol start="11">
<li>When properly applied, inheritance is excellent at producing code that is Reasonable, Usable and Exemplary.</li>
<li><strong>Inheritable code is Reasonable</strong> because small changes in code can produce great changes in behavior. This is because code near the top of the hierarchy is defined once but used by all subclasses. Changing such code allows to alter the behavior of many subclasses.</li>
<li><strong>Inheritable code is Usable</strong> because superclasses are literally designed to be easy to reuse. Inheritance hierarchies adhere to the <a href="https://en.wikipedia.org/wiki/Open%E2%80%93closed_principle">open-closed principle</a>. It’s easy to extend the hierarchy by creating new subclasses without having to touch existing code, thus, reusing it.</li>
<li><strong>Inheritable code is Exemplary</strong> because it intrinsically provides a clear path to extend it. Even novice programmers won’t have a hard time creating new subclasses when the superclasses implement template methods and hooks which guide any extension efforts.</li>
<li>One important disadvantage of inheritance is that the cost of mistakes is considerably high. Incorrect applications of inheritance are costly, whether it be that the wrong abstraction was created or that inheritance was just the wrong tool for the job altogether.</li>
<li>Another disadvantage is that, for other contexts outside of the hierarchy, the ways in which it’s possible to interact with the hierarchy are limited. If the behavior defined within the hierarchy needs to be used, a subclass needs to be created. Other use cases may be incapable of tolerating that dependency.</li>
<li>If you’re writing a framework or a library, this aspect becomes even more important. You can’t know all the scenarios in which your library will be used, and forcing users to create subclasses in order to reuse the logic defined in the hierarchy may be more than what they can afford.</li>
</ol>
<h4 id="pros-and-cons-of-composition">Pros and cons of composition</h4>
<ol start="18">
<li>“When using composition, the natural tendency is to create many small objects that contain straightforward responsibilities that are accessible through clearly defined interfaces”. “Composition results in applications built of simple, pluggable objects that are easy to extend and have a high tolerance for change”.</li>
<li>When properly applied, composition is excellent at producing code that is Transparent, Reasonable and Usable.</li>
<li><strong>Composable code is Transparent</strong> because small objects are easy to understand and changes have obvious effects. These objects also don’t necessarily form part of hierarchies, which means they are not susceptible to changes in superclasses.</li>
<li><strong>Composable code is Reasonable</strong> because its behavior is easy to extend by just implementing new objects that play the role of parts. All new parts need to do is implement the public interface that the container object already expects.</li>
<li><strong>Composable code is Usable</strong> because it’s made up of small independent objects. They don’t have any structural dependencies preventing them from being reused in different contexts, completely unrelated to the container or even the part role.</li>
<li>One disadvantage of composition is the fact that it may be more difficult to understand how the whole application works. While the individual objects may be small and simple, how they come together to solve problems may not be.</li>
<li>Another disadvantage is that objects need to delegate messages to each other manually, as opposed to inheritance where this happens automatically. This means that there is the explicit dependency of the container object having to know which messages to call on its parts.</li>
<li>Composition excels at separating containers from parts and assembling such objects. However, it doesn’t have an answer for the scenario where it is necessary to handle a collection of types of parts that are very similar to each other. That’s where inheritance comes in.</li>
<li>Use inheritance for “is a” relationships. Use duck types, interfaces, and behavior sharing via modules or mixins for “behaves like a” relationships. Use composition for “has a” relationships.</li>
</ol>
<h3 id="chapter-9-designing-cost-effective-tests">Chapter 9: Designing Cost-Effective Tests</h3>
<h4 id="the-benefits-of-testing">The benefits of testing</h4>
<ol>
<li>The goal of design is to write code that is easy to change. In order to do that, we need good object-oriented design skills, good refactoring skills and good testing skills.</li>
<li>Good object-oriented design skills are needed because badly designed code, by definition, is hard to change.</li>
<li>Good refactoring skills are needed because design needs to be evolving constantly. As new requirements and new information about the domain become available, the code needs to adapt.</li>
<li>Good testing skills are needed because high value, solid tests enable continuous refactoring without fear of breaking the code.</li>
<li>Just like design, the true purpose of testing is to reduce costs. Tests do help reduce bugs, provide documentation, and improve the design; but those are only the means through which tests achieve their ultimate goal of reducing the cost of change.</li>
<li><strong>Tests help by finding bugs</strong>: Test help expose bugs early in the process. Bugs are easier to find and cheaper to correct the closer in time we are to their introduction. Testing at the same time a feature is being implemented catches bugs before they go out the door. Bugs caught early are not given the chance to cause problems or have code depend on them.</li>
<li><strong>Tests help by supplying documentation</strong>: Tests represent the best and most reliable documentation of the design. When static documents and memories get outdated and/or disappear, tests remain. Write tests that tell the story of how the code works.</li>
<li><strong>Tests help by deferring design decisions</strong>: As a code base evolves, there will be spots in the code that are less than stellar. Spots which we’re not confident enough in to commit to a particular design. So we postpone making a decision and hack together a solution that works today. However, we know that it will need to be refactored to handle the requirements of tomorrow. These spots represent a lack of knowledge. Knowledge that the design is waiting for in order to improve. While waiting for that knowledge to come, we put the hack behind a clear, stable public interface and write our tests against that. In doing so, we’ll be free to refactor into a good design when the missing information arrives. Knowing that the tests have our backs.</li>
<li><strong>Tests help by supporting abstractions</strong>: “Good design naturally progresses toward small independent objects that rely on abstractions”. One disadvantage to this is that, while small independent objects are easy to understand by themselves, the behavior of the whole application, with all the little objects working together, becomes obscured. Tests solve that problem by highlighting the abstractions, their interfaces, how to work with them, and how they work together.</li>
<li><strong>Tests help by exposing design flaws</strong>: If tests are hard to write, that’s the perfect indicator that the code under test is hard to reuse. Which indicates high coupling, which indicates increased costs of change. If testing an object requires a lot of setup and a lot of other objects, then it requires too much context and has too many dependencies.</li>
</ol>
<h4 id="what-when-and-how-to-test">What, when and how to test</h4>
<p><img src="/blog/2024/02/key-takeaways-from-poodr/object-under-test-are-like-space-capsules.webp" alt="A legend at the bottom shows solid blue and green lines labeled “depended upon by others”, and dashed green and red lines labeled “no dependents”. The main image is labeled “Origins of messages”. The left side is labeled “A. Received from others”, with three solid blue lines moving to the right, terminating at a conical object at the center. The cone is labeled “object under test”, and has a dotted gray line halfway up around the perimeter, with three dotted red lines branching off, looping around, and pointing back to the grey line. This is labeled “B. Sent to self”. Extending from the right side of the cone are three green lines, two solid and one dotted. They are labeled “C. Sent to others”."></p>
<ol start="11">
<li>“One simple way to get better value from tests is to write fewer of them. The safest way to accomplish this is to test everything just once and in the proper place”.</li>
<li><strong>Tests should focus only on the incoming messages</strong> that are defined in each object’s public interface (“A” in the picture above). That is, the public methods. Public interfaces expose the services that a class offers its users. They are also stable, so depending on them is safe. Tests that cover public methods are resilient to refactorings and add value because they exercise objects in the same way that their users in application code do.</li>
<li><strong>Never test private methods</strong> (“B” in the picture above). They are meant to be used by the object internally. They change often and other objects should not depend on them because they will have to change with them. The same applies to tests. Tests that exercise private code break on every refactoring.</li>
<li>An object’s test suite should <strong>never assert the return value of outgoing messages</strong> that it sends to other objects (“C” in the picture above). These messages are part of the public interface of the receiver object, so the receiver’s test suite should be the one testing it.</li>
<li>On the other hand, an object’s test suite should <strong>always assert that necessary outgoing messages are sent</strong>, and with the correct parameters (“C” in the picture above).</li>
<li>A “test of state” is a test that asserts the return value of a method.</li>
<li>A “test of behavior” is a test that asserts whether the object under test calls a particular method on another object, how many times, and with what parameters.</li>
<li>A “query” is a method call with no side effects that is only made to get some value back.</li>
<li>A “command” is a method call that has side effects that are important to the overall application.</li>
<li><strong>All public methods should be covered by tests of state</strong>. A test suite should only cover the public methods of its own object under test.</li>
<li><strong>Queries should not be tested for state or behavior</strong>.</li>
<li><strong>Commands should be tested for behavior</strong>.</li>
<li>Write tests first when it makes sense to do so. Writing tests before implementing the feature establishes an intention for the design from the beginning. Since tests are reused, they make sure the code to be written will have the bare minimum of reusability.</li>
<li>Testing first is no silver bullet, though. Even though it helps steer the design in the right direction, by itself it won’t produce a well-designed application. Fundamental object-oriented design techniques still need to be applied.</li>
<li>Well-designed applications are easy to change. Well-designed tests may never have to change. This is because they depend on abstractions and public interfaces, which are stable.</li>
<li>Two major styles of testing exist: Behavior Driven Development and Test Driven Development.</li>
<li>“BDD takes an outside-in approach, creating objects at the boundary of an application and working its way inward, mocking as necessary to supply as-yet-unwritten objects”.</li>
<li>“TDD takes an inside-out approach, usually starting with tests of domain objects and then reusing these newly created domain objects in the tests of adjacent layers of code”.</li>
<li><strong>Both styles can be followed to produce valuable tests</strong>.</li>
</ol>
<p><img src="/blog/2024/02/key-takeaways-from-poodr/bdd-and-tdd-should-be-viewed-as-on-a-continuum.webp" alt="A blue spectrum line has two objects on either side. At the left end, labeled “More outside in BDD”, are three concentric green circles, with the outermost labeled “1”, the middle labeled “2”, and the innermost labeled “3”. Each label is outside of its circle, with an arrow pointing to the corresponding circle. At the right end, labeled “More inside out TDD”, are three identical concentric green circles, this time with the innermost labeled 1, the middle labeled 2, and the outer labeled 3. Each number label lies in the circle and points outward."></p>
<h4 id="writing-valuable-tests">Writing valuable tests</h4>
<ol start="30">
<li><strong>Incoming messages (i.e. public methods) need to be covered by tests of state</strong>. That is, the values that they return must be validated.</li>
<li>Be on the lookout for public methods that don’t receive any calls. These are good candidates for deletion. If no other object calls it, then it needs to be private.</li>
<li><strong>Objects should be tested in isolation</strong>. When that’s not possible, there’s a problem with the design. It means that the object is too coupled to its collaborators and can’t be reused on its own. It needs to be refactored.</li>
<li>It’s important that objects are tested in isolation because a test suite should care only about the one object that its testing. The other objects will be tested by their own test suites. Changes in an object should affect its own test suite, not that of other objects.</li>
<li>When the object under test depends on another object in order to work properly, refactor into dependency injection and have the test provide the dependency that way.</li>
<li>When using dependency injection, don’t pass in a concrete, “real” object in your tests. Instead, inject a <strong>test double</strong> or <strong>mock</strong> into the object under test.</li>
<li>Dependency injection also hints at the emergence of a role, interface, or duck type. Instead of depending on a concrete class, refactor the object under test to depend on a role. The double or mock in the test suite will be one of the objects that play the collaborator role that the object under test expects.</li>
<li><strong>Private methods and query method calls need not be tested at all</strong>.</li>
<li>Testing private methods is redundant because they are internal to the object under test and invoked by public methods, which already have tests.</li>
<li>Private methods are also unstable. Not meant to be depended on because they change often. That will make for brittle tests.</li>
<li>Private methods covered by tests may mislead others in thinking that the method is stable and, thus, dependable.</li>
<li>Query method call results are not relevant to the overall application, only to the object under test. They are hidden within it. Also, the receiver of the query already includes tests for it, given that it is part of its public interface. So, they should be ignored by the test suite.</li>
<li><strong>Command method calls do need to be covered by tests</strong> in order to prove that they were called with the correct parameters. In other words, it is the responsibility of the object under test to send that message, so its test suite needs to prove it.</li>
<li>Mocks are the way to test that command messages get sent. They help with tests of behavior. Using mocks injected into the object under test, a test case can validate that the method is called properly.</li>
<li>Commands should be tested for behavior, not state. So, mocks should not be used for tests of state. That is, tests should not be concerned with asserting what they return. However, they can be configured to return some value, if the operation requires it.</li>
<li>In dynamically typed languages that don’t have a compiler to do type checks, it is important to test objects for conformance with the public interfaces of their roles, and for proper integration with their inheritance hierarchies.</li>
<li>“<strong>Tests should document the existence of roles, prove that each of their players behave correctly, and show that dependents interact with them appropriately</strong>”.</li>
<li>Pragmatically, this means writing test cases that assert whether objects implement the specific messages defined in their roles' public interface and those that their superclasses expect from them.</li>
<li>Multiple subclasses will undoubtedly share many of the same tests. In those cases, try to encapsulate the test cases into reusable components that every subclass’s test suite can invoke. Just like in application code, it’s better to avoid repetition.</li>
<li>Testing abstract base classes can sometimes be challenging because they are not supposed to be instantiated. In some languages, it is downright impossible to instantiate classes marked as abstract. For these scenarios, a test-specific subclass can be created with minimal implementation that allows tests to exercise the base class.</li>
<li>To avoid test brittleness, test doubles that play certain roles and inherit from superclasses should also be tested for conformance.</li>
</ol>
Setting User-Specific Resource Limits with systemdhttps://www.endpointdev.com/blog/2024/02/setting-user-specific-resource-limits-in-linux/2024-02-03T00:00:00+00:00Seth Jensen
<p><img src="/blog/2024/02/setting-user-specific-resource-limits-in-linux/sun-through-trees.webp" alt="In the center of a dim street lined with houses and cars, the sun illuminates and sillhouettes green foliage. It projects a triangle of light on the ground, first the street and then the grass, centered in the frame. On either side of the light, centered verticaly, a white car catches the light."></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>We recently encountered an issue on one of our servers where some processes were hogging CPU time and slowing down the entire machine to unusable speeds. It turned out that a couple of our developers were using the VS Code extension “Remote - SSH,” which gets SSH credentials to a server from you, logs in by SSH, figures out what kind of server it’s running on, and runs some server-side components that it talks to.</p>
<p>This is fine in principle, but with larger projects it frequently spins out of control and eats up all available CPU and RAM on the server. The developer using VS Code on their desktop likely won’t even notice, leaving a system administrator to kill the VS Code processes after it causes issues for other users.</p>
<p>Our answer to this problem was setting resource limits. In many cases, you could use <a href="https://linuxcommand.org/lc3_man_pages/ulimith.html">ulimit</a> to control a user’s resource limits. But ulimit only controls resources for the shell it’s running in, and since VS Code was connecting through SSH and spawning a bunch of processes, we didn’t have a reasonable way to do this.</p>
<p>Instead of using ulimit, you can use control groups (cgroups) to set overall user limits, not just limits on a specific process or in a specific shell. This is good practice for any multi-user server so that different parts of the server are protected from each other. We will do this by adding a systemd controller to control a specific user slice.</p>
<blockquote>
<p>On PAM systems, you can also use the <code>pam_limits</code> module to set user-wide resource limits. O’Reilly has a <a href="https://www.oreilly.com/library/view/network-security-hacks/0596006438/ch01s20.html">guide on configuring limits</a> this module.</p>
</blockquote>
<h3 id="a-closer-look-at-setting-limits-with-cgroups">A closer look at setting limits with cgroups</h3>
<p>You can add systemd controllers—including resource limits—via drop-in files in <code>/etc/systemd/system/</code>.</p>
<p>To set limits for a specific user, we need the user’s UID. You can find this in the third column of <code>/etc/passwd</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">seth:x:1000:1000:Seth Jensen:/home/seth:/bin/bash
testy:x:1001:1001:Testy Tester:/home/testy:/bin/bash
</code></pre></div><p>For the <code>seth</code> user, the UID is 1000, so we will create a file called <code>50-limits.conf</code> in the <code>/etc/systemd/system/user-1000.slice.d/</code> directory to limit that user’s resources.</p>
<p>The <code>50-</code> in the filename just determines the order in which the config files will be applied, so it shouldn’t matter if this is the only file you’re adding.</p>
<p>Now, in our <code>50-limits.conf</code> file we can add limits:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">[Slice]
CPUQuota=50%
MemoryLimit=1G
TasksMax=40
</code></pre></div><p>Here we’re just limiting CPU time, memory, and the number of tasks run by the <code>seth</code> user. You can see a full list of what’s available with <a href="https://www.freedesktop.org/software/systemd/man/latest/systemd.resource-control.html">man systemd.resource-control</a>.</p>
<p>It’s worth noting that <code>CPUQuota=50%</code> will limit <code>seth</code> to 50% of a <em>single</em> CPU, which would be 25% of total CPU on a dual-core system, etc.</p>
<p>Finding the right values for your limits is a bit of a balancing act: if you set the resource limits too tight, your programs may not be able to run. But, if you set them too loose, the misbehaving program may still end up hogging too many resources. Try different values while running the resource-heavy program, adjusting as needed to find the sweet spot for your case.</p>
Building a real-time application with SignalR, .NET, and three.jshttps://www.endpointdev.com/blog/2024/01/building-real-time-app-signalr-dotnet-threejs/2024-01-13T00:00:00+00:00Bimal Gharti Magar
<p><img src="/blog/2024/01/building-real-time-app-signalr-dotnet-threejs/clock-banner.webp" alt="A white watchface, without any other parts, sits on a surface with a dotted texture"><br>
Image by Pixabay on Pexels</p>
<p>When data in a web application is changing in real time, users want to see those updates reflected in real time without refreshing the application. Adding real-time functionality to a .NET application is easy with the <a href="https://dotnet.microsoft.com/en-us/apps/aspnet/signalr">SignalR</a> library.</p>
<p>Depending on the capabilities of the client and server, SignalR can use WebSockets, server-sent events, or long polling to establish a persistent connection between the client and the server. From the server side, SignalR pushes code to connected clients. SignalR is good for applications that require high-frequency updates from the server, such as real-time gaming.</p>
<p>C# code can be used to write SignalR hubs, which easily integrate with other ASP.NET features like dependency injection, authentication, authorization, and scalability.</p>
<h3 id="what-well-build">What we’ll build</h3>
<p>We will learn how to use SignalR for real-time communication to send the camera position of a cube, built using <a href="https://threejs.org/">three.js</a>, to all users on the page.</p>
<p><img src="/blog/2024/01/building-real-time-app-signalr-dotnet-threejs/signal-real-time-cube.gif" alt="Two browser windows are open side by side, to the same web app. On both, under a “Receiver” title is a window showing a 3D cube, after which a “Controller” title is followed by a “Request Control” button. In the left browser window, the mouse clicks the blue “Request Control” button, which turns green and displays a message underneath saying “Control granted: 120 seconds remaining”, with the number counting down every second. Another identical 3D cube window opens beneath. The clicks and drags in this bottom window, which rotates the cube. After a short delay, the other two cubes in the “Receiver” section follow and move to the position where the controller moved the cube."></p>
<p>You can see the <a href="https://nifty.azurewebsites.net/">demo here</a>.</p>
<p>Here is the description of our testing app:</p>
<ul>
<li>A user can request control of a cube for 2 minutes</li>
<li>Control can be released manually, or it will be automatically released after 2 minutes</li>
<li>A user can control the camera position of cube, which will be seen by all users viewing the page</li>
<li>When one user is controlling the cube, other users requesting control will be added to a queue</li>
<li>Queued users will be granted control automatically when the controlling user’s time is over</li>
</ul>
<p>First we will create a web application and create a SignalR communication between the client and server. After that, we can implement the above features.</p>
<h3 id="creating-the-web-application">Creating the web application</h3>
<p>Create a new .NET application.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">> dotnet new webapp -o EPSignalRControl
</code></pre></div><p>I edited the app using VS Code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">> code .\EPSignalRControl\
</code></pre></div><h4 id="installing-and-configuring-signalr">Installing and configuring SignalR</h4>
<p>The SignalR server library is included in the ASP.NET Core shared framework. However, the JavaScript client library isn’t automatically included in the project. You can use Library Manager (LibMan) to get the client library from unpkg. unpkg is a fast global content delivery network for everything on npm.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">> dotnet tool install -g Microsoft.Web.LibraryManager.Cli
> libman install @microsoft/signalr@latest -p unpkg -d wwwroot/js/signalr --files dist/browser/signalr.js
</code></pre></div><h4 id="creating-a-signalr-hub">Creating a SignalR hub</h4>
<p>We’ll create a <code>CameraData</code> model class for passing data between the SignalR server and client.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">CameraData</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">double</span> x { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#00d;font-weight:bold">0</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">double</span> y { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#00d;font-weight:bold">0</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">double</span> z { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#00d;font-weight:bold">5</span>;
}
</code></pre></div><p>We’ll also create a <code>ControlHub</code> class which inherits from <code>Hub</code>.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.SignalR</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">EPSignalRControl.Hubs</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ControlHub</span> : Hub
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task SendData(CameraData data)
{
<span style="color:#888">// Broadcast the data to all clients
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">await</span> Clients.All.SendAsync(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, data);
}
}
</code></pre></div><p>The <a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub?view=aspnetcore-8.0">Hub</a> class has Clients, Context, and Groups properties:</p>
<ul>
<li><strong><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub.clients?view=aspnetcore-8.0">Clients</a></strong> can be used to invoke methods on the clients connected to this hub.</li>
<li><strong><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub.context?view=aspnetcore-8.0">Context</a></strong> is the hub caller context for accessing information about the hub caller connection.</li>
<li><strong><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub.groups?view=aspnetcore-8.0">Groups</a></strong> is the group manager to manage connections in groups.</li>
</ul>
<blockquote>
<p>One thing to note is that hubs are transient, so we cannot store state in a property of the Hub class. Each hub method call is executed on a new Hub instance.</p>
</blockquote>
<p><code>Clients.All</code> calls a method on all connected clients. As a result, the control hub sends the data received from one client back to all connected clients. Similarly, we can use <code>Clients.Caller</code> to send back data to the client that invoked the hub method.</p>
<p>Then, we need to register the services using <code>AddSignalR()</code> and configure the endpoints using <code>MapHub()</code> required by SignalR in <code>Program.cs</code></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">EPSignalRControl.Hubs</span>;
<span style="color:#888;font-weight:bold">var</span> builder = WebApplication.CreateBuilder(args);
builder.Services.AddRazorPages(); <span style="color:#888">// Add services to the container
</span><span style="color:#888"></span>builder.Services.AddSignalR(); <span style="color:#888">// Register SignalR service
</span><span style="color:#888"></span>
<span style="color:#888;font-weight:bold">var</span> app = builder.Build();
<span style="color:#080;font-weight:bold">if</span> (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler(<span style="color:#d20;background-color:#fff0f0">"/Error"</span>);
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthorization();
app.MapRazorPages();
app.MapHub<ControlHub>(<span style="color:#d20;background-color:#fff0f0">"/controlHub"</span>); <span style="color:#888">// Map ControlHub to the '/controlHub' endpoint
</span><span style="color:#888"></span>
app.Run();
</code></pre></div><h4 id="connecting-to-signalr-hub-using-the-signalr-javascript-client-library">Connecting to SignalR Hub using the SignalR JavaScript client library</h4>
<p>We’ll reference the SignalR JavaScript library as well as create a <code>site.js</code> file and reference it in <code>Index.cshtml</code>.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"~/js/signalr/dist/browser/signalr.js"</span>></<span style="color:#b06;font-weight:bold">script</span>>
<<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">type</span>=<span style="color:#d20;background-color:#fff0f0">"module"</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"~/js/site.js"</span> <span style="color:#369">asp-append-version</span>=<span style="color:#d20;background-color:#fff0f0">"true"</span>></<span style="color:#b06;font-weight:bold">script</span>>
</code></pre></div><p>In the <code>site.js</code> file, we will:</p>
<ul>
<li>Connect to the control hub using the endpoint <code>/connecthub</code></li>
<li>Start the connection with our control hub</li>
<li>Send data to the server by invoking the <code>SendData</code> method of ControlHub</li>
<li>Add a listener for <code>ReceiveData</code> to receive the data sent from SignalR Hub</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">const</span> controlHubConnection = <span style="color:#080;font-weight:bold">new</span> signalR.HubConnectionBuilder()
.withUrl(<span style="color:#d20;background-color:#fff0f0">"/controlhub"</span>)
.build();
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, <span style="color:#080;font-weight:bold">function</span> (cameraData) {
console.log(<span style="color:#d20;background-color:#fff0f0">`position.x = </span><span style="color:#33b;background-color:#fff0f0">${</span>cameraData.x<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">`</span>);
console.log(<span style="color:#d20;background-color:#fff0f0">`position.y = </span><span style="color:#33b;background-color:#fff0f0">${</span>cameraData.y<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">`</span>);
console.log(<span style="color:#d20;background-color:#fff0f0">`position.z = </span><span style="color:#33b;background-color:#fff0f0">${</span>cameraData.z<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">`</span>);
});
controlHubConnection.start()
.then(<span style="color:#080;font-weight:bold">function</span> () {
console.log(<span style="color:#d20;background-color:#fff0f0">"Connected to ControlHub"</span>);
console.log(<span style="color:#d20;background-color:#fff0f0">"Invoking SendData"</span>);
controlHubConnection.invoke(<span style="color:#d20;background-color:#fff0f0">"SendData"</span>, {
x: <span style="color:#00d;font-weight:bold">100</span>,
y: <span style="color:#00d;font-weight:bold">100</span>,
z: <span style="color:#00d;font-weight:bold">100</span>
})
.<span style="color:#080;font-weight:bold">catch</span>(<span style="color:#080;font-weight:bold">function</span> (err) {
<span style="color:#080;font-weight:bold">return</span> console.error(err.toString());
});
}).<span style="color:#080;font-weight:bold">catch</span>(<span style="color:#080;font-weight:bold">function</span> (err) {
console.error(<span style="color:#d20;background-color:#fff0f0">"Error connecting to ControlHub: "</span>, err);
});
</code></pre></div><p>At this point if we run the web application, open the browser, and look at the developer tools console, we will see:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Connected to ControlHub
Invoking SendData
position.x = 100
position.y = 100
position.z = 100
</code></pre></div><p>If we open the web application in a different tab, then in the first tab’s dev tools console we will see the position values are appended. This is because the new tab invoked <code>SendData</code> and the server received and sent back data to all connected clients.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Connected to ControlHub
Invoking SendData
position.x = 100
position.y = 100
position.z = 100
position.x = 100
position.y = 100
position.z = 100
</code></pre></div><p>This is the basic way of implementing SignalR in the project to achieve real-time functionality. Now, we’ll dive deep into creating more features to extend the app to more practical usage.</p>
<h3 id="adding-features">Adding features</h3>
<h4 id="backend">Backend</h4>
<p>Add a <code>ControlRequest.cs</code> model to hold the SignalR connection ID and the time of the request.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ControlRequest</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> ConnectionId { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#888;font-weight:bold">string</span>.Empty;
<span style="color:#080;font-weight:bold">public</span> DateTime RequestTime { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
</code></pre></div><p>Add two variables in <code>ControlHub.cs</code>: <code>currentControl</code> to hold the details of the current active request, and <code>controlQueue</code> to hold the requests in queue.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">static</span> ControlRequest? currentControl;
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">static</span> Queue<ControlRequest> controlQueue = <span style="color:#080;font-weight:bold">new</span> Queue<ControlRequest>();
</code></pre></div><p>Add a <code>ControlTimer.cs</code> class with properties for a ConcurrentDictionary mapping of the connection ID, the ControlHub clients, the ControlHub context, and the ControlTimer object.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Collections.Concurrent</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.SignalR</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ControlTimer</span> : System.Timers.Timer
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> ConcurrentDictionary<<span style="color:#888;font-weight:bold">string</span>, ControlTimer> ControlTimers = <span style="color:#080;font-weight:bold">new</span>();
<span style="color:#080;font-weight:bold">public</span> HubCallerContext hubContext { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> IHubCallerClients hubCallerClients { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> ControlTimer(<span style="color:#888;font-weight:bold">double</span> interval) : <span style="color:#080;font-weight:bold">base</span>(interval) { }
}
</code></pre></div><p>Add two more variables in <code>ControlHub.cs</code>:</p>
<ul>
<li><code>controlTimer</code> to store control timer dictionary, control hub context, and control hub clients that can be accessed during the interval of the timer that runs for the active request.</li>
<li><code>CONTROL_TIME</code> holds the time in seconds for which the active request has control.</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> ControlTimer? controlTimer;
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">const</span> <span style="color:#888;font-weight:bold">int</span> CONTROL_TIME = <span style="color:#00d;font-weight:bold">120</span>; <span style="color:#888">//seconds
</span></code></pre></div><p>Add a <code>RequestControl</code> method to <code>ControlHub</code> that is invoked by the client to request the control. The method adds the connection ID of the invoked request to the queue, and creates and adds the control timer for the currently invoked request. If the queue has only one request, it gives control to the current request, starts the timer and sends a message to the requesting client notifying that access is granted. If the queue has more than one request, the requesting client is notified that their request is queued.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task RequestControl()
{
<span style="color:#888;font-weight:bold">var</span> controlRequest = <span style="color:#080;font-weight:bold">new</span> ControlRequest
{
ConnectionId = Context.ConnectionId,
RequestTime = DateTime.Now
};
<span style="color:#888">// Add the control request to the queue
</span><span style="color:#888"></span> controlQueue.Enqueue(controlRequest);
<span style="color:#888">// Add timer to dictionary for request
</span><span style="color:#888"></span> controlTimer = <span style="color:#080;font-weight:bold">new</span> ControlTimer(<span style="color:#00d;font-weight:bold">500</span>)
{
hubContext = Context,
hubCallerClients = Clients
};
controlTimer = ControlTimer.ControlTimers.GetOrAdd(controlRequest.ConnectionId, controlTimer);
<span style="color:#888">// Grant control if the queue is empty or it's the first request
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (controlQueue.Count == <span style="color:#00d;font-weight:bold">1</span> || controlQueue.Peek() == controlRequest)
{
currentControl = controlRequest;
SetupTimerForRelease(controlRequest);
<span style="color:#080;font-weight:bold">await</span> Clients.Client(controlRequest.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlGranted"</span>);
}
<span style="color:#080;font-weight:bold">else</span>
{
<span style="color:#080;font-weight:bold">await</span> Clients.Client(controlRequest.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlQueued"</span>);
}
}
</code></pre></div><p>Add a <code>SetupTimerForRelease</code> method to add calls to the <code>ReleaseControlMiddleware</code> method on a regular interval from the running timer.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> SetupTimerForRelease(ControlRequest controlRequest)
{
<span style="color:#080;font-weight:bold">if</span> (controlRequest != <span style="color:#080;font-weight:bold">null</span> && controlTimer == <span style="color:#080;font-weight:bold">null</span>)
{
controlTimer = ControlTimer.ControlTimers.GetOrAdd(controlRequest.ConnectionId, controlTimer);
}
controlTimer.Elapsed += <span style="color:#080;font-weight:bold">new</span> ElapsedEventHandler(ReleaseControlMiddleware);
controlTimer.Enabled = <span style="color:#080;font-weight:bold">true</span>;
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> ReleaseControlMiddleware(<span style="color:#888;font-weight:bold">object</span> source, ElapsedEventArgs e)
{
<span style="color:#00d;font-weight:bold">_</span> = AutoReleaseControl(source, e);
}
</code></pre></div><p>Add an <code>AutoReleaseControl</code> method, to be called by a running timer, which checks if the time for the current control has elapsed. The method sends the time remaining message to the user with control. If the time has elapsed, it calls <code>ClearTimerAndControl</code> to clear the timer and release control.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task AutoReleaseControl(<span style="color:#888;font-weight:bold">object</span> source, ElapsedEventArgs e)
{
<span style="color:#888;font-weight:bold">var</span> controlTimer = (ControlTimer)source;
HubCallerContext hcallerContext = controlTimer.hubContext;
IHubCallerClients hubClients = controlTimer.hubCallerClients;
<span style="color:#080;font-weight:bold">if</span> (currentControl != <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#888;font-weight:bold">var</span> elapsedSeconds = Math.Ceiling(DateTime.Now.Subtract(currentControl.RequestTime).TotalSeconds);
<span style="color:#080;font-weight:bold">await</span> hubClients.Client(currentControl.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlRemaining"</span>, CONTROL_TIME - elapsedSeconds);
<span style="color:#080;font-weight:bold">if</span> (elapsedSeconds >= CONTROL_TIME)
{
<span style="color:#080;font-weight:bold">await</span> ClearTimerAndControl(hubClients, hcallerContext);
}
}
}
</code></pre></div><p>Add <code>ClearTimerAndControl</code> method which:</p>
<ul>
<li>Clears the timer for a given connection ID</li>
<li>Sends a message to the controlling client informing that their control time is released</li>
<li>Sends default camera position data to all the clients to reset the cube camera position</li>
<li>Dequeues the current control from the queue, which is the first item</li>
<li>Gives control to the next request in the queue</li>
<li>Sends a message to the client who is granted the control</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">async</span> Task ClearTimerAndControl(IHubCallerClients hubClients, HubCallerContext context)
{
<span style="color:#080;font-weight:bold">try</span>
{
<span style="color:#888">// Clear the timer when control is explicitly released
</span><span style="color:#888"></span> ClearControlTimer(context.ConnectionId);
<span style="color:#080;font-weight:bold">await</span> hubClients.Client(context.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlReleased"</span>);
<span style="color:#080;font-weight:bold">await</span> hubClients.All.SendAsync(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, <span style="color:#080;font-weight:bold">new</span> CameraData());
<span style="color:#888">// Release control
</span><span style="color:#888"></span> currentControl = <span style="color:#080;font-weight:bold">null</span>;
<span style="color:#888">// Remove the first request from the queue
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (controlQueue.Count > <span style="color:#00d;font-weight:bold">0</span>)
controlQueue.Dequeue();
<span style="color:#888">// Grant control to the next in the queue
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (controlQueue.Count > <span style="color:#00d;font-weight:bold">0</span>)
{
currentControl = controlQueue.Peek();
currentControl.RequestTime = DateTime.Now;
SetupTimerForRelease(currentControl);
<span style="color:#080;font-weight:bold">await</span> hubClients.Client(currentControl.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlGranted"</span>, currentControl.RequestTime.ToString());
}
}
<span style="color:#080;font-weight:bold">catch</span> (Exception ex)
{
<span style="color:#080;font-weight:bold">await</span> Clients.All.SendAsync(ex.ToString());
}
}
</code></pre></div><p>Add a <code>ClearControlTimer</code> method which gets the control timer of the given connection ID. Removes the <code>ReleaseControlMiddleware</code> from the control timer and disables the control timer.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> ClearControlTimer(<span style="color:#888;font-weight:bold">string</span> connectionId)
{
controlTimer = ControlTimer.ControlTimers.GetOrAdd(connectionId, <span style="color:#080;font-weight:bold">new</span> ControlTimer(<span style="color:#00d;font-weight:bold">500</span>));
<span style="color:#080;font-weight:bold">if</span> (controlTimer != <span style="color:#080;font-weight:bold">null</span>)
{
controlTimer.Elapsed -= <span style="color:#080;font-weight:bold">new</span> ElapsedEventHandler(ReleaseControlMiddleware);
controlTimer.Enabled = <span style="color:#080;font-weight:bold">false</span>;
controlTimer = <span style="color:#080;font-weight:bold">null</span>;
}
}
</code></pre></div><p>Add a <code>ReleaseControl</code> method to <code>ControlHub</code>, that is invoked manually by the controlling client to release the control. It checks if the requesting client has the current control access and calls the method to clear timer.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task ReleaseControl()
{
<span style="color:#080;font-weight:bold">if</span> (currentControl != <span style="color:#080;font-weight:bold">null</span> && currentControl.ConnectionId == Context.ConnectionId)
{
<span style="color:#080;font-weight:bold">await</span> ClearTimerAndControl(Clients, Context);
}
}
</code></pre></div><p>Add a <code>SendData</code> method, which receives the data from the controlling user and then sends the received data to all the clients.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task SendData(CameraData cameraData)
{
controlTimer = ControlTimer.ControlTimers.GetOrAdd(Context.ConnectionId, <span style="color:#080;font-weight:bold">new</span> ControlTimer(<span style="color:#00d;font-weight:bold">500</span>));
<span style="color:#080;font-weight:bold">if</span> (controlTimer != <span style="color:#080;font-weight:bold">null</span> && controlTimer.hubContext != <span style="color:#080;font-weight:bold">null</span> && Context.ConnectionId == controlTimer.hubContext.ConnectionId)
{
<span style="color:#888">// Broadcast the received sensor data to all clients
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">await</span> controlTimer.hubCallerClients.All.SendAsync(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, cameraData);
}
}
</code></pre></div><p>That’s all the code required for the backend. Now let’s add to the frontend.</p>
<h4 id="frontend">Frontend</h4>
<p>In <code>Index.cshtml</code>, we’ll include:</p>
<ul>
<li>A receiver section to show the three.js cube and update the camera position using the received data from SignalR connection.</li>
<li>A section to show the message from backend.</li>
<li>A controller section with button to request the control using SignalR connection and show the three.js cube. The cube can be rotated with the mouse which sends the data to the backend using the SignalR connection. The backend then sends the data back to all the clients to update the cube camera position in receiver section.</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html">@page
@model IndexModel
@{
ViewData["Title"] = "Home page";
}
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"wrapper"</span>>
<<span style="color:#b06;font-weight:bold">h3</span>>Receiver</<span style="color:#b06;font-weight:bold">h3</span>>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"receiver-wrapper"</span>></<span style="color:#b06;font-weight:bold">div</span>>
<<span style="color:#b06;font-weight:bold">hr</span> />
<<span style="color:#b06;font-weight:bold">h3</span>>Controller</<span style="color:#b06;font-weight:bold">h3</span>>
<<span style="color:#b06;font-weight:bold">button</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"btn btn-primary"</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"requestCtrlBtn"</span> <span style="color:#369">disabled</span>></<span style="color:#b06;font-weight:bold">button</span>>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"alert alert-info d-none"</span> <span style="color:#369">role</span>=<span style="color:#d20;background-color:#fff0f0">"alert"</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"message"</span>></<span style="color:#b06;font-weight:bold">div</span>>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"controller-wrapper"</span>></<span style="color:#b06;font-weight:bold">div</span>>
</<span style="color:#b06;font-weight:bold">div</span>>
<<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"~/js/signalr/dist/browser/signalr.js"</span>></<span style="color:#b06;font-weight:bold">script</span>>
<<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">type</span>=<span style="color:#d20;background-color:#fff0f0">"module"</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"~/js/site.js"</span> <span style="color:#369">asp-append-version</span>=<span style="color:#d20;background-color:#fff0f0">"true"</span>></<span style="color:#b06;font-weight:bold">script</span>>
</code></pre></div><p>Then we can add JavaScript code to implement the app functionality in the template. We’ll add:</p>
<ul>
<li>A click handler to invoke the <code>RequestControl</code> method using the SignalR connection.</li>
<li>Listeners for the <code>ControlGranted</code>, <code>ControlQueued</code>, <code>ControlReleased</code>, and <code>ControlRemaining</code> events to update the UI based on the message included in the corresponding event.</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-js" data-lang="js"><span style="color:#080;font-weight:bold">let</span> controlRequested = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">let</span> controlGranted = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">let</span> destroyController = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">const</span> requestCtrlBtn = <span style="color:#038">document</span>.getElementById(<span style="color:#d20;background-color:#fff0f0">'requestCtrlBtn'</span>)
requestCtrlBtn.disabled = <span style="color:#080;font-weight:bold">true</span>;
requestCtrlBtn.innerText = <span style="color:#d20;background-color:#fff0f0">"Request Control"</span>;
<span style="color:#080;font-weight:bold">const</span> messageCtrl = <span style="color:#038">document</span>.getElementById(<span style="color:#d20;background-color:#fff0f0">'message'</span>)
requestCtrlBtn.addEventListener(<span style="color:#d20;background-color:#fff0f0">'click'</span>, () => {
<span style="color:#080;font-weight:bold">let</span> action = <span style="color:#d20;background-color:#fff0f0">'RequestControl'</span>;
<span style="color:#080;font-weight:bold">if</span> (controlGranted) {
action = <span style="color:#d20;background-color:#fff0f0">'ReleaseControl'</span>;
}
controlHubConnection.invoke(action).<span style="color:#080;font-weight:bold">catch</span>(<span style="color:#080;font-weight:bold">function</span> (err) {
<span style="color:#080;font-weight:bold">return</span> console.error(err.toString());
});
})
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ControlGranted"</span>, <span style="color:#080;font-weight:bold">function</span> () {
controlGranted = <span style="color:#080;font-weight:bold">true</span>;
controlRequested = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.disabled = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.classList.remove(...[<span style="color:#d20;background-color:#fff0f0">'btn-primary'</span>, <span style="color:#d20;background-color:#fff0f0">'btn-warning'</span>]);
requestCtrlBtn.classList.add(<span style="color:#d20;background-color:#fff0f0">'btn-success'</span>);
requestCtrlBtn.innerText = <span style="color:#d20;background-color:#fff0f0">"Release Control"</span>;
destroyController = createController();
});
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ControlQueued"</span>, <span style="color:#080;font-weight:bold">function</span> () {
controlRequested = <span style="color:#080;font-weight:bold">true</span>;
controlGranted = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.disabled = <span style="color:#080;font-weight:bold">true</span>;
requestCtrlBtn.classList.remove(<span style="color:#d20;background-color:#fff0f0">'btn-primary'</span>);
requestCtrlBtn.classList.add(<span style="color:#d20;background-color:#fff0f0">'btn-warning'</span>);
requestCtrlBtn.innerText = <span style="color:#d20;background-color:#fff0f0">"Waiting in Queue"</span>;
});
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ControlReleased"</span>, <span style="color:#080;font-weight:bold">function</span> () {
controlRequested = <span style="color:#080;font-weight:bold">false</span>;
controlGranted = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.disabled = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.innerText = <span style="color:#d20;background-color:#fff0f0">"Request Control"</span>;
requestCtrlBtn.classList.remove(...[<span style="color:#d20;background-color:#fff0f0">'btn-success'</span>, <span style="color:#d20;background-color:#fff0f0">'btn-warning'</span>]);
requestCtrlBtn.classList.add(<span style="color:#d20;background-color:#fff0f0">'btn-primary'</span>);
messageCtrl.innerText = <span style="color:#d20;background-color:#fff0f0">''</span>;
messageCtrl.classList.add(<span style="color:#d20;background-color:#fff0f0">'d-none'</span>);
<span style="color:#080;font-weight:bold">if</span> (destroyController != <span style="color:#080;font-weight:bold">null</span>) {
destroyController();
destroyController = <span style="color:#080;font-weight:bold">null</span>;
}
});
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ControlRemaining"</span>, <span style="color:#080;font-weight:bold">function</span> (seconds) {
messageCtrl.classList.remove(<span style="color:#d20;background-color:#fff0f0">'d-none'</span>);
messageCtrl.innerText = <span style="color:#d20;background-color:#fff0f0">`Control Granted: </span><span style="color:#33b;background-color:#fff0f0">${</span>seconds<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0"> seconds remaining`</span>;
});
</code></pre></div><p>At the top of the file, import the <code>three.js</code> library along with the <code>OrbitalControl</code> add-on to control the cube camera position.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">import</span> * as THREE from <span style="color:#d20;background-color:#fff0f0">'three'</span>;
<span style="color:#080;font-weight:bold">import</span> { OrbitControls } from <span style="color:#d20;background-color:#fff0f0">'three/addons/controls/OrbitControls.js'</span>;
</code></pre></div><p>Add <code>createRenderer</code>, <code>createScene</code>, <code>createCamera</code>, and <code>createCube</code> helper methods that are required for creating cube and attaching it to the receiver as well as the controller section to show a cube.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">function</span> createRenderer(canvasDOMId) {
<span style="color:#888">// Load a Renderer
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">let</span> renderer = <span style="color:#080;font-weight:bold">new</span> THREE.WebGLRenderer({ alpha: <span style="color:#080;font-weight:bold">false</span>, antialias: <span style="color:#080;font-weight:bold">true</span> });
renderer.setClearColor(<span style="color:#00d;font-weight:bold">0xC5C5C3</span>);
renderer.setPixelRatio(<span style="color:#038">window</span>.devicePixelRatio);
renderer.setSize(<span style="color:#00d;font-weight:bold">250</span>, <span style="color:#00d;font-weight:bold">250</span>);
<span style="color:#038">document</span>.getElementById(canvasDOMId).appendChild(renderer.domElement);
<span style="color:#080;font-weight:bold">return</span> renderer;
}
<span style="color:#080;font-weight:bold">function</span> createScene() {
<span style="color:#888">// Load 3D Scene
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">let</span> scene = <span style="color:#080;font-weight:bold">new</span> THREE.Scene();
<span style="color:#888">// Load Light
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">var</span> ambientLight = <span style="color:#080;font-weight:bold">new</span> THREE.AmbientLight(<span style="color:#00d;font-weight:bold">0xcccccc</span>);
scene.add(ambientLight);
<span style="color:#080;font-weight:bold">var</span> directionalLight = <span style="color:#080;font-weight:bold">new</span> THREE.DirectionalLight(<span style="color:#00d;font-weight:bold">0xffffff</span>);
directionalLight.position.set(<span style="color:#00d;font-weight:bold">0</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#00d;font-weight:bold">1</span>).normalize();
scene.add(directionalLight);
<span style="color:#080;font-weight:bold">return</span> scene;
}
<span style="color:#080;font-weight:bold">function</span> createCamera(cameraZPosition = <span style="color:#00d;font-weight:bold">10</span>) {
<span style="color:#888">// Load Camera Perspective
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">let</span> camera = <span style="color:#080;font-weight:bold">new</span> THREE.PerspectiveCamera(<span style="color:#00d;font-weight:bold">50</span>, <span style="color:#00d;font-weight:bold">250</span> / <span style="color:#00d;font-weight:bold">250</span>, <span style="color:#00d;font-weight:bold">1</span>, <span style="color:#00d;font-weight:bold">200</span>);
camera.position.z = <span style="color:#00d;font-weight:bold">5</span>;
<span style="color:#080;font-weight:bold">return</span> camera;
}
<span style="color:#080;font-weight:bold">function</span> createCube() {
<span style="color:#080;font-weight:bold">const</span> geometry = <span style="color:#080;font-weight:bold">new</span> THREE.BoxGeometry(<span style="color:#00d;font-weight:bold">2</span>, <span style="color:#00d;font-weight:bold">2</span>, <span style="color:#00d;font-weight:bold">2</span>).toNonIndexed();;
<span style="color:#080;font-weight:bold">const</span> positionAttribute = geometry.getAttribute(<span style="color:#d20;background-color:#fff0f0">'position'</span>);
<span style="color:#080;font-weight:bold">const</span> colors = [];
<span style="color:#080;font-weight:bold">const</span> color = <span style="color:#080;font-weight:bold">new</span> THREE.Color();
<span style="color:#080;font-weight:bold">for</span> (<span style="color:#080;font-weight:bold">let</span> i = <span style="color:#00d;font-weight:bold">0</span>; i < positionAttribute.count; i += <span style="color:#00d;font-weight:bold">3</span>) {
<span style="color:#080;font-weight:bold">if</span> (i >= <span style="color:#00d;font-weight:bold">0</span> && i <= <span style="color:#00d;font-weight:bold">11</span>) {
color.set(<span style="color:#00d;font-weight:bold">0xffff00</span>); <span style="color:#888">// x facing yellow
</span><span style="color:#888"></span> }
<span style="color:#080;font-weight:bold">else</span> <span style="color:#080;font-weight:bold">if</span> (i >= <span style="color:#00d;font-weight:bold">12</span> && i <= <span style="color:#00d;font-weight:bold">23</span>) {
color.set(<span style="color:#00d;font-weight:bold">0xff0000</span>); <span style="color:#888">// y facing red
</span><span style="color:#888"></span> }
<span style="color:#080;font-weight:bold">else</span> {
color.set(<span style="color:#00d;font-weight:bold">0x0000ff</span>); <span style="color:#888">// z facing blue
</span><span style="color:#888"></span> }
<span style="color:#888">// define the same color for each vertex of a triangle
</span><span style="color:#888"></span> colors.push(color.r, color.g, color.b);
colors.push(color.r, color.g, color.b);
colors.push(color.r, color.g, color.b);
}
geometry.setAttribute(<span style="color:#d20;background-color:#fff0f0">'color'</span>, <span style="color:#080;font-weight:bold">new</span> THREE.Float32BufferAttribute(colors, <span style="color:#00d;font-weight:bold">3</span>));
<span style="color:#080;font-weight:bold">const</span> material = <span style="color:#080;font-weight:bold">new</span> THREE.MeshBasicMaterial({ vertexColors: <span style="color:#080;font-weight:bold">true</span> });
<span style="color:#080;font-weight:bold">let</span> cube = <span style="color:#080;font-weight:bold">new</span> THREE.Mesh(geometry, material);
<span style="color:#888">// wireframe
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">var</span> geo = <span style="color:#080;font-weight:bold">new</span> THREE.EdgesGeometry(cube.geometry); <span style="color:#888">// or WireframeGeometry
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">var</span> mat = <span style="color:#080;font-weight:bold">new</span> THREE.LineBasicMaterial({ color: <span style="color:#00d;font-weight:bold">0xffffff</span> });
<span style="color:#080;font-weight:bold">var</span> wireframe = <span style="color:#080;font-weight:bold">new</span> THREE.LineSegments(geo, mat);
cube.add(wireframe);
<span style="color:#080;font-weight:bold">return</span> cube;
}
</code></pre></div><p>Use the helper methods in the receiver section to create the cube and display it.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#888">// Define variables
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">const</span> receiverRenderer = createRenderer(<span style="color:#d20;background-color:#fff0f0">'receiver-wrapper'</span>);
<span style="color:#080;font-weight:bold">const</span> receiverScene = createScene();
<span style="color:#080;font-weight:bold">const</span> receiverCamera = createCamera();
<span style="color:#080;font-weight:bold">const</span> receiverCube = createCube();
receiverScene.add(receiverCube);
<span style="color:#080;font-weight:bold">let</span> receiverControls = <span style="color:#080;font-weight:bold">new</span> OrbitControls(receiverCamera, receiverRenderer.domElement);
receiverControls.enabled = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">function</span> animate() {
requestAnimationFrame(animate);
receiverControls.update();
receiverRenderer.render(receiverScene, receiverCamera);
}
animate();
</code></pre></div><p>Listen to the <code>ReceiveData</code> event to update the receiver section cube with updated camera position data.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, <span style="color:#080;font-weight:bold">function</span> (cameraData) {
receiverCamera.position.x = cameraData.x;
receiverCamera.position.y = cameraData.y;
receiverCamera.position.z = cameraData.z;
});
</code></pre></div><p>Add a <code>createController</code> method which is called when a user is granted control of the cube. It creates a cube and renders on the controller section and listens to the camera position change.</p>
<p>When the cube is rotated, it calls the server method <code>SendData</code> using the SignalR connection. It also returns a method that cancels the cube renderer and removes the listener from the controller cube. This returned method is called when the <code>ControlReleased</code> event is called from server.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">function</span> createController() {
<span style="color:#080;font-weight:bold">const</span> controllerRenderer = createRenderer(<span style="color:#d20;background-color:#fff0f0">'controller-wrapper'</span>);
<span style="color:#080;font-weight:bold">const</span> controllerScene = createScene();
<span style="color:#080;font-weight:bold">const</span> controllerCamera = createCamera();
<span style="color:#080;font-weight:bold">const</span> controllerCube = createCube();
controllerScene.add(controllerCube);
<span style="color:#080;font-weight:bold">let</span> controls = <span style="color:#080;font-weight:bold">new</span> OrbitControls(controllerCamera, controllerRenderer.domElement);
<span style="color:#080;font-weight:bold">function</span> onPositionChange(o) {
controlHubConnection.invoke(<span style="color:#d20;background-color:#fff0f0">"SendData"</span>, controllerCamera.position).<span style="color:#080;font-weight:bold">catch</span>(<span style="color:#080;font-weight:bold">function</span> (err) {
<span style="color:#080;font-weight:bold">return</span> console.error(err.toString());
});
}
controls.addEventListener(<span style="color:#d20;background-color:#fff0f0">'change'</span>, onPositionChange);
<span style="color:#080;font-weight:bold">let</span> request;
<span style="color:#080;font-weight:bold">function</span> controllerAnimate() {
request = requestAnimationFrame(controllerAnimate);
controls.update();
controllerRenderer.render(controllerScene, controllerCamera);
}
controllerAnimate();
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">function</span> () {
<span style="color:#038">document</span>.getElementById(<span style="color:#d20;background-color:#fff0f0">'controller-wrapper'</span>).innerHTML = <span style="color:#d20;background-color:#fff0f0">''</span>;
cancelAnimationFrame(request);
controls.removeEventListener(<span style="color:#d20;background-color:#fff0f0">'change'</span>, onPositionChange);
}
}
</code></pre></div><h3 id="wrapping-up">Wrapping Up</h3>
<p>We learned the basics of SignalR and how to build a simple application to communicate between the backend and frontend using a SignalR connection. We also looked into a practical implementation on how can we use SignalR to control a graphical object built with three.js. We can extend the application to build apps like a multi-player game, or to send messages to the frontend from long-running backend background jobs.</p>
<p>You can get the code at <a href="https://github.com/bimalghartimagar/EPSignalRControl">GitHub</a>.</p>
End Point meetup in Ljubljana, Sloveniahttps://www.endpointdev.com/blog/2023/12/end-point-ljubljana-meetup/2023-12-29T00:00:00+00:00Jon Jensen
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/20230418-161011.webp" alt="5 men smiling for the camera outside against classic pillar and walls"></p>
<!-- Photos by Jon Jensen except IMG_9314.webp which is by Nicholas Piano -->
<h3 id="remote-first-work">Remote-first work</h3>
<p>End Point has always been a primarily remote work company. We have offices in Manhattan, New York City and Johnson City, Tennessee, yet the people who work in them mostly interact online with remote colleagues.</p>
<p>Most of the time that is wonderful because we have no commute if we work at home, or as short of a commute as we want for those who work in a nearby office or co-working space. We are also able to move without changing jobs. And our meetings with clients are usually remote in any case.</p>
<h3 id="international-interaction">International interaction</h3>
<p>Perhaps best of all, working remote-first has allowed us to work together with colleagues around the world: currently in 13 countries and as many time zones. That is pretty good international representation for a company of 70 people.</p>
<p>A downside of mostly remote work is that we generally have to make an effort to ever meet each other in person. And while remote meetings are convenient, they lack some impact that in-person meetings have. There is especially a big difference between meeting remotely with someone you have never met, and someone you’ve met in person at least once!</p>
<p>So we try to meet up with our co-workers when we are at all “in the neighborhood” of each other. That can happen when more than one of us attends a conference or work on a VisionPort installation, when we meet with a client, when we have meetings at one of our offices, or when one or more of us happens to be traveling and proposes meeting with others in the area.</p>
<p>In 2023 alone I am aware of such meetings among our staff that happened in:</p>
<ul>
<li>Malaysia</li>
<li>Spain</li>
<li>Belgium</li>
<li>Slovenia</li>
<li>New York</li>
<li>Tennessee</li>
<li>Missouri</li>
<li>Indiana</li>
<li>Utah</li>
<li>Washington (state)</li>
<li>California</li>
</ul>
<p>And there are probably a few others I didn’t hear about or forgot!</p>
<h3 id="ljubljana-slovenia">Ljubljana, Slovenia</h3>
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/20230419-143046.webp" alt="A sunny view overlooking a European city with classic and modern architecture, with mountains in the background"></p>
<p>Here I want to share some photos of a brief meet-up four of us had in Ljubljana, the capital of Slovenia, back in April. I was traveling there while on vacation with my daughter Mira, so I asked around among my co-workers who lived nearby, where “nearby” meant within a few hours' travel.</p>
<p>Our crew was Couragyn Chretien from Canada and Nicholas Piano from England, both living in Spain; Marco Pessotto from Italy, living in Croatia; and I was on vacation from my current base in Utah.</p>
<p>None of us had been to Ljubljana before and we all were eager to see it. Couragyn and Nicholas flew, Marco drove, and I took a train from Zagreb, Croatia, the previous stop on my itinerary.</p>
<p>We booked vacation rental apartments that were near each other and close to the river that runs through the city, nestled among town squares, pedestrian areas, and restaurants.</p>
<p>We had dinner together the evening we all arrived, and breakfast in the morning, then met in one of the apartments. The four of us had recently not overlapped very much in End Point projects we were involved with, so we began by having a few hours of “show and tell” to go over several projects we have been working on for clients.</p>
<p>Here is Nicholas giving us an application demonstration:</p>
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/20230418-104650.webp" alt="Man in an apartment looks toward large TV screen mounted on a wall, showing a computer display of a website"></p>
<p>As we are prone to do, we dug into some of our current technical challenges to try to help each other out of problems. That was actually really helpful because it forced us to move beyond high-level descriptions and app demos to implementation and troubleshooting. I don’t remember that we solved any big problem this time, but so it goes.</p>
<p>We next talked about an internal development project that may yet see the light of day.</p>
<p>We also had an impromptu review of our main website’s Google Analytics reports to reflect on the effects of our domain move from endpoint.com to endpointdev.com almost two years prior. Visitor traffic and broken links are some of the many domain move considerations and most of us were not deeply involved in that process so it was good to spread some of the knowledge learned.</p>
<p>We wrapped up with a timed selfie:</p>
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/IMG_9314.webp" alt="4 men standing in a living room smiling for the camera"></p>
<p>Then before dinner we were treated to a walking tour of Ljubljana by our local friend Šime Kodžoman, who I met about a decade earlier through his brother Jure and the European Perl and Interchange open source software developer communities.</p>
<p>Šime knows a ton about the city and we spent an enjoyable afternoon and evening together as we walked 13–14 km ≈ 8–9 miles and saw as much as we could.</p>
<p>A few highlights among many:</p>
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/20230418-162739.webp" alt="4 men walking left across a city square in front of buildings with modern architecture"></p>
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/20230418-143618.webp" alt="Two brutalist concrete high-rise buildings tower over a concrete square"></p>
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/20230418-145303.webp" alt="Colorful 4-story buildings with trees along concrete city walls containing a river reflecting a cloudscape"></p>
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/20230418-145712.webp" alt="A colorful European city square with clocktower and church"></p>
<p><img src="/blog/2023/12/end-point-ljubljana-meetup/20230419-142136.webp" alt="A sunny view of a castle wall and tower with flags flying in the breeze, against a blue sky with white clouds"></p>
<p>If you are interested in learning more about Ljubljana, there are plenty of good resources to read online. Start with <a href="https://www.ljubljana.si/en/">ljubljana.si</a>, <a href="https://www.visitljubljana.com/en/visitors">visitljubljana.com</a>, and <a href="https://en.wikipedia.org/wiki/Ljubljana">Wikipedia’s Ljubljana entry</a>.</p>
<p>We wrapped up our day with another meal together, and the next morning we headed our separate ways.</p>
<p>It was great to meet each other in person, to see Šime again, and to get a feel for a beautiful city!</p>
Client Profile: J.G. Title Companyhttps://www.endpointdev.com/blog/2023/12/client-profile-j-g-title-company/2023-12-27T00:00:00+00:00Juan Pablo Ventoso
<p><img src="/blog/2023/12/client-profile-j-g-title-company/j-g-title-company.webp" alt="The J.G. Title Company logo sits in the center of an image on a blue background with futuristic designs reminiscent of circuit boards. Text on either side of the logo reads “Nationwide titling”, with text under reading “info@jgtitleco.com” and “https://jgtitleco.com”"></p>
<p><a href="https://jgtitleco.com/">J.G. Title</a> is a prominent service company specializing in automotive dealership titling and registration processes across the country. They simplify operations for their dealers, businesses, and individuals by giving tax/fee quotes, document validation, and checklists for all jurisdictions.</p>
<p>Jordan Kivett, the owner of J.G. Title, envisioned an innovative web application to simplify dealership operations. He contacted us to request a quote for developing this solution and making his vision a reality. This project became a great opportunity for our <a href="/team/">.NET team</a> to build a robust system with state-of-the-art technologies functioning seamlessly together.</p>
<h3 id="the-solution">The solution</h3>
<p>J.G. Title submitted a detailed document containing a description of the requirements for the J.G. Title Suite app along with workflow diagrams explaining their current business processes. After analyzing all the documents and having some initial meetings, we decided to divide the project into different stages, and estimate each phase of work separately. We ended up with three phases:</p>
<ul>
<li>Phase I: Create a product that allows users to enter deals into the system and retrieve the quote estimation, including taxes, fees, and charges for services across 50 states.</li>
<li>Phase II: Implement several UI improvements such as a detailed dealership dashboard, financial management, and integration with <a href="https://quickbooks.intuit.com/">QuickBooks</a>.</li>
<li>Phase III: Add an ambitious PDF document management section featuring document recognition, analysis, and manipulation, to perform automatic validations on the document’s contents and automatically generate PDF outputs from the deal’s details.</li>
</ul>
<h3 id="tech-stack">Tech stack</h3>
<ul>
<li><a href="https://dotnet.microsoft.com/en-us/learn/dotnet/what-is-dotnet">.NET</a> 6 with <a href="https://learn.microsoft.com/en-us/dotnet/csharp/tour-of-csharp/">C#</a></li>
<li><a href="https://www.postgresql.org/">PostgreSQL</a> 14</li>
<li><a href="https://rockylinux.org/">Rocky Linux</a> 9 with <a href="https://www.nginx.com/">Nginx</a></li>
</ul>
<p>We had already worked with this combination of technologies for several clients, and we are confident in the combined power in a robust database such as Postgres along with .NET 6 running in a Linux environment. It has proven to be a stable, fast, and reliable setup for our web solutions.</p>
<p>We also used <a href="https://trello.com/">Trello</a> for tracking the project’s progress and tasks, <a href="https://github.com/">GitHub</a> for version control, and <a href="https://visualstudio.microsoft.com/vs/">Visual Studio 2022</a> or <a href="https://code.visualstudio.com/">VS Code</a> with <a href="https://learn.microsoft.com/en-us/dotnet/core/tools/">.NET CLI</a> as a coding environment. Some of us also used <a href="https://code.visualstudio.com/docs/devcontainers/tutorial">VS Code dev containers</a>. <a href="https://www.pgadmin.org/">pgAdmin</a> is our usual tool to manage the local Postgres database, as well as <a href="https://www.postman.com/">Postman</a> to test our API endpoints.</p>
<h3 id="the-team">The team</h3>
<p>Most of our .NET team is involved on this project. We also receive valuable help from our hosting team and our Postgres developers.</p>
<ul>
<li><a href="/team/bimal-gharti-magar/">Bimal Gharti Magar</a></li>
<li><a href="/team/dan-briones/">Dan Briones</a></li>
<li><a href="/team/dylan-wooters/">Dylan Wooters</a></li>
<li><a href="/team/juan-pablo-ventoso/">Juan Pablo Ventoso</a></li>
<li><a href="/team/kevin-campusano/">Kevin Campusano</a></li>
<li><a href="/team/mike-delange/">Mike DeLange</a></li>
</ul>
<p>The client is also highly involved in all aspects of the development process, from requirements gathering to documenting, testing, and providing feedback. We have bi-weekly standup calls, and the entire team (End Point + J.G. Title) interacts actively through Trello, working together and updating each task as the work progresses.</p>
<h3 id="results">Results</h3>
<p>We began working on the project on October 25, 2022, and completed Phase I within the expected timeline, on June 7, 2023. The next phases were launched iteratively, with new deployments usually scheduled twice a week.</p>
<p>The application is now widely used by dealerships and the J.G. Title team. They have over 200 users registered in the system, with more than 8,000 deals quoted, and new deals being added at an increasing rate.</p>
<p><img src="/blog/2023/12/client-profile-j-g-title-company/j-g-title-suite-featured.webp" alt="On the right side is an image of a laptop and phone, each running the J.G. Title Suite application. Text on the left reads “From chaos to clarity. Simplify, streamline, succeed: Embrace clarity in your dealership’s tax and title operations. Our dedicated team of specialists conducted extensive research and established partnerships with local DMVs and state Treasurers to become the trusted partner for automotive dealerships and businesses nationwide, offering efficient and reliable title and registration services. We continue transforming automotive title management on a national scale and welcome you to experience the future of automotive titling.""><br>
The J.G. Title Suite application, featured on the company’s website.</p>
<p>We are continuing work on several tasks related to Phase III of the project. As the application keeps growing and users send feedback, new phases of work will surely be added to the project in the future.</p>
<p>This partnership is a testament to our shared vision for efficiency and innovation, and we’re excited to continue reshaping the industry with J.G. Title Company!</p>
Developers: Time and Teamworkhttps://www.endpointdev.com/blog/2023/12/developers-time-teamwork/2023-12-04T00:00:00+00:00Greg Hanson
<p><img src="/blog/2023/12/developers-time-teamwork/utah-exposed-rock-mountain.webp" alt="A steep golden hillside interspersed with green shrubs, behind power lines. Above the hill is a half moon on a deep blue sky."></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>Time and teamwork. How do these two concepts fit together? The focus of this post is how blending them can improve not only your programming skills, but also your timeline of projects.</p>
<p>For a recent project I am the project manager of, we had an urgent request from a client for a developer who could be available in the next 2 weeks to work on a project that had about a 6 week timeline.</p>
<h3 id="the-project">The project</h3>
<p>Initially, it seemed like a pretty easy request to fill. We had a developer who had worked previously for this client, in the very same environment, albeit on a different project. So the roadmap was there: Get the developer into the systems, get the requirements, and do the work.</p>
<p>This client, like many, had some specific barriers to entry, like needing a company email address, access through a VPN, Duo 2FA, familiarity with Monday.com tracking, Office 365, Microsoft Teams for meetings, and so forth. But, as mentioned, the dev we would be using had worked previously in this environment, and so we already had most of the above in place.</p>
<p>So we got started with the first few initiation meetings, getting the basics of the project laid out, tasks assigned, and timeline established.</p>
<p>Then <a href="https://en.wikipedia.org/wiki/Murphy%27s_law">Mr. Murphy</a> showed up as he often does, and the dev we had planned to use got pulled out of the project for an urgent need elsewhere. Okay, project manager, what do we do now?</p>
<h3 id="time">Time</h3>
<p>Well, we dig into the pool of developers we have, and see what we can do. Here is the first recognition of <em>time</em>. We are already about a week into this thing, and now we are starting over. So time is not currently our friend.</p>
<h3 id="teamwork">Teamwork</h3>
<p>I did, however, have two developers who had worked together on many other projects. Luckily, these two developers had also worked with the first developer who was now leaving the project. So we immediately had a few <em>transfer of information</em> meetings.</p>
<h3 id="time-and-teamwork">Time and teamwork</h3>
<p>We had a lot of specific questions about what had been requested, and where it all lived. Because these two had worked together for so long, the time it took to get the two new developers up to speed was greatly reduced.</p>
<p>They were aware of each other’s strengths and weaknesses. As I thought about how much these developers’ rapport sped things up, I realized that this was a direct effect of <em>time</em> and <em>teamwork</em>. Also, because of the <em>time</em> spent together, they trusted each other. Respect was present instead of uncertainty, and exchanges were filled with value.</p>
<p>As most of you who are reading this know, programming can be performed in a number of different ways. You can be a lone wolf, you can be a tag team, or you can be a cog in a very large machine. There are places where each of those approaches is the most efficient.</p>
<p>In most cases when you have a pair or a small group, <em>two minds are better than one</em>. If you add <em>time</em> in as the multiplier so to speak, you end up with another dimension to that. These two developers have worked together for quite some <em>time</em>, and that is what enabled them to move so quickly in an unfamiliar environment. As the project progressed, these two sounded off to each other, and as a result many issues were solved internally, rather than ending up on the client’s board. <em>Time</em> spent waiting for client responses was minimized.</p>
<h3 id="client-satisfaction">Client satisfaction</h3>
<p>That doesn’t seem like a big deal, and the client may not have noticed that there were fewer clarification issues than on other projects. However, this <em>did</em> make a difference, and we were able to recover all of the time lost due to the unexpected loss of the original developer, even shaving off a little time on the overall project.</p>
<p>The result was a very happy client. Remember, this client found out a week into their project that the developer we had assigned to the project was now unable to continue. So there was quite a swing from losing a developer to finishing the project slightly ahead of schedule. Our client will remember that.</p>
<p>So I guess the moral of the story is something we probably all know inherently: Time contributes to the effectiveness of teamwork. The longer the relationship lasts, the higher the respect and trust level is between the team members. Knowing this, we can try to keep compatible developers together as much as is practically possible. Mix and match, and try to grow this trust over time. Build <em>teams</em> and come back to them, <em>time</em> after <em>time</em>.</p>
Domain Move Considerationshttps://www.endpointdev.com/blog/2023/11/domain-move-considerations/2023-11-30T00:00:00+00:00Josh Ausborne
<p><img src="/blog/2023/11/domain-move-considerations/mormon-row-and-tetons.webp" alt="Photo taken of and from the Mormon Row Homestead at Grand Teton National Park. Photo is taken facing WNW, and has an old home and barn in the foreground with the jagged, rocky peaks of the Teton mountain range in the background."></p>
<!-- Photo by Josh Ausborne, 2019 -->
<p>In 2021 <a href="/blog/2021/10/moving-to-endpointdev-dot-com/">we moved</a> to a new internet domain. We had been on endpoint.com for 26 years, so a lot of things were tied to that domain!</p>
<p>As we expected, there were a lot of little details to deal with. We found the switch to be a bit overwhelming until we started listing things out, at which time we realized that a divide-and-conquer approach would make it achievable.</p>
<p>A domain move is not an extremely common experience for a company to go through, but it’s not unheard of, either, due to acquisitions, mergers, or rebranding like ours. So we want to share our notes from our move to endpointdev.com in case they are helpful to others considering their own move.</p>
<p>How long did we reserve for the move, in calendar time? We planned to work on it over 6 months, but in the end we were done in about 3 months.</p>
<h3 id="make-a-schedule">Make a schedule</h3>
<p>Look at your calendar. Mark any major company or personal events that you do not want any infrastructure disruption around. Block out busy periods, major holidays and vacations, etc. This will help you be realistic about how much time various people can spend on this, and how you can minimize problems for the company by choosing when to break things, or at least risk breaking them.</p>
<p>Start by noting your timeline for starting and your drop-dead end date to be off the old domain.</p>
<p>Is this something that you want or need to do all at one time, with a massive cutover of all services? Or do you expect to move services piecemeal? As we have helped clients move domains, we have done both types of moves.</p>
<p>Plan to divide tasks among the team to preserve sanity and enlist the experience of many people instead of just a few. That shares important knowledge among more people, which is good in any case.</p>
<p>As you settle on a schedule, make sure everybody involved is aware of it. Post it in a shared document all the relevant people have access to, and put important milestones on a shared calendar.</p>
<h3 id="internal-messaging">Internal messaging</h3>
<p>Messaging internally is just as important as messaging externally.</p>
<p>Your own staff may think that the domain has moved everything when it has only moved some. Consider making an internally available status page.</p>
<h3 id="new-domain">New domain</h3>
<p>As soon as possible, get the new domain registered and start using it as a secondary domain. You want to minimize the length of time you use a placeholder name, or have to add the qualifier “or whatever domain we get” every time you mention it. That will also help you see whether you find it nice to say all the time, how often it is misunderstood in speaking, how easy it is to type, etc. If you’re going to have second thoughts about the new domain you’ve chosen, have them early while you can still choose a different one and see if it works better.</p>
<h3 id="identify-all-services">Identify all services</h3>
<p>It’s important to brainstorm with your transition team to identify the list of services that need to be updated, as you could lose the ability to do so after transitioning away from the old domain. These services may include email, websites, ticketing systems, and proof of ownership of logins to SaaS (software as a service) accounts, among other things.</p>
<p>As you start building your list, you may be surprised to find out how many different services and providers you are using. This is a good time to start closing any accounts that you don’t use anymore, as well.</p>
<p>Examples of some services that companies use:</p>
<ul>
<li>Website hosting</li>
<li>Domain registrar accounts and contacts</li>
<li>Google Workspace (G Suite), Microsoft Office 365, or other email services</li>
<li>GitHub (Note that you will lose the GitHub contribution stats associated with the old email address if you delete it from your account.)</li>
<li>GitLab</li>
<li>Bitbucket</li>
<li>Slack</li>
<li>Microsoft Teams</li>
<li>Zoom</li>
<li>Skype</li>
<li>Atlassian (Jira, Trello)</li>
<li>Asana</li>
<li>Monday.com</li>
<li>Redmine</li>
<li>PagerDuty</li>
<li>Adobe Creative Cloud</li>
<li>Windows, Apple iCloud accounts</li>
<li>Shippers such as USPS, FedEx, UPS, DHL</li>
<li>Vendor purchasing accounts</li>
<li>Website and server monitoring services</li>
<li>Payment accounts such as company credit cards, Apple, PayPal, Google Pay</li>
<li>Vendor account logins and contacts (Dell, Microsoft, Apple)</li>
<li>Payroll service</li>
<li>Retirement savings such as 401(k)</li>
<li>Office rental invoicing and payments</li>
<li>Office services such as water/food delivery</li>
</ul>
<h3 id="staff-self-service">Staff self-service</h3>
<p>Have your staff update their own accounts on their own for as many as possible of the services that they use.</p>
<p>Some sites will allow you to add a secondary address, promote it to primary, then delete the old one.</p>
<h3 id="infrastructure">Infrastructure</h3>
<p>Make a list of every hostname in your DNS zone, whether it’s public-facing or internal, where it is hosted (SaaS, a cloud service, on-premises), and how its hostname can be changed.</p>
<p>You should create new DNS entries in the new domain for all of your hosts well in advance of the cutover date. When reviewing your DNS entries, it is as good a time as any to review your existing entries and to do a bit of spring cleaning. We found numerous entries that were no longer needed, and we were able to purge them rather than migrate them into the new domain.</p>
<p>Update reverse DNS entries for all hosts.</p>
<h3 id="email">Email</h3>
<p>Email is the lifeblood of business nowadays. It is how businesses interact with each other: invoicing, payments, sales requests, notifications, reminders, business partner requests, account login resets, etc. For most businesses everything would grind to a halt if email stops working.</p>
<ul>
<li>Monitor outbound mail relay to see if any internal systems are sending notifications to old domain</li>
<li>Notify outsiders (customers, clients, business partners) of new domain</li>
<li>Notify them well in advance of the shutdown date</li>
<li>Some infrequently contacted clients might continue to use the old email address for months, and if the new owner of the domain doesn’t have email set up there, or if it doesn’t send “bounce” messages back when delivery fails, senders may have no idea you’re not receiving their mail.</li>
</ul>
<h3 id="websites">Websites</h3>
<ul>
<li>Change website URLs, email address, redirects</li>
<li>Main website</li>
<li>Blog (posts and comments)</li>
<li>Google Analytics: use domain move function. SEO is crucial for many businesses.</li>
</ul>
<h3 id="social-media">Social media</h3>
<p>Update your company social media accounts on Twitter, LinkedIn, Facebook, Instagram, YouTube, etc. If you move too quickly then you can get ahead of any marketing efforts that you may be making, but if you move too slowly then you might find yourself locked out of accounts or miss out on opportunities to take advantage of those same marketing efforts.</p>
<h3 id="lessons-learned">Lessons learned</h3>
<p>Use a separate infrastructure domain for cloud services. This means that your servers might be located in a different domain than your main marketing domain. For End Point, we use endpointdev.com as our marketing domain, but our infrastructure servers are known by names in epinfra.net. This, in theory, would allow us to easily change our marketing domain yet again (though this is <em>not</em> something that is on the horizon!) without having to go and change the domain in which our servers reside. Even if you never move to a new domain, putting your infrastructure on a separate domain now is a good idea, to separate marketing and public-facing services from internal hostnames.</p>
<p>Consider typing convenience for infrastructure names! epinfra.net is way easier to type correctly than endpoint.com was, and much better than the longer endpointdev.com. You’ll be typing this domain name a <em>lot</em>, so it’s worth taking a few minutes to find one that is easy to type.</p>
<p>Everything you can move earlier, which doesn’t affect the public, do so. The less you have to do up against a deadline, the better.</p>
<p>Pay attention to Single Sign On (SSO) accounts. Since we use Google Workspace with our company domain, we had some staff who used Google’s SSO for services such as Trello. When the users logged into the account using the new domain name, Trello/Atlassian automatically created a new account for them at the new domain name, and users completely lost access to their old Trello accounts that were using the old domain. This caused some issues later on because we had to go back through all Trello boards and cards, add all the new users, then remove all of the old accounts, now dead and never to be used again, from everywhere. This was very tedious.</p>
<p>Something that we’ve found two years after The Great Migration is that the old domain still seems to live on in configuration files in various places. We recently found that we still had a virtual host configured on one of our web servers that was listening for requests for a host in the endpoint.com domain. We also found that many of our custom scripts and configurations reference endpoint.com in their comments. I’m never surprised when I stumble across the old domain in some obscure place. In fact, I kind of laugh about it now, as if I just found an Easter egg.</p>
<h3 id="conclusion">Conclusion</h3>
<p>When you finally decide to make your transition to a new domain, be sure to make lists and plan things out. Migrating can be a challenging task, but it’s doable if you are careful and methodical in your work.</p>
JSforce: A Quick Path to Salesforce Developmenthttps://www.endpointdev.com/blog/2023/10/jsforce-quick-path-to-salesforce-development/2023-10-21T00:00:00+00:00Couragyn Chretien
<p><img src="/blog/2023/10/jsforce-quick-path-to-salesforce-development/desert-sky.webp" alt="A completely clear blue sky is broken by a desert mountain with exposed light rock, covered partly by striking green trees and bushes. Above the mountain is a half-moon."></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>Using JavaScript with JSforce can get you working on a Salesforce project quickly if you don’t have a Salesforce expert on hand. It provides easy access to Salesforce’s API, which will allow you to focus on development instead of learning a new system.</p>
<h3 id="no-salesforce-learning-curve">No Salesforce learning curve</h3>
<p>Apex is a platform-specific language created so that developers can interact with Salesforce classes/objects and write custom code. Apex allows you to do some cool things such as directly triggering custom Apex code based on an action in Salesforce.</p>
<p>The problem with Apex is that it is its own world, with its own IDEs, deployment processes, etc. There’s a steep learning curve to getting up to speed with the Apex ecosphere.</p>
<p>JSforce is a wrapper/abstraction of the Salesforce API. It allows you to do a lot, like search, perform CRUD actions, and even send emails. These functions aren’t as streamlined as their built-in Apex counterpart, but JSforce allows any JS developer to jump right into the code without wasting costly training time.</p>
<h3 id="using-jsforce-cli">Using JSforce CLI</h3>
<p>Below are some examples of connecting and performing basic CRUD operations.</p>
<h4 id="connecting">Connecting</h4>
<p>Installation:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ npm install jsforce -g
</code></pre></div><p>Connection:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ jsforce
> login('user@example.org', 'password123');
{ id: '00550000000vwsFAAQ',
organizationId: '00D500000006xKGEAY',
url: 'https://login.salesforce.com/id/00D500000006xKGEAY/00550000000vwsFAAQ' }
>
</code></pre></div><h4 id="get">GET</h4>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-js" data-lang="js">conn.sobject(<span style="color:#d20;background-color:#fff0f0">"Account"</span>).retrieve(<span style="color:#d20;background-color:#fff0f0">"0017000000hOMChAAO"</span>, <span style="color:#080;font-weight:bold">function</span>(err, account) {
<span style="color:#080;font-weight:bold">if</span> (err) { <span style="color:#080;font-weight:bold">return</span> console.error(err); }
console.log(<span style="color:#d20;background-color:#fff0f0">"Name : "</span> + account.Name);
<span style="color:#888">// ...
</span><span style="color:#888"></span>});
</code></pre></div><h4 id="post">POST</h4>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-js" data-lang="js">conn.sobject(<span style="color:#d20;background-color:#fff0f0">"Account"</span>).create({ Name : <span style="color:#d20;background-color:#fff0f0">'My Account #1'</span> }, <span style="color:#080;font-weight:bold">function</span>(err, ret) {
<span style="color:#080;font-weight:bold">if</span> (err || !ret.success) { <span style="color:#080;font-weight:bold">return</span> console.error(err, ret); }
console.log(<span style="color:#d20;background-color:#fff0f0">"Created record id : "</span> + ret.id);
<span style="color:#888">// ...
</span><span style="color:#888"></span>});
</code></pre></div><h4 id="put">PUT</h4>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-js" data-lang="js">conn.sobject(<span style="color:#d20;background-color:#fff0f0">"Account"</span>).update({
Id : <span style="color:#d20;background-color:#fff0f0">'0017000000hOMChAAO'</span>,
Name : <span style="color:#d20;background-color:#fff0f0">'Updated Account #1'</span>
}, <span style="color:#080;font-weight:bold">function</span>(err, ret) {
<span style="color:#080;font-weight:bold">if</span> (err || !ret.success) { <span style="color:#080;font-weight:bold">return</span> console.error(err, ret); }
console.log(<span style="color:#d20;background-color:#fff0f0">'Updated Successfully : '</span> + ret.id);
<span style="color:#888">// ...
</span><span style="color:#888"></span>});
</code></pre></div><h4 id="delete">DELETE</h4>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-js" data-lang="js">conn.sobject(<span style="color:#d20;background-color:#fff0f0">"Account"</span>).destroy(<span style="color:#d20;background-color:#fff0f0">'0017000000hOMChAAO'</span>, <span style="color:#080;font-weight:bold">function</span>(err, ret) {
<span style="color:#080;font-weight:bold">if</span> (err || !ret.success) { <span style="color:#080;font-weight:bold">return</span> console.error(err, ret); }
console.log(<span style="color:#d20;background-color:#fff0f0">'Deleted Successfully : '</span> + ret.id);
});
</code></pre></div><p>For a deeper dive into setup and uses of JSforce check out <a href="/blog/2020/03/salesforce-integration-with-node/">this post</a> by my coworker Dylan Wooters.</p>
Building Ecommerce Search Using Algoliahttps://www.endpointdev.com/blog/2023/10/building-ecommerce-search-using-algolia/2023-10-12T00:00:00+00:00Dylan Wooters
<p><img src="/blog/2023/10/building-ecommerce-search-using-algolia/east-bay-hills.webp" alt="Looking east from the top of the Berkeley hills over the Briones Reservoir. Rolling hills are seen in the distance with the sun setting to the west."></p>
<!-- Photo by Dylan Wooters, 2020 -->
<p>A common request that developers receive when embarking on a new website project is for the website to have “Google-like search.” For many years, this meant writing custom code to replicate the intelligent and user-friendly aspects of Google search, which was no easy feat. However, now we have many search-as-a-service offerings that do the hard work for us and make this process much easier.</p>
<p>In this blog post, we’ll dive into one of these search-as-a-service platforms, <a href="https://www.algolia.com/">Algolia</a>. We recently worked on an ecommerce website and used Algolia in an interesting way, both as a search engine and as a lightweight backend database to hold product data managed in Salesforce. Algolia worked beautifully, offering users fast and accurate search results, and also allowing us to launch the site within a relatively short time frame.</p>
<p>We will look at how to load Algolia with data, configure search options, and connect the search to the frontend using Algolia’s Vue library.</p>
<h3 id="loading-the-index-with-data">Loading the index with data</h3>
<p>To start using Algolia’s search, you need to load up an index with data. You have the option of manually uploading a JSON file, or using Algolia’s API to programmatically load records. For our backend, we chose to use Algolia’s <a href="https://www.npmjs.com/package/algoliasearch">JavaScript API client</a> in some lightweight TypeScript scripts that are triggered by cron. These scripts allowed us to sync inventory data between Salesforce and the index in Algolia.</p>
<p>Using the Algolia JavaScript client is quite simple. Regardless of where your data comes from—be it in a database, a platform like Salesforce, or elsewhere—once it is in JSON format, you can load it into Algolia with a few lines of code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">import</span> * as algolia from <span style="color:#d20;background-color:#fff0f0">"algoliasearch"</span>;
<span style="color:#080;font-weight:bold">const</span> products = [{
name: <span style="color:#d20;background-color:#fff0f0">"Fender F-5 Acoustic"</span>,
make: <span style="color:#d20;background-color:#fff0f0">"Fender"</span>,
model: <span style="color:#d20;background-color:#fff0f0">"F-5"</span>,
category: <span style="color:#d20;background-color:#fff0f0">"Guitars"</span>,
status: <span style="color:#d20;background-color:#fff0f0">"Used"</span>,
objectID: <span style="color:#d20;background-color:#fff0f0">"Fender-001"</span>
}, {
name: <span style="color:#d20;background-color:#fff0f0">"Fender Player Jaguar"</span>,
make: <span style="color:#d20;background-color:#fff0f0">"Fender"</span>,
model: <span style="color:#d20;background-color:#fff0f0">"Jaguar (Player)"</span>,
category: <span style="color:#d20;background-color:#fff0f0">"Guitars"</span>,
status: <span style="color:#d20;background-color:#fff0f0">"New"</span>,
objectID: <span style="color:#d20;background-color:#fff0f0">"Fender-002"</span>
}];
<span style="color:#080;font-weight:bold">const</span> index = algolia.<span style="color:#080;font-weight:bold">default</span>(process.env.ALGOLIA_APP_ID, process.env.ALGOLIA_API_KEY).initIndex(<span style="color:#d20;background-color:#fff0f0">"store_products"</span>);
<span style="color:#080;font-weight:bold">await</span> index.saveObjects(products);
</code></pre></div><p>Note the <code>objectID</code> property, which is used by Algolia as a primary key. If the <code>objectID</code> does not exist, a new record will be created. If it does exist, the record will be updated. This makes it easy to run a data sync process using a single <code>saveObjects</code> command, without having to worry about differentiating between create and update operations.</p>
<h3 id="configuring-the-index">Configuring the index</h3>
<p>Once you have your index loaded, you’ll want to configure it. Algolia does a good job of walking you through this process using a built-in tutorial when you first load your index. Basically, you will be selecting the <a href="https://www.algolia.com/doc/guides/sending-and-managing-data/prepare-your-data/how-to/setting-searchable-attributes/">searchable properties/attributes</a> from your JSON data, setting how results are ranked and sorted, and adjusting more advanced aspects of search like typo tolerance, stop words, etc.</p>
<p>An important feature we utilized on our recent project is <a href="https://www.algolia.com/doc/guides/managing-results/refine-results/faceting/">faceting</a>. Faceting allows users to easily drill down and refine search by categories, and is also easy to develop using the handy frontend libraries that Algolia provides (more on that in the next section). This feature is powerful and can be used to both refine search and drive homepage category/subcategory links. When you configure your index, you can select which attributes of your data should be used for faceting.</p>
<h3 id="setting-up-search-on-the-frontend">Setting up search on the frontend</h3>
<p>We used <a href="https://nuxt.com/">Nuxt</a> to build the frontend of the website, and we leveraged Algolia’s <a href="https://www.algolia.com/doc/guides/building-search-ui/what-is-instantsearch/vue/">Vue InstantSearch</a> library for the UI. This library really speeds along development, as it wraps all of the search-related functionality in simple widgets, providing the search bar, results, refinements, filtering, and more.</p>
<p>The <a href="https://www.algolia.com/doc/api-reference/widgets/instantsearch/vue/"><code>ais-instant-search</code></a> widget is the parent widget. It serves the search state to its children, which allows you to show the search bar, search hits, hierarchical menus, etc. Here is a simple example of the <code>ais-instant-search</code> widget with a search bar and hits (pulled directly from <a href="https://www.algolia.com/doc/guides/building-search-ui/getting-started/vue/">Algolia’s Vue docs</a>):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">template</span>>
<<span style="color:#b06;font-weight:bold">ais-instant-search</span> <span style="color:#369">:search-client</span>=<span style="color:#d20;background-color:#fff0f0">"searchClient"</span> <span style="color:#369">index-name</span>=<span style="color:#d20;background-color:#fff0f0">"demo_ecommerce"</span>>
<<span style="color:#b06;font-weight:bold">ais-search-box</span> />
<<span style="color:#b06;font-weight:bold">ais-hits</span>>
<<span style="color:#b06;font-weight:bold">template</span> <span style="color:#369">v-slot:item</span>=<span style="color:#d20;background-color:#fff0f0">"{ item }"</span>>
<<span style="color:#b06;font-weight:bold">h2</span>>{{ item.name }}</<span style="color:#b06;font-weight:bold">h2</span>>
</<span style="color:#b06;font-weight:bold">template</span>>
</<span style="color:#b06;font-weight:bold">ais-hits</span>>
</<span style="color:#b06;font-weight:bold">ais-instant-search</span>>
</<span style="color:#b06;font-weight:bold">template</span>>
<<span style="color:#b06;font-weight:bold">script</span>>
<span style="color:#080;font-weight:bold">import</span> algoliasearch from <span style="color:#d20;background-color:#fff0f0">'algoliasearch/lite'</span>;
<span style="color:#080;font-weight:bold">import</span> <span style="color:#d20;background-color:#fff0f0">'instantsearch.css/themes/algolia-min.css'</span>;
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> {
data() {
<span style="color:#080;font-weight:bold">return</span> {
searchClient: algoliasearch(
<span style="color:#d20;background-color:#fff0f0">'[Your app ID]'</span>,
<span style="color:#d20;background-color:#fff0f0">'[Your API key]'</span>
),
};
},
};
</<span style="color:#b06;font-weight:bold">script</span>>
<<span style="color:#b06;font-weight:bold">style</span>>
<span style="color:#b06;font-weight:bold">body</span> {
<span style="color:#080;font-weight:bold">font-family</span>: <span style="color:#080;font-weight:bold">sans-serif</span>;
<span style="color:#080;font-weight:bold">padding</span>: <span style="color:#00d;font-weight:bold">1</span><span style="color:#888;font-weight:bold">em</span>;
}
</<span style="color:#b06;font-weight:bold">style</span>>
</code></pre></div><h3 id="using-faceting-for-search-refinement-and-filtering">Using faceting for search refinement and filtering</h3>
<p>I mentioned faceting above when discussing how to configure your index. Once you have selected the attributes in your JSON data that can be used for faceting (e.g., category, subcategory), you can feed those attributes to the <a href="https://www.algolia.com/doc/api-reference/widgets/hierarchical-menu/vue/"><code>ais-hierarchical-menu</code></a> widget for display on the frontend.</p>
<p>Here is a bit of sample code from the website we built, which offers expandable category refinement via <code>ais-hierarchical-menu</code>.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"sidebar-segment"</span>>
<<span style="color:#b06;font-weight:bold">p</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"sidebar-segment-title"</span>>Category</<span style="color:#b06;font-weight:bold">p</span>>
<<span style="color:#b06;font-weight:bold">ais-hierarchical-menu</span>
<span style="color:#369">:limit</span>=<span style="color:#d20;background-color:#fff0f0">"100"</span>
<span style="color:#369">:attributes</span>=<span style="color:#d20;background-color:#fff0f0">"categoryAttrs"</span>
<span style="color:#369">:sort-by</span>=<span style="color:#d20;background-color:#fff0f0">"hierarchicalMenuSort"</span>
>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">slot-scope</span>=<span style="color:#d20;background-color:#fff0f0">"{ items, refine, createURL }"</span>>
<<span style="color:#b06;font-weight:bold">hierarchical-menu-list</span>
<span style="color:#369">:items</span>=<span style="color:#d20;background-color:#fff0f0">"items"</span>
<span style="color:#369">:refine</span>=<span style="color:#d20;background-color:#fff0f0">"refine"</span>
<span style="color:#369">:create-url</span>=<span style="color:#d20;background-color:#fff0f0">"createURL"</span>
/>
</<span style="color:#b06;font-weight:bold">div</span>>
</<span style="color:#b06;font-weight:bold">ais-hierarchical-menu</span>>
</<span style="color:#b06;font-weight:bold">div</span>>
</code></pre></div><p>Above, we are using the <code>limit</code> property to set the maximum number of items to 100. The <code>attributes</code> property, which targets the JSON attributes in your index data that represent your categories, is set to the following:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">categoryAttrs: [<span style="color:#d20;background-color:#fff0f0">'categories.lvl0'</span>, <span style="color:#d20;background-color:#fff0f0">'categories.lvl1'</span>, <span style="color:#d20;background-color:#fff0f0">'categories.lvl2'</span>],
</code></pre></div><p>These represent the three levels of categories in our data (zero-based), and are used to build the hierarchical menu.</p>
<p>Finally, the <code>sort-by</code> attribute points to a simple function that uses the JavaScript <code>localeCompare</code> method to provide alphanumeric sorting:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">function</span> hierarchicalMenuSort(a, b) {
<span style="color:#080;font-weight:bold">return</span> a.name.localeCompare(b.name, <span style="color:#080;font-weight:bold">undefined</span>, {
numeric: <span style="color:#080;font-weight:bold">true</span>,
sensitivity: <span style="color:#d20;background-color:#fff0f0">'base'</span>,
})
}
</code></pre></div><h3 id="wrapping-up-seeing-algolia-in-action">Wrapping up: seeing Algolia in action</h3>
<p>If you’d like to see Algolia in action on the site that we built, head over to <a href="https://www.eiffeltrading.com/">eiffeltrading.com</a>. If you search the site, you’ll see autocomplete, fast results (thanks to both Algolia and Nuxt), and other aspects of good search that we have all come to expect from modern ecommerce sites.</p>
<p>Next time you are faced with a build involving full-text search, consider search-as-a-service offerings like Algolia. They could save you time and headaches over rolling your own search functionality. Sometimes it’s good to let others do the hard work!</p>
<p>Have questions or feedback on the topic? Let us know in the comments section below.</p>
Introduction to Playwright using C#https://www.endpointdev.com/blog/2023/10/introduction-to-playwright-in-csharp/2023-10-12T00:00:00+00:00Bimal Gharti Magar
<p><img src="/blog/2023/10/introduction-to-playwright-in-csharp/sun-on-green-leaves.webp" alt="In the foreground, the foliage of a light green tree edged with sunlight creates a line across the image. In the background, a mountain hillside with light dirt, intermittent exposed rock, and foliage just starting to turn the yellow, orange, and red colors of fall"></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>Automating web application tasks is a necessary skill for software developers and testers. You need it for performing repetitive tasks, conducting end-to-end testing of web applications, and scraping data from websites. In this blog post, we’ll explore how to use C# with Playwright for automating tasks.</p>
<h3 id="what-is-playwright">What is Playwright?</h3>
<p>Playwright is an open-source automation framework developed by Microsoft that allows you to automate web applications using various programming languages, including C#. Playwright was created specifically to accommodate the needs of end-to-end testing, but we will use it as a library for web automation. Playwright supports Chromium, WebKit, and Firefox. It runs on a variety of systems: Windows, Linux, and macOS, locally or on a continuous integration (CI) platform, headless or headed with native mobile emulation.</p>
<p>To begin web automation using C# and Playwright, follow these steps:</p>
<ol>
<li>
<p><strong>Prerequisites:</strong> You should have .NET Core or .NET 5+ installed on your machine.</p>
</li>
<li>
<p><strong>Create a New C# Project:</strong> Let’s start by creating a new C# console application.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet new console -n EPWebAutomation
cd EPWebAutomation
</code></pre></div></li>
<li>
<p><strong>Install Playwright NuGet Package:</strong> Open new project in Visual Studio or any preferred C# IDE and install the <code>Microsoft.Playwright</code> NuGet package and build the project using the following commands:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet add package Microsoft.Playwright
dotnet build
</code></pre></div></li>
<li>
<p><strong>Install required browsers:</strong> Run the following command to install the required browsers for Playwright:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">pwsh bin/Debug/netX/playwright.ps1 install
# If the pwsh command does not work (throws TypeNotFound), make sure to use an up-to-date version of PowerShell.
dotnet tool update --global PowerShell
</code></pre></div></li>
<li>
<p><strong>Initialize Playwright Instance:</strong> Initialize Playwright in your C# code by creating a new browser instance and take a screenshot in Chromium:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Playwright</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">var</span> playwright = <span style="color:#080;font-weight:bold">await</span> Playwright.CreateAsync();
<span style="color:#080;font-weight:bold">await</span> <span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">var</span> browser = <span style="color:#080;font-weight:bold">await</span> playwright.Chromium.LaunchAsync();
<span style="color:#888;font-weight:bold">var</span> page = <span style="color:#080;font-weight:bold">await</span> browser.NewPageAsync();
</code></pre></div></li>
<li>
<p><strong>Navigate and Interact:</strong> You can now navigate to a web page using <code>GotoAsync</code> and interact with it using Playwright’s APIs. For instance, to navigate to a website and take a screenshot, you can use the below code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">await</span> page.GotoAsync(<span style="color:#d20;background-color:#fff0f0">"https://playwright.dev/dotnet"</span>);
<span style="color:#080;font-weight:bold">await</span> page.ScreenshotAsync(<span style="color:#080;font-weight:bold">new</span> PageScreenshotOptions { Path = <span style="color:#d20;background-color:#fff0f0">"screenshot.png"</span> });
</code></pre></div></li>
<li>
<p><strong>Locators:</strong> Playwright allows you to locate elements on the page using ID, CSS selectors, or XPaths. It also provides inbuilt <a href="https://playwright.dev/dotnet/docs/locators#quick-guide">locators</a>.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">await</span> page.GotoAsync(<span style="color:#d20;background-color:#fff0f0">"https://computer-database.gatling.io/computers"</span>);
<span style="color:#888">// fill dell in search text field using id selector
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">await</span> page.Locator(<span style="color:#d20;background-color:#fff0f0">"#searchbox"</span>).FillAsync(<span style="color:#d20;background-color:#fff0f0">"dell"</span>);
<span style="color:#888">// click on Filter by name button using id selector
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">await</span> page.Locator(<span style="color:#d20;background-color:#fff0f0">"#searchsubmit"</span>).ClickAsync();
<span style="color:#888">// click on a button using inbuild GetByText selector
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">await</span> page.GetByText(<span style="color:#d20;background-color:#fff0f0">"Add a new computer"</span>).ClickAsync();
</code></pre></div></li>
<li>
<p><strong>Fill form, select dropdown value and click elements:</strong> Playwright allows you to perform actions like filling out forms, selecting dropdown value and clicking elements. Here’s an example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">await</span> page.FillAsync(<span style="color:#d20;background-color:#fff0f0">"#name"</span>, <span style="color:#d20;background-color:#fff0f0">"asus abcde"</span>);
<span style="color:#080;font-weight:bold">await</span> page.SelectOptionAsync(<span style="color:#d20;background-color:#fff0f0">"#company"</span>, <span style="color:#080;font-weight:bold">new</span>[] { <span style="color:#080;font-weight:bold">new</span> SelectOptionValue() { Label = <span style="color:#d20;background-color:#fff0f0">"ASUS"</span> } });
<span style="color:#080;font-weight:bold">await</span> page.Locator(<span style="color:#d20;background-color:#fff0f0">"input[type=submit]"</span>).ClickAsync();
</code></pre></div></li>
<li>
<p><strong>Cleanup:</strong> Don’t forget to close the browser when you’re done:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">await</span> browser.CloseAsync();
</code></pre></div></li>
</ol>
<h3 id="advantages-of-using-c-and-playwright">Advantages of Using C# and Playwright</h3>
<p>Some of the useful features of Playwright with C# include:</p>
<ol>
<li><strong>Cross-Browser and cross-platform support:</strong> As mentioned above, Playwright works on modern rendering engines including Chromium, WebKit, and Firefox. You also have the option of running it locally or on CI, and of running headless or headed.</li>
<li><strong>C# Language Familiarity:</strong> The Playwright API can be used with C#. If you’re already familiar with C#, this reduces the learning curve for web automation.</li>
<li><strong>Robust Documentation:</strong> Playwright’s documentation and community support make it easier to troubleshoot issues and find solutions.</li>
<li><strong>Headless and Headful Browsing:</strong> Playwright allows you to choose between headless and headed mode. Debugging issues in headed mode can be much easier, since we can see what’s going on in every line of code.</li>
<li><strong>Powerful APIs:</strong> With Playwright’s APIs, you can perform complex web automation tasks with ease.</li>
<li><strong>Powerful Tooling:</strong> Playwright has tooling like <a href="https://playwright.dev/dotnet/docs/codegen">Codegen</a>, <a href="https://playwright.dev/dotnet/docs/debug#playwright-inspector">Playwright inspector</a>, and <a href="https://playwright.dev/dotnet/docs/trace-viewer-intro">Trace Viewer</a>. They can help make using Playwright easier.</li>
<li><strong>Emulation:</strong> You can test your app on any browser or emulate a real device such as mobile phone or tablet. It can emulate behavior such as <code>userAgent</code>, <code>geolocation</code>, <code>viewport</code>, and many more. You can read more about it in the <a href="https://playwright.dev/dotnet/docs/emulation">emulation</a> documentation.</li>
</ol>
<p>The powerful combination of C# and Playwright makes it a preferred tool for automating web tasks. When there are repetitive tasks or browser testing involved, Playwright can help you with getting the tasks done.</p>
Testing user-defined functions with pgTAPhttps://www.endpointdev.com/blog/2023/09/testing-user-defined-functions-with-pgtap/2023-09-25T00:00:00+00:00Todor “Theo” Dimov
<p><img src="/blog/2023/09/testing-user-defined-functions-with-pgtap/market-movies.webp" alt="An overhead shot of a market full of thousands of colorful DVD cases. Several people peruse the wares in an aisle"></p>
<!-- Image public domain (CC0). Retrieved from https://pxhere.com/en/photo/426847 -->
<p>For a great introduction to the pgTAP protocol, please read over my colleague <a href="/team/josh-tolley/">Josh Tolley’s</a> article, <a href="/blog/2022/03/using-pgtap-automate-database-testing/">using pgTAP to automate database testing</a>. Also check out <a href="/team/edgar-mlowe/">Edgar Mlowe’s</a> article on <a href="/blog/2023/04/pgtap-for-database-unit-tests/">how to set up pgTAP for writing PostgreSQL database unit tests</a>.</p>
<p>As a protocol, pgTAP provides a great list of functions and commands to easily and clearly test all aspects of a database. As I’ve been tasked with more implementations of unit tests for user-defined functions, I thought it helpful to share some useful methods.</p>
<p>First we’ll go over a few basic pgTAP functions that are useful in testing the existence of functions and procedures. Then we’ll use the PostgreSQL port of the Sakila sample database for MySQL, called Pagila. We’ll construct two functions and go over a single test case. Using basic Postgres tools, I’ll walk through a few methods of calling and testing our functions. Finally, we’ll go over a more concise way of testing all cases for said functions.</p>
<p>Feel free to work alongside this article to hopefully leave with a comfortable idea of how to construct unit test files for your database.</p>
<h3 id="functions">Functions:</h3>
<p>For this first example, remember to have the pgTAP extension in your schema and to perform it in a transaction (I’ve omitted <code>BEGIN</code> and <code>ROLLBACK</code> for brevity). Also use <code>SELECT plan(<number of tests you’ll run>)</code> when running the pgTAP functions.</p>
<p>Starting off, let’s create a basic addition function that returns an integer value:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">CREATE</span> <span style="color:#080;font-weight:bold">OR</span> <span style="color:#080;font-weight:bold">REPLACE</span> <span style="color:#080;font-weight:bold">FUNCTION</span> add_numbers(
num1 <span style="color:#038">integer</span>,
num2 <span style="color:#038">integer</span>
)
<span style="color:#080;font-weight:bold">RETURNS</span> <span style="color:#038">integer</span>
<span style="color:#080;font-weight:bold">LANGUAGE</span> plpgsql <span style="color:#080;font-weight:bold">AS</span> <span style="color:#d20;background-color:#fff0f0">$$
</span><span style="color:#d20;background-color:#fff0f0">BEGIN
</span><span style="color:#d20;background-color:#fff0f0"> RETURN num1 + num2;
</span><span style="color:#d20;background-color:#fff0f0">END;
</span><span style="color:#d20;background-color:#fff0f0">$$</span>;
</code></pre></div><p>To test this user-defined function, a few of the most relevant functions in pgTAP we can use are:</p>
<h4 id="has_function"><code>has_function()</code></h4>
<p>Variations:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">SELECT</span> has_function( <span style="color:#369">:schema</span>, <span style="color:#369">:function</span>, <span style="color:#369">:args</span>, <span style="color:#369">:description</span> );
<span style="color:#080;font-weight:bold">SELECT</span> has_function( <span style="color:#369">:schema</span>, <span style="color:#369">:function</span>, <span style="color:#369">:args</span> );
<span style="color:#080;font-weight:bold">SELECT</span> has_function( <span style="color:#369">:schema</span>, <span style="color:#369">:function</span>, <span style="color:#369">:description</span> );
<span style="color:#080;font-weight:bold">SELECT</span> has_function( <span style="color:#369">:schema</span>, <span style="color:#369">:function</span> );
<span style="color:#080;font-weight:bold">SELECT</span> has_function( <span style="color:#369">:function</span>, <span style="color:#369">:args</span>, <span style="color:#369">:description</span> );
<span style="color:#080;font-weight:bold">SELECT</span> has_function( <span style="color:#369">:function</span>, <span style="color:#369">:args</span> );
<span style="color:#080;font-weight:bold">SELECT</span> has_function( <span style="color:#369">:function</span>, <span style="color:#369">:description</span> );
<span style="color:#080;font-weight:bold">SELECT</span> has_function( <span style="color:#369">:function</span> );
</code></pre></div><p>Parameters:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">:schema
Name of a schema in which to find the function.
:function
Name of a function.
:args
Array of data types of the function arguments.
:description
A short description of the test.
</code></pre></div><p>Example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">SELECT</span> has_function(<span style="color:#d20;background-color:#fff0f0">'your_schema'</span>, <span style="color:#d20;background-color:#fff0f0">'add_numbers'</span>, <span style="color:#080;font-weight:bold">ARRAY</span>[<span style="color:#d20;background-color:#fff0f0">'integer'</span>,<span style="color:#d20;background-color:#fff0f0">'integer'</span>]); <span style="color:#888">-- default description used
</span></code></pre></div><p>Results:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">ok 1 - Function add_numbers(integer, integer) should exist
</code></pre></div><p><code>has_function()</code> is useful in checking the existence of a specific function. If you want to parse an entire schema, <code>functions_are()</code> is your command.</p>
<h4 id="functions_are"><code>functions_are()</code></h4>
<p>Variations:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">SELECT</span> functions_are( <span style="color:#369">:schema</span>, <span style="color:#369">:functions</span>, <span style="color:#369">:description</span> );
<span style="color:#080;font-weight:bold">SELECT</span> functions_are( <span style="color:#369">:schema</span>, <span style="color:#369">:functions</span> );
<span style="color:#080;font-weight:bold">SELECT</span> functions_are( <span style="color:#369">:functions</span>, <span style="color:#369">:description</span> );
<span style="color:#080;font-weight:bold">SELECT</span> functions_are( <span style="color:#369">:functions</span> );
</code></pre></div><p>Parameters:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">:schema
Name of a schema in which to find functions.
:functions
An array of function names.
:description
A short description of the test.
</code></pre></div><p>Example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">SELECT</span> functions_are(<span style="color:#d20;background-color:#fff0f0">'your_schema'</span>, <span style="color:#080;font-weight:bold">ARRAY</span>[<span style="color:#d20;background-color:#fff0f0">'add_numbers'</span>, <<span style="color:#080;font-weight:bold">All</span> other <span style="color:#080;font-weight:bold">functions</span> need <span style="color:#080;font-weight:bold">to</span> be listed <span style="color:#080;font-weight:bold">as</span> well>]); <span style="color:#888">-- default description used
</span></code></pre></div><h4 id="ok-and-is"><code>ok()</code> and <code>is()</code></h4>
<p><code>ok()</code> and <code>is()</code> have similar functionality with slightly different parameters:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">SELECT</span> ok( <span style="color:#369">:boolean</span>, <span style="color:#369">:description</span> );
</code></pre></div><p>Example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">SELECT</span> ok(add_numbers(<span style="color:#00d;font-weight:bold">5</span>,<span style="color:#00d;font-weight:bold">5</span>)=<span style="color:#00d;font-weight:bold">10</span>, <span style="color:#d20;background-color:#fff0f0">'add_numbers correctly works'</span>);
</code></pre></div><p>Parameters:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">:boolean
A boolean value indicating success or failure.
:description
A short description of the test.
</code></pre></div><p>While <code>ok()</code> is used for passing a test returning a boolean, <code>is()</code> takes two values to compare:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">SELECT</span> <span style="color:#080;font-weight:bold">is</span>( <span style="color:#369">:have</span>, <span style="color:#369">:want</span>, <span style="color:#369">:description</span> );
</code></pre></div><p>Example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">SELECT</span> <span style="color:#080;font-weight:bold">is</span>(add_numbers(<span style="color:#00d;font-weight:bold">5</span>,<span style="color:#00d;font-weight:bold">5</span>), <span style="color:#00d;font-weight:bold">10</span>, <span style="color:#d20;background-color:#fff0f0">'add_numbers correctly works'</span>);
</code></pre></div><p>You can find functions that cover pretty much anything in PostgreSQL here in <a href="https://pgtap.org/documentation.html">pgTAP’s documentation</a>.</p>
<h3 id="testing-more-complex-user-defined-stored-functions">Testing more complex user-defined stored functions:</h3>
<p>A client product we created utilizes a function that calls several other functions depending on its given argument. This function is called by the application and is passed an <code>integer</code> as an argument from which the function bases an initial <code>SELECT</code> query’s <code>WHERE</code> condition on, in this case <code>WHERE id = [argument]</code>. Variables are then assigned column values from this queried table. These variables are then used as the conditions to <code>PERFORM</code> functions that change parts of the database.</p>
<p>When testing this sort of scenario, it became apparent that we needed a set of test data, as well as the ability to track that test data, run the function with the data, and test to see if the changes are performed correctly.</p>
<p>The nice thing about pgTAP is that unit tests are meant to be rolled back leaving the database as you found it. When faced with a more complex function that nests other functions which loop through multiple tables, it may make more sense to fill those tables temporarily with the data required for the tests.</p>
<p>Let’s take a nostalgia trip and imagine we’re operating a DVD rental store. The Pagila sample database, which you can download on GitHub, represents our scenario. It comes populated and we’ll construct a function which utilizes other functions to perform changes on multiple tables.</p>
<p>Say the DVD rental shop gives extensions of rentals for special occasions, one being that a DVD was rented on Christmas week. Suppose we have an initial function that checks if a new DVD is rented on one of these special occasions. It then calls another function that will extend the return date, based on the specific occasion:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">CREATE</span> <span style="color:#080;font-weight:bold">OR</span> <span style="color:#080;font-weight:bold">REPLACE</span> <span style="color:#080;font-weight:bold">FUNCTION</span> extension_check(_rental_id <span style="color:#038">integer</span>)
<span style="color:#080;font-weight:bold">RETURNS</span> <span style="color:#038">void</span>
<span style="color:#080;font-weight:bold">LANGUAGE</span> plpgsql
<span style="color:#080;font-weight:bold">AS</span> <span style="color:#d20;background-color:#fff0f0">$function$
</span><span style="color:#d20;background-color:#fff0f0">DECLARE
</span><span style="color:#d20;background-color:#fff0f0"> _rental_date date;
</span><span style="color:#d20;background-color:#fff0f0"> _inventory_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _customer_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _return_date date;
</span><span style="color:#d20;background-color:#fff0f0"> christmas_week date;
</span><span style="color:#d20;background-color:#fff0f0">BEGIN
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> SELECT INTO _rental_date
</span><span style="color:#d20;background-color:#fff0f0"> rental_date FROM rental
</span><span style="color:#d20;background-color:#fff0f0"> WHERE rental_id = _rental_id;
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> IF EXTRACT(YEAR FROM _rental_date) = EXTRACT(YEAR FROM CURRENT_DATE) AND
</span><span style="color:#d20;background-color:#fff0f0"> _rental_date <= (EXTRACT(YEAR FROM CURRENT_DATE)::text || '-12-25')::date AND
</span><span style="color:#d20;background-color:#fff0f0"> _rental_date > (EXTRACT(YEAR FROM CURRENT_DATE)::text || '-12-25')::date - INTERVAL '7 days' THEN
</span><span style="color:#d20;background-color:#fff0f0"> PERFORM christmas_extension(_rental_id);
</span><span style="color:#d20;background-color:#fff0f0"> END IF;
</span><span style="color:#d20;background-color:#fff0f0">END;
</span><span style="color:#d20;background-color:#fff0f0">$function$</span>;
<span style="color:#080;font-weight:bold">CREATE</span> <span style="color:#080;font-weight:bold">OR</span> <span style="color:#080;font-weight:bold">REPLACE</span> <span style="color:#080;font-weight:bold">FUNCTION</span> christmas_extension(_rental_id <span style="color:#038">integer</span>)
<span style="color:#080;font-weight:bold">RETURNS</span> <span style="color:#038">void</span>
<span style="color:#080;font-weight:bold">LANGUAGE</span> plpgsql
<span style="color:#080;font-weight:bold">AS</span> <span style="color:#d20;background-color:#fff0f0">$function$
</span><span style="color:#d20;background-color:#fff0f0">DECLARE
</span><span style="color:#d20;background-color:#fff0f0"> _rental_date date;
</span><span style="color:#d20;background-color:#fff0f0"> _return_date date;
</span><span style="color:#d20;background-color:#fff0f0">BEGIN
</span><span style="color:#d20;background-color:#fff0f0"> SELECT return_date FROM rental WHERE rental_id = _rental_id INTO _return_date;
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> IF _return_date IS NULL THEN
</span><span style="color:#d20;background-color:#fff0f0"> UPDATE rental SET return_date = rental_date + INTERVAL '10 days' WHERE rental_id=_rental_id;
</span><span style="color:#d20;background-color:#fff0f0"> ELSE
</span><span style="color:#d20;background-color:#fff0f0"> UPDATE rental SET return_date = return_date + INTERVAL '10 days' WHERE rental_id=_rental_id;
</span><span style="color:#d20;background-color:#fff0f0"> END IF;
</span><span style="color:#d20;background-color:#fff0f0">END;
</span><span style="color:#d20;background-color:#fff0f0">$function$</span>;
</code></pre></div><p>The initial function, <code>extension_check</code>, simply checks if the rental was checked out on the week prior to and on Christmas (It’s a really dedicated DVD shop). We can also assume a function like this has other scenarios where it would extend rental dates but we’ll just show one for this exercise; <code>christmas_extension</code> is our one extension function that extends the return date by ten days.</p>
<p>To hold our unit test, we’ll create a file fittingly named extension_unit_tests.sql. A problem with the current sample database is that we can’t really test out the function
directly since the dates don’t extend to the current year. Rather even if they did, we don’t want to directly affect the existing data especially if the dates were updated and the
database was in production use.</p>
<p>Relatively basic but handy Postgres tools let us create test cases which we can pass to the functions. Afterwards, we can use pgTAP functions to confirm that our user-defined functions <code>extension_check</code> and <code>christmas_extension</code> worked properly.</p>
<p>For test cases in our new file, extension_unit_tests.sql, we’ll need a way to create and track them. For this, we’ll use a <code>temporary table</code> to track the test IDs and a <code>PROCEDURE</code> which can be called at any point to create the test cases. After that, we’ll simply call the procedure and run the user-defined function <code>extension_check</code>. Finally test its existence, test its result, and roll back our changes.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">BEGIN</span>;
<span style="color:#080;font-weight:bold">CREATE</span> <span style="color:#080;font-weight:bold">TEMPORARY</span> <span style="color:#080;font-weight:bold">TABLE</span> test_instance_references (
test_rental_id <span style="color:#038">integer</span> <span style="color:#080;font-weight:bold">PRIMARY</span> <span style="color:#080;font-weight:bold">KEY</span>,
test_number <span style="color:#038">integer</span>
);
<span style="color:#080;font-weight:bold">CREATE</span> <span style="color:#080;font-weight:bold">OR</span> <span style="color:#080;font-weight:bold">REPLACE</span> <span style="color:#080;font-weight:bold">PROCEDURE</span> fill_test_data (
_rental_date <span style="color:#038">date</span>,
_test_number <span style="color:#038">integer</span>
)
<span style="color:#080;font-weight:bold">LANGUAGE</span> PLPGSQL
<span style="color:#080;font-weight:bold">AS</span> <span style="color:#d20;background-color:#fff0f0">$procedure$
</span><span style="color:#d20;background-color:#fff0f0">DECLARE
</span><span style="color:#d20;background-color:#fff0f0"> _address_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _store_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _staff_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _inventory_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _customer_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _rental_id integer;
</span><span style="color:#d20;background-color:#fff0f0">BEGIN
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> SELECT nextval('customer_customer_id_seq1'::regclass) INTO _customer_id;
</span><span style="color:#d20;background-color:#fff0f0"> SELECT nextval('rental_rental_id_seq'::regclass) INTO _rental_id;
</span><span style="color:#d20;background-color:#fff0f0"> SELECT nextval('inventory_inventory_id_seq'::regclass) INTO _inventory_id;
</span><span style="color:#d20;background-color:#fff0f0"> SELECT store_id FROM store LIMIT 1 INTO _store_id; -- any store will do
</span><span style="color:#d20;background-color:#fff0f0"> SELECT address_id FROM address LIMIT 1 INTO _address_id; -- any address will do
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> INSERT INTO customer (customer_id, store_id, first_name, last_name, address_id)
</span><span style="color:#d20;background-color:#fff0f0"> VALUES (
</span><span style="color:#d20;background-color:#fff0f0"> _customer_id,
</span><span style="color:#d20;background-color:#fff0f0"> _store_id,
</span><span style="color:#d20;background-color:#fff0f0"> 'Test First Name',
</span><span style="color:#d20;background-color:#fff0f0"> 'Test Last Name',
</span><span style="color:#d20;background-color:#fff0f0"> _address_id
</span><span style="color:#d20;background-color:#fff0f0"> );
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> INSERT INTO inventory (inventory_id, film_id, store_id)
</span><span style="color:#d20;background-color:#fff0f0"> VALUES (
</span><span style="color:#d20;background-color:#fff0f0"> _inventory_id,
</span><span style="color:#d20;background-color:#fff0f0"> (SELECT film_id FROM film LIMIT 1), -- arbitrary film
</span><span style="color:#d20;background-color:#fff0f0"> _store_id
</span><span style="color:#d20;background-color:#fff0f0"> );
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> INSERT INTO rental (rental_id, rental_date, inventory_id, customer_id, staff_id)
</span><span style="color:#d20;background-color:#fff0f0"> VALUES (
</span><span style="color:#d20;background-color:#fff0f0"> _rental_id,
</span><span style="color:#d20;background-color:#fff0f0"> _rental_date,
</span><span style="color:#d20;background-color:#fff0f0"> _inventory_id,
</span><span style="color:#d20;background-color:#fff0f0"> _customer_id,
</span><span style="color:#d20;background-color:#fff0f0"> (SELECT staff_id FROM staff WHERE store_id = _store_id LIMIT 1) -- select arbitrary staff member from store
</span><span style="color:#d20;background-color:#fff0f0"> );
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> INSERT INTO test_instance_references (test_rental_id, test_number)
</span><span style="color:#d20;background-color:#fff0f0"> values (
</span><span style="color:#d20;background-color:#fff0f0"> _rental_id,
</span><span style="color:#d20;background-color:#fff0f0"> _test_number
</span><span style="color:#d20;background-color:#fff0f0"> );
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0">END;
</span><span style="color:#d20;background-color:#fff0f0">$procedure$</span>;
CALL fill_test_data(<span style="color:#d20;background-color:#fff0f0">'2023-12-25'</span>::<span style="color:#038">date</span>, <span style="color:#00d;font-weight:bold">1</span>); <span style="color:#888">-- Fill the db with our test data
</span><span style="color:#888"></span>
<span style="color:#080;font-weight:bold">SELECT</span> extension_check((<span style="color:#080;font-weight:bold">SELECT</span> test_rental_id <span style="color:#080;font-weight:bold">FROM</span> test_instance_references <span style="color:#080;font-weight:bold">WHERE</span> test_number = <span style="color:#00d;font-weight:bold">1</span>)); <span style="color:#888">-- Call our user-defined function
</span><span style="color:#888"></span>
<span style="color:#080;font-weight:bold">SELECT</span> plan(<span style="color:#00d;font-weight:bold">2</span>); <span style="color:#888">-- State the number of tests we'll be performing
</span><span style="color:#888"></span>
<span style="color:#080;font-weight:bold">SELECT</span> has_function(
<span style="color:#d20;background-color:#fff0f0">'extension_check'</span>,
<span style="color:#080;font-weight:bold">ARRAY</span>[<span style="color:#d20;background-color:#fff0f0">'int'</span>]
);
<span style="color:#080;font-weight:bold">SELECT</span> <span style="color:#080;font-weight:bold">is</span>(
(<span style="color:#080;font-weight:bold">SELECT</span> return_date <span style="color:#080;font-weight:bold">FROM</span> rental <span style="color:#080;font-weight:bold">WHERE</span> rental_id <span style="color:#080;font-weight:bold">in</span> (<span style="color:#080;font-weight:bold">SELECT</span> test_rental_id <span style="color:#080;font-weight:bold">FROM</span> test_instance_references <span style="color:#080;font-weight:bold">WHERE</span> test_number = <span style="color:#00d;font-weight:bold">1</span>))::<span style="color:#038">date</span>,
(<span style="color:#d20;background-color:#fff0f0">'2023-12-25'</span>::<span style="color:#038">date</span> + <span style="color:#038">INTERVAL</span> <span style="color:#d20;background-color:#fff0f0">'10 DAYS'</span>)::<span style="color:#038">date</span>,
<span style="color:#d20;background-color:#fff0f0">'Checking test data 1, week of Christmas'</span>
);
<span style="color:#080;font-weight:bold">ROLLBACK</span>;
</code></pre></div><p>By design, pgTAP tests are performed in a transaction block with a <code>ROLLBACK</code> at the end to reverse all changes. Our first transaction is the creation of a temporary table called <code>test_instance_references</code> which will hold the IDs of the sample data we insert specifically in the rental table. This way we keep track of our specific test cases through their primary identifier.</p>
<p>Our PROCEDURE works similarly to a function, but where a function may have issues being called in a transaction block through pgTAP’s command line pg_prove, the use of a procedure and its CALL command is far more reliable.</p>
<p>In our example, the procedure simply adds in all the required test data. Note the insertion of test data into other tables that are referenced in <code>rental</code>, as well as an insert into the temporary table for tracking. To avoid using any existing data in the database, references are also test samples, like for the customer. Some, however, are selected from existing tables. It’s important to make sure you’re not affecting real data in the database in case a transaction fails. The example is obviously lax when it comes to that, specifically with the references to <code>store</code> and <code>address</code>, so please take note in your own unit tests to be careful when referring to existing data.</p>
<p>After we’ve established our procedure, we’re ready to run the tests. The pgTAP function <code>plan()</code> defines how many unit tests we are going to run. For the first example we run two tests.</p>
<p>Before running tests, we call the procedure <code>fill_test_data</code> which populates the database with test data and the temporary table to track that data.</p>
<p>Our first test is to check if the function exists. pgTAP’s <code>has_function</code> function is perfect for this.</p>
<p>Our procedure is then called with a date for the test rental. This test rental’s ID is saved into the temporary <code>test_instance_references</code> table which is then passed to the main function we want to test, <code>extension_check</code>. We call this function with the SELECT command, and immediately test the outcome with pgTAP’s <code>is()</code>. Our <code>:have</code> parameter holds the <code>return_date</code> of our test rental which is tested to see if it’s been correctly extended by 10 days. Our final parameter is the description which will be returned in the outcome of pg_prove if the test fails.</p>
<p>The <code>ROLLBACK</code> is meant to clean the database of all test data.</p>
<p>Let’s run this unit test on the command line, with pg_prove:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">pg_prove -d <your_database> <path_to_your_unit_test>/extension_unit_tests.sql
</code></pre></div><p>We should see these results:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">extension_unit_tests.sql .. ok
All tests successful.
Files=1, Tests=2, 0 wallclock secs ( 0.01 usr + 0.00 sys = 0.01 CPU)
Result: PASS
</code></pre></div><p>Congrats! We’ve tested out a single case for a function.</p>
<p>However, what if we want to test all possible cases for this function to be sure it performs correctly?</p>
<p>To do this, we can alter the procedure <code>fill_test_data</code> to loop through a date range given in the arguments <code>_start_date</code> and <code>_end_date</code>. For our example we’ll provide it with the week of Christmas, but it’ll work for any date range provided the rental store gives extensions for DVDs rented on other dates.</p>
<p>Instead of passing a test number as an argument, we can simply initialize it as a variable that’s incremented with every loop. Inside the <code>WHILE</code> loop, every iteration will populate the database with the test data, perform <code>extension_check</code> with the test rental ID, and increment the date and test number.</p>
<p>We’ll end the transaction a little differently by setting our pgTAP plan equal to the number of tests we have in our temporary table, to avoid hard-coding our plan count. We’ll also add one to check its existence with <code>has_function</code>.</p>
<p>Finally, our <code>is()</code> will also iterate through our temporary table and perform a check in the rental table to confirm that the return date has been correctly extended by ten days.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-postgresql" data-lang="postgresql"><span style="color:#080;font-weight:bold">BEGIN</span>;
<span style="color:#080;font-weight:bold">CREATE</span> <span style="color:#080;font-weight:bold">TEMPORARY</span> <span style="color:#080;font-weight:bold">TABLE</span> test_instance_references (
test_rental_id <span style="color:#038">integer</span> <span style="color:#080;font-weight:bold">PRIMARY</span> <span style="color:#080;font-weight:bold">KEY</span>,
test_number <span style="color:#038">integer</span>,
start_date <span style="color:#038">date</span>
);
<span style="color:#080;font-weight:bold">CREATE</span> <span style="color:#080;font-weight:bold">OR</span> <span style="color:#080;font-weight:bold">REPLACE</span> <span style="color:#080;font-weight:bold">PROCEDURE</span> fill_test_data (
_start_date <span style="color:#038">date</span>,
_end_date <span style="color:#038">date</span>
)
<span style="color:#080;font-weight:bold">LANGUAGE</span> PLPGSQL
<span style="color:#080;font-weight:bold">AS</span> <span style="color:#d20;background-color:#fff0f0">$procedure$
</span><span style="color:#d20;background-color:#fff0f0"> DECLARE
</span><span style="color:#d20;background-color:#fff0f0"> _address_id smallint;
</span><span style="color:#d20;background-color:#fff0f0"> _store_id smallint;
</span><span style="color:#d20;background-color:#fff0f0"> _inventory_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _customer_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _rental_id integer;
</span><span style="color:#d20;background-color:#fff0f0"> _test_number integer;
</span><span style="color:#d20;background-color:#fff0f0"> BEGIN
</span><span style="color:#d20;background-color:#fff0f0"> _test_number := 1;
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> WHILE _start_date <= _end_date LOOP
</span><span style="color:#d20;background-color:#fff0f0"> SELECT nextval('customer_customer_id_seq1'::regclass) INTO _customer_id;
</span><span style="color:#d20;background-color:#fff0f0"> SELECT nextval('rental_rental_id_seq'::regclass) INTO _rental_id;
</span><span style="color:#d20;background-color:#fff0f0"> SELECT nextval('inventory_inventory_id_seq'::regclass) INTO _inventory_id;
</span><span style="color:#d20;background-color:#fff0f0"> SELECT store_id FROM store LIMIT 1 INTO _store_id; -- any store will do
</span><span style="color:#d20;background-color:#fff0f0"> SELECT address_id FROM address LIMIT 1 INTO _address_id; -- any address will do
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> INSERT INTO customer (customer_id, store_id, first_name, last_name, address_id)
</span><span style="color:#d20;background-color:#fff0f0"> VALUES (
</span><span style="color:#d20;background-color:#fff0f0"> _customer_id,
</span><span style="color:#d20;background-color:#fff0f0"> _store_id,
</span><span style="color:#d20;background-color:#fff0f0"> 'Test First Name',
</span><span style="color:#d20;background-color:#fff0f0"> 'Test Last Name',
</span><span style="color:#d20;background-color:#fff0f0"> _address_id
</span><span style="color:#d20;background-color:#fff0f0"> );
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> INSERT INTO inventory (inventory_id, film_id, store_id)
</span><span style="color:#d20;background-color:#fff0f0"> VALUES (
</span><span style="color:#d20;background-color:#fff0f0"> _inventory_id,
</span><span style="color:#d20;background-color:#fff0f0"> (SELECT film_id FROM film LIMIT 1), -- arbitrary film
</span><span style="color:#d20;background-color:#fff0f0"> _store_id
</span><span style="color:#d20;background-color:#fff0f0"> );
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> INSERT INTO rental (rental_id, rental_date, inventory_id, customer_id, staff_id)
</span><span style="color:#d20;background-color:#fff0f0"> VALUES (
</span><span style="color:#d20;background-color:#fff0f0"> _rental_id,
</span><span style="color:#d20;background-color:#fff0f0"> _start_date,
</span><span style="color:#d20;background-color:#fff0f0"> _inventory_id,
</span><span style="color:#d20;background-color:#fff0f0"> _customer_id::smallint,
</span><span style="color:#d20;background-color:#fff0f0"> (SELECT staff_id FROM staff WHERE store_id = _store_id LIMIT 1) -- select arbitrary staff member from store
</span><span style="color:#d20;background-color:#fff0f0"> );
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> INSERT INTO test_instance_references (test_rental_id, test_number, start_date)
</span><span style="color:#d20;background-color:#fff0f0"> VALUES (
</span><span style="color:#d20;background-color:#fff0f0"> _rental_id,
</span><span style="color:#d20;background-color:#fff0f0"> _test_number,
</span><span style="color:#d20;background-color:#fff0f0"> _start_date
</span><span style="color:#d20;background-color:#fff0f0"> );
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> PERFORM extension_check(_rental_id);
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> _start_date := _start_date + INTERVAL '1 day';
</span><span style="color:#d20;background-color:#fff0f0"> _test_number := _test_number + 1;
</span><span style="color:#d20;background-color:#fff0f0">
</span><span style="color:#d20;background-color:#fff0f0"> END LOOP;
</span><span style="color:#d20;background-color:#fff0f0"> END;
</span><span style="color:#d20;background-color:#fff0f0">$procedure$</span>;
CALL fill_test_data((<span style="color:#d20;background-color:#fff0f0">'2023-12-25'</span>::<span style="color:#038">date</span> - <span style="color:#038">INTERVAL</span> <span style="color:#d20;background-color:#fff0f0">'6 days'</span>)::<span style="color:#038">date</span>, <span style="color:#d20;background-color:#fff0f0">'2023-12-25'</span>::<span style="color:#038">date</span>);
<span style="color:#080;font-weight:bold">SELECT</span> plan(count(*)::<span style="color:#038">integer</span> + <span style="color:#00d;font-weight:bold">1</span>) <span style="color:#080;font-weight:bold">from</span> test_instance_references;
<span style="color:#080;font-weight:bold">SELECT</span> * <span style="color:#080;font-weight:bold">from</span> test_instance_references;
<span style="color:#080;font-weight:bold">SELECT</span> has_function(
<span style="color:#d20;background-color:#fff0f0">'extension_check'</span>,
<span style="color:#080;font-weight:bold">ARRAY</span>[<span style="color:#d20;background-color:#fff0f0">'int'</span>]
<span style="color:#888">-- Description optional - default description usually substantial
</span><span style="color:#888"></span>);
<span style="color:#080;font-weight:bold">SELECT</span> <span style="color:#080;font-weight:bold">is</span>(
(<span style="color:#080;font-weight:bold">SELECT</span> return_date <span style="color:#080;font-weight:bold">FROM</span> rental <span style="color:#080;font-weight:bold">WHERE</span> rental_id = tf<span style="color:#00d;font-weight:bold">.</span>test_rental_id)::<span style="color:#038">date</span>,
(tf<span style="color:#00d;font-weight:bold">.</span>start_date + <span style="color:#038">INTERVAL</span> <span style="color:#d20;background-color:#fff0f0">'10 DAYS'</span>)::<span style="color:#038">date</span>
)
<span style="color:#080;font-weight:bold">FROM</span> test_instance_references <span style="color:#080;font-weight:bold">as</span> tf;
<span style="color:#080;font-weight:bold">ROLLBACK</span>;
</code></pre></div><p>Our results should be:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ pg_prove -d <your_database> <path_to_your_unit_test>/extension_unit_tests.sql
extension_unit_tests.sql .. ok
All tests successful.
Files=1, Tests=8, 0 wallclock secs ( 0.02 usr + 0.00 sys = 0.02 CPU)
Result: PASS
</code></pre></div><p>As someone who found pgTAP fairly straightforward the challenge of intertwining those functions in a repeatable way was what drove me to write this article. Hopefully this makes it easier for the next coder to play around and get the most out of this simple but intuitive “Test Anything Protocol”.</p>
<p>Feel free to share your thoughts and comments below, or to share any more useful tips on pgTAP or general Postgres unit testing.</p>
Ubuntu Touch on a Galaxy S7 & a Pixel 3ahttps://www.endpointdev.com/blog/2023/08/ubuntu-touch-on-galaxy-s7-a-pixel-3a/2023-08-29T00:00:00+00:00Seth Jensen
<p><img src="/blog/2023/08/ubuntu-touch-on-galaxy-s7-a-pixel-3a/arabesque_with_knight-crop.webp" alt="A dark print on very old paper shows an armored knight on a horse, surrounded completely by swirling floral patterns. In a few places the pattern evolves into a flower or what appears to be a curved, long bird head."></p>
<!-- Image: Arabesque with Knight, Italian, niello print, unknown date. Retrieved from https://www.nga.gov/collection/art-object-page.4541.html -->
<p>I have a shoebox in my closet I call my “phone graveyard.” At times I’ve had five or more old phones in there, in various states of decay—some have fully shattered screens, some broken USB ports, etc. But some still have quite usable hardware, their main drawback being how slowly they run modern versions of Android or iOS, and the lack of support for modern features.</p>
<p>It has always troubled me to have such incredible devices, with way more computing power than, say, a Raspberry Pi, gather dust because of software limitations. Even if I don’t use an old phone for daily use any more, what if I could use it as a DNS server (like a Pi-hole), or as a camera or media player?</p>
<p>When I heard about Ubuntu Touch, it seemed like the perfect OS to bring back some long-term functionality to these old devices. Originally created by Canonical, it was soon abandoned but revived by UBports, who started community development in 2015. They actively maintain the OS for around 80 devices, including two I have in my phone graveyard: a Samsung Galaxy S7 and a Google Pixel 3a.</p>
<h3 id="installation">Installation</h3>
<p>Installing Ubuntu Touch is quite straightforward; the UBports installer does most of the heavy lifting. See the <a href="https://devices.ubuntu-touch.io/installer/">UBports website</a> for instructions.</p>
<p>For the Pixel 3a, I had to flash an older version of the stock OS before running the UBports installer. The Ubuntu Touch <a href="https://devices.ubuntu-touch.io/device/sargo">device page</a> for the Pixel 3a specifies which version to flash under “Preparatory Step,” but this seems to be in a different place for each device. Read the device-specific instructions carefully before installing.</p>
<p>See also:</p>
<ul>
<li>This useful <a href="https://android.gadgethacks.com/how-to/complete-guide-flashing-factory-images-android-using-fastboot-0175277/">guide</a> from Gadget Hacks on flashing a factory image</li>
<li>Android’s <a href="https://developer.android.com/studio/releases/platform-tools">SDK platform tools</a>, including fastboot</li>
<li>Android’s docs for <a href="https://source.android.com/docs/core/architecture/bootloader/locking_unlocking">locking/unlocking the bootloader</a></li>
</ul>
<h3 id="usability">Usability</h3>
<p>The native app landscape for Ubuntu Touch is, as you would expect, pretty limited. You won’t be able to play most DRM content, and while there are web apps for Uber, Discord, and others, they tend to be various levels of buggy. You can see the apps, including helpful usability reports, at <a href="https://open-store.io/">open-store.io</a>.</p>
<p>There are plenty of bugs I encountered while briefly testing, like having no volume indicator when using volume keys and not seeing the Libertine installation which should’ve been there, making it hard to install programs with the terminal emulator.</p>
<p>Audio playback had fairly frequent interruptions, at least on my Pixel 3a, making it not viable as a music player. However, for audiobooks or podcasts this probably wouldn’t be a big deal.</p>
<p>However, most of the base functionality for web browsing, photos, and media playback seemed to work well.</p>
<p>The inclusion of a fully-featured terminal emulator expands the possibilities greatly, allowing your old phone to act in place of a Raspberry Pi for projects like a <a href="/blog/2020/12/pihole-great-holiday-gift/">Pi-hole</a>.</p>
<blockquote>
<p>The system is, by default, in read-only mode, requiring containers (the intent is to run the custom container system, <a href="https://docs.ubports.com/en/latest/userguide/dailyuse/libertine.html">Libertine</a>) to run most apps. However, you can change it to read-write for a more traditional Linux system.
For more info on terminal capability, see <a href="https://ubports.com/en/blog/ubports-news-1/post/terminal-chapter-1-3082">the UBports blog</a>.</p>
</blockquote>
<p>While I wouldn’t want to run Ubuntu Touch as my only smartphone OS, it is an exciting option for secondary or old devices, especially if you want to contribute to open-source projects and help out the community!</p>
Upgrade Vue to TypeScripthttps://www.endpointdev.com/blog/2023/08/upgrade-vue-to-typescript/2023-08-28T00:00:00+00:00Nicholas Piano
<p><img src="/blog/2023/08/upgrade-vue-to-typescript/sunset-field.webp" alt="An expansive sky filled with faintly red clouds extends above a field turned red from the sunset. A layer of trees separates the field from the sky"></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>It’s important to keep your code up to date so that time can be dedicated to improving an application instead of version-related mishaps. This is especially true for web development as the landscape changes so quickly.</p>
<p>I recently upgraded a Vue project to exclusively use Vuex. This was a great opportunity to also upgrade the project from JavaScript to TypeScript. This article will cover the steps I took.</p>
<p>Some of the changes can be difficult to understand if you are not familiar with TypeScript. I recommend reading the <a href="https://www.typescriptlang.org/docs/handbook/intro.html">TypeScript Handbook</a> to become more familiar.</p>
<p>Several features of Vue, originally written in JavaScript without types, are hard to convert to TypeScript. These include <code>this.$parent</code>, <code>this.$refs</code>, and <code>this.$emit</code>. These allow you to access the parent component, child components, and emit events respectively. We will make changes to these features along with adding types to the global state handler provided by <code>Vuex.Store</code>.</p>
<h3 id="installation">Installation</h3>
<p>Before you begin, make sure the necessary dependencies are installed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">~$ vue add typescript
</code></pre></div><p>Also make sure that Vuex is installed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">~$ yarn add vuex@next
</code></pre></div><h3 id="convert-your-component-files">Convert your component files</h3>
<p>There are several changes that must be made to component files, such as <code>App.vue</code>.</p>
<ol>
<li>Add <code>lang="ts"</code> to the <code><script></code> tag.</li>
<li>Replace the default export with a class that extends <code>Vue</code> and uses the <code>@Component</code> decorator.</li>
<li>Replace <code>props</code> with class properties marked with the <code>@Prop</code> decorator.</li>
<li>Replace <code>data</code> with individual class properties.</li>
<li>Add a typed <code>$refs</code> class property.</li>
<li>Replace <code>computed</code> with getter and setter class properties.</li>
<li>Lifecycle hooks simply become class methods, so they can be copied directly.</li>
<li>Replace <code>methods</code> with individual class methods.</li>
<li>Replace <code>this.$emit</code> with class methods using the <code>@Emit</code> decorator.</li>
</ol>
<p>Let’s look at each change separately.</p>
<h4 id="add-langts-to-the-script-tag">Add <code>lang="ts"</code> to the <code><script></code> tag</h4>
<p>This is the easiest change. Simply add <code>lang="ts"</code> to the <code><script></code> tag.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#fdd">-<script>
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+<script lang="ts">
</span></code></pre></div><h4 id="replace-the-default-export-with-a-class-that-extends-vue-and-uses-the-component-decorator">Replace the default export with a class that extends <code>Vue</code> and uses the <code>Component</code> decorator</h4>
<p>This change can be complicated to visualise, but the <code>@Component</code> decorator provided by <code>vue-class-component</code> makes it easy to convert a component incrementally by accepting existing properties of the default export as arguments to allow backwards compatibility.</p>
<p>First, replace the default export with a class component:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#fdd">-export default {
</span><span style="color:#000;background-color:#fdd">- name: 'App',
</span><span style="color:#000;background-color:#fdd">- components: {
</span><span style="color:#000;background-color:#fdd">- HelloWorld
</span><span style="color:#000;background-color:#fdd">- },
</span><span style="color:#000;background-color:#fdd">- props: ['msg'],
</span><span style="color:#000;background-color:#fdd">-}
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+import Component from 'vue-class-component'
</span><span style="color:#000;background-color:#dfd">+import { Vue } from 'vue-property-decorator'
</span><span style="color:#000;background-color:#dfd">+
</span><span style="color:#000;background-color:#dfd">+@Component({
</span><span style="color:#000;background-color:#dfd">+ components: {
</span><span style="color:#000;background-color:#dfd">+ HelloWorld
</span><span style="color:#000;background-color:#dfd">+ },
</span><span style="color:#000;background-color:#dfd">+ props: ['msg'],
</span><span style="color:#000;background-color:#dfd">+})
</span><span style="color:#000;background-color:#dfd">+export default class App extends Vue {}
</span></code></pre></div><p>Note that the <code>name</code> property is no longer necessary. The name of the component is now the name of the class.</p>
<p>Also, to illustrate the capability of the <code>@Component</code> decorator, the <code>props</code> property is passed as an argument. This allows the component to be used as before with no other changes. Next, we will replace <code>props</code> with class properties marked by the <code>@Prop</code> decorator.</p>
<h4 id="replace-props-with-class-properties-marked-by-the-prop-decorator">Replace <code>props</code> with class properties marked by the <code>@Prop</code> decorator</h4>
<p>The <code>@Prop</code> decorator is used to mark class properties as props. It accepts an optional argument to specify the type of the prop. If no argument is provided, the type is inferred from the default value.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">import</span> Component <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vue-class-component'</span>
<span style="color:#080;font-weight:bold">import</span> { Prop, Vue } <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vue-property-decorator'</span>
<span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> App <span style="color:#080;font-weight:bold">extends</span> Vue {
<span style="color:#080;font-weight:bold">@Prop</span>() msg!: <span style="color:#888;font-weight:bold">string</span>
<span style="color:#080;font-weight:bold">@Prop</span>({ <span style="color:#080;font-weight:bold">type</span>: <span style="color:#038">Number</span> }) count!: <span style="color:#888;font-weight:bold">number</span> <span style="color:#888">// type can also be specified explicitly
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">@Prop</span>(<span style="color:#038">Boolean</span>) disabled!: <span style="color:#080;font-weight:bold">boolean</span> <span style="color:#888">// shorthand for { type: Boolean }
</span><span style="color:#888"></span>}
</code></pre></div><p>Now, when a prop is referenced in a function or the template, it will be typed correctly.</p>
<h4 id="replace-data-with-individual-class-properties">Replace <code>data</code> with individual class properties</h4>
<p>The <code>data</code> property is replaced with individual class properties.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> {
name: <span style="color:#d20;background-color:#fff0f0">'App'</span>,
data() {
<span style="color:#080;font-weight:bold">return</span> {
count: <span style="color:#00d;font-weight:bold">0</span>,
msg: <span style="color:#d20;background-color:#fff0f0">'Hello World'</span>,
complex: { <span style="color:#888">/* ... */</span> },
},
},
}
</code></pre></div><p>Becomes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> App <span style="color:#080;font-weight:bold">extends</span> Vue {
count = <span style="color:#00d;font-weight:bold">0</span>
msg: <span style="color:#888;font-weight:bold">string</span> = <span style="color:#d20;background-color:#fff0f0">'Hello World'</span>
complex: <span style="color:#888;font-weight:bold">ComplexType</span> = { <span style="color:#888">/* ... */</span> }
}
</code></pre></div><p>As before, types can be specified explicitly or inferred from the default value.</p>
<h4 id="add-a-typed-refs-class-property">Add a typed <code>$refs</code> class property</h4>
<p>The <code>$refs</code> property is typed by adding a class property with the same name and type.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> App <span style="color:#080;font-weight:bold">extends</span> Vue {
$refs!: {
input: <span style="color:#888;font-weight:bold">HTMLInputElement</span>
}
}
</code></pre></div><p>Now, references to <code>$refs.input</code> will be typed correctly.</p>
<h4 id="replace-computed-with-getter-and-setter-class-properties">Replace <code>computed</code> with getter and setter class properties</h4>
<p>The <code>computed</code> property is replaced with getter and setter class properties.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> {
name: <span style="color:#d20;background-color:#fff0f0">'App'</span>,
computed: {
count() {
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">this</span>.$store.state.count
},
msg() {
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">this</span>.$store.state.msg
},
},
}
</code></pre></div><p>Becomes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> App <span style="color:#080;font-weight:bold">extends</span> Vue {
<span style="color:#080;font-weight:bold">get</span> count() {
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">this</span>.$store.state.count
}
<span style="color:#080;font-weight:bold">set</span> count(value: <span style="color:#888;font-weight:bold">number</span>) {
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'setCount'</span>, value)
}
<span style="color:#080;font-weight:bold">get</span> msg() {
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">this</span>.$store.state.msg
}
<span style="color:#080;font-weight:bold">set</span> msg(value: <span style="color:#888;font-weight:bold">string</span>) {
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'setMsg'</span>, value)
}
}
</code></pre></div><p>Now the getters and setters can accept and return the correct types.</p>
<h4 id="lifecycle-hooks-become-class-methods">Lifecycle hooks become class methods</h4>
<p>Lifecycle hooks such as <code>created</code> and <code>mounted</code> are represented as class methods.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> {
name: <span style="color:#d20;background-color:#fff0f0">'App'</span>,
created() {
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'setCount'</span>, <span style="color:#00d;font-weight:bold">0</span>)
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'setMsg'</span>, <span style="color:#d20;background-color:#fff0f0">'Hello World'</span>)
},
}
</code></pre></div><p>Becomes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> App <span style="color:#080;font-weight:bold">extends</span> Vue {
created() {
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'setCount'</span>, <span style="color:#00d;font-weight:bold">0</span>)
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'setMsg'</span>, <span style="color:#d20;background-color:#fff0f0">'Hello World'</span>)
}
}
</code></pre></div><h4 id="replace-methods-with-individual-class-methods">Replace <code>methods</code> with individual class methods</h4>
<p>The <code>methods</code> property is replaced with individual class methods. Not much about the methods needs to change.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> {
name: <span style="color:#d20;background-color:#fff0f0">'App'</span>,
methods: {
increment() {
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'increment'</span>)
},
decrement() {
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'decrement'</span>)
},
},
}
</code></pre></div><p>Becomes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> App <span style="color:#080;font-weight:bold">extends</span> Vue {
increment() {
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'increment'</span>)
}
decrement() {
<span style="color:#080;font-weight:bold">this</span>.$store.commit(<span style="color:#d20;background-color:#fff0f0">'decrement'</span>)
}
}
</code></pre></div><h4 id="replace-thisemit-with-class-methods-using-the-emit-decorator">Replace <code>this.$emit</code> with class methods using the <code>@Emit</code> decorator</h4>
<p>The <code>@Emit</code> decorator is used to mark class methods as emitters. It accepts an optional argument to specify the event name. If no argument is provided, the event name is the name of the method.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> {
name: <span style="color:#d20;background-color:#fff0f0">'App'</span>,
methods: {
setValue(value) {
<span style="color:#080;font-weight:bold">this</span>.$emit(<span style="color:#d20;background-color:#fff0f0">'set-value'</span>, value)
},
},
}
</code></pre></div><p>Becomes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> App <span style="color:#080;font-weight:bold">extends</span> Vue {
<span style="color:#080;font-weight:bold">@Emit</span>(<span style="color:#d20;background-color:#fff0f0">'set-value'</span>) setValue(value: <span style="color:#888;font-weight:bold">string</span>) {
<span style="color:#080;font-weight:bold">return</span> value
}
}
</code></pre></div><p>This would be called from the parent component in the same way:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">template</span>>
<<span style="color:#b06;font-weight:bold">app</span> <span style="color:#a61717;background-color:#e3d2d2">@</span><span style="color:#369">set-value</span>=<span style="color:#d20;background-color:#fff0f0">"onSetValue"</span> />
</<span style="color:#b06;font-weight:bold">template</span>>
</code></pre></div><p>Now that your components have been converted, let’s look at the store.</p>
<h3 id="convert-your-store">Convert your store</h3>
<p>State in Vuex is managed using the <code>Vuex.Store</code> object. Below is a basic example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">import</span> Vue <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vue'</span>
<span style="color:#080;font-weight:bold">import</span> Vuex <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vuex'</span>
Vue.use(Vuex)
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">new</span> Vuex.Store({
state: {
count: <span style="color:#888;font-weight:bold">0</span>,
msg: <span style="color:#d20;background-color:#fff0f0">'Hello World'</span>,
},
mutations: {
setCount(state, value) {
state.count = value
},
setMsg(state, value) {
state.msg = value
},
increment(state) {
state.count++
},
decrement(state) {
state.count--
},
},
getters: {
count: <span style="color:#888;font-weight:bold">state</span> => state.count,
msg: <span style="color:#888;font-weight:bold">state</span> => state.msg,
},
actions: {
increment(context) {
context.commit(<span style="color:#d20;background-color:#fff0f0">'increment'</span>)
},
decrement(context) {
context.commit(<span style="color:#d20;background-color:#fff0f0">'decrement'</span>)
},
},
})
</code></pre></div><p>Inside a component, access to the store can be typed using the <code>@State</code>, <code>@Mutation</code>, <code>@Getter</code>, and <code>@Action</code> decorators.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">import</span> { Component } <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vue-property-decorator'</span>;
<span style="color:#080;font-weight:bold">import</span> { Mutation, Getter, Action } <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vuex-class'</span>;
<span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> MyComponent <span style="color:#080;font-weight:bold">extends</span> Vue {
<span style="color:#080;font-weight:bold">@State</span> count!: <span style="color:#888;font-weight:bold">number</span>
<span style="color:#080;font-weight:bold">@State</span> msg!: <span style="color:#888;font-weight:bold">string</span>
<span style="color:#080;font-weight:bold">@Mutation</span> setCount!: (value: <span style="color:#888;font-weight:bold">number</span>) => <span style="color:#080;font-weight:bold">void</span>
<span style="color:#080;font-weight:bold">@Mutation</span> setMsg!: (value: <span style="color:#888;font-weight:bold">string</span>) => <span style="color:#080;font-weight:bold">void</span>
<span style="color:#080;font-weight:bold">@Mutation</span> increment!: () => <span style="color:#080;font-weight:bold">void</span>
<span style="color:#080;font-weight:bold">@Mutation</span> decrement!: () => <span style="color:#080;font-weight:bold">void</span>
<span style="color:#080;font-weight:bold">@Getter</span> count!: <span style="color:#888;font-weight:bold">number</span>
<span style="color:#080;font-weight:bold">@Getter</span> msg!: <span style="color:#888;font-weight:bold">string</span>
<span style="color:#080;font-weight:bold">@Action</span> increment!: () => <span style="color:#080;font-weight:bold">void</span>
<span style="color:#080;font-weight:bold">@Action</span> decrement!: () => <span style="color:#080;font-weight:bold">void</span>
}
</code></pre></div><p>Parts of the store can also be namespaced using the <code>modules</code> key in the store object:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">import</span> Vue <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vue'</span>
<span style="color:#080;font-weight:bold">import</span> Vuex <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vuex'</span>
Vue.use(Vuex)
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">new</span> Vuex.Store({
<span style="color:#080;font-weight:bold">module</span>s: { ExampleModule },
});
</code></pre></div><p>Where a module is an object with the same structure as the store object. See the <a href="https://vuex.vuejs.org/guide/modules.html">Vuex module documentation</a> for more information.</p>
<p>The namespaced state can then be typed and accessed from components:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-typescript" data-lang="typescript"><span style="color:#080;font-weight:bold">import</span> { Component } <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vue-property-decorator'</span>;
<span style="color:#080;font-weight:bold">import</span> { <span style="color:#080;font-weight:bold">namespace</span> } <span style="color:#080;font-weight:bold">from</span> <span style="color:#d20;background-color:#fff0f0">'vuex-class'</span>;
<span style="color:#080;font-weight:bold">const</span> Store = {
ExampleModule: <span style="color:#888;font-weight:bold">namespace</span>(<span style="color:#d20;background-color:#fff0f0">'ExampleModule'</span>),
};
<span style="color:#080;font-weight:bold">@Component</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> ImportExample <span style="color:#080;font-weight:bold">extends</span> Vue {
<span style="color:#080;font-weight:bold">@Store</span>.ExampleModule.Action(<span style="color:#d20;background-color:#fff0f0">'import'</span>) <span style="color:#080;font-weight:bold">import</span>!: ({ file, config }: { file: <span style="color:#888;font-weight:bold">File</span>; config: <span style="color:#888;font-weight:bold">ExampleConfig</span> }) => Promise<<span style="color:#b06;font-weight:bold">boolean</span>>;
}
</code></pre></div><p>Note that this adds types to the store access points within components, but does not fully statically type the store itself. For more information on how to do this, refer to this excellent article: <a href="https://dev.to/3vilarthas/vuex-typescript-m4j">Vuex + TypeScript</a>.</p>
<h3 id="conclusion">Conclusion</h3>
<p>In conclusion, despite the number of steps needed to convert your Vue components and store, the process is relatively simple. The result is a fully typed Vue application that is easier to maintain and refactor. You won’t regret the ability to trace the flow of data through your application. Additionally, currently developed packages for Vue such as <a href="https://vuetifyjs.com/en/">Vuetify</a> and <a href="https://vuelidate.js.org/">Vuelidate</a> are fully typed and will work seamlessly with your new Vue application.</p>
Synchronous work in asynchronous work environmentshttps://www.endpointdev.com/blog/2023/08/synchronous-work-in-asynchronous-environments/2023-08-15T00:00:00+00:00Vincent Martin
<p><img src="/blog/2023/08/synchronous-work-in-asynchronous-environments/windy-tree.webp" alt="A light green tree blows in strong winds, against a grey stormy sky."></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>Co-authored by <a href="/team/trevor-slocum/">Trevor Slocum</a>.</p>
<p>As a team that spans the globe, asynchronous communication is a necessity for us. While we do use email, of course, we use internal text chat rooms for communication more frequently. These asynchronous tools are useful to have around, but they may not be the best fit for sharing an idea or some other information with others. Sometimes we need to speak with someone in a voice or video conference, or even share our screen and work together to solve a problem. In this article we will share some of the things we have learned while working in these situations.</p>
<h3 id="solving-problems-together">Solving problems together</h3>
<p>When pair programming, we solve problems together. This usually takes the form of one person sharing their screen while the other watches and provides guidance and feedback, but it also takes other forms which we will refer to later. Having someone critique our work in real time as we try to solve complex problems can be stressful. This is an entirely natural reaction to the situation, and happens to everyone who engages in pair programming and other collaborative work.</p>
<p>If you find yourself feeling this way, our advice is to take a brief pause and talk it out with your co-worker. Taking a few minutes to clear your head and get your feelings off of your chest is really all it takes. In the moment, it can be hard to see that our teammate is just trying to help us. Once we have shared how we are feeling, we start to see and understand the situation more clearly.</p>
<p>Another important point is that working together is inherently a teaching and learning activity. Because we were not extruded from a cloning machine, we all have different ideas and perspectives. By virtue of interacting with each other, particularly on anything for which one can acquire a level of understanding and proficiency, we share things that we have picked up along the way, and pick up things that others share along the way.</p>
<p>Pair programming at its best is two friendly engineers enjoying the challenge of building something or solving a problem together while expanding their individual understandings of the problem domain and building trust as a team. Everyone benefits, so try to relax and have fun!</p>
<h3 id="technical-aspects">Technical Aspects</h3>
<p>With all the mental aspects of pair programming addressed, you can now get to programming with your co-worker. While a software developer may share their screen, as mentioned previously, it’s also possible to share an entire development environment. This allows your teammate to see and edit the entire repository of source code in real time, from the comfort of their own editor running on their computer.</p>
<p>These shared environments allow you to follow each other’s cursor and also see what each person has highlighted, or work independently in different files. This approach also has the advantage of not relying on video compression to share code.</p>
<p><img src="/blog/2023/08/synchronous-work-in-asynchronous-environments/vscode.webp" alt="A screenshot demonstrating the initial step of sharing with others using VS Code. VS Code’s command pallette is open, with “live share” typed into the search box. Selected is “Live Share: Start Collaboration Session (Share). Other options are Join Collaboration Session, Focus on Session Details View, Help, Sign out, and Start Read-Only Collaboration Session. Each is prefixed by “Live Share:""><br>
Using the VS Code Live Share plugin</p>
<p>Visual Studio Code offers a plugin called “<a href="https://code.visualstudio.com/learn/collaboration/live-share">Live Share</a>” which can be installed to allow the users to collaborate synchronously on a project. The project itself lives on one of the users' computers, for example if you wanted to pair a program with a friend, you would initiate the session on your computer and send your friend the provided web link, which they would use to attach to your session. This feature not only allows you to share your code, but also allows you to share useful tools like the in-editor debugger.</p>
<p><img src="/blog/2023/08/synchronous-work-in-asynchronous-environments/intellij.webp" alt="A screenshot demonstrating the initial step of sharing with others using IntelliJ IDEA. A popup titled “Code With Me: Start Session” reads “Set permissions for new guests, under which are radio buttons reading Read-Only (which is selected), Edit files, Full access, and Custom. There is also an unchecked checkbox giving the option to “start call.""><br>
Using the IntelliJ Code With Me plugin</p>
<p>The IntelliJ family of editors (e.g. IDEA) offer a similar plugin called “<a href="https://www.jetbrains.com/help/idea/code-with-me.html">Code With Me</a>.” This plugin provides additional features such as allowing more than two users to connect at the same time. Actual live editing of code is limited to five users at the moment, but you can have more if you operate in a teacher-student scenario where you can present code real time with a large number of your teammates by enabling “force others to follow you” mode.</p>
<h3 id="the-flow">The flow</h3>
<p>Pair programming can be quite effective, but the social aspect can cause participants to burn out pretty quickly. It can sometimes be hard to focus and socialize for hours at a time. Because of this we have found it quite important to take breaks. The question is, when do you take these breaks?</p>
<p>Breaking apart and coming back together later is another workflow that we have had a lot of success with. A scenario which demonstrates this is the following: Imagine you and your co-worker are working on a task when you both come to a problem which neither of you knows how to solve. Finding the solution will require a bit of research, reading, and trial & error. A few solutions will have to be tried!</p>
<p>Oftentimes this is a good hint to both of you that you should take a break from the problem and work on it independently, bringing your findings together at a later time. This essentially breaks up the serial nature of pair programming and makes it parallel. In other words, pair programming is not always done together.</p>
<h3 id="wrapping-it-up">Wrapping it up</h3>
<p>Once you are finished working on a problem together, it’s time to commit the work. If you’re using <a href="https://git-scm.com">git</a>, only one author will be attributed by default (whoever runs the command <code>git commit</code>). This means that if/when we find a bug in this code later, and need to identify who authored the code responsible for it to explain why the code was written this way, we might only see half of the picture (or worse).</p>
<p>Adding a co-authored-by line (such as the one below) at the end of a git commit message is the de facto standard for attributing multiple authors to a git commit. This is not officially supported by git, but many third-party git tools recognize this line, such as GitHub and GitLab, and this practice is followed by many programmers.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Co-authored-by: Chuck Person <chuck@eccojams.opn>
</code></pre></div><p>Even when a git tool does not recognize this line, other tools such as grep can be used to extract this information.</p>
<p>Note that you must add a blank line before adding any <code>Co-authored-by</code> lines. For example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Add fizz, fix buzz
We need to include fizz in order for buzz to function properly.
Co-authored-by: Chuck Person <chuck@eccojams.opn>
Co-authored-by: Ramona Langley <ramona@floralshoppe.mpl>
</code></pre></div><p>Happy coding!</p>
How to use dependent dropdowns in Microsoft Excelhttps://www.endpointdev.com/blog/2023/08/dependent-dropdowns-in-microsoft-excel/2023-08-02T00:00:00+00:00Nicholas Piano
<p><img src="/blog/2023/08/dependent-dropdowns-in-microsoft-excel/summer-tree.webp" alt="Red leaves extend on thin branches towards the center of the image. Each is a different distance from the camera, so one, with white blossoms, is in focus, while some are blurred a little, and others so much that they are almost unrecognizable red splotches. There is a vaguely spherical distortion from the lens which gives a convex look to the image."><br></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>Dropdowns are a way to constrain the input of a cell to a set of values. This can be useful for data validation, or for making a spreadsheet more user-friendly. In this article, we will look at how to create two types of dropdowns in Microsoft Excel: the basic dropdown and the dependent dropdown.</p>
<p>Using this, we can expand to more complex dropdown dependencies and spreadsheet generation, including validation using the Axlsx gem in Ruby.</p>
<h3 id="the-basic-dropdown">The basic dropdown</h3>
<p>A dropdown takes its data from a list of values specified by a range. This could be a range of cells, or a named range. To create a dropdown, first select the cell you want to add the dropdown to. Then, go to the Data tab, and click on Data Validation.</p>
<p><img src="/blog/2023/08/dependent-dropdowns-in-microsoft-excel/00_data_validation.webp" alt="The data validation dialog. The leftmost section is selected, “Settings”. Under the single heading, “Validation criteria”, there are several options. “Ignore blank” and “In-cell drop-down” are checked, “Allow:” shows a dropdown set to “List”, and “Source” is empty. A “Data” dropdown is greyed out."><br>
Data Validation Menu</p>
<p>Specify the range, e.g. <code>=Sheet1!$A$1:$A$5</code>. Then, select the “List” option under “Allow”. You can also specify an input message and error message, which will be displayed when the user selects the cell. Note that the range can come from another worksheet.</p>
<p><img src="/blog/2023/08/dependent-dropdowns-in-microsoft-excel/01_basic_range.webp" alt="An Excel column reading “R” has a cell reading “James”. A dropdown is open, showing options Harry, Tommy, George, John, Roger, James (highlighted), and Adrian. Cells in column S hold this list of names."><br>
Basic Range</p>
<p>And that’s it! You can now constrain the input of the cell to the values in the range.</p>
<h3 id="the-dependent-dropdown">The dependent dropdown</h3>
<p>Things get a little more complicated when we want to create a dropdown that depends on another dropdown. For example, we might want to have a dropdown for “Country”, and another dropdown for “State”, where the list of states depends on the country selected.</p>
<p>To do this, the associations between countries and states must be stored in a way that can be easily accessed by Excel. For example:</p>
<p><img src="/blog/2023/08/dependent-dropdowns-in-microsoft-excel/02_country_state_data.webp" alt="A spreadsheet. Column A reads “Spain” in a green header on row 1, then a list of states in Spain: Andalusia, Aragon, Asturias, and so on. Column B reads “Italy” in the header position, with a list of its states: Abruzzo, Valle d’Aosta, Puglia, etc."><br></p>
<p>Given this data, the source for the “Country” dropdown will be <code>=Sheet1!$A$1:$B$1</code>. If our country dropdown is in cell D1, the source for the “State” dropdown will be <code>=INDIRECT($D$1)</code>. Note that the INDIRECT function is used to convert the string into a reference to the range. That is, the string in the target cell is interpreted as a range or reference. The <code>&</code> operator is used to concatenate the string with the cell reference.</p>
<p><img src="/blog/2023/08/dependent-dropdowns-in-microsoft-excel/03_country_state_dropdown.webp" alt="The same spreadsheet, this time showing columns D and E, with headers “Country” and “State”, respectively. Below Country is a dropdown listing Spain and Italy, with Spain selected."><br></p>
<p>There is one more element needed to make this work: the named range. The value of <code>$D$1</code> in this case will simply be the value “Spain” or “Italy”, which does not, by itself, refer to a range. However, we can create a named range that refers to the range of states for each country. For example, we can create a named range called “Spain” that refers to <code>=Sheet1!$B$2:$B$4</code>, and a named range called “Italy” that refers to <code>=Sheet1!$B$5:$B$7</code>. Then, the value of <code>$D$1</code> will be interpreted as a reference to the named range, and the dropdown will be populated with the correct values.</p>
<p>To do this, go to the Formulas tab, and click on Name Manager. Then, click on New, and enter the name and the range. Repeat this for each country. In the end, the name manager should look like this:</p>
<p><img src="/blog/2023/08/dependent-dropdowns-in-microsoft-excel/04_name_manager.webp" alt="The Name Manager dialog. In a table we see Italy and Spain listed, along with the same list of states in a column named “Value”. In the “Refers To” column are “=Sheet1!$B$…” and “=Sheet1!$A$…”, respectively. In the “Scope” column both read “Workbook”. A “Comment” column is empty on both."><br></p>
<h3 id="using-the-axlsx-gem-in-ruby">Using the Axlsx gem in Ruby</h3>
<p>Now that we’ve seen how to create dependent dropdowns in Excel manually, we can use this to create more complex spreadsheets using scripts. We can use the Axlsx gem in Ruby to generate spreadsheets that include the same dropdowns and data validation as before, but now generated automatically from available data.</p>
<p>Let’s recreate our example above using <code>axlsx</code>. First, we need to create a workbook and a worksheet:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-ruby" data-lang="ruby"><span style="color:#038">require</span> <span style="color:#d20;background-color:#fff0f0">'axlsx'</span>
<span style="color:#888"># first, let's add the data</span>
spain_regions = [<span style="color:#d20;background-color:#fff0f0">'Andalusia'</span>, <span style="color:#d20;background-color:#fff0f0">'Aragon'</span>, <span style="color:#d20;background-color:#fff0f0">'Asturias'</span>, <span style="color:#d20;background-color:#fff0f0">'Balearic Islands'</span>, <span style="color:#d20;background-color:#fff0f0">'Basque Country'</span>, <span style="color:#d20;background-color:#fff0f0">'Canary Islands'</span>, <span style="color:#d20;background-color:#fff0f0">'Cantabria'</span>, <span style="color:#d20;background-color:#fff0f0">'Castile and Leon'</span>, <span style="color:#d20;background-color:#fff0f0">'Castile-La Mancha'</span>, <span style="color:#d20;background-color:#fff0f0">'Catalonia'</span>, <span style="color:#d20;background-color:#fff0f0">'Ceuta'</span>, <span style="color:#d20;background-color:#fff0f0">'Extremadura'</span>, <span style="color:#d20;background-color:#fff0f0">'Galicia'</span>, <span style="color:#d20;background-color:#fff0f0">'La Rioja'</span>, <span style="color:#d20;background-color:#fff0f0">'Madrid'</span>, <span style="color:#d20;background-color:#fff0f0">'Melilla'</span>, <span style="color:#d20;background-color:#fff0f0">'Murcia'</span>, <span style="color:#d20;background-color:#fff0f0">'Navarre'</span>, <span style="color:#d20;background-color:#fff0f0">'Valencian Community'</span>]
italy_regions = [<span style="color:#d20;background-color:#fff0f0">'Abruzzo'</span>, <span style="color:#d20;background-color:#fff0f0">'Aosta Valley'</span>, <span style="color:#d20;background-color:#fff0f0">'Apulia'</span>, <span style="color:#d20;background-color:#fff0f0">'Basilicata'</span>, <span style="color:#d20;background-color:#fff0f0">'Calabria'</span>, <span style="color:#d20;background-color:#fff0f0">'Campania'</span>, <span style="color:#d20;background-color:#fff0f0">'Emilia-Romagna'</span>, <span style="color:#d20;background-color:#fff0f0">'Friuli-Venezia Giulia'</span>, <span style="color:#d20;background-color:#fff0f0">'Lazio'</span>, <span style="color:#d20;background-color:#fff0f0">'Liguria'</span>, <span style="color:#d20;background-color:#fff0f0">'Lombardy'</span>, <span style="color:#d20;background-color:#fff0f0">'Marche'</span>, <span style="color:#d20;background-color:#fff0f0">'Molise'</span>, <span style="color:#d20;background-color:#fff0f0">'Piedmont'</span>, <span style="color:#d20;background-color:#fff0f0">'Sardinia'</span>, <span style="color:#d20;background-color:#fff0f0">'Sicily'</span>, <span style="color:#d20;background-color:#fff0f0">'Trentino-South Tyrol'</span>, <span style="color:#d20;background-color:#fff0f0">'Tuscany'</span>, <span style="color:#d20;background-color:#fff0f0">'Umbria'</span>, <span style="color:#d20;background-color:#fff0f0">'Veneto'</span>]
<span style="color:#038">p</span> = <span style="color:#036;font-weight:bold">Axlsx</span>::<span style="color:#036;font-weight:bold">Package</span>.new
wb = <span style="color:#038">p</span>.workbook
<span style="color:#888"># In order to demonstrate how to make references to other sheets in the revelant</span>
<span style="color:#888"># formulas, we will create two sheets, one for the data and one for the dropdowns.</span>
wb.add_worksheet(<span style="color:#038">name</span>: <span style="color:#d20;background-color:#fff0f0">'Countries and Regions'</span>) <span style="color:#080;font-weight:bold">do</span> |sheet|
headers = [<span style="color:#d20;background-color:#fff0f0">'Spain'</span>, <span style="color:#d20;background-color:#fff0f0">'Italy'</span>]
sheet.add_row(headers)
<span style="color:#888"># The data will be organised into two columns for easy access.</span>
<span style="color:#888"># We will use the zip method to iterate over both arrays at the same time.</span>
spain_regions.zip(italy_regions).each <span style="color:#080;font-weight:bold">do</span> |spain_region, italy_region|
sheet.add_row([spain_region, italy_region])
<span style="color:#080;font-weight:bold">end</span>
<span style="color:#888"># The three ranges we will use are:</span>
<span style="color:#888"># - The range of countries (the headers), which will be used for the first dropdown.</span>
<span style="color:#888"># - The range of regions (two ranges) for each country, which will be used for the second dropdown.</span>
<span style="color:#080;font-weight:bold">end</span>
wb.add_worksheet(<span style="color:#038">name</span>: <span style="color:#d20;background-color:#fff0f0">'Dropdowns'</span>) <span style="color:#080;font-weight:bold">do</span> |sheet|
<span style="color:#888"># The first dropdown will be in cell D1.</span>
sheet.add_data_validation(<span style="color:#d20;background-color:#fff0f0">'D1'</span>, {
<span style="color:#a60;background-color:#fff0f0">:type</span> => <span style="color:#a60;background-color:#fff0f0">:list</span>,
<span style="color:#a60;background-color:#fff0f0">:hideDropDown</span> => <span style="color:#080">false</span>,
<span style="color:#a60;background-color:#fff0f0">:formula1</span> => <span style="color:#d20;background-color:#fff0f0">"='Countries and Regions'!$A$1:$B$1"</span>,
<span style="color:#a60;background-color:#fff0f0">:prompt</span> => <span style="color:#d20;background-color:#fff0f0">'Select a country'</span>,
<span style="color:#a60;background-color:#fff0f0">:showErrorMessage</span> => <span style="color:#080">true</span>,
<span style="color:#a60;background-color:#fff0f0">:errorTitle</span> => <span style="color:#d20;background-color:#fff0f0">'Invalid Country'</span>,
<span style="color:#a60;background-color:#fff0f0">:error</span> => <span style="color:#d20;background-color:#fff0f0">'Please select a valid country.'</span>,
<span style="color:#a60;background-color:#fff0f0">:errorStyle</span> => <span style="color:#a60;background-color:#fff0f0">:stop</span>,
<span style="color:#a60;background-color:#fff0f0">:showInputMessage</span> => <span style="color:#080">true</span>,
})
<span style="color:#888"># The second dropdown will be in cell D2.</span>
<span style="color:#888"># Note the INDIRECT function that uses the value of D1 to determine the range.</span>
<span style="color:#888"># This will reference the named range for the selected country.</span>
sheet.add_data_validation(<span style="color:#d20;background-color:#fff0f0">'D2'</span>, {
<span style="color:#a60;background-color:#fff0f0">type</span>: <span style="color:#a60;background-color:#fff0f0">:list</span>,
<span style="color:#a60;background-color:#fff0f0">formula1</span>: <span style="color:#d20;background-color:#fff0f0">"INDIRECT($D$1)"</span>,
<span style="color:#a60;background-color:#fff0f0">allow_blank</span>: <span style="color:#080">true</span>,
<span style="color:#a60;background-color:#fff0f0">show_input_message</span>: <span style="color:#080">true</span>,
<span style="color:#a60;background-color:#fff0f0">show_error_message</span>: <span style="color:#080">true</span>,
<span style="color:#a60;background-color:#fff0f0">error_title</span>: <span style="color:#d20;background-color:#fff0f0">'Invalid Region'</span>,
<span style="color:#a60;background-color:#fff0f0">error_message</span>: <span style="color:#d20;background-color:#fff0f0">'Please select a valid region.'</span>,
<span style="color:#a60;background-color:#fff0f0">error_style</span>: <span style="color:#a60;background-color:#fff0f0">:stop</span>,
<span style="color:#a60;background-color:#fff0f0">show_drop_down</span>: <span style="color:#080">true</span>
})
<span style="color:#080;font-weight:bold">end</span>
<span style="color:#888"># Lastly, add the named ranges for each country</span>
spain_end_row = spain_regions.length + <span style="color:#00d;font-weight:bold">1</span>
wb.add_defined_name(<span style="color:#d20;background-color:#fff0f0">"'Countries and Regions'!$A$2:$A$</span><span style="color:#33b;background-color:#fff0f0">#{</span>spain_end_row<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">"</span>, { <span style="color:#a60;background-color:#fff0f0">:name</span> => <span style="color:#d20;background-color:#fff0f0">"Spain"</span> })
italy_end_row = italy_regions.length + <span style="color:#00d;font-weight:bold">1</span>
wb.add_defined_name(<span style="color:#d20;background-color:#fff0f0">"'Countries and Regions'!$B$2:$B$</span><span style="color:#33b;background-color:#fff0f0">#{</span>italy_end_row<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">"</span>, { <span style="color:#a60;background-color:#fff0f0">:name</span> => <span style="color:#d20;background-color:#fff0f0">"Italy"</span> })
<span style="color:#888"># Note that we have accounted for the length of the array when creating the named range.</span>
</code></pre></div><p>This script should do everything we need to create the spreadsheet. The only thing left to do is to save it:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-ruby" data-lang="ruby"><span style="color:#038">p</span>.serialize(<span style="color:#d20;background-color:#fff0f0">'dropdowns.xlsx'</span>)
</code></pre></div><p>And that’s it! You can now generate a template complete with complex data validation and dropdowns.</p>
<h3 id="conclusion">Conclusion</h3>
<p>Excel spreadsheets are a powerful tool for data analysis and visualisation. They are extremely portable, allowing them to be used as templates for data collection, such as polls.</p>
<p>In order to gather high quality data, it is important to ensure that the data is entered correctly. One way to do this is to use dropdowns and data validation.</p>
<p>In this article, we have seen how to create dependent dropdowns in Excel, and how to use the Axlsx gem in Ruby to generate spreadsheets with dropdowns and data validation.</p>
<p>One limitation to be aware of: while most spreadsheet functionality is common between different spreadsheet software, data validation and dropdowns are not. This means that the spreadsheets generated by the Axlsx gem will only work in Excel, and not in other spreadsheet software such as LibreOffice Calc.</p>
<p>With that, you can take your data collection to the next level!</p>
Text manipulation using regex replace in VS Codehttps://www.endpointdev.com/blog/2023/07/regex-replace-in-vs-code/2023-07-31T00:00:00+00:00Kevin Quaranta Jr.
<p><img src="/blog/2023/07/regex-replace-in-vs-code/light-patterns.webp" alt="A blank white wall and ceiling, with regular rays of light casting striped shadows at an acute angle. As the light hits the ceiling, the angle changes and the strips of light appear vertical."></p>
<!-- Photo by Seth Jensen -->
<p>Regular expressions are incredibly powerful tools that can make life easier for any developer. Being able to quickly and precisely parse text with the syntax you specify has kept regexes relevant from the ’80s through to today.</p>
<p>Visual Studio Code is an excellent modern editor and development environment. According to the Stack Overflow Developer Survey of 2022, it was far and away the most popular IDE for both professional developers and those learning to code—more than twice as popular as the second-place choice.</p>
<p>Like any great editor, VS Code supports using regexes to find and replace text. In this article, we’ll go over how to use this powerful regex mode.</p>
<p>To demonstrate, we’ll extract code values from a JavaScript object so that we can query a SQL database for content that contains one of the code values. We can use standard SQL query WHERE clause multiple matching:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sql" data-lang="sql"><span style="color:#080;font-weight:bold">WHERE</span><span style="color:#bbb"> </span>code<span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">IN</span><span style="color:#bbb"> </span>(<span style="color:#d20;background-color:#fff0f0">'code1'</span>,<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'code2'</span>,<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'code3'</span>)<span style="color:#bbb">
</span></code></pre></div><p>Or we can add the values to a regex alternation group separated by pipes (vertical bars) that we can use as a string match:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sql" data-lang="sql"><span style="color:#080;font-weight:bold">WHERE</span><span style="color:#bbb"> </span>code<span style="color:#bbb"> </span>~<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'^(?:code1|code2|code3)$'</span><span style="color:#bbb">
</span></code></pre></div><p>Each has different query plans and performance impacts in SQL, but for this example we’ll choose the second for its extra regex goodness. Doing the first would just require a few changes to our replacement regexes; feel free to practice your regex skills by figuring out how to fit our data to the standard SQL format!</p>
<p>We have copied and pasted all 287 lines of the object into VS Code. For the sake of this article, we’ll just show a handful of lines to demonstrate the point. Here are some lines from the object:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">{ 'code' : '33866-5','description' : 'HIV 1 Ab [Presence] in Capillary blood by Immunoassay'},
{ 'code' : '34591-8','description' : 'HIV 1 Ab [Presence] in Body fluid by Immunoassay'},
{ 'code' : '35437-3','description' : 'HIV 1 Ab [Presence] in Saliva (oral fluid) by Immunoassay'},
{ 'code' : '35438-1','description' : 'HIV 1 Ab [Units/volume] in Saliva (oral fluid) by Immunoassay'},
{ 'code' : '40437-6','description' : 'HIV 1 p24 Ab [Presence] in Serum by Immunoassay'},
{ 'code' : '40438-4','description' : 'HIV 1 gp41 Ab [Presence] in Serum by Immunoassay'},
{ 'code' : '40439-2','description' : 'HIV 1 gp120+gp160 Ab [Presence] in Serum by Immunoassay'},
{ 'code' : '49905-3','description' : 'HIV 1 Ab [Presence] in Unspecified specimen Qualitative by Rapid immunoassay'},
{ 'code' : '89374-3','description' : 'HIV 1 Ab [Presence] in Unspecified specimen by Immunoassay '},
{ 'code' : '51786-2','description' : 'HIV 2 Ab Signal/Cutoff in Serum by Immunoassay'},
{ 'code' : '12855-3','description' : 'HIV 1 p23 Ab [Presence] in Serum by Immunoblot (IB)'},
{ 'code' : '12856-1','description' : 'HIV 1 p65 Ab [Presence] in Serum by Immunoblot (IB) '},
{ 'code' : '12857-9','description' : 'HIV 1 p28 Ab [Presence] in Serum by Immunoblot (IB)'},
{ 'code' : '12858-7','description' : 'HIV 1 p32 Ab [Presence] in Serum by Immunoblot (IB)'},
{ 'code' : '12859-5','description' : 'HIV 1 p18 Ab [Presence] in Serum by Immunoblot (IB)'},
{ 'code' : '12870-2','description' : 'HIV 1 gp34 Ab [Presence] in Serum by Immunoblot (IB)'},
{ 'code' : '12871-0','description' : 'HIV 1 p26 Ab [Presence] in Serum by Immunoblot (IB)'},
{ 'code' : '12872-8','description' : 'HIV 1 p15 Ab [Presence] in Serum by Immunoblot (IB)'},
</code></pre></div><p>First, use the shortcut Control+f (Command+f on macOS) to bring up the Search bar.</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/find.webp" alt="The find bar of VS Code. A text field reads “Find”, with three buttons on the right side of the field reading “Aa”, “ab” underlined, and “.\*”. To the left of the text field is a right-facing arrow. To the right text reads “No results”."></p>
<p>The more advanced options for Find are the icons on the right hand side of the Find input field. Choose the Use Regular Expression option, indicated by <code>.*</code></p>
<p>With regex mode enabled, the “Find” field allows you to use a regex when searching. Ultimately, we want to replace any matches we get with text, or with nothing at all in order to delete our matches. To reveal the “Replace” field, hit the right-facing arrow on the left side of the Find bar.</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/find-and-replace.webp" alt="The same find bar, now with a second text field below reading “Replace”."></p>
<p>It can be easiest to find and replace in two phases: first, to do so on everything before the values we want to keep, and second, on everything after the values.</p>
<p>In this example, to match everything leading up to our values, we’ll use this regex:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">\{ 'code' : '
</code></pre></div><p>Because braces are special characters in regex, we have to escape the opening brace with a backslash to tell the regex engine that we want to match a literal <code>{</code>. Other than that, our input is a bunch of literal characters. With more varied inputs, you can of course turn to more complex regexes.</p>
<p>Once we enter that regex, we see that all characters leading up to our values are highlighted.</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/before-code-highlighted.webp" alt="A few lines from the earlier codes objects, with the characters “{ ‘code’ : ‘” highlighted in yellow. In the top right the Find/replace bar is visible with the prior regex in the Find field."></p>
<p>To replace these with nothing, simply place the cursor in the Replace input field and hit the Replace All icon to the right of the Replace input field, or hit Control+Enter (Command+Enter on macOS).</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/before-code-replace.webp" alt="The Find/Replace bar, this time with our regex. The Replace field is highlighted, with a popup reading “Replace All (⌘Enter)""></p>
<p>It should now look like this:</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/before-code-no-results.webp" alt="A few lines from the codes object, but the part which was highlighted has been deleted, so each line starts with the code directly. The find/replace bar now reads “no results” with the first regex in the Find field."></p>
<p>All of our lines now begin with the value we want to keep. Now, we’ll enter a regex that will match with everything after the last digits of the values. We can match an apostrophe and everything after that with the regex</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">'.*
</code></pre></div><p>Once we enter that regex, we’ll see that all characters following our values are highlighted</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/after-code-highlighted.webp" alt="The altered object with our new regex in the Find field. Everything after the codes is highlighted."></p>
<p>Once again, to replace all of these characters with nothing, we can place the cursor in the Replace input field and replace all again.</p>
<p>It should now look like this:</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/after-code-no-results.webp" alt="The new altered object. Every line has one code, and nothing else."></p>
<p>So we’ve isolated each individual segment on each line. Our final step is to get all of these segments onto the same line and separate with a pipe.</p>
<p>Let’s match the newline character, represented by <code>\n</code>.</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/newline-highlighted.webp" alt="The codes, the same as the previous image. The space partially overlapping the end of each line is highlighted."></p>
<p>Let’s replace these newlines with a pipe character. We enter the pipe character in the Replace field and replace all. It will result in this:</p>
<p><img src="/blog/2023/07/regex-replace-in-vs-code/pipe-separated.webp" alt="The codes are now all on the same line, separated by a “|” pipe character with no whitespace in between, like “33866-5|34591-8|35437-3|” et cetera."></p>
<p>Once we have this, we can place parentheses around the whole thing and put it into the WHERE clause of our SQL statement.</p>
<p>There you have it! You can imagine the possibilities with this. With decent knowledge of regex, you can use this as a power tool to manipulate text, isolate values that you want, and so on.</p>
Achieving Healthcare Data Interoperabilityhttps://www.endpointdev.com/blog/2023/07/achieving-healthcare-data-interoperability/2023-07-28T00:00:00+00:00Jarrett Smolarkiewicz
<p><img src="/blog/2023/07/achieving-healthcare-data-interoperability/a_stream_running_between_houses_and_a_road.webp" alt="An impressionist watercolor scene of two red-roofed farm houses with a tree-lined path leading forward on the left side of the image. The brushstrokes are very loose."></p>
<!-- Image: Johan Barthold Jongkind, A Stream Running between Houses and a Road, 19th century. Public domain under CC0. -->
<p>(Co-authored by <a href="/team/jon-jensen/">Jon Jensen</a>.)</p>
<h3 id="background">Background</h3>
<p>For many years, healthcare providers, public health experts, and software developers have been hard at work on standards and methods for interconnecting systems to share vast and ever-expanding patient and lab data.</p>
<p>In the late 1980s the non-profit organization Health Level Seven International released the HL7 data standard. That was updated many times up to the year 2007 with <a href="https://www.hl7.org/implement/standards/product_brief.cfm?product_id=144">HL7 v2.5.1</a> which is still widely used today.</p>
<p>New healthcare data standards have emerged, including <a href="https://www.hl7.org/implement/standards/product_section.cfm?section=14">HL7 v3</a>, the Consolidated Clinical Document Architecture (<a href="https://www.hl7.org/implement/standards/product_brief.cfm?product_id=492">C-CDA</a>), the Fast Healthcare Interoperability Resource (<a href="https://www.hl7.org/implement/standards/product_brief.cfm?product_id=491">FHIR</a>, pronounced “fire”), and more. These are not yet as broadly adopted as HL7 v2.5.1.</p>
<p>Under the Health Insurance Portability and Accountability Act of 1996 (<a href="https://www.hhs.gov/sites/default/files/ocr/privacy/hipaa/understanding/consumers/privacy-security-electronic-records.pdf">HIPAA</a>), most of this data is classified as PII (personally identifiable information) and/or PHI (personal health information) and must be kept secure and protected at all times, so that it will not be accessed without proper authorization or patient consent.</p>
<h3 id="the-data-dilemma">The data dilemma</h3>
<p>Healthcare data systems now routinely communicate in many ways, but there are sometimes incompatibilities when patient data travels from one system to another.</p>
<p>The U.S. Centers for Disease Control (CDC) encourages and sometimes incentivizes, but does not mandate, adoption and implementation of common standards. So each jurisdiction’s public health agency (PHA), lab, and vendor may be using different and only partially overlapping standards.</p>
<p>Different labs, doctors, and public health workers use varying conventions for things like facility names and lab codes, which presents problems for interpreting and reporting data. For example, “KU MAIN” may be recognized as the main campus of the University of Kansas Medical Center in nearby communities or states, while providers elsewhere may not recognize this code, or may mistake it for a similarly named facility in their area. The University of Kansas Health System consists of a large network of satellite facilities that span the states of Kansas and Missouri, rather than just one location in a single state. This may create additional confusion about which state jurisdictions should be involved.</p>
<p>This semantic category is called a vocabulary, and if two systems use incompatible vocabulary, one of them, or an intermediary, must have a mapping from the terms in one vocabulary to another.</p>
<p>The syntactic category of challenge is called a grammar, and covers the kinds of message types and actions a system works with.</p>
<p>An article published in December 2022 by Dr. Naheed Ali identifies “inconsistent data and lack of standardized data structure” as the number one issue to be overcome regarding the <a href="https://www.healthit.gov/faq/what-electronic-health-record-ehr">Electronic Health Record (EHR)</a> interoperability challenges facing healthcare professions today:</p>
<blockquote>
<p>One of the biggest EHR interoperability challenges in healthcare interoperability is managing inconsistent data from multiple sources. Information stored in different databases can have a variety of formats and data types that are not easily compatible with one another. A single record may contain different information about a patient’s medical history or treatment plan, making it more difficult for different systems to interpret correctly.
—Dr. Naheed Ali, <a href="https://www.ehrinpractice.com/ehr-interoperability-challenges-solutions.html">EHR Interoperability Challenges and Solutions</a></p>
</blockquote>
<h3 id="cooperating-for-a-healthier-future">Cooperating for a healthier future</h3>
<p>Although great strides have been made to digitize healthcare data and improve interoperability, much remains to be done. Newer standards (like HL7 FHIR, released in 2014) face an uphill battle for industry adoption to become as widely utilized as HL7 v2.5.1 (from 2007) and C-CDA (from 2011).</p>
<p>While networks of hospitals and laboratories typically set up EHR platforms built by companies like Cerner or Epic to manage patient data, public health agencies require additional applications to aid in the identification, tracking, and reporting of reportable health conditions from those patient EHRs.</p>
<p>Public health services in the United States are provided by various federal, state, tribal, territorial, regional, municipal, and other bodies that decide which software they will use to support their communities. Some look to software offered by the CDC or software firms to provide the application functions they require. Others contract with consulting firms to develop custom software. Still others opt to develop and maintain everything in house.</p>
<p>Getting so many diverse software systems to work together is difficult due to a phenomenon known as the quadratic cost problem. This occurs when the need for bi-directional connections between software applications increases exponentially, instead of linearly, with each component added to a system.</p>
<p>An organization called TechChange explains this concept and more in their video <a href="https://www.youtube.com/watch?v=KSEUh-wj7Y0">Standards and Interoperability in Digital Health: Explained</a>.</p>
<p>The book <a href="https://www.ncbi.nlm.nih.gov/books/NBK216088/">Patient Safety: Achieving a New Standard for Care</a> identifies in chapter 4 this concern:</p>
<blockquote>
<p>The fact that there is no standard means of representing the data for any of these datasets or requirements is astonishing and highlights the amount of unnecessary work performed by health care and regulatory organizations to prepare, transmit, and use what amount to custom reports.</p>
</blockquote>
<p>Nearly 20 years after those words were written, and almost a decade after the FHIR standard was introduced, the variety of standards and systems has not narrowed. Change is difficult.</p>
<h3 id="health-information-exchanges">Health Information Exchanges</h3>
<p>One architectural attempt to simplify data sharing is by connecting applications of each jurisdictional segment within a larger governing body to a centralized <a href="https://www.healthit.gov/topic/health-it-and-health-information-exchange-basics/what-hie">Health Information Exchange (HIE)</a>.</p>
<p>An example of this would be an individual state public health organization connecting its own software to an HIE, along with the software currently in use by each local jurisdiction within that state. Each jurisdiction does not have to use the same software, but can communicate by using its connection to the HIE rather than individual bi-directional connections to every other application used throughout the state.</p>
<p>This sounds like just what is needed, and likely is for some use cases.</p>
<p>But HIEs generally work at the syntactic (grammar) level. Many HIEs have agreements with the facilities they serve, that they will not change the content of messages in any way. Unfortunately that means most HIEs essentially work as a passthrough.</p>
<p>They thus are helpless when it comes to the wide variety of nonstandard content semantics that they encounter, and have no way of correcting vocabularies, so they pass on junk data.</p>
<p>Even a small, steady volume of unusable messages can at best clog up manual review queues and overwhelm already overworked staff, and at worst give a false sense that all data are being ingested and reported on, when they are not.</p>
<p>Something like an HIE that is capable of semantic (vocabulary) inspection, translation, and rule-based routing, customizable per reporter for the jurisdiction’s changing needs, is called for.</p>
<h3 id="the-casepointer-software-suite">The CasePointer software suite</h3>
<p>We at End Point are solving the interoperability problem for our client jurisdictions’ public health disease surveillance with <a href="https://www.casepointer.com">CasePointer</a>, a new suite of proven open source software. It interoperates with many other systems, including HIEs, and is configurable to map varying local data conventions into a single system that routes messages and reporting as needed.</p>
<h4 id="epitrax-disease-surveillance-system">EpiTrax disease surveillance system</h4>
<p>We began in 2008 to work with software for public health using the open-source TriSano disease surveillance system.</p>
<p>In 2017, the Utah Department of Health and Human Services (DHHS) released a new open source disease surveillance application called <a href="/expertise/epitrax/">EpiTrax</a> to replace their use of TriSano. EpiTrax is a more capable, comprehensive, and supportable application that supports <a href="https://ndc.services.cdc.gov/">all 170+ reportable health conditions</a> identified by the <a href="https://health.gov/healthypeople/objectives-and-data/data-sources-and-methods/data-sources/national-notifiable-diseases-surveillance-system-nndss">National Notifiable Disease Surveillance System (NNDSS)</a>.</p>
<p>As open source software, we and others can use it, extend its capabilities, and contribute back to a shared source code repository.</p>
<p>Today EpiTrax serves as the central application within the CasePointer suite.</p>
<h4 id="emsa-preprocessing-powerhouse">EMSA preprocessing powerhouse</h4>
<p>The separate application <a href="/expertise/emsa/">Electronic Message Staging Area (EMSA)</a> was also designed by the team at Utah DHHS to work in tandem with EpiTrax. EMSA ingests electronic case and lab reports (eCRs & ELRs) from various reporters and allows public health organizations to create extremely powerful custom configurations capable of message inspection, semantic (vocabulary) mapping and transformation, and routing, tailored to each jurisdiction’s needs and preferences.</p>
<p>Once these EMSA rules are set up, the healthcare data can flow continuously while automatically being pre-processed by EMSA according to its matching jurisdiction with minimal manual intervention by the area’s public health agency. PHA staff can focus on their primary objective: investigating and tracking disease to protect the health of their community.</p>
<p>Investigators can customize their individual dashboards within EpiTrax to analyze, sort, trace, and process the data prepared by EMSA in the way that is most efficient for them. Working together, these two powerhouse applications provide the cost and time saving benefits that public health organizations around the country need.</p>
<h4 id="nmi-automatic-cdc-reporting">NMI automatic CDC reporting</h4>
<p>Once a public health case reaches a certain status within EpiTrax, certain data must be reported per current requirements of the CDC. Historically, this has been a manual and tedious process for investigators involving referencing a text file with the latest CDC guidelines as they process these reports.</p>
<p>More recently, the <a href="https://www.cdc.gov/nndss/about/history.html">NNDSS Modernization Initiative (NMI)</a> by the CDC has provided options for a much more automated method of reporting the necessary case data. This is a critical part of the process for PHAs, as they receive funding for their organization based on their ability to meet CDC reporting requirements and interoperability standards.</p>
<p>The NMI module is integrated into the CasePointer workflow to ensure that once data reporting has been configured for a case, any updates will be automatically sent to the CDC without additional manual processing.</p>
<h4 id="casepointer-disease-reporting-portal">CasePointer Disease Reporting Portal</h4>
<p>During COVID-19, nontraditional reporters such as pharmacies, university health centers, community drive-up testing centers, and others not part of our state clients' formal public health infrastructure, needed to be able to submit test results to the state.</p>
<p>We developed a specialized disease reporting portal, initially focusing on COVID-19, to accept simple batch uploads of test results and, based on changing policy, to forward test data to the disease surveillance system. This separate system had no access to the main EpiTrax disease surveillance system, or to any other reporters' data.</p>
<h3 id="bringing-it-all-together">Bringing it all together</h3>
<p>CasePointer provides two prominent strengths:</p>
<ul>
<li>First, its ability to map the endless local conventions used by those reporting the data into a unified standard for the rest of the workflow.</li>
<li>Second, the customizability and agility of the CasePointer applications are unique among available disease surveillance solutions.</li>
</ul>
<p>CasePointer puts the ability to customize data collection and surveillance in the hands of the public health experts. New fields can be added into an investigation form with the click of a button. The ability to quickly pivot and document disease information is critical in a quickly changing public health landscape.</p>
<h3 id="join-the-evolution-of-public-health-technology">Join the evolution of public health technology</h3>
<p>Such highly configurable, unified, robust tools can be a game changer for public health jurisdictions in need of an affordable, extensible solution that meet all their needs. CasePointer can even work to further streamline systems that are already integrated with an HIE.</p>
<p>Within the United States, four states have already partnered with us to implement EpiTrax. We are helping them reduce unnecessary manual data processing costs, more effectively manage caseloads, and further automate their data workflows.</p>
<p>If this piqued your curiosity, visit <a href="https://www.casepointer.com/">our CasePointer website</a> and <a href="https://www.casepointer.com/contact/">contact us</a> to learn more about CasePointer and how our team is evolving public health support with customized data solutions.</p>
<p>We would love to support your efforts to protect the well-being of people and communities you care about!</p>
SSH and Old Servershttps://www.endpointdev.com/blog/2023/07/ssh-and-old-servers/2023-07-21T00:00:00+00:00Josh Ausborne
<p><img src="/blog/2023/07/ssh-and-old-servers/seashell-beach.webp" alt="A photo of a whole bivalve seashell lying open and flat on the sandy ground."></p>
<!-- Photo by Josh Ausborne -->
<p>We recently updated one of our backup servers from <a href="https://www.oracle.com/linux/">Oracle Linux</a> 8 to Oracle Linux 9, a free rebuild of <a href="https://www.redhat.com/en/technologies/linux-platforms/enterprise-linux">Red Hat Enterprise Linux</a> 9, also known as RHEL 9. As with any newer OS, there are bound to be unexpected little differences which crop up and which need to be handled.</p>
<p>In the case of our backup server, we found that it could no longer SSH to older servers, which includes a couple running CentOS 5 (yes, there really are a few of those out in the wild), some CentOS 6, and one Debian 8. We saw similar connection problems when we first went from CentOS 7 to Oracle Linux 8, and we addressed the issues by creating a Host entry for the servers in our <code>~/.ssh/config</code> file.</p>
<p>Here is an example of of the entry that we used previously:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Host old-server-1 old-server-2 old-server-3
User root
Hostname %h.example.com
KexAlgorithms +diffie-hellman-group-exchange-sha1
</code></pre></div><p>This worked fine for a couple of years until we upgraded from Oracle Linux 8 to 9. At this point we started to see a different problem when trying to connect.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Unable to negotiate with <nnn.nnn.nnn.nnn> port 22: no matching host key type found. Their
offer: ssh-rsa,ssh-dss
</code></pre></div><h3 id="troubleshooting">Troubleshooting</h3>
<p>One of the good troubleshooting methods for diagnosing SSH connectivity problems is to enable some verbosity when connecting. This can be done via the use of the <code>-v</code> option, which can be stacked up to three times for increasing levels of detail. In this case, I used <code>ssh -vvv $hostname</code>, which is the highest level, and I was able to see another error message in the copious output:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">ssh_dispatch_run_fatal: Connection to <nnn.nnn.nnn.nnn> port 22: error in libcrypto
</code></pre></div><p>It was at this time that I had to consult with The Fountain of Knowledge (the Internet) to come up with a solution. There were some suggestions to add <code>HostKeyAlgorithms</code> or <code>PubkeyAcceptedKeyTypes</code> entries for those hosts in the local SSH config, but that didn’t work in this case.</p>
<p>What I did find, though, is that RHEL 9-based OSes now forbid via crypto policy the ability to connect using SHA-1, but SHA-1 is the default crypto policy in use by CentOS 5 and CentOS 6! To get around this, I needed to add SHA-1 support to our existing policy. You can compare the crypto policies for <a href="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/using-the-system-wide-cryptographic-policies_security-hardening">RHEL 8 systems</a> and <a href="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/security_hardening/using-the-system-wide-cryptographic-policies_security-hardening">RHEL 9 systems</a> to see which is right for your use case.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">update-crypto-policies --set DEFAULT:SHA1
</code></pre></div><p>Here I also found a bit of info on <a href="https://access.redhat.com/documentation/fr-fr/red_hat_enterprise_linux/9/html/security_hardening/proc_re-enabling-sha-1_using-the-system-wide-cryptographic-policies">re-enabling SHA-1</a>.</p>
<h3 id="a-problem-for-every-solution">A problem for every solution</h3>
<p>One more problem that we ran into with one of the CentOS 5 servers is that we were getting a different SSH failure even after having updated the crypto policy. This new problem was about the length of the host key on the old remote server.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Bad server host key: Invalid key length
</code></pre></div><p>That is because RHEL 9 by default doesn’t like short RSA host keys. Thankfully, there is an option that can be enabled at the SSH client level to help us overcome the limitation. It is solved in the usual manner of adding an option to the <code>~/.ssh/config</code> file. The option is known as <code>RequiredRSASize</code>, and it specifies the minimum length of RSA key that is required to be considered valid.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Host old-server-1 old-server-2 old-server-3
User root
Hostname %h.example.com
KexAlgorithms +diffie-hellman-group-exchange-sha1
RequiredRSASize 1024
</code></pre></div><h3 id="perfect--enough">Perfect … enough</h3>
<p>Obviously we don’t want to let old systems just sit out there and keep doing their thing. They increase the attack surface that hackers can target, and it’s important to stay on top of updating and upgrading. In an ideal world, we’d prefer that we simply replace and modernize the old, thus securing everything, and deal with the pain of upgrading all at one time. We like to just get the headache over with.</p>
<p>In the real world, though, some old systems just have to stay in place for a little longer (or “much longer” in this case) than we’d prefer. It’s helpful to figure out acceptable ways to work around the limitations when these issues arise, while wherever possible making the exceptions only for the specific old hosts we want to tolerate.</p>
CSTE Conference 2023 Retrospectivehttps://www.endpointdev.com/blog/2023/07/cste-conference-2023-retrospective/2023-07-14T00:00:00+00:00Samuel Stern
<p><img src="/blog/2023/07/cste-conference-2023-retrospective/end-point-cste-team.webp" alt="Seven members of the End Point CasePointer team standing in front of the CSTE 2023 press wall"></p>
<p>End Point is proud to have been an exhibitor at the 2023 Council of State and Territorial Epidemiologists (CSTE) conference in Salt Lake City, Utah.</p>
<p>We are grateful to have had the opportunity to showcase our public health software suite offering, CasePointer, to numerous state, tribal, territorial, and international epidemiologists.</p>
<p>The conference served as a meeting place to hear about the latest research in the fields of informatics and disease surveillance, and discuss the future of data management on state and national levels, with many talks exploring the technologies that will become future standards and practices.</p>
<p>The End Point booth was abuzz with visitors getting hands-on demos using the EpiTrax software as well as learning about the other aspects of the CasePointer suite, such as the Electronic Message Staging Area (EMSA), our new disease reporting portal, and the EpiTrax NMI module.</p>
<p><img src="/blog/2023/07/cste-conference-2023-retrospective/20230625_124940.webp" alt="Trade show booth for CasePointer by End Point, featuring EpiTrax"></p>
<p>The CasePointer demo highlighted its capability of streamlining the management of epidemiological data with a user-friendly and highly customizable interface.</p>
<p>The CSTE conference provides a valuable opportunity to engage with epidemiologists and gain insights into the daily obstacles they encounter in data reception, management, and sharing. It offers a platform to learn about their strategies for overcoming these challenges within their respective jurisdictions.</p>
<p>One of the common topics that came up was tackling the challenge of file transfers using current protocols and systems in place. We were happy to share our successes using EMSA to customize how data is received and uploaded to EpiTrax, making transfers faster, more reliable, and easier to manage.</p>
<p><img src="/blog/2023/07/cste-conference-2023-retrospective/20230627_154336.webp" alt="Booth close-up with flyers, pens, etc. offered by End Point staffer whose name Samuel shows on his conference lanyard"></p>
<p>We look forward to further conversations with the visitors that stopped by our booth, and hope that you enjoyed the locally crafted chocolates and caramels and saved some for the trip home!</p>
<p>Are you interested in a demo of the CasePointer suite? Reach out to us at <a href="mailto:ask@endpointdev.com">ask@endpointdev.com</a> to set up a meeting.</p>
Unix tools & tipshttps://www.endpointdev.com/blog/2023/07/unix-tools/2023-07-05T00:00:00+00:00Muhammad Najmi bin Ahmad Zabidi
<p><img src="/blog/2023/07/unix-tools/idaho-mountains.webp" alt="The view from a mountain ridge. the sky is light blue and partially covered in clouds. Green ridges covered in pine trees lead down to a flat valley populated by a small town."></p>
<!-- Photo by Seth Jensen, 2023. -->
<p>The command line interface (CLI) programs/tools found on UNIX-like (*NIX) systems are among the most useful tools available to system administrators. CLI tools allow a system administrator to handle the *NIX systems either remotely or locally without needing to install Graphical User Interface (GUI) packages.</p>
<p>Almost twenty years ago, I bought two O’Reilly books: “sed & awk, second edition,” written by Dale Dougherty & Arnold Robbins, and “Mastering Regular Expressions,” written by Jeffrey E.F. Friedl. These two books were printed in the late ’90s, and I bought them in 2004. I remembered them recently when my co-workers and I held a few study sessions to learn regular expressions. When a page and chapter are mentioned in this post, I’m referring to the editions of these books printed in that particular year.</p>
<p>In this blog post, I’ll detail some common use cases I encounter day to day, as well as some related tools that help me handle them.</p>
<h3 id="sed">sed</h3>
<p>During my regular work as a system administrator, I usually use <strong>sed</strong> (“<strong>s</strong>tream <strong>ed</strong>itor”) to do string replacement across files and <strong>awk</strong> for log files or file analysis with arbitrary strings as the input field separator using the -F flag.</p>
<p>One common use case for sed I’ve found is updating config files for the Icinga monitoring system. Let’s say that we have a file named server101.cfg and we want to use the same config for server 102. One way to solve this on the command line is doing a simple search and replace with sed:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ cp server101.cfg server102.cfg
$ sed 's/server101/server102/g' server102.cfg
$ icinga -v /etc/icinga/icinga.cfg # This will check that Icinga is able to understand the newly copied file
</code></pre><p>Make sure you use diff or another similar tool to make sure this doesn’t accidentally modify anything it’s not supposed to. You can get more detailed with your regular expressions to avoid such issues, but that’s beyond the scope of this post.</p>
<h3 id="awk">awk</h3>
<p>Sometimes I need to use awk for scripting, but most of the time, I use it for parsing things like IP addresses in log files, or any other string in logs. For example, you can check for plaintext passwords or credit card details in places they’re not supposed to be.</p>
<p>The sed & awk book started with the introduction of ed, an ancient line editor. Then it touched on the field separator, which is represented by the <code>-F</code> flag.</p>
<p>In the following example, I ran the <code>blkid</code> command in order to check the UUID of the SCSI devices that connected to my computer, then used awk to only show the output I needed.</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ sudo blkid /dev/sd{d,e,g}
/dev/sdd: UUID="7eb6302f-e727-4433-8c49-8a7842d18e1e" TYPE="crypto_LUKS"
/dev/sde: UUID="68b2382e-13b8-4bdb-a6cb-15f6844d464b" TYPE="crypto_LUKS"
/dev/sdg: UUID="7237cc7d-0483-4c2a-a503-a11ea88b3690" TYPE="crypto_LUKS"
</code></pre><pre tabindex="0"><code class="language-console" data-lang="console">$ sudo blkid /dev/sd{d,e,g} | awk -F "\"" {'print $2'}
7eb6302f-e727-4433-8c49-8a7842d18e1e
68b2382e-13b8-4bdb-a6cb-15f6844d464b
7237cc7d-0483-4c2a-a503-a11ea88b3690
</code></pre><p>During my day-to-day work I seldom use both <code>sed</code> and <code>awk</code> together, but there are many situations where it can be useful to do so. See page 23 of the sed & awk book for one such example where both of these tools can be used together.</p>
<h3 id="using-regular-expressions-in-nix-tools">Using regular expressions in *NIX tools</h3>
<p>The sed & awk book touches on regular expression syntax, since system administrators might have to use regular expressions in cases where large/repetitive tasks are involved. I do not remember who suggested buying this pair of books, but I am really grateful for the person’s suggestion.</p>
<p>To search a file using regular expressions, I use the program “grep.” I often use the <code>-r</code> flag for recursive parsing, the <code>-w</code> flag to match whole words only, and the <code>--color=always</code> flag to colorize the terminal output. Recent versions of GNU grep use the <code>--color=auto</code> option by default so I don’t have to explicitly specify it when sending output straight to a terminal. We can also use <code>--color=never</code> if we want to remove the color matching.</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ grep 1 -w --color=never tmp/file.txt
Mon Aug 1 11:30:01 AM +08 2022
Thu Sep 1 11:30:01 AM +08 2022
Sat Oct 1 11:30:01 AM +08 2022
Tue Nov 1 11:30:01 AM +08 2022
Thu Dec 1 11:30:01 AM +08 2022
Sun Jan 1 11:30:01 AM +08 2023
Wed Feb 1 11:30:01 AM +08 2023
Wed Mar 1 11:30:01 AM +08 2023
</code></pre><p>To extend the capability of grep for regular expression’s usage, I used <code>-E</code>, but then I learned about the <code>-P</code> flag, which lets you use Perl-compatible regular expressions (PCREs), which I prefer.</p>
<p>Say I have a file named <code>fruit.txt</code> with the following contents:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">apple banana orange
apple orange papaya
durian rambutan
starfruit
</code></pre></div><p>To find lines starting with “durian”:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ grep -P '^durian' fruit.txt
durian rambutan
</code></pre><p>To find lines ending with “papaya”:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ grep -P "papaya$" fruit.txt
apple orange papaya
</code></pre><p>To find lines with three words separated by two spaces:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ grep -P '\w+ \w+ \w+' fruit.txt
apple banana orange
apple orange papaya
</code></pre><p>To find all lines containing a word starting with “a” or “d”:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ grep -P '\b(a|d)\w+' fruit.txt
apple banana orange
apple orange papaya
durian rambutan
</code></pre><p>These are just a few examples of what <code>grep</code> can do; it’s very capable especially when you use PCREs.</p>
<p>Apart from sed, awk and grep, there are many other useful tools for text processing. Some that I commonly use are the sort and uniq programs.</p>
<h3 id="sort">sort</h3>
<p>sort and uniq are frequently used together. sort, as its name implies, is used to sort the parsed text per whatever options are provided.</p>
<p>Assume that we have the following file, “prices.csv”:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ cat prices.csv
chicken,$1.50
duck,$1.20
beef,$6.10
chicken,$1.50
lamb,$3.20
fish,$4.09
</code></pre><p>Using the <code>sort</code> command, we could sort the items according a specified text separator (<code>-t</code>) and the column (key) that we want to prioritize (<code>-k</code>). Here, I want to sort the item based on data in the second column:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ sort -t "," -k 2 prices.csv
duck,$1.20
chicken,$1.50
chicken,$1.50
lamb,$3.20
fish,$4.09
beef,$6.10
</code></pre><p>Without the other parameters, sort will just sort the lines alphabetically.</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ sort prices.csv
beef,$6.10
chicken,$1.50
chicken,$1.50
duck,$1.20
fish,$4.09
lamb,$3.20
</code></pre><p>We can use uniq to eliminate repeated lines. Running <code>uniq</code> without sorting will have no effect, since the duplicate lines <code>chicken,$1.50</code> do not appear sequentially:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ uniq prices.csv
chicken,$1.50
duck,$1.20
beef,$6.10
chicken,$1.50
lamb,$3.20
fish,$4.09
</code></pre><p>To merge them and clean up the data, we can sort first:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ sort prices.csv | uniq
beef,$6.10
chicken,$1.50
duck,$1.20
fish,$4.09
lamb,$3.20
</code></pre><h3 id="find">find</h3>
<p>Say I want to search a full hard disk to check for files that are not needed anymore. I would use the following command, with each option detailed below the output:</p>
<pre tabindex="0"><code class="language-console" data-lang="console">$ find . -maxdepth 1 -mtime +3000 -type d -exec ls {} -ld \;
drwxrwxrwx 2 najmi najmi 4096 Nov 25 2012 ./Terminal
drwxrwxrwx 3 najmi najmi 4096 Dis 27 2012 ./gnome-disk-utility
drwxrwxrwx 8 najmi najmi 4096 Nov 27 2012 ./xfce4
drwxrwxrwx 2 najmi najmi 4096 Nov 25 2012 ./libaccounts-glib
drwxrwxrwx 2 najmi najmi 4096 Dis 1 2012 ./sakura
drwxrwxrwx 2 najmi najmi 4096 Nov 25 2012 ./software-center
drwxrwxrwx 4 najmi najmi 4096 Nov 25 2012 ./evolution
drwxrwxrwx 2 najmi najmi 4096 Nov 25 2012 ./update-notifier
drwxrwxrwx 2 najmi najmi 4096 Dis 22 2012 ./Thunar
drwxrwxrwx 3 najmi najmi 4096 Nov 25 2012 ./compiz-1
drwxrwxrwx 2 najmi najmi 4096 Dis 28 2012 ./tracker
drwxrwxrwx 3 najmi najmi 4096 Dis 9 2012 ./menus
drwxrwxrwx 3 najmi najmi 4096 Dis 27 2012 ./gnome-control-center
drwxrwxrwx 2 najmi najmi 4096 Nov 25 2012 ./goa-1.0
drwxrwxrwx 2 najmi najmi 4096 Nov 28 2012 ./enchant
drwxrwxrwx 2 najmi najmi 4096 Dis 8 2012 ./Pinta
drwxrwxrwx 2 najmi najmi 4096 Nov 25 2012 ./gmusicbrowser
drwxrwxrwx 3 najmi najmi 4096 Dis 14 2012 ./audacious
drwxrwxrwx 3 najmi najmi 4096 Dis 27 2012 ./gnome-session
drwxrwxrwx 5 najmi najmi 4096 Nov 28 2012 ./mate
drwxrwxrwx 2 najmi najmi 4096 Nov 25 2012 ./ristretto
drwxrwxrwx 31 najmi najmi 4096 Dis 24 2012 ./chromium
drwxrwxrwx 5 najmi najmi 4096 Dis 8 2012 ./mono.addins
drwxrwxrwx 3 najmi najmi 4096 Nov 25 2012 ./mate-session
drwxrwxrwx 3 najmi najmi 4096 Nov 25 2012 ./ibus
drwxrwxrwx 4 najmi najmi 4096 Nov 25 2012 ./libreoffice
drwxrwxrwx 3 najmi najmi 4096 Nov 25 2012 ./caja
drwxrwxrwx 2 najmi najmi 4096 Dis 18 2012 ./Empathy
</code></pre><ul>
<li>In this case I search in the current directory with the dot <code>.</code> notation.</li>
<li>The <code>-maxdepth 1</code> option means the search task must not be done beyond the current directory.</li>
<li><code>-mtime +3000</code> means only show directories that have not been modified within the last 3000 days.</li>
<li><code>-type d</code> means match directories, but not files (to only match files, use the <code>-type f</code> flag).</li>
<li>The <code>-exec ls {} -ld \;</code> arguments mean it will run the <code>ls -ld</code> command for each match. This will show the modification time of the directory passed by the <code>{}</code> placeholder, which is replaced by the file name of the current search result.</li>
</ul>
<blockquote>
<p>Note: In Linux with GNU tools it is possible to order the flags like either <code>ls {} -ld</code> or <code>ls -ld {}</code>, but I realized that in some other *NIX variants, you always need to put the flag before the variable/parameter; that is, <code>ls -ld {}</code>.</p>
</blockquote>
<h3 id="conclusion">Conclusion</h3>
<p>There are newer tools which may be provided by more recent Linux distros or BSD variants, but in this short write-up I wanted to stick to the basics which are useful to anyone using *NIX. It is more than worth your time to learn these tools in detail.</p>
<blockquote>
<p>For further reading, I would also recommend “UNIX and Linux System Administration Handbook” by Nemeth, E., Snyder, G., Hein, T., Whaley, B., & Mackin, D. (2020).</p>
</blockquote>
Rocky Linux 9 at Hetzner Robot Made Quick and Easyhttps://www.endpointdev.com/blog/2023/07/rocky-linux-9-at-hetzner-robot-made-quick-and-easy/2023-07-01T00:00:00+00:00Jeffry Johar
<p><img src="/blog/2023/07/rocky-linux-9-at-hetzner-robot-made-quick-and-easy/mangopickles.webp" alt="Malaysian Mango Pickles"><br>
Image <a href="https://www.pexels.com/photo/mango-pickles-17315505/">by Jeffry Johar</a></p>
<h3 id="introduction">Introduction</h3>
<p>In <a href="/blog/2023/06/rocky-linux-9-at-hetzner-robot-for-the-impatient/">my last blog post</a>, I shared my experience of installing Rocky Linux 8 on my Hetzner robot server and subsequently upgrading it to Rocky Linux 9.</p>
<p>Rocky Linux project manager <a href="https://github.com/brianclemens">Brian Clemens</a> said that method is not recommended and suggested using a boot kickstart for an automated installation. Thanks, Brian!</p>
<p>Despite my attempts to utilize kickstart, I encountered difficulties in booting my NVMe disk. During this process, I discovered another workaround for installing Rocky 9 using the installimage script. This method is also experimental, just like the previous one.</p>
<h3 id="the-steps">The Steps</h3>
<ol>
<li>
<p>Access the rescue mode (refer to <a href="/blog/2023/06/rocky-linux-9-at-hetzner-robot-for-the-impatient/#enabling-rescue-mode">my previous blog post</a> if you need guidance).</p>
</li>
<li>
<p>Copy the existing Rocky Linux 9 image to Rocky Linux 8 and place it in the root directory with the following command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">cp /root/images/Rocky-91-amd64-base.tar.gz /root/Rocky-87-amd64-base.tar.gz
</code></pre></div></li>
</ol>
<p>The filename should adhere to the required naming convention for the installimage script to function correctly.</p>
<ol start="3">
<li>
<p>Launch the <code>installimage</code> and select the Custom Image option.</p>
<p><img src="/blog/2023/07/rocky-linux-9-at-hetzner-robot-made-quick-and-easy/installimage.webp" alt="installimage"></p>
</li>
<li>
<p>Configure the disk settings as required, and at the end of the script, select the image accordingly.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">IMAGE /root/Rocky-87-amd64-base.tar.gz
</code></pre></div></li>
<li>
<p>To save the configuration file, simply press F10, allowing the installation process to resume uninterrupted. You may encounter a warning indicating the absence of an image signature, which is perfectly normal. Once the installation is finished, proceed to reboot the system.</p>
<p><img src="/blog/2023/07/rocky-linux-9-at-hetzner-robot-made-quick-and-easy/complete.webp" alt="installation completes with no error"></p>
</li>
<li>
<p>After the system restarts, you will have access to Rocky Linux 9. The installation will provide you with some fairly recent point-release of Rocky Linux, such as 9.1, but as always you need to update to the latest packages with <code>dnf update</code>.</p>
</li>
</ol>
<h3 id="conclusion">Conclusion</h3>
<p>That’s all, folks! This second method is much easier compared to the first one, as it swiftly takes you to Rocky 9 without any complications.</p>
Migrating Rails 6 React to Rails 7 Reacthttps://www.endpointdev.com/blog/2023/06/migrating-rails6-react-rails7-react/2023-06-26T00:00:00+00:00Indra Pranesh Palanisamy
<p><img src="/blog/2023/06/migrating-rails6-react-rails7-react/pexels-indra-pranesh-palanisamy-17019134.webp" alt="A coconut tree stands in the corner of a serene blue sky"></p>
<!-- Photo by Indra Pranesh Palanisamy, 2022 -->
<p>CasePointer’s <a href="https://www.casepointer.com/our-suite/disease-reporting-portal/">disease reporting portal</a> is built on React and Rails 6, and it’s time for an upgrade to Rails 7. This blog post will cover the steps, benefits, and challenges of migrating from Rails 6 to Rails 7, and offer valuable insights into the world of Ruby on Rails.</p>
<p>With the recent release of Rails 7, there are many new features and improvements to explore.
One of the biggest changes in Rails 7 is the retirement of Webpacker in favor of using the native webpack for bundling JavaScript.</p>
<blockquote>
<p>For those who are not familiar, Webpacker is a Rails gem which is a wrapper around the webpack build system that provides a standard webpack configuration and reasonable defaults.</p>
</blockquote>
<h3 id="steps-for-migrating-rails-6-react-to-rails-7-react">Steps for migrating Rails 6 React to Rails 7 React</h3>
<p>To migrate a Rails 6 React application to Rails 7 React, follow these steps:</p>
<h4 id="1-update-the-rails-gem-in-the-gemfile">1. Update the Rails Gem in the Gemfile</h4>
<p>In your application’s Gemfile, update the Rails gem version to Rails 7:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#fdd">-gem "rails", "~> 6.1.4"
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+gem "rails", "~> 7.0.0"
</span></code></pre></div><h4 id="2-upgrade-rails-packages">2. Upgrade Rails packages</h4>
<p>Upgrade the Rails packages using Yarn:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">yarn upgrade @rails/actioncable --latest
yarn upgrade @rails/activestorage --latest
</code></pre></div><h4 id="3-run-the-rails-update-task">3. Run the Rails update task</h4>
<p>Run the following command to initiate the Rails update task:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">bin/rails app:update
</code></pre></div><p>This task guides you file by file to integrate the new Rails 7 defaults. Refer to the <a href="https://guides.rubyonrails.org/upgrading_ruby_on_rails.html">Rails documentation</a> for detailed information on this update task.</p>
<h4 id="4-remove-webpacker">4. Remove Webpacker</h4>
<p>Since Webpacker is no longer the default in Rails 7, follow these steps to remove it:</p>
<p>Remove the webpacker gem from your Gemfile:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#fdd">-gem 'webpacker', '~> 4.0'
</span></code></pre></div><p>Remove the <code>webpacker.yml</code> file and any other files associated with webpacker.</p>
<p>Run <code>bundle install</code> to update the Gemfile.lock:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">bundle install
</code></pre></div><h4 id="5-setting-up-webpack">5. Setting up Webpack</h4>
<p>To set up Webpack for your Rails 7 React application, follow these steps:</p>
<p>Install webpack and other necessary libraries:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">yarn install webpack webpack-cli @babel/core
</code></pre></div><p>Create a webpack.config.js file at the root of your application:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#888">// webpack.config.js
</span><span style="color:#888"></span>
<span style="color:#080;font-weight:bold">const</span> path = require(<span style="color:#d20;background-color:#fff0f0">"path"</span>);
<span style="color:#080;font-weight:bold">const</span> webpack = require(<span style="color:#d20;background-color:#fff0f0">"webpack"</span>);
<span style="color:#888">// Add other necessary plugins and configurations
</span><span style="color:#888"></span>
module.exports = {
<span style="color:#888">// Webpack configuration options
</span><span style="color:#888"></span> output: {
<span style="color:#888">// Make sure to use the path of the rails asset pipeline
</span><span style="color:#888"></span> path: path.join(<span style="color:#00d;font-weight:bold">__</span>dirname, <span style="color:#d20;background-color:#fff0f0">'/app/assets/builds'</span>),
<span style="color:#888">// ...
</span><span style="color:#888"></span> }
module: {
rules: [
<span style="color:#888">// Add CSS/SASS/SCSS rule with loaders
</span><span style="color:#888"></span> {
test: <span style="color:#080;background-color:#fff0ff">/\.(?:sa|sc|c)ss$/i</span>,
use: [MiniCssExtractPlugin.loader, <span style="color:#d20;background-color:#fff0f0">'css-loader'</span>, <span style="color:#d20;background-color:#fff0f0">'sass-loader'</span>],
},
{
test: <span style="color:#080;background-color:#fff0ff">/\.(png|jpe?g|gif|eot|woff2|woff|ttf|svg)$/i</span>,
use: <span style="color:#d20;background-color:#fff0f0">'file-loader'</span>,
},
{
test: <span style="color:#080;background-color:#fff0ff">/\.(js|jsx)$/</span>,
exclude: <span style="color:#080;background-color:#fff0ff">/node_modules/</span>,
use: {
loader: <span style="color:#d20;background-color:#fff0f0">"babel-loader"</span>
}
}
],
},
<span style="color:#888">// ...
</span><span style="color:#888"></span>};
</code></pre></div><p>Add a .babelrc file for Babel configuration:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"presets"</span>: [<span style="color:#d20;background-color:#fff0f0">"@babel/preset-env"</span>, <span style="color:#d20;background-color:#fff0f0">"@babel/preset-react"</span>]
}
</code></pre></div><p>Update the scripts section in your package.json:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"scripts"</span>: {
<span style="color:#b06;font-weight:bold">"build"</span>: <span style="color:#d20;background-color:#fff0f0">"webpack"</span>,
<span style="color:#b06;font-weight:bold">"dev"</span>: <span style="color:#d20;background-color:#fff0f0">"webpack --watch"</span>
}
}
</code></pre></div><h4 id="6-update-rails-asset-tags">6. Update Rails asset tags</h4>
<p>In your application views, update the asset tags from <code>stylesheet_pack_tag</code> and <code>javascript_pack_tag</code> to <code>stylesheet_link_tag</code> and <code>javascript_include_tag</code>, respectively:</p>
<pre tabindex="0"><code class="language-erb" data-lang="erb"><%= stylesheet_link_tag 'application', media: 'all', 'data-turbolinks-track': 'reload' %>
<%= javascript_include_tag 'application', 'data-turbolinks-track': 'reload' %>
</code></pre><h4 id="7-start-the-dev-server">7. Start the dev server</h4>
<p>Start the Rails development server with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">rails server
</code></pre></div><p>Then start the React development server with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">yarn run dev
</code></pre></div><p>By following these steps, we can take advantage of the new features and improvements in Rails 7 while continuing to leverage the power of React.</p>
Announcing CasePointerhttps://www.endpointdev.com/blog/2023/06/announcing-casepointer-epitrax/2023-06-20T00:00:00+00:00Jon Jensen
<p><img src="/blog/2023/06/announcing-casepointer-epitrax/pexels-diva-plavalaguna-6147381.webp" alt="Overhead view of 4 people standing on opposite sides of a table holding huge puzzle pieces"><br>
Photo by <a href="https://www.pexels.com/photo/close-up-photo-of-people-holding-puzzle-pieces-6147381/">Diva Plavalaguna</a>, Pexels license</p>
<p>End Point is pleased to announce CasePointer, the expanded branding of our line of public health systems and services. Before this we referred to it simply by the name of one of its components, EpiTrax.</p>
<p>The name CasePointer more appropriately reflects the broad scope of services we provide within the disease reporting and surveillance sector.</p>
<p>This change coincides with the launching of the dedicated CasePointer website at <a href="https://www.casepointer.com/">www.casepointer.com</a> and our participation next week in this year’s conference of CSTE (Council of State and Territorial Epidemiologists) in Salt Lake City, Utah. We will be staffing a booth there on the exhibition floor. Come visit us!</p>
<p>Starting in 2008, End Point helped develop and maintain the TriSano open source disease surveillance system. In 2019 we migrated our customer, the Kansas Department of Health and Environment, from TriSano to a new disease surveillance system called EpiTrax, which was developed by the state of Utah, and is also open source.</p>
<p>This timeline shows those and other key points on our journey:</p>
<p><object style="max-width:100%" type="image/svg+xml" data="/blog/2023/06/announcing-casepointer-epitrax/casepointer-timeline-20230620.svg"></object></p>
<p>Due to the need for major additional features and capabilities, especially since the onset of the COVID-19 pandemic, End Point is now integrally involved with public health agencies in four states. For each of them we configured, deployed, maintain, and enhance the cloud hosting, relevant software, and databases to run those components they need, including:</p>
<ul>
<li>the EpiTrax disease surveillance system</li>
<li>EMSA (the Electronic Message Staging Area)</li>
<li>the NMI (NNDSS Modernization Initiative) module</li>
<li>End Point’s Disease Reporting Portal</li>
<li>End Point’s merge tool</li>
<li>Mirth Connect</li>
</ul>
<p>All of these interoperate harmoniously around the central EpiTrax application and database.</p>
<p>Our team is composed of public health professionals, software engineers, database specialists, and security experts. We are committed to our public health clients and their needs. We are here to help you help our communities and look forward to discussing the CasePointer approach with you and your team.</p>
Using EndeavourOS as a daily driverhttps://www.endpointdev.com/blog/2023/06/using-endeavouros-as-daily-driver/2023-06-16T00:00:00+00:00Muhammad Najmi bin Ahmad Zabidi
<p><img src="/blog/2023/06/using-endeavouros-as-daily-driver/freeway-signs.webp" alt="A nearby mountain skyline lies against an overcast sky. The lower hills are obscured on the edges by close green trees in the foreground, which lead toward US freeway signs in storage."></p>
<!-- Photo by Seth Jensen, 2023 -->
<h3 id="choosing-a-new-distro">Choosing a new distro</h3>
<p>I have been using Manjaro Linux (based on Arch Linux) for my work desktop and Pop!_OS (based on Ubuntu) for my laptop for quite some time now, and wanted to find another Arch Linux-based distro for my daily driver. Rather than aimlessly searching through the hundreds of Linux distros out there, I made several requirements the chosen distro should meet:</p>
<ul>
<li>Easy to maintain.</li>
<li>Has strong community support and stable development.</li>
<li>Has a Linux Unified Key Setup (LUKS) full-disk encryption option in the installation.</li>
<li>If possible, the distro of choice should be able to trim down the number of default packages. For example, I am a Gnome desktop user, so I don’t want to install the game packages I won’t use.</li>
</ul>
<p>I decided to use <a href="https://endeavouros.com">EndeavourOS</a>. It supports the features that I need: It has a strong community and most of Arch Linux’s documentation/references are still applicable to it, like they are to most Arch Linux-based distros using <code>systemd</code>.</p>
<p>The most notable differences between EndeavourOS and Arch Linux are EndeavourOS’s GUI-based installer—we can select between installing XFCE, Plasma KDE, Gnome, and several others desktop environments—as well as wallpapers and some other tools, for instance, EndeavourOS QuickStart Installer and EndeavourOS log tools.</p>
<p>Also, EndeavourOS notifies the user when there are pending OS updates, making it easy to keep my box up to date, especially when I am busy during the week and forget to apply important OS updates!</p>
<h3 id="installation">Installation</h3>
<p>EndeavourOS is quite easy to install. The Linux Unified Key Setup (LUKS) encryption option is available during the installation process. (Any distro using the <a href="https://calamares.io">Calamares</a> installer tool will also have this feature.)</p>
<p>After the basic installation, I usually need to do some other customizations before it’s suitable for daily usage.</p>
<h3 id="starting-services">Starting services</h3>
<p>One thing that I noticed is that by default the cron service is not enabled. After checking online, I found that I have to enable the <code>cronie</code> service first.</p>
<p>Bluetooth is another service that isn’t running by default. If you want Bluetooth and cron jobs to start automatically, first check whether they are running with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">systemctl status cronie bluetooth
</code></pre></div><p>Enable them with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">systemctl enable --now cronie bluetooth
</code></pre></div><p>Check the service status again:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">systemctl status cronie bluetooth
</code></pre></div><h3 id="diving-a-bit-into-aur">Diving a bit into AUR</h3>
<p>Some modern Arch Linux variants come preloaded with <a href="https://aur.archlinux.org/packages/yay-git">yay</a>, an AUR (Arch Linux User Repository) helper written in the Go language.</p>
<p><a href="https://invent.kde.org/plasma/ksshaskpass">ksshaskpass</a> is a KDE-based ssh-askpass client. We can install it with <code>yay</code>. Check the package’s availability first:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ yay -Ss ksshaskpass
aur/ksshaskpass-git 5.22.80_r196.gdaa2679-1 (+6 0.00)
ssh-add helper that uses kwallet-git and kpassworddialog
extra/ksshaskpass 5.27.3-1 (29.7 KiB 101.9 KiB) [plasma] (Installed)
ssh-add helper that uses kwallet and kpassworddialog
</code></pre></div><p>Since I already installed ksshaskpass, we can see that the status is indicated as “Installed”. If you want to install it for the first time, just type <code>yay -S ksshaskpass</code>.</p>
<h3 id="printing">Printing</h3>
<p>Support for printing is quite easy to set up. It has an application helper which is most useful for newbies, but could also be useful for seasoned Linux users who might have missed the boat for new applications. Since Arch Linux-based distros can use AUR, many packages which are provided independently and do not have an official repository can be updated with the <code>yay</code> command.</p>
<p>I am using a Canon E410 printer, and in the event that the printer is not detected right away by the OS, we can use the CUPS configuration tool (accessible at <code>localhost:631</code>) to configure the printing server. If you are using Gnome, you can also try to add the printer through Gnome’s printer selection in its settings.</p>
<h3 id="desktop-environment-gnome">Desktop environment: Gnome</h3>
<p>I like to assign hotkeys for workspace navigation. I usually use Alt+1, Alt+2, Alt+3, and Alt+4 to switch to the Workspace 1, 2, 3, and 4, respectively.</p>
<p>In Gnome, this can be done by opening the settings app and doing the following:</p>
<p>Go to Keyboard → Keyboard Shortcuts → View and Customize Shortcuts → Navigation → Switch to Workspace #</p>
<p>Apart from the GUI goodness, I also use <a href="https://ohmyz.sh/">Oh My Zsh</a> for my Zsh shell and <a href="https://ohmyposh.dev/">Oh My Posh</a> for Bash. You can see how to install them on their respective sites.</p>
<h3 id="bash-config">Bash config</h3>
<p>I have my own shell configurations but I want to highlight my setup for Oh My Posh, ksshaskpass, and Vim:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">POSHTHEME="easy-term"
eval "$(oh-my-posh --init --shell bash --config ~/.poshthemes/$POSHTHEME.omp.json)"
export SSH_ASKPASS=/usr/bin/ksshaskpass
export EDITOR=/usr/bin/vim
</code></pre></div>