https://www.endpointdev.com/blog/tags/intellij-idea/2022-05-30T00:00:00+00:00End Point DevAligning monospace font text columns with an old Unix toolhttps://www.endpointdev.com/blog/2022/05/aligning-fixed-width-font-text-columns/2022-05-30T00:00:00+00:00Jon Jensen
<p><img src="/blog/2022/05/aligning-fixed-width-font-text-columns/20220316_183928.webp" alt="Photo of old wooden bridge with moss on it">
Photo by Garrett Skinner</p>
<h3 id="a-blast-from-1990-column">A blast from 1990: column</h3>
<p>A while back I learned of a nice old Unix command-line tool called <code>column</code>. It first appeared in 4.3BSD-Reno, released in July 1990. (This is not to be confused with the different, even older Unix tool <code>col</code>.)</p>
<p><code>column</code> formats plain text into nice columns based on the width of the input separated by tabs or groups of spaces.</p>
<p>For example, take this mess found in a server’s <code>/etc/fstab</code> file defining filesystem mount points. It is a real example lightly redacted to remove business details. You may need to scroll right to see the end of the fairly long lines:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">/data3/customer_uploads /home/interch/htdocs/shared/customer_uploads none rw,bind 0 0
/data3/customer_images /home/interch/htdocs/shared/customer_images none rw,bind 0 0
/data3/images/items home/interch/htdocs/images/items none rw,bind 0 0
/data3/images/thumb home/interch/htdocs/images/thumb none rw,bind 0 0
/data3/upload_images /home/interch/upload_images none rw,bind 0 0
/data3/design /home/interch/htdocs/shared/design none rw,bind 0 0
/data3/design_temp /home/interch/htdocs/shared/design_temp none rw,bind 0 0
/data3/reports/cat1 /home/interch/htdocs/cat-numero-uno/images/reports none rw,bind 0 0
/data3/reports/cat2 /home/interch/htdocs/cat-zahl-zwei/images/reports none rw,bind 0 0
/data3/reports/cat3 /home/interch/htdocs/cat-number-three/images/reports none rw,bind 0 0
/data3/reports/cat4 /home/interch/htdocs/cat-quatre/images/reports none rw,bind 0 0
/data3/reports/cat5 /home/interch/htdocs/cat-pět/images/reports none rw,bind 0 0
/data3/shared_var /home/interch/catalogs/shared/var none rw,bind 0 0
</code></pre></div><p>That is really unsightly and includes a mix of spaces and tabs.</p>
<p>If we feed it to <code>column -t</code> we get the same data aligned much nicer:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% column -t /etc/fstab
/data3/customer_uploads /home/interch/htdocs/shared/customer_uploads none rw,bind 0 0
/data3/customer_images /home/interch/htdocs/shared/customer_images none rw,bind 0 0
/data3/images/items home/interch/htdocs/images/items none rw,bind 0 0
/data3/images/thumb home/interch/htdocs/images/thumb none rw,bind 0 0
/data3/upload_images /home/interch/upload_images none rw,bind 0 0
/data3/design /home/interch/htdocs/shared/design none rw,bind 0 0
/data3/design_temp /home/interch/htdocs/shared/design_temp none rw,bind 0 0
/data3/reports/cat1 /home/interch/htdocs/cat-numero-uno/images/reports none rw,bind 0 0
/data3/reports/cat2 /home/interch/htdocs/cat-zahl-zwei/images/reports none rw,bind 0 0
/data3/reports/cat3 /home/interch/htdocs/cat-number-three/images/reports none rw,bind 0 0
/data3/reports/cat4 /home/interch/htdocs/cat-quatre/images/reports none rw,bind 0 0
/data3/reports/cat5 /home/interch/htdocs/cat-pět/images/reports none rw,bind 0 0
/data3/shared_var /home/interch/catalogs/shared/var none rw,bind 0 0
</code></pre></div><p>That isn’t just prettier! It also makes some things stand out prominently at a glance:</p>
<ul>
<li>In the second column there are two mount points not starting with <code>/</code>.</li>
<li>It’s easy to see that most of the paths in the second column start with <code>/home/interch/htdocs/</code> and the few that don’t, stand out.</li>
<li>The final 4 columns are all identical, which was unclear in the unaligned original.</li>
</ul>
<h3 id="text-tables-to-json">Text tables to JSON</h3>
<p>The <a href="https://en.wikipedia.org/wiki/Util-linux">util-linux</a> version of <code>column</code> includes extra options beyond the original. One very useful one is <code>-J</code> or <code>--json</code> which produces a JSON object for each row based on column names you define in the argument <code>-N</code> or <code>--table-columns</code>.</p>
<p>Reviewing <a href="https://www.man7.org/linux/man-pages/man5/fstab.5.html"><code>man fstab</code></a> for details on what each column is used for in the sample above, we can instruct <code>column</code> to produce JSON output like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh">% column -J -n fstab -N spec,file,vfstype,mntops,freq,passno /etc/fstab
</code></pre></div><p>Which gives us this result:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"fstab"</span>: [
{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/customer_uploads"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/shared/customer_uploads"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/customer_images"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/shared/customer_images"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/images/items"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"home/interch/htdocs/images/items"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/images/thumb"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"home/interch/htdocs/images/thumb"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/upload_images"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/upload_images"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/design"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/shared/design"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/design_temp"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/shared/design_temp"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/reports/cat1"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/cat-numero-uno/images/reports"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/reports/cat2"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/cat-zahl-zwei/images/reports"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/reports/cat3"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/cat-number-three/images/reports"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/reports/cat4"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/cat-quatre/images/reports"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/reports/cat5"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/htdocs/cat-pět/images/reports"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
},{
<span style="color:#b06;font-weight:bold">"spec"</span>: <span style="color:#d20;background-color:#fff0f0">"/data3/shared_var"</span>,
<span style="color:#b06;font-weight:bold">"file"</span>: <span style="color:#d20;background-color:#fff0f0">"/home/interch/catalogs/shared/var"</span>,
<span style="color:#b06;font-weight:bold">"vfstype"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#b06;font-weight:bold">"mntops"</span>: <span style="color:#d20;background-color:#fff0f0">"rw,bind"</span>,
<span style="color:#b06;font-weight:bold">"freq"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>,
<span style="color:#b06;font-weight:bold">"passno"</span>: <span style="color:#d20;background-color:#fff0f0">"0"</span>
}
]
}
</code></pre></div><p>JSON takes a lot more room, and of course is not the format Linux expects for this particular file, but transforming tabular data to JSON format in other situations can be more readable for exchange across different systems since each field is labeled, and unused fields can be omitted. Plus JSON syntax is rigorously defined, nested data structures are possible, etc.</p>
<h3 id="columnizing-lists">Columnizing lists</h3>
<p>All versions of <code>column</code> can also columnize lists, either horizontally (across) or vertically (down). Take this list of people’s names:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Amy
Anna
Bob
Brenda
Cameron
Doug
Emily
Frank
Jane
Jill
Jim
Joe
John
Karen
Kate
Liz
Mary
Mike
Sarah
Steve
Victoria
</code></pre></div><p>Put that in file <code>/tmp/names</code> and <code>column</code> will format it in columns fitting the width of your terminal:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% column /tmp/names
Amy Cameron Jane John Mary Victoria
Anna Doug Jill Karen Mike
Bob Emily Jim Kate Sarah
Brenda Frank Joe Liz Steve
</code></pre></div><p>It uses 2 or more tab characters to separate the columns, based on standard terminal 8-space tab stops, so the above doesn’t look right here on the web.</p>
<p>What appears in my terminal looks like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% column /tmp/names
Amy Cameron Jane John Mary Victoria
Anna Doug Jill Karen Mike
Bob Emily Jim Kate Sarah
Brenda Frank Joe Liz Steve
</code></pre></div><p>Or you can use <code>column -t</code> that we discussed earlier to format the columns more compactly with spaces:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% column /tmp/names | column -t
Amy Cameron Jane John Mary Victoria
Anna Doug Jill Karen Mike
Bob Emily Jim Kate Sarah
Brenda Frank Joe Liz Steve
</code></pre></div><p>You can also ask for the list to be delivered horizontally, rather than vertically, with <code>column -x</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% column -x /tmp/names | column -t
Amy Anna Bob Brenda Cameron Doug
Emily Frank Jane Jill Jim Joe
John Karen Kate Liz Mary Mike
Sarah Steve Victoria
</code></pre></div><p>Note that it isn’t doing anything to affect your ordering. You can order the lines in your original file however you want and it will preserve them. But other tools can help you here: Use <code>sort -u</code> to sort alphabetically and remove duplicates.</p>
<p>For more options and details see the <code>column</code> man page for the <a href="https://man7.org/linux/man-pages/man1/column.1.html">util-linux column</a> version or <a href="https://www.freebsd.org/cgi/man.cgi?query=column&apropos=0&sektion=0&manpath=FreeBSD+13.1-RELEASE+and+Ports&arch=default&format=html">FreeBSD column</a> version (same as macOS).</p>
<h3 id="a-blast-from-1974-pr">A blast from 1974: pr</h3>
<p>There is an even older Unix tool for columnizing lists in the same way. It is called <code>pr</code> and dates to First Edition Unix in 1971, but did not gain the options we are using here until Fifth Edition Unix in 1974 as seen in the <a href="https://minnie.tuhs.org/cgi-bin/utree.pl?file=V6/usr/man/man1/pr.1">V6 pr man page</a>.</p>
<p>We need to tell it how many columns to produce, so we will ask for 6 columns as <code>column</code> was doing above. Note that <code>pr</code> emits a curious mix of tabs and spaces, which <code>cat -t</code> reveals here as <code>^I</code> (since a tab is the same thing as Control+I):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% pr -t -6 /tmp/names | cat -t
Amy^I Cameron^IJane^I John^ILiz^I Sarah
Anna^I Doug^IJill^I Karen^IMary^I Steve
Bob^I Emily^IJim^I Kate^IMike^I Victoria
Brenda^I Frank^IJoe
</code></pre></div><p>But in a terminal it looks fine:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% pr -t -6 /tmp/names
Amy Cameron Jane John Liz Sarah
Anna Doug Jill Karen Mary Steve
Bob Emily Jim Kate Mike Victoria
Brenda Frank Joe
</code></pre></div><p>See the many more options of pr in the <a href="https://man7.org/linux/man-pages/man1/pr.1.html">GNU coreutils pr man page</a> and <a href="https://www.freebsd.org/cgi/man.cgi?query=pr&apropos=0&sektion=0&manpath=FreeBSD+13.1-RELEASE+and+Ports&arch=default&format=html">FreeBSD pr man page</a> (same as macOS).</p>
<h3 id="a-blast-from-1979-expand">A blast from 1979: expand</h3>
<p>A useful tool for dealing with tabs is <code>expand</code>, which <a href="https://github.com/dank101/3BSD/blob/master/cmd/expand.c">first appeared in 3BSD</a> in 1979. (Despite the FreeBSD and macOS man pages saying it appeared in 1BSD, I don’t see it there or in 2BSD.)</p>
<p>We can use it to convert tabs to spaces just like a terminal would:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% pr -t -6 /tmp/names | expand | cat -t
Amy Cameron Jane John Liz Sarah
Anna Doug Jill Karen Mary Steve
Bob Emily Jim Kate Mike Victoria
Brenda Frank Joe
</code></pre></div><p><code>expand</code> accepts the optional <code>-t</code> argument with a number to use as tabstop width instead of the default 8.</p>
<h3 id="and-unexpand">And unexpand</h3>
<p>I have used <code>expand</code> for years, but only recently learned of <code>unexpand</code> which goes the other way, converting runs of spaces into tabs:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% pr -t -6 /tmp/names | expand | unexpand -a
Amy Cameron Jane John Liz Sarah
Anna Doug Jill Karen Mary Steve
Bob Emily Jim Kate Mike Victoria
Brenda Frank Joe
</code></pre></div><p>It looks fine in the terminal, but let’s see if it actually used tabs:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% pr -t -6 /tmp/names | expand | unexpand -a | cat -t
Amy^I Cameron^IJane^I John^ILiz^I Sarah
Anna^I Doug^IJill^I Karen^IMary^I Steve
Bob^I Emily^IJim^I Kate^IMike^I Victoria
Brenda^I Frank^IJoe
</code></pre></div><p>Interesting… For some reason <code>unexpand</code> doesn’t actually convert all the spaces to tabs, but just one initial tab per gutter between columns. Running the output through <code>unexpand -a</code> again has no further effect. Strange.</p>
<p>The GNU coreutils version of <code>unexpand</code> that lives in most Linux systems is what I would call bug-compatible with the BSD version in this regard.</p>
<p>Oh, well. There’s probably a reason.</p>
<p>Since it has handled turning the initial tabstop’s varying number of spaces into a tab, we can easily remove the remaining fixed multiples of spaces on our own for a more compact list:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">% pr -t -6 /tmp/names | expand | unexpand -a | sed 's/ //g'
Amy Cameron Jane John Liz Sarah
Anna Doug Jill Karen Mary Steve
Bob Emily Jim Kate Mike Victoria
Brenda Frank Joe
</code></pre></div><h3 id="ready-at-hand">Ready at hand</h3>
<p><code>column</code>, <code>pr</code>, <code>expand</code>, and <code>unexpand</code> all come with most Linux and BSD systems, including macOS. It is amazing what great old tools are on many of our computers all the time, waiting to be used!</p>
<p>You can use these programs as filters inside your favorite text editor or IDE. For example, to achieve the same list columnization as above you can select a block of text and send it to external commands:</p>
<ul>
<li>In Vim, visually select text with <code>v</code> or <code>V</code> and then type <code>!column | expand</code> and press Enter.</li>
<li>In VS Code you can install an extension called Filter Text (by yhirose). Once installed, type Control+K Control+F (⌘K ⌘F on macOS) and then <code>column | expand</code> and Enter.</li>
<li>In IntelliJ IDEA you can install a plugin called Shell Filter (by Dennis Plöger). Once installed, select your text, then choose menu item Edit > Custom Shell Filter and then type <code>column -c 80 | expand</code> and Enter.</li>
<li>Most other editors have a way to do this too. Search for “filter”, “pipe”, and/or “command”.</li>
</ul>
<p>Your selection will be replaced by the output.</p>
<h3 id="unicode">Unicode</h3>
<p>These tools come from an era when one byte of input resulted in one visual character, so for any UTF-8 characters outside the limited classic ASCII character set, we could expect old tools may miscalculate the needed space between columns.</p>
<p>Originally in my testing they appeared not to be aware of some Unicode characters’ width, perhaps due to combining characters or alternate forms. But surprisingly they have been modernized and seem to work fine with most Unicode characters such as Latin letters with diacritics, letters in other alphabets, Chinese characters, and even emoji! 😊</p>
EditorConfig: Ending the Spaces vs. Tabs Confusionhttps://www.endpointdev.com/blog/2022/04/editorconfig-ending-spaces-vs-tabs-confusion/2022-04-30T00:00:00+00:00Jon Jensen
<p><img src="/blog/2022/04/editorconfig-ending-spaces-vs-tabs-confusion/20220316_184133.webp" alt="">
Photo by Garrett Skinner</p>
<h3 id="varieties-of-text-formatting">Varieties of text formatting</h3>
<p>Most everyone who has worked on a software development project with a group of other people has encountered the problem of source code being formatted in different ways by different text editors, IDEs, and operating systems.</p>
<p>The main variations go back to the 1970s or earlier, and include the questions:</p>
<ul>
<li>Will indentation be done by tabs (an ASCII control character) or spaces?
<ul>
<li>If indentation is done by spaces, how many spaces are used for each indentation level?</li>
</ul>
</li>
<li>What will indicate the end of each line (EOL)? The choices are:
<ul>
<li>a line feed (LF), used by the Unix family including Linux and modern macOS</li>
<li>a carriage return (CR), used by old pre-Unix Macintosh and some now-obscure operating systems</li>
<li>both together (CRLF) used by Windows and most Internet protocols</li>
</ul>
</li>
<li>Which character set encoding will be used? Common choices are:
<ul>
<li>Unicode UTF-8 encoding, used by Linux, macOS, and most other Unixes, and standard on the Internet</li>
<li>Unicode UTF-16 encoding (with either little-endian or big-endian encoding), used by modern Windows</li>
<li>legacy ISO-8859 and Windows “code page” encodings in older documents and codebases</li>
</ul>
</li>
</ul>
<h3 id="editor-configurations-in-conflict">Editor configurations in conflict</h3>
<p>Causing widespread frustration, by default, text editors and IDEs generally are each configured differently, and once set, the choices apply broadly from then on. But each developer can simply configure their editor to follow their team’s standards, right? Well, maybe.</p>
<p>First, getting that to happen for every developer and every different editor being used isn’t straightforward. It typically requires a document showing instructions and/or screenshots of how to configure each editor. It may have to be redone after a major upgrade or move to a new computer.</p>
<p>Second, and often a more persistent problem, standards may vary across different projects and even for different types of files within a given project. Ruby code is typically indented with 2 spaces, while perhaps in your project JavaScript uses 4 spaces and HTML uses tabs.</p>
<p>If you start a new project from scratch you can probably settle on a single standard, but in existing large codebases, it can make a lot of version control change “noise” to mess with that.</p>
<p>Computers are good at keeping track of lots of little details, so isn’t there some way to have the computer deal with this?</p>
<h3 id="storing-configuration-in-the-project">Storing configuration in the project</h3>
<p>What if we store the text editor’s or IDE’s configuration in the project instead of per user, so it can go with the project to each new developer and tell their editor how to behave?</p>
<p>For many years that has been possible with some editors, but the configuration had to be set up separately for each editor, and often the feature is disabled by default.</p>
<p>Let’s consider the two most popular terminal-based editors on Unix, partisans in a long-running editor war:</p>
<h4 id="vim">Vim</h4>
<p>Vim has a feature called a “modeline” that allows for configuration settings to appear within the top or bottom 5 lines of the file.</p>
<p>For example, to instruct Vim to use spaces instead of tabs and 4-space tab stops, we can add to the top or bottom of our C source code file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c" data-lang="c"><span style="color:#888">/* vim: tabstop=4 shiftwidth=4 expandtab
</span><span style="color:#888"> */</span>
</code></pre></div><p>Since it gets tedious putting those special configuration comments in each file, Vim has an option to read a <code>.vimrc</code> file from the current directory, which applies to all files there and can be committed to version control.</p>
<p>This feature is disabled by default because Vim has in the past been vulnerable to files with malicious settings running arbitrary code.</p>
<p>You can <code>:set exrc secure</code> to enable the modeline feature in a code base you trust, and also to restrict what it can do.</p>
<h4 id="emacs">Emacs</h4>
<p>In Emacs the same thing can be done on the first or second line of the file. (Of course its setting names differ from Vim’s.) For example consider this configuration in C source code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c" data-lang="c"><span style="color:#888">/* -*- mode: c; indent-tabs-mode: nil; c-basic-offset: 4; tab-width: 4 -*- */</span>
</code></pre></div><p>Alternately you can use “Local Variables” set at the end of the file in as many lines as needed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c" data-lang="c"><span style="color:#888">/* Local Variables: */</span>
<span style="color:#888">/* mode: c */</span>
<span style="color:#888">/* indent-tabs-mode: nil */</span>
<span style="color:#888">/* c-basic-offset: 4 */</span>
<span style="color:#888">/* tab-width: 4 */</span>
<span style="color:#888">/* End: */</span>
</code></pre></div><p>Emacs also has “Directory Variables” that can be set in the file <code>.dir-locals.el</code> for a directory and its subdirectories.</p>
<h4 id="others">Others</h4>
<p>Even if someone has gone to the trouble to set up such editor configuration files and add them to the project code repository, how often has that been done for your editor or IDE?</p>
<p>And how often is one out of sync with the others?</p>
<p>This is not the way to success.</p>
<h3 id="editorconfig-to-the-rescue">EditorConfig to the rescue</h3>
<p>About 10 years ago Trey Hunner and Hong Xu shared with the world EditorConfig, their creation to solve this problem across ideally all editors.</p>
<p>They intentionally kept EditorConfig fairly limited in scope. It covers a limited number of the most important editor options so that the standard would be simple enough to be implemented for every editor either internally or as a plugin, and there would be no arbitrary code execution possible to cause security problems.</p>
<p>In EditorConfig the configuration for our examples and hypotheticals above lives in a <code>.editorconfig</code> file in the root of the project that looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-ini" data-lang="ini"><span style="color:#888"># top-most EditorConfig file</span>
<span style="color:#369">root</span> = <span style="color:#d20;background-color:#fff0f0">true</span>
<span style="color:#888"># basics for all files in our project</span>
<span style="color:#080;font-weight:bold">[*]</span>
<span style="color:#369">charset</span> = <span style="color:#d20;background-color:#fff0f0">utf-8</span>
<span style="color:#369">end_of_line</span> = <span style="color:#d20;background-color:#fff0f0">lf</span>
<span style="color:#888"># C and JavaScript source get 4-space indents</span>
<span style="color:#080;font-weight:bold">[*.{c,js}]</span>
<span style="color:#369">indent_style</span> = <span style="color:#d20;background-color:#fff0f0">space</span>
<span style="color:#369">indent_size</span> = <span style="color:#d20;background-color:#fff0f0">4</span>
<span style="color:#888"># Ruby gets 2-space indents</span>
<span style="color:#080;font-weight:bold">[*.rb]</span>
<span style="color:#369">indent_style</span> = <span style="color:#d20;background-color:#fff0f0">space</span>
<span style="color:#369">indent_size</span> = <span style="color:#d20;background-color:#fff0f0">2</span>
<span style="color:#888"># HTML gets tab indents</span>
<span style="color:#080;font-weight:bold">[*.html]</span>
<span style="color:#369">indent_style</span> = <span style="color:#d20;background-color:#fff0f0">tab</span>
</code></pre></div><p>In a big project you may want to have separate, smaller <code>.editorconfig</code> files in different directories. You can omit the <code>root = true</code> setting in subdirectories to inherit settings from the top-level <code>.editorconfig</code> file.</p>
<p>There are a couple of other options that are nice to specify.</p>
<p>This one removes any tabs or spaces from the end of lines:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-ini" data-lang="ini"><span style="color:#369">trim_trailing_whitespace</span> = <span style="color:#d20;background-color:#fff0f0">true</span>
</code></pre></div><p>Those are rarely needed or semantically meaningful, so it’s nice to remove them. But there are a few cases where they can matter such as in Markdown.</p>
<p>This one determines whether the last line in the file will end with a newline:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-ini" data-lang="ini"><span style="color:#369">insert_final_newline</span> = <span style="color:#d20;background-color:#fff0f0">true</span>
</code></pre></div><p>By default some editors add to the last line a newline (such as Vim) and some don’t (such as Emacs), leading to needless changes as various developers change files.</p>
<p>Typically every line should end with a newline, so that’s a good editor feature to enable. But you could have some text template that should <em>not</em> end with a newline, so might need to specify <code>false</code> for that type of file.</p>
<p>And those are most of the features of EditorConfig! <a href="https://editorconfig.org/#file-format-details">The file format details</a> are easy to digest.</p>
<h3 id="editor-amp-ide-support">Editor & IDE support</h3>
<p>EditorConfig is now widely supported. These popular editors & IDEs recognize <code>.editorconfig</code> files with no extra work:</p>
<ul>
<li>IntelliJ IDEA and most of its language-specific variants</li>
<li>GitHub</li>
<li>GitLab</li>
<li>Visual Studio</li>
<li>BBEdit</li>
<li>and others</li>
</ul>
<p>And these support it with a plugin:</p>
<ul>
<li>VS Code</li>
<li>Vim</li>
<li>Emacs</li>
<li>Sublime Text</li>
<li>TextMate</li>
<li>Eclipse</li>
<li>Atom</li>
<li>Notepad++</li>
<li>Geany</li>
<li>and others</li>
</ul>
<p>The plugins are typically easy to install system-wide from your operating system’s package manager, or else locally for your user only.</p>
<h3 id="do-you-need-it">Do you need it?</h3>
<p>Yes, I think you do.</p>
<p>I know of no reason for any developer not to use EditorConfig, in every editor, for every project. It’s simple and at long last solves this small set of problems well.</p>
<p>One possible counterargument: If, before every version control commit, you run an automatic code formatter such as Prettier (in Node.js, for many languages) or a language-specific one such as <code>gofmt</code>, <code>rustfmt</code>, etc., you could perhaps live without your editor knowing how your files should be saved.</p>
<p>But isn’t it better if your editor knows what kind of line endings and indents to use, rather than waiting for a code formatter to correct such fundamental things after you save? It is easy to start with a single <code>.editorconfig</code> file long before you have a continuous integration set up for the project.</p>
<p>And many projects don’t format code automatically, and instead just “lint” it to report on deviations from the project standards. But that requires work to correct, and can be ignored if not enforced.</p>
<p>Many <a href="https://github.com/editorconfig/editorconfig/wiki/Projects-Using-EditorConfig">open source projects large and small use EditorConfig</a>, including this blog itself. But in recent months I have found several developers who had not yet heard of EditorConfig, so I want to spread awareness of it. I hope you’ll <a href="https://editorconfig.org/">use EditorConfig</a> too!</p>
Word diff: Git as wdiff alternativehttps://www.endpointdev.com/blog/2022/01/wdiff-alternative-git-diff-word-diff/2022-01-05T00:00:00+00:00Jon Jensen
<p><img src="/blog/2022/01/wdiff-alternative-git-diff-word-diff/20210823_183559-sm.jpg" alt="5 drinking fountains mounted on a wall at varying levels"></p>
<!-- Image by Jon Jensen -->
<p>The <code>diff</code> utility to show differences between two files was created in 1974 as part of Unix. It has been incredibly useful and popular ever since, and hasn’t changed much since 1991 when it gained the ability to output the now standard “unified context diff” format.</p>
<p>The comparison <code>diff</code> makes is per line, so if anything on a given line changes, in unified context format we can tell that the previous version of that line was removed by seeing <code>-</code> at the beginning of the old line, and the following line will start with <code>+</code> followed by the new version.</p>
<p>For example see this Dockerfile that had two lines changed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">$ diff -u Dockerfile.old Dockerfile
<span style="color:#000;background-color:#fdd">--- Dockerfile.old 2022-01-05 22:16:21 -0700
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ Dockerfile 2022-01-05 23:08:55 -0700
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -2,7 +2,7 @@
</span><span style="color:#666"></span>
WORKDIR /usr/src/app
<span style="color:#000;background-color:#fdd">-# Bundle app source
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+# Bundle entire source
</span><span style="color:#000;background-color:#dfd"></span> COPY . .
<span style="color:#000;background-color:#fdd">-RUN /usr/src/app/test.sh
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+RUN /usr/src/app/start.sh
</span></code></pre></div><p>That works well for visually reviewing changes to many types of files that developers typically work with.</p>
<p>It can also serve as input to the <code>patch</code> program, which dates to 1985 and is still in wide use as a counterpart to <code>diff</code>. With <code>patch</code> we can apply changes to a file and avoid the need to send an entire new file or apply changes by hand (which is very prone to error).</p>
<p>But let’s leave that aside and focus on humans reading <code>diff</code> output.</p>
<h3 id="diffing-paragraphs">Diffing paragraphs</h3>
<p>For a file containing paragraphs of prose each on their own long lines, it can look like the lines change completely when we change only a few words. This is often the case with HTML destined to be displayed in a web browser, email text, or the Markdown source for this blog post itself.</p>
<p>Consider this file with one long line of sample text gathered from pangrams and typing exercises:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ cat -n paragraph.txt
1 The quick brown fox jumped over the lazy dog's back 1234567890 times. Now is the time for all good men to come to the aid of the party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly quick daft zebras jump! The five boxing wizards jump quickly. Jackdaws love my big sphinx of quartz. Pack my box with five dozen liquor jugs.
</code></pre></div><p>If we change even one character of that, <code>diff</code> will show that its one line changed, but we have to painstakingly visually scan the entire long line to determine what exactly changed and where. That is not very useful.</p>
<p>We can wrap, or split up, the lines into multiple lines of a maximum of 75 characters with the classic Unix tool <code>fmt</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ fmt paragraph.txt | tee wrapped.txt
The quick brown fox jumped over the lazy dog's back 1234567890
times. Now is the time for all good men to come to the aid of the
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
quick daft zebras jump! The five boxing wizards jump quickly.
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div><p>With those shorter lines, changes will be easier to see than with one long line, but it is still hard to pick out small changes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">$ diff -u wrapped.txt wrapped2.txt
<span style="color:#000;background-color:#fdd">--- wrapped.txt 2022-01-05 23:22:46 -0700
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ wrapped2.txt 2022-01-05 23:38:14 -0700
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -1,7 +1,7 @@
</span><span style="color:#666"></span> The quick brown fox jumped over the lazy dog's back 1234567890
<span style="color:#000;background-color:#fdd">-times. Now is the time for all good men to come to the aid of the
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+times. Now is thy time for all good men to come to the aid of the
</span><span style="color:#000;background-color:#dfd"></span> party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
<span style="color:#000;background-color:#fdd">-quick daft zebras jump! The five boxing wizards jump quickly.
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+quick daft zebras jump! Tho five boxing wizards jump quickly.
</span><span style="color:#000;background-color:#dfd"></span> Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div><p>And much worse, every line changes when we make a significant edit early in the text and reflow the paragraph to fit our maximum line length. Here is what happens after adding “an amazing count of” in the first line and re-wrapping the lines:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#333">diff -u wrapped.txt wrapped3.txt
</span><span style="color:#333"></span><span style="color:#000;background-color:#fdd">--- wrapped.txt 2022-01-05 23:22:46 -0700
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ wrapped3.txt 2022-01-06 08:05:33 -0700
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -1,7 +1,7 @@
</span><span style="color:#666"></span><span style="color:#000;background-color:#fdd">-The quick brown fox jumped over the lazy dog's back 1234567890
</span><span style="color:#000;background-color:#fdd">-times. Now is the time for all good men to come to the aid of the
</span><span style="color:#000;background-color:#fdd">-party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
</span><span style="color:#000;background-color:#fdd">-to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
</span><span style="color:#000;background-color:#fdd">-quick daft zebras jump! The five boxing wizards jump quickly.
</span><span style="color:#000;background-color:#fdd">-Jackdaws love my big sphinx of quartz. Pack my box with five dozen
</span><span style="color:#000;background-color:#fdd">-liquor jugs.
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+The quick brown fox jumped over the lazy dog's back an amazing count
</span><span style="color:#000;background-color:#dfd">+of 1234567890 times. Now is thy time for all good men to come to
</span><span style="color:#000;background-color:#dfd">+the aid of the party. Waltz, bad nymph, for quick jigs vex. Glib
</span><span style="color:#000;background-color:#dfd">+jocks quiz nymph to vex dwarf. Sphinx of black quartz, judge my
</span><span style="color:#000;background-color:#dfd">+vow. How vexingly quick daft zebras jump! Tho five boxing wizards
</span><span style="color:#000;background-color:#dfd">+jump quickly. Jackdaws love my big sphinx of quartz. Pack my box
</span><span style="color:#000;background-color:#dfd">+with five dozen liquor jugs.
</span></code></pre></div><p>That gives no aid to a human proofreader!</p>
<h3 id="word-diff-to-the-rescue">Word diff to the rescue</h3>
<p>In 1992 François Pinard wrote the word-based diff program <code>wdiff</code> which is now part of the GNU project. It solves this problem.</p>
<p>Here is how it shows us our example changing two words:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ wdiff wrapped.txt wrapped2.txt
The quick brown fox jumped over the lazy dog's back 1234567890
times. Now is [-the-] {+thy+} time for all good men to come to the aid of the
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
quick daft zebras jump! [-The-] {+Tho+} five boxing wizards jump quickly.
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div><p>Words removed are by default marked with <code>[-…-]</code> and words added with <code>{+…+}</code>.</p>
<p>It even knows how to accommodate word changes appearing on different lines! Trying it out on our example with the reflowed paragraph:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ wdiff wrapped.txt wrapped3.txt
The quick brown fox jumped over the lazy dog's back {+an amazing count
of+} 1234567890 times. Now is [-the-] {+thy+} time for all good men to come to
the aid of the party. Waltz, bad nymph, for quick jigs vex. Glib
jocks quiz nymph to vex dwarf. Sphinx of black quartz, judge my
vow. How vexingly quick daft zebras jump! [-The-] {+Tho+} five boxing wizards
jump quickly. Jackdaws love my big sphinx of quartz. Pack my box
with five dozen liquor jugs.
</code></pre></div><p>So this is very nice, although <code>wdiff</code> often isn’t available by default on the various systems we find ourselves on, and it is perhaps a bit worrisome that the <code>wdiff</code> software has not been updated since 2014.</p>
<p>Too bad this word-diffing feature is not part of standard <code>diff</code>!</p>
<h3 id="a-familiar-friend">A familiar friend</h3>
<p>That’s ok because you probably already have a <code>wdiff</code> alternative available on your computer: Git! More specifically, <code>git diff --word-diff</code>.</p>
<p>Maybe you already use that feature when working with your local clones of Git repositories, to look at what changed in the commit history or local edits. Did you know that <code>git diff</code> can act as a complete replacement of the standalone <code>diff</code> tool? Yes, <code>git diff</code> can also compare two arbitrary files that are not part of a Git repository when given the <code>--no-index</code> option!</p>
<p>And Git can usually tell you mean <code>--no-index</code> without you typing that explicitly because you’re comparing at least one file that is not tracked in a Git clone, so you can just type:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git diff --word-diff <path1> <path2>
</code></pre></div><p>for any two file paths and it will work.</p>
<p>Trying this out with our sample paragraph:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff"><code class="language-diff" data-lang="diff">$ git diff --word-diff wrapped.txt wrapped2.txt
<span style="font-weight: bold">diff --git a/wrapped.txt b/wrapped2.txt
index b1c5775..59ff315 100644
--- a/wrapped.txt
+++ b/wrapped2.txt</span>
<span style="color:blue">@@ -1,7 +1,7 @@</span>
The quick brown fox jumped over the lazy dog's back 1234567890
times. Now is <span style="color:#000;background-color:tomato">[-the-]</span><span style="color:#000;background-color:lightgreen">{+thy+}</span> time for all good men to come to the aid of the
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
quick daft zebras jump! <span style="color:#000;background-color:tomato">[-The-]</span><span style="color:#000;background-color:lightgreen">{+Tho+}</span> five boxing wizards jump quickly.
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div>
<p>It uses the same word deletion and insertion markers as <code>wdiff</code>, but to make them easier for our eyes to spot, by default <code>git diff</code> also shows them in different colors when output is going to an interactive terminal. You can disable the coloring with the additional option <code>--color=never</code>.</p>
<p>Use <code>git diff --word-diff=color</code> for a pretty view using <em>only</em> color to show the changes, without the <code>[-…-]</code> and <code>{+…+}</code> markers. This may be more readable when your input text is full of punctuation confusingly similar to the markers, and is useful if you want to copy from the terminal without any extra surrounding characters:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff"><code class="language-diff" data-lang="diff">$ git diff --word-diff=color wrapped.txt wrapped2.txt
<span style="font-weight: bold">diff --git a/wrapped.txt b/wrapped2.txt
index b1c5775..59ff315 100644
--- a/wrapped.txt
+++ b/wrapped2.txt</span>
<span style="color:blue">@@ -1,7 +1,7 @@</span>
The quick brown fox jumped over the lazy dog's back 1234567890
times. Now is <span style="color:#000;background-color:tomato">the</span><span style="color:#000;background-color:lightgreen">thy</span> time for all good men to come to the aid of the
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
quick daft zebras jump! <span style="color:#000;background-color:tomato">The</span><span style="color:#000;background-color:lightgreen">Tho</span> five boxing wizards jump quickly.
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div>
<p>There is also the option <code>git diff --word-diff=porcelain</code> for an ugly but more easily code-parseable format useful for output sent as input to scripts:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git diff --word-diff=porcelain wrapped.txt wrapped2.txt
diff --git a/wrapped.txt b/wrapped2.txt
index b1c5775..59ff315 100644
--- a/wrapped.txt
+++ b/wrapped2.txt
@@ -1,7 +1,7 @@
The quick brown fox jumped over the lazy dog's back 1234567890
~
times. Now is
-the
+thy
time for all good men to come to the aid of the
~
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
~
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
~
quick daft zebras jump!
-The
+Tho
five boxing wizards jump quickly.
~
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
~
liquor jugs.
~
</code></pre></div><p>I have never needed that yet, but it is good to be aware of in case I ever do need to parse word diff output, to make it easier and more reliable.</p>
<h3 id="customize-word-break-definition">Customize word break definition</h3>
<p>Other kinds of files can present challenges for readability in diff output.</p>
<p>For example consider trying to see small changes in the classic Unix <code>/etc/passwd</code> text “database” which has one user record per line, and within each record line uses <code>:</code> to delimit fields.</p>
<p>First we’ll try traditional line diff:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">$ git diff passwd passwd.mangled
<span style="color:#333">diff --git a/passwd b/passwd.mangled
</span><span style="color:#333">index 981736c..6531f10 100644
</span><span style="color:#333"></span><span style="color:#000;background-color:#fdd">--- a/passwd
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ b/passwd.mangled
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -24,22 +24,22 @@ polkitd:x:996:991:User for polkitd:/:/sbin/nologin
</span><span style="color:#666"></span> rtkit:x:172:172:RealtimeKit:/proc:/sbin/nologin
pulse:x:171:171:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
chrony:x:995:988::/var/lib/chrony:/sbin/nologin
<span style="color:#000;background-color:#fdd">-abrt:x:173:173::/etc/abrt:/sbin/nologin
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+abrt:x:173:1730::/etc/abrt:/sbin/nologin
</span><span style="color:#000;background-color:#dfd"></span> colord:x:994:987:User for colord:/var/lib/colord:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
vboxadd:x:993:1::/var/run/vboxadd:/sbin/nologin
dnsmasq:x:985:985:Dnsmasq DHCP and DNS server:/var/lib/dnsmasq:/sbin/nologin
<span style="color:#000;background-color:#fdd">-tcpdump:x:72:72::/:/sbin/nologin
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+tcpdump:x:72:72::/:/bin/bash
</span><span style="color:#000;background-color:#dfd"></span> systemd-timesync:x:984:984:systemd Time Synchronization:/:/sbin/nologin
pipewire:x:983:983:PipeWire System Daemon:/var/run/pipewire:/sbin/nologin
gluster:x:982:982:GlusterFS daemons:/run/gluster:/sbin/nologin
<span style="color:#000;background-color:#fdd">-radvd:x:75:75:radvd user:/:/sbin/nologin
</span><span style="color:#000;background-color:#fdd">-saslauth:x:981:76:Saslauthd user:/run/saslauthd:/sbin/nologin
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+radvd:x:76:75:radvd user:/:/sbin/nologin
</span><span style="color:#000;background-color:#dfd">+saslauth:x:981:76:Saslauthd user:/ran/saslauthd:/sbin/nologin
</span><span style="color:#000;background-color:#dfd"></span> usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
setroubleshoot:x:980:979::/var/lib/setroubleshoot:/sbin/nologin
openvpn:x:979:978:OpenVPN:/etc/openvpn:/sbin/nologin
<span style="color:#000;background-color:#fdd">-nm-openvpn:x:978:977:Default user for running openvpn spawned by NetworkManager:/:/sbin/nologin
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+mm-openvpn:x:978:977:Default user for running openvpn spawned by NetworkManager:/:/sbin/nologin
</span><span style="color:#000;background-color:#dfd"></span> qemu:x:107:107:qemu user:/:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
</code></pre></div><p>It’s not too hard to “eyeball” changes there if they add or remove characters and thus affect the line lengths. But a line with only a change to a single character isn’t as easy.</p>
<p>Since blank space is not the relevant separator in this file, standard word diff doesn’t help and in some cases is worse than line diff:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff"><code class="language-diff" data-lang="diff">$ git diff --word-diff passwd passwd.mangled
<span style="font-weight: bold">diff --git a/passwd b/passwd.mangled
index 981736c..6531f10 100644
--- a/passwd
+++ b/passwd.mangled</span>
<span style="color:blue">@@ -24,22 +24,22 @@</span> polkitd:x:996:991:User for polkitd:/:/sbin/nologin
rtkit:x:172:172:RealtimeKit:/proc:/sbin/nologin
pulse:x:171:171:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
chrony:x:995:988::/var/lib/chrony:/sbin/nologin
<span style="color:#000;background-color:tomato">[-abrt:x:173:173::/etc/abrt:/sbin/nologin-]</span><span style="color:#000;background-color:lightgreen">{+abrt:x:173:1730::/etc/abrt:/sbin/nologin+}</span>
colord:x:994:987:User for colord:/var/lib/colord:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
vboxadd:x:993:1::/var/run/vboxadd:/sbin/nologin
dnsmasq:x:985:985:Dnsmasq DHCP and DNS server:/var/lib/dnsmasq:/sbin/nologin
<span style="color:#000;background-color:tomato">[-tcpdump:x:72:72::/:/sbin/nologin-]</span><span style="color:#000;background-color:lightgreen">{+tcpdump:x:72:72::/:/bin/bash+}</span>
systemd-timesync:x:984:984:systemd Time Synchronization:/:/sbin/nologin
pipewire:x:983:983:PipeWire System Daemon:/var/run/pipewire:/sbin/nologin
gluster:x:982:982:GlusterFS daemons:/run/gluster:/sbin/nologin
<span style="color:#000;background-color:tomato">[-radvd:x:75:75:radvd-]</span><span style="color:#000;background-color:lightgreen">{+radvd:x:76:75:radvd+}</span> user:/:/sbin/nologin
saslauth:x:981:76:Saslauthd <span style="color:#000;background-color:tomato">[-user:/run/saslauthd:/sbin/nologin-]</span><span style="color:#000;background-color:lightgreen">{+user:/ran/saslauthd:/sbin/nologin+}</span>
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
setroubleshoot:x:980:979::/var/lib/setroubleshoot:/sbin/nologin
openvpn:x:979:978:OpenVPN:/etc/openvpn:/sbin/nologin
<span style="color:#000;background-color:tomato">[-nm-openvpn:x:978:977:Default-]</span><span style="color:#000;background-color:lightgreen">{+mm-openvpn:x:978:977:Default+}</span> user for running openvpn spawned by NetworkManager:/:/sbin/nologin
qemu:x:107:107:qemu user:/:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
</code></pre></div>
<p>Another venerable program similar to <code>wdiff</code> that is still maintained is <code>dwdiff</code>. In its self-description we read something intriguing:</p>
<blockquote>
<p>It is different from wdiff in that it allows the user to specify what should be considered whitespace …</p>
</blockquote>
<p>That sounds useful. But <code>dwdiff</code> is still a separate program and is even less common than <code>wdiff</code>. Can the versatile <code>git diff</code> help us here too?</p>
<p>Yes! <code>git diff</code> has the option <code>--word-diff-regex</code> to specify a regular expression to use instead of whitespace as a delimiter, like <code>dwdiff</code> does. The man page explanation notes:</p>
<blockquote>
<p>For example, <code>--word-diff-regex=.</code> will treat each character as a word and, correspondingly, show differences character by character.</p>
</blockquote>
<p>It also notes that <code>--word-diff</code> is assumed and can be omitted when using <code>--word-diff-regex</code>.</p>
<p>So let’s try that:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff"><code class="language-diff" data-lang="diff">$ git diff --word-diff-regex=. passwd passwd.mangled
<span style="font-weight: bold">diff --git a/passwd b/passwd.mangled
index 981736c..6531f10 100644
--- a/passwd
+++ b/passwd.mangled</span>
<span style="color:blue">@@ -24,22 +24,22 @@</span> polkitd:x:996:991:User for polkitd:/:/sbin/nologin
rtkit:x:172:172:RealtimeKit:/proc:/sbin/nologin
pulse:x:171:171:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
chrony:x:995:988::/var/lib/chrony:/sbin/nologin
abrt:x:173:173<span style="color:#000;background-color:lightgreen">{+0+}</span>::/etc/abrt:/sbin/nologin
colord:x:994:987:User for colord:/var/lib/colord:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
vboxadd:x:993:1::/var/run/vboxadd:/sbin/nologin
dnsmasq:x:985:985:Dnsmasq DHCP and DNS server:/var/lib/dnsmasq:/sbin/nologin
tcpdump:x:72:72::/:/<span style="color:#000;background-color:tomato">[-s-]</span>bin/<span style="color:#000;background-color:tomato">[-nologin-]</span><span style="color:#000;background-color:lightgreen">{+bash+}</span>
systemd-timesync:x:984:984:systemd Time Synchronization:/:/sbin/nologin
pipewire:x:983:983:PipeWire System Daemon:/var/run/pipewire:/sbin/nologin
gluster:x:982:982:GlusterFS daemons:/run/gluster:/sbin/nologin
radvd:x:7<span style="color:#000;background-color:tomato">[-5-]</span><span style="color:#000;background-color:lightgreen">{+6+}</span>:75:radvd user:/:/sbin/nologin
saslauth:x:981:76:Saslauthd user:/r<span style="color:#000;background-color:tomato">[-u-]</span><span style="color:#000;background-color:lightgreen">{+a+}</span>n/saslauthd:/sbin/nologin
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
setroubleshoot:x:980:979::/var/lib/setroubleshoot:/sbin/nologin
openvpn:x:979:978:OpenVPN:/etc/openvpn:/sbin/nologin
<span style="color:#000;background-color:tomato">[-n-]</span><span style="color:#000;background-color:lightgreen">{+m+}</span>m-openvpn:x:978:977:Default user for running openvpn spawned by NetworkManager:/:/sbin/nologin
qemu:x:107:107:qemu user:/:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
</code></pre></div>
<p>That’s quite good, at least to my eyes.</p>
<h3 id="on-the-web">On the web</h3>
<p>GitHub, GitLab, and Bitbucket do a good job of showing readable diffs for most common cases: line-oriented, but within each line also word or character diffs highlighted via color. Open up each of the following links to see how they present a few of our earlier examples as commit diffs:</p>
<ul>
<li>Change 2 letters in prose paragraph: <a href="https://github.com/jonjensen/word-diff-examples/commit/07ab55f0acb6410e30688515a7a0bc95ed6a37b1">🔗 in GitHub</a>, <a href="https://gitlab.com/jonjensen/word-diff-examples/-/commit/07ab55f0acb6410e30688515a7a0bc95ed6a37b1">🔗 in GitLab</a>, <a href="https://bitbucket.org/jonjensen/word-diff-examples/commits/07ab55f0acb6410e30688515a7a0bc95ed6a37b1">🔗 in Bitbucket</a></li>
<li>Change a few characters in <code>/etc/passwd</code>: <a href="https://github.com/jonjensen/word-diff-examples/commit/faa5f309a6d83a898a2b84fdf4c1c16189bb0be8">🔗 in GitHub</a>, <a href="https://gitlab.com/jonjensen/word-diff-examples/-/commit/faa5f309a6d83a898a2b84fdf4c1c16189bb0be8">🔗 in GitLab</a>, <a href="https://bitbucket.org/jonjensen/word-diff-examples/commits/faa5f309a6d83a898a2b84fdf4c1c16189bb0be8">🔗 in Bitbucket</a></li>
</ul>
<p>But GitHub and GitLab both break down on a reflowed paragraph, while Bitbucket shows a sensible diff of what logically changed, including spaces to newlines and vice versa:</p>
<ul>
<li>Insert words early in paragraph and reflow: <a href="https://github.com/jonjensen/word-diff-examples/commit/986ab1450f434d675b69c60b67a837f4bf11c84f">🔗 in GitHub</a>, <a href="https://gitlab.com/jonjensen/word-diff-examples/-/commit/986ab1450f434d675b69c60b67a837f4bf11c84f">🔗 in GitLab</a>, <a href="https://bitbucket.org/jonjensen/word-diff-examples/commits/986ab1450f434d675b69c60b67a837f4bf11c84f">🔗 in Bitbucket</a></li>
</ul>
<p>It appears that GitLab may soon gain proper cross-line word diff ability as seen in the project’s issue <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/325856">Add word-diff option to commits view</a>, which states its “Problem to solve” as:</p>
<blockquote>
<p>When working with markdown (or any type of prose/text in general), the “classic” git-diff (intended for code) is of limited use.</p>
</blockquote>
<p>Exactly right.</p>
<h3 id="ides">IDEs</h3>
<p>Visual Studio Code (VS Code) handles the above cases well out of the box for uncommitted changes in the current Git clone, and the GitLens extension helps it do the same for showing past commit diffs.</p>
<p>IntelliJ IDEA handles both cases well by default.</p>
<h3 id="for-those-left-behind">For those left behind</h3>
<p>To make the most of Git, you’ll want a fairly recent version, since new features are being added all the time. If you’re working on a server using the popular but aging CentOS 7 which comes with the ancient Git 1.8.3, you can follow our <a href="/blog/2021/12/installing-git-2-on-centos-7/">simple tutorial to upgrade to Git 2.34.1 or newer on CentOS 7</a>.</p>
<p>Enjoy!</p>
<h3 id="reference">Reference</h3>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Diff">diff</a> on Wikipedia, including history and samples of original, context, and unified context diff output</li>
<li><a href="https://en.wikipedia.org/wiki/Patch_(Unix)">patch</a> on Wikipedia</li>
<li><a href="https://git-scm.com/docs/git-diff">git-diff</a> man page</li>
<li><a href="https://www.gnu.org/software/wdiff/">wdiff</a></li>
<li><a href="https://os.ghalkes.nl/dwdiff.html">dwdiff</a></li>
<li><a href="https://en.wikipedia.org/wiki/Pangram">Pangrams</a> on Wikipedia, the source of our sample prose here</li>
</ul>
Hot-deploy Java classes and assets in Wildfly 8/9/10https://www.endpointdev.com/blog/2017/10/hot-deploy-java-classes-and-assets-in/2017-10-27T00:00:00+00:00Piotr Hankiewicz
<p><img border="0" src="/blog/2017/10/hot-deploy-java-classes-and-assets-in/image-0.jpg" style="width: 100%; max-width: 100%;" /></p>
<h3 id="introduction">Introduction</h3>
<p>Java development can be really frustrating when you need to re-build your project and restart a server every time you change something. I know about JRebel, but while it’s a good tool, it’s also pretty expensive. You can use the open-source version, but then you need to send project statistics to the JRebel server, which is not a viable option for your more serious projects.</p>
<p>Fortunately, there is an open-source project called HotSwapAgent and it does the same thing as JRebel, for free (thank you, guys!).</p>
<p>I will explain how to combine it with Widlfly in order to hot-deploy Java classes as well as how to hot-deploy other resources (JavaScript, CSS, images).</p>
<h3 id="wildfly-configuration">Wildfly configuration</h3>
<p>Let’s assume that we use the <code>standalone-full.xml</code> configuration file.</p>
<p>We need to use exploded deployment instead of deploying WAR or EAR. You can do this in production as well to allow for application changes with zero downtime.</p>
<p>Start by configuring the metaspace size; we had to increase defaults for our application, but it’s possible that it will be just fine in your case. It’s encouraged that you play with these values after completing all steps.</p>
<p>In:</p>
<p><code>WILDFLY_DIR/bin/standalone.conf</code></p>
<p>set:</p>
<p><code>-XX:MetaspaceSize=256M -XX:MaxMetaspaceSize=512m</code></p>
<p>so it looks like this:</p>
<p><code>JAVA_OPTS="-Xms512m -Xmx1024m -XX:MetaspaceSize=256M -XX:MaxMetaspaceSize=512m"</code>.</p>
<p>Now, look for the <code>deployment-scanner</code> node in:</p>
<p><code>WILDFLY_DIR/standalone/configuration/standalone-full.xml</code></p>
<p>Replace it so it looks like this:</p>
<p><code><deployment-scanner path="PATH_TO_DEPLOYMENT_DIR" relative-to="RELATIVE_TO_PATH" scan-enabled="true" scan-interval="2000" auto-deploy-exploded="false" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}"/></code></p>
<p>Note:</p>
<p><code>PATH_TO_DEPLOYMENT_DIR</code> is <code>WILDFLY_DIR/standalone/deployments</code></p>
<p><code>RELATIVE_TO_PATH</code> is, as the name suggests the dir that the <code>PATH_TO_DEPLOYMENT_DIR</code> is relative to.</p>
<h3 id="hotswapagent-installation-and-configuration">HotSwapAgent installation and configuration</h3>
<p>We need to download and install the latest release of DCEVM Java patch from here: <a href="https://github.com/dcevm/dcevm/releases">https://github.com/dcevm/dcevm/releases</a>. Why it’s needed? It will allow us unlimited redefinition of loaded classes at runtime. This is not possible with the original Java HotSpot VM. Make sure you update to the same Java version that you’re going to use to run the Wildfly server.</p>
<p>Now, download the latest release of the Hotswap agent from here:</p>
<p><a href="https://github.com/HotswapProjects/HotswapAgent/releases">https://github.com/HotswapProjects/HotswapAgent/releases</a></p>
<p>The only thing that you need to do is get the JAR and put it anywhere on your hard drive (I recommend to add it to your Java project).</p>
<p>Ok, great, now just some configuration.</p>
<p>Open:</p>
<p><code>WILDFLY_DIR/bin/standalone.conf</code></p>
<p>and add new Java opts:</p>
<p><code>-XXaltjvm=dcevm -javaagent:PATH_TO_HOTSWAPAGENT_JAR</code>.</p>
<p>What does this do?</p>
<ul>
<li>The <code>altjvm</code> option sets an alternative Java Virtual Machine.</li>
<li>The <code>javagent</code> is just an interceptor on the top of your classes that allows the HotSwapAgent library to manipulate your code on the fly.</li>
</ul>
<p>That’s all you need. It’s a good idea to create a configuration file for the HotSwapAgent. This is well explained here:</p>
<p><a href="http://hotswapagent.org/mydoc_configuration.html">http://hotswapagent.org/mydoc_configuration.html</a></p>
<p>Basically create a new file, name it hotswap-agent.properties, set all needed configuration inside and add it to the classpath of the application.</p>
<p>If you use Netbeans, Eclipse or Intellij you should check the HotSwapAgent page for some helpful plugins here: <a href="http://hotswapagent.org/mydoc_setup_intellij_idea.html">http://hotswapagent.org/mydoc_setup_intellij_idea.html</a>, <a href="http://hotswapagent.org/mydoc_setup_eclipse.html">http://hotswapagent.org/mydoc_setup_eclipse.html</a>, and <a href="http://hotswapagent.org/mydoc_setup_netbeans.html">http://hotswapagent.org/mydoc_setup_netbeans.html</a>.</p>
<h3 id="application-configuration">Application configuration</h3>
<p>Now that we have everything in place, I will explain how to put it all together.
I doesn’t really matter which build-tool you use (Ant, Gradle or Maven). The process should look like this (you can do it in many ways, in our case, it’s pretty specific as our build process is really complicated):</p>
<ol>
<li>Build your application and deploy it to the <code>PATH_TO_DEPLOYMENT_DIR</code> in the exploded version,</li>
<li>Create a script that will look for changes in the application directory (this one is interesting: <a href="https://gist.github.com/peter-hank/3ecf7fc285ba4b9c50cf8cace1badaf4">https://gist.github.com/peter-hank/3ecf7fc285ba4b9c50cf8cace1badaf4</a>),</li>
<li>On change, trigger a job that will:
<ol>
<li>Copy all resources like JSP, JavaScript, CSS and copy to the <code>PATH_TO_DEPLOYMENT_DIR</code>,</li>
<li>Compile classes and copy them to the <code>PATH_TO_DEPLOYMENT_DIR</code>.</li>
</ol>
</li>
</ol>
<p>That’s it, after you replace files in the <code>PATH_TO_DEPLOYMENT_DIR</code> HotSwapAgent and Wildfly will do the rest really fast. We have a ton of assets and classes and the whole process takes only a few seconds!</p>
<h3 id="summary">Summary</h3>
<p>I feel this process is really worth doing. It doesn’t take a lot of time to configure everything and saves a lot of manual work. Just multiply the number of manual deployments and the number of developers in your team and you understand how much time you lose everyday without hot-deployment.</p>
<p>From now on, focus on development, forget about deployment!</p>
<p>Lastly, good luck!</p>