https://www.endpointdev.com/blog/tags/git/2022-10-05T00:00:00+00:00End Point DevGit 2.38 packages for RHEL, CentOS, Rocky Linux, AlmaLinux, Oracle Linux 7 and 8https://www.endpointdev.com/blog/2022/10/git-2.38-for-rhel7-rhel8/2022-10-05T00:00:00+00:00Jon Jensen
<p><img src="/blog/2022/10/git-2.38-for-rhel7-rhel8/20220903-162550-sm.webp" alt="Photo of a wilderness footpath over stone with scattered evergreen trees alongside a clear lake, with a blue sky"></p>
<!-- photo by Jon Jensen -->
<p>Git 2.38 was released 2 days ago, and it has some interesting new features. Let’s look at a couple of them, which you may find make upgrading worthwhile.</p>
<h3 id="scalar">Scalar</h3>
<p>Git now includes a new command-line tool called <code>scalar</code> which is adapted to working with very large Git repositories. It reminds me of Google’s “repo” tool used to manage a group of multiple large Git repositories for the Android project, but unlike that, Scalar focuses on a single repository.</p>
<p>By default it makes a partial clone so that you need far less network transfer and local disk space to work with a large repository. One of our clients has a repository whose <code>.git</code> directory for a complete clone takes 7.1 GB, but when using <code>scalar clone</code> the partial clone of that repository uses only 46 MB. That’s only 0.6% the size! Your experience will vary based on how much of the repository data is in history vs. currently checked-out files.</p>
<p>I use SSH agent confirmation with <code>ssh-add -c</code> (on Linux; see references below for tips on doing it on Windows and macOS). Because of that, I am aware of every SSH connection that uses my SSH secret key, and noticed that Scalar had Git make 3 network connections to the origin server rather than just 1.</p>
<p>During later Git operations, having only the most recent commit stored locally leads to some interesting new network calls. For example, when you do plain <code>git log</code> to see commits, there’s no remote access needed. But doing <code>git log -p</code> to see the diffs per commit causes 1 SSH request per commit, quite a bundle of SSH agent confirmations for even a small handful of commits!</p>
<p>Cloning with Scalar also leaves some warnings for your enjoyment, which currently happen no matter which remote Git server I’m cloning from, but maybe these will go away later when servers gain support for some new features being used here:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">warning: filtering not recognized by server, ignoring
warning: filtering not recognized by server, ignoring
warning: fetch normally indicates which branches had a forced update,
but that check has been disabled; to re-enable, use '--show-forced-updates'
flag or run 'git config fetch.showForcedUpdates true'
warning: fetch normally indicates which branches had a forced update,
but that check has been disabled; to re-enable, use '--show-forced-updates'
flag or run 'git config fetch.showForcedUpdates true'
</code></pre></div><p>Another default feature of Scalar is scheduled background maintenance. On a Linux system with systemd during your first <code>scalar clone</code> operation you’ll see:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Created symlink /home/someuser/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer → /home/someuser/.config/systemd/user/git-maintenance@.timer.
Created symlink /home/someuser/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer → /home/someuser/.config/systemd/user/git-maintenance@.timer.
Created symlink /home/someuser/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer → /home/someuser/.config/systemd/user/git-maintenance@.timer.
</code></pre></div><p>showing that it set up systemd timers to maintain your Git clones.</p>
<h3 id="rebasing-dependent-branches">Rebasing dependent branches</h3>
<p>This is a really helpful new feature!</p>
<p>In short, when rebasing, you can now use <code>git rebase --update-refs</code> and Git will update other branches that are affected by the changed commit hashes you are currently rebasing.</p>
<p>You probably only want to do that when you’re dealing with local private branches of your own, or you’re working against a repository where you can force-push branch history rewrites for multiple branches.</p>
<h3 id="and-more-">And more …</h3>
<p>There are several other great new features too. This was just to whet your appetite.</p>
<p>The <a href="https://github.blog/2022-10-03-highlights-from-git-2-38/">GitHub blog highlights from Git 2.38</a> gives an accessible overview of the major new features.</p>
<h3 id="new-git-on-an-older-enterprise-linux-os">New Git on an older Enterprise Linux OS</h3>
<p>But how soon will you be able to use the new Git? On your own development computer, hopefully soon.</p>
<p>If you’re using a server running Red Hat Enterprise Linux or its derivatives CentOS, Rocky Linux, AlmaLinux, or Oracle Linux, you probably only have an old version of Git there that the distribution won’t upgrade beyond that major version to avoid introducing incompatibilities.</p>
<p>We have packaged Git 2.38 for RHEL 7 and 8 so anyone can upgrade and use a modern Git version.</p>
<p>Follow <a href="/blog/2021/12/installing-git-2-on-centos-7/">our instructions for installing newer Git</a>. That write-up highlights Git 2.34.1, but since our repositories now include Git 2.38.0, you’ll get that newer version.</p>
<p>Enjoy!</p>
<h3 id="reference">Reference</h3>
<ul>
<li><a href="/blog/2021/12/installing-git-2-on-centos-7/">Installing Git 2 on CentOS 7</a></li>
<li><a href="https://packages.endpointdev.com/">End Point package repositories</a> for EL 8 also include Git 2.38</li>
<li><a href="https://git-scm.com/">Git website</a></li>
<li><a href="https://github.blog/2022-10-03-highlights-from-git-2-38/">Highlights from Git 2.38</a> on the GitHub blog</li>
<li>SSH agent confirmation:
<ul>
<li>On Linux and other Unixes: <code>ssh-add -c</code></li>
<li>On Windows, see <a href="/blog/2022/07/windows-ssh-key-agent-forwarding-confirmation/">Windows SSH key agent confirmation</a></li>
<li>On macOS, you can use <a href="https://github.com/theseal/ssh-askpass">ssh-askpass for macOS</a> for normal keys or <a href="https://github.com/maxgoedjen/secretive">Secretive</a> for keys generated in and stored in Apple’s Secure Enclave.</li>
</ul>
</li>
<li><a href="https://github.com/microsoft/scalar">Scalar</a> project website</li>
<li><a href="https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/">Get up to speed with partial clone and shallow clone</a> on the GitHub blog</li>
<li><a href="https://android.googlesource.com/tools/repo">repo</a> multi-repository tool for Git</li>
</ul>
Word diff: Git as wdiff alternativehttps://www.endpointdev.com/blog/2022/01/wdiff-alternative-git-diff-word-diff/2022-01-05T00:00:00+00:00Jon Jensen
<p><img src="/blog/2022/01/wdiff-alternative-git-diff-word-diff/20210823_183559-sm.jpg" alt="5 drinking fountains mounted on a wall at varying levels"></p>
<!-- Image by Jon Jensen -->
<p>The <code>diff</code> utility to show differences between two files was created in 1974 as part of Unix. It has been incredibly useful and popular ever since, and hasn’t changed much since 1991 when it gained the ability to output the now standard “unified context diff” format.</p>
<p>The comparison <code>diff</code> makes is per line, so if anything on a given line changes, in unified context format we can tell that the previous version of that line was removed by seeing <code>-</code> at the beginning of the old line, and the following line will start with <code>+</code> followed by the new version.</p>
<p>For example see this Dockerfile that had two lines changed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">$ diff -u Dockerfile.old Dockerfile
<span style="color:#000;background-color:#fdd">--- Dockerfile.old 2022-01-05 22:16:21 -0700
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ Dockerfile 2022-01-05 23:08:55 -0700
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -2,7 +2,7 @@
</span><span style="color:#666"></span>
WORKDIR /usr/src/app
<span style="color:#000;background-color:#fdd">-# Bundle app source
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+# Bundle entire source
</span><span style="color:#000;background-color:#dfd"></span> COPY . .
<span style="color:#000;background-color:#fdd">-RUN /usr/src/app/test.sh
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+RUN /usr/src/app/start.sh
</span></code></pre></div><p>That works well for visually reviewing changes to many types of files that developers typically work with.</p>
<p>It can also serve as input to the <code>patch</code> program, which dates to 1985 and is still in wide use as a counterpart to <code>diff</code>. With <code>patch</code> we can apply changes to a file and avoid the need to send an entire new file or apply changes by hand (which is very prone to error).</p>
<p>But let’s leave that aside and focus on humans reading <code>diff</code> output.</p>
<h3 id="diffing-paragraphs">Diffing paragraphs</h3>
<p>For a file containing paragraphs of prose each on their own long lines, it can look like the lines change completely when we change only a few words. This is often the case with HTML destined to be displayed in a web browser, email text, or the Markdown source for this blog post itself.</p>
<p>Consider this file with one long line of sample text gathered from pangrams and typing exercises:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ cat -n paragraph.txt
1 The quick brown fox jumped over the lazy dog's back 1234567890 times. Now is the time for all good men to come to the aid of the party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly quick daft zebras jump! The five boxing wizards jump quickly. Jackdaws love my big sphinx of quartz. Pack my box with five dozen liquor jugs.
</code></pre></div><p>If we change even one character of that, <code>diff</code> will show that its one line changed, but we have to painstakingly visually scan the entire long line to determine what exactly changed and where. That is not very useful.</p>
<p>We can wrap, or split up, the lines into multiple lines of a maximum of 75 characters with the classic Unix tool <code>fmt</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ fmt paragraph.txt | tee wrapped.txt
The quick brown fox jumped over the lazy dog's back 1234567890
times. Now is the time for all good men to come to the aid of the
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
quick daft zebras jump! The five boxing wizards jump quickly.
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div><p>With those shorter lines, changes will be easier to see than with one long line, but it is still hard to pick out small changes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">$ diff -u wrapped.txt wrapped2.txt
<span style="color:#000;background-color:#fdd">--- wrapped.txt 2022-01-05 23:22:46 -0700
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ wrapped2.txt 2022-01-05 23:38:14 -0700
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -1,7 +1,7 @@
</span><span style="color:#666"></span> The quick brown fox jumped over the lazy dog's back 1234567890
<span style="color:#000;background-color:#fdd">-times. Now is the time for all good men to come to the aid of the
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+times. Now is thy time for all good men to come to the aid of the
</span><span style="color:#000;background-color:#dfd"></span> party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
<span style="color:#000;background-color:#fdd">-quick daft zebras jump! The five boxing wizards jump quickly.
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+quick daft zebras jump! Tho five boxing wizards jump quickly.
</span><span style="color:#000;background-color:#dfd"></span> Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div><p>And much worse, every line changes when we make a significant edit early in the text and reflow the paragraph to fit our maximum line length. Here is what happens after adding “an amazing count of” in the first line and re-wrapping the lines:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#333">diff -u wrapped.txt wrapped3.txt
</span><span style="color:#333"></span><span style="color:#000;background-color:#fdd">--- wrapped.txt 2022-01-05 23:22:46 -0700
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ wrapped3.txt 2022-01-06 08:05:33 -0700
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -1,7 +1,7 @@
</span><span style="color:#666"></span><span style="color:#000;background-color:#fdd">-The quick brown fox jumped over the lazy dog's back 1234567890
</span><span style="color:#000;background-color:#fdd">-times. Now is the time for all good men to come to the aid of the
</span><span style="color:#000;background-color:#fdd">-party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
</span><span style="color:#000;background-color:#fdd">-to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
</span><span style="color:#000;background-color:#fdd">-quick daft zebras jump! The five boxing wizards jump quickly.
</span><span style="color:#000;background-color:#fdd">-Jackdaws love my big sphinx of quartz. Pack my box with five dozen
</span><span style="color:#000;background-color:#fdd">-liquor jugs.
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+The quick brown fox jumped over the lazy dog's back an amazing count
</span><span style="color:#000;background-color:#dfd">+of 1234567890 times. Now is thy time for all good men to come to
</span><span style="color:#000;background-color:#dfd">+the aid of the party. Waltz, bad nymph, for quick jigs vex. Glib
</span><span style="color:#000;background-color:#dfd">+jocks quiz nymph to vex dwarf. Sphinx of black quartz, judge my
</span><span style="color:#000;background-color:#dfd">+vow. How vexingly quick daft zebras jump! Tho five boxing wizards
</span><span style="color:#000;background-color:#dfd">+jump quickly. Jackdaws love my big sphinx of quartz. Pack my box
</span><span style="color:#000;background-color:#dfd">+with five dozen liquor jugs.
</span></code></pre></div><p>That gives no aid to a human proofreader!</p>
<h3 id="word-diff-to-the-rescue">Word diff to the rescue</h3>
<p>In 1992 François Pinard wrote the word-based diff program <code>wdiff</code> which is now part of the GNU project. It solves this problem.</p>
<p>Here is how it shows us our example changing two words:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ wdiff wrapped.txt wrapped2.txt
The quick brown fox jumped over the lazy dog's back 1234567890
times. Now is [-the-] {+thy+} time for all good men to come to the aid of the
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
quick daft zebras jump! [-The-] {+Tho+} five boxing wizards jump quickly.
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div><p>Words removed are by default marked with <code>[-…-]</code> and words added with <code>{+…+}</code>.</p>
<p>It even knows how to accommodate word changes appearing on different lines! Trying it out on our example with the reflowed paragraph:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ wdiff wrapped.txt wrapped3.txt
The quick brown fox jumped over the lazy dog's back {+an amazing count
of+} 1234567890 times. Now is [-the-] {+thy+} time for all good men to come to
the aid of the party. Waltz, bad nymph, for quick jigs vex. Glib
jocks quiz nymph to vex dwarf. Sphinx of black quartz, judge my
vow. How vexingly quick daft zebras jump! [-The-] {+Tho+} five boxing wizards
jump quickly. Jackdaws love my big sphinx of quartz. Pack my box
with five dozen liquor jugs.
</code></pre></div><p>So this is very nice, although <code>wdiff</code> often isn’t available by default on the various systems we find ourselves on, and it is perhaps a bit worrisome that the <code>wdiff</code> software has not been updated since 2014.</p>
<p>Too bad this word-diffing feature is not part of standard <code>diff</code>!</p>
<h3 id="a-familiar-friend">A familiar friend</h3>
<p>That’s ok because you probably already have a <code>wdiff</code> alternative available on your computer: Git! More specifically, <code>git diff --word-diff</code>.</p>
<p>Maybe you already use that feature when working with your local clones of Git repositories, to look at what changed in the commit history or local edits. Did you know that <code>git diff</code> can act as a complete replacement of the standalone <code>diff</code> tool? Yes, <code>git diff</code> can also compare two arbitrary files that are not part of a Git repository when given the <code>--no-index</code> option!</p>
<p>And Git can usually tell you mean <code>--no-index</code> without you typing that explicitly because you’re comparing at least one file that is not tracked in a Git clone, so you can just type:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git diff --word-diff <path1> <path2>
</code></pre></div><p>for any two file paths and it will work.</p>
<p>Trying this out with our sample paragraph:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff"><code class="language-diff" data-lang="diff">$ git diff --word-diff wrapped.txt wrapped2.txt
<span style="font-weight: bold">diff --git a/wrapped.txt b/wrapped2.txt
index b1c5775..59ff315 100644
--- a/wrapped.txt
+++ b/wrapped2.txt</span>
<span style="color:blue">@@ -1,7 +1,7 @@</span>
The quick brown fox jumped over the lazy dog's back 1234567890
times. Now is <span style="color:#000;background-color:tomato">[-the-]</span><span style="color:#000;background-color:lightgreen">{+thy+}</span> time for all good men to come to the aid of the
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
quick daft zebras jump! <span style="color:#000;background-color:tomato">[-The-]</span><span style="color:#000;background-color:lightgreen">{+Tho+}</span> five boxing wizards jump quickly.
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div>
<p>It uses the same word deletion and insertion markers as <code>wdiff</code>, but to make them easier for our eyes to spot, by default <code>git diff</code> also shows them in different colors when output is going to an interactive terminal. You can disable the coloring with the additional option <code>--color=never</code>.</p>
<p>Use <code>git diff --word-diff=color</code> for a pretty view using <em>only</em> color to show the changes, without the <code>[-…-]</code> and <code>{+…+}</code> markers. This may be more readable when your input text is full of punctuation confusingly similar to the markers, and is useful if you want to copy from the terminal without any extra surrounding characters:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff"><code class="language-diff" data-lang="diff">$ git diff --word-diff=color wrapped.txt wrapped2.txt
<span style="font-weight: bold">diff --git a/wrapped.txt b/wrapped2.txt
index b1c5775..59ff315 100644
--- a/wrapped.txt
+++ b/wrapped2.txt</span>
<span style="color:blue">@@ -1,7 +1,7 @@</span>
The quick brown fox jumped over the lazy dog's back 1234567890
times. Now is <span style="color:#000;background-color:tomato">the</span><span style="color:#000;background-color:lightgreen">thy</span> time for all good men to come to the aid of the
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
quick daft zebras jump! <span style="color:#000;background-color:tomato">The</span><span style="color:#000;background-color:lightgreen">Tho</span> five boxing wizards jump quickly.
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
liquor jugs.
</code></pre></div>
<p>There is also the option <code>git diff --word-diff=porcelain</code> for an ugly but more easily code-parseable format useful for output sent as input to scripts:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git diff --word-diff=porcelain wrapped.txt wrapped2.txt
diff --git a/wrapped.txt b/wrapped2.txt
index b1c5775..59ff315 100644
--- a/wrapped.txt
+++ b/wrapped2.txt
@@ -1,7 +1,7 @@
The quick brown fox jumped over the lazy dog's back 1234567890
~
times. Now is
-the
+thy
time for all good men to come to the aid of the
~
party. Waltz, bad nymph, for quick jigs vex. Glib jocks quiz nymph
~
to vex dwarf. Sphinx of black quartz, judge my vow. How vexingly
~
quick daft zebras jump!
-The
+Tho
five boxing wizards jump quickly.
~
Jackdaws love my big sphinx of quartz. Pack my box with five dozen
~
liquor jugs.
~
</code></pre></div><p>I have never needed that yet, but it is good to be aware of in case I ever do need to parse word diff output, to make it easier and more reliable.</p>
<h3 id="customize-word-break-definition">Customize word break definition</h3>
<p>Other kinds of files can present challenges for readability in diff output.</p>
<p>For example consider trying to see small changes in the classic Unix <code>/etc/passwd</code> text “database” which has one user record per line, and within each record line uses <code>:</code> to delimit fields.</p>
<p>First we’ll try traditional line diff:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">$ git diff passwd passwd.mangled
<span style="color:#333">diff --git a/passwd b/passwd.mangled
</span><span style="color:#333">index 981736c..6531f10 100644
</span><span style="color:#333"></span><span style="color:#000;background-color:#fdd">--- a/passwd
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ b/passwd.mangled
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -24,22 +24,22 @@ polkitd:x:996:991:User for polkitd:/:/sbin/nologin
</span><span style="color:#666"></span> rtkit:x:172:172:RealtimeKit:/proc:/sbin/nologin
pulse:x:171:171:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
chrony:x:995:988::/var/lib/chrony:/sbin/nologin
<span style="color:#000;background-color:#fdd">-abrt:x:173:173::/etc/abrt:/sbin/nologin
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+abrt:x:173:1730::/etc/abrt:/sbin/nologin
</span><span style="color:#000;background-color:#dfd"></span> colord:x:994:987:User for colord:/var/lib/colord:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
vboxadd:x:993:1::/var/run/vboxadd:/sbin/nologin
dnsmasq:x:985:985:Dnsmasq DHCP and DNS server:/var/lib/dnsmasq:/sbin/nologin
<span style="color:#000;background-color:#fdd">-tcpdump:x:72:72::/:/sbin/nologin
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+tcpdump:x:72:72::/:/bin/bash
</span><span style="color:#000;background-color:#dfd"></span> systemd-timesync:x:984:984:systemd Time Synchronization:/:/sbin/nologin
pipewire:x:983:983:PipeWire System Daemon:/var/run/pipewire:/sbin/nologin
gluster:x:982:982:GlusterFS daemons:/run/gluster:/sbin/nologin
<span style="color:#000;background-color:#fdd">-radvd:x:75:75:radvd user:/:/sbin/nologin
</span><span style="color:#000;background-color:#fdd">-saslauth:x:981:76:Saslauthd user:/run/saslauthd:/sbin/nologin
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+radvd:x:76:75:radvd user:/:/sbin/nologin
</span><span style="color:#000;background-color:#dfd">+saslauth:x:981:76:Saslauthd user:/ran/saslauthd:/sbin/nologin
</span><span style="color:#000;background-color:#dfd"></span> usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
setroubleshoot:x:980:979::/var/lib/setroubleshoot:/sbin/nologin
openvpn:x:979:978:OpenVPN:/etc/openvpn:/sbin/nologin
<span style="color:#000;background-color:#fdd">-nm-openvpn:x:978:977:Default user for running openvpn spawned by NetworkManager:/:/sbin/nologin
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+mm-openvpn:x:978:977:Default user for running openvpn spawned by NetworkManager:/:/sbin/nologin
</span><span style="color:#000;background-color:#dfd"></span> qemu:x:107:107:qemu user:/:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
</code></pre></div><p>It’s not too hard to “eyeball” changes there if they add or remove characters and thus affect the line lengths. But a line with only a change to a single character isn’t as easy.</p>
<p>Since blank space is not the relevant separator in this file, standard word diff doesn’t help and in some cases is worse than line diff:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff"><code class="language-diff" data-lang="diff">$ git diff --word-diff passwd passwd.mangled
<span style="font-weight: bold">diff --git a/passwd b/passwd.mangled
index 981736c..6531f10 100644
--- a/passwd
+++ b/passwd.mangled</span>
<span style="color:blue">@@ -24,22 +24,22 @@</span> polkitd:x:996:991:User for polkitd:/:/sbin/nologin
rtkit:x:172:172:RealtimeKit:/proc:/sbin/nologin
pulse:x:171:171:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
chrony:x:995:988::/var/lib/chrony:/sbin/nologin
<span style="color:#000;background-color:tomato">[-abrt:x:173:173::/etc/abrt:/sbin/nologin-]</span><span style="color:#000;background-color:lightgreen">{+abrt:x:173:1730::/etc/abrt:/sbin/nologin+}</span>
colord:x:994:987:User for colord:/var/lib/colord:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
vboxadd:x:993:1::/var/run/vboxadd:/sbin/nologin
dnsmasq:x:985:985:Dnsmasq DHCP and DNS server:/var/lib/dnsmasq:/sbin/nologin
<span style="color:#000;background-color:tomato">[-tcpdump:x:72:72::/:/sbin/nologin-]</span><span style="color:#000;background-color:lightgreen">{+tcpdump:x:72:72::/:/bin/bash+}</span>
systemd-timesync:x:984:984:systemd Time Synchronization:/:/sbin/nologin
pipewire:x:983:983:PipeWire System Daemon:/var/run/pipewire:/sbin/nologin
gluster:x:982:982:GlusterFS daemons:/run/gluster:/sbin/nologin
<span style="color:#000;background-color:tomato">[-radvd:x:75:75:radvd-]</span><span style="color:#000;background-color:lightgreen">{+radvd:x:76:75:radvd+}</span> user:/:/sbin/nologin
saslauth:x:981:76:Saslauthd <span style="color:#000;background-color:tomato">[-user:/run/saslauthd:/sbin/nologin-]</span><span style="color:#000;background-color:lightgreen">{+user:/ran/saslauthd:/sbin/nologin+}</span>
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
setroubleshoot:x:980:979::/var/lib/setroubleshoot:/sbin/nologin
openvpn:x:979:978:OpenVPN:/etc/openvpn:/sbin/nologin
<span style="color:#000;background-color:tomato">[-nm-openvpn:x:978:977:Default-]</span><span style="color:#000;background-color:lightgreen">{+mm-openvpn:x:978:977:Default+}</span> user for running openvpn spawned by NetworkManager:/:/sbin/nologin
qemu:x:107:107:qemu user:/:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
</code></pre></div>
<p>Another venerable program similar to <code>wdiff</code> that is still maintained is <code>dwdiff</code>. In its self-description we read something intriguing:</p>
<blockquote>
<p>It is different from wdiff in that it allows the user to specify what should be considered whitespace …</p>
</blockquote>
<p>That sounds useful. But <code>dwdiff</code> is still a separate program and is even less common than <code>wdiff</code>. Can the versatile <code>git diff</code> help us here too?</p>
<p>Yes! <code>git diff</code> has the option <code>--word-diff-regex</code> to specify a regular expression to use instead of whitespace as a delimiter, like <code>dwdiff</code> does. The man page explanation notes:</p>
<blockquote>
<p>For example, <code>--word-diff-regex=.</code> will treat each character as a word and, correspondingly, show differences character by character.</p>
</blockquote>
<p>It also notes that <code>--word-diff</code> is assumed and can be omitted when using <code>--word-diff-regex</code>.</p>
<p>So let’s try that:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff"><code class="language-diff" data-lang="diff">$ git diff --word-diff-regex=. passwd passwd.mangled
<span style="font-weight: bold">diff --git a/passwd b/passwd.mangled
index 981736c..6531f10 100644
--- a/passwd
+++ b/passwd.mangled</span>
<span style="color:blue">@@ -24,22 +24,22 @@</span> polkitd:x:996:991:User for polkitd:/:/sbin/nologin
rtkit:x:172:172:RealtimeKit:/proc:/sbin/nologin
pulse:x:171:171:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
chrony:x:995:988::/var/lib/chrony:/sbin/nologin
abrt:x:173:173<span style="color:#000;background-color:lightgreen">{+0+}</span>::/etc/abrt:/sbin/nologin
colord:x:994:987:User for colord:/var/lib/colord:/sbin/nologin
rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
vboxadd:x:993:1::/var/run/vboxadd:/sbin/nologin
dnsmasq:x:985:985:Dnsmasq DHCP and DNS server:/var/lib/dnsmasq:/sbin/nologin
tcpdump:x:72:72::/:/<span style="color:#000;background-color:tomato">[-s-]</span>bin/<span style="color:#000;background-color:tomato">[-nologin-]</span><span style="color:#000;background-color:lightgreen">{+bash+}</span>
systemd-timesync:x:984:984:systemd Time Synchronization:/:/sbin/nologin
pipewire:x:983:983:PipeWire System Daemon:/var/run/pipewire:/sbin/nologin
gluster:x:982:982:GlusterFS daemons:/run/gluster:/sbin/nologin
radvd:x:7<span style="color:#000;background-color:tomato">[-5-]</span><span style="color:#000;background-color:lightgreen">{+6+}</span>:75:radvd user:/:/sbin/nologin
saslauth:x:981:76:Saslauthd user:/r<span style="color:#000;background-color:tomato">[-u-]</span><span style="color:#000;background-color:lightgreen">{+a+}</span>n/saslauthd:/sbin/nologin
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
setroubleshoot:x:980:979::/var/lib/setroubleshoot:/sbin/nologin
openvpn:x:979:978:OpenVPN:/etc/openvpn:/sbin/nologin
<span style="color:#000;background-color:tomato">[-n-]</span><span style="color:#000;background-color:lightgreen">{+m+}</span>m-openvpn:x:978:977:Default user for running openvpn spawned by NetworkManager:/:/sbin/nologin
qemu:x:107:107:qemu user:/:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
</code></pre></div>
<p>That’s quite good, at least to my eyes.</p>
<h3 id="on-the-web">On the web</h3>
<p>GitHub, GitLab, and Bitbucket do a good job of showing readable diffs for most common cases: line-oriented, but within each line also word or character diffs highlighted via color. Open up each of the following links to see how they present a few of our earlier examples as commit diffs:</p>
<ul>
<li>Change 2 letters in prose paragraph: <a href="https://github.com/jonjensen/word-diff-examples/commit/07ab55f0acb6410e30688515a7a0bc95ed6a37b1">🔗 in GitHub</a>, <a href="https://gitlab.com/jonjensen/word-diff-examples/-/commit/07ab55f0acb6410e30688515a7a0bc95ed6a37b1">🔗 in GitLab</a>, <a href="https://bitbucket.org/jonjensen/word-diff-examples/commits/07ab55f0acb6410e30688515a7a0bc95ed6a37b1">🔗 in Bitbucket</a></li>
<li>Change a few characters in <code>/etc/passwd</code>: <a href="https://github.com/jonjensen/word-diff-examples/commit/faa5f309a6d83a898a2b84fdf4c1c16189bb0be8">🔗 in GitHub</a>, <a href="https://gitlab.com/jonjensen/word-diff-examples/-/commit/faa5f309a6d83a898a2b84fdf4c1c16189bb0be8">🔗 in GitLab</a>, <a href="https://bitbucket.org/jonjensen/word-diff-examples/commits/faa5f309a6d83a898a2b84fdf4c1c16189bb0be8">🔗 in Bitbucket</a></li>
</ul>
<p>But GitHub and GitLab both break down on a reflowed paragraph, while Bitbucket shows a sensible diff of what logically changed, including spaces to newlines and vice versa:</p>
<ul>
<li>Insert words early in paragraph and reflow: <a href="https://github.com/jonjensen/word-diff-examples/commit/986ab1450f434d675b69c60b67a837f4bf11c84f">🔗 in GitHub</a>, <a href="https://gitlab.com/jonjensen/word-diff-examples/-/commit/986ab1450f434d675b69c60b67a837f4bf11c84f">🔗 in GitLab</a>, <a href="https://bitbucket.org/jonjensen/word-diff-examples/commits/986ab1450f434d675b69c60b67a837f4bf11c84f">🔗 in Bitbucket</a></li>
</ul>
<p>It appears that GitLab may soon gain proper cross-line word diff ability as seen in the project’s issue <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/325856">Add word-diff option to commits view</a>, which states its “Problem to solve” as:</p>
<blockquote>
<p>When working with markdown (or any type of prose/text in general), the “classic” git-diff (intended for code) is of limited use.</p>
</blockquote>
<p>Exactly right.</p>
<h3 id="ides">IDEs</h3>
<p>Visual Studio Code (VS Code) handles the above cases well out of the box for uncommitted changes in the current Git clone, and the GitLens extension helps it do the same for showing past commit diffs.</p>
<p>IntelliJ IDEA handles both cases well by default.</p>
<h3 id="for-those-left-behind">For those left behind</h3>
<p>To make the most of Git, you’ll want a fairly recent version, since new features are being added all the time. If you’re working on a server using the popular but aging CentOS 7 which comes with the ancient Git 1.8.3, you can follow our <a href="/blog/2021/12/installing-git-2-on-centos-7/">simple tutorial to upgrade to Git 2.34.1 or newer on CentOS 7</a>.</p>
<p>Enjoy!</p>
<h3 id="reference">Reference</h3>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Diff">diff</a> on Wikipedia, including history and samples of original, context, and unified context diff output</li>
<li><a href="https://en.wikipedia.org/wiki/Patch_(Unix)">patch</a> on Wikipedia</li>
<li><a href="https://git-scm.com/docs/git-diff">git-diff</a> man page</li>
<li><a href="https://www.gnu.org/software/wdiff/">wdiff</a></li>
<li><a href="https://os.ghalkes.nl/dwdiff.html">dwdiff</a></li>
<li><a href="https://en.wikipedia.org/wiki/Pangram">Pangrams</a> on Wikipedia, the source of our sample prose here</li>
</ul>
Installing Git 2 on CentOS 7https://www.endpointdev.com/blog/2021/12/installing-git-2-on-centos-7/2021-12-09T00:00:00+00:00Jon Jensen
<p><img src="/blog/2021/12/installing-git-2-on-centos-7/20210718-174926-sm.jpg" alt="Photo of a room with colorful translucent glass windows; wooden beams, walls, and ceiling; and heating vents"></p>
<!-- photo by Jon Jensen -->
<h3 id="git-ing-a-bit-stale">Git-ing a bit stale?</h3>
<p>RHEL/CentOS 7 is starting to feel a somewhat dated, but it still has over 2½ years before it reaches the end of its support lifetime that Red Hat has set for the end of June 2024.</p>
<p>One component that is far enough outdated to cause serious annoyance is Git.</p>
<p>Git is by far the most-used version control system in the world. It popularized the distributed model of tracking changes to source code files and greatly simplified collaboration by multiple developers. It is open source and free software and is used by most public and many internal software projects, and also by solo developers. IDEs such as VS Code and IntelliJ idea integrate with it. SaaS code hosting providers GitHub, GitLab, Bitbucket, and others are built around it. We have been using and <a href="/blog/2007/12/better-git-it-in-your-soul/">advocating Git since 2007</a> — see <a href="/blog/tags/git/">our blog posts about Git</a> for a variety of helpful articles.</p>
<p>CentOS/RHEL 7 includes Git version 1.8.3, which was released in May 2013. There have been 2 major, 36 minor, and <strong>216</strong> patch releases of Git in the 8½ years between then and the current version 2.34.1!</p>
<p>The makers of CentOS/RHEL have good reasons to stick with versions they shipped with for the lifetime of the operating system: compatibility, stability, predictability. They only want to release updates that are entirely compatible, mostly for bugfixes and security updates.</p>
<p>Using a newer version will likely have some differences, but for software primarily used interactively by humans who can adapt to change, this is often worth the tradeoff of more features vs. occasional unexpected change.</p>
<h3 id="we-help-you-freshen-up">We help you freshen up</h3>
<p>To relieve the pain of aging software, we here at End Point Dev package up newer Git versions for RHEL 7 as needed for ourselves and our clients. You can use it too! Our current packaged version as of today is the latest Git, version 2.34.1, which was released on 24 November 2021.</p>
<p>Why not just build the latest version from source? That works well on a single developer workstation if you don’t mind staying abreast of each new Git release on your own, and doing a bit of work to build them.</p>
<p>We recommend using packages specific to your OS because it is faster, easier, and fits well with automation tools such as Ansible, Salt, Chef, and Puppet. And when you configure a Yum repository that includes ongoing package updates, the updates are automatically applied to all your systems as part of your routine OS maintenance.</p>
<p>Here are instructions showing how you can install the latest Git package we built, on CentOS/RHEL 7 systems.</p>
<h3 id="check-your-version-of-git">Check your version of git</h3>
<p>First, see what version you have installed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git --version
git version 1.8.3.1
</code></pre></div><p>Now see where it is installed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ which git
/usr/bin/git
</code></pre></div><p>If you have <code>git</code> installed somewhere else, such as <code>/usr/local/bin/git</code>, it was probably built from source and installed there, and you should consider deleting that other installation before you install this new packaged one.</p>
<p>To make sure it was installed from an RPM using yum:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ rpm -qi git
Name : git
Version : 1.8.3.1
Release : 23.el7_8
Source RPM : git-1.8.3.1-23.el7_8.src.rpm
Build Date : Thu 28 May 2020 08:37:56 PM UTC
Build Host : x86-02.bsys.centos.org
Packager : CentOS BuildSystem <http://bugs.centos.org>
Vendor : CentOS
</code></pre></div><p>(I omitted a few uninteresting lines from that output so we can focus on the essentials.)</p>
<p>If instead of the above you see this from <code>rpm</code> and/or don’t have <code>git</code> installed at all:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ rpm -qi git
package git is not installed
$ which git
/usr/bin/which: no git in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
</code></pre></div><p>That’s fine. In that case you will just be installing Git on this system for the first time.</p>
<h3 id="add-the-end-point-yum-repository">Add the End Point Yum repository</h3>
<p>Now install the End Point package repository Yum repo package:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ sudo yum install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm
</code></pre></div><p>That adds two important files to your system:</p>
<h4 id="yum-repo-config-file">Yum repo config file</h4>
<p>The first is <code>/etc/yum.repos.d/endpoint.repo</code> which is configuration for <code>yum</code>, an extension to its main <code>/etc/yum.conf</code> configuration file. Here is where we point your Yum setup to our <a href="https://packages.endpointdev.com/">packages.endpointdev.com</a> repository to look for packages in the future.</p>
<h4 id="gnupg-key">GnuPG key</h4>
<p>The second is <code>/etc/pki/rpm-gpg/RPM-GPG-KEY-endpoint-7</code> which is the PGP/GnuPG public key matching the secret key we use to sign packages in our repository, so your <code>yum</code> can verify the packages have not been corrupted, either accidentally during transmission, or intentionally by Bad Folks.</p>
<h3 id="install-or-upgrade-git">Install or upgrade git</h3>
<p>Now installing or upgrading to the new version of Git is as simple as:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ sudo yum install git
</code></pre></div><p>You can use <code>yum upgrade git</code> if you already have it installed, but <code>yum install git</code> works in either case.</p>
<p>You should see output similar to this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: centos.mirror.constant.com
* epel: d2lzkl7pfhq30w.cloudfront.net
* extras: bay.uchicago.edu
* updates: ftp.usf.edu
Resolving Dependencies
--> Running transaction check
---> Package git.x86_64 0:1.8.3.1-23.el7_8 will be updated
--> Processing Dependency: git = 1.8.3.1-23.el7_8 for package: perl-Git-1.8.3.1-23.el7_8.noarch
---> Package git.x86_64 0:2.34.1-1.ep7 will be an update
--> Processing Dependency: git-core = 2.34.1-1.ep7 for package: git-2.34.1-1.ep7.x86_64
--> Processing Dependency: git-core-doc = 2.34.1-1.ep7 for package: git-2.34.1-1.ep7.x86_64
--> Processing Dependency: emacs-filesystem >= 24.3 for package: git-2.34.1-1.ep7.x86_64
--> Running transaction check
---> Package emacs-filesystem.noarch 1:24.3-23.el7 will be installed
---> Package git-core.x86_64 0:2.34.1-1.ep7 will be installed
--> Processing Dependency: libpcre2-8.so.0()(64bit) for package: git-core-2.34.1-1.ep7.x86_64
---> Package git-core-doc.noarch 0:2.34.1-1.ep7 will be installed
---> Package perl-Git.noarch 0:1.8.3.1-23.el7_8 will be updated
---> Package perl-Git.noarch 0:2.34.1-1.ep7 will be an update
--> Running transaction check
---> Package pcre2.x86_64 0:10.23-2.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
===============================================================
Package Arch Version Repository Size
===============================================================
Updating:
git x86_64 2.34.1-1.ep7 endpoint 69 k
Installing for dependencies:
emacs-filesystem noarch 1:24.3-23.el7 base 58 k
git-core x86_64 2.34.1-1.ep7 endpoint 5.7 M
git-core-doc noarch 2.34.1-1.ep7 endpoint 2.7 M
pcre2 x86_64 10.23-2.el7 base 201 k
Updating for dependencies:
perl-Git noarch 2.34.1-1.ep7 endpoint 43 k
Transaction Summary
===============================================================
Install ( 4 Dependent packages)
Upgrade 1 Package (+1 Dependent package)
Total download size: 8.7 M
Is this ok [y/d/N]: y
</code></pre></div><p>If you don’t see the new <code>git</code> version available, you may need to clear your Yum caches with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ sudo yum clean all
</code></pre></div><p>In the install/upgrade output above from <code>yum</code>, notice the question: “Is this ok”? As long as what you see is similar to the above, everything should be fine. But investigate further before proceeding if it proposed any unexpected package upgrades or removals.</p>
<p>If you agree to continue, you’ll see something like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/6): emacs-filesystem-24.3-23.el7.noarch.rpm | 58 kB 00:00:00
(2/6): git-2.34.1-1.ep7.x86_64.rpm | 69 kB 00:00:00
(3/6): pcre2-10.23-2.el7.x86_64.rpm | 201 kB 00:00:00
(4/6): git-core-doc-2.34.1-1.ep7.noarch.rpm | 2.7 MB 00:00:00
(5/6): perl-Git-2.34.1-1.ep7.noarch.rpm | 43 kB 00:00:00
(6/6): git-core-2.34.1-1.ep7.x86_64.rpm | 5.7 MB 00:00:01
-------------------------------------------------------------------
Total 6.9 MB/s | 8.7 MB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : 1:emacs-filesystem-24.3-23.el7.noarch 1/8
Installing : pcre2-10.23-2.el7.x86_64 2/8
Installing : git-core-2.34.1-1.ep7.x86_64 3/8
Installing : git-core-doc-2.34.1-1.ep7.noarch 4/8
Updating : perl-Git-2.34.1-1.ep7.noarch 5/8
Updating : git-2.34.1-1.ep7.x86_64 6/8
Cleanup : perl-Git-1.8.3.1-23.el7_8.noarch 7/8
Cleanup : git-1.8.3.1-23.el7_8.x86_64 8/8
Verifying : pcre2-10.23-2.el7.x86_64 1/8
Verifying : 1:emacs-filesystem-24.3-23.el7.noarch 2/8
Verifying : git-core-2.34.1-1.ep7.x86_64 3/8
Verifying : git-2.34.1-1.ep7.x86_64 4/8
Verifying : git-core-doc-2.34.1-1.ep7.noarch 5/8
Verifying : perl-Git-2.34.1-1.ep7.noarch 6/8
Verifying : git-1.8.3.1-23.el7_8.x86_64 7/8
Verifying : perl-Git-1.8.3.1-23.el7_8.noarch 8/8
Dependency Installed:
emacs-filesystem.noarch 1:24.3-23.el7 git-core.x86_64 0:2.34.1-1.ep7
git-core-doc.noarch 0:2.34.1-1.ep7 pcre2.x86_64 0:10.23-2.el7
Updated:
git.x86_64 0:2.34.1-1.ep7
Dependency Updated:
perl-Git.noarch 0:2.34.1-1.ep7
Complete!
</code></pre></div><p>Now check again which version of <code>git</code> is installed, and what the RPM database contains:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git --version
git version 2.34.1
$ rpm -qi git
Name : git
Version : 2.34.1
Release : 1.ep7
Architecture: x86_64
Source RPM : git-2.34.1-1.ep7.src.rpm
Build Date : Thu 09 Dec 2021 01:15:52 AM UTC
Build Host : rhel7-build64.epinfra.net
Packager : End Point Hosting Team <hosting@endpointdev.com>
Vendor : End Point Dev - https://packages.endpointdev.com/
</code></pre></div><p>(Again I removed a few uninteresting lines.)</p>
<p>We can see that this new version came from the End Point repository and is very fresh as of the time of this writing.</p>
<h3 id="bonus-newer-tmux">Bonus: Newer tmux!</h3>
<p>You may also be interested in our much newer tmux 3.2a vs. the tmux 1.8 that comes with CentOS 7.</p>
<p>If you want it, now that you have the End Point Yum repository configured, you can simply do:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ sudo yum install tmux
</code></pre></div><h3 id="reference">Reference</h3>
<ul>
<li><a href="https://packages.endpointdev.com/">End Point package repositories</a></li>
<li><a href="https://pkgs.org/">pkgs.org</a> makes it easy to find RPM packages across the various Linux distros, architectures, and repositories. We often use it to find source RPMs from Fedora Rawhide and other distros that we can rebuild for RHEL.</li>
<li><a href="https://centos.org/centos-linux/">CentOS Linux</a></li>
<li><a href="https://git-scm.com/">Git website</a></li>
<li><a href="https://github.blog/2021-11-15-highlights-from-git-2-34/">GitHub blog Highlights from Git 2.34</a> is a good exploration of important changes in the most recent version of Git. GitHub’s articles about each Git release are worth reading.</li>
<li><a href="https://en.wikipedia.org/wiki/Tmux">tmux on Wikipedia</a></li>
</ul>
How to split Git repositories into twohttps://www.endpointdev.com/blog/2017/08/how-to-split-git-repositories-into-two/2017-08-14T00:00:00+00:00Wojtek Ziniewicz
<p>Ever wondered how to split your Git repo into two repos?</p>
<div class="separator" style="clear: both; text-align: center;"><a href="/blog/2017/08/how-to-split-git-repositories-into-two/image-0-big.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="/blog/2017/08/how-to-split-git-repositories-into-two/image-0.png"/></a></div>
<p>First you need to find out what files and directories you want to move to separate repos. In the above example we’re moving dir3, dir4, and dir7 to repo A, and dir1, dir2, dir5, and dir8 to repo B.</p>
<h3 id="steps">Steps</h3>
<p>What you need to do is to go through <strong>each and every commit</strong> in git history for every branch and filter out commits that modify directories that you dont care about in your new repo. The only flaw of this method is that it will leave those empty, filtered out commits in the history.</p>
<h4 id="track-all-branches">Track all branches</h4>
<p>First we need to start tracking all branches locally:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"><span style="color:#080;font-weight:bold">for</span> i in <span style="color:#080;font-weight:bold">$(</span>git branch -r | grep -vE <span style="color:#d20;background-color:#fff0f0">"HEAD|master"</span> | sed <span style="color:#d20;background-color:#fff0f0">'s/^[ ]\+//'</span><span style="color:#080;font-weight:bold">)</span>;
<span style="color:#080;font-weight:bold">do</span> git checkout --track <span style="color:#369">$i</span>
<span style="color:#080;font-weight:bold">done</span>
</code></pre></div><p>Then copy your original repo to two separate dirs: repo_a and repo_b.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">cp -a source_repo repo_a
cp -a source_repo repo_b
</code></pre></div><h4 id="filter-the-history">Filter the history</h4>
<p>Following command will delete all dirs that exclusively belong to repo B, thus we create repo A. Filtering is not limited to directories. You can provide relative paths to files, dirs etc.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"><span style="color:#038">cd</span> repo_a
git filter-branch --index-filter <span style="color:#d20;background-color:#fff0f0">'git rm --cached -r dir8 dir2 || true'</span> -- --all
<span style="color:#038">cd</span> repo_b
git filter-branch --index-filter <span style="color:#d20;background-color:#fff0f0">'git rm --cached -r dir3 dir4 dir7 || true'</span> -- --all
</code></pre></div><p>Note that the <code>|| true</code> prevents git from failing to filter our dirs mentioned in the <code>rm</code> clause in early stages of the git history where the dirs did not yet exist.</p>
<p>Look at the list of branches once again (in both repos):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">git branch -l
</code></pre></div><h4 id="set-new-origins-and-push">Set new origins and push</h4>
<p>In every repo, we need to remove the old origin and set up new origin. After it’s done, we’re ready to push.</p>
<p>Remove old origin:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">git remote rm origin
</code></pre></div><p>Add new origin:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">git remote add origin git@github.com:YourOrg/repo_a.git
</code></pre></div><p>Push all tracked branches:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">git push origin --all
</code></pre></div><p>That’s it!</p>
Git: pre-receive hook error on CentOS 7https://www.endpointdev.com/blog/2015/12/git-pre-receive-hook-fatal-hang-up-error-centos-7/2015-12-29T00:00:00+00:00Marco Matarazzo
<p>We recently had to move a git repository from an old CentOS 5 to a new CentOS 7 server.</p>
<p>On the old CentOS 5 we had a recent, custom compiled version of git while on the new server we are using the system default old 1.8 version, shipped by the official CentOS repositories. And, as usual when you tell yourself “What could possibly go wrong?”, something did: every push began to return the dreaded “fatal: The remote end hung up unexpectedly” error.</p>
<p>After some time spent trying to debug the problem, we managed to isolate the problem to the pre-receive hook, specifically active on that repository. The script was very simple:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"> <span style="color:#888">#!/bin/bash</span>
<span style="color:#369">read_only_users</span>=<span style="color:#d20;background-color:#fff0f0">"alice bob"</span>
<span style="color:#080;font-weight:bold">for</span> user in <span style="color:#369">$read_only_users</span>
<span style="color:#080;font-weight:bold">do</span>
<span style="color:#080;font-weight:bold">if</span> [ <span style="color:#369">$USER</span> == <span style="color:#369">$user</span> ]; <span style="color:#080;font-weight:bold">then</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"User </span><span style="color:#369">$USER</span><span style="color:#d20;background-color:#fff0f0"> has read-only access, push blocked."</span>
<span style="color:#038">exit</span> <span style="color:#00d;font-weight:bold">1</span>
<span style="color:#080;font-weight:bold">fi</span>
<span style="color:#080;font-weight:bold">done</span>
</code></pre></div><p>… which apparently had no visible mistakes. On top of the lack of errors, this very same script used to work perfectly for years on the old server. Unfortunately, and quite disappointingly, even changing it to a simple:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"> <span style="color:#888">#!/bin/bash</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"These are not the droids you are looking for. Move along."</span>
</code></pre></div><p>…did not help and the error still persisted.</p>
<p>Searching for clues around forums and wikis, we found <a href="https://spuder.wordpress.com/2014/03/26/git-pre-receive-hooks/">this</a> blog post talking about parameters passed through stdin.</p>
<p>On <a href="https://git-scm.com/docs/githooks">Git docs</a>, we read that pre-receive hooks takes no arguments, but for each ref to be updated it receives on standard input a line of the format: <strong><old-value> SP <new-value> SP <ref-name> LF</strong>.</p>
<p>At that point, we tried with a sample script that actually reads and does something with stdin:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"> <span style="color:#888">#!/bin/bash</span>
<span style="color:#080;font-weight:bold">while</span> <span style="color:#038">read</span> oldrev newrev refname
<span style="color:#080;font-weight:bold">do</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"OLDREV: </span><span style="color:#369">$oldrev</span><span style="color:#d20;background-color:#fff0f0"> - NEWREV: </span><span style="color:#369">$newrev</span><span style="color:#d20;background-color:#fff0f0"> - REFNAME: </span><span style="color:#369">$refname</span><span style="color:#d20;background-color:#fff0f0">"</span>
<span style="color:#080;font-weight:bold">done</span>
</code></pre></div><p>…and voilà: pushes started working again. Lesson learned: never ignore stdin.</p>
Happy 10th birthday, Git!https://www.endpointdev.com/blog/2015/04/happy-10th-birthday-git/2015-04-08T00:00:00+00:00Jon Jensen
<p>Git’s birthday was yesterday. It is now 10 years old! Happy birthday, Git!</p>
<p>Git was born on 7 April 2005, as its creator <a href="http://marc.info/?l=git&m=117254154130732">Linus Torvalds recounted in a 2007 mailing list post</a>. At least if we consider the achievement of self-hosting to be “birth” for software like this. :)</p>
<p>Birthdays are really arbitrary moments in time, but they give us a reason to pause and reflect back. Why is Git a big deal?</p>
<p>Even if Git were still relatively obscure, for any serious software project to survive a decade and still be useful and maintained is an accomplishment. But Git is not just surviving.</p>
<p>Over the past 5–6 years, Git has become the standard version control system in the free software / open source world, and more recently, it is becoming the default version control system everywhere, including in the proprietary software world. It is amazing to consider how fast it has overtaken the older systems, and won out against competing newer systems too. It is not unreasonable these days to expect anyone who does software development, and especially anyone who claims to be familiar with version control systems, to be comfortable with Git.</p>
<p>So how did I get to be friends with Git, and end up at this birthday celebration?</p>
<p>After experimenting with Git and other distributed version control systems for a while in early and mid-2007, I started using Git for real work in July 2007. That is the earliest commit date in one my personal Git repositories (which was converted from an earlier CVS repository I started in 2000). Within a few weeks I was in love with Git. It was so obviously vastly superior to CVS and Subversion that I had mostly used before. It offered so much more power, control, and flexibility. The learning curve was real but tractable and it was so much easier to prevent or repair mistakes that I didn’t mind the retraining at all.</p>
<p>So I’m sounding like a fanboy. What was so much better?</p>
<p>First, the design. A distributed system where all commits were objects with a SHA-1 hash to identify them and the parent commit(s). Locally editable history. Piecemeal committing thanks to the staging power of the Git index. Cheap and quick branching. Better merging. A commit log that was really useful. Implicit rename tracking. Easy tagging and commit naming. And nothing missing from other systems that I needed.</p>
<p>Next, the implementation. Trivial setup, with no political and system administrative fuss for client or server. No messing with users and permissions and committer identities, just name & email address like we’re all used to. An efficient wire protocol. Simple ssh transport for pushes and/or pulls of remote repositories, if needed. A single .git directory at the repository root, rather than RCS, CVS, or .svn directories scattered throughout the checkout. A simple .git/config configuration file. And speed, so much speed, even for very large repositories with lots of binary blobs.</p>
<p>The speed is worth talking about more.</p>
<p>The speed of Git mattered, and was more than just a bonus. It proved true once again the adage that a big enough quantitative difference becomes a qualitative difference. Some people believed that speed of operations wasn’t all that important, but once you are able to complete your version control tasks so quickly that they’re not at all bothersome, it changes the way you work.</p>
<p>The ease of setting up an in-place repository on a whim, without worrying about where or if a central repository would ever be made, let alone wasting any time with access control, is a huge benefit. I used to administer CVS and Subversion repositories and life is so much better with the diminished role a “Git repository administrator” plays now.</p>
<p>Cheap topic branches for little experiments are easy. Committing every little thing separately makes sense if I can later reorder or combine or split my commits, and craft a sane commit before pushing it out where anyone else sees it.</p>
<p>Git subsumed everything else for us.</p>
<p>RCS, despite its major limitations, stuck around because CVS and Subversion didn’t do the nice quick in-place versioning that RCS does. That kind of workflow is so useful for a system administrator or ad-hoc local development work. But RCS has an ugly implementation and is based on changing single files, not sets. It can’t be promoted to real distributed version control later if needed. At End Point we used to use CVS, Subversion, and SVK (a distributed system built on top of Subversion), and also RCS for those cases where it still proved useful. Git replaced them all.</p>
<p>Distributed is better. Even for those who mostly use Git working against a central repository.</p>
<p>The RCS use case was a special limited subset of the bigger topic of distributed version control, which many people resisted and thought was overkill, or out of control, or whatever. But it is essential, and was key to fixing a lot of the problems of CVS and Subversion. Getting over the mental block of Git not having a single sequential integer revision number for each commit as Subversion did was hard for some people, but it forces us to confront the new reality: Commits are their own objects with an independent identity in a distributed world.</p>
<p>When I started using Git, Mercurial and Bazaar were the strongest distributed version control competitors. They were roughly feature-equivalent, and were solid contenders. But they were never as fast or compact on disk, and didn’t have Git’s index, cheap branching, stashing, or so many other niceties.</p>
<p>Then there is the ecosystem.</p>
<p>GitHub arrived on the scene as an unnecessary appendage at first, but its ease of use and popularity, and social coding encouragement, quickly made it an essential part of the Git community. It turned the occasional propagandistic accusation that Git was antisocial and would encourage project forks, into a virtue, by calling everyone’s clone a “fork”.</p>
<p>Over time GitHub has played a major role in making Git as popular as it is. Bypassing the need to set up any server software at all to get a central Git repository going removed a hurdle for many people. GitHub is a centralized service that can go down, but that is no serious risk in a distributed system where you generally have full repository mirrors all over the place, and can switch to other hosting any time if needed.</p>
<p>I realize I’m gushing praise embarrassingly at this point. I find it is warranted, based on my nearly 8 years of using Git and with good familiarity with the alternatives, old and new.</p>
<p>Thanks, Linus, for Git. Thanks, Junio C Hamano, who has maintained the Git open source project since early on.</p>
<p>Presumably someday something better will come along. Until then let’s enjoy this rare period of calm where there is an obvious winner to a common technology question and thus no needless debate before work can begin. And the current tool stability means we don’t have to learn new version control skills every few years.</p>
<p>To commemorate this 10-year birthday, <a href="http://www.linux.com/news/featured-blogs/185-jennifer-cloer/821541-10-years-of-git-an-interview-with-git-creator-linus-torvalds">Linux.com interviewed Linus</a> and it is a worthwhile read.</p>
<p>You may also be interested to read more in the <a href="https://en.wikipedia.org/wiki/Git_(software)">Wikipedia article on Git</a> or the <a href="https://git.wiki.kernel.org/index.php/GitHistory">Git wiki’s history of Git</a>.</p>
<p>Finally, an author named Stephen has written an article called <a href="http://www.netinstructions.com/the-case-for-git/">The case for Git in 2015</a>, which revisits the question of which version control system to use as if it were not yet a settled question. It has many good reminders of why Git has earned its prominent position.</p>
Testing your chef repo pull requests with chef-zero, Vagrant and Jenkinshttps://www.endpointdev.com/blog/2015/02/testing-your-chef-repo-pull-requests/2015-02-18T00:00:00+00:00Wojtek Ziniewicz
<p>All <a href="https://www.visionport.com/">Liquid Galaxy</a> setups deployed by End Point are managed by <a href="https://www.chef.io/chef/">Chef</a>. Typical deployment consists of approx 3 to 9 Linux boxes from which only 1 is managed and the rest is an ISO booted from this machine via network with copy-on-write root filesystem. Because of this, typical deployment involves more steps than just updating your code and restarting application. Deployment + rollback may be even 10 times longer compared with typical web application. Due to this fact, we need to test our infrastructure extensively.</p>
<p>What are we to do in order to make sure that our infrastructure is <strong>tested</strong> <strong>well</strong> <strong>before it hits production?</strong></p>
<div class="separator" style="clear: both; text-align: center;">
<br/></div>
<p>Scary? It’s not.</p>
<div class="separator" style="clear: both; text-align: center;">
<a href="/blog/2015/02/testing-your-chef-repo-pull-requests/image-0.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: left;"><img border="0" height="550" src="/blog/2015/02/testing-your-chef-repo-pull-requests/image-0.png" width="640"/></a></div>
<h4 id="workflow-broken-down-by-pieces">Workflow broken down by pieces</h4>
<ul>
<li><strong>lg_chef.git repo</strong>—where we keep cookbooks, environments and node definitions</li>
<li><strong>GitHub pull request</strong>—artifact of infrastructure source code tested by Jenkins</li>
<li><strong>Vagrant</strong>—virtual environment in which chef is run in order to test the artifact. There’s always 1 master node and few Vagrant boxes that boot an ISO from master via tftp protocol</li>
<li><strong>chef-zero</strong>—Chef flavor used to converge and test the infrastructure on the basis of GitHub pull request</li>
<li><strong>chef-server/chef-client</strong>—Chef flavor used to converge and test production and pre-production environment</li>
<li><strong>Jenkins</strong> — Continuous Integration environment that runs the converge process and part of the tests</li>
<li><strong>Tests</strong>—two frameworks used—<a href="https://github.com/sstephenson/bats">BATS</a> (for the integration tests on the top) and <a href="https://github.com/seattlerb/minitest">minitest</a> (for after-converge tests)</li>
<li><strong>lg-live-build</strong>—our fork of <a href="https://www.debian.org/devel/debian-live/">Debian live build</a> used to build the ISO that is booted by Vagrant slaves</li>
</ul>
<h4 id="workflow-broken-down-by-the-order-of-actions">Workflow broken down by the order of actions</h4>
<ol>
<li>User submits <strong>GitHub pull request</strong> to <strong>lg_chef.git</strong> repo</li>
<li><strong>GitHub pull request</strong> gets picked up by <strong>Jenkins</strong></li>
<li><strong>Jenkins</strong> creates 1 <strong>master</strong> <strong>Vagrant</strong> node and several slave nodes to act as <strong>slaves</strong></li>
<li><strong>chef-zero</strong> converges master <strong>Vagrant</strong> box and runs <strong>minitest</strong></li>
<li><strong>BATS</strong> tests run on the freshly converged <strong>Vagrant master</strong>box. Few steps are performed here: ISO is built, it’s distributed to the slaves, slaves boot the ISO and final integration tests are run to see whether slaves have all the goodness.</li>
<li>If points 1 to 5 are **green,**developer merges the changes, uploads the updated cookbooks, node definitions, roles and environments and runs the final tests.</li>
</ol>
<p><strong>What didn’t work for us and why</strong></p>
<ul>
<li><a href="https://github.com/test-kitchen/kitchen-vagrant">kitchen-vagrant </a>—because it didn’t play well with Jenkins (or JVM itself) and didn’t know how to use advanced Vagrant features for specifying multiple networking options, interfaces and drivers. However it supports using your own <a href="https://github.com/test-kitchen/kitchen-vagrant/blob/master/templates/Vagrantfile.erb">Vagrantfile.erb</a></li>
<li>We’ve had some doubts about keeping all the cookbooks, environments and node definitions in one repo because chef-server/chef-client tests can only test your stuff if it’s uploaded to the Chef server, but <strong>chef-zero</strong> came in handy</li>
</ul>
<h4 id="the-code">The code</h4>
<p>As previously mentioned, we needed our own vagrant template file.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-ruby" data-lang="ruby"><span style="color:#036;font-weight:bold">Vagrant</span>.configure(<span style="color:#d20;background-color:#fff0f0">"2"</span>) <span style="color:#080;font-weight:bold">do</span> |config|
<<span style="color:#2b2;background-color:#f0fff0">% if </span> <span style="color:#33b">@data</span>[<span style="color:#a60;background-color:#fff0f0">:chef_type</span>] == <span style="color:#d20;background-color:#fff0f0">"chef_zero"</span> <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> config.chef_zero.enabled = true
</span><span style="color:#d20;background-color:#fff0f0"> config.chef_zero.chef_server_url = "<%= @data[:chef_server_url] %></span><span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> config.chef_zero.roles = "</span>../../roles/<span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> config.chef_zero.cookbooks = "</span>../../cookbooks/<span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> config.chef_zero.environments = "</span>../../environments/<span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> config.chef_zero.data_bags = "</span>../integration/default/data_bags/<span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> <% else %>
</span><span style="color:#d20;background-color:#fff0f0"> config.chef_zero.enabled = false
</span><span style="color:#d20;background-color:#fff0f0"> <% end %>
</span><span style="color:#d20;background-color:#fff0f0"> config.omnibus.chef_version = "</span><<span style="color:#d20;background-color:#fff0f0">%= @data[:chef_version] %>"
</span><span style="color:#d20;background-color:#fff0f0"> config.vm.define "<%=</span> <span style="color:#33b">@data</span>[<span style="color:#a60;background-color:#fff0f0">:headnode</span>][<span style="color:#a60;background-color:#fff0f0">:slug</span>] <span style="color:#d20;background-color:#fff0f0">%>" do |h|
</span><span style="color:#d20;background-color:#fff0f0"> h.vm.box = "<%= @data[:headnode][:box] %></span><span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> h.vm.box_url = "</span><<span style="color:#d20;background-color:#fff0f0">%= @data[:headnode][:box_url] %>"
</span><span style="color:#d20;background-color:#fff0f0"> h.vm.hostname =</span> <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:hostname] %>"</span>
h.vm.network(<span style="color:#a60;background-color:#fff0f0">:private_network</span>, {<span style="color:#a60;background-color:#fff0f0">:ip</span> => <span style="color:#d20;background-color:#fff0f0">'10.42.41.1'</span>})
h.vm.synced_folder <span style="color:#d20;background-color:#fff0f0">"."</span>, <span style="color:#d20;background-color:#fff0f0">"/vagrant"</span>, <span style="color:#a60;background-color:#fff0f0">disabled</span>: <span style="color:#080">true</span>
h.vm.provider <span style="color:#a60;background-color:#fff0f0">:virtualbox</span> <span style="color:#080;font-weight:bold">do</span> |<span style="color:#038">p</span>|
<<span style="color:#2b2;background-color:#f0fff0">% @data[:headnode][:customizations].each </span> <span style="color:#080;font-weight:bold">do</span> |key, value| <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["modifyvm", :id, "<%= key %></span><span style="color:#d20;background-color:#fff0f0">", "</span><<span style="color:#d20;background-color:#fff0f0">%= value %>"]
</span><span style="color:#d20;background-color:#fff0f0"> <% end %>
</span><span style="color:#d20;background-color:#fff0f0"> end
</span><span style="color:#d20;background-color:#fff0f0"> h.vm.provision :chef_client do |chef|
</span><span style="color:#d20;background-color:#fff0f0"> <% if @data[:chef_type] =</span>= <span style="color:#d20;background-color:#fff0f0">"chef_zero"</span> <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> chef.environment = "<%= @data[:headnode][:provision][:environment] %></span><span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> chef.run_list = <%= @data[:run_list] %>
</span><span style="color:#d20;background-color:#fff0f0"> chef.json = <%= @data[:node_definition] %>
</span><span style="color:#d20;background-color:#fff0f0"> chef.chef_server_url = "</span><<span style="color:#d20;background-color:#fff0f0">%= @data[:chef_server_url] %>"
</span><span style="color:#d20;background-color:#fff0f0"> <% else %>
</span><span style="color:#d20;background-color:#fff0f0"> chef.chef_server_url =</span> <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:provision][:chef_server_url] %>"</span>
<<span style="color:#2b2;background-color:#f0fff0">% end </span> <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> chef.validation_key_path = "<%= @data[:headnode][:provision][:validation_key_path] %></span><span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> chef.encrypted_data_bag_secret_key_path = "</span><<span style="color:#d20;background-color:#fff0f0">%= @data[:headnode][:provision][:encrypted_data_bag_secret_key_path] %>"
</span><span style="color:#d20;background-color:#fff0f0"> chef.verbose_logging =</span> <<span style="color:#d20;background-color:#fff0f0">%= @data[:headnode][:provision][:verbose_logging] %>
</span><span style="color:#d20;background-color:#fff0f0"> chef.log_level =</span> <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:provision][:log_level] %>"</span>
<<span style="color:#2b2;background-color:#f0fff0">% end </span> <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> config.omnibus.chef_version = "<%= @data[:chef_version] %></span><span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> config.vm.define "</span><<span style="color:#d20;background-color:#fff0f0">%= @data[:headnode][:slug] %>" do |h|
</span><span style="color:#d20;background-color:#fff0f0"> h.vm.box =</span> <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:box] %>"</span>
h.vm.box_url = <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:box_url] %>"</span>
h.vm.hostname = <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:hostname] %>"</span>
h.vm.network(<span style="color:#a60;background-color:#fff0f0">:private_network</span>, {<span style="color:#a60;background-color:#fff0f0">:ip</span> => <span style="color:#d20;background-color:#fff0f0">'10.42.41.1'</span>})
h.vm.synced_folder <span style="color:#d20;background-color:#fff0f0">"."</span>, <span style="color:#d20;background-color:#fff0f0">"/vagrant"</span>, <span style="color:#a60;background-color:#fff0f0">disabled</span>: <span style="color:#080">true</span>
h.vm.provider <span style="color:#a60;background-color:#fff0f0">:virtualbox</span> <span style="color:#080;font-weight:bold">do</span> |<span style="color:#038">p</span>|
<<span style="color:#2b2;background-color:#f0fff0">% @data[:headnode][:customizations].each </span> <span style="color:#080;font-weight:bold">do</span> |key, value| <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["modifyvm", :id, "<%= key %></span><span style="color:#d20;background-color:#fff0f0">", "</span><<span style="color:#d20;background-color:#fff0f0">%= value %>"]
</span><span style="color:#d20;background-color:#fff0f0"> <% end %>
</span><span style="color:#d20;background-color:#fff0f0"> end
</span><span style="color:#d20;background-color:#fff0f0"> h.vm.provision :chef_client do |chef|
</span><span style="color:#d20;background-color:#fff0f0"> <% if @data[:chef_type] =</span>= <span style="color:#d20;background-color:#fff0f0">"chef_zero"</span> <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> chef.environment = "<%= @data[:headnode][:provision][:environment] %></span><span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> chef.run_list = <%= @data[:run_list] %>
</span><span style="color:#d20;background-color:#fff0f0"> chef.json = <%= @data[:node_definition] %>
</span><span style="color:#d20;background-color:#fff0f0"> chef.chef_server_url = "</span><<span style="color:#d20;background-color:#fff0f0">%= @data[:chef_server_url] %>"
</span><span style="color:#d20;background-color:#fff0f0"> <% else %>
</span><span style="color:#d20;background-color:#fff0f0"> chef.chef_server_url =</span> <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:provision][:chef_server_url] %>"</span>
<<span style="color:#2b2;background-color:#f0fff0">% end </span> <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> chef.validation_key_path = "<%= @data[:headnode][:provision][:validation_key_path] %></span><span style="color:#d20;background-color:#fff0f0">"
</span><span style="color:#d20;background-color:#fff0f0"> chef.encrypted_data_bag_secret_key_path = "</span><<span style="color:#d20;background-color:#fff0f0">%= @data[:headnode][:provision][:encrypted_data_bag_secret_key_path] %>"
</span><span style="color:#d20;background-color:#fff0f0"> chef.verbose_logging =</span> <<span style="color:#d20;background-color:#fff0f0">%= @data[:headnode][:provision][:verbose_logging] %>
</span><span style="color:#d20;background-color:#fff0f0"> chef.log_level =</span> <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:provision][:log_level] %>"</span>
chef.node_name = <span style="color:#d20;background-color:#fff0f0">"<%= @data[:headnode][:provision][:node_name] %>"</span>
<span style="color:#080;font-weight:bold">end</span>
<span style="color:#080;font-weight:bold">end</span>
<span style="color:#888">#display nodes</span>
<<span style="color:#2b2;background-color:#f0fff0">% @data[:display_nodes][:nodes].each </span> <span style="color:#080;font-weight:bold">do</span> |dn| <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> config.vm.define "<%= dn[:slug] %></span><span style="color:#d20;background-color:#fff0f0">" do |dn_config|
</span><span style="color:#d20;background-color:#fff0f0"> dn_config.vm.box_url = "</span><<span style="color:#d20;background-color:#fff0f0">%= @data[:display_nodes][:global][:box_url] %>"
</span><span style="color:#d20;background-color:#fff0f0"> dn_config.vm.hostname =</span> <span style="color:#d20;background-color:#fff0f0">"<%= dn[:hostname] %>"</span>
dn_config.vm.box = <span style="color:#d20;background-color:#fff0f0">"<%= @data[:display_nodes][:global][:box] %>"</span>
dn_config.vm.synced_folder <span style="color:#d20;background-color:#fff0f0">"."</span>, <span style="color:#d20;background-color:#fff0f0">"/vagrant"</span>, <span style="color:#a60;background-color:#fff0f0">disabled</span>: <span style="color:#080">true</span>
dn_config.vm.boot_timeout = <span style="color:#00d;font-weight:bold">1</span>
dn_config.vm.provider <span style="color:#a60;background-color:#fff0f0">:virtualbox</span> <span style="color:#080;font-weight:bold">do</span> |<span style="color:#038">p</span>|
<<span style="color:#2b2;background-color:#f0fff0">% @data[:display_nodes][:global][:customizations].each </span> <span style="color:#080;font-weight:bold">do</span> |key, value| <span style="color:#d20;background-color:#fff0f0">%>
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["modifyvm", :id, "<%= key %></span><span style="color:#d20;background-color:#fff0f0">", "</span><<span style="color:#d20;background-color:#fff0f0">%= value %>"]
</span><span style="color:#d20;background-color:#fff0f0"> <% end %>
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["modifyvm", :id, "--macaddress1", "<%=</span> dn[<span style="color:#a60;background-color:#fff0f0">:mac</span>] <span style="color:#d20;background-color:#fff0f0">%>"]
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["createhd", "--filename", "../files/<%= dn[:slug] %></span>.vmdk<span style="color:#d20;background-color:#fff0f0">", "</span>--size<span style="color:#d20;background-color:#fff0f0">", 80*1024]
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["</span>storageattach<span style="color:#d20;background-color:#fff0f0">", :id, "</span>--storagectl<span style="color:#d20;background-color:#fff0f0">", "</span><span style="color:#036;font-weight:bold">IDE</span> <span style="color:#036;font-weight:bold">Controller</span><span style="color:#d20;background-color:#fff0f0">", "</span>--port<span style="color:#d20;background-color:#fff0f0">", 0, "</span>--device<span style="color:#d20;background-color:#fff0f0">", 0, "</span>--type<span style="color:#d20;background-color:#fff0f0">", "</span>hdd<span style="color:#d20;background-color:#fff0f0">", "</span>--medium<span style="color:#d20;background-color:#fff0f0">", "</span>none<span style="color:#d20;background-color:#fff0f0">"]
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["</span>storageattach<span style="color:#d20;background-color:#fff0f0">", :id, "</span>--storagectl<span style="color:#d20;background-color:#fff0f0">", "</span><span style="color:#036;font-weight:bold">IDE</span> <span style="color:#036;font-weight:bold">Controller</span><span style="color:#d20;background-color:#fff0f0">", "</span>--port<span style="color:#d20;background-color:#fff0f0">", 0, "</span>--device<span style="color:#d20;background-color:#fff0f0">", 0, "</span>--type<span style="color:#d20;background-color:#fff0f0">", "</span>hdd<span style="color:#d20;background-color:#fff0f0">", "</span>--medium<span style="color:#d20;background-color:#fff0f0">", "</span>../files/<<span style="color:#d20;background-color:#fff0f0">%= dn[:slug] %>.vmdk"]
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["storagectl", :id, "--name", "SATA Controller", "--add", "sata", "--controller", "IntelAHCI", "--hostiocache", "on"]
</span><span style="color:#d20;background-color:#fff0f0"> p.customize ["storageattach", :id, "--storagectl", "SATA Controller", "--port", 1, "--device", 0, "--type", "hdd", "--medium", "../files/ipxe_<%=</span> dn[<span style="color:#a60;background-color:#fff0f0">:slug</span>] <span style="color:#d20;background-color:#fff0f0">%>.vmdk"]
</span><span style="color:#d20;background-color:#fff0f0"> end
</span><span style="color:#d20;background-color:#fff0f0"> end
</span><span style="color:#d20;background-color:#fff0f0"> <% end %></span>
<span style="color:#080;font-weight:bold">end</span>
</code></pre></div><p>It renders a Vagrant File out of following data:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">{
<span style="color:#d20;background-color:#fff0f0">"description"</span> : <span style="color:#d20;background-color:#fff0f0">"This file is used to generate Vagrantfile and run_test.sh and also run tests. It should contain _all_ data needed to render the templates and run teh tests."</span>,
<span style="color:#d20;background-color:#fff0f0">"chef_version"</span> : <span style="color:#d20;background-color:#fff0f0">"11.12.4"</span>,
<span style="color:#d20;background-color:#fff0f0">"chef_type"</span> : <span style="color:#d20;background-color:#fff0f0">"chef_zero"</span>,
<span style="color:#d20;background-color:#fff0f0">"vagrant_template_file"</span> : <span style="color:#d20;background-color:#fff0f0">"vagrantfile.erb"</span>,
<span style="color:#d20;background-color:#fff0f0">"run_tests_template_file"</span> : <span style="color:#d20;background-color:#fff0f0">"run_tests.sh.erb"</span>,
<span style="color:#d20;background-color:#fff0f0">"chef_server_url"</span> : <span style="color:#d20;background-color:#fff0f0">"http://192.168.1.2:4000"</span>,
<span style="color:#d20;background-color:#fff0f0">"headnode"</span> :
{
<span style="color:#d20;background-color:#fff0f0">"slug"</span> : <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests"</span>,
<span style="color:#d20;background-color:#fff0f0">"box"</span> : <span style="color:#d20;background-color:#fff0f0">"opscode-ubuntu-14.04"</span>,
<span style="color:#d20;background-color:#fff0f0">"box_url"</span> : <span style="color:#d20;background-color:#fff0f0">"https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-14.04_chef-provisionerless.box"</span>,
<span style="color:#d20;background-color:#fff0f0">"hostname"</span> : <span style="color:#d20;background-color:#fff0f0">"lg-head"</span>,
<span style="color:#d20;background-color:#fff0f0">"bats_tests_dir"</span> : <span style="color:#d20;background-color:#fff0f0">"projectX-pr"</span>,
<span style="color:#d20;background-color:#fff0f0">"customizations"</span> : {
<span style="color:#d20;background-color:#fff0f0">"--memory"</span> : <span style="color:#d20;background-color:#fff0f0">"2048"</span>,
<span style="color:#d20;background-color:#fff0f0">"--cpus"</span>: <span style="color:#d20;background-color:#fff0f0">"2"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nic1"</span> : <span style="color:#d20;background-color:#fff0f0">"nat"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nic2"</span>: <span style="color:#d20;background-color:#fff0f0">"intnet"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nic3"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nic4"</span> : <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nictype1"</span>: <span style="color:#d20;background-color:#fff0f0">"Am79C970A"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nictype2"</span>: <span style="color:#d20;background-color:#fff0f0">"Am79C970A"</span>,
<span style="color:#d20;background-color:#fff0f0">"--intnet2"</span>: <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests"</span>
},
<span style="color:#d20;background-color:#fff0f0">"provision"</span> : {
<span style="color:#d20;background-color:#fff0f0">"chef_server_url"</span> : <span style="color:#d20;background-color:#fff0f0">"https://chefserver.ourdomain.com:40443"</span>,
<span style="color:#d20;background-color:#fff0f0">"validation_key_path"</span> : <span style="color:#d20;background-color:#fff0f0">"~/.chef/validation.pem"</span>,
<span style="color:#d20;background-color:#fff0f0">"encrypted_data_bag_secret_key_path"</span> : <span style="color:#d20;background-color:#fff0f0">"~/.chef/encrypted_data_bag_secret"</span>,
<span style="color:#d20;background-color:#fff0f0">"node_name"</span> : <span style="color:#d20;background-color:#fff0f0">"lg-head-projectXtest.liquid.glx"</span>,
<span style="color:#d20;background-color:#fff0f0">"environment"</span> : <span style="color:#d20;background-color:#fff0f0">"pull_requests"</span>,
<span style="color:#d20;background-color:#fff0f0">"verbose_logging"</span> : <span style="color:#080;font-weight:bold">true</span>,
<span style="color:#d20;background-color:#fff0f0">"log_level"</span> : <span style="color:#d20;background-color:#fff0f0">"info"</span>
}
},
<span style="color:#d20;background-color:#fff0f0">"display_nodes"</span> : {
<span style="color:#d20;background-color:#fff0f0">"global"</span> : {
<span style="color:#d20;background-color:#fff0f0">"box"</span> : <span style="color:#d20;background-color:#fff0f0">"opscode-ubuntu-14.04"</span>,
<span style="color:#d20;background-color:#fff0f0">"box_url"</span> : <span style="color:#d20;background-color:#fff0f0">"https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-14.04_chef-provisionerless.box"</span>,
<span style="color:#d20;background-color:#fff0f0">"customizations"</span> : {
<span style="color:#d20;background-color:#fff0f0">"--memory"</span> : <span style="color:#d20;background-color:#fff0f0">"2048"</span>,
<span style="color:#d20;background-color:#fff0f0">"--cpus"</span> : <span style="color:#d20;background-color:#fff0f0">"1"</span>,
<span style="color:#d20;background-color:#fff0f0">"--boot1"</span> : <span style="color:#d20;background-color:#fff0f0">"floppy"</span>,
<span style="color:#d20;background-color:#fff0f0">"--boot2"</span> : <span style="color:#d20;background-color:#fff0f0">"net"</span>,
<span style="color:#d20;background-color:#fff0f0">"--boot3"</span> : <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#d20;background-color:#fff0f0">"--boot4"</span> : <span style="color:#d20;background-color:#fff0f0">"none"</span>
},
<span style="color:#d20;background-color:#fff0f0">"provision"</span> : {
<span style="color:#d20;background-color:#fff0f0">"chef_server_url"</span> : <span style="color:#d20;background-color:#fff0f0">"https://chefserver.ourdomain.com:40443"</span>,
<span style="color:#d20;background-color:#fff0f0">"validation_key_path"</span> : <span style="color:#d20;background-color:#fff0f0">"~/.chef/validation.pem"</span>,
<span style="color:#d20;background-color:#fff0f0">"encrypted_data_bag_secret_key_path"</span> : <span style="color:#d20;background-color:#fff0f0">"~/.chef/encrypted_data_bag_secret"</span>,
<span style="color:#d20;background-color:#fff0f0">"node_name"</span> : <span style="color:#d20;background-color:#fff0f0">"lg-head-projectXtest.liquid.glx"</span>,
<span style="color:#d20;background-color:#fff0f0">"environment"</span> : <span style="color:#d20;background-color:#fff0f0">"pull_requests"</span>,
<span style="color:#d20;background-color:#fff0f0">"verbose_logging"</span> : <span style="color:#080;font-weight:bold">true</span>,
<span style="color:#d20;background-color:#fff0f0">"log_level"</span> : <span style="color:#d20;background-color:#fff0f0">"info"</span>
}
},
<span style="color:#d20;background-color:#fff0f0">"display_nodes"</span> : {
<span style="color:#d20;background-color:#fff0f0">"global"</span> : {
<span style="color:#d20;background-color:#fff0f0">"box"</span> : <span style="color:#d20;background-color:#fff0f0">"opscode-ubuntu-14.04"</span>,
<span style="color:#d20;background-color:#fff0f0">"box_url"</span> : <span style="color:#d20;background-color:#fff0f0">"https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-14.04_chef-provisionerless.box"</span>,
<span style="color:#d20;background-color:#fff0f0">"customizations"</span> : {
<span style="color:#d20;background-color:#fff0f0">"--memory"</span> : <span style="color:#d20;background-color:#fff0f0">"2048"</span>,
<span style="color:#d20;background-color:#fff0f0">"--cpus"</span> : <span style="color:#d20;background-color:#fff0f0">"1"</span>,
<span style="color:#d20;background-color:#fff0f0">"--boot1"</span> : <span style="color:#d20;background-color:#fff0f0">"floppy"</span>,
<span style="color:#d20;background-color:#fff0f0">"--boot2"</span> : <span style="color:#d20;background-color:#fff0f0">"net"</span>,
<span style="color:#d20;background-color:#fff0f0">"--boot3"</span> : <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#d20;background-color:#fff0f0">"--boot4"</span> : <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#d20;background-color:#fff0f0">"--intnet1"</span> : <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nicpromisc1"</span>: <span style="color:#d20;background-color:#fff0f0">"allow-all"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nic1"</span> : <span style="color:#d20;background-color:#fff0f0">"intnet"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nic2"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nic3"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nic4"</span>: <span style="color:#d20;background-color:#fff0f0">"none"</span>,
<span style="color:#d20;background-color:#fff0f0">"--nictype1"</span>: <span style="color:#d20;background-color:#fff0f0">"Am79C970A"</span>,
<span style="color:#d20;background-color:#fff0f0">"--ioapic"</span>: <span style="color:#d20;background-color:#fff0f0">"on"</span>
}
},
<span style="color:#d20;background-color:#fff0f0">"nodes"</span> : [
{
<span style="color:#d20;background-color:#fff0f0">"slug"</span> : <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests-kiosk"</span>,
<span style="color:#d20;background-color:#fff0f0">"hostname"</span> : <span style="color:#d20;background-color:#fff0f0">"kiosk"</span>,
<span style="color:#d20;background-color:#fff0f0">"mac"</span> : <span style="color:#d20;background-color:#fff0f0">"5ca1ab1e0001"</span>
},
{
<span style="color:#d20;background-color:#fff0f0">"slug"</span> : <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests-display"</span>,
<span style="color:#d20;background-color:#fff0f0">"hostname"</span> : <span style="color:#d20;background-color:#fff0f0">"display"</span>,
<span style="color:#d20;background-color:#fff0f0">"mac"</span> : <span style="color:#d20;background-color:#fff0f0">"5ca1ab1e0002"</span>
}
]
}
}
</code></pre></div><p>As a result we get an on-the-fly Vagrantfile that’s used during the testing:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">Vagrant.configure(<span style="color:#d20;background-color:#fff0f0">"2"</span>) <span style="color:#080;font-weight:bold">do</span> |config|
config.chef_zero.enabled = <span style="color:#080;font-weight:bold">true</span>
config.chef_zero.chef_server_url = <span style="color:#d20;background-color:#fff0f0">"http://192.168.1.2:4000"</span>
config.chef_zero.roles = <span style="color:#d20;background-color:#fff0f0">"../../roles/"</span>
config.chef_zero.cookbooks = <span style="color:#d20;background-color:#fff0f0">"../../cookbooks/"</span>
config.chef_zero.environments = <span style="color:#d20;background-color:#fff0f0">"../../environments/"</span>
config.chef_zero.data_bags = <span style="color:#d20;background-color:#fff0f0">"../integration/default/data_bags/"</span>
config.omnibus.chef_version = <span style="color:#d20;background-color:#fff0f0">"11.12.4"</span>
config.vm.define <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests"</span> <span style="color:#080;font-weight:bold">do</span> |h|
h.vm.box = <span style="color:#d20;background-color:#fff0f0">"opscode-ubuntu-14.04"</span>
h.vm.box_url = <span style="color:#d20;background-color:#fff0f0">"https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-14.04_chef-provisionerless.box"</span>
h.vm.hostname = <span style="color:#d20;background-color:#fff0f0">"lg-head"</span>
h.vm.network(:private_network, {:ip => <span style="color:#d20;background-color:#fff0f0">'10.42.41.1'</span>})
h.vm.synced_folder <span style="color:#d20;background-color:#fff0f0">"."</span>, <span style="color:#d20;background-color:#fff0f0">"/vagrant"</span>, disabled: <span style="color:#080;font-weight:bold">true</span>
h.vm.provider :virtualbox <span style="color:#080;font-weight:bold">do</span> |p|
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--memory"</span>, <span style="color:#d20;background-color:#fff0f0">"2048"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--cpus"</span>, <span style="color:#d20;background-color:#fff0f0">"2"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic1"</span>, <span style="color:#d20;background-color:#fff0f0">"nat"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic2"</span>, <span style="color:#d20;background-color:#fff0f0">"intnet"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic3"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic4"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nictype1"</span>, <span style="color:#d20;background-color:#fff0f0">"Am79C970A"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nictype2"</span>, <span style="color:#d20;background-color:#fff0f0">"Am79C970A"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--intnet2"</span>, <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests"</span>]
end
h.vm.provision :chef_client <span style="color:#080;font-weight:bold">do</span> |chef|
chef.environment = <span style="color:#d20;background-color:#fff0f0">"pull_requests"</span>
chef.run_list = [<span style="color:#d20;background-color:#fff0f0">"role[lg-head-nocms]"</span>, <span style="color:#d20;background-color:#fff0f0">"recipe[lg_live_build]"</span>, <span style="color:#d20;background-color:#fff0f0">"recipe[lg_tftproot]"</span>, <span style="color:#d20;background-color:#fff0f0">"recipe[lg_projectX]"</span>, <span style="color:#d20;background-color:#fff0f0">"recipe[lg_test]"</span>, <span style="color:#d20;background-color:#fff0f0">"recipe[test_mode::bats]"</span>, <span style="color:#d20;background-color:#fff0f0">"recipe[hostsfile::projectX]"</span>]
chef.json = {:sysctl=>{:params=>{:vm=>{:swappiness=><span style="color:#00d;font-weight:bold">20</span>}, :net=>{:ipv4=>{:ip_forward=><span style="color:#00d;font-weight:bold">1</span>}}}}, :test_mode=><span style="color:#080;font-weight:bold">true</span>, :call_ep=>{:keyname=><span style="color:#d20;background-color:#fff0f0">"ProjectX CI Production"</span>, :fwdport=><span style="color:#00d;font-weight:bold">33299</span>}, :tftproot=>{:managed=><span style="color:#080;font-weight:bold">true</span>}, :tags=>[], :lg_cms=>{:remote=><span style="color:#d20;background-color:#fff0f0">"github"</span>}, :monitor=><span style="color:#080;font-weight:bold">true</span>, :lg_grub=>{:cmdline=><span style="color:#d20;background-color:#fff0f0">"nomodeset biosdevname=0"</span>}, :projectX=>{:repo_branch=><span style="color:#d20;background-color:#fff0f0">"development"</span>, :display_host=><span style="color:#d20;background-color:#fff0f0">"42-a"</span>, :kiosk_host=><span style="color:#d20;background-color:#fff0f0">"42-b"</span>, :sensors_host=><span style="color:#d20;background-color:#fff0f0">"42-b"</span>, :maps_url=><span style="color:#d20;background-color:#fff0f0">"https://www.google.com/maps/@8.135687,-75.0973243,17856994a,40.4y,1.23h/data=!3m1!1e3?esrch=Tactile::TactileAcme,Tactile::ImmersiveModeEnabled"</span>}, :liquid_galaxy=>{:touchscreen_link=><span style="color:#d20;background-color:#fff0f0">"/dev/input/lg_active_touch"</span>, :screenshotd=>{:screen_rows=><span style="color:#d20;background-color:#fff0f0">"1"</span>, :screen_columns=><span style="color:#d20;background-color:#fff0f0">"1"</span>}, :screenshot_service=><span style="color:#080;font-weight:bold">true</span>, :display_nodes=>[{:hostname=><span style="color:#d20;background-color:#fff0f0">"42-c"</span>}, {:allowed_pages=>[<span style="color:#d20;background-color:#fff0f0">"Google Maps"</span>, <span style="color:#d20;background-color:#fff0f0">"Pacman Doodle"</span>, <span style="color:#d20;background-color:#fff0f0">"jellyfish"</span>, <span style="color:#d20;background-color:#fff0f0">"Doodle Selection"</span>, <span style="color:#d20;background-color:#fff0f0">"ProjectX Video Player"</span>, <span style="color:#d20;background-color:#fff0f0">"Composer kiosk"</span>], :hostname=><span style="color:#d20;background-color:#fff0f0">"42-b"</span>, :mac=><span style="color:#d20;background-color:#fff0f0">"5c:a1:ab:1e:00:02"</span>, :features=><span style="color:#d20;background-color:#fff0f0">"mandatory_windows, plain_gray, starry_skies"</span>, :bad_windows_names=><span style="color:#d20;background-color:#fff0f0">"Google Earth - Login Status"</span>, :mandatory_windows_names=><span style="color:#d20;background-color:#fff0f0">"awesome"</span>, :screens=>[{:display=><span style="color:#d20;background-color:#fff0f0">":0"</span>, :crtc=><span style="color:#d20;background-color:#fff0f0">"default"</span>, :grid_order=><span style="color:#d20;background-color:#fff0f0">"0"</span>}], :screen_rotation=><span style="color:#d20;background-color:#fff0f0">"normal"</span>, :audio_device=><span style="color:#d20;background-color:#fff0f0">"{type hw; card DGX; device 0}"</span>, :onboard_enable=><span style="color:#080;font-weight:bold">true</span>, :keyboard_enable=><span style="color:#080;font-weight:bold">true</span>, :mouse_enable=><span style="color:#080;font-weight:bold">true</span>, :cursor_enable=><span style="color:#080;font-weight:bold">true</span>, :background_extension=><span style="color:#d20;background-color:#fff0f0">"jpg"</span>, :background_mode=><span style="color:#d20;background-color:#fff0f0">"zoom-fill"</span>, :projectX=>{:extensions=>{:kiosk=><span style="color:#d20;background-color:#fff0f0">"ProjectX Kiosk"</span>, :google_properties_menu=><span style="color:#d20;background-color:#fff0f0">"Google Properties Menu"</span>, :onboard=><span style="color:#d20;background-color:#fff0f0">"Onboard"</span>, :no_right_click=><span style="color:#d20;background-color:#fff0f0">"Right Click Killer"</span>, :render_statistics=><span style="color:#d20;background-color:#fff0f0">"Render Statistics"</span>}, :browser_slug=><span style="color:#d20;background-color:#fff0f0">"lgS0"</span>, :urls=><span style="color:#d20;background-color:#fff0f0">"https://www.google.com/maps"</span>, :ros_nodes=>[{:name=><span style="color:#d20;background-color:#fff0f0">"rfreceiver_reset"</span>, :pkg=><span style="color:#d20;background-color:#fff0f0">"rfreceiver"</span>, :type=><span style="color:#d20;background-color:#fff0f0">"kill_browser.py"</span>}, {:name=><span style="color:#d20;background-color:#fff0f0">"proximity"</span>, :pkg=><span style="color:#d20;background-color:#fff0f0">"maxbotix"</span>, :type=><span style="color:#d20;background-color:#fff0f0">"sender.py"</span>}, {:name=><span style="color:#d20;background-color:#fff0f0">"spacenav"</span>, :pkg=><span style="color:#d20;background-color:#fff0f0">"spacenav_node"</span>, :type=><span style="color:#d20;background-color:#fff0f0">"spacenav_node"</span>}, {:name=><span style="color:#d20;background-color:#fff0f0">"leap"</span>, :pkg=><span style="color:#d20;background-color:#fff0f0">"leap_motion"</span>, :type=><span style="color:#d20;background-color:#fff0f0">"sender.py"</span>}, {:name=><span style="color:#d20;background-color:#fff0f0">"projectX_nav"</span>, :pkg=><span style="color:#d20;background-color:#fff0f0">"projectX_nav"</span>, :type=><span style="color:#d20;background-color:#fff0f0">"projectX_nav"</span>}, {:name=><span style="color:#d20;background-color:#fff0f0">"onboard"</span>, :pkg=><span style="color:#d20;background-color:#fff0f0">"onboard"</span>, :type=><span style="color:#d20;background-color:#fff0f0">"listener.py"</span>}, {:name=><span style="color:#d20;background-color:#fff0f0">"rosbridge"</span>, :pkg=><span style="color:#d20;background-color:#fff0f0">"rosbridge_server"</span>, :type=><span style="color:#d20;background-color:#fff0f0">"rosbridge_websocket"</span>, :params=>[{:name=><span style="color:#d20;background-color:#fff0f0">"certfile"</span>, :value=><span style="color:#d20;background-color:#fff0f0">"/home/lg/etc/ros.crt"</span>}, {:name=><span style="color:#d20;background-color:#fff0f0">"keyfile"</span>, :value=><span style="color:#d20;background-color:#fff0f0">"/home/lg/etc/ros.key"</span>}]}]}, :browser_infinite_url=><span style="color:#d20;background-color:#fff0f0">"http://lg-head/projectX-loader.html"</span>}, {:hostname=><span style="color:#d20;background-color:#fff0f0">"42-a"</span>, :mac=><span style="color:#d20;background-color:#fff0f0">"5c:a1:ab:1e:00:01"</span>, :features=><span style="color:#d20;background-color:#fff0f0">"mandatory_windows, plain_gray, starry_skies, erroneous_text"</span>, :bad_windows_names=><span style="color:#d20;background-color:#fff0f0">"Google Earth - Login Status"</span>, :mandatory_windows_names=><span style="color:#d20;background-color:#fff0f0">"awesome"</span>, :screens=>[{:display=><span style="color:#d20;background-color:#fff0f0">":0"</span>, :crtc=><span style="color:#d20;background-color:#fff0f0">"default"</span>, :grid_order=><span style="color:#d20;background-color:#fff0f0">"1"</span>}], :keyboard_enable=><span style="color:#080;font-weight:bold">true</span>, :mouse_enable=><span style="color:#080;font-weight:bold">true</span>, :cursor_enable=><span style="color:#080;font-weight:bold">true</span>, :background_extension=><span style="color:#d20;background-color:#fff0f0">"jpg"</span>, :background_mode=><span style="color:#d20;background-color:#fff0f0">"zoom-fill"</span>, :nvidia_mosaic=><span style="color:#080;font-weight:bold">true</span>, :manual_layout=>{:<span style="color:#080;font-weight:bold">default</span>=><span style="color:#d20;background-color:#fff0f0">"1024x768+0+0"</span>}, :projectX=>{:extensions=>{:display=><span style="color:#d20;background-color:#fff0f0">"ProjectX Large Display"</span>, :pacman=><span style="color:#d20;background-color:#fff0f0">"pacman"</span>, :render_statistics=><span style="color:#d20;background-color:#fff0f0">"Render Statistics"</span>}, :browser_slug=><span style="color:#d20;background-color:#fff0f0">"lgS0"</span>, :urls=><span style="color:#d20;background-color:#fff0f0">"https://www.google.com/maps"</span>, :ros_nodes=>[{:name=><span style="color:#d20;background-color:#fff0f0">"geodata"</span>, :pkg=><span style="color:#d20;background-color:#fff0f0">"geodata"</span>, :type=><span style="color:#d20;background-color:#fff0f0">"geodata_server.py"</span>}]}, :browser_infinite_url=><span style="color:#d20;background-color:#fff0f0">"http://lg-head/projectX-loader.html"</span>, :default_browser_bin=><span style="color:#d20;background-color:#fff0f0">"google-chrome"</span>, :allowed_pages=>[<span style="color:#d20;background-color:#fff0f0">"Google Maps"</span>, <span style="color:#d20;background-color:#fff0f0">"Pacman Doodle"</span>, <span style="color:#d20;background-color:#fff0f0">"jellyfish"</span>, <span style="color:#d20;background-color:#fff0f0">"ProjectX Video Player"</span>, <span style="color:#d20;background-color:#fff0f0">"Composer wall"</span>]}], :has_cec=><span style="color:#080;font-weight:bold">false</span>, :google_office=><span style="color:#080;font-weight:bold">false</span>, :viewsync_master=><span style="color:#d20;background-color:#fff0f0">"42-b"</span>, :has_touchscreen=><span style="color:#080;font-weight:bold">false</span>, :has_spacenav=><span style="color:#080;font-weight:bold">true</span>, :support_name=><span style="color:#d20;background-color:#fff0f0">"projectX-ci"</span>, :podium_interface=><span style="color:#d20;background-color:#fff0f0">"http://lg-head"</span>, :podium_display=><span style="color:#d20;background-color:#fff0f0">"42-b:default"</span>}}
chef.chef_server_url = <span style="color:#d20;background-color:#fff0f0">"http://192.168.1.2:4000"</span>
chef.validation_key_path = <span style="color:#d20;background-color:#fff0f0">"~/.chef/validation.pem"</span>
chef.encrypted_data_bag_secret_key_path = <span style="color:#d20;background-color:#fff0f0">"~/.chef/encrypted_data_bag_secret"</span>
chef.verbose_logging = <span style="color:#080;font-weight:bold">true</span>
chef.log_level = <span style="color:#d20;background-color:#fff0f0">"info"</span>
chef.node_name = <span style="color:#d20;background-color:#fff0f0">"lg-head-projectXtest.liquid.glx"</span>
end
end
<span style="color:#a61717;background-color:#e3d2d2">#</span>display nodes
config.vm.define <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests-kiosk"</span> <span style="color:#080;font-weight:bold">do</span> |dn_config|
dn_config.vm.box_url = <span style="color:#d20;background-color:#fff0f0">"https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-14.04_chef-provisionerless.box"</span>
dn_config.vm.hostname = <span style="color:#d20;background-color:#fff0f0">"kiosk"</span>
dn_config.vm.box = <span style="color:#d20;background-color:#fff0f0">"opscode-ubuntu-14.04"</span>
dn_config.vm.synced_folder <span style="color:#d20;background-color:#fff0f0">"."</span>, <span style="color:#d20;background-color:#fff0f0">"/vagrant"</span>, disabled: <span style="color:#080;font-weight:bold">true</span>
dn_config.vm.boot_timeout = <span style="color:#00d;font-weight:bold">1</span>
dn_config.vm.provider :virtualbox <span style="color:#080;font-weight:bold">do</span> |p|
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--memory"</span>, <span style="color:#d20;background-color:#fff0f0">"2048"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--cpus"</span>, <span style="color:#d20;background-color:#fff0f0">"1"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--boot1"</span>, <span style="color:#d20;background-color:#fff0f0">"floppy"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--boot2"</span>, <span style="color:#d20;background-color:#fff0f0">"net"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--boot3"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--boot4"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--intnet1"</span>, <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nicpromisc1"</span>, <span style="color:#d20;background-color:#fff0f0">"allow-all"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic1"</span>, <span style="color:#d20;background-color:#fff0f0">"intnet"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic2"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic3"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic4"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nictype1"</span>, <span style="color:#d20;background-color:#fff0f0">"Am79C970A"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--ioapic"</span>, <span style="color:#d20;background-color:#fff0f0">"on"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--macaddress1"</span>, <span style="color:#d20;background-color:#fff0f0">"5ca1ab1e0001"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"createhd"</span>, <span style="color:#d20;background-color:#fff0f0">"--filename"</span>, <span style="color:#d20;background-color:#fff0f0">"../files/projectX-pull-requests-kiosk.vmdk"</span>, <span style="color:#d20;background-color:#fff0f0">"--size"</span>, <span style="color:#00d;font-weight:bold">80</span>*<span style="color:#00d;font-weight:bold">1024</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"storageattach"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--storagectl"</span>, <span style="color:#d20;background-color:#fff0f0">"IDE Controller"</span>, <span style="color:#d20;background-color:#fff0f0">"--port"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--device"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--type"</span>, <span style="color:#d20;background-color:#fff0f0">"hdd"</span>, <span style="color:#d20;background-color:#fff0f0">"--medium"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"storageattach"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--storagectl"</span>, <span style="color:#d20;background-color:#fff0f0">"IDE Controller"</span>, <span style="color:#d20;background-color:#fff0f0">"--port"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--device"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--type"</span>, <span style="color:#d20;background-color:#fff0f0">"hdd"</span>, <span style="color:#d20;background-color:#fff0f0">"--medium"</span>, <span style="color:#d20;background-color:#fff0f0">"../files/projectX-pull-requests-kiosk.vmdk"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"storagectl"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--name"</span>, <span style="color:#d20;background-color:#fff0f0">"SATA Controller"</span>, <span style="color:#d20;background-color:#fff0f0">"--add"</span>, <span style="color:#d20;background-color:#fff0f0">"sata"</span>, <span style="color:#d20;background-color:#fff0f0">"--controller"</span>, <span style="color:#d20;background-color:#fff0f0">"IntelAHCI"</span>, <span style="color:#d20;background-color:#fff0f0">"--hostiocache"</span>, <span style="color:#d20;background-color:#fff0f0">"on"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"storageattach"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--storagectl"</span>, <span style="color:#d20;background-color:#fff0f0">"SATA Controller"</span>, <span style="color:#d20;background-color:#fff0f0">"--port"</span>, <span style="color:#00d;font-weight:bold">1</span>, <span style="color:#d20;background-color:#fff0f0">"--device"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--type"</span>, <span style="color:#d20;background-color:#fff0f0">"hdd"</span>, <span style="color:#d20;background-color:#fff0f0">"--medium"</span>, <span style="color:#d20;background-color:#fff0f0">"../files/ipxe_projectX-pull-requests-kiosk.vmdk"</span>]
end
end
config.vm.define <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests-display"</span> <span style="color:#080;font-weight:bold">do</span> |dn_config|
dn_config.vm.box_url = <span style="color:#d20;background-color:#fff0f0">"https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-14.04_chef-provisionerless.box"</span>
dn_config.vm.hostname = <span style="color:#d20;background-color:#fff0f0">"display"</span>
dn_config.vm.box = <span style="color:#d20;background-color:#fff0f0">"opscode-ubuntu-14.04"</span>
dn_config.vm.synced_folder <span style="color:#d20;background-color:#fff0f0">"."</span>, <span style="color:#d20;background-color:#fff0f0">"/vagrant"</span>, disabled: <span style="color:#080;font-weight:bold">true</span>
dn_config.vm.boot_timeout = <span style="color:#00d;font-weight:bold">1</span>
dn_config.vm.provider :virtualbox <span style="color:#080;font-weight:bold">do</span> |p|
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--memory"</span>, <span style="color:#d20;background-color:#fff0f0">"2048"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--cpus"</span>, <span style="color:#d20;background-color:#fff0f0">"1"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--boot1"</span>, <span style="color:#d20;background-color:#fff0f0">"floppy"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--boot2"</span>, <span style="color:#d20;background-color:#fff0f0">"net"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--boot3"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--boot4"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--intnet1"</span>, <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nicpromisc1"</span>, <span style="color:#d20;background-color:#fff0f0">"allow-all"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic1"</span>, <span style="color:#d20;background-color:#fff0f0">"intnet"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic2"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic3"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nic4"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--nictype1"</span>, <span style="color:#d20;background-color:#fff0f0">"Am79C970A"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--ioapic"</span>, <span style="color:#d20;background-color:#fff0f0">"on"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"modifyvm"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--macaddress1"</span>, <span style="color:#d20;background-color:#fff0f0">"5ca1ab1e0002"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"createhd"</span>, <span style="color:#d20;background-color:#fff0f0">"--filename"</span>, <span style="color:#d20;background-color:#fff0f0">"../files/projectX-pull-requests-display.vmdk"</span>, <span style="color:#d20;background-color:#fff0f0">"--size"</span>, <span style="color:#00d;font-weight:bold">80</span>*<span style="color:#00d;font-weight:bold">1024</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"storageattach"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--storagectl"</span>, <span style="color:#d20;background-color:#fff0f0">"IDE Controller"</span>, <span style="color:#d20;background-color:#fff0f0">"--port"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--device"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--type"</span>, <span style="color:#d20;background-color:#fff0f0">"hdd"</span>, <span style="color:#d20;background-color:#fff0f0">"--medium"</span>, <span style="color:#d20;background-color:#fff0f0">"none"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"storageattach"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--storagectl"</span>, <span style="color:#d20;background-color:#fff0f0">"IDE Controller"</span>, <span style="color:#d20;background-color:#fff0f0">"--port"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--device"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--type"</span>, <span style="color:#d20;background-color:#fff0f0">"hdd"</span>, <span style="color:#d20;background-color:#fff0f0">"--medium"</span>, <span style="color:#d20;background-color:#fff0f0">"../files/projectX-pull-requests-display.vmdk"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"storagectl"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--name"</span>, <span style="color:#d20;background-color:#fff0f0">"SATA Controller"</span>, <span style="color:#d20;background-color:#fff0f0">"--add"</span>, <span style="color:#d20;background-color:#fff0f0">"sata"</span>, <span style="color:#d20;background-color:#fff0f0">"--controller"</span>, <span style="color:#d20;background-color:#fff0f0">"IntelAHCI"</span>, <span style="color:#d20;background-color:#fff0f0">"--hostiocache"</span>, <span style="color:#d20;background-color:#fff0f0">"on"</span>]
p.customize [<span style="color:#d20;background-color:#fff0f0">"storageattach"</span>, :id, <span style="color:#d20;background-color:#fff0f0">"--storagectl"</span>, <span style="color:#d20;background-color:#fff0f0">"SATA Controller"</span>, <span style="color:#d20;background-color:#fff0f0">"--port"</span>, <span style="color:#00d;font-weight:bold">1</span>, <span style="color:#d20;background-color:#fff0f0">"--device"</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#d20;background-color:#fff0f0">"--type"</span>, <span style="color:#d20;background-color:#fff0f0">"hdd"</span>, <span style="color:#d20;background-color:#fff0f0">"--medium"</span>, <span style="color:#d20;background-color:#fff0f0">"../files/ipxe_projectX-pull-requests-display.vmdk"</span>]
end
end
end
</code></pre></div><p>Finally we have the environment stored in one Vagrantfile. The missing part is to how to run tests on it.</p>
<p>The testing script does the following:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell"><span style="color:#888">#/bin/bash</span>
<span style="color:#038">set</span> -e
<span style="color:#888"># FUNCTIONS</span>
<span style="color:#080;font-weight:bold">function</span> halt_vm () {
<span style="color:#369">vms</span>=<span style="color:#d20;background-color:#fff0f0">`</span>vboxmanage list vms | grep <span style="color:#d20;background-color:#fff0f0">"vagrant_</span><span style="color:#369">$1</span><span style="color:#d20;background-color:#fff0f0">_"</span> | awk {<span style="color:#d20;background-color:#fff0f0">'print $1'</span>} | sed s/<span style="color:#d20;background-color:#fff0f0">'"'</span>//g<span style="color:#d20;background-color:#fff0f0">`</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Stopping VM </span><span style="color:#369">$vms</span><span style="color:#d20;background-color:#fff0f0">"</span>
<span style="color:#369">stop_result</span>=<span style="color:#080;font-weight:bold">$(for</span> vm in <span style="color:#369">$vms</span> ; <span style="color:#080;font-weight:bold">do</span> vboxmanage controlvm <span style="color:#369">$vm</span> poweroff; <span style="color:#038">echo</span> <span style="color:#369">$?</span>; <span style="color:#080;font-weight:bold">done)</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Output of stopping VM </span><span style="color:#369">$1</span><span style="color:#d20;background-color:#fff0f0"> : </span><span style="color:#369">$stop_result</span><span style="color:#d20;background-color:#fff0f0">"</span>
}
<span style="color:#080;font-weight:bold">function</span> boot_vm () {
<span style="color:#369">vms</span>=<span style="color:#d20;background-color:#fff0f0">`</span>vboxmanage list vms | grep <span style="color:#d20;background-color:#fff0f0">"vagrant_</span><span style="color:#369">$1</span><span style="color:#d20;background-color:#fff0f0">_"</span> | awk {<span style="color:#d20;background-color:#fff0f0">'print $1'</span>} | sed s/<span style="color:#d20;background-color:#fff0f0">'"'</span>//g<span style="color:#d20;background-color:#fff0f0">`</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Booting VM </span><span style="color:#369">$vms</span><span style="color:#d20;background-color:#fff0f0">"</span>
<span style="color:#369">start_result</span>=<span style="color:#080;font-weight:bold">$(for</span> vm in <span style="color:#369">$vms</span> ; <span style="color:#080;font-weight:bold">do</span> vboxmanage startvm <span style="color:#369">$vm</span> --type headless; <span style="color:#038">echo</span> <span style="color:#369">$?</span>; <span style="color:#080;font-weight:bold">done)</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Output of booting VM </span><span style="color:#369">$1</span><span style="color:#d20;background-color:#fff0f0"> : </span><span style="color:#369">$start_result</span><span style="color:#d20;background-color:#fff0f0">"</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Sleeping additional 15 secs after peacefull boot"</span>
sleep <span style="color:#00d;font-weight:bold">15</span>
}
<span style="color:#080;font-weight:bold">function</span> add_keys () {
<span style="color:#080;font-weight:bold">for</span> i in <span style="color:#d20;background-color:#fff0f0">`</span>find /var/lib/jenkins/.ssh/id_rsa* | grep -v <span style="color:#d20;background-color:#fff0f0">'.pub'</span><span style="color:#d20;background-color:#fff0f0">`</span> ; <span style="color:#080;font-weight:bold">do</span> ssh-add <span style="color:#369">$i</span> ; <span style="color:#080;font-weight:bold">done</span>
}
<span style="color:#888">#vars</span>
<span style="color:#369">knifeclient_name</span>=lg-head-projectXtest.liquid.glx
<span style="color:#369">headnode_name</span>=projectX-pull-requests
<span style="color:#888"># TEST SCENARIO</span>
<span style="color:#038">cd</span> test/vagrant
<span style="color:#888"># teardown of previous sessions</span>
vagrant destroy projectX-pull-requests-kiosk -f
vagrant destroy projectX-pull-requests-display -f
vagrant destroy <span style="color:#369">$headnode_name</span> -f
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Not managing knife client because => chef_zero "</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"All ssh keys presented below"</span>
ssh-add -l
<span style="color:#888"># headnode</span>
vagrant up <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span>
<span style="color:#888"># displaynodes</span>
<span style="color:#369">result</span>=<span style="color:#080;font-weight:bold">$(</span>vboxmanage convertfromraw ../files/ipxe.usb ../files/ipxe_projectX-pull-requests-kiosk.vmdk --format=VMDK ; vagrant up projectX-pull-requests-kiosk; <span style="color:#038">echo</span> <span style="color:#369">$?</span><span style="color:#080;font-weight:bold">)</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests-kiosk : </span><span style="color:#369">$result</span><span style="color:#d20;background-color:#fff0f0">"</span>
<span style="color:#369">result</span>=<span style="color:#080;font-weight:bold">$(</span>vboxmanage convertfromraw ../files/ipxe.usb ../files/ipxe_projectX-pull-requests-display.vmdk --format=VMDK ; vagrant up projectX-pull-requests-display; <span style="color:#038">echo</span> <span style="color:#369">$?</span><span style="color:#080;font-weight:bold">)</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"projectX-pull-requests-display : </span><span style="color:#369">$result</span><span style="color:#d20;background-color:#fff0f0">"</span>
<span style="color:#888"># test phase</span>
<span style="color:#369">OPTIONS</span>=<span style="color:#d20;background-color:#fff0f0">`</span>vagrant ssh-config <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span> | grep -v <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span> | awk -v <span style="color:#369">ORS</span>=<span style="color:#d20;background-color:#fff0f0">' '</span> <span style="color:#d20;background-color:#fff0f0">'{print "-o " $1 "=" $2}'</span><span style="color:#d20;background-color:#fff0f0">`</span>
scp <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">OPTIONS</span><span style="color:#33b;background-color:#fff0f0">}</span> ../integration/projectX-pr/bats/*.bats vagrant@<span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span>:/tmp/bats_tests
ssh <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">OPTIONS</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#d20;background-color:#fff0f0">'/usr/local/bin/bats /tmp/bats_tests/pre_build_checks.bats'</span>
halt_vm projectX-pull-requests-kiosk
halt_vm projectX-pull-requests-display
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Building teh ISO (it may take a long time)"</span>
ssh <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">OPTIONS</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#d20;background-color:#fff0f0">'/usr/local/bin/bats /tmp/bats_tests/build_iso.bats'</span>
ssh <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">OPTIONS</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#d20;background-color:#fff0f0">'/usr/local/bin/bats /tmp/bats_tests/set_grub_to_make_partitions.bats'</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Booting nodes"</span>
boot_vm projectX-pull-requests-kiosk
boot_vm projectX-pull-requests-display
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Sleeping 30 secs for the DNS to boot and setting the grub to boot the ISO"</span>
sleep <span style="color:#00d;font-weight:bold">30</span>
ssh <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">OPTIONS</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#d20;background-color:#fff0f0">'/usr/local/bin/bats /tmp/bats_tests/set_grub_to_boot_the_iso.bats'</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Sleeping for 4 mins for the displaynodes to boot fresh ISO"</span>
sleep <span style="color:#00d;font-weight:bold">240</span>
<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"Running the tests inside the headnode:"</span>
ssh <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">OPTIONS</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#33b;background-color:#fff0f0">${</span><span style="color:#369">headnode_name</span><span style="color:#33b;background-color:#fff0f0">}</span> <span style="color:#d20;background-color:#fff0f0">'/usr/local/bin/bats /tmp/bats_tests/post_checks.bats'</span>
</code></pre></div><p>So finally we get the following pipeline:</p>
<ol>
<li>Clone Chef pull request from GitHub</li>
<li>Create Vagrantfile on the basis of Vagrantfile template</li>
<li>Create run_tests.sh script for running the tests</li>
<li>Destroy all previously created Vagrant boxes</li>
<li>Create one Chef Vagrant box</li>
<li>Create ISO Vagrant boxes with ipxe bootloader</li>
<li>Converge the Vagrant box with Chef</li>
<li>Copy BATS tests onto the headnode</li>
<li>Run initial BATS tests that build an ISO</li>
<li>Boot display nodes with the newly created ISO</li>
<li>Run final integration tests on the stack</li>
</ol>
<p>Elapsed time—between 40 and 50 minutes.</p>
Finding specific Git commit at a point in timehttps://www.endpointdev.com/blog/2014/11/finding-specific-git-commit-at-point-in/2014-11-10T00:00:00+00:00Greg Sabino Mullane
<div class="separator" style="clear: both; float: right; text-align: center;"><a href="/blog/2014/11/finding-specific-git-commit-at-point-in/image-0-big.jpeg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="/blog/2014/11/finding-specific-git-commit-at-point-in/image-0.jpeg"/></a><br/><small><a href="https://flic.kr/p/fFXKMu">Some car</a> by <a href="https://www.flickr.com/photos/wwarby/">William Warby</a></small></div>
<p>When using Git, being able to track down a particular version of a file is an important debugging skill. The common use case for this is when someone is reporting a bug in your project, but they do not know the exact version they are using. While normal software versioning resolves this, bug reports often come in from people using the HEAD of a project, and thus the software version number does not help. Finding the exact set of files the user has is key to being able to duplicate the bug, understand it, and then fix it.</p>
<p>How you get to the correct set of files (which means finding the proper Git commit) depends on what information you can tease out of the user. There are three classes of clues I have come across, each of which is solved a different way. You may be given clues about:</p>
<ol>
<li><strong>Date</strong>: The date they downloaded the files (e.g. last time they ran a <strong>git pull</strong>)</li>
<li><strong>File</strong>: A specific file’s size, checksum, or even contents.</li>
<li><strong>Error</strong>: An error message that helps guide to the right version (especially by giving a line number)</li>
</ol>
<h3 id="finding-a-git-commit-by-date">Finding a Git commit by date</h3>
<p>This is the easiest one to solve. If all you need is to see how the repository looked around a certain point in time, you can use git checkout with git-rev-parse to get it. I covered this in detail in <a href="/blog/2014/05/git-checkout-at-specific-date/">an earlier post</a>, but the best answer is below. For all of these examples, I am using the public Bucardo repository at <code>git clone git://bucardo.org/bucardo.git</code></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ DATE='Sep 3 2014'
$ git checkout `git rev-list -1 --before="$DATE" master`
Note: checking out '79ad22cfb7d1ea950f4ffa2860f63bd4d0f31692'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
HEAD is now at 79ad22c... Need to update validate_sync with new columns</small>
</code></pre></div><p>Or if you prefer xargs over backticks:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ <span style="color:#369">DATE</span>=<span style="color:#d20;background-color:#fff0f0">'Sep 3 2014'</span>
$ git rev-list -1 --before=<span style="color:#d20;background-color:#fff0f0">"</span><span style="color:#369">$DATE</span><span style="color:#d20;background-color:#fff0f0">"</span> master | xargs -Iz git checkout z
</code></pre></div><p>What about the case in which there were multiple important commits on the given day? If the user doesn’t know the exact time, you will have to make some educated guesses. You might add the <strong>-p</strong> flag to git log to examine what changes were made and how likely they are to interact with the bug in question. If it is still not clear, you may just want to have the user mail you a copy or a checksum of one of the key files, and use the method below.</p>
<p>Once you have found the commit you want, it’s a good idea to tag it right away. This applies to any of the three classes of clues in this article. I usually add a lightweight Git tag immediately after doing the checkout. Then you can easily come back to this commit simply by using the name of the tag. Give it something memorable and easy, such as the bug number being reported. For example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git checkout <span style="color:#d20;background-color:#fff0f0">`</span>git rev-list -1 --before=<span style="color:#d20;background-color:#fff0f0">"</span><span style="color:#369">$DATE</span><span style="color:#d20;background-color:#fff0f0">"</span> master<span style="color:#d20;background-color:#fff0f0">`</span>
<span style="color:#888">## Give a lightweight tag to the current commit</span>
$ git tag bug_23142
<span style="color:#888">## We need to get back to our main work now</span>
$ git checkout master
<span style="color:#888">## Later on, we want to revisit that bug</span>
$ git checkout bug_23142
<span style="color:#888">## Of course, you may also want to simply create a branch</span>
</code></pre></div><h3 id="finding-a-git-commit-by-checksum-size-or-exact-file">Finding a Git commit by checksum, size, or exact file</h3>
<p>Sometimes you can find the commit you need by looking for a specific version of an important file. One of the “main” files in the repository that changes often is your best bet for this. You can ask the user for the size, or just a checksum of the file, and then see which repository commits have a matching entry.</p>
<h4 id="finding-a-git-commit-when-given-a-checksum">Finding a Git commit when given a checksum</h4>
<p>As an example, a user in the Bucardo project has encountered a problem when running HEAD, but all they know is that they checked it out of sometime in the last four months. They also run “md5sum Bucardo.pm” and report that the MD5 of the file Bucardo.pm is
767571a828199b6720f6be7ac543036e. Here’s the easiest way to find what version of the repository they are using:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ SUM=767571a828199b6720f6be7ac543036e
$ git log --format=%H \
| xargs -Iz sh -c \
'echo -n "z "; git show z:Bucardo.pm | md5sum' \
| grep -m1 $SUM \
| cut -d " " -f 1 \
| xargs -Iz git log z -1
xargs: sh: terminated by signal 13
commit b462c256e62e7438878d5dc62155f2504353be7f
Author: Greg Sabino Mullane <greg@endpoint.com>
Date: Fri Feb 24 08:34:50 2012 -0500
Fix typo regarding piddir
</code></pre></div><p>I’m using variables in these examples both to make copy and paste easier, and because it’s always a good idea to save away constant but hard-to-remember bits of information. The first part of the pipeline grabs a list of all commit IDs: <strong>git log –format=%H</strong>.</p>
<p>We then use xargs to feed list of commit ids one by one to a shell. The shell grabs a copy of the Bucardo.pm file as it existed at the time of that commit, and generates an MD5 checksum of it. We echo the commit on the line as well as we will need it later on. So we now generate the commit hash and the md5 of the Bucardo.pm file.</p>
<p>Next, we pipe this list to grep so we only match the MD5 we are looking for. We use -m1 to stop processing once the first match is found (this is important, as the extraction and checksumming of files is fairly expensive, so we want to short-circuit it as soon as possible). Once we have a match, we use the <strong>cut</strong> utility to extract just the commit ID, and pipe that back into <strong>git log</strong>. Voila! Now we know the very last time the file existed with that MD5, and can checkout the given commit. (The “terminated by signal 13” is normal and expected)</p>
<p>You may wonder if a sha1sum would be better, as Git uses those internally. Sadly, the process remains the same, as the algorithm Git uses to generate its internal SHA1 checksums is <code>sha1("blob " . length(file) . "\0" . contents(file))</code>, and you can’t expect a random user to compute that and send it to you! :)</p>
<h4 id="finding-a-git-commit-when-given-a-file-size">Finding a Git commit when given a file size</h4>
<p>Another piece of information the user can give you very easily is the size of a file. For example, they may tell you that their copy of Bucardo.pm weighs in at 167092 bytes. As this file changes often, it can be a unique-enough marker to help you determine when they checkout out the repository. Finding the matching size is a matter of walking backwards through each commit and checking the file size of every Bucardo.pm as it existed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ SIZE=167092
$ git rev-list --all \
| while read commit
do if git ls-tree -l -r $commit \
| grep -q -w $SIZE
then echo $commit
break
fi
done
d91807d59a6326e48077311e96e4d5730f24304c
</code></pre></div><p>The git ls-tree command generates a list of all blobs (files) for a given commit. The -l option tells it to also print the file size, and the -r option asks it to recurse. So we use git rev-list to generate a list of all the commits (by default, these are output from newest to oldest). Then we pass each commit to the ls-tree command, and use grep to see if that number appears anywhere in the output. If it does, grep returns truth, making the if statement fire the echo, which shows is the commit. The break ensures we stop after the first match. We now have the (probable) commit that the user checked the file out of. As we are not matching by filename, it’s probably a good idea to double-check by running git ls-tree -l -r on the given commit.</p>
<h4 id="finding-a-git-commit-when-given-a-copy-of-the-file-itself">Finding a Git commit when given a copy of the file itself</h4>
<p>This is very similar to the size method above, except that we are given the file itself, not the size, so we need to generate some metadata about it. You could run a checksum or a filesize and use one of the recipes above, or you could do it the Git way and find the SHA1 checksum that Git uses for this file (aka a blob) by using
<a href="https://www.kernel.org/pub/software/scm/git/docs/git-hash-object.html">git hash-object</a>. Once you find that, you can use git ls-tree as before, as the blob hash is listed next to the filename. Thus:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ HASH=`git hash-object ./bucardo.clue`
$ echo $HASH
639b247aab027b79bda788182c8b6785ed319662
$ git rev-list --all \
| while read commit
do if git ls-tree -r $commit \
| grep -F -q $HASH
then echo $commit
break
fi
done
cd1d776307204cb77a731aa1b15c3c43a275c70e
</code></pre></div><h3 id="finding-a-git-commit-by-error-message">Finding a Git commit by error message</h3>
<p>Sometimes the only clue you are given is an error message, or some other snippet that you can trace back to one or more commits. For example, someone once mailed the list to ask about this error that they received:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">DBI connect('dbname=bucardo;host=localhost;port=5432',
'bucardo',...) failed: fe_sendauth: no password supplied at
/usr/local/bin/bucardo line 8627.
</code></pre></div><p>A quick glance at line 8627 of the file “bucardo” in HEAD showed only a closing brace, so it must be an earlier version of the file. What was needed was to walk backwards in time and check that line for every commit until we find one that could have triggered the error. Here is one way to do that:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git log --format=%h \
| xargs -n 1 -I sh -c \
"echo -n {}; git show {}:bucardo | head -8627 | tail -1" \
| less
## About 35 lines down:
379c9006 $dbh = DBI->connect($BDSN, 'bucardo'...
</code></pre></div><p>Therefore, we can do a “git checkout 379c9006” and see if we can solve the user’s problem.</p>
<p>These are some of the techniques I use to hunt down specific commits in a Git repository. Are there other clues you have run up against? Better recipes for hunting down commits? Let me know in the comments below.</p>
Version differences via GitHub from the command linehttps://www.endpointdev.com/blog/2014/07/version-differences-via-github-from/2014-07-09T00:00:00+00:00Greg Sabino Mullane
<div class="separator" style="clear: both; float:right; text-align: center;"><a href="/blog/2014/07/version-differences-via-github-from/image-0.jpeg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="/blog/2014/07/version-differences-via-github-from/image-0.jpeg"/></a><br/><small><a href="https://flic.kr/p/7pSEW1">Photo</a> by <a href="https://www.flickr.com/photos/bensonkua/">Benson Kua</a></small></div>
<p>I work with a lot of open source projects, and I use the command-line for almost everything. It often happens that I need to examine a file from a project, and thanks to bash, <a href="https://github.com/">Github</a>, and curl, I can do so easily, without even needing to have the repo handy. One of the things I do sometimes is compare a file across versions to see what has changed. For example, I needed to see what changes were made between versions 1.22 and 1.23 to the file includes/UserMailer.php which is part of the <a href="https://www.mediawiki.org/wiki/MediaWiki">MediaWiki</a> project. For this trick to work, the project must be on Github, and must label their versions in a consistent manner, either via git branches or git tags.</p>
<p>MediaWiki exists on Github as <a href="https://github.com/wikimedia/mediawiki-core">wikimedia/mediawiki-core</a>. The MediaWiki project tags all of their releases in the format X.Y.Z, so in this example we can use the git tags <strong>1.22.0</strong> and <strong>1.23.0</strong>. Github is very nice because you can view a specific file at a certain commit (aka a tag), and even grab it over the web as a plain text file. The format is:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">https://raw.githubusercontent.com/PROJECTNAME/BRANCH-OR-TAG/FILE
</code></pre></div><p>Note that you can use a tag <em><strong>OR</strong></em> a branch! So to compare these two files, we can use one of these pairs:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">https://raw.githubusercontent.com/wikimedia/mediawiki-core/REL1_21/includes/UserMailer.php
https://raw.githubusercontent.com/wikimedia/mediawiki-core/REL1_22/includes/UserMailer.php
https://raw.githubusercontent.com/wikimedia/mediawiki-core/1.21.0/includes/UserMailer.php
https://raw.githubusercontent.com/wikimedia/mediawiki-core/1.22.0/includes/UserMailer.php
</code></pre></div><p>All that is left is to treat git as a web service and compare the two files at the command line ourselves. The program <strong>curl</strong> is a great tool for downloading the files, as it dumps to stdout by default. We will add a <strong>-s</strong> flag (for “silent”) to prevent it from showing the progress meter as it usually does. The last bit of the puzzle is to use <(), bash’s process substitution feature, to trick diff into comparing the curl outputs as if they were files. So our final command is:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">diff <(curl -s https://raw.githubusercontent.com/wikimedia/mediawiki-core/1.21.0/includes/UserMailer.php) \
<(curl -s https://raw.githubusercontent.com/wikimedia/mediawiki-core/1.22.0/includes/UserMailer.php) \
| more
</code></pre></div><p>Voila! A quick and simple glance at what changed between those two tags. This should work for any project on Github. You can also replace the branch or tag with the word “master” to see the current version. For example, the PostgreSQL project lives on github as postgres/postgres. They use the format RELX_Y_Z in their tags. To see what has changed since release 9.3.4 in the psql help file (as a context diff), run:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sql" data-lang="sql">diff<span style="color:#bbb"> </span>-<span style="color:#080;font-weight:bold">c</span><span style="color:#bbb"> </span><(curl<span style="color:#bbb"> </span>-s<span style="color:#bbb"> </span>https://raw.githubusercontent.com/postgres/postgres/REL9_3_4/src/bin/psql/help.<span style="color:#080;font-weight:bold">c</span>)<span style="color:#bbb"> </span><span style="color:#a61717;background-color:#e3d2d2">\</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><(curl<span style="color:#bbb"> </span>-s<span style="color:#bbb"> </span>https://raw.githubusercontent.com/postgres/postgres/master/src/bin/psql/help.<span style="color:#080;font-weight:bold">c</span>)<span style="color:#bbb">
</span></code></pre></div><p>You are not limited to diff, of course. For a final example, let’s see how many times Tom Lane is mentioned in the version 9 release notes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">for i in {0,1,2,3,4}
do grep -Fc 'Tom Lane' \
<(curl -s https://raw.githubusercontent.com/postgres/postgres/master/doc/src/sgml/release-9.$i.sgml)
done
272
206
174
115
16
</code></pre></div><p>The last number is so low relative to the rest because 9.4 is still under development. Rest assured Tom’s contributions have not slowed down! :) Thanks to Github for providing such a useful service for so many open source projects, and for providing the raw text to allow useful hacks like this.</p>
git checkout at a specific datehttps://www.endpointdev.com/blog/2014/05/git-checkout-at-specific-date/2014-05-19T00:00:00+00:00Greg Sabino Mullane
<div class="separator" style="clear: both; float:right; text-align: center;"><a href="/blog/2014/05/git-checkout-at-specific-date/image-0.jpeg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="/blog/2014/05/git-checkout-at-specific-date/image-0.jpeg"/></a>
<br/><small><a href="https://flic.kr/p/eqKF87">Porcupine</a> via <a href="https://www.flickr.com/photos/pinti1/">Flickr user Holly Occhipinti</a></small></div>
<p>There are times when you need to view a git repository as it was at a certain point in time. For example, someone sends your project an error report and says they were using the git head version from around January 17, 2014. The short (and wrong!) way to do it is to pass the date to the checkout command like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git checkout <span style="color:#d20;background-color:#fff0f0">'HEAD@{Jan 17 2014}'</span> <span style="color:#888">## do not do this</span>
</code></pre></div><p>While I used to rely on this, I no longer do so, as I consider it somewhat of a footgun. To understand why, you first have to know that the ability to checkout using the format above only works for a short window of time, as defined by the git parameter <strong>gc.reflogExpire</strong>. This defaults to a measly 90 days. You can view yours with <strong>git config gc.reflogExpire</strong>. The problem is that when you go over the 90 day limit, git outputs a warning, but them spews a mountain of output as it performs the checkout anyway! It uses the latest entry it has in the reflog (e.g. 90 days ago). This commit has no relation at all with the date you requested, so unless you catch the warning, you have checked out a repository that is useless to your efforts.</p>
<p>For example, the Bucardo project can be cloned via:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git clone git://bucardo.org/bucardo.git/
</code></pre></div><p>Now let’s say we want to examine the project as it looked on January 17, 2014. As I am writing this, the date is May 19, 2014, so that date occurred about four months ago: well over 90 days. Watch what happens:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git checkout <span style="color:#d20;background-color:#fff0f0">'HEAD@{Jan 17 2014}'</span> <span style="color:#888">## do not do this</span>
warning: Log <span style="color:#080;font-weight:bold">for</span> <span style="color:#d20;background-color:#fff0f0">'HEAD'</span> only goes back to Sat, <span style="color:#00d;font-weight:bold">22</span> Feb <span style="color:#00d;font-weight:bold">2014</span> 11:47:33 -0500.
Note: checking out <span style="color:#d20;background-color:#fff0f0">'HEAD@{Jan 17 2014}'</span>.
You are in ‘detached HEAD’ state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
<span style="color:#080;font-weight:bold">do</span> so (now or later) by using -b with the checkout <span style="color:#038">command</span> again. Example:
git checkout -b new_branch_name
HEAD is now at d7f89dd... Bucardo now accepts pg_service <span style="color:#080;font-weight:bold">for</span> databases
</code></pre></div><p>So, we get the warning that HEAD only goes back to Feb 22, but then git goes ahead and checks us out anyway! If you were not paying attention—perhaps because you only glanced over that perfectly ordinary looking last line—you might not realize that the checkout you received is not what you requested.</p>
<p>Since this behavior cannot, to my knowledge, be turned off, I avoid this method and use other ways to checkout the repo as it existed on a certain date. The simplest is to find the closest commit by viewing the output of <strong>git log</strong>. In smaller projects, you can simply do this in a text editor and search for the date you want, then find a good commit sha-1 hash to checkout (i.e. <strong>git log > log.txt; emacs log.txt</strong>). Another somewhat canonical way is to use git-rev-list:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git checkout <span style="color:#d20;background-color:#fff0f0">`</span>git rev-list -1 --before=<span style="color:#d20;background-color:#fff0f0">"Jan 17 2014"</span> master<span style="color:#d20;background-color:#fff0f0">`</span>
</code></pre></div><p>This command works fine, although it is a little clunky and hard to remember. It’s requesting a list of all commits on the master branch, which happened before the given date, ordered by date, and stop once a single row has been output. Since I deal with SQL all day, I think of this as:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sql" data-lang="sql"><span style="color:#080;font-weight:bold">SELECT</span><span style="color:#bbb"> </span>repository<span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">WHERE</span><span style="color:#bbb"> </span>commit_id<span style="color:#bbb"> </span>=<span style="color:#bbb">
</span><span style="color:#bbb"> </span>(<span style="color:#080;font-weight:bold">SELECT</span><span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">commit</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">FROM</span><span style="color:#bbb"> </span>rev-list<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">WHERE</span><span style="color:#bbb"> </span>commit_date<span style="color:#bbb"> </span><=<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Jan 10, 2014'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">AND</span><span style="color:#bbb"> </span>branch<span style="color:#bbb"> </span>=<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'master'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">ORDER</span><span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">BY</span><span style="color:#bbb"> </span>commit_date<span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">DESC</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">LIMIT</span><span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">1</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>);<span style="color:#bbb">
</span></code></pre></div><p>This is one of the cases where the date IS inclusive. With git, you should always test when using date ranges if the given date is inclusive or exclusive, as reading the fine manual does not always reveal this information. Here is one way to prove the date is inclusive for the rev-list command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git rev-list -1 --before="Jan 17 2014" master --format=medium
commit d4b565bf46b6f478b969a378578b0cff3b24e82d
Author: Greg Sabino Mullane <greg@endpoint.com>
Date: Fri Jan 17 10:49:09 2014 -0500
Make our statement_chunk_size default match up.
</code></pre></div><p>As a final nail in the coffin for doing a checkout via the reflog date, the reflog actually is local to you and will pull the date of the repo as it existed for you at that point in time. This may or may not line up with the commits, depending on how often you are syncing with other people via git pull or other methods! So play it safe and request a specific commit by sha-1 hash, or use the rev-list trick.</p>
MediaWiki extensions and wfLoadExtensionMessageshttps://www.endpointdev.com/blog/2014/05/mediawiki-extensions-and/2014-05-09T00:00:00+00:00Greg Sabino Mullane
<div class="separator" style="clear: both; float: right; text-align: center;"><a href="/blog/2014/05/mediawiki-extensions-and/image-0.jpeg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="/blog/2014/05/mediawiki-extensions-and/image-0.jpeg"/></a><br/><small><a href="https://flic.kr/p/dykcFR">Image</a> by <a href="https://www.flickr.com/photos/yukonlife/">Susan Drury</a></small>
</div>
<p>Upgrading MediaWiki can be a challenging task, especially if you use a lot of extensions.
While the core upgrade process usually goes smoothly, it’s rare you can upgrade a major
version or two without having to muddle with your collection of extensions. Extensions are bits of code that extend what MediaWiki can do. Only a few are packaged with and maintained alongside MediaWiki itself—the great majority are written by third-party developers. When the MediaWiki API changes, it is up to those developers to update their extension so it works with the new version of MediaWiki. This does not always happen. Take for example one of the more common errors seen on a MediaWiki upgrade since 1.21 was released:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">[Tue May 06 11:21:52 2014] [error] [client 12.34.56.78] PHP Fatal error: Call to undefined function wfLoadExtensionMessages() in /home/beckett/mediawiki/extensions/PdfExport/PdfExport.php on line 83, referer: http://test.ziggy.com/wiki/Main_Page
</code></pre></div><p>This is because the <strong>wfLoadExtensionMessages</strong> function, which many extensions use, has
been deprecated since MediaWiki version 1.16 and was finally removed in 1.21, resulting in the
error seen above. Luckily, this function has been a no-op since 1.16, so it is safe
to comment it out and/or make a dummy function in your LocalSettings.php file (see below).</p>
<p>Sadly, the release notes for 1.21 make no mention of
<a href="https://lists.gt.net/wiki/wikitech/214619">this fairly major change</a>. Let’s
walk through as if we didn’t know anything about it and see how we could solve the
given error with the help of git. For this example, we’ll use the
<a href="https://www.mediawiki.org/wiki/Extension:Pdf_Export">Pdf Export extension</a>,
which allows you to export your wiki pages into PDF form. A pretty handy extension, and
one which completely fails to work in MediaWiki version 1.21 or better.</p>
<p>First, let’s verify that wfLoadExtensionMessages does not exist at all in version 1.21 of MediaWiki. For
these examples, I’ve checked out the MediaWiki code via git, and am relying on
the fact that lightweight git tags were made for all the versions we are interested in.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git clone https://github.com/SemanticMediaWiki/SemanticMediaWiki.git mediawiki
$ cd mediawiki
$ git grep wfLoadExtensionMessages 1.21.0
1.21.0:HISTORY:* (bug 12880) wfLoadExtensionMessages does not use $fallback from MessagesXx.php
</code></pre></div><p>A nice feature of <code>git grep</code> is the ability to simply use a tag after the search string. In this
case, we see that the only mention of wfLoadExtensionMessages in the entire codebase is an
old mention of it in the history file. Let’s see what version that bug is from:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git grep -n wfLoadExtensionMessages 1.21.0
1.21.0:HISTORY:5280:* (bug 12880) wfLoadExtensionMessages does not use $fallback from MessagesXx.php
$ git show 1.21.0:HISTORY | head -5280 | tac | grep '===' -m1
=== Bug fixes in 1.12 ===
</code></pre></div><p>That message is from way back in version 1.12, and doesn’t concern us. Let’s take a look at
what tags exist in the 1.20 branch so we can scan the latest one:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git tag | grep '^1\.20'
1.20.0
1.20.0rc1
1.20.0rc2
1.20.1
1.20.2
1.20.3
1.20.4
1.20.5
1.20.6
1.20.7
1.20.8
</code></pre></div><p>Now we can peek inside version 1.20.8 and see what that function did before it was removed.
By using the -A and -B (after and before) arguments to grep, we can see the entire function in
context:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git grep wfLoadExtensionMessages 1.20.0
1.20.0:HISTORY:* (bug 12880) wfLoadExtensionMessages does not use $fallback from MessagesXx.php
1.20.0:includes/GlobalFunctions.php:function wfLoadExtensionMessages() {
$ git show 1.20.8:includes/GlobalFunctions.php | grep -B6 -A2 LoadExtensionMessages
/**
* Load an extension messages file
*
* @deprecated since 1.16, warnings in 1.18, remove in 1.20
* @codeCoverageIgnore
*/
function wfLoadExtensionMessages() {
wfDeprecated( __FUNCTION__, '1.16' );
}
</code></pre></div><p>Thus wfLoadExtensionMessages was basically a no-op in MediaWiki version 1.20, with the caveat that it will write
a deprecation warning to your error log (or, in modern versions, the debug log unless $wgDevelopmentWarnings is set).
Next we want to find the last time this function did something useful—which should be version 1.15 according to
the comment above. Thus:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git show 1.15.0:includes/GlobalFunctions.php | grep -A4 LoadExtensionMessages
function wfLoadExtensionMessages( $extensionName, $langcode = false ) {
global $wgExtensionMessagesFiles, $wgMessageCache, $wgLang, $wgContLang;
#For recording whether extension message files have been loaded in a given language.
static $loaded = array();
</code></pre></div><p>So, it’s a pretty safe bet that unless you are upgrading from 1.15.0 or earlier, it should
be completely safe to remove it. When was 1.16.0 released? There are no dates in the HISTORY
file (shame), but the date it was tagged should be a good guess:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git show 1.16.0 | grep -m1 Date
Date: Wed Jul 28 07:11:03 2010 +0000
</code></pre></div><p>So what should you do with extensions that are still using this deprecated function? There are
two quick solutions: comment it out inside the extension, or add a dummy function to your version
of MediaWiki.</p>
<p>Changing the extension itself is certainly quick and easy. To get the PdfExport extension to work,
we only have to comments out two calls to wfLoadExtensionMessages inside of the file
PdfExport.php, and one inside of PdfExport_body.php. The diff:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git difftool -y -x "diff -u1"
--- /tmp/7YqvXv_PdfExport.php 2014-05-08 12:45:03 -0400
+++ PdfExport.php 2014-05-08 12:34:39 -0400
@@ -82,3 +82,3 @@
if ($img_page > 0 || $img_page === false) {
- wfLoadExtensionMessages('PdfPrint');
+ //wfLoadExtensionMessages('PdfPrint');
$nav_urls['pdfprint'] = array(
@@ -92,3 +92,3 @@
function wfSpecialPdfToolbox (&$monobook) {
- wfLoadExtensionMessages('PdfPrint');
+ //wfLoadExtensionMessages('PdfPrint');
if (isset($monobook->data['nav_urls']['pdfprint']))
--- /tmp/7gO8Hz_PdfExport_body.php 2014-05-08 12:45:03 -0400
+++ PdfExport_body.php 2014-05-08 12:34:44 -0400
@@ -44,3 +44,3 @@
// For backwards compatibility
- wfLoadExtensionMessages('PdfPrint');
+ //wfLoadExtensionMessages('PdfPrint');
</code></pre></div><p>A better way is to add a dummy function to LocalSettings.php. This ensures that any extension
we add in the future will continue to work unmodified. Just throw this at the bottom
on your LocalSettings.php:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-php" data-lang="php"><span style="color:#080;font-weight:bold">function</span> <span style="color:#06b;font-weight:bold">wfLoadExtensionMessages</span>() { }
</code></pre></div><p>Probably the best overall solution is to not only add that to your LocalSettings.php,
but to try to get the extension changed as well. You can notify the author, or try to
fix it yourself and release a new version if the extension has been abandoned. You might
also look to see if the extension has been superseded by a different extension, as sometime
happens.</p>
<p>While there may be other compatibility issues when upgrading MediaWiki, for some extensions
(such as PdfExport), this is the only change needed to make it work again on newer versions of MediaWiki!</p>
Git Workflows That Workhttps://www.endpointdev.com/blog/2014/05/git-workflows-that-work/2014-05-02T00:00:00+00:00Spencer Christensen
<div class="separator" style="clear: both; text-align: center;"><a href="/blog/2014/05/git-workflows-that-work/image-0-big.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="/blog/2014/05/git-workflows-that-work/image-0.png"/></a></div>
<p>There was a <a href="http://nvie.com/posts/a-successful-git-branching-model/">significant blog post some years ago</a>. It introduced a “successful” workflow for using Git. This workflow was named Gitflow. One of the reasons this blog post was significant is that it was the first structured workflow that many developers had been exposed to for using Git. Before Gitflow was introduced, most developers didn’t work with Git like that. And if you remember when it was introduced back in 2010, it created quite a buzz. People praised it as “the” way to work with Git. Some adopted it so quickly and full heartedly that they dismissed any other way to use Git as immature or childish. It became, in a way, a movement.</p>
<p>I start with this little bit of history to talk about the void that was filled by Gitflow. There was clearly something that drew people to it that wasn’t there before. It questioned the way they were working with Git and offered something different that worked “successfully” for someone else. I supposed many developers didn’t have much confidence or strong feelings about their use of Git before they heard of Gitflow. And so they followed someone who clearly did have confidence and strong feelings about a particular workflow. Some of you may be questioning your current Git workflow now and can relate to what I’m describing. However, I’m not going to prescribe a particular workflow for you as “the” way to do it.</p>
<p>Instead, let’s talk about the purpose of a workflow. Let’s reword that so we’re clear- the purpose of a software development workflow using Git. What is the purpose? Let’s back up and ask what is the purpose of software? <a href="https://www.codesimplicity.com/post/the-purpose-of-software/">The purpose of software is to help people.</a> Period. Yes it can help servers, and networks, and robots, and telephones, etc. But help them do what? Help people. They are tools to help us (people) do things better, faster, simpler, etc. I submit to you that the purpose of a software development workflow using Git should be the same. It should help people release software. Specifically, it should help match the software development process with business expectations for the people responsible for the software. That list of people responsible for the software should include more than just the developers. It also includes operations engineers, project managers, and certainly business owners.</p>
<p>Does your Git workflow help your business owners? Does it help your project managers or the Operations team? These are questions you should be thinking about. And by doing so, you should realize that there is no “one size fits all” workflow that will do all that for every case. There are many different workflows based on different needs and uses. Some are for large complex projects and some are extremely simple. What you need to ask is- what will best help my team/project/organization to develop, release, and maintain software effectively? Let’s look at a few workflow examples and see.</p>
<h3 id="github-flow">GitHub Flow</h3>
<div class="separator" style="clear: both; text-align: center;"><a href="/blog/2014/05/git-workflows-that-work/image-1-big.jpeg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="/blog/2014/05/git-workflows-that-work/image-1.jpeg"/></a></div>
<p>GitHub’s own workflow, <a href="http://scottchacon.com/2011/08/31/github-flow.html">their internal workflow</a>, is quite different from what everyone else does who uses GitHub. It is based on a set of simple business choices:</p>
<ul>
<li>Anything in the master branch is deployable</li>
<li>To work on something new, create a descriptively named branch off of master (ie: new-oauth2-scopes)</li>
<li>Commit to that branch locally and regularly push your work to the same named branch on the server</li>
<li>When you need feedback or help, or you think the branch is ready for merging, open a pull request</li>
<li>After someone else has reviewed and signed off on the feature, you can merge it into master</li>
<li>Once it is merged and pushed to ‘master’ on the origin, you can and should deploy immediately</li>
</ul>
<p>They release many times per day to production using this model. They branch off master for every change they make, hot fixes and features are treated the same. Then they merge back into master and release. They even have automated their releases using an irc bot.</p>
<h3 id="skullcandys-workflow">Skullcandy’s workflow</h3>
<div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-AX-_EhIL6v0/U2Po8uyGNcI/AAAAAAAAAUo/5EYh2pZhwd8/s1600/git_workflow_skullcandy+(1).jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="/blog/2014/05/git-workflows-that-work/image-2.jpeg"/></a></div>
<p>When I worked for Skullcandy we used a workflow loosely based on the GitHub Flow model, but altered a bit. We used a Scrum Agile methodology with well defined sprints of work and deliverables at the end of each sprint. The workflow followed these business choices:</p>
<ul>
<li>A userstory or defect in our tracking system represented a single deliverable, and a Git branch was created for each userstory or defect. We used a naming convention for branches (skdy/schristensen/US1234-cool-new-feature, for example). Yes, you can use ‘/’ characters in branch names.</li>
<li>Everything branches off master. Features and hot fixes are treated the same.</li>
<li>After code review, then the branch was merged into a QA branch and deployed to the QA environment where business owners tested and approved the changes.</li>
<li>The QA branch is just another branch off master and can be blown away and recreated when needed at any time.</li>
<li>We released once a week, and only those changes that have been approved by the business owners in QA got merged into master and released.</li>
<li>Since branch names and items in our issue tracking system were tied together we could easily verify the status of a change, the who, when, and what, and why of it, and even automate things- like auto merging of approved branches and deployment, auto updating tickets in the tracking system, and notifying developers of any merge issues or when their branch got released.</li>
</ul>
<h3 id="master-only-workflow">Master only workflow</h3>
<div class="separator" style="clear: both; text-align: center;"><a href="/blog/2014/05/git-workflows-that-work/image-3-big.jpeg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="/blog/2014/05/git-workflows-that-work/image-3.jpeg"/></a></div>
<p>Not every team or project is going to work like this. And it may be too complicated for some. It may be appropriate to just work on master without branching and merging. I do this now with some of the clients I work with.</p>
<ul>
<li>Each feature or hot fix is worked on in dev environment that is similar to production, that allows business owner direct access for testing and approval. Changes are committed locally.</li>
<li>Once approved by the business owner, commit and push changes to master on origin, and then deploy to production immediately.</li>
</ul>
<p>You may not be working for a business, and so the term “business owner” may not fit your
situation. But there should always be <strong>someone</strong> who approves the changes as acceptable for release. That person should be the same one who requested the change in the first place.</p>
<h3 id="gitflow">Gitflow</h3>
<div class="separator" style="clear: both; text-align: center;"><a href="/blog/2014/05/git-workflows-that-work/image-4-big.jpeg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="/blog/2014/05/git-workflows-that-work/image-4.jpeg"/></a></div>
<p>On the other end of the spectrum from a master only workflow, is Gitflow. Here there are at least three main branches: develop (or development), release, and master. There are other branches as well for features and hot fixes. Many of these are long running. For example, you merge develop into the release branch but then you continue working on develop and add more commits. The workflow looks like this:</p>
<ul>
<li>All work is done in a branch. Features are branched off develop. Hot fixes are treated different and are branched off master.</li>
<li>Features are merged back into develop after approval.</li>
<li>Develop is merged into a release branch.</li>
<li>Hot fixes are merged back into master, but also must be merged into develop and the release branch.</li>
<li>The release branch is merged into master.</li>
<li>Master is deployed to production.</li>
</ul>
<h3 id="backcountry-workflow">Backcountry workflow</h3>
<p>When I worked for Backcountry.com we used a similar workflow, however we used different names for the branches. All development happened on master, feature branches were branched off and then merged back into master. Then we branched master to create a new release branch. And then we merged the release branch into a branch called “production”. And since master is just a branch and doesn’t have to be special, you could use a branch named whatever you want for your production code.</p>
<h3 id="guidelines">Guidelines</h3>
<p>There are many other examples we could go over and discuss, but these should be enough to get you thinking about different possibilities. There are a few guidelines that you should consider for your workflow:</p>
<ul>
<li>Branches should be used to represent a single deliverable request from the business- like a single user story or bug fix. Something that can be approved by the business that contains everything needed for that single request to be released- and nothing more!</li>
<li>The longer a feature branch lives without getting merged in for a release, the greater risk for merge conflicts and challenges for deployment. Short lived branches merge and deploy cleaner.</li>
<li>Business owner involvement in your workflow is essential. Don’t merge, don’t deploy, don’t work without their input. Otherwise pain and tears will ensue (or worse).</li>
<li>Avoid reverts. Test, test, test your branch before a merge. When merging use <strong>git merge –no-ff</strong>, which will ease merge reverts if really needed.</li>
<li>Your workflow should fit how you release. Do you release continually, multiple times a day? Do you have 2 week sprints with completed work to release on a regular schedule? Do you have a business Change Control Board where all released items must get reviewed and approved first? Does someone else run your releases, like the Operations team or a Release manager? Your branching and merging strategy needs to make releasing easier.</li>
<li>Complicated workflows drive people crazy. Make it simple. Review your workflow and ask how you can simplify it. In actively making things more simple, you will also make them easier to understand and work with as well as easier for others to adopt and maintain.</li>
</ul>
<p>These should help you adjust your software development workflow using Git to fulfill its purpose of helping people. Helping <em><strong>you</strong></em>.</p>
<h3 id="further-reading">Further Reading</h3>
<p>There is a lot more you can read about on this topic, and here are several good places to start:</p>
<ul>
<li><a href="https://www.codesimplicity.com/post/the-purpose-of-software/">https://www.codesimplicity.com/post/the-purpose-of-software/</a></li>
<li><a href="http://scottchacon.com/2011/08/31/github-flow.html">http://scottchacon.com/2011/08/31/github-flow.html</a></li>
<li><a href="http://nvie.com/posts/a-successful-git-branching-model/">http://nvie.com/posts/a-successful-git-branching-model/</a></li>
<li><a href="http://www.kdgregory.com/index.php?page=scm.git">http://www.kdgregory.com/index.php?page=scm.git</a></li>
<li><a href="https://sandofsky.com/blog/git-workflow.html">https://sandofsky.com/blog/git-workflow.html</a></li>
<li><a href="https://stackoverflow.com/questions/2428722/git-branch-strategy-for-small-dev-team">https://stackoverflow.com/questions/2428722/git-branch-strategy-for-small-dev-team</a></li>
<li><a href="https://git-scm.com/book/en/Git-Branching-Branching-Workflows">https://git-scm.com/book/en/Git-Branching-Branching-Workflows</a></li>
<li><a href="https://www.atlassian.com/git/workflows">https://www.atlassian.com/git/workflows</a></li>
</ul>
A Git and symlink mistakehttps://www.endpointdev.com/blog/2014/02/a-git-and-symlink-mistake/2014-02-17T00:00:00+00:00Steph Skardal
<p>A couple of times, I’ve accidentally created an infinite symlink loop and lost files from that directory from a single Git commit. Here’s how it happened:</p>
<ol>
<li>First, on the production app, images tied to products in a database were uploaded to the server. Let’s say I was working on a Rails app, so these images were uploaded to the RAILS_ROOT/public/system/ directory. I added “public/system/” to my .gitignore file, and all appeared to be good.</li>
<li>Next on a <a href="https://www.devcamps.org/">camps instance</a>, I created a symlink from CAMP_ROOT/public/system pointing to the production app public/system directory. This is common practice in End Point’s camps setup because we often don’t need the redundancy of uploaded files on our dev camp instance, and we don’t want the extra disk space used up for these types of files. The <em>make camp</em> script is designed to allow a user to toggle symlink functionality on and off for various directories during the make camp process.</li>
<li>Next, I accidentally committed and push the public/system symlink from my development instance.</li>
<li>Finally, I pulled the commit onto my production instance. The pull resulted in public/system symlinking to itself, and all of the files vanished (poof). Since they were Git-ignored in the first place, I couldn’t recover them from Git.</li>
</ol>
<p>This is a pretty simple mistake to mitigate or avoid completely:</p>
<h3 id="backups">Backups</h3>
<p>Have backups! Have backups! Did I say have backups?! Our awesome hosting team here at End Point was able to recover the lost files from the nightly backups when this has happened.</p>
<h3 id="gitignore-update">Gitignore Update</h3>
<p>Second, .gitignore should be modified to ignore “public/system” which includes the symlink and the directory, instead of the directory only with “public/system/”. In the case of Rails, the Rails app will automatically create the “system” directory if it does not exist, so that directory does not need to be committed with a hidden file (e.g. .empty). But in other cases where the directory is not automatically created, it might make sense to include “public/some_folder”, “public/some_folder/*”, and “!public/some_folder/.empty” in your gitignore so that the symlink and directory contents (except for .empty) are ignored.</p>
<p>It seems silly that a thing such as a single character (a “/” in this case) can wreak havoc on websites for me more than once, but such is life in programming. Backups and attention to detail are important!</p>
My Favorite Git Commandshttps://www.endpointdev.com/blog/2013/09/my-favorite-git-commands/2013-09-16T00:00:00+00:00Steph Skardal
<p>Git is a tool that all of us End Pointers use frequently. I was recently reviewing history on a server that I work on frequently, and I took note of the various git commands I use. I put together a list of the top git commands (and/or techniques) that I use with a brief explanation.</p>
<p><strong>git commit -m “****"</strong></p>
<p>This is a no-brainer as it commits a set of changes to the repository. I always use the -m to set the git commit message instead of using an editor to do so. <strong>Edit:</strong> <a href="/team/jon-jensen/">Jon</a> recommends that new users not use <em>-m</em>, and that more advanced users use this sparingly, for good reasons described in the comments!</p>
<p><strong>git checkout -b branchname</strong></p>
<p>This is the first step to setting up a local branch. I use this one often as I set up local branches to separate changes for the various tasks I work on. This command creates and moves you to the new branch. Of course, if your branch already exists, <em>git checkout branchname</em> will check out the changes for that local branch that already exists.</p>
<p><strong>git push origin branchname</strong></p>
<p>After I’ve done a bit of work on my branch, I push it to the origin to a) back it up in another location (if applicable) and b) provide the ability for others to reference the branch.</p>
<p><strong>git rebase origin/master</strong></p>
<p>This one is very important to me, and our blog has featured a couple of articles about it (<a href="/blog/2010/10/git-branches-and-rebasing/">#1</a> and <a href="/blog/2009/05/git-rebase-just-workingness-baked-right/">#2</a>). A rebase rewinds your current changes (on your local branch), applies the changes from origin/master (or whatever branch you are rebasing against), and then reapplies your changes one by one. If there are any conflicts along the way, you are asked to resolve the conflicts, skip the commit, or abort the rebase. Using a rebase allows you to avoid those pesky merge commits which are not explicit in what changes they include and helps you keep a cleaner git history.</p>
<p><strong>git push -f origin branchname</strong></p>
<p>I use this one sparingly, and <strong>only</strong> if I’m the only one that’s working on branchname. This comes up when you’ve rebased one of your local branches resulting in an altered history of branchname. When you attempt to push it to origin, you may see a message that origin/branchname has X commits different from your local branch. This command will forcefully push your branch to origin and overwrite its history.</p>
<p><strong>git merge –squash branchname</strong></p>
<p>After you’ve done a bit of work on branchname and you are ready to merge it into the master branch, you can use the <em>–squash</em> argument to squash/smush/combine all of your commits into one clump of changes. This command <strong>does not</strong> perform the commit itself, therefore it must be followed by a) review of the changes and b) git commit.</p>
<p><strong>git branch -D branchname</strong></p>
<p>If you are done with all of your work on branchname and it has been merged into master, you can delete it with this command! <strong>Edit:</strong> Phunk tells me that there is a difference between <em>-D</em> and <em>-d</em>, as with the latter option, git will refuse to delete a branch with unmerged changes, so <em>-d</em> is a safer option.</p>
<p><strong>git push origin :branchname</strong></p>
<p>Want to delete branchname from the origin? Run this command. You can leave branchname on the origin repository if you want, but I like to keep things clean with this command.</p>
<p><strong>git checkout -t origin/someone_elses_branch</strong></p>
<p>Use this command to set up a local branch to track another developers branch. As the acting technical project manager for one of my clients, I use this command to track <a href="/blog/authors/kamil-ciemniewski/">Kamil’s</a> branch, in combination with the next command (cherry-pick), to get his work cleanly merged into master.</p>
<p><strong>git cherry-pick hashhashhash</strong></p>
<p>Git cherry-pick applies changes from a single commit (identified by hash) to your current working branch. As noted above, I typically use this after I’ve set up a local tracking branch from another developer to cherry-pick his or her commits onto the master branch in preparation for a deploy.</p>
<p><strong>git stash, git stash apply</strong></p>
<p>I only learned about <em>git stash</em> in the last year, however, it’s become a go-to tool of mine. If I have some working changes that I don’t want to commit, but a client asks me to commit another quick change, I will often stash the current changes (save them but not commit them), run a rebase to get my branch up to date, then push out the commit, then run <em>git stash apply</em> to restore my uncommitted changes.</p>
<p>Admittedly, several of my coworkers are Git experts and have many more Git tools in their toolboxes—I should ask one of them to follow-up on this article with additional advanced git commands I should be using! Also take note that for us End Pointers, <a href="http://www.devcamps.org/">DevCamps</a> may influence our Git toolbox because it allows us to have multiple instances (and copies of the production database) running at a time, which may require less management of Git branches.</p>
Git as rsynchttps://www.endpointdev.com/blog/2013/02/git-as-rsync/2013-02-14T00:00:00+00:00Jeff Boes
<p>I had a quick-and-dirty problem to solve recently:</p>
<p>The clients had uploaded many assorted images to a development <a href="http://devcamps.org">camp</a>, but the .gitignore meant those updates were not picked up when we committed and pushed and rolled out to the live site. Normally, one would just rsync the files, but for various reasons this was not practical.</p>
<p>So my solution, which I think can get filed under “Stupid ‘git’ tricks (as opposed to Tricks of a Stupid Git)”:</p>
<p>(on the source repo)</p>
<pre tabindex="0"><code>$ git checkout -b images_update
$ git add -f path-to-missing-images
$ git commit -m 'Do not push me! I'm just a silly temporary commit'
</code></pre><p>(“add -f” forces the images into the index, overriding our gitignore settings)</p>
<p>(on the target repo)</p>
<pre tabindex="0"><code>$ git remote add images /path/to/source/repo
$ git fetch
$ git checkout -f images/images_update path-to-missing-images
$ git remote rm images
$ git reset HEAD path-to-missing-images
</code></pre><p>That last “git reset” is because the newly-restored images will be git-added by default, and we didn’t want them committed to the central repo.</p>
<p>So what did we do here? For those dumbfounded by the level of silly, we used git to record the state of all the files in a certain path; then we pulled them back out into another location without disturbing anything already there. But as a benefit, we do have a record of what happened, in case we need to reproduce it.</p>
DevCamps: Creating new camps from a non-default Git branchhttps://www.endpointdev.com/blog/2012/08/devcamps-creating-new-camps-from-non/2012-08-31T00:00:00+00:00Brian Gadoury
<p>I recently set up part of a new Rails project <a href="http://www.devcamps.org/">DevCamps installation</a> with a unique Git repo setup and discovered a trick for creating camps from a Git branch other than master. Admittedly, the circumstances that led to me discovering this trick are a bit specific to this project, but the trick itself can be useful in other situations as well.</p>
<p>The Git repo specified in local-config had a master branch with nothing in it but the standard “initial commit.” This relatively new project uses a simplifed <a href="http://jeffkreeftmeijer.com/2010/why-arent-you-using-git-flow/">git-flow workflow</a> and as such, all its code was still in the “develop” branch.</p>
<p>In my case, this empty-ish master branch meant there were no tracked files in <strong>CAMP_PATH</strong>/public directory. This meant that Git did not create that directory when the repo is cloned by <code>mkcamp</code>. This meant that apache2 would refuse to start. Camping without a web server makes my back hurt, so I snooped around a little bit…</p>
<p>I discovered two things:</p>
<ol>
<li>You can tell <code>git clone</code> which branch to checkout initially by passing it a ‘–branch $your_non_default_branch’ switch</li>
<li>The <code>mkcamp</code> command will happily pass that switch (as well as any other spicy options you include) along to the <code>git clone</code> system command it executes. To do that, just add it to your camp type’s local-config file as part of the ‘repo_path_git’ config variable. For example:</li>
</ol>
<p>repo_path_git:git@github.com:somegituser/somegitrepo.git –branch develop</p>
<p>Note that this option means your fresh new camp won’t have a ‘master’ branch checked out. This might confuse some users, but we all know the ‘master’ branch is nothing but a tracking branch with some convention mixed in. A simple <code>git checkout master</code> will create that expected master branch easily enough. It’s probably worth giving your devs a heads up about this, lest they think something wonky is afoot with mkcamp.</p>
<p>Now, there are people out there that may try to find fault with my solution. These detractors, these misanthropes, these malingering sluggards might cry “Why don’t you just commit an empty <strong>CAMP_PATH</strong>/public/.gitkeep” to your master branch?" Well, I like a clean git history. So, to those people I would say, “David, that’s messy and silly and wouldn’t make a very good blog article at all. I’m embarrassed for you for even bringing it up, <em>David.</em>”</p>
Git: Delete your files and keep them, toohttps://www.endpointdev.com/blog/2012/08/git-delete-your-files-and-keep-them-too/2012-08-30T00:00:00+00:00Jeff Boes
<p>I was charged with cleaning up a particularly large, sprawling set of files comprising a git repository. One whole “wing” of that structure consisted of files that needed to stay around in production (they were various PDFs, PowerPoint presentations, and Windows EXEs that were only ever needed by the customer’s partners, and downloaded from the live site – our <a href="http://www.devcamps.org/">developer camps</a> never wanted to have local copies of these files, which amounted to over 280 MB (and since we have dozens of camps shadowing this repository, all on the same server, this will save a few GB at least).</p>
<p>I should point out that our preferred deployment is to have production, QA, and development all be working clones of a central repository. Yes, we even push from production, especially when clients are the ones making changes there. (Gasp!)</p>
<p>So: the aim here is to make the stuff vanish from all the other clones (when they are updated), but to preserve the stuff in one particular clone (production). Also, we want to ensure that no future updates in that “wing” are tracked.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"><span style="color:#888"># From the "production" clone:</span>
$ <span style="color:#038">cd</span> stuff
$ git rm -r --cached .
$ <span style="color:#038">cd</span> ..
$ <span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"stuff"</span> >>.gitignore
$ git commit ...
$ git push ...
</code></pre></div><p>Now, everything that was in the “stuff” tree remains, for “production”, but every other clone will remove these files when they update from the central repository:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git pull origin master
...
delete mode <span style="color:#00d;font-weight:bold">100644</span> stuff/aaa
delete mode <span style="color:#00d;font-weight:bold">100644</span> stuff/aab
...
</code></pre></div>
Moving a Commit to Another Branch in Githttps://www.endpointdev.com/blog/2012/06/moving-commit-to-another-branch-in-git/2012-06-21T00:00:00+00:00Mike Farmer
<p>Perhaps you’ve made the same mistake I have. You’re right in the middle of developing a feature when a request comes up to fix a different completely unrelated problem. So, you jump right in and fix the issue and then you realize you forgot to start a new git feature branch. Suddenly you realize that you need to merge just the fix you made, but don’t want to merge the commits from the previous feature your working on.</p>
<p>Git rocks at manipulating branches and I knew this, but I wasn’t sure how to just move one commit to the master branch. After some digging and a little trial and error, I finally figured it out. This may not be the simplest approach, but it worked for me and wanted to share.</p>
<p>The branches I’ll be working with are master and feature. In the current scenario, the feature branch is 4 commits ahead of the master and the branch that I want to bring over is just the most recent.</p>
<p>First things first, I need to ensure my master branch is up to date.</p>
<pre tabindex="0"><code>git checkout master
git pull origin master
</code></pre><p>Then I’ll checkout my feature branch and make sure it’s completely up to date with the master branch.</p>
<pre tabindex="0"><code>git checkout feature
git rebase origin/master
</code></pre><p>Next, I’ll create a temporary feature branch that I’ll use later on to bring over the commit that I want.</p>
<pre tabindex="0"><code>git checkout -b feature_tmp
</code></pre><p>I’ll do the same for master so that I can perform my merging and rebasing in isolation from the master branch.</p>
<pre tabindex="0"><code>git checkout master
git checkout -b master_tmp
</code></pre><p>Now I’m going to merge the two tmp branches so that I have a history that contains all of my commits. This will give me the history that I want, but will include the 3 commits I don’t want.</p>
<pre tabindex="0"><code>git merge feature_tmp
</code></pre><p>Here’s where the magic happens. I’m going to rebase this branch using interactive mode. I want to rebase everything back to the last commit on the master branch. For simplicity in the commands here, we’ll just use SHA-MASTER in place of the actual SHA1 hash.</p>
<pre tabindex="0"><code>git rebase -i SHA-MASTER
</code></pre><p>This loads the commits into my editor and from here I just delete the 3 commits that I didn’t want on my master branch. This will give me the history I want with the 4th commit coming right after the last commit on the master branch. After deleting the commits, I just save and quit my editor.</p>
<p>Next, I merge my tmp branch into the master branch.</p>
<pre tabindex="0"><code>git checkout master
git merge master_tmp
git log
</code></pre><p>Now in the log, I can see the history is in the correct order, just how I wanted it. To finish things up, I’ll just push my changes and then rebase my feature branch which will reorder my commits to match the master branch and place my feature commits as the last three commits in the log.</p>
<pre tabindex="0"><code>git push origin master
git checkout feature
git rebase origin/master
git log
</code></pre><p>The last thing to do is delete my tmp branches.</p>
<pre tabindex="0"><code>git branch -D tmp_master
git branch -D tmp_feature
</code></pre>
EP Meeting: Clean Editor and Git Workflowshttps://www.endpointdev.com/blog/2012/06/ep-meeting-clean-editor-and-git/2012-06-14T00:00:00+00:00Matt Vollrath
<p>Having good editor configurations and Git habits is a great way to make work easier and less tedious. David Christensen showed us how to reduce cruft and leverage advanced features of Git to take control of the code.</p>
<p><a href="https://www.flickr.com/photos/80083124@N08/7187478135/" title="IMG_0830.JPG by endpoint920, on Flickr"><img alt="IMG_0830.JPG" height="375" src="/blog/2012/06/ep-meeting-clean-editor-and-git/image-0.jpeg" width="500"/></a></p>
<p>Indentation is a big part of reading and understanding code, too important to be ignored. Tabs can be interpreted differently in different editors, so using spaces makes life easier for you and your coworkers. Most editors have automatic indentation and tab translation settings to standardize the workflow. Remember, code should be optimized for humans.</p>
<p>Commit often! If your commit can not be summarized in one sentence, it is probably not granular enough. Don’t hesitate to make multiple commits per work session as you accomplish separate tasks. In your commit messages, describe the ‘why,’ not the ‘how.’ Don’t mix trivial style or whitespace tweaks with actual code modifications, because it makes it harder to catch important changes in diffs. If you make multiple changes to a single file, you can use -p/–interactive mode to commit hunks of code separately.</p>
Git Workflowshttps://www.endpointdev.com/blog/2012/06/git-workflows/2012-06-13T00:00:00+00:00Mike Farmer
<p><a href="/blog/authors/david-christensen/">David Christensen</a> is talking today about Git workflows.</p>
<p><a href="https://www.flickr.com/photos/80083124@N08/7369039912/" title="IMG_0721.JPG by endpoint920, on Flickr"><img alt="IMG_0721.JPG" height="375" src="/blog/2012/06/git-workflows/image-0.jpeg" width="500"/></a></p>
<p>There are different ways that you can work with Git. Git doesn’t dictate a certain workflow so you are free to implement one that works best for you. Understanding git and how it works will help you develop an effective workflow.</p>
<p>The Git object model provides Git’s flexibility and is as follows:</p>
<ul>
<li>trees, blobs</li>
<li>commits</li>
<li>named commits: tags, branches</li>
</ul>
<p>Branch flexibility comes through combining of branches (merges, rebase).</p>
<p>Good commits are key to flexibility/tools and should encapsulate the smallest logical change and a good log message describing the commit. It’s important to provide the why in your commit message in addition to what was fixed so that it’s clear to future developers.</p>
<p>Branches contain all the magic of Git in that it’s just a pointer to a commit.</p>
<p>Topic branches are convention driven branches that are merged off the master branch. They usually deal with a single topic and can be rebased onto master to provide a clean history. They can also be thrown away later so they don’t clutter up the repository.</p>
<p>Integration branches are usually for different levels of the application integration, for example, staging and production. They can be used to resolved conflicts and other small issues with the code.</p>
<p>Git also makes it easy to get quick version control ‘git init; git add’ is all you need. Then you can use ‘git grep’, which is faster than ‘grep -R’ for searching. A helpful command for understanding Git workflows is ‘git help workflows’.</p>
Reverting Git Commitshttps://www.endpointdev.com/blog/2012/05/reverting-git-commits/2012-05-02T00:00:00+00:00Szymon Lipiński
<p>Git is great, but it’s not always easy to use. For example, reverting a commit is a very nice feature. There are git commands for reverting a commit which has not been pushed to the main repository. However after pushing it, things are not so easy.</p>
<p>While I was working for one of our clients, I made about 20 commits and then I pushed them to the main repository. After that I realised that I was working on a wrong branch.
The new branch I should have used wasn’t created yet. I had to revert all my commits, create the new branch, and load all my changes into it.</p>
<p>Creating the branch named NEW_BRANCH is as easy as:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git branch NEW_BRANCH
</code></pre></div><p>Now the harder part… how to delete the commits pushed to the main repo. After reading through tons of documentation it turned out that it is not possible. You cannot just delete a pushed commit. However you can do something else.</p>
<p>As an example of this, I created a simple file, added a couple of lines there, and made four commits. The git log looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git log
commit dc47a884f7b303fc8b207550104f5a1de192c91c
Author: Szymon Guz
Date: Mon Apr 30 12:14:21 2012 +0200
replaced b with d
commit 68f56d3321324bd14cd1e73d003b1e151c4d43b4
Author: Szymon Guz
Date: Mon Apr 30 12:14:05 2012 +0200
added c
commit a77427d8151f143cacb85f00eb6c8170079dc290
Author: Szymon Guz
Date: Mon Apr 30 12:13:58 2012 +0200
added b
commit 73e586bb6d401f4049cf977703f25bf47c93b227
Author: Szymon Guz
Date: Mon Apr 30 12:13:49 2012 +0200
added a
</code></pre></div><p>Now let’s move the last 3 commits to another branch. I will create one diff for reverting the changes and one for replaying them on the new branch.
Let’s call these the ‘down’ and ‘up’ diff files: ‘down’ for reverting, and ‘up’ for recreating the changes.</p>
<p>The up diff can be created with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git diff 73e586bb6d401f4049cf977703f25bf47c93b227 dc47a884f7b303fc8b207550104f5a1de192c91c
diff --git a/test b/test
index 7898192..3171744 100644
--- a/test
+++ b/test
@@ -1 +1,3 @@
a
+d
+c
</code></pre></div><p>The down diff can be created using exactly the same command, but with switched parameters:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git diff dc47a884f7b303fc8b207550104f5a1de192c91c 73e586bb6d401f4049cf977703f25bf47c93b227
diff --git a/test b/test
index 3171744..7898192 100644
--- a/test
+++ b/test
@@ -1,3 +1 @@
a
-d
-c
</code></pre></div><p>I saved the diffs into files called ‘up.diff’ and ‘down.diff’.</p>
<p>On the old branch I want to revert the changes, after doing this I will just commit the changes and the branch will look like it was before all the commits. However all the commits stay in the branch. This something like a revert commit.</p>
<p>I reverted the changes on current branch with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ patch -p1 < down.diff
patching file test
$ git commit -a -m "reverted the changes, moved to another branch"
</code></pre></div><p>Now let’s move the changes into the new branch.
I need to create the new branch from the repo after the first commit:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git branch NEW_BRANCH 73e586bb6d401f4049cf977703f25bf47c93b227
</code></pre></div><p>Switch to the new branch:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git checkout NEW_BRANCH
</code></pre></div><p>Apply the up.diff patch to the new branch:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">patch -p1 < up.diff
</code></pre></div><p>And commit the changes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ git commit -a -m "Applied changes from the other branch"
</code></pre></div><p>I know that all the steps can be replaced with different ones, however this solution worked for me pretty well and without any problem.</p>
Hurray for tracking configuration files in source controlhttps://www.endpointdev.com/blog/2011/12/hurray-for-tracking-configuration-files/2011-12-15T00:00:00+00:00Josh Williams
<p>In a number of places we’ve started tracking configuration files locally in Git. It’s great for Postgres configs, Apache or nginx, DNS zone files, Nagios, all kinds of things. A few clients have private offsite repos we push to, like at GitHub, but for the most part they’re independent repos. It’s still great for keeping track of what was changed when, and by whom.</p>
<p>In one case we have a centralized Nagios instance that does little more than receive passive checks from a number of remote systems. I’d set the checks on the remote systems but not loaded that configuration in yet. However while getting the central system set up, muscle memory kicked in and I suddenly had a half-red console as it’s loading in stale data.</p>
<p>We don’t need a flood of false alerts over email, but I don’t want to completely revert the config and lose all those services…</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">[root nagios]# git stash; service nagios restart; git stash apply
Saved working directory and index state WIP on master: 0e9113b Made up commit for blog
HEAD is now at 0e9113b Made up commit for blog
Running configuration check...done.
Stopping nagios: .done.
Starting nagios: done.
# On branch master
# (etc)
</code></pre></div><p>Green! A small victory, for sure, but it shows one more advantage of modern source code management.</p>
DBD::Pg UTF-8 for PostgreSQL server_encodinghttps://www.endpointdev.com/blog/2011/06/dbdpg-utf-8-for-postgresql/2011-06-20T00:00:00+00:00Greg Sabino Mullane
<p><a href="/blog/2011/06/dbdpg-utf-8-for-postgresql/image-0-big.png" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" id="BLOGGER_PHOTO_ID_5620313937936840898" src="/blog/2011/06/dbdpg-utf-8-for-postgresql/image-0.png" style="float:right; margin:0 0 10px 10px;cursor:pointer; cursor:hand;width: 167px; height: 276px;"/></a></p>
<p>We are preparing to make a major version bump in DBD::Pg, the Perl interface for <a href="https://www.postgresql.org/">PostgreSQL</a>, from the 2.x series to 3.x. This is due to a reworking of how we handle UTF-8. The change is not going to be backwards compatible, but will probably not affect many people. If you are using the pg_enable_utf8 flag, however, you definitely need to read on for the details.</p>
<p>The short version is that DBD::Pg is going return all strings from the Postgres server with the Perl utf8 flag on. The sole exception will be databases in which the server_encoding is SQL_ASCII, in which case the flag will never be turned on.</p>
<p>For backwards compatibility and fine-tuning control, there is a new attribute called <strong>pg_utf8_strings</strong> that can be set at connection time to override the decision above. For example, if you need your connection to return byte-soup, non-utf8-marked strings, despite coming from a UTF-8 Postgres database, you can say:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-perl" data-lang="perl"> <span style="color:#080;font-weight:bold">my</span> <span style="color:#369">$dsn</span> = <span style="color:#d20;background-color:#fff0f0">'dbi:Pg:dbname=foobar'</span>;
<span style="color:#080;font-weight:bold">my</span> <span style="color:#369">$dbh</span> = <span style="color:#b06;font-weight:bold">DBI</span>-><span style="color:#038">connect</span>(<span style="color:#369">$dsn</span>, <span style="color:#369">$dbuser</span>, <span style="color:#369">$dbpass</span>,
{ AutoCommit => <span style="color:#00d;font-weight:bold">0</span>,
RaiseError => <span style="color:#00d;font-weight:bold">0</span>,
PrintError => <span style="color:#00d;font-weight:bold">0</span>,
pg_utf8_strings => <span style="color:#00d;font-weight:bold">0</span>,
}
);
</code></pre></div><p>Similarly, you can set pg_utf8_strings to 1 and it will force settings returned strings as utf8, even if the backend is SQL_ASCII. You should not be using SQL_ASCII of course, and certainly not forcing the strings returned from it to UTF-8. :)</p>
<p>All Perl variables (be they strings or otherwise) are actually Perl objects, with some internal attributes defined on them. One of those is the utf8 flag, which can be flipped on to indicate that the string should be treated as possibly containing multi-byte characters, or it can be left off, to indicate the string should always be treated on a byte-by-byte basis. This will affect things like the Perl <strong>length</strong> function, and the Perl <strong>\w</strong> regex flag. This is completely unrelated to the Perl pragma <strong>use utf8</strong>, which DBD::Pg has nothing at all to do with. Have I mentioned that UTF-8, and UTF-8 in Perl in particular, can be quite confusing?</p>
<p>There are a few exceptions as to what things DBD::Pg will mark as utf8. Integers and other numbers will not, boolean values will not, and no bytea data will ever have the flag set. When in doubt, assume that it is set.</p>
<p>The old attribute, <strong>pg_enable_utf8</strong>, will be deprecated, and have no effect. We thought about re-using that but it seemed clearer and cleaner to simply create a new variable (pg_utf8_strings), as the behavior has significantly changed.</p>
<p>A beta version of DBD::Pg (2.99.9_1) with these changes has been uploaded to CPAN for anyone to experiment with. Right now, none of this is set in stone, but we did want to get a working version out there to start the discussion and see how it interacts with applications that were making use of the
pg_enable_utf8 flag. You can web search for “dbdpg” and look for the “Latest Dev. Release”, or jump straight to <a href="https://metacpan.org/release/TURNSTEP/DBD-Pg-2.99.9_1">the page for DBD::Pg 2.99.9_1</a>. The trailing underscore is a CPAN convention that indicates this is a development version only, and thus will not replace the latest production version (2.18.1 as of this writing).</p>
<p>As a reminder, DBD::Pg has <a href="/blog/2011/06/dbdpg-moves-to-git/">switched to using git</a>, so you can follow along with the development
with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">git clone git://bucardo.org/dbdpg.git
</code></pre></div><p>There is also a commits mailing list you can join to receive notifications of commits as they are pushed to the main repo. To sign up, send an email to <strong><a href="mailto:dbd-pg-changes-subscribe@perl.org">dbd-pg-changes-subscribe@perl.org</a></strong>.</p>
DBD::Pg moves to Git!https://www.endpointdev.com/blog/2011/06/dbdpg-moves-to-git/2011-06-14T00:00:00+00:00David Christensen
<p>Just a note to everyone that development the official DBD::Pg DBI driver for PostgreSQL source code repository has moved from its old home in Subversion to a Git repository. All development has now moved to this repo.</p>
<p>We have imported the SVN revision history, so it’s just a matter of pointing your Git clients to:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git clone git://bucardo.org/dbdpg.git
</code></pre></div><p>For those who prefer, there is a GitHub mirror:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ git clone git://github.com/bucardo/dbdpg.git
</code></pre></div><p>Git is available via many package managers or by following the download links at <a href="https://git-scm.com/download">https://git-scm.com/download</a> for your platform.</p>
<p>Enjoy!</p>
Interactive Git: My New Found Friend(s)https://www.endpointdev.com/blog/2011/03/interactive-git-my-new-found-friends/2011-03-29T00:00:00+00:00Brian J. Miller
<p>As a software engineer I’m naturally inclined to be at least somewhat introverted :-), combine that with the fact that End Point is PhysicalWaterCooler challenged and you have a recipe for two things to naturally occur, 1) talking to oneself (but then who doesn’t do that really? no, really.), 2) finding friends in unusual places. Feeling a bit socially lacking after a personal residence move, I was determined to set out to find new friends, so I found one, his name is “–interactive”, or Mr. git add –interactive.</p>
<p>“How did we meet?” You ask. While working on a rather “long winded” project I started to notice myself sprinkling in TODOs throughout the source code, not a bad habit really (presuming they do actually eventually get fixed), but unfortunately the end result is having a lot of changed files in git that you don’t really need to commit, but at the same time don’t really need to see every time you want to review code. I’m fairly anal about reviewing code and so I was generally in the habit of running a <code>git status</code> followed by a <code>git diff </code> on every file that was mentioned by status. These are two great friends, but of late they just don’t seem to be providing the inspiration they once did. Enter my new friend <code>git add --interactive</code>. Basically he combines the two steps for me in a nice, neat controlled way while adding a bit of spice to my life, in particular per change inclusion capability. When running <code>git add</code> with the interactive flag you are provided with an overall index status immediately followed by a prompt. At that prompt you have an option of “5) patch”, by entering “5”, then return, you are then provided the index (basically) again. From that index you can select from a list of files for which you would like to review patches. For each reviewed patch you can then specify whether to include that patch for commit, skip it, divide (split) it into smaller patches for further review, or even edit it. When selecting the files to review the patches for it is simple to choose a range of files by entering a specifically formatted string, i.e. “1-12,15-18,19”. With –interactive the time it takes to review the code pending commit and skip through the TODOs is greatly reduced, something the client definitely appreciates.</p>
<p>“But what about your other old friends?” You then ask. Well, as it turns out my spending so much time with interactive add made <code>git stash</code> feel a bit lonely, and it dawned on me that tracking those TODOs in the working tree at all may be a bit silly. What could a guy do, perhaps these two friends might actually like to party together? As it turns out they had already been introduced and do like to party together (not sure why they couldn’t have just invited me before, though it might have something to do with my past friendship with SVN and RCS). Either way, to once and for all get those unsightly TODOs out from under my immediate purview while keeping other changes I still needed in the index I found <code>git stash save --patch --no-keep-index “TODO Tracking”</code>. “save” instructs git stash to save a new stash, “–patch” tosses it into an interactive mode similar to the one described above for add, “–no-keep-index” instructs stash not to keep the changes in the working tree that are added to the created stash, and the “TODO Tracking” is just a message to make it easy for a human to understand what the stash contains (I made this one up for my specific immediate purpose). This leaves my working tree and index clean for me to do more pressing work and to know that when I have the time/need to restore those past TODOs I can, so that they may be worked on as well. Note that I’ve not really used this technique much (read: I’ve just done it now for the first time) so we’ll see if it really is that useful, but the interactive patching I’ve used and it is definitely worth it.</p>
<p>As a further side bar I was discussing multiple commit indexes in a Git repo with someone in the #yui channel, and as soon as I found the above it occurred to me that using multiple stashes where you pop them could work in effect the same way, though I don’t know if there is a way to add patches to an already created stash. That might make a neat feature to investigate and/or request from the Git core.</p>
<p>Just so you aren’t too concerned, there is still a place in my heart for <code>git add</code> and <code>git status</code> even if I don’t see them as frequently as I once did.</p>