• Home

  • Custom Ecommerce
  • Application Development
  • Database Consulting
  • Cloud Hosting
  • Systems Integration
  • Legacy Business Systems
  • Security & Compliance
  • GIS

  • Expertise

  • About Us
  • Our Team
  • Clients
  • Blog
  • Careers

  • VisionPort

  • Contact
  • Our Blog

    Ongoing observations by End Point Dev people

    Practical Linux Command Line Tips for Productivity and Efficiency

    Edgar Mlowe

    By Edgar Mlowe
    June 22, 2024

    A shadow draws sharp lines on brightly lit concrete. On the left side is a thick line descending across the full height diagonally, connected at the top to a subtly curved line which spans the full width of the image. It is connected by a thinner parallel diagonal line, which has two more horizontal lines equally spaced across the image.

    Feeling stuck with basic Linux commands? You’re not alone! Many people know some commands but don’t feel efficient. The good news is that with some know-how and common commands, you can transform your skills.

    In this post, you’ll learn to combine commands with pipes, master shell techniques, efficiently recall and edit past commands, and navigate the filesystem with speed. Let’s dive in!

    1. Combining Commands with Pipes

    Understanding How Pipes Work

    Ever wish your commands could work together? Pipes make it happen! They let one command pass its output to another, creating a smooth workflow.

    Commands take input (stdin) and give output (stdout). The pipe symbol (|) links them, making life easier.

    For example, ls -l /bin lists files in the /bin directory, but the amount of output is overwhelming. Try ls -l /bin | less to view one screen at a time—like having a personal organizer! This command pipes the output of the ls command into the pager less, which is very useful for reading large amounts of data.

    Let’s dive into pipes and practice them with commands like wc, head, cut, grep, sort, and uniq to handle data like a pro. If any of these commands are new to you, use man (short for “manual”) followed by the command name to learn more. For example:

    man grep

    Practical Examples: Real-World Use Cases for Combining Commands

    Example 1: Extracting Information from Log Files

    Task: Find all error messages in a log file and count them.

    Sample Data (application.log):

    INFO 2024-06-07 User login successful
    ERROR 2024-06-07 Failed to connect to database
    INFO 2024-06-07 User logout
    ERROR 2024-06-07 Timeout while retrieving data
    INFO 2024-06-07 Session expired
    ERROR 2024-06-07 Null pointer exception


    grep 'ERROR' application.log | wc -l


    • grep 'ERROR' application.log searches for lines containing “ERROR” in application.log.
    • The output is then passed to wc -l, which counts the number of lines, giving you the total number of error messages. wc is short for “word count”, while -l stands for lines.

    Expected Output:

    Example 2: Processing XML Files for Public Health Data

    Task: Extract all patient IDs from multiple XML files and sort them uniquely.

    Sample Data (file1.xml, file2.xml):

    • file1.xml:

              <name>John Doe</name>
              <name>Jane Smith</name>
    • file2.xml:

              <name>Jane Smith</name>
              <name>Mary Johnson</name>


    grep '<patientId>' *.xml | cut -d '>' -f 2 | cut -d '<' -f 1 | sort | uniq


    • grep '<patientId>' *.xml searches for lines containing <patientId> in all XML files.
    • The output is then piped to cut -d'>' -f2 to extract the content after the > character.
    • cut -d'<' -f1 further extracts the content before the < character.
    • sort sorts the IDs alphabetically.
    • uniq removes duplicate entries.

    Expected Output:

    Example 3: Organizing Text Data into a CSV File

    Task: Convert a space-separated file to a CSV file.

    Sample Data (data.txt):

    Name Age Location
    Alice 30 New_York
    Bob 25 Los_Angeles
    Charlie 35 Chicago


    cat data.txt | tr ' ' ',' > data.csv


    • cat data.txt reads the content of data.txt.
    • The output is then piped to tr ' ' ',', which translates spaces to commas.
    • > data.csv redirects the final output to data.csv. We’ll cover redirection in more detail later.

    Expected Output (data.csv):


    These examples demonstrate how simple commands can be combined to achieve powerful and useful results. Experiment with these commands and modify them to fit your specific needs!

    2. Mastering Shell Techniques

    The shell is the heart of your command line, acting as the bridge between you and the operating system. It does more than just run commands; it helps you manage and streamline your tasks efficiently. Let’s dive into some essential shell techniques.

    Understanding the Shell with ls

    To get a clear picture of what the shell does versus what a command does, let’s look at the ls command. When you run ls, the shell and the command work together to get things done:

    • Shell’s Job: The shell processes wildcards (like *.py), sets up input and output redirections, and manages environment variables.
    • Command’s Job: The ls command lists directory contents.

    Example: List all Python files in a directory.

    ls *.py

    In this example:

    • The shell expands *.py to match all Python files (e.g., data.py, main.py).
    • The ls Command receives the list of Python files and displays them.
    1. Pattern Matching for Filenames

    Pattern matching allows you to select multiple files with a single command using wildcards.

    Example 1: List all text files.

    ls *.txt

    This command lists all files ending with .txt, like report.txt, notes.txt, and summary.txt.

    Example 2: List all files that start with “data”.

    ls data*

    This command lists all files starting with “data”, like data1.csv, data2.csv, and database.db.

    Example 3: List all files that have numbers in their names.

    ls *[0-9]*

    This command lists files that contain any digit, like file1.txt, report2.doc, and data2023.csv.

    2. Variables to Store Values

    Variables let you store data that can be reused in your commands.

    Example 1: Create a variable to store your project directory path.


    This command sets the PROJECT_DIR variable and then uses it to change to that directory.

    Example 2: Store the result of a command in a variable.

    DATE=$(date +%Y-%m-%d)
    echo "Today's date is $DATE"

    This command sets the DATE variable to the current date using command substitution and then prints the contents of the DATE variable.

    3. Redirection of Input and Output

    Redirection allows you to control where your command’s input and output go.

    Example 1: Save the output of a command to a file.

    ls -l > filelist.txt

    This command lists all files in long format and saves the output to filelist.txt. Redirection with the single right bracket will replace the contents of the target file.

    Example 2: Append the output of a command to a file.

    echo "New entry" >> filelist.txt

    This command adds “New entry” to the end of filelist.txt. Redirection with two right brackets will append to the contents of the target file.

    Example 3: Use a file as input for a command.

    sort < unsorted.txt

    This command sorts the contents of unsorted.txt and displays the result. The single left bracket reads the contents of the file on the right (unsorted.txt in this case) and uses them as input to the command on the left (sort in this case).

    4. Quoting and Escaping to Disable Shell Features

    Quoting and escaping are used to handle special characters in your commands. Double quotes won’t expand wildcards or break the string on spaces, but will still expand variables and substitute commands with $(). Content in single quotes isn’t processed by the shell at all; all characters are retained literally.

    Example 1: Search for a phrase with spaces in a file.

    grep "my search phrase" file.txt

    This command searches for the exact phrase “my search phrase” in file.txt.

    Example 2: Use a variable inside a double-quoted string.

    echo "Hello, $NAME"

    This command prints “Hello, Alice” using the NAME variable.

    Example 3: Use single quotes to prevent variable expansion.

    echo 'Hello, $NAME'

    This command prints Hello, $NAME without expanding the variable.

    Example 4: Use backslashes to escape special characters in a double-quoted string.

    echo "This path costs \$139: C:\\Windows\\System32"

    This command prints This path costs $139: C:\Windows\System32. The dollar sign and backlashes must be escaped with a \ so that they aren’t treated as special characters.

    Example 5: Include a literal double quote in a string.

    echo "He said, \"Hello, World!\""

    This command prints He said, "Hello, World!" by escaping the double quotes.

    Example 6:: Include a literal single quote in a single-quoted string.

    echo 'What'\''s up?'

    This command prints “What’s up?” using a more complex escape sequence—first, you close the single-quote string, then without a space you add a literal single-quote character with \', then you start a single-quoted string again. In bash, when there are no spaces between two strings, the strings will be joined together.

    5. The Search Path for Locating Programs to Run

    The shell uses the PATH variable to find programs to execute.

    Example 1: View your current PATH.

    echo $PATH

    This command displays the directories in your current PATH.

    Example 2: Add a directory to your PATH temporarily.

    export PATH=$PATH:/home/user/bin

    This command adds /home/user/bin to your PATH for the current session. It reads the PATH variable, then concatenates the string “:/home/user/bin” to the end, similarly to how we escaped a single quote.

    Example 3: Add a directory to your PATH permanently.

    echo 'export PATH=$PATH:/home/user/bin' >> ~/.bashrc
    source ~/.bashrc

    This command appends the new PATH to your .bashrc file (using redirection!). .bashrc is run every time a new shell is started. source reloads .bashrc for the current shell, using the new PATH for the current session.

    6. Saving Changes to Your Shell Environment

    Save environment changes by adding them to your shell configuration file.

    Example 1: Set an environment variable for the current session.

    export EDITOR=nano

    This command sets the default text editor to nano for the current session.

    Example 2: Make an environment variable permanent.

    echo 'export EDITOR=nano' >> ~/.bashrc
    source ~/.bashrc

    This command sets the default text editor to nano by adding it to your .bashrc file.

    Example 3: Create an alias for a command.

    echo 'alias ll="ls -l"' >> ~/.bashrc
    source ~/.bashrc

    This command creates an alias ll for ls -l and makes it permanent by adding it to your .bashrc file.

    Mastering these shell techniques will boost your productivity and make your command line experience more enjoyable. Dive in and start practicing these essential commands!

    3. Efficient Command Recall and Editing

    Tired of retyping long commands or fixing typos from scratch? The shell’s got you covered with command history and command-line editing, designed to save you time and effort.

    Imagine you just ran a complex command:

    md5sum *.jpg | cut -c 1-32 | sort | uniq -c | sort -nr

    Need to run it again? No need to retype it. The shell keeps a history of every command you run, so you can quickly recall and rerun them. This feature, known as command history, is a favorite among power users for speeding up their workflow and cutting down on repetitive typing.

    Accidentally typed *.jg instead of *.jpg?

    md5sum *.jg | cut -c 1-32 | sort | uniq -c | sort -nr

    No problem! Instead of starting over, just edit the command directly. The shell’s command-line editing lets you fix typos on the fly, similar to using a text editor.

    In this section, you’ll discover how to leverage command history and command-line editing to enhance your efficiency and minimize errors.

    Command History

    The shell keeps a record of every command you execute, allowing you to recall and reuse them with ease.

    Viewing the Command History

    The history command lists all previously executed commands in your shell session. Each command is assigned an ID number for easy reference.




    1000  cd $HOME/Music
    1001  ls
    1002  mv jazz.mp3 jazzy-song.mp3
    1003  play jazzy-song.mp3
    1481  cd
    1482  firefox https://google.com
    1483  history

    To view the most recent commands, limit the output with a number.


    history 3


    1482  firefox https://google.com
    1483  history
    1484  history 3

    You can also filter the history to find specific commands by piping to grep.


    history | grep "cd"


    1000  cd $HOME/Music
    1092  cd ..
    1123  cd Finances
    1375  cd Checking
    1481  cd
    Environment Variables for History

    Environment variables control how your shell handles command history. Key variables include HISTSIZE, HISTFILESIZE, and HISTCONTROL.

    HISTSIZE: Determines the number of commands to remember in the current session.


    echo $HISTSIZE



    To change the size:

    export HISTSIZE=1000

    HISTFILESIZE: Sets the number of commands to save in the history file.





    To change the file size:

    export HISTFILESIZE=2000

    HISTCONTROL: Controls what commands are saved. Common values include ignorespace (ignore commands starting with a space) and ignoredups (ignore duplicate commands).


    export HISTCONTROL=ignoredups

    To combine options:

    export HISTCONTROL=ignoredups:ignorespace

    HISTIGNORE: Excludes specific commands from being saved.


    export HISTIGNORE="ls:cd:pwd"

    This setting prevents ls, cd, and pwd commands from being saved in the history.

    Note that you should add these export commands to your .bashrc for them to take effect outside the current shell.

    History Expansion

    History expansion is a powerful shell feature that lets you reuse and modify previous commands quickly. It uses special expressions, typically starting with an exclamation mark (!), to reference commands from your history.

    Basic History Expansion

    You can repeat the last command with !!.


    echo "Hello, World!"

    This reruns echo "Hello, World!".

    Referencing Specific Commands

    To repeat a specific command from your history, use its ID number.



    This reruns the command with ID 1002 (mv jazz.mp3 jazzy-song.mp3 in our example).

    Using Substring Matches

    You can recall the most recent command that starts with a specific string.



    This reruns the most recent mv command.

    To recall a command containing a specific substring anywhere, use !?string?.



    This reruns the most recent command containing cd.

    Modifying Previous Commands

    You can also modify the previous command before running it. For instance, use ^old^new to replace the first occurrence of old with new in the last command.


    echo "Hello, Alice"

    This changes Alice to Bob, rerunning the command as echo "Hello, Bob".

    Using !$ and !*

    !$ refers to the last argument of the previous command.


    ls /some/directory
    cd !$

    This changes to /some/directory.

    !* refers to all arguments of the previous command.


    cp file1.txt file2.txt /backup/
    rm !*

    This removes both file1.txt and file2.txt from /backup/.

    Safety with :p

    To avoid mistakes, use :p to print the command without executing it.



    This prints the command with ID 1002 without running it. You can then confirm it before rerunning it.

    Command-Line Editing

    Command-line editing is essential for fixing mistakes, modifying commands, and creating new commands efficiently. Here are some key techniques to help you edit commands quickly and effectively.

    Cursoring Within a Command

    Use the left and right arrow keys to move the cursor back and forth within a command line. This allows you to make changes without retyping the entire command.

    Example: To correct a typo, press the left arrow key to move the cursor to the mistake, fix it, and then press Enter.

    Useful Keystrokes:

    • Left arrow: Move left by one character
    • Right arrow: Move right by one character
    • Ctrl + left arrow: Move left by one word (on macOS, try Option + left arrow)
    • Ctrl + right arrow: Move right by one word (on macOS, try Option + right arrow)
    • Ctrl + a: Move to the beginning of the command line
    • Ctrl + e: Move to the end of the command line

    Press Ctrl + r to initiate an incremental search through your command history. This lets you search for a previously run command without scrolling through the entire history.

    Example: Press Ctrl + r and start typing part of a previous command. The shell will display matching commands as you type. Press Ctrl + r again to cycle through other matches. Once you find the command, press Enter to run it or use the arrow keys to edit it.

    Additional Tips:

    • To recall the most recent search string, press Ctrl + r twice.
    • To stop the search and continue editing the current command, press Esc, Ctrl + j, or any arrow key.
    • To quit the search and clear the command line, press Ctrl + g or Ctrl + c.

    By mastering these command-line editing techniques, you can save time, reduce errors, and improve your efficiency when working in the shell.

    4. Navigating the Filesystem with Speed

    Efficient Directory Navigation

    Navigating the filesystem efficiently can significantly boost productivity. Here are some techniques:

    Using cd

    The cd (change directory) command is fundamental for navigation. Here are some practical tips:

    • Jump to the Home Directory:


      This command takes you directly to your home directory, regardless of your current location in the filesystem.

    • Navigate Using Variables:

      cd $HOME/Work
      cd ~/Work

      Both $HOME and ~ are shortcuts to your home directory, making it easy to navigate to subdirectories within it.

    • Navigate to Another User’s Home Directory:

      cd ~username

    Set the CDPATH variable to define base directories for quick navigation.

    • Setting CDPATH:

      export CDPATH=$HOME:$HOME/Work
    • Using CDPATH:

      cd Work

    Organizing Your Home Directory

    A well-organized home directory structure can streamline your workflow.

    Strategies for Fast Navigation
    • Use Descriptive Names:

      mkdir -p $HOME/Projects/{Work,Personal,OpenSource}
    • Create Aliases:

      alias work="cd $HOME/Projects/Work"
      alias personal="cd $HOME/Projects/Personal"

    Using pushd and popd for Directory Management

    The pushd and popd commands are used to manage a stack of directories, allowing for efficient navigation between them. Here’s how they work:

    • pushd: Adds a directory to the stack and changes to that directory.
    • popd: Removes the top directory from the stack and changes to the new top directory.
    Basic Usage
    1. Pushing a Directory:

      pushd /path/to/directory

      This command adds /path/to/directory to the stack and changes the current directory to it.

    2. Popping a Directory:


      This command removes the top directory from the stack and changes the current directory to the new top directory.

    Example Workflow

    Let’s say you start in your home directory (~/) and execute the following commands:

    1. Pushing Directories:

      pushd /var/www/html
      pushd /etc/apache2

      The stack now looks like this:

      ~/           (top)
    2. Popping Directories:


      After this command, the stack looks like this:

      ~/           (top)
    3. Navigating with Aliases:

      You can create aliases for pushd and popd to make navigation easier:

      alias pd="popd"
      alias gd="pushd"

      Now you can use gd to push directories onto the stack and pd to pop directories off the stack.

    These techniques will help you navigate your filesystem efficiently, allowing you to focus more on your tasks and less on typing long paths.


    Mastering the command line can transform your productivity and efficiency. We’ve covered combining commands with pipes, mastering shell techniques, recalling and editing commands efficiently, and navigating the filesystem with speed. Dive in, experiment, and watch your skills grow!

    For further reading, I highly recommend the book “Efficient Linux at the Command Line”. It took my command-line skills to the next level and inspired many of the tips and tricks shared in this blog.

    linux shell tips