Introducing EP Audit

2026-05-11T00:00:00+00:00

Photo by Bimal Gharti Magar, 2026.

In the complex landscape of cloud security, staying compliant and prepared for audits is important. Enter EP Audit, a compliance-focused audit process, AI-powered but led by senior engineers. It’s designed for Azure, AWS, and Google Cloud setups. EP Audit emerged from necessity: When a financial services client required an Azure security audit, we at End Point built a solution that not only met their immediate needs, but also evolved into an internal tool we continue to refine. Today, it serves as a key part of our multi-cloud security audit framework and is tested against our own cloud accounts before being used in client engagements.

The Problem

Cloud environments are dynamic: New projects, personnel changes, and vendor updates can significantly alter configurations over time. As environments evolve, the security posture originally reviewed and approved by your engineers may no longer fully match what is running in production. Unused resources, overly permissive access, and orphaned infrastructure can also accumulate over time, increasing both risk and monthly costs. This often goes unnoticed until an audit, customer review, or security incident forces a closer look.

Modern compliance frameworks—SOC 2, ISO 27001, HIPAA, PCI DSS—require regular, documented security reviews. Yet many organizations still struggle to provide consistent documentation with timestamps, severity tracking, and proof of remediation. EP Audit helps address this gap by generating a structured audit trail throughout the review process.

What EP Audit Looks Like

An EP Audit engagement is structured as a scheduled review cycle with two phases: a Base scan to establish a security snapshot, and a Final scan performed after remediation work to verify fixes. Both phases use a command-line runner with a read-only audit identity, ensuring no changes are made during the audit itself. Each Base scan produces nine deliverables designed to provide both high-level summaries and detailed technical findings:

Executive Summary: Severity-ranked audit findings.
Client Report: Full narrative with risk scorecard and roadmap.
Structured Findings Report: Engineer-friendly findings by category.
Pre-filled Hardening Checklist: CIS-aligned checklist with auto-populated FAIL statuses.
Fillable Hardening Checklist: A remediation tool for your team.
Network Diagram: Auto-generated deployment topology.
Findings Workbook: Excel file with color-coded severity tabs.
Remediation Plan: Phased plan with affected resources.
Partner Remediation Identity Setup: Guide for temporary write scope during remediation.

These deliverables are standardized across Azure, AWS, and GCP, helping simplify multi-cloud audit reviews and recurring engagements.

Sample summary of findings by severity and category on EP Audit.

Every finding in EP Audit is mapped to controls from recognized security frameworks. For Azure, we reference the CIS Microsoft Azure Foundations Benchmark and the Microsoft Cloud Security Benchmark. AWS findings align with the CIS AWS Foundations Benchmark, AWS Foundational Security Best Practices, and related guidance. GCP findings follow the CIS GCP Foundation Benchmark and the Google Cloud Architecture Framework Security Pillar. This helps keep audit findings grounded in established industry standards rather than proprietary scoring systems or arbitrary recommendations.

A Service, Not Just a Tool

There are already plenty of cloud configuration scanners available. What matters is how the findings are reviewed, prioritized, and applied in real environments.

Senior Engineering Review: Our engineers provide context and actionable recommendations for each finding, bridging the gap between automated results and practical solutions.
End Point-Specific Enhancements: We include checks beyond published frameworks, such as billing data analysis for orphaned resources and third-party security tool integration.
Internal Validation: Every framework change is tested on End Point’s own cloud accounts before client deployment.

A single audit provides a snapshot in time, but recurring audits make it easier to identify trends, validate remediation progress, and detect configuration drift over time. EP Audit includes year-over-year comparison reporting to help highlight both improvements and regressions. In practice, this helps support compliance reviews, procurement questionnaires, and ongoing security review processes while providing better visibility into how environments evolve over time.

Phased Remediation

EP Audit generates remediation scripts and recommendations organized by severity:

P1 (0–7 days): Critical issues like exposed management ports.
P2 (30 days): High-priority issues such as encryption gaps.
P3 (90 days): Medium-priority defense-in-depth weaknesses.
P4 (Maintenance): Low-priority configuration hygiene.

Scripts default to dry-run mode, allowing proposed changes to be reviewed before execution. Our team can assist with remediation directly, or clients can implement the recommendations independently using the provided guidance.

Over multiple engagements, organizations begin building a clearer picture of how their cloud environments evolve over time. Recurring audits make it easier to track remediation progress, identify recurring issues, and detect configuration drift before it becomes a larger problem. The resulting documentation can also help support compliance reviews, customer security questionnaires, cyber insurance requirements, internal security tracking, and reduce monthly costs.

Working with End Point

If your organization operates workloads in Azure, AWS, or Google Cloud and needs a structured, standards-aligned cloud security review process, feel free to contact us. If you are working with us already, you can talk with your representative to find out more.

Risks of Paper Checks for Secure Transactions

2025-08-18T00:00:00+00:00

Fraud is on the rise, and paper checks are one of the easiest targets. In the not-so-distant past, a grandparent could tuck a crisp $5 bill into an envelope and trust it would arrive. Today, even checks sent to pay your bills are being stolen, altered, and cashed by criminals before they reach their destination.

So, the practice of sending a paper check in the mail to reach a creditor should be stopped. According to the US Financial Crimes Enforcement Network (FinCEN), there were over 660,000 check fraud reports in 2024. This number has stayed in the 600,000s every year since 2022, when it nearly doubled from the previous year.¹ The U.S. Postal Service has seen similar increases in mail theft, with criminals targeting collection boxes and even robbing postal carriers for paper checks.

Fraudsters have many ways to manipulate checks. Intercepting checks that are not in a secured location by simply paying attention to where mail is lying around is at the top of the list. Once they get a check in hand, they can “wash” original amounts and payees and replace them with higher numbers and their own names.

A recent incident at our company showed how risky paper checks can be. A check mailed to End Point by one of our long time customers was intercepted and fraudulently cashed by someone else. We contacted the customer multiple times to collect on the open invoice. They were sure they had paid the invoice. When they sent us a photo of the front and back of the cancelled check, we could see that an unknown person had endorsed the check and cashed it! In some cases, the sender is left holding the financial loss with no way to recover the funds.

At End Point, we don’t cash customer checks in person — all payments are deposited directly into our business accounts. While this adds some protection, it’s useless if the check never reaches us.

The safest way to pay today is electronically. We strongly encourage customers to switch to secure electronic methods such as ACH transfers or direct deposits. If a check is unavoidable, send it via Certified Mail with tracking. Protecting your payment protects both your business and ours.

Data aggregated from https://www.fincen.gov/reports/sar-stats ↩︎

How to Analyze Application Logs and Extract Actionable Insights

2025-02-28T00:00:00+00:00

Photo by Timo C. Dinger on Unsplash

Logs often accumulate unnoticed—until something breaks. Then, they suddenly become vital tools for diagnosing issues. By combining just a few command-line techniques, you can quickly spot recurring problems, identify suspicious activity, and strengthen your application’s defenses.

Below is a sample error log we’ll reference in the examples:

[2025-01-01 12:34:56] [client 192.168.1.1:12345] PHP Notice: Undefined variable $foo in /var/www/html/index.php on line 45 | referer: http://example.com/index.php
[2025-01-01 12:34:56] [client 192.168.1.1:12345] PHP Notice: Undefined variable $foo in /var/www/html/index.php on line 45 | referer: http://example.com/index.php
[2025-01-01 12:34:57] [client 192.168.1.1:12345] PHP Notice: Undefined variable $foo in /var/www/html/index.php on line 45 | referer: http://example.com/index.php
[2025-01-01 12:35:00] [client 10.0.0.2:6789] PHP Warning: Division by zero in /var/www/html/script.php on line 23 | referer: http://example.com/script.php
[2025-01-01 13:35:01] [client 10.0.0.2:6789] PHP Warning: Division by zero in /var/www/html/script.php on line 23 | referer: http://example.com/script.php
[2025-01-01 13:36:10] [client 192.168.1.1:12345] PHP Fatal error: Call to undefined function baz() in /var/www/html/lib.php on line 78 | referer: http://example.com/index.php
[2025-01-01 14:36:11] [client 172.16.0.5:54321] PHP Fatal error: Call to undefined function baz() in /var/www/html/lib.php on line 78 | referer: http://example.com/index.php

A Note on Commands

Throughout this blog, you’ll notice repeated use of commands like awk, grep, sed, uniq, and sort. These tools are indispensable for log analysis, allowing you to filter, transform, and summarize data efficiently. Here are some tips for learning these tools:

Use man <command> (e.g., man awk) to access the manual and dive deeper into a specific command
Experiment with small examples to understand how commands like awk process patterns or how grep extracts data
For a broader look at how these commands enhance productivity, check out my blog post: Practical Linux Command Line Tips for Productivity and Efficiency

These commands are not just for log analysis—they’re powerful for any data manipulation task you encounter. With practice, they can become essential to your workflow.

1. Debugging Application Errors

1.1 Summarize Frequent Issues

sed -E 's/^\[.*\] //' error.log | sort | uniq -c | sort -nr

Purpose: Remove timestamps/client’s IP, then sort and count duplicates
Outcome: A quick snapshot of which errors appear most often

3 PHP Notice: Undefined variable $foo ...
2 PHP Warning: Division by zero ...
2 PHP Fatal error: Call to undefined function baz() ...

1.2 Categorize Errors by Type

grep -oE 'PHP [^:]+' error.log | sort | uniq -c | sort -nr

Purpose: Extract error types (e.g., Notice, Warning, Fatal error)
Outcome: Know whether notices, warnings, or fatal errors dominate

3 Notice
2 Warning
2 Fatal error

1.3 Find Problematic Scripts

awk -F'in ' '{print $2}' error.log | awk '{print $1}' | sort | uniq -c | sort -nr

Purpose: Identify which files generate the most errors
Outcome: Pinpoint error hotspots for focused debugging

3 /var/www/html/index.php
2 /var/www/html/script.php
2 /var/www/html/lib.php

2. Monitoring System Behavior

2.1 Track Problematic IPs

awk -F'\[client ' '{print $2}' error.log | awk -F':' '{print $1}' | sort | uniq -c | sort -nr

Purpose: Count how many errors each IP triggers
Outcome: Identify suspicious or high-traffic IPs

4 192.168.1.1
2 10.0.0.2
1 172.16.0.5

2.2 Spot High-Error Time Periods

awk -F'[][]' '{split($2, time, ":"); print $1, time[1]":"time[2]":00"}' error.log | sort | uniq -c | sort -nr

Purpose: Group errors by minute, revealing spikes or trends
Outcome: Correlate error surges with deployments or traffic peaks

3 2025-01-01 12:34:00
1 2025-01-01 14:36:00
1 2025-01-01 13:36:00
...

2.3 Analyze IP–Referer Pairs

grep "referer:" error.log | awk -F'\\[client ' '{print $2}' | awk -F':| referer: ' '{print $1, $3}' | sort | uniq -c | sort -nr

Purpose: Match IP addresses with the URLs they refer from
Outcome: Detect repeat offenders or malicious traffic patterns

3 192.168.1.1 Undefined variable $foo ...
2 10.0.0.2 Division by zero ...
1 172.16.0.5 Call to undefined function baz() ...

3. Enhancing Security

3.1 Watch Sensitive URLs

grep -E "admin|login" script_error_frequency.txt | sort -nr

Purpose: Flag frequent hits on admin or login pages
Outcome: Gauge whether sensitive endpoints are under attack

2 /var/www/html/login.php
2 /var/www/html/admin.php

3.2 Pinpoint Critical Times

grep -E "admin|login" error.log | awk -F'[][]' '{split($2, time, ":"); print $1, time[1]":"time[2]":00"}' | sort | uniq -c | sort -nr

Purpose: Identify specific time windows for sensitive access
Outcome: Cross-reference suspicious activity with your security logs

2 2025-01-01 12:34:00
1 2025-01-01 13:35:00
1 2025-01-01 12:35:00

Conclusion

These commands demonstrate how quickly you can glean insights from logs. With a little creativity, you can expand them to track response times, detect performance bottlenecks, and safeguard critical endpoints. Whether you’re tackling PHP errors or any other type of log data, the same principles apply: filter, sort, count, and investigate.

Curious to learn more? Combine these strategies with automation tools, integrate them into CI/CD pipelines, or hook them up to visual dashboards. Your logs will become a gold mine of actionable information.

Streamlining SELinux Policies: From Policy Modules to Modules and Silent SELinux Denials

2024-08-26T00:00:00+00:00

Introduction

SELinux (Security-Enhanced Linux) provides a robust security layer that enforces security policies to control system access. When dealing with SELinux, you often encounter the terms “policy_module” and “module”. Understanding the difference between these and knowing how to convert between them is crucial for efficient system administration.

What is a policy_module?

A policy_module in SELinux is a type of module used to define additional policies. These modules encapsulate specific security rules that can be loaded into the SELinux policy to grant or restrict permissions. Policy modules are particularly useful for adding or modifying policies without changing the base SELinux policy.

policy_module(my_policy, 1.0)

require {
    type my_app_t;
}

#============= my_app_t ==============
allow my_app_t my_log_t:file read;

What is a module?

A module in SELinux is a compiled version of a policy module. The compilation process translates the high-level policy rules into a binary format that SELinux can enforce. Modules are loaded into the SELinux policy store to extend or modify the active policy.

module my_module 1.0;

require {
    type my_app_t;
    class file { read write };
}

#============= my_app_t ===============
allow my_app_t my_log_t:file { read write };

Converting policy_module to module

In many scenarios, it’s necessary to convert a policy_module to a module. This conversion ensures compatibility and avoids the need for additional utilities such as selinux-polgenui.

Here’s how you can do it:

Create a .te file. This file contains the policy rules.

policy_module(my_policy, 1.0)

require {
    type my_app_t;
}

#============= my_app_t ==============
allow my_app_t my_log_t:file read;

Compile the policy module. Use the checkmodule and semodule_package tools to compile the policy module into a module.

checkmodule -M -m -o my_policy.mod my_policy.te
semodule_package -o my_policy.pp -m my_policy.mod

Load the module. Use semodule to load the compiled module into SELinux (using -i for “install”).

semodule -i my_policy.pp

The silent blocks of SELinux

Sometimes, SELinux can quietly block your software, leading to silent failures. This can be particularly tricky because SELinux usually logs issues in /var/log/audit/auditd.log or /var/log/messages. However, if a permission or property has the dontaudit setting applied, it won’t be logged, making it harder to troubleshoot.

In many cases, system administrators expect SELinux to be vocal about problems. When SELinux doesn’t log an issue due to dontaudit, it can quietly block your software without any visible errors.

Troubleshooting silent blocks

To diagnose whether SELinux is causing a problem, you can temporarily set SELinux to “permissive” mode:

setenforce 0

If your script works in permissive mode and stops when you switch back to enforcing mode (with setenforce 1), SELinux is likely the culprit.

The next step is to temporarily disable dontaudit settings by building all policy modules with the -D/--disable_dontaudit option.

semodule -DB

This command will allow you to collect all the failing rules related to your problem. You can then use these logs to create a custom SELinux module.

After you’ve finished creating your module, re-enable the dontaudit silencing feature by rebuilding policy modules:

semodule -B

This prevents your logs from becoming cluttered with too many messages.

Secure Your Dockerized Nginx with Let's Encrypt SSL Certificates

2024-06-27T00:00:00+00:00

Photo by Animesh Srivastava from Pexels.

In this tutorial I will demonstrate how to secure Nginx on Docker using HTTPS, leveraging free certificates from Let’s Encrypt. Let’s Encrypt certificates provide trusted and secure encryption at no cost, although they require renewal every 90 days. Fortunately, this renewal process can be automated with various tools. We will use acme.sh, a versatile Bash script compatible with major platforms. The tutorial will guide you through obtaining Let’s Encrypt certificates on the host system and mounting them as a volume in the Nginx container. Please ensure the following prerequisites are met before proceeding:

Working Docker Engine
Working domain name
A host with ports 80 and 443 that is accessible from the internet

1. Domain validation

First, we need an Nginx instance on Docker that will expose port 80 and have a directory on the host mounted for its web root. This is required by acme.sh for its file-based domain validation. I’ve prepared a Docker Compose file (docker-compose.yml) and an Nginx configuration file (nginx.conf) for this purpose. Git clone the following repository and change into the directory

git clone https://github.com/aburayyanjeffry/nginx-docker-acme.git
cd nginx-docker-acme

In nginx.conf, please note that the lines exposing port 443 and adding SSL certificates are commented because we don’t have the certificate yet. Listening on port 444 and trying to enable SSL without a valid certificate will cause errors and prevent Nginx from starting up. Port 443 is also commented out in docker-compose.yml because Nginx is not exposing port 443 yet, as is the line copying the ssh folder. Let’s start Nginx using Docker Compose:

docker compose up -d

2. Get the acme.sh

The following command will install acme.sh in the current user’s home directory. Make sure to use your email in the command.

curl https://get.acme.sh | sh -s email=jeffry@email.com

Log out and log in again to enable the acme.sh alias for the user. If acme.sh is not working, it’s probably because you missed this step. If the alias is not enabled, the acme.sh script is not defined.

3. Set the CA

Set Let’s Encrypt as the default Certificate Authority.

acme.sh --set-default-ca --server letsencrypt

4. Issue the certificate

Now we’ll proceed with issuing the certificate, a step that involves domain validation. Upon successful validation, the certificate will be issued.

acme.sh --issue -d jeffry.temphost.net -w /home/jeffry/nginx-docker-acme/nginx/html --keylength 4096

Make sure to replace jeffry.temphost.net with your domain and /home/jeffry/nginx-docker-acme/nginx/html with your web root directory. Note that /home/jeffry is the directory where the code was downloaded, making it the working directory. Be sure to update it to reflect your own working directory.

The -d flag specifies the domain, while -w designates the web root directory. This directory will be mounted as Nginx’s web root in Docker, where acme.sh will write the validation file.

We need to know the container name in order to restart it. The container name is the string in the last column from the docker ps output. In this example the container name is nginx-docker-acme-web-1.

[jeffry@docker ~]$ docker ps
CONTAINER ID   IMAGE     COMMAND                  CREATED          STATUS          PORTS                               NAMES
754c055d5b5e   nginx     "/docker-entrypoint...."   16 minutes ago   Up 16 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp   nginx-docker-acme-web-1

5. Install the certificate

Uncomment the port 443 and SSL lines in nginx.conf and docker-compose.yaml. This will enable port 443 for Nginx and will make Docker expose it to the host after a restart it through Docker Compose later.

docker-compose.yaml

services:
  web:
    image: nginx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
      - ./nginx/html:/usr/share/nginx/html
      - ./nginx/ssl:/etc/nginx/ssl

nginx/nginx.conf

server {
    listen 80;
    listen 443 ssl;

    server_name localhost;

    ssl_certificate /etc/nginx/ssl/cert.pem;
    ssl_certificate_key /etc/nginx/ssl/key.pem;

    location / {
        root /usr/share/nginx/html;
        index index.html;
    }
}

Create the ssl directory for Nginx and install the certificates there. This directory will be mounted by Nginx in Docker. Use the container name obtained from the previous steps as the value for the --reloadcmd switch. This is crucial because it will be used to restart Nginx when certificates are updated.

mkdir /home/jeffry/nginx-docker-acme/nginx/ssl
acme.sh --install-cert -d jeffry.temphost.net --key-file /home/jeffry/nginx-docker-acme/nginx/ssl/key.pem --fullchain-file /home/jeffry/nginx-docker-acme/nginx/ssl/cert.pem --reloadcmd "docker exec nginx-docker-acme-web-1 nginx -s reload"

6. Restart the Nginx container

Refresh the Nginx container by stopping and starting it from Docker Compose. You should be able to access Nginx over HTTPS with the Let’s Encrypt certificates.

docker compose down
docker compose up -d

7. Certificate Renewal Mechanism

The certificate will be automatically renewed by the cron job which was installed by acme.sh. You can verify the cron job by running crontab -l .

[jeffry@docker ~]$ crontab -l
13 7 * * * "/home/jeffry/.acme.sh"/acme.sh --cron --home "/home/jeffry/.acme.sh" > /dev/null

This ensures that the renewal process runs regularly and without manual intervention.

Setting up Let’s Encrypt SSL certificates for Nginx in a Docker environment using acme.sh is an easy process that enhances the security of your web applications. By leveraging acme.sh, you automate the certificate issuance and renewal process, ensuring your sites remain secure without manual intervention. Following the steps outlined in this tutorial, you now have a robust setup where Nginx serves your applications over HTTPS, backed by trusted SSL certificates from Let’s Encrypt. Thank you for following this tutorial. Have a great day!

Google Chrome Yum/RPM package update fails on RHEL/CentOS 7

2023-06-08T00:00:00+00:00

Painting by Willgard Krause, Pixabay license

One of our clients uses the Chrome web browser running on their continuous integration server with Jenkins for automated e2e (end-to-end) testing of their website. That server runs Red Hat Enterprise Linux (RHEL) 7—actually the rebuild CentOS 7.

Last month, in May 2023, Google started signing Chrome RPMs with a GnuPG subkey, where they before had signed with the main key. Now yum upgrade fails when trying to update Chrome, giving this error:

warning: /var/cache/yum/x86_64/7/google-chrome/packages/google-chrome-stable-114.0.5735.106-1.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID a3b88b8b: NOKEY
Retrieving key from https://dl.google.com/linux/linux_signing_key.pub


The GPG keys listed for the "google-chrome" repository are already installed but they are not correct for this package.
Check that the correct key URLs are configured for this repository.


 Failing package is: google-chrome-stable-114.0.5735.106-1.x86_64
 GPG Keys are configured as: https://dl.google.com/linux/linux_signing_key.pub

To double-check, we tried to manually verify the signature on the downloaded RPM package with:

# rpm -K /var/cache/yum/x86_64/7/google-chrome/packages/google-chrome-stable-114.0.5735.106-1.x86_64.rpm
/var/cache/yum/x86_64/7/google-chrome/packages/google-chrome-stable-114.0.5735.106-1.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK (MISSING KEYS: (MD5) PGP#a3b88b8b)

That showed it is not just a Yum problem, but affects RPM too.

Sweatin’ to the oldies

Long ago (see reference below), people reported that RPM wasn’t working with GnuPG subkeys for signatures, and Red Hat confirmed this is the case for RHEL 7 and earlier. They added support, but that first appeared in RHEL 8.

RHEL 7 has another year of support left, until end of June 2024. But it was released in 2014 and is so old that apparently Google isn’t testing the RPM packages it produces on RHEL 7 anymore.

Our client is planning to move this system that runs tests with Chrome to Rocky Linux 9, but for the next few months they need it to keep working on CentOS 7.

So to cope, we used scp to copy that RPM file to a RHEL 8 or 9 server, imported the Google signing public key, and used the newer version of rpm there to verify the signature:

# rpm -K google-chrome-stable-114.0.5735.106-1.x86_64.rpm
google-chrome-stable-114.0.5735.106-1.x86_64.rpm: digests signatures OK

Then back on the RHEL 7 server we had no qualms skipping the signature check during upgrade because we had just manually checked it elsewhere:

# rpm -Uvh --nosignature /var/cache/yum/x86_64/7/google-chrome/packages/google-chrome-stable-114.0.5735.106-1.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:google-chrome-stable-114.0.5735.1################################# [ 50%]
Cleaning up / removing...
   2:google-chrome-stable-113.0.5672.6################################# [100%]

Then we ran yum upgrade again to get the rest of that server’s pending package updates. Yum didn’t care about Chrome anymore since we had updated it already.

References

Reddit discussion on this situation—thanks especially to user Jskud
Red Hat Bugzilla #227632 where the problem is confirmed and the future fix announced

Identifying Vulnerabilities in Code Using Horusec

2023-03-28T00:00:00+00:00

Horusec is an open source tool which, by orchestrating other security tools, identifies security flaws and vulnerabilities in source code. It puts all the possible vulnerabilities it finds into a database for analysis.

Currently, Horusec supports C#, Java, Kotlin, Python, Ruby, Go, JavaScript, TypeScript, PHP, Swift, C, Dart, Elixir, shell, Terraform, Kubernetes, nginx, HTML, and JSON. You can see an up-to-date list of supported languages in Horusec’s docs.

It can also be integrated with CI/CD to execute the scan every time a developer creates a pull request or merge request.

Horusec CLI Installation

Requirements: Docker, Git.

The easiest installation method listed in the docs is curling Horusec’s install script and piping it into bash:

curl -fsSL https://raw.githubusercontent.com/ZupIT/horusec/main/deployments/scripts/install.sh | bash -s latest

Be aware that there is risk to piping unseen commands into the shell like this: It can lead to unintended consequences and it is a bad security practice.

If a user blindly pipes the output of a website response to be run by a shell without fully understanding what each command does, they may inadvertently execute malicious code or perform actions that could compromise the security of the system.

To mitigate these risks, one solution is to use a virtual machine specifically set up for code scanning. This virtual machine should have no production data, no secret keys, and no other sensitive information. It should be used only for the purpose of scanning source code and nothing else.

To instead download an executable directly, as well as for instructions for different platforms, see the installation docs.

VS Code Extension

Horusec has a VS Code extension which is helpful for making complete code analysis with a single click.

Usage

Navigate to the application directory and run the generate command to make a configuration file called horusec-config.json which has a set of customisations:

cd /path/to/project
horusec generate

The start command executes the code scan throughout the repository, searching for possible vulnerabilities:

horusec start -p . --output-format=json --json-output-file=<filename>.json

All the detected vulnerabilities are stored in the named JSON output file.

Classification of Vulnerabilities

Horusec may identify (or, as the docs say, accuse) possible vulnerabilities that aren’t vulnerabilities at all. Possible vulnerabilites need to be classified to sort out those that are wrong.

Here are the classification types from Horusec’s docs:

False positive: Vulnerability found is wrong, because it is accused in a test file or it is not a vulnerability in fact, and is safe code.
Accepted Risk: Vulnerability that was accused, but at the moment, you don’t have the option to correct it, so it is classified as an accepted risk to not raise alarms in future runs.
Corrected: Vulnerability that doesn’t exist and can be considered as corrected.
Vulnerability: A possible security problem found and accused by the analysis.

The hashes to be ignored can be added to the configuration file, horusec-config.json:

{
  ...
  "horusecCliFalsePositiveHashes": [
      "f9e5abe187ad4246daa4e9113e0a11a175347e793fbc40acd7663df67b2f89d2",
      "878c35116d043311403a0a2e8a64f2c8d00479a1c23373dcd50372ff35e123c8"
  ],
  "horusecCliRiskAcceptHashes": [
      "5c9af42834ca77233a0e7afc98df317fb0e6041ea69a109754f278331039f844",
      "b494003277bcd7792390f032ba39e9a860da66ddb1ccfcd8a581978eab744561"
  ],
  ...
}

Ignore Files

To entirely ignore a file or certain paths under the directory, add the paths to the configuration file:

{
  ...
  "horusecCliFilesOrPathsToIgnore": [
      "**/tmp/**",
      "**/.vscode/**"
  ],
  ...
}

Code Scanning in EpiTrax

The use of a code scan tool has become an integral part of our work with the EpiTrax disease surveillance system for public health.

By integrating Horusec into our software development process, we have been able to identify potential security vulnerabilities, code quality issues, and other problems before they can cause any significant harm.

Moreover, it has helped us reduce the overall cost and time associated with bug-fixing and maintenance tasks, making our projects more efficient and effective. By automating the scanning process, we can identify problems quickly and accurately, allowing us to prioritize and address the most critical ones first.

By leveraging the power of code scans, we can ensure that our code is of the highest quality, and our projects are as secure and reliable as possible.

Special thanks to my colleagues Kürşat Kutlu Aydemir and Edgar Mlowe for their insights and feedback on this blog post.

ssh-askpass on macOS for SSH agent confirmation

2022-11-23T00:00:00+00:00

Photo by Kristoffer Trolle, CC BY 2.0

At End Point Dev we mostly use SSH keys for authentication when connecting to remote servers and Git services. The majority of the time, the servers we are trying to visit are barred from direct access and require a middle “jump server” instead.

Enabling SSH agent forwarding makes it easier to reuse SSH private keys. It keeps the private keys on our local machine and uses them to authenticate with each server in the chain without entering a password.

However, this approach comes with an inherent risk of the agent being hijacked if one of the servers is compromised. This means a bad guy could use the SSH keys to compromise downstream servers.

In this post, we’ll cover a simple way to protect against SSH agent hijacking. We will see in detail on macOS how to configure a system-wide agent using ssh-askpass to pop up a graphical window to ask for confirmation before using the agent.

How it works

It is strongly recommended to use the -c option on the ssh-add command when adding your SSH keys to the agent in order to protect yourself against SSH agent hijacking.

With this, every time a request is made to utilize the private key stored in the SSH agent, ssh-askpass will display a prompt on your local computer asking you to approve the usage of the key. By doing this, it becomes more difficult for a remote attacker to use the private key without authorization.

If you don’t want to use the ssh-askpass agent confirmation, I recommend using the OpenSSH feature ProxyJump rather than agent forwarding to get an equivalent level of security.

Installing ssh-askpass on macOS

So let’s set this up. The recommended way is with Homebrew:

Install ssh-askpass with Homebrew

$ brew tap theseal/ssh-askpass
$ brew install ssh-askpass

You might see some warnings. Go ahead and proceed with them.

Now we need to start the Homebrew services. Note that this is a brew service, not a regular macOS daemon service.

$ brew services start theseal/ssh-askpass/ssh-askpass
=> Successfully started `ssh-askpass` (label: homebrew.mxcl.ssh-askpass)

Behind the scenes, it just sets the SSH_ASKPASS and SUDO_ASKPASS environment variables and stops ssh-agent, so that the SSH agent can pick up these environment variables when it restarts.

To list the services and make sure it’s started:

$ brew services list | grep ssh-askpass
ssh-askpass started ~/Library/LaunchAgents/homebrew.mxcl.ssh-askpass.plist

Install ssh-askpass with MacPorts

If you prefer MacPorts, do:

$ sudo port install ssh-askpass

Install ssh-askpass from source code

And of course you can install from source if you wish:

$ cp ssh-askpass /usr/local/bin/
$ cp ssh-askpass.plist ~/Library/LaunchAgents/
$ launchctl load -w ~/Library/LaunchAgents/ssh-askpass.plist

Configure the SSH agent with the `ssh-add -c` option

Let’s first verify that the agent is running, then add the private key with the confirmation option:

$ ssh-add -l
The agent has no identities.
$ ssh-add -c .ssh/id_rsa
Enter passphrase for .ssh/id_rsa (will confirm after each use):
Identity added: .ssh/id_rsa (.ssh/id_rsa)
The user must confirm each use of the key

The Identity will get added if you provide the correct passphrase for the key. This can be confirmed by listing the keys again with ssh-add -l.

ssh-askpass agent confirmation

Now let’s log into a remote server.

You will be prompted to confirm the private key’s usage with the pop-up window:

Set up keyboard shortcuts

Since it’s too easy to hit the spacebar and accept a connection, ssh-askpass defaults to the cancel option. We can use keyboard shortcuts to press “OK” by following the below steps.

Go to “System Preferences” and then “Keyboard”.

macOS 10.15 Catalina, 11 Big Sur, 12 Monterey

Go to “Shortcuts” tab
check the option “Use keyboard navigation to move focus between controls”.

macOS 13 Ventura

Turn on the “Keyboard navigation” option.

Now you can press tab ⇥ and then the spacebar to select “OK”.

Enjoy!

Vulnerability Scanning

2022-09-15T00:00:00+00:00

Define Your Terms

A security vulnerability is a flaw or bug that could be exploited by a threat agent/threat actor. According to CrowdStrike, “A threat actor, also known as a malicious actor, is any person or organization that intentionally causes harm in the digital sphere.”

Once a bug or flaw is deemed a vulnerability, it is registered by the MITRE Corporation as a Common Vulnerability or Exposure (CVE) and stored in their CVE database. A CVE is given an identifying number by a CVE Numbering Authority (CNA), for example, Red Hat, Microsoft, and other designated authorities.

Threat levels are quantified by assigning a Common Vulnerability Scoring System (CVSS) score from 0 to 10. CVSS is a free and open standard for evaluating the level of threat to a business or organization maintained by the Forum of Incident Response and Security Teams (FIRST).

The National Institute of Standards and Technology (NIST) is a federal agency housing the National Vulnerability Database (NVD). NIST provides a CVSS calculator. The NIST NVD database synchronizes with the MITRE CVE database. Only the NVD includes CVSS scores.

The Attackers

Real live people spend a lot of time and money trying to break into specific high-value targets, as do bots that clever people have weaponized to attack more cheaply and broadly at all hours of the day.

Main Security Vulnerability Categories

The main information security vulnerability categories are:

Broken Authentication

When security credentials are stolen, attackers can usurp user identities and sessions as if they were the user.
SQL Injection

Attackers can hijack database content by injecting malicious code. It can allow attackers to acquire sensitive data, modify or delete data, impersonate identities, and conduct other nefarious activities.
Cross-site scripting (XSS)

This type of attack inserts malicious code into a website. Its target is the website user, threatening sensitive user information.
Cross-Site Request Forgery (CSRF)

This attack may mislead the user into an action that they would do only unwittingly. For example, social engineering may hoodwink the user into clicking a link in an email or chat to get the user to perform an action of the attacker’s choice. It could be having their browser use JavaScript to submit a form that they don’t even know about.
Security Misconfiguration

A configuration error that can be exploited by attackers. For example:
- Default passwords left in place
- No password strength requirements to prevent users from setting weak passwords that can easily be found in dictionary attacks
- Web server directory listings left enabled, possibly exposing files that shouldn’t be seen
- Unused software modules or plugins left enabled, increasing the attack surface
Software Vulnerabilities

Especially out-of-date software. As more time passes without getting security updates, attackers can examine code for bugs and flaws that could be exploited. Thus, a given piece of software that was secure at one time may be vulnerable six months later or even the next day.

For more on vulnerability categories, see our blog post on the OWASP Top 10. OWASP is the Open Web Application Security Project, which defines itself as “a nonprofit foundation that works to improve the security of software. Through community-led open-source software projects, hundreds of local chapters worldwide, tens of thousands of members, and leading educational and training conferences, the OWASP Foundation is the source for developers and technologists to secure the web.”

Solutions

Applications to scan for vulnerabilities are available. Some are free and/or open source, some are paid, and some have a free version and a paid version.

Vulnerability scanners are automated tools that look into a database of known CVEs for the specific software and library versions running on a given server. They also examine systems for flaw types that could be exploited.

In general, the vulnerabilities fall into three classifications:

System/Network: The application scans for system misconfiguration and network CVEs.
Web: The application scans for SQL Injections, XSS, CSRF, etc.
Software Analysis: Static Application Security Testing (SAST). SAST analyzes source code to find security vulnerabilities. Each SAST has a set of languages it can scan, sometimes one or two, sometimes up to a dozen or more.

In summary, vulnerability scanning is a vital process that spans many aspects of Information Technology (IT). Its solutions are constantly evolving to keep up with new threats, an endless cat-and-mouse game where the cats must be ever adaptive to keep ahead of the ever-evolving malicious mice.

References

Implementing Authentication in ASP.NET Core Web APIs

2022-06-17T00:00:00+00:00

Authentication is a complex space. There are many problem scenarios and many more solutions. When it comes to Web APIs written with ASP.NET Core, there are various fully featured options like Duende IdentityServer or Azure Active Directory. These promise to be “everything but the kitchen sink” solutions which are robust and allow you to deal with many complex requirements.

But what if our requirements dictate that we need something simpler? Do we have to roll out our own from scratch? Or does ASP.NET Core offer smaller, customizable, somewhat independent puzzle pieces that we can put together without having to write all the code ourselves and still have a good amount of control?

Spoiler alert: The answer to that last question is yes. And we’re going to talk about it in this very article.

There is a Table of contents at the end of this post.

Two approaches to authentication: JWT and API Keys

In this article, we’ll take an existing ASP.NET Core Web API and add authentication capabilities to it. Specifically, we’ll support two authentication schemes commonly used for Web APIs: JWT and API Keys. Also, we will use our own database for storage of user accounts and credentials.

The project that we will work with is a simple ASP.NET Web API backed by a Postgres database. It has a few endpoints for CRUDing automotive related data and for calculating values of vehicles based on various aspects of them. You can read all about the process of building it here. I also added a few database integration tests for it here.

You can find it on GitHub. If you’d like to follow along, clone the repository and checkout this commit: cd290c765fcd2c6693008d3dc76fa931098dcaa0. It represents the project as it was before applying all the changes from this article.

You can follow the instructions in the project’s README file in order to get the app up and running.

Managing user accounts with ASP.NET Core Identity

Let’s deal first with the requirement of storing the user accounts in our own database.

Luckily for us, ASP.NET Core provides us with Identity. This is an API that offers a comprehensive solution for authentication. It can connect with the aforementioned Duende IdentityServer for OpenID Connect and OAuth 2.0, supports authentication via third parties like Facebook or Google, and can fully integrate with user interfaces on web apps, complete with scaffolding/autogeneration capabilities too.

Most importantly for us, it supports management of user accounts stored in our own database. It includes data for third party logins, passwords, roles for authorization, email confirmation, access tokens, etc.

That’s a lot of of functionality baked in there. But we don’t need all that, so let’s see how we can take only the bits and pieces that we need in order to fulfill our specific requirements.

Specifically, we want:

Tables in our database for storage of user accounts and passwords.
A programmatic way to create, fetch and validate users using those tables.

It’s actually pretty simple.

Install the necessary NuGet packages

First, we need to install a couple of NuGet packages:

Microsoft.AspNetCore.Identity which contains the core library.
Microsoft.AspNetCore.Identity.EntityFrameworkCore which includes the classes that Identity needs in order to properly interact with Entity Framework, which is what we’re using in our Web API for interfacing with the database.

We can install them by running these two commands from the VehicleQuotes directory:

$ dotnet add package Microsoft.AspNetCore.Identity
$ dotnet add package Microsoft.AspNetCore.Identity.EntityFrameworkCore

That will add the following lines to the VehicleQuotes/VehicleQuotes.csproj project file:

 <Project Sdk="Microsoft.NET.Sdk.Web">
   ...
   <ItemGroup>
     ...
+    <PackageReference Include="Microsoft.AspNetCore.Identity" Version="2.2.0" />
+    <PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="6.0.5" />
     ...
   </ItemGroup>
 </Project>

Update the DbContext to include the Identity tables

Next step is to configure our DbContext class so that it includes the new tables that we need from the Identity library. So let’s go to VehicleQuotes/Data/VehicleQuotesContext.cs and update it to do so.

We need to include these new using statements at the top of the file so that we have access to the classes that we need from the NuGet packages we just installed:

using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Identity.EntityFrameworkCore;

Next, instead of DbContext, the VehicleQuotesContext class should inherit from IdentityUserContext<IdentityUser>.

IdentityUserContext is a class provided by ASP.NET Core Identity that’s designed so that our DbContext can inherit from it and gain user management functionality. Namely, it includes a new DbSet (and consequently, a table) for holding user accounts (aptly named Users), among other things that we won’t need.

IdentityUserContext has a more feature rich counterpart called IdentityDbContext, which also includes DbSets to support roles based authorization. We don’t need all that so we use its simpler cousin. Feel free to explore the source code on GitHub to see all it offers.

The generic type parameter that we give it, IdentityUser, is a class that’s also provided by the Identity library. Its purpose is to serve as a default Entity Type for our user model and, as a consequence, our Users DbSet.

In summary, by having our DbContext class inherit from IdentityUserContext<IdentityUser>, we’re telling Identity that we want it to augment our DbContext (and database) to include the core user management tables and that we want our users table to have the same columns as IdentityUser.

If we wanted to include more columns in our users table, what we would have to do is create a new class, make it inherit from IdentityUser, define any additional fields that we want on it, and use that class as a type parameter to IdentityUserContext. For us for now, the default works just fine. You can learn more about customizing Identity in the official docs.

The change looks like this:

-public class VehicleQuotesContext : DbContext
+public class VehicleQuotesContext : IdentityUserContext<IdentityUser>

Finally, IdentityUserContext has some logic that it needs to run when it is being created. In order to allow it to run that logic, let’s add the following line to our VehicleQuotesContext’s OnModelCreating method:

 protected override void OnModelCreating(ModelBuilder modelBuilder)
 {
+    base.OnModelCreating(modelBuilder);
     // ...
 }

This calls IdentityUserContext’s own OnModelCreating implementation so that it can set itself up properly.

The complete file should be looking like this now:

using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Identity.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;
using VehicleQuotes.Models;

namespace VehicleQuotes
{
    public class VehicleQuotesContext : IdentityUserContext<IdentityUser>
    {
        public VehicleQuotesContext (DbContextOptions<VehicleQuotesContext> options)
            : base(options)
        {
        }

        public DbSet<Make> Makes { get; set; }
        public DbSet<Size> Sizes { get; set; }
        public DbSet<BodyType> BodyTypes { get; set; }

        public DbSet<Model> Models { get; set; }
        public DbSet<ModelStyle> ModelStyles { get; set; }
        public DbSet<ModelStyleYear> ModelStyleYears { get; set; }

        public DbSet<QuoteRule> QuoteRules { get; set; }
        public DbSet<QuoteOverride> QuoteOverides { get; set; }

        public DbSet<Quote> Quotes { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            base.OnModelCreating(modelBuilder);

            modelBuilder.Entity<Size>().HasData(
                new Size { ID = 1, Name = "Subcompact" },
                new Size { ID = 2, Name = "Compact" },
                new Size { ID = 3, Name = "Mid Size" },
                new Size { ID = 5, Name = "Full Size" }
            );

            modelBuilder.Entity<BodyType>().HasData(
                new BodyType { ID = 1, Name = "Coupe" },
                new BodyType { ID = 2, Name = "Sedan" },
                new BodyType { ID = 3, Name = "Hatchback" },
                new BodyType { ID = 4, Name = "Wagon" },
                new BodyType { ID = 5, Name = "Convertible" },
                new BodyType { ID = 6, Name = "SUV" },
                new BodyType { ID = 7, Name = "Truck" },
                new BodyType { ID = 8, Name = "Mini Van" },
                new BodyType { ID = 9, Name = "Roadster" }
            );
        }
    }
}

Applying database changes

The Users DbSet that we added into our data model by inheriting from IdentityUserContext<IdentityUser> will become a table once we create and apply a database migration. That’s what we will do next.

That’s simple in ASP.NET Core. All we need to do is run a command like this:

dotnet ef migrations add AddIdentityTables

That should produce a new migration file named something like 20220605003253_AddIdentityTables.cs. If you explore it, you’ll see how it contains the definitions for a few new database tables. Including the one that we want: AspNetUsers. That’s the one that we will use to store our user account records.

Next, we apply the migration with:

dotnet ef database update

If you now connect to the database, you’ll see the new tables in there.

If you have the database running in a Docker container like we discussed in the beginning of the article, you should be able to connect to it with:

$ psql -h localhost -U vehicle_quotes

Then, to see the tables:

vehicle_quotes=# \dt
                    List of relations
 Schema |         Name          | Type  |     Owner      
--------+-----------------------+-------+----------------
 public | AspNetUserClaims      | table | vehicle_quotes
 public | AspNetUserLogins      | table | vehicle_quotes
 public | AspNetUserTokens      | table | vehicle_quotes
 public | AspNetUsers           | table | vehicle_quotes
 ...
(14 rows)

And that’s pretty much it when it comes to having a sensible storage for user accounts. That was pretty inexpensive, wasn’t it? All we had to do was install some NuGet packages, tweak our existing DbContext, and run some migrations.

The best thing is that we’re not done yet. We can also take advantage of ASP.NET Core Identity to manage users programmatically. Instead of interacting with these tables directly, we will use the service classes provided by Identity to create new users, fetch existing ones and validate their credentials.

Before we can do that though, we must first do some configuration in our app’s Startup class so that said services are properly set up to our liking and are made available to our application.

Configuring the Identity services

We need to add some code to VehicleQuotes/Startup.cs. With it, we configure the Identity system and add a few service classes to ASP.NET Core’s IoC container so that they are available to our app via Dependency Injection.

We need a new using statement:

using Microsoft.AspNetCore.Identity;

And the following code added to the ConfigureServices method:

services
    .AddIdentityCore<IdentityUser>(options => {
        options.SignIn.RequireConfirmedAccount = false;
        options.User.RequireUniqueEmail = true;
        options.Password.RequireDigit = false;
        options.Password.RequiredLength = 6;
        options.Password.RequireNonAlphanumeric = false;
        options.Password.RequireUppercase = false;
        options.Password.RequireLowercase = false;
    })
    .AddEntityFrameworkStores<VehicleQuotesContext>();

The call to AddIdentityCore makes several Identity utility classes available to the application. Among those, UserManager is the only one we will use. We will use it later to… well, manage users. You can also see how we’ve set a few options related to how the user accounts are handled. options.SignIn.RequireConfirmedAccount controls whether new accounts need to be confirmed via email before they are available. With options.User.RequireUniqueEmail, we tell Identity to enforce uniqueness of emails on user accounts. And finally the options.Password.* options configure the password strength requirements.

You can explore all the available options in the official docs.

Then, the call to AddEntityFrameworkStores tells the Identity system that it should use our VehicleQuotesContext for data storage.

Creating users

With that configuration out of the way, we can now write some code to create new user accounts. To keep things simple, we’ll add a new UsersController to our project that will expose a new endpoint that offers that functionality.

Let’s start with this in VehicleQuotes/Controllers/UsersController.cs:

using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Mvc;

namespace VehicleQuotes.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class UsersController : ControllerBase
    {
        private readonly UserManager<IdentityUser> _userManager;

        public UsersController(
            UserManager<IdentityUser> userManager
        ) {
            _userManager = userManager;
        }
    }
}

As you can see, this controller defines a dependency on UserManager<IdentityUser>, an instance of which is injected via the constructor. This is one of the classes made available to us when we configured the Identity core services in our app’s Startup. We will use it to create new user records.

Before that though, we need to define a new class that encapsulates the payload for a request to our endpoint. When it comes to creating user accounts, all we need is a username, a password, and an email. As such, we add the following class in a new VehicleQuotes/ResourceModels/User.cs file:

using System.ComponentModel.DataAnnotations;

namespace VehicleQuotes.ResourceModels
{
    public class User
    {
        [Required]
        public string UserName { get; set; }
        [Required]
        public string Password { get; set; }
        [Required]
        public string Email { get; set; }
    }
}

Now, we shall use this type as the parameter for a new PostUser action method in the UserController which will expose the new user account creation endpoint in our API. The method looks like this:

// POST: api/Users
[HttpPost]
public async Task<ActionResult<User>> PostUser(User user)
{
    if (!ModelState.IsValid)
    {
        return BadRequest(ModelState);
    }

    var result = await _userManager.CreateAsync(
        new IdentityUser() { UserName = user.UserName, Email = user.Email },
        user.Password
    );

    if (!result.Succeeded)
    {
        return BadRequest(result.Errors);
    }

    user.Password = null;
    return Created("", user);
}

Be sure to also add the following using statements:

using System.Threading.Tasks;
using VehicleQuotes.ResourceModels;

System.Threading.Tasks allows us to reference the class Task<> which is the return type of the new PostUser method. VehicleQuotes.ResourceModels is where our new User class lives.

Thanks to the instance of UserManager<IdentityUser> that we’re holding onto, this method is very straightforward. The most interesting portion is this:

var result = await _userManager.CreateAsync(
    new IdentityUser() { UserName = user.UserName, Email = user.Email },
    user.Password
);

Here we’re newing up an IdentityUser instance using the given request parameters and passing it on to UserManager<IdentityUser>’s CreateAsync method, along with the password that was also given in the incoming request. This puts the Identity system to work for us and properly create a new user account.

Then, we can inspect its return value (which we capture in the result variable) to determine if the operation was successful. That way we can respond appropriately to our API’s caller.

Finally, with…

user.Password = null;
return Created("", user);

we return the data representing the newly created user account but we’re discreet and make sure not to include the password.

With that, we can test our API. Fire it up with dotnet run (or dotnet watch!) and send a POST request to our new endpoint at http://0.0.0.0:5000/api/Users with a payload like this:

{
  "userName": "newuser000",
  "email": "newuser000@endpointdev.com",
  "password": "password"
}

I just tried it in Postman and this is what it looked like:

Feel free to test it out further. Try repeated emails or usernames. Try passwords that don’t meet the criteria we defined when configuring Identity in the app’s Startup class. It all works as you’d expect.

You can inspect the database and see the newly created record too:

vehicle_quotes=# select id, user_name, email, password_hash from "AspNetUsers";
-[ RECORD 1 ]-+--------------------------------------
id            | aaad07b4-f109-4255-8caa-185fd7694c72
user_name     | newuser000
email         | newuser000@endpointdev.com
password_hash | AQAAAAEAACcQAAAAEJJTV7M2Ejqd3K3iC...

And there it is, that’s our record. With a hashed password and everything.

Now let’s see what would an endpoint to fetch users look like:

Fetching users

UserManager<IdentityUser> offers a FindByNameAsync method that we can use to retrieve users by name. We can add a new endpoint that leverages it like so:

// GET: api/Users/username
[HttpGet("{username}")]
public async Task<ActionResult<User>> GetUser(string username)
{
    IdentityUser user = await _userManager.FindByNameAsync(username);

    if (user == null)
    {
        return NotFound();
    }

    return new User
    {
        UserName = user.UserName,
        Email = user.Email
    };
}

This is even more straightforward than the previous one. We just call the FindByNameAsync method by giving it the username of the account we want, make sure that we actually found it, and then return the data.

For the response, we use the same User class that we created to represent the input for the creation endpoint. If we added more fields to the user profile, we could include them here. Alas, we only have username and email for now.

Restart the app and we can now make a request like this:

Pretty neat, huh? Try a username that does not exist and you should see the API respond with a 404.

One quick improvement that we can do before we move on: let’s change PostUser’s return statement to this:
return CreatedAtAction("GetUser", new { username = user.UserName }, user);
All that does is include a Location header on the POST Users endpoint’s response that contains the URL for the newly created user. That’s just being a good citizen.

Implementing JWT Bearer Token authentication

Now that we have the user management capabilities that we need, let’s implement some actual authentication. We will start by adding JWT-based authentication to our API.

The strategy that we will use is to create a new API endpoint that clients can POST credentials to and that will respond to them with a fresh, short-lived token. They will then be able to use that token for subsequent requests by including it via headers.

We will pick a random endpoint to secure, just to serve as an example. That is, an endpoint that requires authentication in order to be accessed. GET api/BodyTypes is a good candidate. It is defined in VehicleQuotes/Controllers/BodyTypesController.cs’s GetBodyTypes action method. Feel free to test it out.

Creating tokens

Let’s start by creating the new endpoint that clients will call to obtain the auth tokens. In order to do so, we need a few things:

A class that represents incoming request data.
A class that represents outgoing response data.
A class that can generate tokens for a given user.
A new action method in UsersController that does the work.

The request data structure

To deal with step one, let’s define a new class in VehicleQuotes/ResourceModels/AuthenticationRequest.cs that looks like this:

using System.ComponentModel.DataAnnotations;

namespace VehicleQuotes.ResourceModels
{
    public class AuthenticationRequest
    {
        [Required]
        public string UserName { get; set; }
        [Required]
        public string Password { get; set; }
    }
}

All users need to authenticate is provide a set of credentials: username and password. So we have this class that contains fields for both and that will be used as input for our endpoint.

The response data structure

Next, we need to define a class to represent that endpoint’s response. I’ve added the following class in VehicleQuotes/ResourceModels/AuthenticationResponse.cs:

using System;

namespace VehicleQuotes.ResourceModels
{
    public class AuthenticationResponse
    {
        public string Token { get; set; }
        public DateTime Expiration { get; set; }
    }
}

Just a simple data structure containing the token itself and a date letting clients know when they can expect it to expire.

The class that creates JWTs

Step 3 is creating a class that can produce the tokens. This is the most interesting part in terms of complexity. Before we can do that though, let’s add the following configurations to VehicleQuotes/appsettings.json:

  "Jwt": {
    "Key": "this is the secret key for the jwt, it must be kept secure",
    "Issuer": "vehiclequotes.endpointdev.com",
    "Audience": "vehiclequotes.endpointdev.com",
    "Subject": "JWT for vehiclequotes.endpointdev.com"
  },

These are values that we’ll need when creating the tokens. We’ll go over the purpose of each them as they come up as we continue writing our code.

Here we’ll gloss over some details on the inner workings of JWTs as a standard. You can learn more about them at jwt.io.

For now, let’s add the following class in VehicleQuotes/Services/JwtService.cs:

using System;
using System.IdentityModel.Tokens.Jwt;
using System.Security.Claims;
using System.Text;
using Microsoft.AspNetCore.Identity;
using Microsoft.Extensions.Configuration;
using Microsoft.IdentityModel.Tokens;
using VehicleQuotes.ResourceModels;

namespace VehicleQuotes.Services
{
    public class JwtService
    {
        private const int EXPIRATION_MINUTES = 1;

        private readonly IConfiguration _configuration;

        public JwtService(IConfiguration configuration)
        {
            _configuration = configuration;
        }

        public AuthenticationResponse CreateToken(IdentityUser user)
        {
            var expiration = DateTime.UtcNow.AddMinutes(EXPIRATION_MINUTES);

            var token = CreateJwtToken(
                CreateClaims(user),
                CreateSigningCredentials(),
                expiration
            );

            var tokenHandler = new JwtSecurityTokenHandler();

            return new AuthenticationResponse {
                Token = tokenHandler.WriteToken(token),
                Expiration = expiration
            };
        }

        private JwtSecurityToken CreateJwtToken(Claim[] claims, SigningCredentials credentials, DateTime expiration) =>
            new JwtSecurityToken(
                _configuration["Jwt:Issuer"],
                _configuration["Jwt:Audience"],
                claims,
                expires: expiration,
                signingCredentials: credentials
            );

        private Claim[] CreateClaims(IdentityUser user) =>
            new[] {
                new Claim(JwtRegisteredClaimNames.Sub, _configuration["Jwt:Subject"]),
                new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()),
                new Claim(JwtRegisteredClaimNames.Iat, DateTime.UtcNow.ToString()),
                new Claim(ClaimTypes.NameIdentifier, user.Id),
                new Claim(ClaimTypes.Name, user.UserName),
                new Claim(ClaimTypes.Email, user.Email)
            };

        private SigningCredentials CreateSigningCredentials() =>
            new SigningCredentials(
                new SymmetricSecurityKey(
                    Encoding.UTF8.GetBytes(_configuration["Jwt:Key"])
                ),
                SecurityAlgorithms.HmacSha256
            );
    }
}

The CreateToken method is the element that’s most worth discussing here. It receives an instance of IdentityUser as a parameter (which, remember, is the entity class that represents our user accounts), uses it to construct a JWT, and returns it within a AuthenticationResponse object (which is what we decided that our endpoint would return). To do so, we use various classes built into .NET.

The main class that represents the JWT is JwtSecurityToken. We new up one of those in the CreateJwtToken method with this call:

new JwtSecurityToken (
    _configuration["Jwt:Issuer"],
    _configuration["Jwt:Audience"],
    claims,
    expires: expiration,
    signingCredentials: credentials
);

“Issuer” and “Audience” are two important values for how JWTs work. They specify which entity is creating the token (i.e. the “Issuer”) and which entity is the token intended for (i.e. the “Audience”). We use the IConfiguration instance that we got as a dependency to fetch their values from the VehicleQuotes/appsettings.json file.

The “Issuer” and “Audience” parameters in JwtSecurityToken’s constructor correspond to JWT claims iss and aud, respectively. You can learn more about them and other claims in the RFC.

The next parameter that JwtSecurityToken’s constructor needs is the claims array. In JWT terms, a claim is essentially a statement about the entity for which the token is generated, some data that identifies it. For example, if we’re generating a token for a user, what you would expect to see in such a token’s claims are things like username, email, and any other non-secret profile info.

In our case, as you can see in the CreateClaims method, we add a number of claims:

new[] {
    new Claim(JwtRegisteredClaimNames.Sub, _configuration["Jwt:Subject"]),
    new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()),
    new Claim(JwtRegisteredClaimNames.Iat, DateTime.UtcNow.ToString()),
    new Claim(ClaimTypes.NameIdentifier, user.Id),
    new Claim(ClaimTypes.Name, user.UserName),
    new Claim(ClaimTypes.Email, user.Email)
};

Along with the user id, name and email, we also add sub, jti and iat claims. These are standardized claims of which you can learn more about in the RFC. The data that we put in here will make its way into the encoded token that our API caller eventually sees. We’ll see that later.

The next parameter to JwtSecurityToken is the expiration date of the token. Here we are setting it to just one minute in the future to comply with our original requirement that the token should be short-lived so that it only allows a handful of requests over a short period of time.

Finally there’s the signingCredentials parameter which tells the JwtSecurityToken how to cryptographically sign the token. As you can see in the code:

new SigningCredentials(
    new SymmetricSecurityKey(
        Encoding.UTF8.GetBytes(_configuration["Jwt:Key"])
    ),
    SecurityAlgorithms.HmacSha256
);

That’s a SigningCredentials instance that we create using the “key” that we have configured in VehicleQuotes/appsettings.json, along with the algorithm to use to produce it.

And that’s about it. There isn’t much else to this class. It is somewhat involved but that’s just how JWTs are created in .NET.

The action method that puts it all together

Now all we need to do is actually create an endpoint that exposes this functionality to clients. Since we have the core logic encapsulated in our JwtService class, the actual action method is simple. Here are the changes that we need to make in order to implement it:

First, on VehicleQuotes/Controllers/UsersController.cs, we add a new using statement so that we can reference our new JwtService class:

using VehicleQuotes.Services;

Next, we declare a new parameter of type JwtService in the controller’s constructor so that we signal to ASP.NET Core’s Dependency Injection subsystem that we want to use one of those. We also hold onto it via a new instance variable called _jwtService:

 // ...
 public class UsersController : ControllerBase
 {
     private readonly UserManager<IdentityUser> _userManager;
+    private readonly JwtService _jwtService;
 
     public UsersController(
         UserManager<IdentityUser> userManager,
+        JwtService jwtService
     ) {
         _userManager = userManager;
+        _jwtService = jwtService;
     }
     // ...
 }

We also need to tell ASP.NET Core that JwtService should be available for Dependency Injection. To do so, we can add this line in VehicleQuotes/Startup.cs’s ConfigureServices method:

services.AddScoped<Services.JwtService>();

Note that this way of defining dependencies is not recommended and only done this way here to keep things simple for an illustrative app that’s never going to run in production. Ideally what you want to do here is define the dependency as an abstraction (i.e. an interface) and a concrete implementation that fulfills it. For example:
services.AddScoped<ITokenCreationService, JwtService>();
This way, classes that depend on this service can reference the interface, not the concrete type. In doing that, they adhere to the Dependency Inversion principle and become more easily testable because they allow mocks to be provided as dependencies.

Finally, we write the actual action method:

// POST: api/Users/BearerToken
[HttpPost("BearerToken")]
public async Task<ActionResult<AuthenticationResponse>> CreateBearerToken(AuthenticationRequest request)
{
    if (!ModelState.IsValid)
    {
        return BadRequest("Bad credentials");
    }

    var user = await _userManager.FindByNameAsync(request.UserName);

    if (user == null)
    {
        return BadRequest("Bad credentials");
    }

    var isPasswordValid = await _userManager.CheckPasswordAsync(user, request.Password);

    if (!isPasswordValid)
    {
        return BadRequest("Bad credentials");
    }

    var token = _jwtService.CreateToken(user);

    return Ok(token);
}

Here too we’re leveraging the UserManager instance that ASP.NET Core Identity so graciously provided us with. In summary, we find the user account by name using the incoming request data, check if the given password is correct, then ask our JwtService to create a token for this user, and finally return it wrapped in a 200 response.

If you hit that endpoint with a POST and a set of existing credentials, you should see something like this:

If you take that big string that came back in the "token" field in the response JSON, and paste it in jwt.io’s token decoder, you should be able to see all the claims that we added to the token:

Notice that the keywords here are “Encoded” and “Decoded”. The claims that we put in our JWTs are not protected in any way. As such, we should never put secrets in there.

Securing an endpoint with JWT authentication

Now that we have a way for obtaining tokens, let’s see how we can actually use them to gain access to some resources. To demonstrate that, let’s secure an endpoint in a way that it denies unauthenticated requests.

Applying the Authorize attribute

First we need to signal ASP.NET Core that the endpoint requires auth. We do that by annotating the corresponding action method with the Authorize attribute. We’ve already decided that we were going to use VehicleQuotes/Controllers/BodyTypesController.cs’s GetBodyTypes method as a guinea pig. So, let’s go into that file and add the following using statement:

using Microsoft.AspNetCore.Authorization;

That will allow us access to the attribute, which we can apply to the action method like so:

 // GET: api/BodyTypes
+ [Authorize]
  [HttpGet]
  public async Task<ActionResult<IEnumerable<BodyType>>> GetBodyTypes()
  {
      return await _context.BodyTypes.ToListAsync();
  }

With this, we’ve told ASP.NET Core that we want it to require auth for this endpoint.

Enabling and configuring the JWT authentication

Now, we need to tell it how to actually perform the check. To do that, we need to install the Microsoft.AspNetCore.Authentication.JwtBearer NuGet package. We can do that from the VehicleQuotes directory with the following command:

$ dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer

Once we have that installed, we get access to additional services which we can use to configure the “JwtBearer” authentication scheme. As usual, we do the configuration in VehicleQuotes/Startup.cs’s. Here’s what we need to do:

Add a few new using statements:

using Microsoft.AspNetCore.Authentication.JwtBearer;
using Microsoft.IdentityModel.Tokens;
using System.Text;

Add the following code at the end of the ConfigureServices method:

services
    .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.TokenValidationParameters = new TokenValidationParameters()
        {
            ValidateIssuer = true,
            ValidateAudience = true,
            ValidateLifetime = true,
            ValidateIssuerSigningKey = true,
            ValidAudience = Configuration["Jwt:Audience"],
            ValidIssuer = Configuration["Jwt:Issuer"],
            IssuerSigningKey = new SymmetricSecurityKey(
                Encoding.UTF8.GetBytes(Configuration["Jwt:Key"])
            )
        };
    });

The call to AddAuthentication includes all the internal core service classes that are needed to do authentication in our app.

The JwtBearerDefaults.AuthenticationScheme parameter that we give it is the name of the authentication scheme to use as the default. More on that later. For now, know that ASP.NET Core supports multiple authentication schemes to be used at the same time. In fact, our own plan here is to eventually support two auth schemes: JWTs and API Keys. We’re starting with JWT first and as such, that’s the one we specify as the default.

The call to AddJwtBearer configures the JWT authentication scheme. That is, it allows the app to perform authentication checks based on an incoming JWT token.

The most important part of the configuration is the values that we are passing to TokenValidationParameters. As you can see, we are able to specify which aspects of the incoming JWTs to validate. You can see all available options for TokenValidationParameters in the official documentation. Here we’ve chosen to validate obvious things like the issuer, audience, and signing key.

The last configuration step is to add this line right before app.UseAuthorization(); in the Configure method:

app.UseAuthentication();

That will enable the authentication middleware. That way the framework will perform authentication checks as part of the request processing pipeline. In other words, actually put all the configuration that we’ve done to good use.

Alright, now that we have all the configuration ready and have secured an endpoint, let’s try hitting it with a GET on api/BodyTypes and see what happens:

Ok! So far so good. We made an unauthenticated request into an endpoint that requires authentication and as a result we got a 401 back. That’s just what we wanted.

Now, let’s get a token by POSTing to api/Users/BearerToken:

We can copy that token and include as a header in the GET request to api/BodyTypes. The header key should be Authorization and the value should be Bearer <our token>. In Postman, we can setup a similar request if we choose the “Authorization” tab, select “Bearer Token” in the “Type” drop-down list and paste the token in the “Token” text box.

Do that, and you should now see a 200 response from the endpoint:

Neat! Now that we passed a valid token, it wants to talk to us again. Let a few minutes pass, enough for the token to expire and try the request again to see how it’s 401’ing again.

Implementing API Key authentication

Now let’s change gears from JWT and implement an alternate authentication strategy in our Web API: API Keys. Authentication with API Keys is fairly common in the web service world.

The core idea of API Keys is that the API provider (in this case, us) produces a secret string that is given to the clients for safekeeping. The clients then can use that key to access the API and make as many requests as they want. So it’s essentially just a glorified password.

The main functional differences when it comes to JWT as we have implemented it is that the API Keys won’t expire and that we will store them in our database. Then, when processing incoming requests, as part of the authentication step, our app will check if the given API Key exists in our internal records. If it does, then the request is allowed to go through, if it does not, then a 401 is sent back.

This is going to be interesting because ASP.NET Core does not include an authentication scheme out of the box for this kind of behavior. For JWT, we were able to use the Microsoft.AspNetCore.Authentication.JwtBearer package. For this we have to get our hands dirty and actually implement a custom authentication handler that will run the logic that we need.

Let’s get started.

Creating API Keys

Similarly to when we implemented JWTs, we’ll start by creating the new endpoint that clients will call to obtain their keys. In order to make it work, we’ll need:

A new model and database table to store the keys
A service class to create the keys.
A new action method in UsersController that does the work.

The API Key model and database table

For storing API Keys, we just need a simple table that is linked to users via a one-to-many relationship and includes a name and a value. Using Entity Framework, our model could look like this in VehicleQuotes/Models/UserApiKey.cs:

using System.ComponentModel.DataAnnotations;
using System.Text.Json.Serialization;
using Microsoft.AspNetCore.Identity;
using Microsoft.EntityFrameworkCore;

namespace VehicleQuotes.Models
{
    [Index(nameof(Value), IsUnique = true)]
    public class UserApiKey
    {
        [JsonIgnore]
        public int ID { get; set; }

        [Required]
        public string Value { get; set; }

        [JsonIgnore]
        [Required]
        public string UserID { get; set; }

        [JsonIgnore]
        public IdentityUser User { get; set; }
    }
}

Easy enough. We express the relationship between the “keys” and “users” tables via the User navigation property and the UserID field. We also define a unique index on the Value field and annotate some fields with Required because we don’t want them to allow null. Interestingly, we also annotated some of the fields with the JsonIgnore attribute so that when we include objects of this type in API responses (which, as you’ll see, we will), those fields are not included.

We also need to add a new DbSet on our DbContext class at VehicleQuotes/Data/VehicleQuotesContext.cs so that Entity Framework picks it up as part of the data model:

public DbSet<UserApiKey> UserApiKeys { get; set; }

With that, let’s now create and run a migration so that the new table can be added to the database. All it takes is running this to create the migration:

dotnet ef migrations add AddUserApiKeysTable

And this to apply it:

dotnet ef database update

Now the new table should appear in the database:

vehicle_quotes=# \dt
                    List of relations
 Schema |         Name          | Type  |     Owner      
--------+-----------------------+-------+----------------
...
 public | user_api_keys         | table | vehicle_quotes
 (15 rows)

The class that creates API Keys

The next step is very straightforward. All we need is some logic that, when given a user account (in the form of an instance of IdentityUser), can generate a new API Key, insert it in the database, and return it. The following unremarkable class, defined in VehicleQuotes/Services/ApiKeyService.cs, encapsulates said logic:

using System;
using Microsoft.AspNetCore.Identity;
using VehicleQuotes.Models;

namespace VehicleQuotes.Services
{
    public class ApiKeyService
    {
        private readonly VehicleQuotesContext _context;

        public ApiKeyService(VehicleQuotesContext context)
        {
            _context = context;
        }

        public UserApiKey CreateApiKey(IdentityUser user)
        {
            var newApiKey = new UserApiKey
            {
                User = user,
                Value = GenerateApiKeyValue()
            };

            _context.UserApiKeys.Add(newApiKey);

            _context.SaveChanges();

            return newApiKey;
        }

        private string GenerateApiKeyValue() =>
            $"{Guid.NewGuid().ToString()}-{Guid.NewGuid().ToString()}";
    }
}

As you can see, this class’s single public method, aptly named CreateApiKey, just takes the given user and creates a new UserApiKey that’s associated with it and saves it into the database. The key values themselves are just two GUIDs concatenated.

This is an admittedly simplistic and not-all-that-secure method for generating the keys themselves. For simplicity, we’ve gone with this. In a production app however, it’d be better to use a true cryptographically secure string. Here’s an article that explains how to do that with .NET.

Naturally, this class needs to be used elsewhere in the application, so let’s make it available for Dependency Injection by adding the following line to VehicleQuotes/Startup.cs’s ConfigureServices method.

services.AddScoped<Services.ApiKeyService>();

The action method that puts it all together

Now let’s finally write a new action method on VehicleQuotes/Controllers/UsersController.cs so that there’s an endpoint that clients can call to get API Keys. Before adding the action method though, as usual, we need to declare the following dependency as a constructor parameter and hold it in an instance variable:

 public class UsersController : ControllerBase
 {
     private readonly UserManager<IdentityUser> _userManager;
     private readonly JwtService _jwtService;
+    private readonly ApiKeyService _apiKeyService;
 
     public UsersController(
         UserManager<IdentityUser> userManager,
         JwtService jwtService,
+        ApiKeyService apiKeyService 
     ) {
         _userManager = userManager;
         _jwtService = jwtService;
+        _apiKeyService = apiKeyService;
     }}

The action method itself would look like this:

// POST: api/Users/ApiKey
[HttpPost("ApiKey")]
public async Task<ActionResult> CreateApiKey(AuthenticationRequest request)
{
    if (!ModelState.IsValid)
    {
        return BadRequest(ModelState);
    }

    var user = await _userManager.FindByNameAsync(request.UserName);

    if (user == null)
    {
        return BadRequest("Bad credentials");
    }

    var isPasswordValid = await _userManager.CheckPasswordAsync(user, request.Password);

    if (!isPasswordValid)
    {
        return BadRequest("Bad credentials");
    }

    var token = _apiKeyService.CreateApiKey(user);

    return Ok(token);
}

This method is very similar to the other one that generates JWTs. Painfully so. In fact the only difference is that this one calls the ApiKeyService’s CreateApiKey method instead of JwtService’s CreateToken, for obvious reasons.

These two are ripe for some refactoring to eliminate duplication but we won’t do that here. Take a look at the finished project on GitHub to see a neat refactoring option.

Anyway, you can now try a POST into api/Users/ApiKey, pass it a valid set of credentials, and you should see it respond with a brand new API Key:

The key can also be found in the database:

vehicle_quotes=# select id, SUBSTRING(value,1,13), user_id from user_api_keys;
 id |   substring   |               user_id                
----+---------------+--------------------------------------
  1 | efd2133b-cc4a | aaad07b4-f109-4255-8caa-185fd7694c72
(1 row)

Securing an endpoint with API Key authentication

Now that users have a way of getting API Keys, we can start securing endpoints with that authentication scheme. In order to acomplish that, we need to take three steps:

Implement a custom authentication handler that runs the authentication logic.
Configure the new authentication handler, plugging it into ASP.NET Core’s auth mechanisms.
Secure certain endpoints via the Authorization attribute, specifying the scheme that we want to use to do so.

We’ll do that next.

Implementing the API Key authentication handler

Like we’ve touched on before, the “authentication handler” is a class that can plug into ASP.NET Core’s authentication middleware and can run the authentication logic for a given authentication scheme. In practical terms, it’s a class that inherits from Microsoft.AspNetCore.Authentication.AuthenticationHandler and implements the HandleAuthenticateAsync method. Ours will live in VehicleQuotes/Authentication/ApiKey/ApiKeyAuthenticationHandler.cs and it looks like this:

using System.Security.Claims;
using System.Text.Encodings.Web;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Identity;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace VehicleQuotes.Authentication.ApiKey
{
    class ApiKeyAuthenticationHandler : AuthenticationHandler<AuthenticationSchemeOptions>
    {
        private const string API_KEY_HEADER = "Api-Key";

        private readonly VehicleQuotesContext _context;

        public ApiKeyAuthenticationHandler(
            IOptionsMonitor<AuthenticationSchemeOptions> options,
            ILoggerFactory logger,
            UrlEncoder encoder,
            ISystemClock clock,
            VehicleQuotesContext context
        ) : base(options, logger, encoder, clock)
        {
            _context = context;
        }

        protected override async Task<AuthenticateResult> HandleAuthenticateAsync()
        {
            if (!Request.Headers.ContainsKey(API_KEY_HEADER))
            {
                return AuthenticateResult.Fail("Header Not Found.");
            }

            string apiKeyToValidate = Request.Headers[API_KEY_HEADER];

            var apiKey = await _context.UserApiKeys
                .Include(uak => uak.User)
                .SingleOrDefaultAsync(uak => uak.Value == apiKeyToValidate);

            if (apiKey == null)
            {
                return AuthenticateResult.Fail("Invalid key.");
            }

            return AuthenticateResult.Success(CreateTicket(apiKey.User));
        }

        private AuthenticationTicket CreateTicket(IdentityUser user)
        {
            var claims = new[] {
                new Claim(ClaimTypes.NameIdentifier, user.Id),
                new Claim(ClaimTypes.Name, user.UserName),
                new Claim(ClaimTypes.Email, user.Email)
            };

            var identity = new ClaimsIdentity(claims, Scheme.Name);
            var principal = new ClaimsPrincipal(identity);
            var ticket = new AuthenticationTicket(principal, Scheme.Name);

            return ticket;
        }
    }
}

The first thing to note is how we’ve defined the base class: AuthenticationHandler<AuthenticationSchemeOptions>. AuthenticationHandler is a generic, and it allows us to control the type of options object that it accepts. In this case, we’ve used AuthenticationSchemeOptions, which is just a default one already provided by the framework. This is because we don’t really need any custom options.

If we did, then we would define a new class that inherits from AuthenticationSchemeOptions, give it the new fields that we need, and use that as a generic time parameter instead. We would then be able to set values for these new fields during configuration in the app’s Startup class. Just like we’ve done with the “Jwt Bearer” auth scheme via the AddJwtBearer method.

The next point of interest in this class is its constructor. The key aspect of it is that it calls base and passes it a series of parameters that it itself gets. This is because AuthenticationHandler does have some constructor logic that needs to be run for things to work properly, so we make sure to do that.

Finally, there’s the HandleAuthenticateAsync method which does the actual authentication. This method inspects the incoming request looking for an “Api-Key” header. If it finds it, it takes its value and makes a query into the database to try and find a record in the user_api_keys table that matches the incoming value. If it finds that, it creates an AuthenticationTicket which contains the identity of the user that the key belongs to, and returns it wrapped in a AuthenticateResult.Success. That return value signals ASP.NET Core’s authentication middleware that the request is authentic. Otherwise, it returns AuthenticateResult.Fail, which prompts ASP.NET Core to halt the request and return a 401.

Something interesting to note is how the AuthenticationTicket, similarly to the JWT Bearer scheme’s JwtSecurityToken, also includes an array of “claims” that serve to identify the user that has been authenticated. The “claims” concept is central to how auth works in ASP.NET Core.

Enabling and configuring the API Key authentication

Now we need to tell the framework that we want it to use a new authentication scheme for our app, spearheaded by the custom authentication handler that we just wrote. There are two steps to make that happen.

First, we configure our new scheme in the app’s Startup class. For that, we need these two new using statements:

using Microsoft.AspNetCore.Authentication;
using VehicleQuotes.Authentication.ApiKey;

Then we add this right after the AddJwtBearer call in the ConfigureServices method:

 public void ConfigureServices(IServiceCollection services)
 {
     // ...
     services
         .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
         .AddJwtBearer(options =>
         {
             options.TokenValidationParameters = new TokenValidationParameters()
             {
                 ValidateIssuer = true,
                 ValidateAudience = true,
                 ValidateLifetime = true,
                 ValidateIssuerSigningKey = true,
                 ValidAudience = Configuration["Jwt:Audience"],
                 ValidIssuer = Configuration["Jwt:Issuer"],
                 IssuerSigningKey = new SymmetricSecurityKey(
                     Encoding.UTF8.GetBytes(Configuration["Jwt:Key"])
                 )
             };
         })
+        .AddScheme<AuthenticationSchemeOptions, ApiKeyAuthenticationHandler>(
+            "ApiKey",
+            options => { }
+        );
 }

A simple configuration as far as auth schemes go. We specify both the types of the authentication handler and the options object that it accepts via the generic type parameters to the AddScheme method. "ApiKey" is just the name we’ve given our custom auth scheme. It can be anything as long as it’s not “Bearer”, which is the name of the Jwt Bearer scheme (which is the value returned by JwtBearerDefaults.AuthenticationScheme). We’ll refer back to it later. Finally, since we have no special options to give it, we specify a lambda that does nothing with its options.

Updating the Authorize attribute to use our two schemes

Now that the auth scheme is configured, we need to use it to secure an endpoint. We can use the same that we used for JWT: POST api/BodyTypes, defined in BodyTypesController’s GetBodyTypes action method. Remember though, that we wanted to have both auth schemes (JWT and API Key) work on the endpoint. For that, the Authorize attribute allows a comma separated string of scheme names as a parameter. So, we can get both our configured schemes working if we update the attribute like this:

[Authorize(AuthenticationSchemes = $"{JwtBearerDefaults.AuthenticationScheme},ApiKey")]
public async Task<ActionResult<IEnumerable<BodyType>>> GetBodyTypes()
{
    return await _context.BodyTypes.ToListAsync();
}

JwtBearerDefaults.AuthenticationScheme contains the name of the JWT Bearer auth scheme. Next to it, we just put “ApiKey”, which is the name we have given our new custom one.

It’d be nice to put that “ApiKey” string somewhere safe similar to how the “Jwt Bearer” scheme has it in a constant. Take a look at the finished project on GitHub to see one way to organize that and also how to make the Startup configuration a little less verbose by using extension methods.

And finally, let’s test our endpoint and marvel at the fruit of our work.

Make sure to create a new API Key by POSTing to api/Users/ApiKey, and copy the “value” from the response. We can use it as the value for the Api-Key header a GET request to api/BodyTypes. In Postman, we can do that by choosing the “Authorization” tab, selecting “API Key” in the “Type” drop-down list, writing “Api-Key” in the “Key” text box, and putting our key in the “Value” text box.

With that, we can make the request and see how the endpoint now allows authentication via API Keys:

Of course, requests authenticated via JWT will also work for this endpoint.

That’s all

And there you have it! We’ve updated an existing ASP.NET Core Web API application so that it supports authentication using two strategies: JWT and API Keys. We leveraged the Identity libraries to securely store and manage user accounts. We used ASP.NET Core’s built-in authentication capabilities to enable JWT generation and usage. For API Keys, the framework didn’t provide an implementation out of the box. However, it proved to be extensible enough so that we could implement our own.

Using pgTAP to automate database testing

2022-03-16T00:00:00+00:00

Photo from PxHere

Recently I started learning to tune pianos. There are many techniques and variations, but the traditional method, and the one apparently most accepted by ardent piano tuning purists, involves tuning one note to a reference, tuning several other notes in relation to the first, and testing the results by listening closely to different combinations of notes.

The tuner adjusts each new note in relation to several previously tuned notes. Physics being what it is, no piano can play all its tones perfectly, and one of the tricks of it all is adjusting each note to minimize audible imperfections. The tuner achieves this with an exacting series of musical intervals tested against each other.

Databases need tests too

One of our customers needed to add security policies to their PostgreSQL database, to limit data visibility for certain new users. This can quickly become complicated and ticklish, ensuring that the rules work properly for the affected users while leaving other users unmolested.

This struck me as an excellent opportunity to create some unit tests, not that there’s any short supply of good opportunities to add unit tests! This is not just because it helps prove that these security policies really do work properly, but because (confession time) I recently did a similar project for a different database without the help of unit tests, and it wasn’t much fun.

So this seemed like a good time to use pgTAP, a set of database functions designed to allow writing unit tests within the database itself. They produce “Test Anything Protocol” (TAP) output, a simple protocol that displays unit test results in an easily understood report.

What to test?

A good first step in writing unit tests is deciding on something to test. In my case, I figured I should make sure row-level security policies were turned on for the tables I was interested in, which is available from the rowsecurity field in the pg_tables view:

select ok(rowsecurity, tablename || ' has row security enabled') from pg_tables
    where schemaname = 'public'
        and (
            tablename in (
                -- ... some hard coded table names
            ) or tablename like 'some_other_tables_%'
        );

The ok() function comes from pgTAP. It counts as one test each time it’s called; the test passes when the first argument is true, and fails when the argument is something else. The second argument is an optional comment describing what’s being tested. Following a pretty common TAP-related naming convention, I put this in a file called 00-test.sql in a directory under the root of my project, simply called t.

A more complicated set of tests could include several different files, where the numeric part of the name helps sort the tests in the desired run order, and the rest of the filename describes the subject of the tests within. But this will do just to get started. I can run it with pg_prove, included with the pgTAP package:

pg_prove -d mydatabase t/00-test.sql

Iteratively improving

This fails, for several reasons.

First, I haven’t yet installed the pgTAP extension in my database, with CREATE EXTENSION pgtap.

I also haven’t actually done anything in my test to run the code I’m testing. The actual code in this project consists of some database functions, which we need to run to create the database security policies, and I haven’t run any of them yet.

And finally, pgTAP requires me to “plan” my tests first, or in other words, I need to inform pgTAP how many tests I plan to run, before I run them. It’s also nice to call finish() so pgTAP can clean up after itself.

I installed the pgTAP extension in my database, and modified the test as follows:

begin;

\i create_policy.sql

select plan(1);  -- plan for a single test

select ok(rowsecurity, tablename || ' has row security enabled') from pg_tables
    where schemaname = 'public'
        and (
            tablename in (
                -- ... some hard coded table names
            ) or tablename like 'some_other_tables_%'
        );

select finish();

rollback;

This wraps my test in a transaction, so that I can roll everything back to leave the database essentially as I found it. It also calls the actual code I’m testing, in create_policy.sql, and plans one test. And it gives me this new failure:

t/m.sql .. All 1 subtests passed

Test Summary Report
-------------------
t/m.sql (Wstat: 0 Tests: 226 Failed: 225)
  Failed tests:  2-226
Parse errors: Bad plan.  You planned 1 tests but ran 226.
Files=1, Tests=226,  1 wallclock secs ( 0.04 usr  0.00 sys +  0.03 cusr  0.01 csys =  0.08 CPU)
Result: FAIL

The problem here is that each call to ok() counts as one test, and my test apparently found 226 tables to check for row-level security. I can improve the planning like this:

select plan(count(*)::integer)
    from pg_tables where schemaname = 'public'
        and (
            tablename in (
                -- ... some hard coded table names
            ) or tablename like 'some_other_tables_%'
        );

count() returns a bigint, and plan() expects integer, so this requires a typecast, but is otherwise pretty simple. And now my tests pass:

josh@here:~dw$ pg_prove -d nedss t/00-test.sql
t/00-test.sql .. ok
All tests successful.
Files=1, Tests=226,  1 wallclock secs ( 0.03 usr  0.01 sys +  0.03 cusr  0.01 csys =  0.08 CPU)
Result: PASS

Looking back and ahead

Suffice it to say that pgTAP includes many functions similar to ok(), to test various aspects of the database, its structure, and its behavior, and I’d recommend interested users review the documentation for more details. I intended this post only as an introduction.

In its completed state, my test suite comprised several tests ensuring various required preliminaries were in place, a few tests like the one above that check for necessary table-specific settings, others that ensure the affected roles were created, and finally some which create some sample data and use SET ROLE to test the data visibility directly for roles with various policies applied.

And to be honest, I was surprised at the sense of security that came over me with this completed test suite. As I mentioned, I’d done similar work previously, and knew that although I was confident in the code when it was written, that confidence came only through fairly extensive manual testing. I know very well the struggles of bit rot, and I knew it would be at least as hard to repeat that testing regimen by hand sometime down the road after a year or two.

I also recognized that if I ever needed to set up similar policies again, I could use these tests themselves as a reference, because they show exactly how to run the code in question. Though of course I included that information in the project’s associated README file as well … right?

Let us know if you’ve used pgTAP, and what effect it has had on your database development.

Using a YubiKey as authentication for an encrypted disk

2022-03-07T00:00:00+00:00

Image by Silas Köhler on Unsplash

Recently I built a small desktop computer to run applications that were a bit much for my laptop to handle, intending to bring it with me when I work outside my apartment. However, there was an immediate issue with this plan. Because this computer was intended for use with sensitive information/source code, I needed to encrypt the disk, which meant that I’d need to enter a passphrase before I could boot it up.

I didn’t really want to haul a keyboard and monitor around with me, so I came up with an alternative solution: using a YubiKey as my method of authentication. This allowed me to avoid the need to type a password without giving up security. In this post I’ll show you how you can do the same.

Preparation

First off, you need a YubiKey, if you don’t have one already. I ended up getting the YubiKey 5C NFC.

While I waited for my YubiKey to arrive, I installed Ubuntu 20.04 with full-disk encryption (using the default option of LUKS, or Linux Unified Key Setup) on the computer. I set a passphrase like normal—the process I describe in this post allows access with either this passphrase or the YubiKey.

Next, there were two packages that I needed to configure everything:

yubikey-personalization allows you to change the settings on your YubiKey. I installed it from the Ubuntu repository and had no problems.
yubikey-luks is what lets you use the YubiKey as an authentication method for a LUKS setup. I initially installed this from Ubuntu’s repository as well, but the version they’ve got is fairly out of date and required both a YubiKey and passphrase instead of just the YubiKey. As I mentioned earlier, the main objective of setting this up was booting without a keyboard, so I installed the tool from source as detailed in its README.

Setup

Once you’ve got the above libraries installed, setup is simple. Step by step:

1. Configure your YubiKey to use challenge-response mode

A YubiKey has at least 2 “slots” for keys, depending on the model.

We will change only the second YubiKey slot so you will still be able to use your YubiKey for two-factor auth like normal.

Plug in your YubiKey and run the following command:

ykpersonalize -2 -ochal-resp -ochal-hmac -ohmac-lt64 -oserial-api-visible

2. Find a free LUKS slot to use for your YubiKey

LUKS also allows for multiple key slots so that you can have different passphrases to unlock the encrypted data. Up to 8 key slots are available for LUKS1, and up to 32 for LUKS2.

Most setups only use the first slot for the main passphrase, but we can check by following these steps:

First run lsblk and figure out the name of your LUKS-encrypted disk partition. Mine was nvme0n1p3.
Now run sudo cryptsetup luksDump /dev/nvme0n1p3. The output should look something like this:

LUKS header information
Version:        2
Epoch:          11
Metadata area:  [a smallish number] [bytes]
Keyslots area:  [a medium number] [bytes]
UUID:           [a UUID]
Label:          (no label)
Subsystem:      (no subsystem)
Flags:          (no flags)

Data segments:
  0: crypt
        offset: [a big number] [bytes]
        length: (whole device)
        cipher: aes-xts-plain64
        sector: 512 [bytes]

Keyslots:
  0: luks2
				[Lots of information about this slot]
Tokens:
Digests:
  0: pbkdf2
        Hash:       sha256
        Iterations: 370259
        Salt:       [A bunch of bytes in hex format]
        Digest:     [A bunch of bytes in hex format]

You’re looking specifically for a free keyslot, and the output here only shows anything in slot 0, so slot 1 should be free.

3. Assign your YubiKey to a free slot

You can do this with the following command (substituting in your own partitition name and slot number):

sudo yubikey-luks-enroll -d /dev/nvme0n1p3 -s 1

This command will ask you for a passphrase. It doesn’t need to be a particularly complex one, because it’ll only work with your YubiKey.

4. Update crypttab and ykluks.cfg

Now you need to add keyscript=/usr/share/yubikey-luks/ykluks-keyscript to /etc/crypttab. For example, mine started as:

nvme0n1p3_crypt UUID=[uuid-here] none luks,discard

After the change, it should look like this:

nvme0n1p3_crypt UUID=[uuid-here] none luks,discard,keyscript=/usr/share/yubikey-luks/ykluks-keyscript

Finally, you need to configure yubikey-luks to give the passphrase you just set so you don’t have to. Open /etc/ykluks.cfg and add the line

YUBIKEY_CHALLENGE="[your new passphrase here]"

Once you’ve added this line, run sudo update-initramfs -u and you’re done!

Conclusion

Now if you shut your machine off, plug in your YubiKey, and turn it on, it should boot all the way without needing a passphrase. If you forget to plug in the YubiKey before turning the computer on, you’ll probably need to hold the contact button on it for a second or two and then it should boot just the same.

And there you go! A YubiKey provides neat way to securely start up a computer with an encrypted disk without needing a passphrase.

Fixing a PostgreSQL cluster that has no superuser

2022-01-07T00:00:00+00:00

Normally in a newly-created PostgreSQL database cluster there is a single all-powerful administrative role (user) with “superuser” privileges, conventionally named postgres, though it can have any name.

After the initial cluster setup you can create other roles as needed. You may optionally grant one or more of your new roles the superuser privilege, but it is best to avoid granting superuser to any other roles because you and your applications should generally connect as roles with lower privilege to reduce the risk of accidental or malicious damage to your database.

Let’s break something 😎

Imagine you have a cluster with two or more superuser roles. If you accidentally remove superuser privilege from one role, you can simply connect as the other superuser and re-grant it.

But if you have a cluster where only the single postgres role is a superuser, what happens if you connect as that role and try to remove its superuser privilege?

$ psql -U postgres postgres
psql (14.1)
Type "help" for help.

postgres=# \du
                       List of roles
 Role name |            Attributes             | Member of
-----------+-----------------------------------+-----------
 postgres  | Superuser, Create role, Create DB | {}
 somebody  | Create DB                         | {}

postgres=> \conninfo
You are connected to database "postgres" as user "postgres" via socket in "/tmp" at port "5432".
postgres=# ALTER ROLE postgres NOSUPERUSER;
ALTER ROLE
postgres=# \du
                 List of roles
 Role name |       Attributes       | Member of
-----------+------------------------+-----------
 postgres  | Create role, Create DB | {}
 somebody  | Create DB              | {}

postgres=# ALTER ROLE postgres SUPERUSER;
ERROR:  must be superuser to alter superuser roles or change superuser attribute
postgres=# \q

PostgreSQL happily lets us do that, and now we have no superuser, and so we cannot re-grant the privilege to that role or any other!

Homebrew PostgreSQL problem

Aside from such a severe operator error, there are other situations where you may find no superuser exists. One happened to me recently while experimenting with PostgreSQL installed by Homebrew on macOS.

I used Homebrew to install postgresql@14 and later noticed that it left me with a single role named after my OS user, and it was not a superuser. It couldn’t even create other roles. I’m not sure how that happened, perhaps somehow caused by an earlier installation of postgresql on the same computer, but so it was.

Since there wasn’t any data in there yet, I could have simply deleted the existing PostgreSQL cluster and created a new one. But in other circumstances there could have been data in there that I needed to preserve, which wasn’t accessible to my one less-privileged user, and which caused errors in pg_dumpall.

So how can we solve this problem the right way?

First, stop the server

We need to get lower-level access to our database. To do that, first we stop the running database server.

On a typical modern Linux server running systemd, that looks like:

# systemctl stop postgresql-14

On macOS using Homebrew services, that could be:

$ brew services stop postgresql

Or in my experimental case with Homebrew using a temporary Postgres server which I’m showing here:

$ pg_ctl -D /opt/homebrew/var/postgresql@14 stop
waiting for server to shut down.... done
server stopped

Next, start the PostgreSQL stand-alone backend

Next we start the “stand-alone backend” which a single user can interact with directly, not using a separate client.

No privilege checks are done here, so we can re-grant the superuser privilege to our postgres role.

Interestingly, SQL statements entered here end with a newline, no ; needed, though adding one doesn’t hurt. And statements here can’t span multiple lines without being continued with \ at the end of each intermediate line.

In the command below, note that the --single option must come first, and the postgres at the end of the command is the name of the database we want to connect to:

$ postgres --single -D /opt/homebrew/var/postgresql@14 postgres

PostgreSQL stand-alone backend 14.1
backend> ALTER ROLE postgres SUPERUSER
2022-01-07 20:32:51.321 MST [27246] LOG:  statement: ALTER ROLE postgres SUPERUSER

2022-01-07 20:32:51.322 MST [27246] LOG:  duration: 1.242 ms
backend>

The postgres server stand-alone prompt does not have all the niceties of psql including line editing features, history, and backslash metacommands such as \q to quit, so we type control-D there to mark “end of file” on our input stream and exit the program.

Back to normal

Now we can again start the normal multi-user client/server PostgreSQL service:

$ pg_ctl -D /opt/homebrew/var/postgresql@14 start
waiting for server to start.... done
server started

And finally we can connect to the server and confirm our change persisted:

$ psql -U postgres postgres
psql (14.1)
Type "help" for help.

postgres=# \du
                       List of roles
 Role name |            Attributes             | Member of
-----------+-----------------------------------+-----------
 postgres  | Superuser, Create role, Create DB | {}
 somebody  | Create DB                         | {}

postgres=#

Reference

Remote Access Control with AWS Security Groups

2021-12-23T00:00:00+00:00

With the onset of COVID-19, a lot of companies started remote work and had to allow their employees to access systems remotely while keeping unwanted traffic out.

As a company of almost all remote workers, even pre-pandemic, we have long had a solution for this. We have a web application that allows people to register their current IP address with one or more servers to gain access to staging sites, SSH, SFTP, databases, and other services that aren’t open to the public.

The application relies on host-based firewalls iptables (Linux) or pf (OpenBSD) to give people access to the system. For customers using cloud providers like AWS, we added support for this application to work with AWS Security Groups.

The Basics

The system uses a single web application and relies on the Apache mod_authn_file format and htpasswd command to create users and hashed passwords for the system. htpasswd allows you to create bcrypt, MD5, SHA-1, and Unix crypt passwords. We suggest using bcrypt wherever possible, since it is the only one in that list which is not fairly easily reversible nowadays.

The system also allows for grouping of users to allow them access to different sets of systems. These are handled by the simple Apache mod_authz_groupfile format that looks like this:

web: richard, joe, frank
dbs: richard, frank

The data format is the group name followed by a colon (:), then a comma-separated list of users who should be allowed to access that service. In this example, the users richard, joe, and frank would be allowed to access the web group but only richard and frank would be allowed to access the dbs group.

The groups in this file then dictate what servers and services you are allowed to access. For example, the dbs group might allow you ssh and database connectivity to the specific production and replication database systems in AWS. It does this by adding rules to the proper security groups in AWS via their API.

We typically set up two different security groups that a set of servers can use: a static whitelist and a dynamic whitelist. The static whitelist is populated with IP addresses that should always be allowed in. This could be anything from a set of IP addresses for people still working at the office to employees’ dedicated IP addresses from their ISPs. This could also be certain other systems (remote data sources or such) that would need access to these systems. The dynamic whitelist is what the web application will add rules to after someone enters their proper username and password.

The API layer

Now that we’ve reviewed the way to setup users and the way users can be organized into groups, we can talk about how the web application interfaces with AWS itself.

After the user passes the username and password validation, the system will use the AWS API key to add the user to the appropriate security groups. We need certain information about each security group we are going to add people to: the Security Group ID, description, profile, and region. We also need to know what ports we are going to open for someone for this group.

So using our previous example, if we are going to be granting access to the user frank and giving him access to the web servers then we’d want to open up TCP & UDP (for HTTP/3!) ports 443 (https) and probably port 80 (http). The same goes with granting access to the database servers: give frank access to whatever port the database allows you to connect on.

The Security Group Layer

The secret sauce in this is interfacing with AWS’s security group.

Create a new security group in the account—one per server or logical group (e.g. TCP port 22 for SSH on all servers).

Then create a new IAM account. Go to the IAM menu and click on Groups in the side tab.

Click “Create a Group”
- Name the group whitelist-user-group_id

Click “Create Policy”

Choose Service “Ec2”

Set the following permissions:

List (4 of 123 actions)
DescribeSecurityGroupReferences
DescribeSecurityGroups
DescribeStaleSecurityGroups
DescribeVpcs

Write (4 of 285 actions)
AuthorizeSecurityGroupEgress
AuthorizeSecurityGroupIngress
RevokeSecurityGroupEgress
RevokeSecurityGroupIngress

Hint: You can find all but one of these by searching ‘SecurityGroup’, and the last one by searching for DescribeVpcs

The write functions you’ll need to attach to a specific security group or groups, which you created earlier.

When you’re done, you should have:

User (whitelist_groupid) → in group (whitelist_group_groupid) → with attached policy (pol-whitelist) → with the above eight permissions, the four latter ones pointed at your new security group.

Attach the security group to your VM.

You’ll need to make calls to the security groups over the API. You can find more info about the API call for changing firewall rules in AWS’s docs.

When the layers combine

With these pieces in place, you can start thinking about what groups would give people access to what. How broad or granular you want to be is up to you! If you want to have a group that gives access to every system and all the ports (maybe for your operations team), go for it! If you want to have a group that just needs access to https on certain systems (like a sales and marketing team) then you can easily control that as well.

Generating TOTP QR codes as Unicode text from the command line

2021-10-28T00:00:00+00:00

(QR = “Quick Response” — good to know!)

Python’s QR code generator library qrcode generates QR codes from a secret key and outputs to a terminal using Unicode characters, not a PNG graphic as most other libraries do. We can store that in a text file. This is a neat thing to do, but how is this functionality useful?

Benefits of having Unicode QR code as a text file:

Storing the QR code as a text file takes less disk space than a PNG image.
It is easy to read the QR code over ssh using the cat command; you don’t even have to download the file to your own workstation.
It is simpler to manage QR codes in Git as text files than as PNG images.

This can be used for any kind of QR code, but we have found it especially useful for managing shared multi-factor authentication (MFA, including 2FA for 2-factor authentication) secrets for TOTPs (Time-based One-Time Passwords).

Multi-factor authentication (MFA)

Many services provide a separate account and login for each user so that accounts do not need to be shared, and thus passwords and multi-factor authentication secrets do not need to be shared either. This is ideal, and what we insist on for our most important accounts.

Unfortunately, however, some services provide only a single login per account, or only a single primary account login with the other accounts being limited in serious ways (no access to billing, account management, etc.) so that any business relying on them needs to share the access between several authorized users. A single point of failure in an account login is a serious problem when that one person is unavailable.

TOTP mobile apps

There are many good mobile apps for managing TOTP keys and codes, including Aegis, FreeOTP, Google Authenticator, and many others. Look for one that works with no connection to the outside world, so that you won’t be stuck when off internet & data networks.

Most applications support scanning QR codes with the phone’s camera, or else typing in a secret key to import the accounts.

For those shared accounts with no option for fully empowered individual user accounts, we can convert secret keys into QR codes for easy sharing and easy imports.

First, note that you should never use online QR code generators for MFA secrets! You risk exposing your extra authentication factors and defeating the purpose of your extra work.

Python’s ‘qrcode’ library

The qrcode Python library provides a qr executable that can print your QR code using UTF-8 characters on the console.

Installation

apt install python3-qrcode

Or visit https://pypi.org/project/qrcode/

The contents of the QR code are a URL in the format:

otpauth://totp/{username}?secret={key}&issuer={provider_name}

The provider_name can contain spaces; however, they need to be URL-encoded and entered as %20 for auth to work correctly on iOS. Otherwise, an invalid barcode error will be shown when adding the code.

For example, if you generate the QR code with key JSZE5V4676DZFCUCFW4GLPAHEFDNY447 for the account root@example.com, the resulting command would be:

$ qr "otpauth://totp/Example:root@example.com?secret=JSZE5V4676DZFCUCFW4GLPAHEFDNY447&issuer=Superhost"

Here’s what its output looks like:

Providing the username and issuer will display it properly in the list of configured accounts in your authenticator application. For example: Superhost (Example:root@example.com)

Reference

See a simple explanation of otpauth URI format used by TOTP.
See RFC 6238 for full details about TOTP.
See RFC 4648 for the base 32 specification used to encode the secret key.
A recent similar Perl implementation Terminal: QR Code with Unicode characters by Flavio Poletti that builds on Text::QRCode, which uses libqrencode.

Lock down your security with GPG on a YubiKey

2021-09-10T00:00:00+00:00

Photo by Mauro Sbicego on Unsplash

Gnu Privacy Guard (GnuPG or GPG) is a tool we use a lot at End Point. Its ubiquity and quite decent security is a perfect fit for us — and there’s a way to make it even safer.

GPG uses the OpenPGP standard to encrypt files. Normally, one creates a PGP key on their computer and just keeps the keyfile safe. A password is generally used, but as with any private key, it’s only as safe as the computer it’s on.

Got a YubiKey and not sure what to do with it? Want to get a little more secure with your encryption?

In case you haven’t heard of them, YubiKeys are hardware USB keys that can be used as a multi-factor authentication (MFA) token, or to fill in one-time password (OTP) fields (like those generated by Google Authenticator) on sites that don’t support the YubiKey directly as an MFA token.

Using a smart card like a YubiKey can increase GPG’s security, especially if the key is generated on an air-gapped machine. This way the keyfile is stored in the hardware security token, and is never exposed to the internet.

In addition, you can even store an SSH key on the card, which will enable you to log in to remote Linux machines while keeping your private key secured.

While there isn’t full password locking on hardware tokens, YubiKey and almost all OpenPGP keys have two PINs — a user PIN and an administrative PIN to reset the user PIN. If you enter either or both three times incorrectly, the card will lock and you’ll need to reload from backup (or, in some cases, throw the card away) which is why it’s critical to have a backup.

There are several options for smart cards beyond the YubiKey. You can use any OpenPGP compatible card and reader, or an all-in-one solution that’s compatible with OpenPGP.

The following instructions do require a basic understanding of the command line and of how to create a live CD/USB stick, but if you need to use GPG, you’re probably already at least somewhat familiar with these requirements.

An air-gapped machine isn’t required for these instructions. You could do this on any machine you trust, but using a machine with a fresh OS that hasn’t been connected to the internet affords the highest level of security. Simply booting from a live CD/USB is fairly easy. Choosing an operating sytem that comes with the smart card daemon (scdaemon) will help. If you don’t, make sure you download the scdaemon package for your operating system to use with the live CD/USB.

Prepare

Before you begin, you’ll need:

A smart card solution as described above.
A backup smart card, or external media on which to store an encrypted copy of the key.
A machine on which to generate the key.
A live OS. For this demonstration I used Tails, The Amnesic Incognito Live System. It’s specifically designed to not store logs or keep data from one reboot to another.
A way to access these instructions, like a second computer, phone, printout, or a very good memory.

Boot up the live machine. Note that if you’re using Tails, there are two settings you’ll need to choose on the welcome screen. Click the plus button and do the following:

Set an administrative password. Tails doesn’t set a root password by default, and thus disallows root access for better security. You can set one yourself.
Disable networking in the ‘Network Connection’ section.

Access and configure the card

Once booted, run an admin terminal, or load a terminal and run sudo -i. It’ll prompt you for the password you just set.

Ensure you can access the card and that the smart card daemon is installed by running gpg --edit-card. It should display information about your smart card.

If you’re using an air-gapped machine and your live OS is missing requisite packages, don’t access the internet with the machine in order to install it, since that would break the air gap. Instead, copy the installation files across using sneakernet: Add the files to a USB drive, perhaps the one you’ll be using to back up your PGP key). The packages for a Debian-based machine are: scdaemon, libccid, pcscd, rng-tools, and gnupg2. Debian has instructions on how to get packages to an air-gapped machine.

Set a PIN for your card, if you haven’t already. Start with:

$ gpg --edit-card

The default PIN for a YubiKey should be 123456, and the default admin PIN should be 12345678. Check the documentation that came with your key, though!

Enable admin features first:

gpg/card> admin

This should return Admin commands are allowed.

Set the passwords, both for the regular PIN and the admin PIN:

gpg/card> passwd

1 - change PIN <- Default 123456
2 - unblock PIN <- To reset the pin with the AdminPin / Reset Code
3 - change Admin PIN <- Default 12345678
4 - set the Reset Code <- See below
Q - quit

Do not mix up your PIN and admin PIN! You can lock up your card, which will require a factory reset.

The reset code is set if you are setting up the card for someone else to use, and wish to give them a way to reset the PIN without having full access to the rest of the admin functions.

Generate the keys

Run the following to generate the key:

$ gpg --expert --full-gen-key

Key type: 1 (RSA & RSA). (You can also use “ECC & ECC” if you’re brave. These types of keys may not work with older systems and implementations of GPG, however, so your mileage may vary. If you do want to use ECC & ECC, use Curve 25519 in the next step.)
Key size: this should be the maximum supported by your key. YubiKey 4 or 5 can support up to 4096. Use this for both main and subkey.
Expiry: This is your choice. I’d set it to a year or two.
Real name, email, and comment: I recommend leaving the comment blank, since most of the time the email address will be enough information.
Next, GPG will ask you to move your mouse around — don’t sprain anything while generating entropy!

Now your key is generated. If you only have one spare storage device which you want to use for backups, copy the revocation certificate and the SSH public key to storage, sneak this on to your main computer, and only then copy the keys over to the backup drive. Don’t attach the backup drive to anything but an air-gapped machine once it holds your key!

You might as well generate an SSH key now. Even if you don’t use it, there’s no harm in having it.

$ gpg --expert --edit-key <your email/key id>

Run gpg/card> addkey to add a key.
Type 8, RSA (set your own capabilities). If this option doesn’t show up, ensure you used --expert above.
Enable authentication and disable signing and encrypting — type s, e, a, then q to save.
4096 bits is the best number to use, at least for RSA on modern YubiKeys. If you have an older key you may be limited to 3072 or 2048.
You can choose to have the key expire, but ensure you have a backup method of logging in.
Confirm your choices, then quit, confirming you want to save.

For more in-depth instructions, visit https://opensource.com/article/19/4/gpg-subkeys-ssh

Export the public keys:

$ gpg --export --armor <key ID> > /path/to/thumbdrive/<email>_pub.asc

$ gpg --export-ssh <key ID> > /path/to/thumbdrive/YubiKey_id_rsa.pub

Move the keys to your card

Once you have the key added to your keyring, you’ll need to transfer that key to your card:

$ gpg --edit-key <key id>

To export the public SSH key you’ll need to put on remote servers, you can run the command: gpg --export-ssh-key 0x123456789ABCDE

gpg> keytocard — confirm you want to move the primary key and store this in position 1 of the card.
- To select the encryption key, type key 1.
- Run keytocard to store the encryption key in the encryption slot.
If you have an encryption key:
- To select the authentication key, run key 2.
- To deselect the key first key, run key 1. You should only have one key with * marking it.
- Store in the authentication slot: keytocard.
Repeat this for as many subkeys as you have.
Once you’re done, quit and confirm the saved changes.

Time to test your new key!

And that’s it. Publish your new public GPG key, use your new SSH key, secure in the knowledge that your private key is protected from malicious attacks by an additional hardware layer.

What do I do if it all went wrong or I locked up the card?

You can start again. The following command will restore the GPG-compatible portion of your YubiKey to factory settings. You will lose any keys stored on the card. I don’t believe it’ll cause any OTP/2FA set up with the card to be lost, but I make no guarantees.

ONLY RUN THE FOLLOWING IF YOUR PIN IS LOCKED AND THE SMART CARD IS UNUSABLE:

$ gpg --edit-card
gpg/card> factory-reset

Follow the confirmation steps on screen.

Media erasure in the time of SSD

2020-12-10T00:00:00+00:00

Photo by Alex Fu from Pexels

How valuable is your data? Losing it to a third party is usually a business’s worst nightmare—and can cause legal or even criminal repercussions, depending on the drive’s contents and the business’s jurisdiction.

Every system adminstrator worth their salt knows that running “rm” (or equivalent delete operations) doesn’t actually remove data, it simply removes the file name from the filesystem and leaves the data in place on the disk.

When dealing with traditional storage, destroying (intentionally or otherwise) your data used to be relatively easy. A wise system admin could simply run:

shred /dev/sda

And be fairly certain of the result. A cautious one might run a demagnetizing wand over the drive. Only the most paranoid might destroy it physically.

The Age of SSDs

Nowadays, most servers have switched away from storing data on rotating metal or glass platters. Solid state drives, or SSDs, are faster, less prone to errors from physical impact, and generally more sought after.

SSDs have issues with speed if the drives are too full, and have a limited lifespan—only a certain number of write operations can be achieved. This is less of an issue with modern drives thanks to wear leveling built into the firmware of the drives. However, this leads to some issues as well.

Complicated systems introduce issues

Because SSDs manage which blocks of storage they write to, a simple shred won’t do. There could be hundreds of bytes, or even kilobytes or megabytes, of data that the shred doesn’t reach.

Even some “traditional” storage can run into such issues these days. Hybrid drives offer some speed advantages: By leveraging a small amount of SSD storage, these drives save data to SSD first, then write it at slower speeds to the actual magnetic platters. The same issues with SSD storage can affect this cache of data.

So how to be sure?

Ideally, I would recommend using a combination of methods for security. Here are the main methods that are used at present:

Run shred and hope for the best

This is an option, for sure. Writing to every block of the disk will generally wipe the data securely enough. However, if there are sectors that have been marked bad by the drive firmware, these won’t be covered.

nwipe, DBAN, and other free options

Free software exists to securely run shred over operating systems. The old gold standard for this was Darik’s Boot and Nuke (DBAN), written by Darik Horn, but this software was acquired by Blancco, a for-profit data erasure company (more on them later). DBAN is still available, but lacks features necessary for a 100% wipe.

A fork called nwipe exists, and is available on many live operating systems. nwipe does a more thorough job than shred, but it still can’t get to sectors the firmware hides from the operating system.

Wikipedia has a list of data-erasing software. Most of these are open source or freeware, but it includes a few paid options.

SATA secure erase with hdparm

Drive manufacturers have thought of this issue as well. Most drive manufacturers offer a secure erase program that works with their drives, and the SATA standard has a procedure in place for securely erasing drive contents.

hdparm --user-master u --security-set-pass TheEnd /dev/X
hdparm -I /dev/X  # Check that the master password is enabled
time hdparm --user-master u --security-erase TheEnd /dev/X

This will show some output indicating success or failure. Once this is done…

hdparm -I /dev/X  # Check that the master password is not enabled, which indicates the wipe was successful

This is likely the best option for a savvy home user, combined with shred/nwipe. However, if you are going to attempt this, I highly recommend reading the full instructions!

Commercial software

Blancco, aforementioned as the purchasers of DBAN, offer an enterprise level product for destroying data called Drive Eraser. Importantly, for business users, it provides certification that the data are gone for good.

There are also many other options. Ask your favourite security vendor and they will be happy to sell you a product for this, though you usually have to take them at their word.

Be prepared—encrypt your disk!

Encrypt your disk. This can be achieved by encrypting either your home folder or by encrypting your whole disk. One downside is that your password is required after every reboot—a problem especially for servers, but even cloud providers offer virtual terminals these days, so for secure operations, this is the best option.

A secure, long password is usually enough to ensure the disk can’t be cracked, though be aware that cybersecurity changes quickly—what’s impossible to brute-force today might not be in five years.

So how can we be really, really sure?

Thermite

Physical destruction of the media is always the most secure way to destroy a disk—crushing, drilling, or in the fanciful, dangerous dreams of some systems administrators, covering the disk in thermite and lighting it with a magnesium flare. (Do not actually use thermite.)

At home, disassembling the drive and taking a hammer to the data bearing chips will do the trick, though again, combine this with the above options to be sure.

For spinning disks, a similar procedure is advised, though there are other options, of course…

Check your local laws and follow all safety procedures before engaging in creative drive destruction techniques!

Final thoughts

Use any or all of the above options and you’ll be ahead of the game. Taking the time to sanitize your data, or better, encrypting it from the beginning, is always a good investment.

A great gift for the holidays: No ads!

2020-12-03T00:00:00+00:00

Many people will bring home a pie during the holiday season, but perhaps you’ll find a place in your home network for a Raspberry Pi instead?

With more people than ever working from home, many more people are using their personal infrastructure to conduct business, and aren’t able to rely on a crack team of network engineers to make sure their system is secure. While there are many things one can do to improve network security, from using a VPN to ensuring you update your system, a Pi-hole is one quick, inexpensive way to help keep your network a little safer not just on your phone or laptop, but on every device that connects to your router.

It’s great not only for technical types, but for everyone who connects to your network. You can even set it up with remote access and gift it to a relative, as long as you’re willing to fix it if it breaks. With the holiday season coming up, it’s surely something to consider.

Shut the door with Pi-hole

Pi-hole is an open source DNS server for your local network which blocks advertising and, after adding some extra block lists, some malicious websites.

This is done before the data even gets downloaded, by redirecting requests for ads and garbage to a blank page, which means your internet will be faster as well as safer—this is especially important if you’re sharing networks with spouses and kids. It doesn’t rely on any client-side software, either, so it works regardless of platform, even on some smart TVs and apps. (Remember to support the websites and apps you use by other means if possible, though!)

In addition to securing your network by allowing you to block malware, there’s an optional step that you can take to secure your DNS entirely: Encrypted DNS. With this, you can stop your ISP or any other entity from not only tampering with your DNS results, but seeing them at all. It won’t stop state actors, but it should help keep your browsing data from being sold.

Here’s some example stats from the author’s home. This is one (rather heavy) day’s worth of traffic!

Installing Pi-hole

What sort of knowledge do you need in order to be able to install Pi-hole? Well, it does require some minimal command line knowledge. However, if you don’t already have this, then this is a great first project to learn with.

The Pi-hole website offers great instructions for installing the software. While you may want to use it as initially intended—on a $35 Raspberry Pi board—you don’t have to purchase new hardware. You can run Pi-hole on any Linux server, or even on a Docker instance or using a virtual machine on many different services.

Just make sure that whatever system you install Pi-hole on is always turned on and stable (unplug the Pi-hole and your internet may stop working for the house, something I know my own family hates to have happen).

Adding blocklists

Adding blocklists is as simple as putting in a link to a text file with the offending hostnames. There are quite a few blocklists curated on the page. The trick is to choose blocklists that will meet your requirements—Some of the block lists can generate false positives. If you’re using Pi-hole for yourself, that’s probably not an issue since you can manually whitelist things as you go, but if you have a larger household, you may want to stick with the more conservative whitelists. Same if you’re setting this up for a relative or family member where you won’t be on site to fix things.

Firebog.net has a good list of block lists, sorted by category. Green checkmarks are best for minimal maintenance situations; the other categories may incur false positives that require whitelisting.

Encrypted DNS

While blocking ads is a great start, to add further security to your DNS, you have several options.

You can enable DNSSEC, which doesn’t encrypt your DNS but does validate it, assuming the domain owner actually enabled it. This is a good thing to enable, but it doesn’t replace true encrypted DNS.

Cloudflare is the service that Pi-hole officially recommends using, via the tool cloudflared. This routes all DNS traffic over https to Cloudflare’s DNS servers by default. There are also several other options, such as DNSCrypt. There’s a debate going on about DNS over HTTPS vs DNS over TLS, which is summed up nicely by this IEEE spectrum article.

Do you need to do this? No. But taking this step is relatively easy and protects you and your family from slippery man-in-the-middle attacks as well as your ISP snooping on what websites you go to and selling this to advertisers.

What can’t it do?

For now, it won’t block all advertising. YouTube ads aren’t blocked, nor are most streaming services that display ads from their internal servers. It won’t block ads hosted directly on the website you’re visiting.

To keep things running smoothly, there is some maintenance required: the OS and Pi-hole need to be updated periodically, and the blocklists as well, though these tasks are easily automated.

The Pi-hole also won’t protect you from concerted attempts to get into your system. This isn’t a replacement for a good firewall or antivirus/malware protection software, and you should of course remember to practice good browsing habits. But as a relatively simple project to set up, it’s a good way to make your browsing experience at least a little less annoying.

Testing to defend against nginx add_header surprises

2020-05-29T00:00:00+00:00

These days when hosting websites it is common to configure the web server to send several HTTP response headers with every single request for security purposes.

For example, using the nginx web server we may add these directives to our http configuration scope to apply to everything served, or to specific server configuration scopes to apply only to particular websites we serve:

add_header Strict-Transport-Security max-age=2592000 always;
add_header X-Content-Type-Options    nosniff         always;

(See HTTP Strict Transport Security and X-Content-Type-Options at MDN for details about these two particular headers.)

The surprise (problem)

Once upon a time I ran into a case where nginx usually added the expected HTTP response headers, but later appeared to be inconsistent and sometimes did not. This is distressing!

Troubleshooting leads to the (re-)discovery that add_header directives are not always additive throughout the configuration as one would expect, and as every other server I can think of typically does.

If you define your add_header directives in the http block and then use an add_header directive in a server block, those from the http block will disappear.

If you define some add_header directives in the server block and then add another add_header directive in a location block, those from the http and/or server blocks will disappear.

This is even the case in an if block.

In the nginx add_header documentation we find the reason for the behavior explained:

There could be several add_header directives. These directives are inherited from the previous level if and only if there are no add_header directives defined on the current level.

This nginx directive has always behaved this way. Various people have warned about it in blog posts and online discussions for many years. But the situation remains the same, a trap for the unwary.

I have tried to imagine the rationale behind this behavior. Response headers often are set in groups, so the programmer who created this feature may have decided that any new scope’s add_header directives should start with a clean slate, unaffected by those set elsewhere. Hmm. The need for exclusive grouping of response headers is rare in my experience, and adding headers to the existing stack of tentative response headers is far more commonly what I want.

So while this behavior may make sense somewhere, it has not ever done so for me or anyone I have talked to about it. For us it is simply misbehavior, silent and easy to overlook when making later seemingly unrelated configuration adjustments.

Dangers

It often has security implications when headers you thought were being added to every response are not. Consider more fine-tuned and consequential security-related headers such as Content-Security-Policy, Vary for cache object separation, CORS headers Access-Control-*, etc.

Headers such as these are especially important when they need to be added based on logic spread across various configuration blocks, and that is exactly when nginx add_headers doesn’t work as expected.

Another pitfall is omitting the always option to add_header. Without that, the header will only be added to success responses (2XX and 3XX, but see the docs for specifics). We usually want security-related headers to be added even to 4XX and 5XX error responses.

Workaround using include

My first instinct was to work around the problems caused by this behavior by putting the standard add_header list in a file that I include everywhere. In some cases that works.

But despite the nginx include documentation saying that directive is allowed in “Context: any”, include is not allowed in an if block and will result in the fatal startup error:

“include” directive is not allowed here

So the only recourse in those cases is to repeat all needed add_header directives in every if block that uses add_header. Gross.

Repeating configuration manually means almost surely having the add_header directives in different configuration areas drift over time. So if we have to repeat ourselves, at least let’s do it with automation, such as by using configuration templating and preprocessing.

That is what I have most recently done. And we can still use native nginx include directives everywhere those are allowed.

nginx Headers More module

Many people have run into exactly this problem, and some of them developed a separate nginx module ngx_headers_more to solve most of these problems.

By using its more_set_headers directive, you get the expected additive behavior with previously-declared headers, regardless of the block scope:

Directives inherited from an upper level scope (say, http block or server blocks) are executed before the directives in the location block.

Note that although more_set_headers is allowed in location if blocks, it is not allowed in the server if blocks …

Fortunately I have not needed to use this in an if block in the server scope, so that one remaining limitation doesn’t pose a problem for me.

It also has options to set a header only for responses of a certain HTTP content type or status code.

The more_clear_headers directive allows the * wildcard for clearing all headers with the same prefix at once, such as Access-Control-*.

Installing ngx_headers_more

Because “Headers More” is a separate module, not part of standard nginx, it is not usually available without some extra work.

You can build it from source and install it manually, but of course that isn’t good to do on a production machine since it won’t get updated on its own.

You can use the OpenResty server built around nginx, which “Headers More” is part of. But you may not want all of that if you’re not writing a Lua web application.

Many Linux distributions and 3rd-party package repositories have prebuilt packages for “Headers More” which you can use:

Alpine
- nginx-mod-http-headers-more
Debian & Ubuntu
- nginx-extras
- libnginx-mod-http-headers-more-filter
RHEL/CentOS
- GetPageSpeed & Webtatic repos nginx-module-headers-more
- Aeris repo nginx-more

Search the excellent pkgs.org to find what you need if it isn’t already available through your package manager.

Apache

Apache httpd is still alive and well — actually better than ever. So depending on your situation, you may want to use that instead.

Apache’s Header directive has intuitive (to me) default behavior for setting response headers across the whole configuration, and many ways to deal with a possibly already-existing header:

add another header, or set exclusively (replace), or set only if this header doesn’t already exist
append to or merge into an existing header (for headers that accept multiple values)
edit an existing header with a regular expression search-and-replace
unset a header if one was previously set

I don’t know a way to have Apache clear a group of headers with a wildcard, or all headers at once, so they need to be individually cleared by name if that’s what you want.

Доверяй, но проверяй (Trust, but verify)

nginx was written by Igor Sysoev. Despite my disagreement with this one feature’s behavior, overall I find that nginx is excellent. Because of its open source release, excellent performance, and wide use, it has provided much-needed competition to Apache and Microsoft IIS. Thank you, Igor and all other contributors!

In the relevant spirit, since Igor is Russian, I close with the Russian proverb Доверяй, но проверяй: Trust, but verify.

Let us code (and configure) defensively, yet also test to avoid being surprised by missing headers.

We can manually test various HTTP responses are as we expect using curl -v or other HTTP clients to exercise various requests.

Even better, we can add to our automated test suite to confirm these HTTP response headers appear everywhere we expect, for static files and API endpoints backed by different application servers, and for various success and error responses.

Here is a test adapted from one I put together for one of our clients. It uses JavaScript in Node.js, the Jest test framework, and the Axios HTTP client. It ensures the security headers example I showed at the beginning of this article keeps working, even as we make nginx configuration changes over time:

const axios = require('axios');

const http = axios.create({
  baseURL: 'https://your.dom.ain',
});

describe('Check security headers', () => {
  const verifs = [
    { header: 'strict-transport-security', expect: (x) => x.toMatch(/max-age=\d{3,}/) },
    { header: 'x-content-type-options',    expect: (x) => x.toEqual('nosniff')        },
  ];

  const locs = [
    { path: '/robots.txt',                status: 200 },  // static
    { path: '/feed/endpoint/of/interest', status: 200 },  // API backend in PHP
    { path: '/api/other/auth/endpoint',   status: 403 },  // API backend in Perl
    { path: '/never/gonna/give/you/up!',  status: 404 },
    { path: '/api/dies/for/testing',      status: 500 },
  ];

  // throw no exceptions for non-success HTTP response status
  const conf = { validateStatus: () => true };

  for (const l of locs) {
    test(`${l.status} ${l.path}`, async () => {
      const res = await http.get(l.path, conf);
      expect(res.status).toBe(l.status);
      for (const v of verifs) {
        v.expect(expect(res.headers[v.header]));
      }
    });
  }
});

Here I run just this one test rather than the whole suite:

% jest -w 6 ./__tests__/webserver/security-headers.test.js
Determining test suites to run...
testing on https://https://your.dom.ain

 PASS  webserver/security-headers.test.js
  Check security headers
    ✓ 200 /robots.txt (55ms)
    ✓ 200 /feed/endpoint/of/interest (408ms)
    ✓ 403 /api/other/auth/endpoint (18ms)
    ✓ 404 /never/gonna/give/you/up! (6ms)
    ✓ 500 /api/dies/for/testing (12ms)

Test Suites: 1 passed, 1 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        2.721s, estimated 3s
Ran all test suites matching /.\/__tests__\/webserver\/security-headers.test.js/i.

This can also be extended to ensure that certain headers do not exist, or do not contain details that you do not want exposed:

the Server header should not reveal the nginx (see server_tokens) or Apache (see ServerTokens) version numbers
the X-Powered-By header should be absent, not exposing the fact that you are using PHP, and the version number — see the expose_php directive for php.ini
or with the Java Wildfly server, both of those headers are sent by default! — see instructions on how to omit them by editing XML or using jboss-cli

Add to the verifs array in the code above:

    { header: 'server',                    expect: (x) => x.not.toMatch(/\d/)         },
    { header: 'x-powered-by',              expect: (x) => x.toBeUndefined()           },

Now if (when) I forget about the nginx add_headers behavior, make changes, and inadvertently break things? Instead of it being unnoticed, my test suite will alert me so I can fix it before it goes into production!

Consolidating Multiple SFTP Accounts Into One Master Account

2020-03-16T00:00:00+00:00

Photo by Dan Meyers

Recently, a client implemented a data-intensive workflow to generate various reports and insights from a list of facilities as part of an EpiTrax installation. Because a significant portion of these files contain sensitive healthcare data, they needed to strictly comply with HIPAA. Optimally, facilities should be able to transfer files securely and exclusively to our server. One of the best methods of achieving this is to create individual SSH File Transfer Protocol (SFTP) accounts for each source.

SFTP account

Private SFTP accounts were established for each facility and the data was received at a designated path. At these individual points of contact, a third-party application picks up the data and processes further into the pipeline. The following demonstrates how SFTP accounts are developed and configured:

Create a user group for SFTP accounts:

$ addgroup sftpusers

Configure the following settings in sshd_config (this enables an SFTP account and sets the default location as the home path):

$ vi /etc/ssh/sshd_config
...
# override default of no subsystems
Subsystem       sftp    internal-sftp...

Match Group sftpusers
    ChrootDirectory /home/%u
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp

Restart SSH server to apply changes:

$ systemctl restart ssh

Create an SFTP user account for a facility and place in a folder on the home path to receive data:

# set new user name
sftpuser=the-new-username
useradd $sftpuser
usermod -g sftpusers -s /usr/sbin/nologin $sftpuser
mkdir -p /home/$sftpuser/INPUT_PATH/
chown -R root:root /home/$sftpuser

Mount multiple accounts to one account

The goal here is to point the data from many facilities to one location, but using a single account and path for multiple sites’ data could result in a breach of security and/or privacy. Mounting the receiving path of a facility’s data onto a single master account and then to a “mount point” with a unique facility name takes care of this issue. The process next consolidates files from individual paths on a master account in one place where the application picks up messages for further processing.

The SFTP accounts and the master account should be attached to the same group. This will permit individual accounts to write on the master account-mounted path. In turn, the master account can read files from the same location. This location now has administrative rights for both the SFTP user the group. Group permission of the mounted folder is set to sftpgroup and user permission is set to the facility account.

# create master user account
$ adduser master
$ passwd master

# Add master user to sftpgroup group
usermod -a -G sftpgroup master
getent group sftpgroup

# MOUNT_PATH and sub folders
mkdir -p /home/master/MOUNT_PATH/{Input,Pickup,Backup,Archive}
chown -R master:master /home/master/MOUNT_PATH

We wrote a script to automate the process of creating an SFTP account, mounting it at the master account path, and adding fstab entries to save the mount in the case of a reboot. The script not only saves time, but also avoids human error when creating accounts for all facilities.

#!/bin/bash

# ./create_sftp_account.sh sftpfacilityone FACILITY_ONE

sftpuser=$1
facility_name=$2

# Create SFTP account and add to sftpgroup
useradd $sftpuser
usermod -g sftpusers -s /usr/sbin/nologin $sftpuser
usermod -a -G sftpgroup $sftpuser
mkdir -p /home/$sftpuser/INPUT_PATH
chown root:root /home/$sftpuser

# Create path specific to facility in master account Input folder
mkdir -p /home/master/MOUNT_PATH/Input/$facility_name
chown -R $sftpuser:sftpgroup /home/master/MOUNT_PATH/Input/$facility_name

# Mount sftp account into master account and set permissions
mount --bind /home/master/MOUNT_PATH/Input/$facility_name /home/$sftpuser/INPUT_PATH
chown -R $sftpuser:sftpgroup /home/$sftpuser/INPUT_PATH
chmod g=rwxs /home/$sftpuser/INPUT_PATH

# Add fstab entry to persist and mount on reboot
echo "/home/master/MOUNT_PATH/Input/$facility_name /home/$sftpuser/INPUT_PATH none bind 0 0" >> /etc/fstab
echo "Created user $sftpuser at $facility_name mount point successfully"

Files at one location

Now, data files from facilities are available at individual folders under MOUNT_PATH/Input on the master account. This enables third-party applications to pick up files in a straightforward way to proceed with further processing. It also helps our client access the files for review from the master account easily without navigating into each separate account.

Summary

Mounting multiple SFTP accounts onto one master account turns out to be an efficient and beneficial method of consolidating data. Both safe and secure, running separate SFTP accounts establishes an exclusive private link between facilities and servers. The master account has the unique ability to access files belonging to each facility in order to process the data further.

Tip: In order to avoid broken mounts, check the status by using the command mount -fav. Problems with mount configurations can cause broken mounts after rebooting the server.

Cooking with CAS

2020-03-10T00:00:00+00:00

Photo by Flickr user reidrac, licensed under CC BY-SA 2.0

One of our customers asked us to host a new suite of web-based applications for them and to protect them with a single sign-on (SSO) solution. Ok, easy enough; these applications were in fact designed with a particular SSO system in mind already, but our situation required a different one, and we eventually chose Apereo’s open source Central Authentication Server project, or CAS. I’d like to describe the conversion process we went through.

The ingredients

Our customer’s application suite included:

The principal Java application using JAAS authentication
Another Java application based on Spring Security
A pair of PHP applications
A few automated tasks that needed to authenticate.

The original SSO system sets a header on each request, identifying an authenticated user. This requires a gateway system to sanitize request headers to ensure malicious users cannot forge a header themselves. It also requires each application inspect request headers and respond appropriately.

CAS is a bit more complex: applications redirect unauthenticated requests to a CAS server, which authenticates the user through any of various configurable methods. The CAS server then redirects the user back to the original application with a parameter called a “Service Ticket”, a seemingly random number identifying an individual authentication request. The original application contacts the CAS server directly to validate the service ticket and to collect information to identify the user. It can then establish a session for that user, and proceed normally.

To CAS-enable an application, we incorporate one of the CAS client libraries, which exist for various languages. In fact we won’t use the Java client directly, but rather we’ll incorporate components that extend it. When evaluating CAS, I was a bit concerned by what appeared to be a surprisingly limited selection of actively supported client libraries, and of course your results may vary, but we found software to meet our own needs without too much difficulty.

Configuring Wildfly Authentication

The most important application in the suite depends on the JAAS-based security subsystem of the Wildfly application server it’s deployed to. Originally it used a custom LoginModule that inspected request headers to find the ID of the authenticated user. Our first task was to configure our proxy server to remove this header from every request. We planned to disable the old authentication system entirely, of course, but this change ensured that even if we mistakenly left it enabled somewhere, malicious users couldn’t exploit it for access.

This application uses a declarative security policy: the deployment descriptor identifies a set of user roles, and another set of “security constraints”. Each security constraint describes one or more URL patterns used in the application, and the set of user roles allowed to access URLs matching those patterns. Here are two examples:

<!-- clients -->
<security-constraint>
    <web-resource-collection>
        <web-resource-name>Secure Area</web-resource-name>
        <url-pattern>/views/clients/*</url-pattern>
        <url-pattern>/client/edit/*</url-pattern>
        <url-pattern>/client/edit_tree/*</url-pattern>
        <url-pattern>/client/view/*</url-pattern>
        <url-pattern>/client/view/id/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <role-name>client_role</role-name>
    </auth-constraint>
</security-constraint>

<!-- places -->
<security-constraint>
    <web-resource-collection>
        <web-resource-name>Secure Area</web-resource-name>
        <url-pattern>/views/places/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <role-name>manage_places</role-name>
    </auth-constraint>
</security-constraint>

We installed the cas-extension library in our Wildfly server to handle the CAS protocol. When an unauthenticated user attempts to access a URL matching a pattern in one of the application’s security constraints, the CAS extension automatically intercepts control and redirects the user to the CAS server. Assuming the user authenticates successfully, our application will receive another request with a service ticket parameter. The cas-extension intercepts this request as well, validates the service ticket, and creates an “identity assertion”, which it sends to the Wildfly security system. Wildfly’s role mapper queries a database to find the user’s roles, after which the authentication process is complete.

Configuration of the cas-extension begins with a CAS profile, a combination of the URL of the CAS server and the URL of the service the extension should protect.

<subsystem xmlns="urn:soulwing.org:cas:1.0">
    <cas-profile name="default" service-url="https://our.application.server/application" server-url="https://our.cas.server"/>
</subsystem>

This tells the extension where to send authentication requests, where to listen for requests returning from the CAS server, and where to validate service tickets. Next we need to validate that identity assertion, and figure out what roles belong to the user. This happens in a Wildfly security domain.

<security-domain name="MySecurityDomain">
    <authentication>
        <login-module name="IdentityAssertion" code="org.soulwing.cas.jaas.IdentityAssertionLoginModule" flag="required" module="org.soulwing.cas"/>
    </authentication>
    <mapping>
        <mapping-module code="DatabaseRoles" type="role">
            <module-option name="dsJndiName" value="java:/comp/env/jdbc/databaseConnection"/>
            <module-option name="rolesQuery" value="select role from user_roles where user_id = ?"/>
        </mapping-module>
    </mapping>
</security-domain>

The authentication portion of the security domain refers to a JAAS LoginModule shipped with the CAS extension, which simply verifies that the identity assertion comes from the CAS extension and not somewhere else. Then the mapping portion (documented here) looks up the given user in a database to find what roles it should be assigned.

The last piece of the puzzle is a CAS deployment descriptor for our application, which activates cas-extension for that application. In its simplest form, this is an empty file in the right place, but ours ended up a little more complex. It identifies both the CAS profile we want to use (unnecessary in this case, as there’s only one CAS profile on the system, but it helped keep things more clear in our minds), and instructs the extension to load some other libraries into our application.

<?xml version="1.0" encoding="UTF-8"?>
<cas xmlns="urn:soulwing.org:cas:1.0">
  <profile>default</profile>
  <add-api-dependencies/>
</cas>

This configuration proved sufficient to let users log in and use the application, but as is sometimes the case with single sign-on, we needed a little more work to let them log out properly. Each application in the suite sets a cookie in the user’s browser to identify its session. The CAS server likewise sets a cookie. When a user logs out of the application, that application’s session cookie is destroyed, but we also need to destroy the CAS server’s session cookie as well as the other applications’ cookies. Single log-out can be complicated, and I won’t go into the full setup here. One reason the CAS deployment descriptor loads cas-extension libraries was so we could use cas-extension to generate the proper logout URL, and redirect our users to that URL once the application has destroyed its own session.

Configuring Spring Authentication

Another Java-based application in our suite uses Spring Security. This document describes the bulk of the configuration, which seemed less straightforward than for cas-extension, but follows essentially the same mechanism. Here we configured a Spring UserDetailsService to execute the same database query we used above to find the user’s roles. I’m not fully conversant in the large stack of beans Spring uses to manage the process, and it took some time to get this configuration sorted out.

Enter PHP

Two of these applications use PHP, which meant yet another configuration. Apereo maintains a PHP client, which includes several helpful examples. I tracked down the part of the application that authenticates users, and replaced the existing code with calls to phpCAS:

phpCAS::client(CAS_VERSION_2_0, $phpCASHost, $phpCASPort, $phpCASContext);
phpCAS::forceAuthentication();
$_SESSION[EXPORT_SERVERNAME]['umdid'] = phpCAS::getUser();

The forceAuthentication call determines the current phase of the authentication process this request is in, whether it’s unauthenticated, fully authenticated, or requires service ticket validation, and responds appropriately. We then set a session variable to the ID of the authenticated user, which replicates what the original authentication code would have done.

One of these two applications requires authenticated access to a REST API exposed by the Wildfly application. CAS calls this “proxy” authentication, when one application requests access to another. Here, CAS issues not only its usual service ticket, but also a “proxy granting ticket”. When the application wants to use the API, it asks the CAS server for a service ticket, using the proxy granting ticket. The CAS server itself requires some new configuration in this case, but for the PHP code, the only difference in the login phase from the simpler, non-proxy case is that we call phpCAS::proxy instead of phpCAS::client to configure CAS. Later, when calling the service itself, we used more phpCAS services in place of the cURL library the original used.

$service = \phpCAS::getProxiedService(PHPCAS_PROXIED_SERVICE_HTTP_GET);
$service->setUrl($serviceURL);
$service->send();

The return of header authentication

Finally, we have a few automated tasks which use various APIs, and need to authenticate. We can’t redirect requests to a CAS server and expect a user to provide credentials, so we’ve taken our cue from the applications’ original form, and configured CAS to recognize a “trusted header”. We add this header to any requests issued by these automated jobs. Of course, we’ve also configured the proxy to disallow this header from any systems outside our internal network.

A few loose ends

Of course, there were other considerations in this project that I’ve not covered here. Configuring CAS itself wasn’t necessarily straightforward, and included a custom authentication module I hope to describe in a later article. Selecting among CAS server’s available deployment options and fitting the winner into our existing infrastructure in a way that makes it easy to manage and monitor was another task entirely. We needed to customize the server’s default user interface to prevent it from offering users an imaginary method to recover forgotten passwords. The series of redirected browser requests involved in the CAS protocol presents a notable performance impact under some circumstances. And it has taken me no small effort to learn to appreciate the CAS server’s sometimes distressingly circular documentation. But several months into the project, CAS seems to be working well enough for our purposes that other users of the same application suite have begun to express interest.

End Point Security Tips: Securing your Infrastructure

2020-02-05T00:00:00+00:00

Photo from comparitech.com

Implement Security Measures to Protect Your Organization & Employees

In this post, I’ll address what I believe are the three important initiatives every organization should implement to protect your organization and employees:

Train employees on security culture.
Implement the best technical tools to aid with organizational security.
Implement recovery tools in case you need to recover from a security breach.

Habits of a Security Culture

Train everyone in your organization on these fundamentals:

The only time you should be requested to reset your password by email is when you initiate it. There are rare exceptions to this rule, such as when accounts are compromised and providers request all users reset their passwords, but those events should be publicly announced. Staff can confirm with security personnel before acting on such requests.
If you are asked to reset your password, it will typically be after you successfully logged into a website and the old one has expired.
If you receive an email and do not know the sender, do not trust the contents or open attachments. Get advice from security personnel if needed.
If you think the email is from your bank, keep in mind that banks do not ask their clients for private information via email.
If you think the social security office emailed you to obtain your personal information, keep in mind that they do not initiate or solicit private information via email.
Companies should not solicit private information unless you initiate first.
Online retailers should not ask for your private information unless you initiate first.

A Security Concern: Going Phishing!

Photo by Epic Top 10 Site

One of the more common ways to steal someone’s private information is through phishing. Phishing is like fishing: you try to catch something. In this case, the ‘fish’ is your data. Someone with malicious intent sends you email to attempt to get you to click on the link, picture, content, etc. within the email. Once you click the link or content within the email, it might take you to a website to enter or reset your password, even ask for your social security number or other personal information. The person with malicious intent would use the information collect to open accounts, purchase items online or resell your personal data. The links within the phishing email might even redirect you to a fake website that mimics a real website to collect your personal data. The goal is to confuse the email recipient into believing that the email message is legit in its attempt to collect personal information from the user.

Phishing Exercises

Photo by Christiaan Colen

One way to help the staff better their understanding of a phishing email is to practice phishing exercises by setting up an experiment to see which users would click on your test phishing email. If the user clicks on the experiment phishing email, the email administrator would notify the compliance officer and he or she would re-train the staff on how to properly differentiate real emails from phishing emails. Phishing exercises make a great activity for the employees to avoid the email scams and exert more caution in the future.

Essential Security Tools

Firewall

A firewall should be a mandatory technology for all consumers or businesses. This is basically your main door. The office or organization’s technology is what the firewall is protecting. When you sit in your office working on the computer, just imagine that you are in a fort surrounded by walls. If someone needs to come in, they must be given permission by a gatekeeper to enter the fort. The firewall is very similar. The firewall allows network traffic to go in and out of the office based on the configuration set by a network engineer. The engineer determines what network traffic comes into the office and goes out.

Companies typically test their firewall with penetration tests and network scans to determine if there are any security concerns after implementing the firewall. The testing of the firewall verifies that good security practices are implemented and the firewall is setup properly and securely.

Some hardware firewall devices to consider include:

Network Assessment & Internal Threat Protection

Why is security vulnerability testing necessary? Many organizations have legacy servers, or desktops/laptops with operating systems that are no longer supported. For example, outdated Microsoft Windows XP and 7 can be compromised by malware while browsing the Internet due to the BlueKeep vulnerability. Systems that are not patched could expose your organization to be infected with malware such as ransomware which would hold your data ransom until you pay to unlock the data.

Security vulnerability testing typically results in a report outlining problematic areas such as outdated operating systems, private data that does not belong on a file server, such as social security or credit card number (personally identifying information, or PII). If PII is needed for the organization to operate, then higher security standards and an encryption system to store the private data is needed. The security vulnerabilities testing could also check if your environment is HIPAA or PCI DSS compliant if those security standards apply to you.

Some security vulnerability testing technology includes:

Enterprise Antivirus Systems

Antivirus software is the last line of defense if malware enters your computer. The antivirus itself would not protect you 100% from being infected by malware or virus but if you have multiple layers of security in place, then the chances are much lower that your organization’s systems would not be compromised. A company called AV-Comparatives assessed some of the popular antivirus software in the market.

The battle with malware is endless. Case in point: The WannaCry malware affected over 200,000 machines across the world and spread quickly. Security researchers quickly realized the malware was spreading like a computer worm, across computers and over the network, using the Windows SMB protocol.

New variants of viruses and malware are developed every day, so the antivirus companies are also daily hard at work developing a way to block and remove malware.

Some antivirus software includes:

Web Filtering Technology

Web filtering technology blocks websites that are malicious or deemed not appropriate to visit from an organization’s network. For example, websites for gambling could be blocked to reduce employee distractions, but also to reduce access to sites popularly infected with malware, reducing the possibility of malware coming into your network.

There are many competitors out there in this competitive market, and some vendors offer free proof-of-concept testing with their product before you make a big investment. Take a look at:

Data Loss Prevention

Employees’ and businesses’ private information is sensitive and should be protected. Businesses, whether audited or not, should always protect their employee private information. 20 years ago paper was used to store private information and locked in a file cabinet, but in 2020 most private information is stored digitally. How do companies keep private information from leaving their office?

Physically you probably can’t stop someone from walking out with private information, but digitally there is technology called digital loss prevention (DLP) that can help keep confidential information from leaving the office. For example, if someone in the office decides to copy private information onto USB storage, or tries to send a social security number or credit card information via email, this can often be prevented. Prepare your business with solutions tailored for regulations involving personal information using DLP software.

Some DLP solutions to consider include:

Email Filtering

Probably one of the easiest and oldest methods to infect or phish someone is via email. There are multiple mechanisms email filtering uses to stop malicious emails: SPF and DKIM, blacklists, etc. On top of these configurable items, email filtering software vendors often release updates throughout the day to block the latest known malware or spam based on heuristics.

Cloud email services such as Microsoft Office 365 are for many businesses superior to on-premise email servers due to having all the bells and whistles to proactively protect your email environment, such as spam & virus filtering, and email archiving for retention purposes. If you need email encryption to protect sensitive emails, this feature is also available.

The distinct advantage of using hosted email service is that the cost is predictable and includes maintenance, so your email administrators do not have to worry about updating the email server software or hardware.

Email filtering technology available includes:

Two Factor Authentication (2FA)

Companies like Duo (owned by Cisco) and Google’s two-factor authentication technology are great tools to implement to improve your overall security. Beyond the usual user name and password, your smartphone or a hardware token are used to authorize the access to a website, an application, or a network. This technology is now well-established and available in most online services, and can be added to your own custom business applications.

Some starting points:

VPN (Virtual Private Network)

Virtual private network technology allows businesses to provide secure access to office systems or applications to employees who travel or work from a remote location. That is important so that data traveling to and from the source, and many details of traffic patterns, are encrypted, so that the data cannot be captured and decrypted. VPN solutions have been around for years and are an important tool to securely and safely protect data in transit.

Password Reset Systems

Why are self-service password reset systems necessary, and are they secure? They eliminate many manual password assignment mistakes, keep passwords private to the user alone, and allow an organization to integrate two-factor authentication to securely reset user passwords, such as Active Directory for Windows, or other single sign-on password. Onboarding is needed for each user, which is done by sending them an email with a link to register.

The password reset system could be an internal system only available to your organization. Another possibility is to access the system via VPN or even through a proxy server in the DMZ to allow password reset from anywhere.

Some password reset systems include:

Recovery Tools

Data and System Backups/Disaster Recovery

Data backup and disaster recovery practice is a critical component for a business to speed up the process of recovery if malware attacks your systems or if a system becomes inoperable.

If you are attacked by ransomware and all the critical systems and desktops are compromised and rendered useless, then the only way to recover is from backups. At that point, the disaster recovery preparation you made will be well worth what you invested into it. If you did not have an adequate backup or disaster recovery plan, your organization or company would have to start from scratch and rebuild all systems and desktops which could take weeks if not months to recover from. The lost wages, and possibly clients, due to unavailable systems and desktop to operate, probably would cost you far more than preparation does. The more options you have, the better the chances that you will get out of your bind and not prolong the situation.

Some backup solutions I have worked with have the flexibility to mix and match to fit your configuration needs:

Acronis Cloud Backup allows you to back up to local storage, the cloud, and off-premise, recover a system using a USB drive, back up Office 365 emails, or backup VMware or Hyper-V environments. It also allows you to recover a single file if needed.

Additional Security Recommendations

SSL/TLS Certificates

What are SSL and TLS? SSL is outdated and replaced by TLS, but people often still use the familiar SSL name. They are an encryption system to keep sensitive information sent across the Internet encrypted so that only the intended recipient can access it.

When an SSL certificate is validated, the transmitted data is not only unreadable by anyone except for the intended server, but you are relatively well assured you are communicating with the organization you expected and not with a malicious intermediary. Read more details in SSL Shopper’s Why SSL? article.

SSL certificates were historically used primarily for higher-security web servers, but now are commonly used on almost all web servers. They are also used for other remote server access, B2B server-to-server communication, VPN access, email systems, etc.

Recent Destructive Incidents

These incidents from just a few weeks in fall 2019 show that we have a long way to go in protecting our technology usage:

Speak With Us!

Keeping up with security can be a full-time job. If you need professional consulting on security tools and implementing them, contact our team today.

SCAM ALERT: EndPoint Petroleum Corporation, P&Z Petroleum

2019-06-22T00:00:00+00:00

Photo by Jefferson Santos on Unsplash

This blog post is to alert the public, targeted individuals, and relevant authorities about a scam being perpetrated under the guise of a fictitious company called “EndPoint Petroleum Corporation”. The scammers have set up a website for this fake company, copying much of the content from Noble Energy. The scammers have adapted an old version of our logo for their nefarious purposes.

We will refrain from posting the URL for the scammers’ website since we don’t want to increase its SEO ranking, but it is shown in the image capture above. At first glance this website looks legitimate, but closer observation provides abundant evidence to the contrary. Rather than posting a list of its defects which would assist the scammers, I urge the interested reader to spend five minutes on the website.

An email forwarded to us from one of their targets provides an example of how the scammers are trying to use their website. They are targeting people outside the US with experience in the Oil and Gas Industry, telling them that they have been “shortlisted” for an Operations Manager position with the bogus company. The scammers then tell their targets to contact their “required travel agency” to make arrangements for a trip to the United States for an in-person interview.

We know that the scammers are using LinkedIn to contact their targets. They may also be contacting potential targets via other means as well. And it stands to reason that they may have posted their fake job offer on one or more job boards.

Trying to Take This Down

We have contacted and followed up with various authorities and organizations about this scam. We are going to continue our efforts in this regard. We welcome any assistance or advice that anyone can offer that would help with this. Please use our contact form to contact us.

Sample Scam Letters and Job Description

Below are several attachments that have been forwarded to us by one of their intended victims:

December 2019 Update: P&Z Petroleum

Several potential victims have informed us that the scammers have begun using another name, P&Z Petroleum or PZ Petroleum, signing letters with:

Mr. Gary Mason
Human Resource Manager
3422 San Felipe Street
Houston, TX 77056-2723
United States
hr.department@pzpetroleum.com

Contact Person: Mr. Kris Clark
STAR TRAVELS AGENCY
2400 Augusta Dr
Houston, TX 77057, USA
Phone Number: +1 832-861-1505
Email: kris.clark@startravelsagency.com

They are also trying to lure medical doctors and nurses, in addition to various management and engineering roles.

If they have contacted you, run away and report them to the authorities!

OWASP Top Ten Application Security Risks

2019-02-27T00:00:00+00:00

I don’t consider myself a security expert. Still, to my surprise, I was asked to give a talk about security to all the End Point developers. Obviously I realized too late what I was getting myself into! Such an audience is not only pretty large, it is also challenging, many are more competent than me, and the risk to bore them is very high. Yet, the slides were prepared, the talk was given and the feedback was good.

It goes without saying that a broad, generic training about security, which still can give something to the listener, can’t be really improvised.

The platform for the talk was the OWASP Top Ten 2017 Project, which discusses the most critical security risks to web applications.

OWASP stands for Open Web Application Security Project and describes itself as “an open community dedicated to enabling organizations to develop, purchase, and maintain applications and APIs that can be trusted.” Its website provides plenty of resources to the developers.

The Top Ten consists of 10 broad classes of vulnerabilities. The data behind that comes from specialized firms and surveys, gathering information from 100,000 real-world applications and APIs. Some of these classes are very broad and have a taxonomic value. Saying, for example, “Broken access control” or “Security misconfiguration”, doesn’t say much what you are doing wrong. It still reminds us that a lot of things can actually go wrong and that when working on a service you should ask yourself how are you doing with regard to security.

The current (2017) Top Ten classes are:

Injection
Broken Authentication
Sensitive Data Exposure
XML External Entities (XXE)
Broken Access Control
Security Misconfiguration
Cross-Site Scripting (XSS)
Insecure Deserialization
Using Components with Known Vulnerabilities
Insufficient Logging & Monitoring

I’m not going over the Top Ten in detail like I did at the training, but I’m going to leave here a couple of brief notes.

Injections: in my naivety I was convinced that this class of vulnerability, which is very specific and whose fix is well-known, was something belonging to the past, to legacy and neglected applications. Still, it’s not so. It’s 2019 and injections are still winning.

If by chance you are still interpolating variables inside the SQL and not using placeholders, it’s time to take a look at Bobby Tables (or equivalent) and start doing it right, as every major language has a library supporting them. Maybe you avoid this vulnerability in your own code, but have inherited code from others who did not defend against it. Look and see!

Broken Authentication: On the other hand, finding the authentication problems in second place was not a surprise at all, given the frequency of leaked databases. The recommended practice appears to be checking the password against a list of known passwords and ensuring it has a decent length. (Please note: no requiring regular password rotation, nor uppercase/lowercase/digit enforcing.)

XSS: For web developers an interesting class is the ubiquitous Cross-Site Scripting (XSS) vulnerability, which is found in two-thirds of all applications. Statistically speaking, it’s probable that the application you are currently working on is affected.

XSS comes in many flavors, usually generated server-side, when a variable coming from user input is interpolated without proper escaping into the HTML, opening the door to JavaScript execution, but DOM XSS is also common. How many times did you see something like this, for example, with jQuery:

$('#app').html(string_from_user)

The html() method stuffs whatever string is provided by the user into the HTML page. This should be rewritten as:

$('#app').text(string_from_user)

If formatting is required, the append() method is your friend.

CSRF: Interestingly enough, Cross Site Request Forgery (when an application doesn’t check if a POST request comes from the site itself and not from a random site on the internet which is fooling the authenticated browser to perform an operation on your site) didn’t make into the Top Ten. This is probably due to the fact that mainstream frameworks like Ruby on Rails and Django come with CSRF protections almost out of the box.

The goal of the OWASP document is to increase the awareness about security problem. We are trying to do our part to prevent security problems at End Point.

Where are you with your Windows OS in 2019?

2019-02-12T00:00:00+00:00

Photo by bradleypjohnson · CC BY 2.0

It should be of little surprise that on January 14, 2020, after a decade of Windows 7, Microsoft will stop providing security updates and support for this older operating system. Windows 7 was released in 2009, and due to its stability enjoyed many years as the go-to operating system for home and business alike.

Even now, it is estimated by NetMarketShare that over 40% of businesses still rely on it. Despite Microsoft having ended mainstream support for Windows 7 in 2015, it still offered extended support because of the operating system’s popularity, and the generally slow adoption of newer releases. However, that support shall soon end, as will support for Windows Server 2008 R2 (release 2), which also remains in wide use. Organizations of all kinds will need to upgrade to newer operating systems to remain secure.

The corporate adoption of Windows 8 and 8.1 may have been slow in part due to Microsoft’s radical changes to the user interface, such as replacing navigation menus with information-filled “live” tiles. Windows 10, however, was designed as a compromise, providing a Windows 7-like Start menu, while preserving the live tile interface for accessing applications and services from the desktop. There are many compelling reasons for moving from Windows 7 to Windows 10. The most notable is that Windows 7 will no longer receive any security updates, creating serious security risk and compliance issues.

In 2018, Microsoft adopted a Semi-Annual Channel (SAC) governed by the Modern Lifecycle Policy. This means that feature updates are released to the public twice a year, around March and September. Updates are cumulative. Updating to these releases is required to remain eligible for support from Microsoft. Security updates are released monthly and quarterly as rollups, and critical updates are released as needed. This is part of Microsoft’s plan to minimize vulnerabilities and security exposure in an ever-changing digital world.

Time to change

At End Point we consider keeping our clients’ systems updated both fundamental and essential. Companies need stable systems in order to operate, and uptime is of critical importance. This is one reason we offer our RMM (Remote Management and Monitoring) services. We connect with our clients’ systems to receive custom-defined policies for patching, then monitor and test updates before we deploy them on the cycle that best suits our customer’s workflow.

We are committed to helping our clients transition safely and efficiently to Windows 10. It can be a lengthy process when accounting for the time it takes to properly test all business applications and processes before transitioning to a new OS, and the time it takes to make any necessary hardware changes and complete the update.

See the charts below to learn when service will end for various versions of Windows, and contact us for a consultation on the best ways to manage your IT to keep your organization secure.

Windows 10

OS version	Date of availability	End of service for Home, Pro, and Pro for Workstation Editions	End of service for Enterprise and Education Editions
Windows 10, version 1809	November 13, 2018	May 12, 2020	May 11, 2021
Windows 10, version 1803	April 30, 2018	November 12, 2019	November 10, 2020
Windows 10, version 1709	October 17, 2017	April 9, 2019	April 14, 2020
Windows 10, version 1703	April 5, 2017	October 9, 2018	October 8, 2019
Windows 10, version 1607	August 2, 2016	April 10, 2018	April 9, 2019
Windows 10, version 1511	November 10, 2015	October 10, 2017	October 10, 2017
Windows 10, released July 2015 (version 1507)	July 29, 2015	May 9, 2017	May 9, 2017

Enterprise LTSC/LTSB Editions

Windows 10 LTSC/LTSB editions also follow the Fixed Lifecycle policy. To learn more, see Microsoft Business, Developer and Desktop Operating Systems Policy.

OS version	Date of availability	Mainstream support end date	Extended support end date
Windows 10 Enterprise LTSC 2019 / Windows 10 IoT Enterprise LTSC 2019	November 13, 2018	January 9, 2024	January 9, 2029
Windows 10 Enterprise 2016 LTSB / Windows 10 IoT Enterprise 2016 LTSB	August 2, 2016	October 12, 2021	October 13, 2026
Windows 10 Enterprise 2015 LTSB / Windows 10 IoT Enterprise 2015 LTSB	July 29, 2015	October 13, 2020	October 14, 2025

Note: Not all features in an update will work on all devices. A device may not be able to receive updates if the device hardware is incompatible, lacks current drivers, or is otherwise outside the original equipment manufacturer’s (OEM) support period.

For more information on the Windows 10 lifecycle, see Windows 10 Client and Windows Server Semi-Annual Channel Lifecycle Policy update (February 1, 2018) or the Windows 10 release information page. To learn more about the Windows 10 mobile lifecycle, see Windows 10 Mobile.

Windows 8.1 and 7

Prior releases of the Windows operating system are likewise governed by the Fixed Lifecycle Policy. This policy comprises two phases: mainstream support and extended support. See Microsoft Business, Developer and Desktop Operating Systems Policy for more details.

OS version	End of mainstream support	End of extended support
Windows 8.1	January 9, 2018	January 10, 2023
Windows 7, service pack 1*	January 13, 2015	January 14, 2020

* Support for Windows 7 RTM without service packs ended on April 9, 2013. Be sure to install Windows 7 Service Pack 1 to continue to receive support and updates.

Prior versions of Windows, including Windows 7 and Windows 8.1, have limited support when running on new processors and chipsets from manufacturers like Intel, AMD, Nvidia, and Qualcomm. For more information, see the Microsoft Lifecycle Policy. A device may not be able to run prior versions of Windows if the device hardware is incompatible, lacks current drivers, or is otherwise outside the original equipment manufacturer’s (OEM) support period.

(Windows life cycle charts and information were taken from Microsoft’s windows support site.)

Introducing EP Audit

The Problem

What EP Audit Looks Like

A Service, Not Just a Tool

Phased Remediation

Working with End Point

Risks of Paper Checks for Secure Transactions

How to Analyze Application Logs and Extract Actionable Insights

A Note on Commands

1. Debugging Application Errors

1.1 Summarize Frequent Issues

1.2 Categorize Errors by Type

1.3 Find Problematic Scripts

2. Monitoring System Behavior

2.1 Track Problematic IPs

2.2 Spot High-Error Time Periods

2.3 Analyze IP–Referer Pairs

3. Enhancing Security

3.1 Watch Sensitive URLs

3.2 Pinpoint Critical Times

Conclusion

Streamlining SELinux Policies: From Policy Modules to Modules and Silent SELinux Denials

Introduction

What is a policy_module?

What is a module?

Converting policy_module to module

The silent blocks of SELinux

Troubleshooting silent blocks

Secure Your Dockerized Nginx with Let's Encrypt SSL Certificates

1. Domain validation

2. Get the acme.sh

3. Set the CA

4. Issue the certificate

5. Install the certificate

6. Restart the Nginx container

7. Certificate Renewal Mechanism

Google Chrome Yum/​RPM package update fails on RHEL/​CentOS 7

Sweatin’ to the oldies

References

Identifying Vulnerabilities in Code Using Horusec

Horusec CLI Installation

VS Code Extension

Usage

Classification of Vulnerabilities

Ignore Files

Code Scanning in EpiTrax

ssh-askpass on macOS for SSH agent confirmation

How it works

Installing ssh-askpass on macOS

Install ssh-askpass with Homebrew

Install ssh-askpass with MacPorts

Install ssh-askpass from source code

Configure the SSH agent with the ssh-add -c option

ssh-askpass agent confirmation

Set up keyboard shortcuts

macOS 10.15 Catalina, 11 Big Sur, 12 Monterey

macOS 13 Ventura

Vulnerability Scanning

Define Your Terms

The Attackers

Main Security Vulnerability Categories

Solutions

References

Implementing Authentication in ASP.NET Core Web APIs

Two approaches to authentication: JWT and API Keys

Managing user accounts with ASP.NET Core Identity

Install the necessary NuGet packages

Update the DbContext to include the Identity tables

Applying database changes

Configuring the Identity services

Creating users

Fetching users

Implementing JWT Bearer Token authentication

Creating tokens

The request data structure

The response data structure

The class that creates JWTs

The action method that puts it all together

Securing an endpoint with JWT authentication

Applying the Authorize attribute

Google Chrome Yum/RPM package update fails on RHEL/CentOS 7

Configure the SSH agent with the `ssh-add -c` option

Be prepared—encrypt your disk!