Understanding the Relationship Between Apex and Salesforce

2025-05-02T00:00:00+00:00

Salesforce has become a cornerstone for businesses looking to streamline their customer relationship management. At the heart of its customization capabilities lies Apex, a powerful programming language designed specifically for the Salesforce platform and based on Java syntax. In this post, we’ll explore the relationship between Apex and Salesforce and touch on how Apex differs from the general-purpose language Java.

Apex and Salesforce

Apex is Salesforce’s proprietary programming language, introduced to extend the platform’s functionality beyond its out-of-the-box features. Think of Salesforce as a robust cloud-based ecosystem with pre-built tools for managing leads, contacts, opportunities, etc. While these tools are configurable through point-and-click interfaces, there are times when businesses need custom solutions. That’s where Apex comes in.

Apex runs natively on Salesforce’s platform. This tight integration allows Apex to interact seamlessly with Salesforce data (like objects and fields), execute triggers based on record changes, and build custom APIs. For example, a developer might use Apex to automate a discount approval process or sync Salesforce with an ERP system — all without leaving the Salesforce ecosystem.

The relationship is symbiotic: Salesforce provides the infrastructure (data models, security, scalability), and Apex empowers developers to tailor it. Apex is built exclusively for Salesforce, meaning it can only be compiled and run on Salesforce servers. That also means it has optimizations for rapid development within the platform’s guardrails, such as governor limits that ensure resource fairness in a multi-tenant setup.

Apex vs. Java: Key Differences

Apex is based on Java but not built off it. The style of programming for Apex is quite different as it’s more similar to web development than standalone application development.

Here’s a quick comparison, including examples:

Syntax: Although quite similar, there are distinct differences in syntax. Here’s an example of a simple for loop written in both:

Java:

for (int i = 1; i <= 5; i++) {
    System.out.println("Number: " + i);
}

Apex:

for (Integer i = 1; i <= 5; i++) {
    System.debug('Number: ' + i);
}

Purpose and Environment: Java is a general-purpose language, used everywhere from mobile apps to enterprise applications. Apex, by contrast, is Salesforce-specific, designed to operate within the SalesForce platform’s constraints and leverage its metadata-driven architecture.
Execution Context: Java runs on the Java Virtual Machine (JVM), which you can deploy anywhere with a compatible runtime. Apex runs exclusively on Salesforce’s servers. This removes the need for infrastructure management by tying it to the platform, but also restricting its usefulness only to Salesforce organizations.
Database Interaction: In Java, you use libraries like JDBC to connect to databases. Apex has built-in access to Salesforce objects (e.g., Account, Contact) via SOQL (Salesforce Object Query Language), making data manipulation more declarative and less boilerplate-heavy than Java’s approach. Here’s an example of a DB call to fetch accounts with revenue over $1M:

Java:
```
PreparedStatement stmt = conn.prepareStatement("SELECT name, revenue FROM accounts WHERE revenue > 1000000");
```
Apex:
```
List<Account> accounts = [SELECT Name, AnnualRevenue FROM Account WHERE AnnualRevenue > 1000000];
```
Governor Limits: Apex operates under strict governor limits such as caps on CPU time or query rows to maintain performance in a shared environment. Java has no such restrictions, giving developers more freedom but also more responsibility.
Learning Curve: If you know Java, Apex feels familiar. Classes, methods, and loops are quite similar. However, Apex’s Salesforce-specific features like triggers and Visualforce require specific platform knowledge. On the other hand Java’s versatility demands broader programming expertise.

Why It Matters

The Apex-Salesforce relationship enables businesses to go beyond configuration and into customization, while staying within a secure, scalable cloud platform. While Java requires you to build much of the scaffolding yourself, Apex provides a shortcut to customization within Salesforce, at the cost of some flexibility.

How to Migrate from Struts 2 to Struts 6

2025-04-17T00:00:00+00:00

With the introduction of Struts 6, developers are provided enhanced features, security improvements, and modern practices that align with contemporary Java development. If you’re currently using Struts 2, migrating to Struts 6 is a worthwhile endeavor that can future-proof your application.

Also, since Struts 2.5.x reached its end of life, there won’t be any security updates for this version. This guide will walk you through the key differences between Struts 2 and Struts 6, including some significant changes in Struts 6 with practical configuration and code examples.

Getting Ready for Migration from Struts 2 to Struts 6

Starting from Struts 6.0.0 the framework requires Java 8 at minimum. So, if you are running a Struts 2.x environment on an old version of Java, it needs to be upgraded to Java 8 at least. Check out the Struts 6.0.0 version notes for a list of changes.

Config and Code Changes

Servlet API Dependency

Struts 6.0.0 requires Servlet API 3.1 or newer and won’t work with the older versions. The Maven dependency that you can use for this is below:

<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>javax.servlet-api</artifactId>
    <version>3.1.0</version>
    <scope>provided</scope>
</dependency>

Deprecated Sitegraph Plugin Removed

Support for the sitegraph plugin, which was already deprecated, has now been removed entirely.

Velocity Plugin

Velocity support has been moved into a dedicated plugin. To continue using the Velocity plugin in your Struts application, you can extend the Struts config package definition, adding velocity-default in struts.xml as in the below example:

<package name="mystrutsapp" extends="struts-default, velocity-default">
  ...
</package>

DTD of struts.xml

The proper DTD (Document Type Definition) header of struts.xml should be updated as below:

<!DOCTYPE struts PUBLIC
    "-//Apache Software Foundation//DTD Struts Configuration 6.0//EN"
    "https://struts.apache.org/dtds/struts-6.0.dtd">

Class Changes

There are some class changes you might need to consider.

com.opensymphony.xwork2.config.providers.XmlConfigurationProvider was made abstract. You should now use org.apache.struts2.config.StrutsXmlConfigurationProvider instead.
com.opensymphony.xwork2.conversion.TypeConversionException was replaced byorg.apache.struts2.conversion.TypeConversionException.
com.opensymphony.xwork2.XWorkException was replaced by org.apache.struts2.StrutsException.

Constant Changes

Xwork config constants like devMode were already deprecated and replaced with new constants like struts.devMode. In Struts 6.0.0, the deprecated constants were completely removed. Also, new constants struts.i18n.search.defaultbundles.first and struts.ui.escapeHtmlBody were introduced.

OGNL

Starting from Struts 6.0.0 OGNL expression length is limited to 256 characters by default as longer expressions can be considered harmful. However, the limit can be changed by adjusting the struts.ognl.expressionMaxLength constant in struts.xml.

Static method access is not possible using OGNL expressions like @com.mypack.MyClass@MyStaticMethod(). This usage should be converted to alternative approaches such as Value Stack access through Action methods:

// MyAction.java
public class MyAction extends ActionSupport {

    public String getMyData() {
        return MyStaticClass.getMyStaticData();
    }

    public String execute() {
        return SUCCESS;
    }
}

// MyClass.java
public class MyStaticClass {
    public static String getMyStaticData() {
        return "Data from static method";
    }
}

In your JSP you can access the data using an OGNL expression:

<s:property value="myData" />

HTML Escaping

Struts 6 uses a newer FreeMarker version with auto-escaping enabled by default, affecting how output is rendered. You should avoid manual escaping (e.g., ?html) in FreeMarker templates when migrating to Struts 6.

Struts 2 manual escaping:

${user.name?html}

Struts 6 auto-escaping happens by default, so no ?html escape is necessary:

${user.name}

Tiles Support

Until version 6.3.0, Struts 2 and Struts 6 had been supporting Tiles through Apache Tiles. The Apache Tiles project has been retired and is not updated anymore. The Struts team incorporated the Apache Tiles codebase into Struts and essentially added built-in support for Tiles starting with Struts version 6.3.0 through the new Tiles Plugin.

If you are using Maven you can use the following dependency to setup the Tiles plugin in Struts 6.x:

<dependency>
  <groupId>org.apache.struts</groupId>
  <artifactId>struts2-tiles-plugin</artifactId>
  <version>${version.tiles}</version>
</dependency>

And then register the Tile listener in your web.xml:

<listener>
  <listener-class>org.apache.struts2.tiles.StrutsTilesListener</listener-class>
</listener>

Conclusion

Migrating from Struts 2 to Struts 6 involves several significant changes due to updates in the framework’s architecture, dependencies, and requirements. Struts 6 (starting with version 6.0.0) introduces modernizations to align with newer Java versions, servlet specifications, and security practices.

Getting Output from jps with NRPE

2025-03-22T00:00:00+00:00

One of the tools our hosting team uses for server and site monitoring is Icinga (which is based on Nagios). When monitoring host resources, one of the tools we use is Nagios Remote Plugin Executor or NRPE.

We encountered an issue when executing NRPE: though NRPE runs on the server being monitored, it wasn’t giving the same output as a script which was executed on the server itself. The NRPE-related call should have no issues be executed on the target server, as it is declared in the sudoers file (commonly /etc/sudoers). In this post, I will explain how to get the output from jps (Java Virtual Machine Process Status Tool), which can only be executed as root.

Getting process information with jps

Let’s say we have a “hello world” program named Hello.java. How do we get the process’s state from Icinga’s head server?

First, let’s compile and run the program.

public class Hello {
    public static void main(String[] args) {
        System.out.println("Hello, World!");

        try {
            Thread.sleep(3000);  // Sleep for 3 seconds
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

java Hello.java
Hello, World!

The program will run until it times out.

First, let’s check the process by running ps on the server:

# ps aux | grep Hello.java | grep -v grep
najmi      61609 16.7  0.3 11541184 109360 pts/5 Sl+  01:19   0:00 java Hello.java

We can also get the process ID with jps. Here’s the output from the “najmi” user view:

# sudo -s -u najmi jps -l
61609 jdk.compiler/com.sun.tools.javac.launcher.SourceLauncher
61695 jdk.jcmd/sun.tools.jps.Jps

And from the “root” user view:

# whoami
root
# jps -l
61609 jdk.compiler/com.sun.tools.javac.launcher.SourceLauncher
61852 jdk.jcmd/sun.tools.jps.Jps
# ps aux | grep Hello.java | grep -v grep
najmi      61609  1.2  0.3 11541184 109360 pts/5 Sl+  01:19   0:01 java Hello.java

However, the “nrpe” user does not see it:

# sudo -s -u nrpe jps -l
61790 jdk.jcmd/sun.tools.jps.Jps

Instead of running the jps command directly as Nagios, we will let the system run jps as root and dump the results into a file. An NRPE-based script will later read the output and feed the result to the dashboard.

Running jps as root, with output to a file

For this example, I started the jetty process with sudo.

sudo /usr/share/jetty/bin/jetty.sh start

Now let’s check whether the “jetty” service is running or not.

First we’ll check the status as a non-root user. It sees the process as “not running”.

$ /usr/share/jetty/bin/jetty.sh status
** WARNING: JETTY_LOGS is Deprecated. Please configure logging within the jetty base.
Jetty NOT running

JAVA                  =  /usr/bin/java
JAVA_OPTIONS          =
JETTY_HOME            =  /usr/share/jetty
JETTY_BASE            =  /usr/share/jetty
START_D               =  /usr/share/jetty/start.d
START_INI             =  /usr/share/jetty/start.ini
JETTY_START           =  /usr/share/jetty/start.jar
JETTY_CONF            =  /usr/share/jetty/etc/jetty.conf
JETTY_ARGS            =  jetty.state=/run/jetty/jetty.state jetty.pid=/run/jetty/jetty.pid --module=pid,state
JETTY_RUN             =  /run/jetty
JETTY_PID             =  /run/jetty/jetty.pid
JETTY_START_LOG       =  /run/jetty/jetty-start.log
JETTY_STATE           =  /run/jetty/jetty.state
JETTY_START_TIMEOUT   =  60
JETTY_SYS_PROPS       =
RUN_ARGS              =  -Djava.io.tmpdir=/tmp -Djetty.home=/usr/share/jetty -Djetty.base=/usr/share/jetty --class-path /etc/jetty/resources:/usr/share/jetty/lib/logging/slf4j-api-2.0.16.jar:/usr/share/jetty/lib/logging/jetty-slf4j-impl-12.0.16.jar:/usr/share/jetty/lib/jetty-http-12.0.16.jar:/usr/share/jetty/lib/jetty-server-12.0.16.jar:/usr/share/jetty/lib/jetty-xml-12.0.16.jar:/usr/share/jetty/lib/jetty-util-12.0.16.jar:/usr/share/jetty/lib/jetty-io-12.0.16.jar org.eclipse.jetty.xml.XmlConfiguration java.version=23.0.1 jetty.base=/usr/share/jetty jetty.base.uri=file:///usr/share/jetty jetty.home=/usr/share/jetty jetty.home.uri=file:///usr/share/jetty jetty.pid=/run/jetty/jetty.pid jetty.state=/run/jetty/jetty.state jetty.webapp.addHiddenClasses=org.eclipse.jetty.logging.,file:///usr/share/jetty/lib/logging/,org.slf4j. runtime.feature.alpn=true slf4j.version=2.0.16 /etc/jetty/jetty-bytebufferpool.xml /etc/jetty/jetty-pid.xml /etc/jetty/jetty-threadpool.xml /etc/jetty/jetty.xml /etc/jetty/jetty-state.xml
ID                    =  uid=1000(user) gid=1000(user) groups=1000(user),90(network),98(power),984(users),987(storage),991(lp),994(input),996(audio),998(wheel)
JETTY_USER            =  jetty
USE_START_STOP_DAEMON =  0
START_STOP_DAEMON     =  0

Now, let’s check the status of the jetty service with sudo:

$ sudo /usr/share/jetty/bin/jetty.sh status
** WARNING: JETTY_LOGS is Deprecated. Please configure logging within the jetty base.
Jetty running pid=89969

JAVA                  =  /usr/bin/java
JAVA_OPTIONS          =
JETTY_HOME            =  /usr/share/jetty
JETTY_BASE            =  /usr/share/jetty
START_D               =  /usr/share/jetty/start.d
START_INI             =  /usr/share/jetty/start.ini
JETTY_START           =  /usr/share/jetty/start.jar
JETTY_CONF            =  /usr/share/jetty/etc/jetty.conf
JETTY_ARGS            =  jetty.state=/run/jetty/jetty.state jetty.pid=/run/jetty/jetty.pid --module=pid,state
JETTY_RUN             =  /run/jetty
JETTY_PID             =  /run/jetty/jetty.pid
JETTY_START_LOG       =  /run/jetty/jetty-start.log
JETTY_STATE           =  /run/jetty/jetty.state
JETTY_START_TIMEOUT   =  60
JETTY_SYS_PROPS       =
RUN_ARGS              =  -Djava.io.tmpdir=/tmp -Djetty.home=/usr/share/jetty -Djetty.base=/usr/share/jetty --class-path /etc/jetty/resources:/usr/share/jetty/lib/logging/slf4j-api-2.0.16.jar:/usr/share/jetty/lib/logging/jetty-slf4j-impl-12.0.16.jar:/usr/share/jetty/lib/jetty-http-12.0.16.jar:/usr/share/jetty/lib/jetty-server-12.0.16.jar:/usr/share/jetty/lib/jetty-xml-12.0.16.jar:/usr/share/jetty/lib/jetty-util-12.0.16.jar:/usr/share/jetty/lib/jetty-io-12.0.16.jar org.eclipse.jetty.xml.XmlConfiguration java.version=23.0.1 jetty.base=/usr/share/jetty jetty.base.uri=file:///usr/share/jetty jetty.home=/usr/share/jetty jetty.home.uri=file:///usr/share/jetty jetty.pid=/run/jetty/jetty.pid jetty.state=/run/jetty/jetty.state jetty.webapp.addHiddenClasses=org.eclipse.jetty.logging.,file:///usr/share/jetty/lib/logging/,org.slf4j. runtime.feature.alpn=true slf4j.version=2.0.16 /etc/jetty/jetty-bytebufferpool.xml /etc/jetty/jetty-pid.xml /etc/jetty/jetty-threadpool.xml /etc/jetty/jetty.xml /etc/jetty/jetty-state.xml
ID                    =  uid=0(root) gid=0(root) groups=0(root)
JETTY_USER            =  jetty
USE_START_STOP_DAEMON =  0
START_STOP_DAEMON     =  0

Then, check that it’s running with jps as root, and note that the nrpe user still can’t see the process using jps:

$ sudo /usr/share/jetty/bin/jetty.sh status 2>/dev/null | grep "running pid"
Jetty running pid=89969

$ sudo jps -l
89969 org.eclipse.jetty.xml.XmlConfiguration
91635 jdk.jcmd/sun.tools.jps.Jps

$ sudo -s -u nrpe jps -l
91711 jdk.jcmd/sun.tools.jps.Jps

We can decide whether to use the “jetty” service script as the process check tool, or whether to use jps. For this post, we’ll focus on jps.

We can run jps as root in a Bash script to dump the Java process ID into a temporary file. We can then use this script as an NRPE script.

#!/bin/bash

# Check if program name was provided
if [ -z "$1" ]; then
    echo "UNKNOWN: Please specify a program name to check"
    exit 3
fi

TEMP_FILE="/tmp/java_pid.txt"

# Clear the temp file first
> $TEMP_FILE

# Run jps and save output to temp file
jps -l > $TEMP_FILE

# Check if program is in the temp file
if grep -q "$1" $TEMP_FILE; then
    PID=$(grep "$1" $TEMP_FILE | awk '{print $1}')
    echo "OK - $1 is running with PID $PID"
    exit 0
else
    echo "CRITICAL - $1 is down"
    exit 2
fi

Conclusion

Most of time, using ps or pgrep will be adequate to check for running processes. This post just shows how to get the output from jps, when you need to use jps. As for checking the status of the jetty service, /usr/share/jetty/bin/jetty.sh status might be fine as well, but the process of filtering, parsing, and feeding the output to the NRPE plugin should be pretty much the same.

There could be another way to solve this issue, which I am not be aware of. The method I shared above is just one way to get jps’s report working with NRPE. Please let me know if you have experience with jps and Icinga/Nagios, and how you handle their reporting.

Migration from Struts 1 to Struts 2

2024-10-04T00:00:00+00:00

Apache Struts is an open source web framework. Struts 1 was first released in 2000 and its latest version (1.3.10) was released in December 2008, marking the EOL of Struts 1. So Struts 1 is an ancient, dead version. Organizations still using it need to move away from Struts 1.

Apache adapted the WebWork framework as Struts 2. Its architecture significantly differs from that of Struts 1. There are key points and obvious differences provided by Apache to cover a migration plan from Struts 1 to Struts 2.

The market share of Apache Struts is around 0.01%, which gives relatively lower relevance to Apache Struts. However, there are companies still using Apache Struts 1 in their legacy applications, which may not be counted in market share statistics.

A Comparison of Struts 1 and Struts 2

Struts 1 and Struts 2 differ significantly in their approach to handling web requests. Struts 1 follows a more rigid structure where Action classes must extend an abstract base class, limiting flexibility. In contrast, Struts 2 is more dynamic, allowing Action classes to implement interfaces or even function as simple POJOs, enhancing adaptability. Struts 2 also includes the ActionSupport class, which implements commonly used interfaces, making development easier and more versatile.

The threading model in Struts 1 is based on a singleton pattern, meaning only one instance of an Action class exists to handle all requests, which requires careful management of thread safety. Struts 2, on the other hand, creates a new Action object for every request, which eliminates concerns about thread safety.

When it comes to servlet dependency, Struts 1 is tightly coupled with the servlet API, requiring Actions to work directly with HttpServletRequest and HttpServletResponse. Struts 2 decouples the Action from the servlet container, using simple maps to represent the context. This allows for easier testing and more modular code. Struts 2 still offers the flexibility to access the request and response objects when necessary, but often through other architectural patterns that minimize the need for direct servlet interaction.

Testing Struts 1 applications is challenging due to its tight coupling with the servlet API. External libraries like Struts TestCase are often required to mock servlet objects for testing purposes. Struts 2 makes testing far more straightforward by supporting dependency injection and allowing Action classes to be tested in isolation, without needing complex mocks.

Input handling in Struts 1 revolves around ActionForm objects, which must extend a base class which often leads to redundancy. Struts 2 simplifies this by using Action properties directly, eliminating the need for separate ActionForm objects. Additionally, Struts 2 supports complex object types for input and output, allowing domain specific objects to be used.

In terms of expression languages, Struts 1 integrates with JSTL, which provides basic object graph traversal but lacks robust support for collections. Struts 2 uses OGNL (Object Graph Notation Language), offering more advanced and flexible expression capabilities, giving developers more control over data handling in views.

Binding values into views is handled differently as well. Struts 1 uses standard JSP mechanisms, while Struts 2 introduces the ValueStack, a system that decouples the view from the object being rendered. This allows for a greater degree of reuse and flexibility, especially when dealing with varying property types across different objects.

Type conversion in Struts 1 is handled by Commons-Beanutils, which operates on a per-class basis. This can be limiting as it doesn’t allow for instance-level customization. Struts 2 improves on this by utilizing OGNL for type conversion, offering built-in converters for common types while also providing a flexible system for adding custom converters.

Lastly, the control over action execution is much more granular in Struts 2. While Struts 1 uses a single request processor per module, Struts 2 introduces Interceptor Stacks, allowing developers to define different lifecycles for individual Actions. This customization helps tailor the flow of execution to fit specific application requirements.

Actions are central components in both Struts 1 and 2 and play a major role in the architectures of both Struts versions. Struts 2 offers a more modern, flexible, and test-friendly framework, improving upon Struts 1’s limitations by embracing POJO actions, better type conversion, enhanced validation, and a decoupled threading model. These changes make Struts 2 more adaptable and easier to maintain, particularly for complex web applications.

You can see more about request processing flow in Struts 2 in this InfoQ article.

Migration

Dependencies

Add the Struts 2.0 Jars to the existing Struts 1.3 application. The latest Jar files can be downloaded from the Apache Struts download page. Alternatively, you can download the Jar files for any of the previous versions from the archive.

Request handling

Configure Struts 2 to handle requests with the *.action extension, differently from Struts 1, which uses the *.do extension.

Configure web application

The web application is enabled in the web.xml configuration file. There are a few changes to this file needed for Struts 2. Dispatcher configuration is done by simply renaming the configuration tags from <servlet> and <servlet-mapping> to <filter> and <filter-mapping>.

A very basic web.xml configuration for Struts 1 would look like:

<servlet>
    <servlet-name>action</servlet-name>
    <servlet-class>org.apache.struts.action.ActionServlet</servlet-class>
    <init-param>
        <param-name>config</param-name>
        <param-value>/WEB-INF/struts-config.xml</param-value>
    </init-param>
    <load-on-startup>2</load-on-startup>
</servlet>

<servlet-mapping>
    <servlet-name>action</servlet-name>
    <url-pattern>*.do</url-pattern>
</servlet-mapping>

The corresponding conversion to Struts 2 would look like:

<filter>
    <filter-name>webwork</filter-name>
    <filter-class>org.apache.struts.action2.dispatcher.FilterDispatcher</filter-class>
</filter>

<filter-mapping>
    <filter-name>webwork</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

Migrating actions

A general structure of a Struts 1 action class is below.

public class CustomAction extends Action {
    public ActionForward execute(
        ActionMapping mapping,
        ActionForm form,
        HttpServletRequest request,
        HttpServletResponse response) throws Exception {
        // add logic
        return (mapping.findForward("success"));
    }
}

Struts 1 actions are singletons and extend the base Action class. Alternatively, DispatchAction can be used instead of the base class. In either case, the execute method is the entry point to an Action class. Actions have to be thread-safe and hence all the necessary variables should be handled in method scope. Finally, the execute method returns an ActionForward response.

On the other hand, in Struts 2 a basic action class structure would look like this:

public class MyAction extends ActionSupport {
   public String execute() throws Exception {
        // add logic
        return "success";
   }
}

A Struts 2 action class will usually (but not necessarily) extend ActionSupport. The usual entry point is again the execute method—however, this method has no parameters. In Struts 2 action classes are not singletons. Instead an instance of the action class is created for each request, so class scope variables can be used safely, alleviating many thread-safety concerns in Struts 1 actions. Struts 2 will inject a HttpServletRequest for the current request into actions implementing the ServletRequestAware interface, allowing you access to the underlying request object.

Action Configuration

In Struts 1 applications, action configurations are made in struts-config.xml located in the WEB-INF directory. Struts 1 applications need to configure actions and action form beans in this configuration file.

<struts-config>

     <form-beans>
         <form-bean name="myForm" type="com.endpoint.blog.struts.MyForm"/>
         ...
     </form-beans>

    <global-forwards>
       <forward name="home" path="/home.do" redirect="true" />
       <forward name="projects" path="/myactionProcess.do?dispatch=myDispatch" redirect="true"/>
       ...
   </global-forwards>

   <action-mappings>
       <action path="/myPackage/add" type="org.apache.struts.actions.ForwardAction" />
       ...
   </action-mappings>

 </struts-config>

Struts 2 configuration involves a package wrapper around the action definitions.

<struts>
  <package name="myPackage" extends="struts-default">
      <action name="add" >
          <result>/myPackage/add.jsp</result>
        </action>
        ...
    </package>
 </struts>

However, in Struts 2, actions can be defined in a more general way using Struts conventions:

<struts>
  <constant name="struts.action.extension" value="ep"/>
    <constant name="struts.convention.action.packages" value="com.endpoint.blog.actions.*"/>
    ...
</struts>

In this article we covered migrating the most basic components from Struts 1 to Struts 2 and compared several features between each version. Complete configuration and code migration from Struts 1 to Struts 2 can be done gradually, involving actions, interceptors, tags, and many other details. Not completely straightforward, but the patterns and convention changes adopted in Struts 2 become clearer as you dive in.

Getting started with Java development using Visual Studio Code

2023-03-17T00:00:00+00:00

Photo by Garrett Skinner, 2022

Visual Studio Code is a free source-code editor available for Windows, macOS, and Linux. While it includes a lot of features out of the box, you will likely need to extend its functionality to suit your purpose for using it. There are many extensions available, each providing their own set of features and functions.

In this guide we will install the Extension Pack for Java, which is a bundle of several extensions. Installing this extension pack will add the following features to Visual Studio Code:

Java language support for parsing and highlighting our code
Java test runner for testing our code
Java debugger for debugging our code
Java project manager for managing resources related to our code
Maven support for building and packaging our code

Note: This guide assumes you have already installed a Java Development Kit. If you haven’t done that yet, OpenJDK is a great option.

Step 1: Install Visual Studio Code

If you haven’t yet, download Visual Studio Code and install it. If you need more help with this step, review the installation instructions linked on this page. Click the link that applies to your operating system to access the installation instructions.

Step 2: Install the extension pack

Open Visual Studio Code and click on the extensions icon in the left sidebar, which looks like this:

This will take you to the extensions marketplace. Type “Extension Pack for Java” into the search bar and press enter. The extension we are looking for is authored by Microsoft. Once you find the extension pack in this list, click the Install button.

Step 3: Use the extension pack

Once the installation is complete, a “Get Started with Java Development” screen is displayed. The first item on this screen is “Get your runtime ready” with an “Install JDK” button. We already have a JDK installed, so we can skip this. The remaining items on the screen are helpful to review, as they give a brief overview of the features we have just added to our editor.

To ensure our JDK and extension pack are working together correctly, click File > New file…. You will be prompted to choose a file type. Enter “Java” and select “Java Class”. You will be prompted for a file name. Enter HelloWorld.java. You will then be prompted to choose where to save the file. Create a new directory somewhere, and save the file inside of the newly created directory.

The file HelloWorld.java is now open in our editor. Add the following text to the file:

class HelloWorld {
    public static void main (String args[])
    {
        System.out.println("Hello, world!");
    }
}

Save the file, then click the play icon at the top right of the window. This is the Run button. By default, it will run your Java application normally. You can also click the dropdown next to the icon and select Debug to run your Java application with the debugger attached, which is very helpful when problem-solving.

When you click the Run button, assuming everything is set up properly, your code will be compiled and executed. A terminal window will be opened, where your program may take in user input and/or write output. In this terminal window, the text “Hello, world!” will be written by the program.

If you are seeing the expected behavior, then congratulations are in order! You have just finished setting up Visual Studio Code for Java development.

If you are not seeing the expected behavior, restart from the beginning and confirm that you have followed each step correctly.

Happy coding!

For more guides and a deeper dive on the content in this one, see VS Code’s Java docs.

Creating a Messaging App Using Spring for Apache Kafka, Part 5

2022-10-31T00:00:00+00:00

I guess this is the longest break between posts in this series. But finally I had a chance to prepare a working example to finalize this series. So, up-to-date code is available in the GitHub repository.

This was already implemented in the previous parts. However, we haven’t shown it in action yet. There are some code fixes and small changes within the workflow of activation and login steps, so you can refer to the GitHub repository for the latest code.

Authentication and activation are managed through the AuthController class where these activation and login requests are handled. Let’s take a look at the REST endpoints handling these requests and explain the steps.

Activation

Our activation step uses a dummy mobile phone number. You can think of it similarly to activating a messaging application (like WhatsApp) using a phone number. I didn’t introduce any restrictions on the phone number for this application, so the phone number activation is just a pseudo-step and you can supply any number. In real life, the phone number activation would use SMS or other activation services to activate your chat application against the user’s phone number.

The activation endpoint is /api/auth/getcode, and takes the mobile number as payload. A sample POST request is below:

curl -H "Content-Type: application/json" -X POST localhost:8080/api/auth/getcode -d '{ "mobile": "01234" }'

The response of this endpoint is of the class ActivationResponse which, in return, gives an activation code along with the provided mobile number. This activationCode will later be used to login and retrieve an access token for further loginless connections. Here is an example output of this response:

{
  "mobile": "01234",
  "activationCode": "309652"
}

We provide a random activationCode and the client now needs to use this activation code to move one step further towards getting an access token for further authentication.

The login step is supposed to be a one-time login after getting the activation code (phone number) which is used for login credentials. Our application returns an access token in return. The login endpoint is /api/auth/login. Request and response examples are below.

curl -H "Content-Type: application/json" -X POST localhost:8080/api/auth/login -d '{ "mobile": "01234", "activationCode": "309652" }'

POSTing to the login endpoint returns an access token. The access token is needed to make message-sending requests so that the client can be authenticated. Storing locally and reusing this access token is the responsibility of the client.

{
  "accessToken": "5eab27f8-8748-4fdc-a4de-ac782ce17a74"
}

Chat Application

The chat application is the final part of this series, and will show the messaging in action. The chat application is designed as a simple proof of concept and simulates a chatting session.

To add a web UI to our Spring application I’ve added two static files, app.js and index.html, in the resources directory. index.html serves as the chat application’s UI and app.js is responsible for making AJAX calls and WebSocket connections from the client browser.

I added all three steps to the chat application UI to make the workflow clearer. As you see in the chat screen above, these three steps are activation, login, and starting a new chat. Let’s walk through these steps.

Activation simply requires a pretend mobile phone number. When you click on Activate it sends a POST request to the /api/auth/getcode end point we mentioned above.

function activate() {
  var mobileFormData = JSON.stringify({
    'mobile': $("#mobile").val(),
  });

  $.ajax({
    type: "POST",
    url: "http://localhost:8080/api/auth/getcode",
    data: mobileFormData,
    contentType: "application/json; charset=utf-8",
    success: function(result) {
      $("#activationCode").val(result.activationCode);
    },
  });
}

The resulting activationCode is automatically inputted into the login field. We already know what this activationCode is and where it is stored in the backend, so now our chat client can use this along with the mobile number to log in and get the access token. When you click on Login it POSTs to /api/auth/login and puts the access token into the Access Token input box this time.

function login() {
  var loginFormData = JSON.stringify({
    'mobile': $("#mobile").val(),
    'activationCode': $("#activationCode").val(),
  });

  $.ajax({
    type: "POST",
    url: "http://localhost:8080/api/auth/login",
    data: loginFormData,
    contentType: "application/json; charset=utf-8",
    success: function(result) {
      $("#accessToken").val(result.accessToken);
    },
  });
}

Now finally our web client can connect to our WebSocket URI. Without a login step we could let our application accept WebSocket connections as well. However, we simulated an authentication step and we want to know who is connecting to the WebSocket handler.

function connect() {
  ws = new WebSocket('ws://localhost:8080/messaging?accessToken=' + $("#accessToken").val());
    ws.addEventListener('error', (error) => {
    console.log("Error: ", error.message);
    setConnected(false);
    });
    ws.addEventListener('close', (event) => {
    console.log("Close: ", event.code + " - " + event.reason);
      setConnected(false);
    })
  ws.onmessage = function(event) {
    helloWorld(event.data);
    console.log("Connected to WebSocket " + event.data);
  }
  setConnected(true);
}

WebSocket Handshake

In our WebSocketConfig class I added an WebSocket handshake interceptor WSHandshakeInterceptor to authenticate the connecting user by checking the accessToken. So, the accessToken is necessary at the first step while connecting to the WebSocket registry. If the client doesn’t provide an accessToken or provides an invalid one then the handshake will fail.

@Override
public boolean beforeHandshake(ServerHttpRequest serverHttpRequest, ServerHttpResponse serverHttpResponse, org.springframework.web.socket.WebSocketHandler webSocketHandler, Map<String, Object> map) throws Exception {

    LOG.info("hand shaking for ws session url  " + serverHttpRequest.getURI().getPath() + " ? " + serverHttpRequest.getURI().getQuery());
    String parameters[] = serverHttpRequest.getURI().getQuery().split("=");

    if (parameters.length == 2 && parameters[0].equals("accessToken")) {
        String accessToken = parameters[1];

        Long senderUserId = 0L;
        String senderId = cacheRepository.getUserIdByAccessToken(accessToken);

        if (senderId == null) {
            User sender = userRepository.findByToken(accessToken);
            if(sender != null) {
                senderUserId = sender.getUserId();
            }
        } else {
            senderUserId = Long.valueOf(senderId);
        }

        LOG.info("Handshake found userId: " + senderUserId);

        if (senderUserId == 0L) {
            LOG.info("Handshake failed: user not found for given accessToken");
            map.put("CUSTOM-HEADER", "Handshake failed: user not found for given accessToken");
            return false;
        }

        map.put("CUSTOM-HEADER", "Handshake successful");
        LOG.info("Handshake successful");
        return true;
    }

    map.put("CUSTOM-HEADER", "Handshake failed: accessToken not found");
    LOG.info("Handshake failed: accessToken not found");
    return false;

}

If everything goes well and our web client is connected to WebSocket then the client can start sending messages to the other WebSocket sessions. We don’t manage offline messaging or other capabilities in this application. As you’ll see a WebSocket session will be able to send messages to only corresponding active WebSocket sessions. In real life examples you would manage offline messaging and deliver to the destination users whenever they become online.

Chat in action

When I open the chat application in two different browsers and login with different mobile numbers I am able to make them have a chat. I assume mobile number 01234 has two contacts 12345 and 23456 in their contact list.

On this screen I chose the 12345 contact to start messaging. I entered my message and clicked on Send button. Since both clients connected to the WebSocket I was able to send the message to 12345, as you can see on the second client’s window.

The messages are not tidied here; I made the WebSocket message list the message on the client where it connected to its WebSocket session.

Here is the JavaScript code sending the message to WebSocket session when you click on the Send button:

function sendData() {
  var data = JSON.stringify({
    'topic': 'SEND_MESSAGE',
    'message': {
      'accessToken': $("#accessToken").val(),
      'sendTo': $("#sendTo").val(),
      'msg': $("#messageArea").val(),
    },
  })
  ws.send(data);
}

When we send the message to WebSocket, our WebSocketHandler handles the received message through WebSocket session. However, it doesn’t directly send the message to the corresponding WebSocket session. We send our message to the Kafka topic SEND_MESSAGE using MessageSender.send() method at this point and Kafka manages the streaming messages.

On the Kafka listener side of our application MessageReceiver.messagesSendToUser() is receiving the messages and it redirects the messages to the corresponding WebSocket session. So we took this workload from the WebSocket handler to the Kafka listener so it would be busy only with the messages sent by the clients. Another approach would be sending the messages directly to Kafka topics instead of WebSocket handler so we would give all the workload to Kafka services.

@KafkaListener(topics = "SEND_MESSAGE", groupId = "foo")
public void messagesSendToUser(@Payload String message, @Headers MessageHeaders headers) {

    JSONObject jsonObject = new JSONObject(message);

    LOG.info("Websocket message will be sent if corresponding destination websocket session is found");
    if (jsonObject.get("sendTo") != null
            && WebSocketPool.websockets.get(jsonObject.getLong("sendTo")) != null
            && WebSocketPool.websockets.get(jsonObject.getLong("sendTo")).size() > 0) {

        String accessToken = jsonObject.getString("accessToken");
        Long sendTo = jsonObject.getLong("sendTo");
        String msg = jsonObject.getString("msg");

        LOG.info("Websocket message is sent to " + sendTo);

        String topic = "SEND_MESSAGE";

        messageService.sendMessage(accessToken, sendTo, msg, topic);

    } else {
        LOG.info("Websocket session not found for given sendTo");
    }
}

Here is our logging for the message sent through the chat application.

2022-10-07 16:36:41.371  INFO 37554 --- [nio-8080-exec-1] c.e.S.websocket.WebSocketHandler         : {"topic":"SEND_MESSAGE","message":{"accessToken":"5eab27f8-8748-4fdc-a4de-ac782ce17a74","sendTo":"5","msg":"Hey dude, howdy?"}}
2022-10-07 16:36:41.411  INFO 37554 --- [ntainer#0-0-C-1] c.e.S.message.broker.MessageReceiver     : Websocket message will be sent if corresponding destination websocket session is found
2022-10-07 16:36:41.411  INFO 37554 --- [ntainer#0-0-C-1] c.e.S.message.broker.MessageReceiver     : Websocket message is sent to 5
2022-10-07 16:36:41.412  INFO 37554 --- [ntainer#0-0-C-1] c.e.S.message.broker.MessageReceiver     : Sending websocket message {"msg":"Hey dude, howdy?","senderId":4,"topic":"SEND_MESSAGE"}

What is the point of using Kafka and WebSocket?

Let’s go back to the question of why we used Kafka and WebSockets for a messaging app. We questioned this paradigm in part 1 of this series while discussing the design and architecture. It all depends on what you aim to do with a messaging application.

There are times when WebSocket has the advantage over HTTP or XMPP, and we can compare these protocols on a case-by-case basis. However, those debates won’t be had in this post. WebSocket is simply a fast relaying and simple protocol which enables a web server app to communicate with a web browser, or more broadly, WebSocket clients, easily. So you can use WebSocket to create chat applications over the web and let the same user’s multiple sessions run concurrently on multiple clients.

Kafka, on the other hand, acts as a message orchestrator, highly aware of what is coming in and going out and upwardly scalable. You are guaranteed to be able to process each message which arrives at Kafka without worrying about concurrency and delay issues which you often face with traditional threads or services. Kafka is not essentially a chat application backend but can handle the millions and even billions of messages easily. The main purpose of this example application is not creating a chat application, but as the title says, a messaging application.

So, while chatting would designate a narrow scope, messaging covers a broad range of applications handling high loads of data streaming environments.

Conclusion

Kafka is used in several environments by lots of giants of industry and startups. A few industry examples would be financial services, telecom, manufacturing, and healthcare. You can make data streaming seamless and real-time using Kafka.

Please share with us in the comments your experiences using Kafka and related technologies, even including ML and model training streams.

Quartz Scheduler as a Service

2022-04-18T00:00:00+00:00

Photo by Mat Brown from Pexels

Quartz Job Scheduler

“Quartz is a richly featured, open source job scheduling library that can be integrated within virtually any Java application — from the smallest stand-alone application to the largest e-commerce system.” (Quartz Scheduler overview)

Besides its advanced features, most basic and frequently used feature is job scheduling and job execution. Some frameworks like Spring Scheduler have their integration practice using Quartz Scheduler which allows using its default scheduling method.

In this post I am going to tell you a different approach to show how we can use Quartz Scheduler to schedule our jobs. We actually still will be using the existing scheduling mechanism of Quartz but we’re going to show how we can manage the scheduled and unscheduled jobs online. This way you can manage all the available jobs or create new ones on the fly.

Quartz Scheduler as a Service

Previously I led development of an enterprise “Business Service Management” software to replace IBM’s TBSM product at a major telco company in Turkey. This was a challenging project and found a solid place in the customer environment after a successful release.

Scheduled key performance indicators (KPI) retrieval and background reporting jobs were a significant part of this project. KPIs were either internal business service availability and health metrics or measured metrics calculated and stored in external data sources. Reports are also another type of schedulable jobs as many organizations need the data to be reported at certain intervals.

In an enterprise web application with such needs you would need to allow your customer to create their own customized scheduled jobs (KPIs, reports, etc.) in an easily manageable way. For this I came up with a simple solution by blending the existing Quartz Scheduler scheduling mechanism with some spice.

So here is the model we used:

A database table for creating/updating scheduler job definitions
Observer Schduler Job for observing the scheduler job table to watch for any updates in the scheduled jobs: new job, updated job, or disabled job, etc.
Business Job: You might define several schedulable business job types. KPI is one of those and I am going to give an example of it.

Simplicity should be a design goal, however the details can have their complexities.

This design doesn’t replace or provide an alternative to how Quartz Scheduler schedules its jobs. That is subject to job persistence and is out of this article’s scope. I am assuming we are scheduling the jobs all in Quartz Scheduler’s RAM-store or Job-store.

Read and Manage Job Data

Ideally you should store and manage the jobs as services in a database and you can then connect to this job storage either via DB connection or API. For security reasons even if you think that your application or services are internal and totally authenticated and authorised you should still perform DB operations via APIs. But for capability perspective yes you can use many ways to read and manage a data storage.

For this simple project I am not going to use a database but instead a JSON file as job service definitions repository. But you can simply convert this method to a database or API method.

I am going to use a JSON file named kpi.json in my project and define a simple set of attributes for each KPI item. Any service or scheduled job can have more or fewer attributes according to the requirements of the business use case.

Spring Application

You can use any framework or even without using any framework you can create your application from scratch and build a JAR. Here in this project I chose to go with Spring framework. You can also simply initialize a Spring application here.

Design

As I suggested a model above as a scheduling service solution, here is a high-level design of the model.

The overall solution would have a data storage for holding scheduled job service definitions and a UI for managing their attributes like enabling/disabling or changing scheduling dates etc.

In this solution we have two different Quartz Scheduler job types: observer job and business job. Observer job is a single job triggered frequently, say, every 5 seconds or every 1 minute, and checks the existing job definitions in the job storage. If it sees any update on the job definitions or new jobs it behaves accordingly. Business jobs are the job definitions found in job storage and designed to perform certain business actions. The business jobs can be notification jobs, KPI measuring jobs, and any other scheduled business jobs which should have their own scheduling interval.

In this example project I specifically used KPI term as the business case just to make it more relevant.

Scheduler

KPIJobWatcher class is responsible to schedule the observer job. In Spring application startup this is going to be our starting point to the scheduling service management.

Spring Application Startup

@SpringBootApplication
public class QSchedulerApplication {

	public static void main(String[] args) {
		SpringApplication.run(QSchedulerApplication.class, args);
	}

	Scheduler kpiScheduler;

	@EventListener(ApplicationReadyEvent.class)
	public void onAppStartUp() {
		try {
			// initializing KPI Trigger
			SchedulerFactory sf = new StdSchedulerFactory();
			kpiScheduler = sf.getScheduler();

			// watcher runs an observer job which monitors and manages KPI jobs
			KPIJobWatcher watcher = new KPIJobWatcher(kpiScheduler);
			watcher.run();
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}

KPIJobWatcher schedules the Observer job:

public class KPIJobWatcher {
    private static final Logger logger = LoggerFactory.getLogger(KPIJobWatcher.class);

    Scheduler kpiScheduler;
    public KPIJobWatcher(Scheduler s) {
        kpiScheduler = s;
    }

    /**
     * run KPIJobWatcher
     * @throws Exception
     */
    public void run() throws Exception {

        try {
            // Setting the KPI Job factory of observerScheduler
            KPIJobFactory jf = new KPIJobFactory((StdScheduler)kpiScheduler);
            kpiScheduler.setJobFactory(jf);

            // Scheduling KPI Observer Job
            JobDetail observerJob = newJob(KPIObserverJob.class)
                    .withIdentity("observerJob", "observergroup")
                    .build();

            SimpleTrigger trigger = newTrigger()
                    .withIdentity(observerJob.getKey() + "_trigger", "observergroup")
                    .withSchedule(org.quartz.SimpleScheduleBuilder.simpleSchedule()
                            .withIntervalInSeconds(10)
                            .repeatForever())
                    .build();

            Date ft = kpiScheduler.scheduleJob(observerJob, trigger);
            logger.info(observerJob.getKey() + " has been scheduled to run at: " + ft);

            // Starting KPI Observer Scheduler
            kpiScheduler.start();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Before moving on to observer job here I want to notice that you can use a custom JobFactory and attach it to the current scheduler object so that you can use custom jobs with custom constructors created within this custom JobFactory as part of the factory design pattern.

JobFactory

public class KPIJobFactory implements JobFactory {
    Scheduler kpiScheduler;
    public KPIJobFactory(Scheduler s) {
        kpiScheduler = s;
    }

    public KPIObserverJob newJob(TriggerFiredBundle bundle, Scheduler Scheduler) throws SchedulerException {

        JobDetail jobDetail = bundle.getJobDetail();
        Class<KPIObserverJob> jobClass = (Class<KPIObserverJob>) jobDetail.getJobClass();
        try {
            // this is how we construct our custom job with custom factory
            return jobClass.getConstructor(Scheduler.getClass()).newInstance(kpiScheduler);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

Observer Job

As suggested in the model above the observer job is triggered frequently and manages the overall scheduling job service in the background. I created the KPIObserverJob class as the observer job in this project and as you can see in the previous section KPIJobFactory creates instances of this observer job.

KPIObserverJob

Observer Job has some specific methods like ScheduleJob and UnscheduleJob to manage scheduling jobs.

public class KPIObserverJob implements Job {

    private static final Logger logger = LoggerFactory.getLogger(KPIObserverJob.class);

    List<JobDetail> jobList;
    Scheduler kpiScheduler;

    public KPIObserverJob(StdScheduler s) {
        kpiScheduler = s;
    }

    List<String> scheduledJobList;
    HashMap<String, JobDetail> alreadyScheduledJobList;

    String cronFormat = "SECOND MINUTE HOUR DAY_OF_MON MONTH DAY_OF_WEEK";

    @Override
    public void execute(JobExecutionContext context) throws JobExecutionException {
        scheduledJobList = new ArrayList<String>();
        alreadyScheduledJobList = new HashMap<String, JobDetail>();
        //JobKey jobKey = context.getJobDetail().getKey();

        jobList = new ArrayList<JobDetail>();

        // Get all KPIs and create their jobs
        CreateJobs();

        // Get the list of currently scheduled KPI jobs
        try {
            for (String groupName : kpiScheduler.getJobGroupNames()) {
                for (JobKey jk : kpiScheduler.getJobKeys(GroupMatcher.jobGroupEquals(groupName))) {
                    String jobName = jk.getName();
                    String jobGroup = jk.getGroup();

                    scheduledJobList.add(jobName);

                    JobDetail jd = kpiScheduler.getJobDetail(jk);
                    alreadyScheduledJobList.put(jobName, jd);

                    logger.info("already scheduled jobName {}", jobName);
                }
            }
        } catch (SchedulerException e) {
            e.printStackTrace();
        }

        // Schedule or unschedule KPI jobs if not done yet
        for (JobDetail job : jobList) {
            try {
                if (!scheduledJobList.contains(job.getKey().getName()))
                {
                    if (job.getJobDataMap().getInt("isRunning") == 1) {
                        logger.info("scheduling job: kpiJobName_{}", job.getJobDataMap().getString("kpiName"));
                        ScheduleJob(job);
                    }
                } else {
                    // Check any changes in the KPI job definition
                    JobDetail sJD = alreadyScheduledJobList.get("kpiJobName_" + job.getJobDataMap().getString("kpiName"));
                    if (!job.getJobDataMap().getString("cron").equals(sJD.getJobDataMap().getString("cron"))) {
                        logger.info("rescheduling job: kpiJobName {} , new cron: {}",
                                job.getJobDataMap().getString("kpiName"), job.getJobDataMap().getString("cron"));
                        UnscheduleJob(job.getJobDataMap().getString("kpiName"));
                        ScheduleJob(job);
                    }

                    if (job.getJobDataMap().getInt("isRunning") == 0) {
                        logger.info("Unscheduling: kpiJobName {}" + job.getJobDataMap().getString("kpiName"));
                        UnscheduleJob(job.getJobDataMap().getString("kpiName"));
                    }
                }
            } catch (SchedulerException e) {
                e.printStackTrace();
            }
        }

        // Finally unschedule deleted jobs if they are not listed anymore
        for (String kpiName : scheduledJobList) {
            boolean unschedule = true;
            if (!kpiName.equals("observerJob")) {
                JobDetail toBeRemovedJob = null;
                for (JobDetail jdetail : jobList) {
                    if (jdetail.getKey().getName().equals(kpiName)) {
                        unschedule = false;
                    }
                }

                if (unschedule) {
                    logger.info("Unscheduling: " + "kpiJobId" + kpiName.split("_")[1]);
                    UnscheduleJob(kpiName.split("_")[1]);
                }
            }
        }
    }

    private static final Type KPI_JSON_TYPE = new TypeToken<List<KPI_JSON>>() {}.getType();

    /**
     * Create Quartz Scheduler jobs from the job records read from a data source
     */
    private void CreateJobs() {

        Gson gson = new Gson();
        try {
            // kpi.json as a service data storage where we get KPI job data to be scheduled
            JsonReader reader = new JsonReader(new FileReader("kpi.json"));
            List<KPI_JSON> kpiList = gson.fromJson(reader, KPI_JSON_TYPE);

            for (KPI_JSON kpiItem : kpiList) {
                logger.info("Found KPI in kpi.json: {} , enabled: {}", kpiItem.getName(), kpiItem.getIsRunning());

                JobDetail job = newJob(KPIJSONJob.class)
                        .withIdentity("kpiJobName_" + kpiItem.getName(), "kpigroup")
                        .usingJobData("kpiName", kpiItem.getName())
                        .usingJobData("cron", kpiItem.getCron())
                        .usingJobData("lastRan", kpiItem.getLastRan())
                        .usingJobData("kpiDescription", kpiItem.getKpiDescription())
                        .usingJobData("lastMeasuredValue", kpiItem.getLastMeasuredValue())
                        .usingJobData("filename", kpiItem.getFilename())
                        .usingJobData("type", kpiItem.getType())
                        .usingJobData("isRunning", kpiItem.getIsRunning())
                        .build();

                jobList.add(job);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    /**
     * Schedule a job
     * @param job
     * @throws SchedulerException
     */
    private void ScheduleJob(JobDetail job) throws SchedulerException {
        String cron = job.getJobDataMap().getString("cron");
        CronTrigger trigger = newTrigger()
                .withIdentity(job.getKey().getName() + "_trigger", "kpigroup")
                .withSchedule(cronSchedule(cron))
                .startNow()
                .build();
        Date ft = kpiScheduler.scheduleJob(job, trigger);
    }

    /**
     * Unschedule a job
     * @param kpiName
     */
    private void UnscheduleJob(String kpiName) {
        TriggerKey tk = new TriggerKey("kpiJobName_" + kpiName + "_trigger", "kpigroup");
        try {
            kpiScheduler.unscheduleJob(tk);
            kpiScheduler.deleteJob(new JobKey("kpiJobName_" + kpiName, "kpigroup"));
        } catch (SchedulerException e) {
            e.printStackTrace();
        }
    }
}

Business Jobs

Business jobs, as suggested in the model, can be any schedulable jobs. Managing/updating the business jobs frequently is a key point here. As the enterprise demands grow and change continuously, KPIs are generated at intervals (daily, weekly, monthly, etc.) and for frequent notification needs this kind of scheduling job management can be an important part of a solution.

Here I created KPIJSONJob as my business job:

public class KPIJSONJob implements Job {
    private static final Logger logger = LoggerFactory.getLogger(KPIJSONJob.class);
    private KPI_JSON kpi;

    @Override
    public void execute(JobExecutionContext context) throws JobExecutionException {
        JobDataMap dataMap = context.getJobDetail().getJobDataMap();

        kpi.setName(dataMap.getString("kpiName"));
        kpi.setKpiDescription(dataMap.getString("kpiDescription"));
        kpi.setIsRunning(dataMap.getInt("isRunning"));
        kpi.setFilename(dataMap.getString("filename"));
        kpi.setCron(dataMap.getString("cron"));
        kpi.setLastRan(dataMap.getString("lastRan"));
        kpi.setType(dataMap.getString("type"));
        kpi.setLastMeasuredValue(dataMap.getString("lastMeasuredValue"));

        this.processKPI();
    }

    public class KPIMeasured {
        public String name;
        public String value;
    }

    private static final Type KPIMEASURED_TYPE = new TypeToken<List<KPIMeasured>>() {}.getType();
    protected void processKPI() {
        // processKPI is supposed to get the KPI measured value from an external datasource and updates kpi.json
        // ...
    }

}

Running this Solution

Let’s give it a try and see it in action. Say we have the kpi.json as our job storage, with the following KPI jobs defined:

[
  {
    "name": "critical_ticket_count",
    "type": "JSON_FILE",
    "cron": "0 0 1 ? * * *",
    "isRunning": 0,
    "lastRan": "2022-04-01 01:00:00",
    "kpiDescription": "Open critical ticket count",
    "lastMeasuredValue": "7",
    "filename": "kpi_measured.json"
  },
  {
    "name": "failed_customer_api_call",
    "type": "JSON_FILE",
    "cron": "0 0 2 ? * * *",
    "isRunning": 0,
    "lastRan": "2022-04-01 02:00:00",
    "kpiDescription": "Last 24-Hour failed API call count",
    "lastMeasuredValue": "23",
    "filename": "kpi_measured.json"
  }
]

When we run the Spring application it starts logging like below:

2022-04-11 13:48:12.354  INFO 13033 --- [eduler_Worker-1] com.example.qscheduler.KPIObserverJob    : Found KPI in kpi.json: critical_ticket_count , enabled: 0
2022-04-11 13:48:12.355  INFO 13033 --- [eduler_Worker-1] com.example.qscheduler.KPIObserverJob    : Found KPI in kpi.json: failed_customer_api_call , enabled: 0
2022-04-11 13:48:12.355  INFO 13033 --- [eduler_Worker-1] com.example.qscheduler.KPIObserverJob    : already scheduled jobName observerJob

Initially I set the isRunning attribute of those KPI jobs to 0 and my scheduler service is not scheduling them. My KPIObserverJob triggers every 10 seconds because I set it to trigger that way in KPIJobWatcher.

Now let’s see if I update critical_ticket_count KPI’s isRunning value to 1:

2022-04-11 13:52:32.347  INFO 13033 --- [eduler_Worker-7] com.example.qscheduler.KPIObserverJob    : Found KPI in kpi.json: critical_ticket_count , enabled: 1
2022-04-11 13:52:32.348  INFO 13033 --- [eduler_Worker-7] com.example.qscheduler.KPIObserverJob    : Found KPI in kpi.json: failed_customer_api_call , enabled: 0
2022-04-11 13:52:32.348  INFO 13033 --- [eduler_Worker-7] com.example.qscheduler.KPIObserverJob    : already scheduled jobName observerJob
2022-04-11 13:52:32.349  INFO 13033 --- [eduler_Worker-7] com.example.qscheduler.KPIObserverJob    : scheduling job: kpiJobName_critical_ticket_count

As you can see from the logs ObserverJob noticed the enabled job and scheduled it.

Let’s change the cron scheduling rule of critical_ticket_count job to 0 0 3 ? * * * and see the logs again:

2022-04-11 13:55:12.354  INFO 13033 --- [eduler_Worker-3] com.example.qscheduler.KPIObserverJob    : Found KPI in kpi.json: critical_ticket_count , enabled: 1
2022-04-11 13:55:12.355  INFO 13033 --- [eduler_Worker-3] com.example.qscheduler.KPIObserverJob    : Found KPI in kpi.json: failed_customer_api_call , enabled: 0
2022-04-11 13:55:12.356  INFO 13033 --- [eduler_Worker-3] com.example.qscheduler.KPIObserverJob    : already scheduled jobName observerJob
2022-04-11 13:55:12.356  INFO 13033 --- [eduler_Worker-3] com.example.qscheduler.KPIObserverJob    : already scheduled jobName kpiJobName_critical_ticket_count
2022-04-11 13:55:12.356  INFO 13033 --- [eduler_Worker-3] com.example.qscheduler.KPIObserverJob    : rescheduling job: kpiJobName critical_ticket_count , new cron: 0 0 3 ? * * *

The Observer job now rescheduled the job since we changed its cron rule. These are all how we make observer job manage the KPI business jobs. If you have more attributes and if you want your observer job to reschedule or perform different operations on business job definition updates you should enrich your ObserverJob.

Extend by Creating a Management UI

Managing the scheduler jobs using a UI is not in the scope of this post. But that is not much different than managing any data on a web application. I encourage you to do your own implementations if this solution sounds useful to you.

Conclusion

This solution helps you create your own scheduling job management solution on the fly and lets you create, update, or delete the Quartz Scheduler jobs dynamically.

The complete implementation can be found in the GitHub project.

Extending Your Jetty Distribution’s Capabilities

2022-03-31T00:00:00+00:00

What is Jetty?

“Jetty is a lightweight highly scalable Java-based web server and servlet engine.” (Jetty Project)

Jetty can run standalone or embedded in a Java application and the details about running a Jetty webserver can be found in the Jetty Project Git repository and documentation as well. The Jetty project has been hosted at the Eclipse Foundation since 2009 (Jetty, Eclipse).

Know Your Jetty

In many legacy environments using the Jetty web server there may be an older version of Jetty. If you know the version of the Jetty distribution in your environment then you can find its source code in the Jetty project GitHub repo. Some of the distributions are in project releases but most of the distributions can be found in the tags as well.

For instance jetty-9.4.15.v20190215 distribution can be found in the Jetty project tags at this URL: https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.15.v20190215

When you clone the jetty.project Git repo, you can then easily switch to any specific release tag:

$ git clone git@github.com:eclipse/jetty.project.git
$ git checkout jetty-9.4.15.v20190215

Then you can build or add your custom code in that version.

Extending Your Jetty Capabilities

The reason you might want to build Jetty yourself is that you have a specific Jetty version in your environment and want to add some custom handlers or wrappers so that you can add additional capabilities in your environment.

Jetty is written in Java and you can add new features or patch your own fork like other open-source Java projects.

Build

Once you have your target version code base you can just work on that individually. This is one way to add new features to your Jetty distribution.

After you add your custom code you’ll need to build. You can find the building instructions on Jetty Project GitHub home, which is simply:

$ mvn clean install

If you want to skip the tests the option below is your friend:

$ mvn clean install -DskipTests

Compile Classes Individually

This is a tricky way to inject your newly created custom classes into your Jetty distribution. In this way, instead of building the whole Jetty project, you can just create individual custom Java classes consuming Jetty libraries and compile them manually. You don’t need the whole project this way.

If we come back to the question: what new features would I want to add to my new or ancient local Jetty distribution? Well, that really depends on the issues you face or improvements you need to add.

For one of our customers, once we needed to log request and response headers in Jetty. We couldn’t find an existing way to do that. So I decided to create a custom RequestLog handler class and inject this into the Jetty deployment we already have rather than building the whole project.

Even if you don’t build the whole project it is still useful and handy to get the whole project code to refer the existing code and prepare your code by learning the existing way things are done in the project.

I found RequestLog interface in jetty-server sub-project and it is created under org.eclipse.jetty.server package. There is also a class RequestLogCollection in the same level implementing RequestLog which may give you some idea about the implementations.

So I followed the structure and created my custom handler in the same level and implemented RequestLog. Below is a part of my CustomRequestLog class:

package org.eclipse.jetty.server;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.eclipse.jetty.http.pathmap.PathMappings;
import org.eclipse.jetty.util.component.ContainerLifeCycle;
import org.eclipse.jetty.util.log.Log;
import org.eclipse.jetty.util.log.Logger;

import java.io.IOException;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.*;

public class CustomRequestLog extends ContainerLifeCycle implements RequestLog
{
    protected static final Logger LOG = Log.getLogger(CustomRequestLog.class);

    private static ThreadLocal<StringBuilder> _buffers = ThreadLocal.withInitial(() -> new StringBuilder(256));

    protected final Writer _requestLogWriter;

    private String[] _ignorePaths;
    private transient PathMappings<String> _ignorePathMap;

    public CustomRequestLog(Writer requestLogWriter)
    {
        this._requestLogWriter = requestLogWriter;
        addBean(_requestLogWriter);
    }

    /**
     * Is logging enabled
     *
     * @return true if logging is enabled
     */
    protected boolean isEnabled()
    {
        return true;
    }

    /**
     * Write requestEntry out. (to disk or slf4j log)
     *
     * @param requestEntry the request entry
     * @throws IOException if unable to write the entry
     */
    public void write(String requestEntry) throws IOException
    {
        _requestLogWriter.write(requestEntry);
    }

    private void append(StringBuilder buf, String s)
    {
        if (s == null || s.length() == 0)
            buf.append('-');
        else
            buf.append(s);
    }

    /**
     * Writes the request and response information to the output stream.
     *
     * @see RequestLog#log(Request, Response)
     */
    @Override
    public void log(Request request, Response response)
    {
        try
        {
            if (_ignorePathMap != null && _ignorePathMap.getMatch(request.getRequestURI()) != null)
                return;

            if (!isEnabled())
                return;

            StringBuilder buf = _buffers.get();
            buf.setLength(0);

            Gson gsonObj = new GsonBuilder().disableHtmlEscaping().create();
            Map<String, Object> reqLogMap = new HashMap<String, Object>();
            Map<String, String> reqHeaderMap = new HashMap<String, String>();
            // epoch timestamp
            reqLogMap.put("timestamp_epoch", System.currentTimeMillis());

            // timestamp
            DateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSXXX");
            String nowAsString = df.format(new Date());
            reqLogMap.put("timestamp", nowAsString);

            // request headers
            List<String> reqHeaderList = Collections.list(request.getHeaderNames());
            for(String headerName : reqHeaderList) {
                reqHeaderMap.put(headerName.toLowerCase(), request.getHeader(headerName));
            }
            reqLogMap.put("request_headers", reqHeaderMap);

            // response headers
            Map<String, String> resHeaderMap = new HashMap<String, String>();
            for(String headerName : response.getHeaderNames()) {
                resHeaderMap.put(headerName.toLowerCase(), response.getHeader(headerName));
            }
            reqLogMap.put("response_headers", resHeaderMap);

            // http method
            reqLogMap.put("http_method", request.getMethod());

            // original URI
            reqLogMap.put("original_uri", request.getOriginalURI());

            // protocol
            reqLogMap.put("protocol", request.getProtocol());

            // http status
            reqLogMap.put("http_status", response.getStatus());

            // query string
            reqLogMap.put("query_string", request.getQueryString());

            String reqJSONStr = gsonObj.toJson(reqLogMap);
            buf.append(reqJSONStr);

            String log = buf.toString();
            write(log);
        }
        catch (IOException e)
        {
            LOG.warn(e);
        }
    }
}

In this custom RequestLog class the most interesting part is public void log(Request request, Response response) method where the logging operation is actually done. You can simply override the existing logging behaviour and put anything you want. Here I added the raw request and response headers coming and going through Jetty server.

Now it is time to compile this class. You can find many tutorials about compiling a single Java class using classpath. Here’s how I did it:

$ javac -cp ".:$JETTY_HOME/lib/jetty-server-9.4.15.v20190215.jar:$JETTY_HOME/lib/jetty-http-9.4.15.v20190215.jar:$JETTY_HOME/lib/jetty-util-9.4.15.v20190215.jar:$JETTY_HOME/lib/servlet-api-3.1.jar:$JETTY_HOME/lib/gson-2.8.2.jar" CustomRequestLog.java

If you look at my classpath I even added a third party library gson-2.8.2.jar since I also used this in my custom code. Remember to put this in your $JETTY_HOME directory as well.

The command above generates the CustomRequestLog.class file which is now available to be injected. So where do you need to inject this?

Since I followed where the RequestLog interface is located and packaged we better inject this into the same project JAR file, which is jetty-server.jar. In my environment it is jetty-server-9.4.15.v20190215.jar. I also added other required dependencies in the classpath to compile this code.

Now, I want to inject CustomRequestLog.class into jetty-server-9.4.15.v20190215.jar. I copied this jar into a temporary directory and I extracted the content of jetty-server-9.4.15.v20190215.jar into the temp directory using this command:

$ jar xf jetty-server-9.4.15.v20190215.jar

This command extracts all the content of the jar file including resource files and the classes in their corresponding directory structure org/eclipse/jetty/server. You would see RequestLog.class also extracted in this directory.

So what we need to do is now simply copy our CustomRequestLog.class into this extracted org/eclipse/jetty/server directory and pack up the JAR file again by running this command:

$ jar cvf jetty-server-9.4.15.v20190215.jar org/ META-INF/

This command re-bundles compiled code along with the other extracted resources (in this case the META-INF/ directory only) and creates our injected JAR file. You’d better create this injected Jetty JAR file in the temp directory so that you can control the backup of existing original JAR files.

For this specific case I added this custom RequestLog handler in my Jetty config file jetty.xml. It may not be the case for all the custom changes or extensions you’d add to your Jetty instance.

Here is an example RequestLog config entry for this custom handler:

<Set name="RequestLog">
  <New id="RequestLog" class="org.eclipse.jetty.server.CustomRequestLog">
    <!-- Writer -->
    <Arg>
      <New class="org.eclipse.jetty.server.AsyncRequestLogWriter">
        <Arg>
          <Property name="jetty.base" default="." />/
          <Property>
            <Name>jetty.requestlog.filePath</Name>
            <Default>
              <Property name="jetty.requestlog.dir" default="logs"/>/yyyy_mm_dd.request.log
            </Default>
          </Property>
        </Arg>
        <Arg/>
        <Set name="filenameDateFormat">
          <Property name="jetty.requestlog.filenameDateFormat" default="yyyy_MM_dd"/>
        </Set>
        <Set name="retainDays">
          <Property name="jetty.requestlog.retainDays" default="90"/>
        </Set>
        <Set name="append">
          <Property name="jetty.requestlog.append" default="false"/>
        </Set>
        <Set name="timeZone">
          <Property name="jetty.requestlog.timezone" default="GMT"/>
        </Set>
      </New>
    </Arg>
  </New>
</Set>

That’s all.

Java developer job opening

2021-11-03T00:00:00+00:00

We are seeking a full-time Java server-side software developer based in the United States to work with us on our clients’ applications.

End Point Dev is an Internet technology consulting company based in New York City, with 50 employees serving many clients ranging from small family businesses to large corporations. The company is going strong after 26 years in business!

Even before the pandemic most of us worked remotely from home offices. We collaborate using SSH, Git, project tracking tools, Zulip chat, video conferencing, and of course email and phones.

What you will be doing:

Develop new web applications and support existing ones for our clients.
Work together with End Point Dev co-workers and our clients’ in-house staff.
Use your desktop operating system of choice: Linux, macOS, or Windows.
Enhance open source software and contribute back as opportunity arises.

You’ll need professional development experience with:

4+ years of development with Java for server-side applications
Frameworks and libraries such as Wildfly, Hibernate, Spring, Struts, Play
Databases such as PostgreSQL, Redis, Solr, Elasticsearch, etc.
Security consciousness, such as under HIPAA for safe medical & PII data handling
Git version control
Automated testing
HTTP, REST APIs

You have these important work traits:

Strong verbal and written communication skills
An eye for detail
Tenacity in solving problems and focusing on customer needs
A feeling of ownership of your projects
Work both independently and as part of a team
A good remote work environment

For some of our clients you will need to submit to and pass a criminal background check.

What work here offers:

Collaboration with knowledgeable, friendly, helpful, and diligent co-workers around the world
Freedom from being tied to an office location
Flexible, sane work hours
Paid holidays and vacation
Health insurance subsidy and 401(k) retirement savings plan
Annual bonus opportunity

Get in touch with us:

~~Please email us an introduction to jobs@endpointdev.com to apply.~~ (This job has been filled.)

Include your location, a resume/CV, your Git repository or LinkedIn URLs, and whatever else may help us get to know you.

We look forward to hearing from you! Direct work seekers only, please—this role is not for agencies or subcontractors.

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of sex/gender, race, religion, color, national origin, sexual orientation, age, marital status, veteran status, or disability status.

Creating a Messaging App Using Spring for Apache Kafka, Part 4

2021-01-14T00:00:00+00:00

Photo by Daniel Tran on Unsplash

This article is part of a series. The GitHub repository with code examples can be found here.

After a long break from this series let’s keep moving further.

Let’s configure and prepare the WebSocket session pool. As we go through some custom operations like authentication and storing messages on the time of socket messages and sessions received, we need to create a WebSocketHandler for WebSocket configuration. When a WebSocket session message is received we’re going to send the message to a Kafka topic. In order to achieve this we need to define our WebSocket message handler MessageHandler and Kafka message producer MessageSender. We also need a session pool WebSocketPool so we can manage the client sessions.

WebSocketPool

package com.endpoint.SpringKafkaMessaging.websocket;

import org.springframework.web.socket.WebSocketSession;

import java.util.HashMap;
import java.util.Map;
import java.util.Set;

public class WebSocketPool {
  public static Map<Long, Set<WebSocketSession>> websockets = new HashMap<>();
}

WebSocketPool holds client sessions in a map of <user_id, <set of WebSocketSession>>. This map allows multiple sessions for one user, ensuring it will work from multiple client applications.

MessageHandler

package com.endpoint.SpringKafkaMessaging.websocket;

import org.springframework.web.socket.WebSocketSession;

import java.io.IOException;

public interface MessageHandler {
  public void addSessionToPool(Long userId, WebSocketSession session);
  public void sendMessageToUser(Long userId, String message) throws IOException;
  void removeFromSessionToPool(Long userId, WebSocketSession session);
}

MessageHandlerImpl

package com.endpoint.SpringKafkaMessaging.websocket;

import org.springframework.stereotype.Service;
import org.springframework.web.socket.TextMessage;
import org.springframework.web.socket.WebSocketSession;

import java.io.IOException;
import java.util.HashSet;
import java.util.Set;

@Service
public class MessageHandlerImpl implements MessageHandler {

  @Override
  public void addSessionToPool(Long userId, WebSocketSession session) {

    Set<WebSocketSession> userSessions = WebSocketPool.websockets.get(userId);

    if (userSessions != null) {
      userSessions.add(session);
      WebSocketPool.websockets.put(userId, userSessions);
    } else {
      Set<WebSocketSession> newUserSessions = new HashSet<>();
      newUserSessions.add(session);
      WebSocketPool.websockets.put(userId, newUserSessions);
    }

  }

  @Override
  public void sendMessageToUser(Long userId, String message) throws IOException {

    Set<WebSocketSession> userSessions = WebSocketPool.websockets.get(userId);

    if (userSessions == null) {
      return;
    }

    TextMessage textMessage = new TextMessage(message);
    for (WebSocketSession session : userSessions) {
      session.sendMessage(textMessage);
    }

  }

  @Override
  public void removeFromSessionToPool(Long userId, WebSocketSession session) {
    Set<WebSocketSession> userSessions = WebSocketPool.websockets.get(userId);

    if (userSessions != null) {
      for (WebSocketSession sessionItem : userSessions) {
        if (sessionItem.equals(session)) {
          userSessions.remove(session);
        }
      }
    }
    WebSocketPool.websockets.put(userId, userSessions);
  }
}

With MessageHandler we are able to handle adding or removing WebSocketSession sessions from the pool as well as sending messages to a target user’s WebSocket sessions. On the other hand, MessageSender is going to define a KafkaTemplate broker to send the messages to a specified Kafka topic.

MessageSender

package com.endpoint.SpringKafkaMessaging.message.broker;

import org.json.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;

@Service
public class MessageSender {

  private static final Logger LOG = LoggerFactory.getLogger(MessageSender.class);

  @Autowired
  private KafkaTemplate<String, String> kafkaTemplate;

  public void send(String topic, String message) {
    kafkaTemplate.send(topic, message);
  }
}

Now we can create our WebSocketHandler. It will utilize MessageHandler to add and remove user sessions and MessageSender to send WebSocketSession messages to Kafka topics. I only used one topic in this messaging app, SEND_MESSAGE, but in a real application there would be a bunch of Kafka topics for sending different types of messages like new contact requests, notifications, etc.

WebSocketHandler

package com.endpoint.SpringKafkaMessaging.websocket;

import com.endpoint.SpringKafkaMessaging.cache.respository.CacheRepository;
import com.endpoint.SpringKafkaMessaging.message.broker.MessageSender;
import com.endpoint.SpringKafkaMessaging.persistent.model.User;
import com.endpoint.SpringKafkaMessaging.persistent.repository.UserRepository;
import org.json.JSONObject;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.web.socket.CloseStatus;
import org.springframework.web.socket.TextMessage;
import org.springframework.web.socket.WebSocketSession;
import org.springframework.web.socket.handler.TextWebSocketHandler;

@Component
public class WebSocketHandler extends TextWebSocketHandler {

  @Autowired
  CacheRepository cacheRepository;

  @Autowired
  UserRepository userRepository;

  @Autowired
  MessageHandler messageHandler;

  @Override
  public void afterConnectionClosed(WebSocketSession session, CloseStatus status) throws Exception {

    String parameters[] = session.getUri().getQuery().split("=");

    if (parameters.length == 2 && parameters[0].equals("accessToken")) {
      String accessToken = parameters[1];

      Long senderUserId = 0L;
      String senderId = cacheRepository.getUserIdByAccessToken(accessToken);

      if (senderId == null) {
        User sender = userRepository.findByToken(accessToken);
        if (sender != null) {
          senderUserId = sender.getUserId();
        }
      } else {
        senderUserId = Long.valueOf(senderId);
      }
      if (senderUserId == 0L) {
        return;
      }

      messageHandler.removeFromSessionToPool(senderUserId, session);
    }

  }

  @Override
  public void afterConnectionEstablished(WebSocketSession session) throws Exception {

    String parameters[] = session.getUri().getQuery().split("=");

    if (parameters.length == 2 && parameters[0].equals("accessToken")) {
      String accessToken = parameters[1];

      Long senderUserId = 0L;
      String senderId = cacheRepository.getUserIdByAccessToken(accessToken);

      if (senderId == null) {
        User sender = userRepository.findByToken(accessToken);
        if (sender != null) {
          senderUserId = sender.getUserId();
        }
      } else {
        senderUserId = Long.valueOf(senderId);
      }
      if (senderUserId == 0L) {
        return;
      }

      messageHandler.addSessionToPool(senderUserId, session);
    }
    else {
      session.close();
    }

  }

  @Autowired
  private MessageSender sender;

  @Override
  protected void handleTextMessage(WebSocketSession session, TextMessage textMessage) throws Exception {

    JSONObject jsonObject = new JSONObject(textMessage.getPayload());
    String topic = jsonObject.getString("topic");

    // only SEND_MESSAGE topic is available
    if (topic == null && !topic.equals("SEND_MESSAGE")) {
      return;
    }

    sender.send(topic, textMessage.getPayload());
  }
}

We can configure WebSocket in our application by adding the WebSocketHandler Bean and our custom MessageHandler.

WebSocketConfig

package com.endpoint.SpringKafkaMessaging.websocket;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.socket.config.annotation.EnableWebSocket;
import org.springframework.web.socket.config.annotation.WebSocketConfigurer;
import org.springframework.web.socket.config.annotation.WebSocketHandlerRegistry;

@Configuration
@EnableWebSocket
public class WebSocketConfig implements WebSocketConfigurer {

  @Bean
  public WebSocketHandler myMessageHandler() {
    return new WebSocketHandler();
  }

  @Override
  public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
    registry.addHandler(myMessageHandler(), "/messaging").setAllowedOrigins("*");
  }

}

Now our WebSocket configuration is ready. Let’s configure our MessageService:

MessageService

package com.endpoint.SpringKafkaMessaging.message;

import com.endpoint.SpringKafkaMessaging.persistent.model.Message;

import java.util.List;

public interface MessageService {
  public void sendMessage(String accessToken, Long sendTo, String msg);
  List<Message> getMessageHistory(Long fromUserId, Long toUserId);
}

With our MessageService interface we can define common messaging methods to be utilized by the Kafka listeners. Notice the sendMessage method where we take accessToken, sendTo and msg parameters. accessToken will be used to authenticate user sending the message. sendTo is the user_id of the recipient.

MessageServiceImpl

package com.endpoint.SpringKafkaMessaging.message;

import com.endpoint.SpringKafkaMessaging.cache.respository.CacheRepository;
import com.endpoint.SpringKafkaMessaging.persistent.model.Message;
import com.endpoint.SpringKafkaMessaging.persistent.model.User;
import com.endpoint.SpringKafkaMessaging.persistent.repository.MessageRepository;
import com.endpoint.SpringKafkaMessaging.persistent.repository.UserRepository;
import com.endpoint.SpringKafkaMessaging.websocket.MessageHandler;
import org.json.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.List;

@Service
public class MessageServiceImpl implements MessageService {

  private static final Logger LOGGER = LoggerFactory.getLogger(MessageServiceImpl.class);

  @Autowired
  MessageRepository messageRepository;

  @Autowired
  CacheRepository cacheRepository;

  @Autowired
  UserRepository userRepository;

  @Autowired
  MessageHandler messageHandler;

  @Override
  public void sendMessage(String accessToken, Long sendTo, String msg) {

    Long senderUserId = 0L;
    String senderId = cacheRepository.getUserIdByAccessToken(accessToken);

    if (senderId == null) {
      User sender = userRepository.findByToken(accessToken);
      if (sender != null) {
        senderUserId = sender.getUserId();
      }
    } else {
      senderUserId = Long.valueOf(senderId);
    }
    if (senderUserId == 0L) {
      return;
    }

    try {
      // enrich message with senderId
      JSONObject msgJson = new JSONObject();
      msgJson.put("msg", msg);
      msgJson.put("senderId", senderUserId);
      messageHandler.sendMessageToUser(sendTo, msgJson.toString());
    } catch (IOException e) {
      return;
    }
  }

  @Override
  public List<Message> getMessageHistory(Long fromUserId, Long toUserId) {
    return messageRepository.findByFromUserIdAndToUserId(fromUserId, toUserId);
  }

  private void storeMessageToUser(Message message) {
    messageRepository.save(message);
  }
}

We are going to use message brokers as MessageSender and MessageReceiver. We’re going to define a KafkaListener listening to SEND_MESSAGE topic inside MessageReceiver, which is actually sending message to the target user. Here notice that MessageReceiver is not a client method; because this is a server side message orchestration application, it does not directly serve as a client interface. Above we implemented the WebSocket sessions where the actual communication with the clients happens, but the client application is out of our scope.

MessageReceiver

package com.endpoint.SpringKafkaMessaging.message.broker;

import com.endpoint.SpringKafkaMessaging.message.MessageService;
import com.endpoint.SpringKafkaMessaging.websocket.MessageHandler;
import com.endpoint.SpringKafkaMessaging.websocket.WebSocketPool;
import org.json.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.messaging.MessageHeaders;
import org.springframework.messaging.handler.annotation.Headers;
import org.springframework.messaging.handler.annotation.Payload;
import org.springframework.stereotype.Service;

@Service
public class MessageReceiver {

  private static final Logger LOG = LoggerFactory.getLogger(MessageReceiver.class);

  @Autowired
  MessageService messageService;

  @Autowired
  MessageHandler messageHandler;

  @KafkaListener(topics = "SEND_MESSAGE")
  public void messagesSendToUser(@Payload String message, @Headers MessageHeaders headers) {

    JSONObject jsonObject = new JSONObject(message);

    if (WebSocketPool.websockets.get(jsonObject.getString("sendTo")) != null) {

      String accessToken = jsonObject.getString("accessToken");
      Long sendTo = Long.parseLong(jsonObject.getString("sendTo"));
      String msg = jsonObject.getString("msg");

      messageService.sendMessage(accessToken, sendTo, msg);

    }
  }

}

Here’s what our simple authentication service looks like:

AuthService

package com.endpoint.SpringKafkaMessaging.auth;

public interface AuthService {
  void putAccessToken(String accessToken, Long userId);
  Long loginWithAccessToken(String code);
}

AuthServiceImpl

package com.endpoint.SpringKafkaMessaging.auth;

import com.endpoint.SpringKafkaMessaging.persistent.model.User;
import com.endpoint.SpringKafkaMessaging.persistent.repository.UserRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import com.endpoint.SpringKafkaMessaging.cache.respository.CacheRepository;
import com.endpoint.SpringKafkaMessaging.persistent.model.AccessToken;
import com.endpoint.SpringKafkaMessaging.persistent.repository.AccessTokenRepository;

import java.util.Calendar;
import java.util.UUID;

@Service
public class AuthServiceImpl implements AuthService {

  @Autowired
  CacheRepository cacheRepository;

  @Autowired
  AccessTokenRepository accessTokenRepository;

  @Autowired
  UserRepository userRepository;

  @Override
  public void putAccessToken(String token, Long userId) {

    // store token in the cache
    cacheRepository.putAccessToken(token, String.valueOf(userId));

    // store token in the persistence
    AccessToken accessToken = AccessToken.builder()
                  .token(token)
                  .userId(userId)
                  .createdAt(Calendar.getInstance().getTime())
                  .build();
    accessTokenRepository.save(accessToken);
  }

  @Override
  public Long loginWithAccessToken(String token) {
    String userIdStr = cacheRepository.getUserIdByAccessToken(token);
    if (userIdStr == null) {
      User user = userRepository.findByToken(token);
      if (user != null)
        return user.getUserId();
      else
        return 0L;
    }
    return Long.valueOf(userIdStr);
  }
}

AuthController

package com.endpoint.SpringKafkaMessaging.auth.controller;

import javax.validation.Valid;

import com.endpoint.SpringKafkaMessaging.persistent.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import com.endpoint.SpringKafkaMessaging.auth.AuthService;
import com.endpoint.SpringKafkaMessaging.cache.respository.CacheRepository;
import com.endpoint.SpringKafkaMessaging.persistent.repository.UserRepository;
import com.endpoint.SpringKafkaMessaging.util.StringHelper;

import java.util.UUID;


@RestController
@RequestMapping("/api/auth")
public class AuthController {

  @Autowired
  UserRepository userRepository;

  @Autowired
  AuthService authService;

  @Autowired
  CacheRepository cacheRepository;
  
  @RequestMapping(value = "/getcode", method = RequestMethod.POST, consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
  public ResponseEntity<Object> getCode(@Valid @RequestBody ActivationRequest activationRequest) {
    
    int code = StringHelper.generateRandomNumber(6);
    
    // save the activation code to the cache repository (cached auth token)
    cacheRepository.putActivationCode(activationRequest.getMobile(), String.valueOf(code));

    ActivationResponse activationResponse = ActivationResponse.builder()
        .mobile(activationRequest.getMobile())
        .activationCode(String.valueOf(code))
        .build();
    
    return new ResponseEntity<>(
        activationResponse,
        HttpStatus.OK);
  }
  
  @RequestMapping(value = "/login", method = RequestMethod.POST, consumes = MediaType.APPLICATION_JSON_UTF8_VALUE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
  public ResponseEntity<Object> login(@RequestBody LoginRequest loginRequest) {
    String mobile = cacheRepository.queryMobileActivationCode(loginRequest.getMobile(), loginRequest.getActivationCode());

    if (mobile == null) {
      return new ResponseEntity<>(
          "Mobile number not found!",
          HttpStatus.NOT_FOUND);
    } else {
      Long userId = 0L;
      User user = userRepository.findByMobile(loginRequest.getMobile());
      if (user == null) {
        // save user in persistence
        userRepository.save(
            User.builder()
            .mobile(loginRequest.getMobile())
            .build()
        );
        user = userRepository.findByMobile(loginRequest.getMobile());
      }
      userId = user.getUserId();
      String accessToken = UUID.randomUUID().toString();
      authService.putAccessToken(accessToken, userId);

      return new ResponseEntity<>(
          LoginResponse.builder()
              .accessToken(accessToken)
              .build(),
          HttpStatus.OK);
    }
  }

}

Here the authentication controller is consumed by a client application, which will use the /getcode endpoint to get an activation code for a mobile number. Then the client application will use the /login endpoint, passing their activation code. Once the /login endpoint confirms the activation code, it will return an access token to the client for future requests. The client application will be responsible for adding the accessToken to all WebSocket session messages. Here we used a StringHelper method to generate the activation code and a random UUID string to generate an accessToken. Since UUID is not an ideal random token generator for authentication, you should use your own token generator algorithm.

Demonstrating the HotSwap JVM

2020-11-25T00:00:00+00:00

Photo by Neil and Zulma Scott

For a recent Java development project I spent a while setting up an environment to take advantage of the HotSwap JVM, a Java virtual machine that automatically reloads classes when they change. This feature can potentially eliminate the need to redeploy each time code changes, reducing development cycle time considerably. While setting up the environment, I found I wanted a simple example of hot swapping available for my own experimentation, and I thought I’d share that example here.

First, let’s create a simple Java program. It needs to be slightly more complex than the ubiquitous “Hello, World!” application, because we need it to keep running for a while; if it just prints some message and exits immediately, we won’t have time to compile new code and see the hot swap feature in action. Here’s an example that uses a simple infinite loop, wherein it sleeps for one second, prints a message, and then repeats.

import java.lang.Thread;

public class HotSwapTest {
  public static void main(String[] args) {
    while (true) {
      try {
        System.out.println("Hi");
        Thread.sleep(1000);
      } catch (InterruptedException e) {
        // Ignore this
      }
    }
  }
}
~

If I build this into build/classes/java/main and run it, as expected it prints out “Hi” every second:

josh@igtre:~/hotswaptest$ java -cp build/classes/java/main/ HotSwapTest
Hi
Hi
Hi
...

The usual JVM doesn’t include the HotSwap feature. For my purposes I downloaded DCEVM, an alternative JVM which includes HotSwap. It’s also possible to patch some existing JVMs to add HotSwap, if you’d prefer. When I run the same code with DCEVM, it runs the code just like it did with the normal JVM, with additional debugging output:

Starting HotswapAgent '/home/josh/hotswaptest/dcevm/lib/hotswap/hotswap-agent.jar'
HOTSWAP AGENT: 15:01:23.423 INFO (org.hotswap.agent.HotswapAgent) - Loading Hotswap agent {1.4.1} - unlimited runtime class redefinition.
HOTSWAP AGENT: 15:01:24.189 INFO (org.hotswap.agent.config.PluginRegistry) - Discovered plugins: [JdkPlugin, Hotswapper, WatchResources, ClassInitPlugin, AnonymousClassPatch, Hibernate, Hibernate3JPA, Hibernate3, Spring, Jersey1, Jersey2, Jetty, Tomcat, ZK, Logback, Log4j2, MyFaces, Mojarra, Omnifaces, ELResolver, WildFlyELResolver, OsgiEquinox, Owb, Proxy, WebObjects, Weld, JBossModules, ResteasyRegistry, Deltaspike, GlassFish, Vaadin, Wicket, CxfJAXRS, FreeMarker, Undertow, MyBatis]

To make hot swapping work automatically, we need to provide the JVM with a properties file in the JVM’s classpath. Mine looks like this, and lives in build/classes/java/main, next to the compiled class files:

autoHotswap=true
LOGGER=debug

These properties are pretty self-explanatory: they tell the JVM to hot swap automatically when it finds new code, and turn up logging to DEBUG level.

So, with that all set up, let’s run the program again, change the code and rebuild it, and see what happens. For this test, I’ll just edit the message printed in each loop from “Hi” to “Hello”.

josh@igtre:~/hotswaptest$ ./dcevm/bin/java -cp build/classes/java/main/ HotSwapTest
Starting HotswapAgent '/home/josh/hotswaptest/dcevm/lib/hotswap/hotswap-agent.jar'
HOTSWAP AGENT: 15:36:22.132 INFO (org.hotswap.agent.HotswapAgent) - Loading Hotswap agent {1.4.1} - unlimited runtime class redefinition.
HOTSWAP AGENT: 15:36:22.543 DEBUG (org.hotswap.agent.annotation.handler.OnClassLoadedHandler) - Init for method public static void org.hotswap.agent.plugin.jdk.JdkPlugin.flushIntrospectClassInfoCache(java.lang.ClassLoader,org.hotswap.agent.javassist.CtClass)
HOTSWAP AGENT: 15:36:22.546 DEBUG (org.hotswap.agent.util.HotswapTransformer) - Registering transformer for class regexp '.*'.
HOTSWAP AGENT: 15:36:22.549 DEBUG (org.hotswap.agent.annotation.handler.OnClassLoadedHandler) - Init for method public static void org.hotswap.agent.plugin.jdk.JdkPlugin.flushObjectStreamCaches(java.lang.ClassLoader,org.hotswap.agent.javassist.CtClass)
HOTSWAP AGENT: 15:36:22.550 DEBUG (org.hotswap.agent.util.HotswapTransformer) - Registering transformer for class regexp '.*'.
...

The flurry of DEBUG messages tells me that it must have read my properties file correctly, and when I change the code and rebuild, I see the JVM respond with still more debug messages, saying it found and reloaded my code:

HOTSWAP AGENT: 15:38:27.066 DEBUG (org.hotswap.agent.watch.nio.WatcherNIO2) - Watch event 'ENTRY_DELETE' on '/home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class' --> HotSwapTest.class
HOTSWAP AGENT: 15:38:27.118 DEBUG (org.hotswap.agent.watch.nio.WatcherNIO2) - Watch event 'ENTRY_CREATE' on '/home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class' --> HotSwapTest.class
HOTSWAP AGENT: 15:38:27.119 DEBUG (org.hotswap.agent.watch.nio.WatcherNIO2) - Watch event 'ENTRY_MODIFY' on '/home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class' --> HotSwapTest.class
HOTSWAP AGENT: 15:38:27.280 DEBUG (org.hotswap.agent.annotation.handler.WatchEventCommand) - Executing resource changed method watchReload on class org.hotswap.agent.plugin.hotswapper.HotswapperPlugin for event WatchFileEvent on path /home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class for event ENTRY_MODIFY
...
HOTSWAP AGENT: 15:38:27.439 DEBUG (org.hotswap.agent.plugin.jdk.JdkPlugin) - Flushing HotSwapTest from introspector
Hi
HOTSWAP AGENT: 15:38:27.482 DEBUG (org.hotswap.agent.config.PluginManager) - ... reloaded classes [HotSwapTest] (autoHotswap)
Hi
Hi
Hi

But although it says it swapped in the new code successfully, it’s still printing “Hi”, not “Hello”. Why?

It turns out this ability to hot swap new code isn’t unlimited, and one important limitation is that methods you’re already running aren’t reloaded. Since the main() method was running, it didn’t get swapped out. What if I change the code so that instead of printing a string every second, it calls a method, and that method prints the string? Here’s some code to test that technique.

import java.lang.Thread;

public class HotSwapTest {
  public static void printMsg() {
    System.out.println("Here is printMsg");
  }

  public static void main(String[] args) {
    while (true) {
      try {
        HotSwapTest.printMsg();
        Thread.sleep(1000);
      } catch (InterruptedException e) {
        // Ignore this
      }
    }
  }
}

Now, I start the JVM over again, and as expected, it prints “Here is printMsg” once every second. When I change to “Here is printMsg v2.0” and rebuild, this happens:

Here is printMsg
Here is printMsg
HOTSWAP AGENT: 15:48:34.923 DEBUG (org.hotswap.agent.watch.nio.WatcherNIO2) - Watch event 'ENTRY_DELETE' on '/home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class' --> HotSwapTest.class
HOTSWAP AGENT: 15:48:35.007 DEBUG (org.hotswap.agent.watch.nio.WatcherNIO2) - Watch event 'ENTRY_CREATE' on '/home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class' --> HotSwapTest.class
HOTSWAP AGENT: 15:48:35.008 DEBUG (org.hotswap.agent.watch.nio.WatcherNIO2) - Watch event 'ENTRY_MODIFY' on '/home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class' --> HotSwapTest.class
HOTSWAP AGENT: 15:48:35.174 DEBUG (org.hotswap.agent.annotation.handler.WatchEventCommand) - Executing resource changed method watchReload on class org.hotswap.agent.plugin.hotswapper.HotswapperPlugin for event WatchFileEvent on path /home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class for event ENTRY_CREATE
HOTSWAP AGENT: 15:48:35.188 DEBUG (org.hotswap.agent.plugin.hotswapper.HotswapperPlugin) - Class HotSwapTest will be reloaded from URL file:/home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class
HOTSWAP AGENT: 15:48:35.175 DEBUG (org.hotswap.agent.annotation.handler.WatchEventCommand) - Executing resource changed method watchReload on class org.hotswap.agent.plugin.hotswapper.HotswapperPlugin for event WatchFileEvent on path /home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class for event ENTRY_MODIFY
HOTSWAP AGENT: 15:48:35.191 DEBUG (org.hotswap.agent.plugin.hotswapper.HotswapperPlugin) - Class HotSwapTest will be reloaded from URL file:/home/josh/hotswaptest/build/classes/java/main/HotSwapTest.class
HOTSWAP AGENT: 15:48:35.352 DEBUG (org.hotswap.agent.command.impl.SchedulerImpl) - Executing pluginManager.hotswap([class HotSwapTest])
HOTSWAP AGENT: 15:48:35.355 RELOAD (org.hotswap.agent.config.PluginManager) - Reloading classes [HotSwapTest] (autoHotswap)
HOTSWAP AGENT: 15:48:35.370 DEBUG (org.hotswap.agent.plugin.jdk.JdkPlugin) - Flushing HotSwapTest from com.sun.beans.introspect.ClassInfo cache
HOTSWAP AGENT: 15:48:35.375 DEBUG (org.hotswap.agent.plugin.jdk.JdkPlugin) - Flushing HotSwapTest from ObjectStreamClass caches
HOTSWAP AGENT: 15:48:35.376 DEBUG (org.hotswap.agent.plugin.jdk.JdkPlugin) - Flushing HotSwapTest from introspector
HOTSWAP AGENT: 15:48:35.415 DEBUG (org.hotswap.agent.config.PluginManager) - ... reloaded classes [HotSwapTest] (autoHotswap)
Here is printMsg v2.0
Here is printMsg v2.0

As you can see, it swapped in the new code correctly, and now prints the new message. HotSwap was a success!

I imagine there are very few production environments where this feature would be applicable, and even in development, getting this to work properly for something like a JEE app deployed to some application container isn’t necessarily a simple task. But if it can cut down on redeployment cycles, it can certainly be a valuable developer tool.

Job opening: Java & JavaScript developer

2020-11-13T00:00:00+00:00

We are seeking a full-time Java software engineer to work with us on our clients’ applications.

End Point is an Internet technology consulting company based in New York City, with 50 employees serving many clients ranging from small family businesses to large corporations. The company turned 25 years old this year!

Even before the pandemic most of us worked remotely from home offices. We collaborate using SSH, Git, project tracking tools, Zulip chat, video conferencing, and of course email and phones.

What you will be doing:

Develop new web applications and support existing ones for our clients
Work together with End Point co-workers and our clients’ in-house staff
Use your desktop OS of choice: Linux, macOS, Windows
Use open source tools and contribute back as opportunity arises

What you bring:

Professional experience developing and supporting web applications in these technical areas:

3+ years of development with Java and front-end JavaScript
Frameworks and libraries such as Wildfly, Hibernate, Spring, Struts, Play, and Vue.js, React, Angular
Databases such as PostgreSQL, MySQL, Redis, Solr, Elasticsearch, etc.
Security consciousness
Git version control
Automated testing
HTTP, REST APIs
Bonus for familiarity with other ecosystems such as Ruby on Rails, PHP, Python/Django, .NET/C#, Node.js, Go, Rust, Scala, Kotlin, Swift …

These work traits are just as important:

Strong verbal and written communication skills
An eye for detail
Tenacity in solving problems and focusing on customer needs
A feeling of ownership of your projects
Work both independently and as part of a team
Ability to pass a criminal background check when clients require
A good remote work environment

What work here offers:

Collaborate with knowledgeable, friendly, helpful, and diligent co-workers around the world
Flexible, sane work hours
Paid holidays and vacation
Annual bonus opportunity
Freedom from being tied to an office location
For U.S. employees: health insurance subsidy and 401(k) retirement savings plan

Get in touch with us:

~~Please email us an introduction to jobs@endpointdev.com to apply.~~ (This job has been filled.) Include your location, a resume/CV, your Git repository or LinkedIn URLs, and whatever else may help us get to know you.

We look forward to hearing from you! Direct work seekers only, please—this role is not for agencies or subcontractors.

Creating a Messaging App Using Spring for Apache Kafka, Part 3

2020-05-21T00:00:00+00:00

Photo by Pascal Debrunner on Unsplash

This article is part of a series. The GitHub repository with code examples can be found here.

In this article we’ll create the persistence and cache models and repositories. We’re also going to create our PostgreSQL database and the basic schema that we’re going to map to the persistence model.

Persistence

Database

We are going to keep the persistence model as simple as possible so we can focus on the overall functionality. Let’s first create our PostgreSQL database and schema. Here is the list of tables that we’re going to create:

users: will hold the users who are registered to use this messaging service.
access_token: will hold the unique authentication tokens per session. We’re not going to implement an authentication and authorization server specifically in this series but rather will generate a simple token and store it in this table.
contacts: will hold relationships of existing users.
messages: will hold messages sent to users.

Let’s create our tables:

CREATE TABLE kafkamessaging.users (
    user_id BIGSERIAL PRIMARY KEY,
    fname VARCHAR(32) NOT NULL,
    lname VARCHAR(32) NOT NULL,
    mobile VARCHAR(32) NOT NULL,
    created_at DATE NOT NULL
);

CREATE TABLE kafkamessaging.access_token (
    token_id BIGSERIAL PRIMARY KEY,    
    token VARCHAR(256) NOT NULL,
    user_id BIGINT NOT NULL REFERENCES kafkamessaging.users(user_id),
    created_at DATE NOT NULL
);

CREATE TABLE kafkamessaging.contacts (
    contact_id BIGSERIAL PRIMARY KEY,
    user_id BIGINT NOT NULL REFERENCES kafkamessaging.users(user_id),
    contact_user_id BIGINT NOT NULL REFERENCES kafkamessaging.users(user_id),
);

CREATE TABLE kafkamessaging.messages (
    message_id BIGSERIAL PRIMARY KEY,
    from_user_id BIGINT NOT NULL REFERENCES kafkamessaging.users(user_id),
    to_user_id BIGINT NOT NULL REFERENCES kafkamessaging.users(user_id),
    message VARCHAR(512) NOT NULL,
    sent_at DATE NOT NULL
);

Model

Before creating the models we’ll add another dependency called Lombok in pom.xml as shown below. Lombok provides very helpful annotations which automatically create getters, setters and many other parts of a class.

    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
    </dependency>

So here are the persistent model classes of the corresponding tables we created in the database. Notice the Lombok and javax.persistence annotations in the model classes:

User

package com.endpoint.SpringKafkaMessaging.persistent.model;

import java.io.Serializable;
import java.util.Date;
import java.util.Set;

import javax.persistence.CascadeType;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.FetchType;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.OneToMany;
import javax.persistence.Table;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Entity
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@Table(name="users")
public class User implements Serializable {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @Column(name="user_id")
    private Long userId;

    @Column
    private String fname;

    @Column
    private String lname;

    @Column
    private String mobile;
    
    @Column(name="created_at")
    private Date createdAt;
    
    @OneToMany(mappedBy = "User", fetch = FetchType.EAGER,
            cascade = CascadeType.ALL)
    private Set<Contact> contacts;

}

AccessToken

package com.endpoint.SpringKafkaMessaging.persistent.model;

import java.io.Serializable;
import java.util.Date;
import java.util.Map;
import java.util.UUID;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Entity
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@Table(name="access_token")
public class AccessToken implements Serializable {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long token_id;
    
    @Column(name="token")
    private String token;

    @Column(name="user_id")
    private Long userId;

    @Column(name="created_at")
    private Date createdAt;

}

Contact

package com.endpoint.SpringKafkaMessaging.persistent.model;

import java.io.Serializable;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.FetchType;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.JoinColumn;
import javax.persistence.ManyToOne;
import javax.persistence.Table;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Entity
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@Table(name="contacts")
public class Contact implements Serializable {
    
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @Column(name="contact_id")
    private Long contactId;
    
    @Column(name="user_id")
    private Long userId;

    @Column(name="contact_user_id")
    private Long contactUserId;
    
    @ManyToOne(fetch = FetchType.LAZY, optional = false)
    @JoinColumn(name = "user_id", nullable = false)
    private User user;
}

Message

package com.endpoint.SpringKafkaMessaging.persistent.model;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.io.Serializable;
import java.util.Date;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;

@Entity
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@Table(name="messages")
public class Message implements Serializable {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @Column(name="message_id")
    private String messageId;

    @Column(name="from_user_id")
    private Long fromUserId;

    @Column(name="to_user_id")
    private Long toUserId;

    @Column(name="message")
    private String message;
    
    @Column(name="sent_at")
    private Date sentAt;

}

Note also that we didn’t use underscores in the class field names for the corresponding table field names like userId for user_id.

We’re going to use Spring’s CrudRepository interface to create our data repositories. CrudRepository can use keywords to automatically create logic using the given interface method names. Underscores are reserved characters, and even though you can still escape using double underscore in the CrudRepository method names, it doesn’t look good. I chose to use camel case, which also complies with Java convention.

Repository

Now let’s add the corresponding persistent repositories for each data model:

UserRepository

package com.endpoint.SpringKafkaMessaging.persistent.repository;

import java.util.List;

import org.springframework.data.repository.CrudRepository;
import org.springframework.stereotype.Repository;

import com.endpoint.SpringKafkaMessaging.persistent.model.User;

@Repository
public interface UserRepository extends CrudRepository<User, Long> {

    List<User> findAll();

    User findByUserId(Long userId);
    
    User findByMobile(String mobile);
    
    User findByFname(String fname);
    
    User findByLname(String lname);

    void deleteById(Long userId);

}

ContactRepository

package com.endpoint.SpringKafkaMessaging.persistent.repository;

import java.util.List;

import org.springframework.data.repository.CrudRepository;
import org.springframework.stereotype.Repository;

import com.endpoint.SpringKafkaMessaging.persistent.model.Contact;

@Repository
public interface ContactRepository extends CrudRepository<Contact, Long> {

    List<Contact> findAllByUserId(Long userId);
    
    Contact findByContactUserId(Long contactUserId);

    void deleteByContactUserId(Long contactUserId);
}

AccessTokenRepository

package com.endpoint.SpringKafkaMessaging.persistent.repository;
import org.springframework.data.repository.CrudRepository;
import org.springframework.stereotype.Repository;

import com.endpoint.SpringKafkaMessaging.persistent.model.AccessToken;

@Repository
public interface AccessTokenRepository extends CrudRepository<AccessToken, Long> {

    AccessToken findByUserId(Long userId);

    void deleteByUserId(Long userId);

}

MessageRepository

package com.endpoint.SpringKafkaMessaging.persistent.repository;

import org.springframework.data.repository.CrudRepository;
import org.springframework.stereotype.Repository;

import com.endpoint.SpringKafkaMessaging.persistent.model.Message;
@Repository
public interface MessageRepository extends CrudRepository<Message, Long> {
    
}

Cache

We’re not going to integrate the cache environment as Spring persistent data, so we won’t be using the CrudRepository implementation for the cache repository. Instead, let’s create the cache repository interface and create an implementation of it. Caching is going to be used for quick activation and authentication responses. To achieve this we’re going to store and query simple key-value pairs with Redis.

Repository

CacheRepository

package com.endpoint.SpringKafkaMessaging.cache.respository;

public interface CacheRepository {

    void putAccessToken(String token, String userId);
    
    String getUserIdByAccessToken(String token);

    void putActivationCode(String mobile, String activationCode);
    
    String queryMobileActivationCode(String mobile, String activationCode);
}

Since the business logic of this interface is not automatically created by Spring Boot, we need to create our own logic in Spring’s @Service like below.

CacheRepositoryImpl

package com.endpoint.SpringKafkaMessaging.cache.respository;

import org.springframework.stereotype.Service;

import com.endpoint.SpringKafkaMessaging.cache.JedisFactory;

import redis.clients.jedis.Jedis;

@Service
public class CacheRepositoryImpl implements CacheRepository {

    @Override
    public void putAccessToken(String token, String userId) {

        try (Jedis jedis = JedisFactory.getConnection()) {

            jedis.set(token, userId);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Override
    public String getUserIdByAccessToken(String token) {

        try (Jedis jedis = JedisFactory.getConnection()) {

            return jedis.get(token);

        } catch (Exception e) {
            e.printStackTrace();
        }

        return null;
    }

    @Override
    public void putActivationCode(String mobile, String activationCode) {

        try (Jedis jedis = JedisFactory.getConnection()) {

            jedis.hset(mobile, String.valueOf(activationCode), "");
            jedis.expire(mobile, 15 * 60);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Override
    public String queryMobileActivationCode(String mobile, String code) {

        try (Jedis jedis = JedisFactory.getConnection()) {

            return jedis.hget(mobile, code);
        } catch (Exception e) {
            e.printStackTrace();
        }

        return null;
    }
}

Activation and Authentication

Activation is a one-time process to activate a mobile number for our messaging service client. After an activation our simple authentication service will provide an access token to messaging client, and this access token will be used for future client logins. To achieve these simple processes let’s create our authentication service interface.

AuthService

package com.endpoint.SpringKafkaMessaging.auth;

public interface AuthService {
    void putAccessToken(String code, Long userId);

    void loginWithAccessToken(String mobile, String code);
}

AuthServiceImpl

package com.endpoint.SpringKafkaMessaging.auth;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import com.endpoint.SpringKafkaMessaging.cache.respository.CacheRepository;
import com.endpoint.SpringKafkaMessaging.persistent.model.AccessToken;
import com.endpoint.SpringKafkaMessaging.persistent.repository.AccessTokenRepository;

import java.util.Calendar;

@Service
public class AuthServiceImpl implements AuthService {

    @Autowired
    CacheRepository cacheRepository;

    @Autowired
    AccessTokenRepository accessTokenRepository;

    @Override
    public void putAccessToken(String token, Long userId) {

        // store token in the cache
        cacheRepository.putAccessToken(token, String.valueOf(userId));

        // store token in the persistence
        AccessToken accessToken = AccessToken.builder()
                                    .token(token)
                                    .userId(userId)
                                    .createdAt(Calendar.getInstance().getTime())
                                    .build();
        accessTokenRepository.save(accessToken);
    }

    @Override
    public void loginWithAccessToken(String mobile, String code) {
        // TODO
    }
}

We won’t implement a complex auth server here.

Let’s look at the draft form of the authentication controller below. The authentication controller here simulates the mobile number activation and one time login with the activation code and provides a unique access token to client. To achieve this I defined activation request and response models.

ActivationRequest

package com.endpoint.SpringKafkaMessaging.auth.controller;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class ActivationRequest {
    
    private String mobile;
    
}

ActivationResponse

package com.endpoint.SpringKafkaMessaging.auth.controller;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class ActivationResponse {
    
    private String mobile;
    
    private String activationCode;
    
}

LoginRequest

package com.endpoint.SpringKafkaMessaging.auth.controller;

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class LoginRequest {
    
    private String mobile;
    
    private String activationCode;
    
}

AuthController

package com.endpoint.SpringKafkaMessaging.auth.controller;

import javax.validation.Valid;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import com.endpoint.SpringKafkaMessaging.auth.AuthService;
import com.endpoint.SpringKafkaMessaging.cache.respository.CacheRepository;
import com.endpoint.SpringKafkaMessaging.persistent.model.User;
import com.endpoint.SpringKafkaMessaging.persistent.repository.UserRepository;
@RestController
@RequestMapping("/api/auth")
public class AuthController {

    @Autowired
    UserRepository userRepository;

    @Autowired
    AuthService authService;

    @Autowired
    CacheRepository cacheRepository;
    
    @RequestMapping(value = "/getcode", method = RequestMethod.POST, consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
    public ResponseEntity<Object> getCode(@Valid @RequestBody ActivationRequest activationRequest) {
        
        // TODO
        
        return null;
    }
    
    @RequestMapping(value = "/login", method = RequestMethod.POST, consumes = MediaType.APPLICATION_JSON_UTF8_VALUE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
    public ResponseEntity<String> login(@RequestBody LoginRequest loginRequest) {
        
        // TODO
        
        return null;
    }
}

In the next chapter, we’ll shape and complete the authentication service and controller and add message sender and receiver services. We’ll also configure and enable Spring WebSocket.

In the final chapter, we’ll create a simple web app interface as a messaging client to test our spring-kafka messaging application.

Creating a Messaging App Using Spring for Apache Kafka, Part 2

2020-04-29T00:00:00+00:00

This article is part of a series. The GitHub repository with code examples can be found here.

In this part I’ll walk through Kafka’s servers and processes, the basics of spring-kafka producers and consumers, persistence, and caching configurations.

Kafka Servers

Kafka uses Apache ZooKeeper as the distributed coordination server. You can download the Apache Kafka with ZooKeeper bundle here.

When you download and untar the Kafka bundle Kafka’s console scripts can be found in the bin directory. To enable Kafka connectivity and prepare the Kafka configuration let’s start the Kafka servers and see how to create Kafka topics and test console producers and consumers.

ZooKeeper

To start ZooKeeper with the default properties run the following command:

bin/zookeeper-server-start.sh config/zookeeper.properties

Kafka Server

A single Kafka server with the default properties can be started with following command:

bin/kafka-server-start.sh config/server.properties

Kafka Topics

Creating Kafka Topics

Let’s create a test Kafka topic:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic myTestTopic

List Topics

To list all previously created Kafka topics:

bin/kafka-topics.sh --list --bootstrap-server localhost:9092

Start a Producer

To start a console producer run the following command and send some messages from console:

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic myTestTopic
> This is a message
> This is another message

Start a Consumer

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic myTestTopic --from-beginning

When you run the consumer on the console with the from-beginning parameter you’ll see all the messages sent previously shown in the console.

Here we ran Kafka as a single server. You’ll need to optimize and scale the Kafka clusters for production and large-scale distributed systems. So far, we’ve become familiar with some Kafka components but for further Kafka configuration you can refer to the corresponding tutorials.

spring-kafka Configuration

Consumer Configuration

In the Spring Boot project let’s put the lines below in the application.properties to configure the Spring Kafka consumer:

#Consumer
spring.kafka.consumer.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=foo
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer

A simple Kafka consumer is defined as a Spring @KafkaListener annotated method like this:

@Configuration
public class MyKafkaConsumer {

    @KafkaListener(topics = "myTestTopic")
    public void listenTopic(ConsumerRecord<String, String> kafkaMessage) {
        System.out.print(String.format("Received a message: %s", kafkaMessage.value()));
    }

}

We are going to define different Kafka consumer methods listening to different topics for different purposes in our messaging app.

Producer Configuration

For producer configuration let’s add the following lines in the application.properties in our Spring Kafka project.

#Producer
spring.kafka.producer.bootstrap-servers=localhost:9092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

A very simple Kafka producer could be configured like below. Spring KafkaTemplate provides a producer model and methods for sending messages to specified Kafka topics.

@Configuration
public class MyKafkaProducer {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public void sendMessage(String topic, String message) {
        System.out.println(String.format("Message is being sent to topic %s", topic));
        kafkaTemplate.send(topic, message);
    }

}

So far we have configured Kafka in a Spring Boot project and seen simple consumer and producer examples. Before going further with Kafka configuration, let’s configure the persistence and cache repositories.

Persistence Configuration

As I mentioned in the first part of this blog series, I’m going to use PostgreSQL as a persistence environment and Spring data configuration will be like below in the application.properties:

spring.datasource.url=jdbc:postgresql://localhost:5432/epmessagingdb
spring.datasource.username=epmessaging
spring.datasource.password=epmessagingdb_password
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.configuration.maximum-pool-size=30
spring.jpa.database-platform=PostgreSQL
# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
# Hibernate ddl auto (create, create-drop, validate, update)
spring.jpa.hibernate.ddl-auto=none

In the properties we set the spring.jpa.hibernate.ddl-auto Spring JPA property to none to avoid Hibernate populating the schema automatically. In some cases it can be useful to allow auto-population. We leave the base configuration here for now as it is, in the next part we’ll create our Spring Data models in the project.

Caching Configuration

I also mentioned that we’re going to use Redis as the cache environment. Redis is developed using C and a very fast in-memory cache.

Let’s put the following lines in application.properties to enable Redis configuration in our Spring Kafka project.

cache.redis.host=localhost
cache.redis.port=6379
cache.redis.timeout=5000
cache.redis.password=

Redis Pooling Factory

We’re going to use Jedis as the Redis client in our project. So let’s create a Jedis pooling factory class in our project called JedisFactory like below:

package com.endpoint.SpringKafkaMessaging.cache;

import org.springframework.beans.factory.annotation.Value;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

public class JedisFactory {

    @Value("${cache.redis.host}")
    private static String host;

    @Value("${cache.redis.port}")
    private static Integer port;

    @Value("${cache.redis.timeout}")
    private static Integer timeout;

    @Value("${cache.redis.password}")
    private static String password;

    // hide the constructor
    private JedisFactory() {

    }

    private static JedisPool jedisPool;

    static {
        JedisPoolConfig poolConfig = new JedisPoolConfig();
        poolConfig.setMaxTotal(128);

        jedisPool = new JedisPool(
            poolConfig,
            host,
            port,
            timeout,
            password
        );
    }

    public static Jedis getConnection() {
        return jedisPool.getResource();
    }
}

We’ll create a persistence model, repository, controllers, and a cache repository in the next part of this blog series.

Convenient Reporting with Jasper

2020-04-28T00:00:00+00:00

Business Intelligence (BI) reporting is a huge problem space in custom software. There’s a wide range of business needs for looking at past and predictive behavior. Building a reporting tool can be a very cost effective way to get this data, especially compared to writing individual queries or manually generating reports.

I’ve been working with Jasper in the Java project space and wanted to write about some research I’ve collected on the topic.

JasperReports takes .jrxml files as input and outputs a .jasper report. Possible output targets include:

Screen
Printer
PDF
HTML
Excel files
RTF
ODT
CSV
XML

Jasper history

June 2001: Teodor Danciu began working on JasperReports.
September 2001: Jasper was registered on SourceForge.
November 2001: JasperReports 0.1.5 was released.
2004: Panscopic teamed up with Teodor Danciu, acquired ownership of the product and changed its name to Jaspersoft.
2005: JasperReports 1.0 was released.
2007: Brian Gentile became CEO of the company.
2014: TIBCO acquired Jaspersoft for ~$185 million.

Best reporting tools

Let’s compare some popular reporting tools:

JasperReports is a free and open source Java-based reporting tool, which supports lots of possible outputs, as mentioned earlier. Generating reports can be difficult if you’re less technical. More technical aspects can be more difficult as well; embedding JasperReports into a project is not necessarily simple, but once it’s done, the tool will be reliable.
Crystal Reports supports many inputs, including Access, Excel, XML, ODBC, and JDBC. It also has good multi-language support. It’s easy to embed into a .NET project, but software updates are unstable. The process can be very slow and there is no control of data-level security. A trial version is offered, but if one wants to use it long-term, then the price is a one-time payment of $495 or more, for larger companies.
Domo is another a popular reporting tool. It provides a trial version, and a 5 user plan costs $5700/year*.
Zoho analytics is an easy-to-use BI reporting tool, priced between $22–$445, depending on the number of users and data.
Host analytics is a great tool for finance automation. Pricing not publicized.
Tableau is an excellent reporting tool, with a thriving community online, but its quote-based price is high.
Pentaho is a Java-based reporting tool, which provides data integration, online analytical processing and reporting, among other features. Pentaho offers a 30-day trial period. Contract pricing isn’t disclosed.

So, if you are writing software and already use Java, or using Java reporting is an option, JasperReports is a great choice. It supports a variety of outputs, is free to use, and open source.

Installing JasperReports Server

To install JasperReports Server, you need a computer with a fully functional JRE (Java Runtime Environment). The server might be Tomcat or GlassFish. An RDBMS is also needed. since JasperReports has its own database, this could be PostgreSQL, Oracle, MySQL, DB2, or SQL Server. JasperReports prefers PostgreSQL and Tomcat, so these will be included with an automatic install. You may choose to use your existing Tomcat/PostgreSQL or have it install them as well.

Manual installation is also possible, as described here. At my first encounter with Jasper I installed Tomcat with the installer and used it for generating JasperReports, while the application I was working with was running WildFly (formerly JBoss), using a MySQL database. Needless to say, this was unnecessary, but I was not aware of that at the time. JasperServer can be configured to work with JBoss and MySQL as well.

The core of JasperReports is the JasperReports Library, which was already integrated into the project I was working with and is integrated into JasperReports Server as well as into popular IDEs, like TIBCO Jaspersoft Studio or iReport Designer.

Database

JasperReports provides example databases for imaginary companies, like FoodMart:

As we can see these are normal tables, having a primary key and some other fields.

.jrxml

jrxml, which stands for Jasper XML, contains report definitions in XML format. This type of file can be edited as code or visually to be compiled into .jasper files. The community provides samples that can be used and understood. Among them we can find a JFreeChart sample, which contains a jrxml file, a preview HTML and an actual PDF. The .jrxml file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jasperReport PUBLIC "-//JasperReports//DTD Report Design//EN" "http://jasperreports.sourceforge.net/dtds/jasperreport.dtd">

<jasperReport name="JFreeChartReport" pageWidth="595" pageHeight="842" columnWidth="515" leftMargin="40" rightMargin="40" topMargin="50" bottomMargin="50" scriptletClass="JFreeChartScriptlet">
  <variable name="Chart" class="net.sf.jasperreports.engine.JRRenderable" calculation="System"/>
  <title>
    <band height="742">
      <line>
        <reportElement x="0" y="0" width="515" height="1"/>
        <graphicElement/>
      </line>
      <staticText>
        <reportElement x="0" y="10" width="515" height="30"/>
        <textElement textAlignment="Center">
          <font size="22"/>
        </textElement>
        <text><![CDATA[JFreeChart Sample]]></text>
      </staticText>
      <textField>
        <reportElement x="0" y="50" width="515" height="50"/>
        <textElement textAlignment="Center">
          <font size="12"/>
        </textElement>
        <textFieldExpression class="java.lang.String"><![CDATA["This sample uses JFreeChart Version 1.0.0-pre2\n" + "Written by David Gilbert (david.gilbert@object-refinery.com) and others.\n" + "(C)opyright 2000-2004, by Object Refinery Limited and Contributors."]]></textFieldExpression>
      </textField>
      <image scaleImage="Clip" hAlign="Center" hyperlinkType="Reference">
        <reportElement x="0" y="110" width="515" height="300"/>
        <graphicElement/>
        <imageExpression class="net.sf.jasperreports.engine.JRRenderable"><![CDATA[$V{Chart}]]></imageExpression>
        <hyperlinkReferenceExpression><![CDATA["http://www.jfree.org/jfreechart"]]></hyperlinkReferenceExpression>
      </image>
    </band>
  </title>
</jasperReport>

It starts with the xml tag, specifying that this file should be interpreted as XML. Then comes the DOCTYPE and finally the jasperReport node, which contains the actual report nodes. A variable is defined, called Chart, which is used later in the inner XML of the image node. A hyperlink is defined for the image. The preview for this report looks like this:

Don’t worry about the broken images; this is just the preview, the actual result looks like this:

Nice, isn’t it?

Data source

It’s nice to generate reports, but in many cases the content is not fully known at programming time. It’s quite possible that we need to provide some input for the template. For this purpose, the interface of JRDataSource was defined to be iterated with a .next() method and is readable via the .getFieldValue() method. To make sure that we can read fields, another interface, JRField was defined as well. We therefore will need to use implementations of these classes, possibly writing our own if the currently available implementations are not fulfilling our needs.

Let’s consider a datasource sample, also taken from the community. It has this .jrxml template:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jasperReport PUBLIC "-//JasperReports//DTD Report Design//EN" "http://jasperreports.sourceforge.net/dtds/jasperreport.dtd">

<jasperReport name="DataSourceReport" pageWidth="595" pageHeight="842" columnWidth="515" leftMargin="40" rightMargin="40" topMargin="50" bottomMargin="50">
  <style name="Arial_Normal" isDefault="true" fontName="Arial" fontSize="12" isBold="false" isItalic="false" isUnderline="false" isStrikeThrough="false" pdfFontName="Helvetica" pdfEncoding="Cp1252" isPdfEmbedded="false"/>
  <style name="Arial_Bold" isDefault="false" fontName="Arial" fontSize="12" isBold="true" isItalic="false" isUnderline="false" isStrikeThrough="false" pdfFontName="Helvetica-Bold" pdfEncoding="Cp1252" isPdfEmbedded="false"/>
  <style name="Arial_Italic" isDefault="false" fontName="Arial" fontSize="12" isBold="false" isItalic="true" isUnderline="false" isStrikeThrough="false" pdfFontName="Helvetica-Oblique" pdfEncoding="Cp1252" isPdfEmbedded="false"/>
  <parameter name="ReportTitle" class="java.lang.String"/>
  <parameter name="DataFile" class="java.lang.String"/>
  <field name="id" class="java.lang.Integer"/>
  <field name="name" class="java.lang.String"/>
  <field name="street" class="java.lang.String"/>
  <field name="the_city" class="java.lang.String">
    <fieldDescription>me.me.city</fieldDescription>
  </field>
  <variable name="CityNumber" class="java.lang.Integer" incrementType="Group" incrementGroup="CityGroup" calculation="Count">
    <variableExpression><![CDATA[Boolean.TRUE]]></variableExpression>
  </variable>
  <group name="CityGroup" minHeightToStartNewPage="60">
    <groupExpression><![CDATA[$F{the_city}]]></groupExpression>
    <groupHeader>
    <band height="20">
      <textField evaluationTime="Group" evaluationGroup="CityGroup" bookmarkLevel="1">
        <reportElement mode="Opaque" x="0" y="5" width="515" height="15" backcolor="#c0c0c0" style="Arial_Bold"/>
        <box leftPadding="10" bottomBorder="1Point"/>
        <textFieldExpression class="java.lang.String"><![CDATA["  " + String.valueOf($V{CityNumber}) + ". " + String.valueOf($F{the_city})]]></textFieldExpression>
        <anchorNameExpression><![CDATA[String.valueOf($F{the_city})]]></anchorNameExpression>
      </textField>
    </band>
    </groupHeader>
    <groupFooter>
    <band height="20">
      <staticText>
        <reportElement x="400" y="1" width="60" height="15" style="Arial_Bold"/>
        <textElement textAlignment="Right"/>
        <text><![CDATA[Count :]]></text>
      </staticText>
      <textField>
        <reportElement x="460" y="1" width="30" height="15" style="Arial_Bold"/>
        <textElement textAlignment="Right"/>
        <textFieldExpression class="java.lang.Integer"><![CDATA[$V{CityGroup_COUNT}]]></textFieldExpression>
      </textField>
    </band>
    </groupFooter>
  </group>
  <title>
    <band height="70">
      <line>
        <reportElement x="0" y="0" width="515" height="1"/>
        <graphicElement/>
      </line>
      <textField isBlankWhenNull="true" bookmarkLevel="1">
        <reportElement x="0" y="10" width="515" height="30" style="Arial_Normal"/>
        <textElement textAlignment="Center">
          <font size="22"/>
        </textElement>
        <textFieldExpression class="java.lang.String"><![CDATA[$P{ReportTitle}]]></textFieldExpression>
        <anchorNameExpression><![CDATA["Title"]]></anchorNameExpression>
      </textField>
      <textField isBlankWhenNull="true">
        <reportElement x="0" y="40" width="515" height="20" style="Arial_Normal"/>
        <textElement textAlignment="Center">
          <font size="14"/>
        </textElement>
        <textFieldExpression class="java.lang.String"><![CDATA[$P{DataFile}]]></textFieldExpression>
      </textField>
    </band>
  </title>
  <pageHeader>
    <band height="20">
      <rectangle>
        <reportElement x="0" y="5" width="515" height="15" forecolor="#333333" backcolor="#333333"/>
        <graphicElement/>
      </rectangle>
      <staticText>
        <reportElement mode="Opaque" x="0" y="5" width="55" height="15" forecolor="#ffffff" backcolor="#333333" style="Arial_Bold"/>
        <textElement textAlignment="Center"/>
        <text><![CDATA[ID]]></text>
      </staticText>
      <staticText>
        <reportElement mode="Opaque" x="55" y="5" width="205" height="15" forecolor="#ffffff" backcolor="#333333" style="Arial_Bold"/>
        <text><![CDATA[Name]]></text>
      </staticText>
      <staticText>
        <reportElement mode="Opaque" x="260" y="5" width="255" height="15" forecolor="#ffffff" backcolor="#333333" style="Arial_Bold"/>
        <text><![CDATA[Street]]></text>
      </staticText>
    </band>
  </pageHeader>
  <detail>
    <band height="15">
      <textField bookmarkLevel="2">
        <reportElement x="0" y="0" width="50" height="15"/>
        <box leftBorder="Thin" bottomBorder="Thin" leftPadding="10" rightPadding="10"/>
        <textElement textAlignment="Right"/>
        <textFieldExpression class="java.lang.Integer"><![CDATA[$F{id}]]></textFieldExpression>
        <anchorNameExpression><![CDATA[$F{name} + " (" + $F{id} + ")"]]></anchorNameExpression>
      </textField>
      <textField isStretchWithOverflow="true">
        <reportElement positionType="Float" x="50" y="0" width="200" height="15"/>
        <box leftBorder="Thin" bottomBorder="Thin" leftPadding="10" rightPadding="10"/>
        <textElement/>
        <textFieldExpression class="java.lang.String"><![CDATA[$F{name}]]></textFieldExpression>
      </textField>
      <textField isStretchWithOverflow="true">
        <reportElement positionType="Float" x="250" y="0" width="265" height="15"/>
        <box leftBorder="Thin" bottomBorder="Thin" rightBorder="Thin" leftPadding="10" rightPadding="10"/>
        <textElement/>
        <textFieldExpression class="java.lang.String"><![CDATA[$F{street}]]></textFieldExpression>
      </textField>
    </band>
  </detail>
  <pageFooter>
    <band height="40">
      <line>
        <reportElement x="0" y="10" width="515" height="1"/>
        <graphicElement/>
      </line>
      <textField>
        <reportElement x="200" y="20" width="80" height="15"/>
        <textElement textAlignment="Right"/>
        <textFieldExpression class="java.lang.String"><![CDATA["Page " + String.valueOf($V{PAGE_NUMBER}) + " of"]]></textFieldExpression>
      </textField>
      <textField evaluationTime="Report">
        <reportElement x="280" y="20" width="75" height="15"/>
        <textElement/>
        <textFieldExpression class="java.lang.String"><![CDATA[" " + String.valueOf($V{PAGE_NUMBER})]]></textFieldExpression>
      </textField>
    </band>
  </pageFooter>
  <lastPageFooter>
    <band height="60">
      <textField bookmarkLevel="1">
        <reportElement x="0" y="10" width="515" height="15"/>
        <textElement textAlignment="Center"/>
        <textFieldExpression class="java.lang.String"><![CDATA["There were " +
          String.valueOf($V{REPORT_COUNT}) +
          " address records on this report."]]></textFieldExpression>
        <anchorNameExpression><![CDATA["Summary"]]></anchorNameExpression>
      </textField>
      <line>
        <reportElement x="0" y="30" width="515" height="1"/>
        <graphicElement/>
      </line>
      <textField>
        <reportElement x="200" y="40" width="80" height="15"/>
        <textElement textAlignment="Right"/>
        <textFieldExpression class="java.lang.String"><![CDATA["Page " + String.valueOf($V{PAGE_NUMBER}) + " of"]]></textFieldExpression>
      </textField>
      <textField evaluationTime="Report">
        <reportElement x="280" y="40" width="75" height="15"/>
        <textElement/>
        <textFieldExpression class="java.lang.String"><![CDATA[" " + String.valueOf($V{PAGE_NUMBER})]]></textFieldExpression>
      </textField>
    </band>
  </lastPageFooter>
</jasperReport>

As we can see, there are fields defined like id, name, street, and the_city. We also have a group called CityGroup, so when the items from the data source are iterated through, their group is known via the_city. It’s worth looking at how the paging works. The key is evaluationTime, which is telling the engine to not evaluate a given element at iteration time, but rather when an event occurs. evaluationTime="Report" means that we need to evaluate the value when the Report event occurs. At that time $V{PAGE_NUMBER} already has the value equal to the number of pages. Let’s see the preview:

Again, we don’t need to worry about the missing image icons, since this is only a preview and this is the actual result:

Since we have a few interfaces that we need to respect, we can easily integrate Hibernate with our Jasper reports as data source, we just need to make sure we are using the field and data source interfaces they defined. Here we have a few examples, notably the following:

List cats = session.find("from eg.Cat");

Map parameters = new HashMap();
parameters.put("Title", "The Cat Report");

InputStream reportStream = this.class.getResourceAsStream("/the-cat-report.xml");
JasperDesign jasperDesign = JasperManager.loadXmlDesign(reportStream);
JasperReport jasperReport = JasperManager.compileReport(jasperDesign);

JRBeanCollectionDataSource ds = new JRBeanCollectionDataSource(cats);
JasperPrint jasperPrint = JasperManager.fillReport(jasperReport, parameters, ds);

JasperManager.printReportToPdfFile(jasperPrint, "the-cat-report.pdf");

We gather a List of Cat instances, then define some parameters, like Title, followed by the creation of an InputStream object which will be used to generate a JasperDesign object, which is then the input for the instantiation of JasperReport. Now, we define cats to be our data source and call fillReport, passing the jasperReport object we have just created, the parameters, which contain the title and the data source. Finally, we print the report to a PDF. Note that you can use a compiled .jasper file as input for getResourceAsStream.

TIBCO Jaspersoft Studio

This IDE is excellent for compiling the jrxml files into jasper files. I use it as a desktop application. There are three tabs:

Design
Source
Preview

In the Design tab one can interactively design the report, not worrying about source code, XML, and the like, which enables non-programmers to work on report creation as well. Handling the variables, parameters and fields needs some algorithmic understanding, but nonetheless, code is generated interactively in a point-and-click manner. The designer cannot do everything we need, but the Source tab comes to the rescue if needed. Sometimes I do not even need to use the Source tab.

There’s still one problem: testing is a high-cost operation in the project I am using Jasper for. We need to build the .jasper files, but that’s low-cost. The higher cost is that we need to build the application that actually generates the Jasper reports in order to test and then deploy it with JBoss. Then the actual test in the application occurs, so a test takes more than a minute. Luckily there is a Preview tab, where I can more or less see whether it’s a good idea to invest time into building and deploying, or if we need to do some tweaking first.

Long story short

For more information, see Jaspersoft’s website.

JasperReports is free and open source. However, you have to pay a fee to consult the help documentation. It’s easily a competitor in quality with other reporting tools, which are less reasonably priced. When I first had to work with Jasper reports, I didn’t know anything about it, but was able to complete the tasks at hand in a few hours, which shows that getting into the Jasper universe isn’t too hard. One may have difficulty understanding .jrxml files at first, but trust me, it’s worth it.

If you are already using Java and want to generate reports, Jasper is a good candidate. It can periodically generate reports for you and even send them attached to an email to you. I couldn’t write better closing words than a poster I found:

* Domo plan pricing was found at yurbi.com.

Creating a Messaging App Using Spring for Apache Kafka, Part 1

2020-04-08T00:00:00+00:00

Photo by Click and Learn Photography at Unsplash

This article is part of a series. The GitHub repository with code examples can be found here.

Spring is a popular Java application framework. Apache Kafka is a fault-tolerant, fast, and horizontally scalable distributed stream-message broker. Spring for Apache Kafka applies the overall concepts of Spring to Java applications based on Kafka.

Since Kafka can establish a fast and fault-tolerant stream data pipeline it can be used as an orchestrator. In this article I’ll explain how to create a spring-kafka project, add dependencies and use Kafka to create a messaging app.

Initialize Spring project

Spring projects can be built from scratch using Spring Initializr. I like to keep the default options. Most Spring projects use Maven. I set the group id as com.endpoint and the artifact as SpringKafkaMessaging which makes the base package name com.endpoint.SpringKafkaMessaging.

When we are done with the initial project setup we press the “GENERATE” button to download an empty Spring Boot project in a zip file. You can then use your favorite IDE to open and start developing your project. I prefer Eclipse for Java projects. Here’s what it looks like when I open the project up:

I won’t address detailed configuration or adding dependencies of Spring and Maven projects in this post. If you are not familiar with Spring and Maven, I recommend that you have a look at the Spring documentation first.

Design and architecture

Before adding the dependencies, including Kafka, we need to make a high-level design of this simple project and figure out how to proceed with development. Messaging apps seem simple at first glance but the architecture behind them can be quite complex.

There are different kinds of technology stacks you can choose from. Which base protocol we choose (XMPP, SIP, or WebSocket) depends on what our app’s aim is. Sometimes multiple protocols can be used and interconnected to provide more features; XMPP is mostly used for chatting, SIP is designed for VoIP and media transfer. We’ll use WebSocket to communicate with Kafka over TCP.

By understanding the architectural model of Kafka, you’ll get an understanding of how Kafka is going to maintain most of the backend processes.

Kafka, as I mentioned previously, is horizontally scalable, and Kafka clusters can grow to span several data sources. Message producers and message consumers (all client messaging apps are both producers and consumers) produce and consume messages through Kafka topics.

So, taking into account the principles for designing the architecture of such a client–server-based messaging app, here are the components and their communication directions:

Kafka Cluster
Spring Boot REST API, which will handle user authentication and login
Persistence (here I chose PostgreSQL)
Cache (Redis) for fast read-write cache operations
WebSocket for messaging app clients

spring-kafka dependencies

After creating a model and components let’s add our dependencies to the pom.xml file to finish creating our project. Below we add spring-boot-starter, spring-boot-starter-web, spring-kafka, spring-boot-starter-jdbc, and redis.clients:jedis for the corresponding REST, Kafka, Persistent (JDBC), and Redis components.

<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
  </dependency>

  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>

  <dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
  </dependency>

  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jdbc</artifactId>
  </dependency>

  <dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
  </dependency>

  <dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
  </dependency>

  <dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
  </dependency>

  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
    <exclusions>
      <exclusion>
        <groupId>org.junit.vintage</groupId>
        <artifactId>junit-vintage-engine</artifactId>
      </exclusion>
    </exclusions>
  </dependency>

  <dependency>
      <groupId>org.springframework.kafka</groupId>
      <artifactId>spring-kafka-test</artifactId>
      <scope>test</scope>
  </dependency>
</dependencies>

Continued in Part 2!

Serialization and Deserialization Issues in Spring REST

2020-03-17T00:00:00+00:00

Photo by Annie Spratt

Spring Boot projects primarily use the JSON library Jackson to serialize and deserialize objects. It is especially useful that Jackson automatically serializes objects returned from REST APIs and deserializes complex type parameters like @RequestBody.

In a Spring Boot project the automatically registered MappingJackson2HttpMessageConverter is usually enough and makes JSON conversions simple, but this may have some issues which need custom configuration. Let’s go over a few good practices for them.

Configuring a Custom Jackson ObjectMapper

In Spring REST projects a custom implementation of MappingJackson2HttpMessageConverter helps to create the custom ObjectMapper, as seen below. Whatever custom implementation you need to add to the custom ObjectMapper can be handled by this custom converter:

public class CustomHttpMessageConverter extends MappingJackson2HttpMessageConverter {

    private ObjectMapper initCustomObjectMapper() {
        ObjectMapper customObjectMapper = new ObjectMapper();
        return customObjectMapper;
    }

    // ...
}

Additionally, some MappingJackson2HttpMessageConverter methods, such as writeInternal, can be useful to override in certain cases. I’ll give a few examples in this article.

In Spring Boot you also need to register a custom MappingJackson2HttpMessageConverter like below:

@Bean
MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter() {
    return new CustomHttpMessageConverter();
}

Serialization

Pretty-printing

Pretty-printing in Jackson is disabled by default. By enabling SerializationFeature.INDENT_OUTPUT in the ObjectMapper configuration pretty-print output is enabled (as in the example below). Normally a custom ObjectMapper is not necessary for setting the pretty-print configuration. In some cases, however, like one case of mine in a recent customer project, this configuration might be necessary.

For example, passing a URL parameter can enable pretty-printing. In this case having a custom ObjectMapper with pretty-print enabled and keeping the default ObjectMapper of MappingJackson2HttpMessageConverter as is could be a better option.

public class CustomHttpMessageConverter extends MappingJackson2HttpMessageConverter {

    private ObjectMapper initiatePrettyObjectMapper() {
        ObjectMapper customObjectMapper = new ObjectMapper();
        customObjectMapper.configure(SerializationFeature.INDENT_OUTPUT, true);

        // additional indentation for arrays
        DefaultPrettyPrinter pp = new DefaultPrettyPrinter();
        pp.indentArraysWith(new DefaultIndenter());
        customObjectMapper.setDefaultPrettyPrinter(pp);

        return customObjectMapper;
    }

}

Conditionally Filtering the Fields

When serializing a response object you may need to include or ignore one or more fields depending on their values. Let’s assume a model class UserResponse like below.

Notice that we used @JsonIgnore which is completely discarding the annotated field from serialization. Conditional filtering is different and it can be done using SimpleBeanPropertyFilter objects set to the filter provider of the ObjectMapper objects. Also notice that @JsonFilter annotation is used for UserResponse which points to which filter will be used by ObjectMapper during the serialization.

@JsonFilter("userCodeFilter")
public class UserResponse {

    public Integer userId;
    public String username;
    public Integer code;

    @JsonIgnore
    public String status;

}

Here we add a filter called userCodeFilter—like the one we added to the custom ObjectMapper of CustomHttpMessageConverter—which will include the UserResponse class’s code field in the serialization if its value is greater than 0. You can add multiple filters to ObjectMapper for different models.

public class CustomHttpMessageConverter extends MappingJackson2HttpMessageConverter {

    private ObjectMapper initiatePrettyObjectMapper() {
        ObjectMapper customObjectMapper = new ObjectMapper();
        customObjectMapper.configure(SerializationFeature.INDENT_OUTPUT, true);

        // additional indentation for arrays
        DefaultPrettyPrinter pp = new DefaultPrettyPrinter();
        pp.indentArraysWith(new DefaultIndenter());
        customObjectMapper.setDefaultPrettyPrinter(pp);

        PropertyFilter userCodeFilter = new SimpleBeanPropertyFilter() {
            @Override
            public void serializeAsField(Object pojo, JsonGenerator jgen, SerializerProvider provider, PropertyWriter writer)
                    throws Exception {
                if (include(writer)) {
                    if (!writer.getName().equals("code")) {
                        writer.serializeAsField(pojo, jgen, provider);
                        return;
                    }
                    int intValue = ((UserResponse) pojo).code;
                    if (intValue > 0) {
                        writer.serializeAsField(pojo, jgen, provider);
                    }
                } else if (!jgen.canOmitFields()) {
                    writer.serializeAsOmittedField(pojo, jgen, provider);
                }
            }

            @Override
            protected boolean include(BeanPropertyWriter writer) {
                return true;
            }

            @Override
            protected boolean include(PropertyWriter writer) {
                return true;
            }
        };

        FilterProvider filters = new SimpleFilterProvider().addFilter("userCodeFilter", userCodeFilter);
        customObjectMapper.setFilterProvider(filters);

        return customObjectMapper;
    }

}

Deserialization

JSON String Parse Error Handling in Spring Boot

This one is a little tricky. Deserialization of a JSON @RequestParam object can cause parsing errors if the JSON object is not well-formed. The errors thrown in Jackson’s deserialization level just before it’s pushed to Spring Boot occur at that level, so Spring Boot doesn’t catch these errors.

Deserialization of Jackson maps JSON to POJOs and finally returns the expected Java class object. If the JSON is not well-formed, parsing cannot be done and MappingJackson2HttpMessageConverter internally throws a parsing error. Since this exception is not caught by Spring Boot and no object is returned, the REST controller would be unresponsive, having a badly-formed JSON payload.

Here we can override the internal read method of MappingJackson2HttpMessageConverter, hack the ReadJavaType with a customReadJavaType method, and make it return an internal error when the deserialization fails to parse the JSON input, rather than throwing an exception which is not seen or handled by Spring Boot.

@Override
public Object read(Type type, @Nullable Class<?> contextClass, HttpInputMessage inputMessage)
        throws IOException, HttpMessageNotReadableException {

    objectMapper.enable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);

    JavaType javaType = getJavaType(type, contextClass);
    return customReadJavaType(javaType, inputMessage);
}

private Object customReadJavaType(JavaType javaType, HttpInputMessage inputMessage) throws IOException {
    try {
        if (inputMessage instanceof MappingJacksonInputMessage) {
            Class<?> deserializationView = ((MappingJacksonInputMessage) inputMessage).getDeserializationView();
            if (deserializationView != null) {
                return this.objectMapper.readerWithView(deserializationView).forType(javaType).
                        readValue(inputMessage.getBody());
            }
        }
        return this.objectMapper.readValue(inputMessage.getBody(), javaType);
    }
    catch (InvalidDefinitionException ex) {
        //throw new HttpMessageConversionException("Type definition error: " + ex.getType(), ex);
        return "Type definition error";
    }
    catch (JsonProcessingException ex) {
        //throw new HttpMessageNotReadableException("JSON parse error: " + ex.getOriginalMessage(), ex, inputMessage);
        return "JSON parse error";
    }
}

This way you can return errors occurring at the deserialization level to Spring Boot, which expects a deserialized object but gets a String value which can be caught and translated into a ControllerAdvice handled exception. This also makes it easier to catch JSON parsing errors without using any third party JSON libraries like Gson.

Cooking with CAS

2020-03-10T00:00:00+00:00

Photo by Flickr user reidrac, licensed under CC BY-SA 2.0

One of our customers asked us to host a new suite of web-based applications for them and to protect them with a single sign-on (SSO) solution. Ok, easy enough; these applications were in fact designed with a particular SSO system in mind already, but our situation required a different one, and we eventually chose Apereo’s open source Central Authentication Server project, or CAS. I’d like to describe the conversion process we went through.

The ingredients

Our customer’s application suite included:

The principal Java application using JAAS authentication
Another Java application based on Spring Security
A pair of PHP applications
A few automated tasks that needed to authenticate.

The original SSO system sets a header on each request, identifying an authenticated user. This requires a gateway system to sanitize request headers to ensure malicious users cannot forge a header themselves. It also requires each application inspect request headers and respond appropriately.

CAS is a bit more complex: applications redirect unauthenticated requests to a CAS server, which authenticates the user through any of various configurable methods. The CAS server then redirects the user back to the original application with a parameter called a “Service Ticket”, a seemingly random number identifying an individual authentication request. The original application contacts the CAS server directly to validate the service ticket and to collect information to identify the user. It can then establish a session for that user, and proceed normally.

To CAS-enable an application, we incorporate one of the CAS client libraries, which exist for various languages. In fact we won’t use the Java client directly, but rather we’ll incorporate components that extend it. When evaluating CAS, I was a bit concerned by what appeared to be a surprisingly limited selection of actively supported client libraries, and of course your results may vary, but we found software to meet our own needs without too much difficulty.

Configuring Wildfly Authentication

The most important application in the suite depends on the JAAS-based security subsystem of the Wildfly application server it’s deployed to. Originally it used a custom LoginModule that inspected request headers to find the ID of the authenticated user. Our first task was to configure our proxy server to remove this header from every request. We planned to disable the old authentication system entirely, of course, but this change ensured that even if we mistakenly left it enabled somewhere, malicious users couldn’t exploit it for access.

This application uses a declarative security policy: the deployment descriptor identifies a set of user roles, and another set of “security constraints”. Each security constraint describes one or more URL patterns used in the application, and the set of user roles allowed to access URLs matching those patterns. Here are two examples:

<!-- clients -->
<security-constraint>
    <web-resource-collection>
        <web-resource-name>Secure Area</web-resource-name>
        <url-pattern>/views/clients/*</url-pattern>
        <url-pattern>/client/edit/*</url-pattern>
        <url-pattern>/client/edit_tree/*</url-pattern>
        <url-pattern>/client/view/*</url-pattern>
        <url-pattern>/client/view/id/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <role-name>client_role</role-name>
    </auth-constraint>
</security-constraint>

<!-- places -->
<security-constraint>
    <web-resource-collection>
        <web-resource-name>Secure Area</web-resource-name>
        <url-pattern>/views/places/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <role-name>manage_places</role-name>
    </auth-constraint>
</security-constraint>

We installed the cas-extension library in our Wildfly server to handle the CAS protocol. When an unauthenticated user attempts to access a URL matching a pattern in one of the application’s security constraints, the CAS extension automatically intercepts control and redirects the user to the CAS server. Assuming the user authenticates successfully, our application will receive another request with a service ticket parameter. The cas-extension intercepts this request as well, validates the service ticket, and creates an “identity assertion”, which it sends to the Wildfly security system. Wildfly’s role mapper queries a database to find the user’s roles, after which the authentication process is complete.

Configuration of the cas-extension begins with a CAS profile, a combination of the URL of the CAS server and the URL of the service the extension should protect.

<subsystem xmlns="urn:soulwing.org:cas:1.0">
    <cas-profile name="default" service-url="https://our.application.server/application" server-url="https://our.cas.server"/>
</subsystem>

This tells the extension where to send authentication requests, where to listen for requests returning from the CAS server, and where to validate service tickets. Next we need to validate that identity assertion, and figure out what roles belong to the user. This happens in a Wildfly security domain.

<security-domain name="MySecurityDomain">
    <authentication>
        <login-module name="IdentityAssertion" code="org.soulwing.cas.jaas.IdentityAssertionLoginModule" flag="required" module="org.soulwing.cas"/>
    </authentication>
    <mapping>
        <mapping-module code="DatabaseRoles" type="role">
            <module-option name="dsJndiName" value="java:/comp/env/jdbc/databaseConnection"/>
            <module-option name="rolesQuery" value="select role from user_roles where user_id = ?"/>
        </mapping-module>
    </mapping>
</security-domain>

The authentication portion of the security domain refers to a JAAS LoginModule shipped with the CAS extension, which simply verifies that the identity assertion comes from the CAS extension and not somewhere else. Then the mapping portion (documented here) looks up the given user in a database to find what roles it should be assigned.

The last piece of the puzzle is a CAS deployment descriptor for our application, which activates cas-extension for that application. In its simplest form, this is an empty file in the right place, but ours ended up a little more complex. It identifies both the CAS profile we want to use (unnecessary in this case, as there’s only one CAS profile on the system, but it helped keep things more clear in our minds), and instructs the extension to load some other libraries into our application.

<?xml version="1.0" encoding="UTF-8"?>
<cas xmlns="urn:soulwing.org:cas:1.0">
  <profile>default</profile>
  <add-api-dependencies/>
</cas>

This configuration proved sufficient to let users log in and use the application, but as is sometimes the case with single sign-on, we needed a little more work to let them log out properly. Each application in the suite sets a cookie in the user’s browser to identify its session. The CAS server likewise sets a cookie. When a user logs out of the application, that application’s session cookie is destroyed, but we also need to destroy the CAS server’s session cookie as well as the other applications’ cookies. Single log-out can be complicated, and I won’t go into the full setup here. One reason the CAS deployment descriptor loads cas-extension libraries was so we could use cas-extension to generate the proper logout URL, and redirect our users to that URL once the application has destroyed its own session.

Configuring Spring Authentication

Another Java-based application in our suite uses Spring Security. This document describes the bulk of the configuration, which seemed less straightforward than for cas-extension, but follows essentially the same mechanism. Here we configured a Spring UserDetailsService to execute the same database query we used above to find the user’s roles. I’m not fully conversant in the large stack of beans Spring uses to manage the process, and it took some time to get this configuration sorted out.

Enter PHP

Two of these applications use PHP, which meant yet another configuration. Apereo maintains a PHP client, which includes several helpful examples. I tracked down the part of the application that authenticates users, and replaced the existing code with calls to phpCAS:

phpCAS::client(CAS_VERSION_2_0, $phpCASHost, $phpCASPort, $phpCASContext);
phpCAS::forceAuthentication();
$_SESSION[EXPORT_SERVERNAME]['umdid'] = phpCAS::getUser();

The forceAuthentication call determines the current phase of the authentication process this request is in, whether it’s unauthenticated, fully authenticated, or requires service ticket validation, and responds appropriately. We then set a session variable to the ID of the authenticated user, which replicates what the original authentication code would have done.

One of these two applications requires authenticated access to a REST API exposed by the Wildfly application. CAS calls this “proxy” authentication, when one application requests access to another. Here, CAS issues not only its usual service ticket, but also a “proxy granting ticket”. When the application wants to use the API, it asks the CAS server for a service ticket, using the proxy granting ticket. The CAS server itself requires some new configuration in this case, but for the PHP code, the only difference in the login phase from the simpler, non-proxy case is that we call phpCAS::proxy instead of phpCAS::client to configure CAS. Later, when calling the service itself, we used more phpCAS services in place of the cURL library the original used.

$service = \phpCAS::getProxiedService(PHPCAS_PROXIED_SERVICE_HTTP_GET);
$service->setUrl($serviceURL);
$service->send();

The return of header authentication

Finally, we have a few automated tasks which use various APIs, and need to authenticate. We can’t redirect requests to a CAS server and expect a user to provide credentials, so we’ve taken our cue from the applications’ original form, and configured CAS to recognize a “trusted header”. We add this header to any requests issued by these automated jobs. Of course, we’ve also configured the proxy to disallow this header from any systems outside our internal network.

A few loose ends

Of course, there were other considerations in this project that I’ve not covered here. Configuring CAS itself wasn’t necessarily straightforward, and included a custom authentication module I hope to describe in a later article. Selecting among CAS server’s available deployment options and fitting the winner into our existing infrastructure in a way that makes it easy to manage and monitor was another task entirely. We needed to customize the server’s default user interface to prevent it from offering users an imaginary method to recover forgotten passwords. The series of redirected browser requests involved in the CAS protocol presents a notable performance impact under some circumstances. And it has taken me no small effort to learn to appreciate the CAS server’s sometimes distressingly circular documentation. But several months into the project, CAS seems to be working well enough for our purposes that other users of the same application suite have begun to express interest.

A primer on Java

2020-02-10T00:00:00+00:00

What is Java and why is it interesting?

Java is a descendant of C++ and it is a C-based language. C was therefore the original language and it is probably not an overstatement to say that C is the most popular programming language in history. Most programmers speak at least C or one of its descendants. Let’s take a quick look at the most popular on the list of C-based languages:

C
C++
C#
Fantom
Go
Java
JavaScript
Objective C
Perl
PHP
Swift

Java is a member of a large family of programming languages and as a result, if someone learns Java, then they will have an easier time learning one of its cousins. And at the same time, if someone already speaks a C-based language, then Java is not too difficult to learn. Also, if someone is already a programmer and does not speak a C-based language yet, then it is in his/her interest in most cases to learn a C-based language and thus to have an understanding of the most popular language family.

Let’s see the list of popular languages according to Stackify:

Programming Language	Ratings	Change
Java	16.028%	-0.85%
C	15.154%	+0.19%
Python	10.020%	+3.03%
C++	6.057%	-1.41%
C#	3.842%	+0.30%
Visual Basic .NET	3.695%	-1.07%
JavaScript	2.258%	-0.15%
PHP	2.075%	-0.85%
Objective-C	1.690%	+0.33%
SQL	1.625%	-0.69%
Ruby	1.316%	+0.13%
MATLAB	1.274%	-0.09%
Groovy	1.225%	+1.04%
Delphi/Object Pascal	1.194%	-0.18%
Assembly language	1.114%	-0.30%
Visual Basic	1.025%	+0.10%
Go	0.973%	-0.02%
Swift	0.890%	-0.49%
Perl	0.860%	-0.31%
R	0.822%	-0.14%

Yes, Java has lost almost a percent from its popularity according to the list above, but even though the trend is suboptimal for Java at the time of this writing, it’s still ahead of the second-placed C in a convincing manner, since between the two there is almost a whole percent and has an advantage of more than 6% over Python, which is not a C-based language, but nevertheless it’s popular and rapidly gaining popularity, but it’s still very far from Java. Any other language is way below the popularity of Java, not even close.

What this means:

If you get a programming job and you are agnostic to languages, then Java is a language you may well work in.
Due to the popularity of the language for most problems you will likely find a well written and well tested, reliable library.
If you search for a programmer, he/she will more likely speak Java than other languages.

According to Towards Data Science, based on the data acquired from TIOBE, Java has been the most popular language since 2002 with two “pauses”, when C briefly took over, but even then, Java was the second:

As a result, taking into account the current situation, it is an imperative to speak a C-based language at least and due to Java’s popularity, it is advisable for programmers to be prepared to work in Java, by acquiring at least some practical knowledge about the language. Web programmers will naturally speak at least one C-based language if they work on the front-end, since in browsers JavaScript is standard.

Since there are many servers using Java and on smartphones Android is Java-based, Java’s future looks bright, so, if someone knows how to work with Java, then this knowledge will not lose its relevance in the foreseeable future.

“A long time ago in a galaxy far, far away…”

(Or, in other words, when I was still a university student, when a deadline was closely linked to homework and exam sessions.)

If you ask me what my favorite language is, I will tell you that I’m agnostic, because programming languages are just means of transforming thoughts into software. The main feature of languages – even outside of the realm of programming – is their ability of transmitting thoughts between actors in a conversation. Yes, Java’s syntax is very neat and there are other languages having a neat syntax as well. So, at this point I have no favorite programming language, but back in the day, Java offered some mouth-watering novelties for me, that filled me with enthusiasm. In the first two hours of my encounter with Java I learned that there is:

no need to worry about pointers
or function pointers
or allocating and freeing memory (due to the Garbage Collector)
or different platforms (due to the Java Virtual Machine)

Java enforces object-oriented coding, which helps us a lot, because that way, if we work in Java, then we do not need to worry about encountering procedural code written by others. Also, it forces us to think about the concepts we are about to use in our project. All these were great things to see.

If one just starts to code with Java, or has to work with classes or methods he/she is not very experienced with, then the brilliantly written documentation can help them.

This method is the add method of ArrayList. We know that ArrayList is logically a collection (technically it’s inherited from AbstractList, which is inherited from AbstractCollection), so, adding an item to it is the capability of putting an item at the end of the set of items present in the collection. We can also see that it is public and it will return a boolean, which represents success (or lack of it). Some parameter of the type E is passed to it. Since the ArrayList is denoted as ArrayList<E> at the start of the article, it’s clear that E is the type of the elements. So, taking a single glance on the first line inside the box one already understands the core information.

Of course, I had some prerequisite knowledge in C and C++ before sitting down for two hours to learn Java, but even then, being able to learn how to work with Java and being very comfortable with doing so in just two hours is a great achievement, but it’s not my achievement. It’s Java’s wonderful syntax, documentation and a great friend with the patience to teach, who achieved this for me. I only had to be curious.

Java consulting

My first work was in C++, but after realizing that I do not intend to go to an office each morning in the long term, I had the bold idea of finding remote work. So I started to work as a freelancer, doing small and poorly paid work remotely, often investing more energy into winning projects via negotiation than actually doing the work, but I was confident that my efforts will pay off long term.

In this period I learned PHP, JavaScript, CSS, HTML, MySQL (this was easy, because I was comfortable with Oracle), and realized that Java is not an objective best, at least not for me. It is just a language that I subjectively like and have fond memories with, but objectively I can perform well in other technologies as well.

After a lot of struggle, doing a lot of negotiation and freelancing work for very different people, each having his/her own preferences, cultural background, and temperament, I ended up working on .NET projects. Even though I enjoyed working with .NET as well, which was mostly in ASP.NET, I had difficulty working with Visual Basic, since my only coding with BASIC was some work I did with QBasic, when I was 11 years old. Even though I enjoyed working with QBasic as an early teen, writing code to display some graphic in motion, when I encountered Visual Basic with the purpose of developing the backend of ASP.NET projects, I had to relearn BASIC and having experience with C-based languages, I found it strange to see code like:

If foo IsNot Nothing AndAlso foo.IsValid Then
  'Some code
ElseIf foo IsNot Nothing Then
  'Some code
Else
  'Some code
End If

instead of

if (foo != null && foo.isValid) {
  //Some code
} else if (foo != null) {
  //Some code
} else {
  //Some code
}

I could work with either of the syntaxes, but the C-based syntax required much less characters to be typed on my part and the use of whitespace as operation separators is a source of worry for me (what happens if for any reason tabs are switched with spaces and vice-versa in Python, for example), which is a small problem for a small code but a bigger a problem if the code is bigger. But even though the situation was not very dramatic for me, I enjoyed a parody about Java and .NET immensely.

Over the years every now and then I received Java projects, mostly console or desktop applications, but occasionally I had some web work with Java as well. Since I began working with End Point I have been working on several Java projects running under Wildfly, a very nice application server, developed by Red Hat.

In general, when some work is completed, I build it with an ant command and deploy it using jboss.

In Linux I run jboss-cli.sh, for Windows there is a jboss-cli.bat alternative. After running it, we need to:

connect

and then deploy, like:

deploy <path to the ear file>

You can use the --force parameter to replace an older version of the deployment with the newer one.

Of course, this will only work if Wildfly is running. In order to make sure it is running, one needs to execute:

./standalone.sh -server-config=standalone-full.xml

Of course, one may use a different server config. It also has a Windows alternative, a file named standalone.bat, having the same purpose.

Java ORM

I used several different database object-relational mappers (ORMs) in the past, like LINQ, Flourishlib, Doctrine, to name a few examples. In Java I have been using Hibernate in recent times and am very satisfied with it in general. I can nicely work with object entities to represent table rows using Hibernate.

The entities in Hibernate are defined as POJO (plain old Java object) classes, where data members are private and they are accessible via public setters and getters. The classes and tables can be mapped via a mapping configuration XML. There are nice examples available online, for instance this one. One has the option to use Hibernate annotations as well, like here.

Besides methods that can be used with the help of session objects, one can write custom Hibernate queries as well, like:

String hql = "UPDATE Employee set salary = :salary "  +
             "WHERE id = :employee_id";
Query query = session.createQuery(hql);
query.setParameter("salary", 1000);
query.setParameter("employee_id", 10);
int result = query.executeUpdate();
System.out.println("Rows affected: " + result);

That example is from Tutorials Point.

Is Java slow, or is that a myth?

Java is a language. It’s not slow or fast. Performance depends on what is running and how. Natively compiled languages like C or C++ are in general faster, but the difference is not in the language itself, but the way the code is executed. Java runs through the JVM (Java Virtual Machine).

Taken from net-informations.com

That’s slower than running the compiler separately for each target platform and avoiding the usage of a virtual machine at runtime. The JVM helps us to avoid worrying about different platforms, and performance issues due to the language itself are an unlikely cause of performance issues in general. However, if one has a Java project with performance problems and the culprit is JVM, then one can compile Java bytecode into machine code. Alternatively, Java code can be converted automatically into C or C++.

If one is worried about using Java which then might become C or C++ code, then it’s worth considering the fact that C or C++ code on their own are already abstractions in comparison to machine code. Python and Ruby environments are slower to run than Java these days in my experience, so Java’s slowness is a myth nowadays and the truth in it is rooted in the past, when JVM was slow to start. However, at that point it was not Java itself that was slow, but the virtual machine that was running it for the platform.

Naturally, if we do not convert our code into machine code, but we run it through a virtual machine, that, upon each run will have to detect what the platform is and to interpret the JAR file, that convenience at the time of development comes at a price at runtime. But this problem is only theoretically frustrating these days, since with the ever more performant computers, if we find our Java program to be slow, then we should review the actual code before blaming the JVM for our performance problems.

Consider the example when someone is speaking in English and a translator has to translate the speech into French in real-time. What would we say if the interpreter is arriving late by a minute? Would we consider English to be a slow language, because of our initial wait for the translator? Naturally not. The situation with JVM slowness is similar. If the project is very large, that is, there is a lot of work to do for the JVM, then we might still experience slowness these days as well at program start, but if that becomes a serious problem, then we of course can solve that. Note that slow, underperforming or buggy code can be written in any language.

In short, Java, as a language is not slow, especially if it was translated into machine code. If it’s running via the JVM, then there is a performance price we pay in order to have no worries about platforms, but that’s rarely felt nowadays because of today’s general computing power.

Primitive and reference types

In Java there are primitive types and reference types. Reference types are Object and anything inherited from it. Primitive types, like float, int, or double are not objects and hence they cannot be used in a generic manner. To overcome this issue, Java offers wrapper classes for primitives, which are like Float, Integer or Double.

One cannot create a List of int, but a List of Integer can be created without problems. Integer is a reference type, hence we can use it as a specification for generics. I consider this to be a drawback of the language. I think that conceptually one might say that primitives and references are all types. So, if we could define some ultimately generic type as something special that represents the ultimate generic, even more abstract than Object, with the convention that any primitive or Object is of this generic type, then we should be able to create lists or collections of int or double.

I understand that this is not possible in the language, but I believe that this abstraction should be done under the hood. For example ArrayList<int> should be either inherently supported, or interpreted as ArrayList<Integer>. Of course, in order to establish this, some rules need to be implemented and tested thoroughly, but as a result we would be able to get rid of the unnecessary rule that primitive types cannot be referred to by generics.

Summary

Java is the most popular programming language and it is a C-based language.
C-based languages are the most popular programming language family and it’s the interest of every programmer to have an understanding of C-like syntaxes.
Due to its popularity it’s not difficult to find programmers who speak this language.
The language enforces development to comply to object-orientation.
Primitive types cannot be used as generic, one needs to use wrapper classes for this purpose.
Java is a language and as such it cannot be fast or slow; its environment can be fast or slow though.
JVM was slow to start in the past, but today this is less of a problem.
Java has a bright future due to the many popular projects in Java and Android.

Reviewing the Code

2019-11-26T00:00:00+00:00

Code review is analysis of source code of a software project by reading the code itself or sometimes using automated tools to analyze it, and it is part of the Software Quality Assurance (SQA) activities which define the quality assurance of the whole software development lifecycle.

As we go through the flow of a code-review, we’ll understand what to assure by doing code review. Code review lifecycle covers the guidelines and the code review process itself.

Code review guidelines

Preparing for code review

Pick the right reviewers. The reviewer should be an experienced software architect. Sometimes, less experienced groups can join the review team. Communication is also important, since frequent round table code review meetings with the developers are not usually productive.
Let authors annotate the code before review indicating which files have precedence for review. Annotation by the developers to orient the reviewer so they know where changes have taken place and where to review is a good practice.
Request code review. Requesting code review just before testing is another good practice.

Tips for reviewer during the code review

Review small pieces of code at a time, for instance around 400 lines of code.
Review for around 60 minutes at a time.
Set goals and metrics.
Use SMART criteria:

Specific: target one area clearly and precisely.

Measurable: quantify progress toward success by using metrics.

Actionable: ready to start and possible to accomplish.

Relevant: connected to what’s being done, considered, and/or resourced.

Timely: occurring at a favorable or useful time and opportune.

Measure inspection rate and defect rate.
Use checklists.

Code review flow

Regardless of what programming language is used to develop, code review should cover the following common areas:

Maintainability
Reusability
Performance
Security
Bug analysis
Knowledge sharing

Other practices can also be reviewed, like extensibility or OOP (Object Oriented Programming) software design patterns. These depend on possible future plans and the nature of the software being developed.

An Example Checklist for Java projects

I’d like to share a checklist below that I used for a project recently. I didn’t add my comments on the checklist items. The comments should be added to each item and if there are important details in the code, they should be added in the detailed report indicating the code blocks with the comments.

Clean Code & General

Use solution-problem domain names
Classes should be small
Functions should be small
Do one thing
Don’t repeat yourself (avoid duplication)—reusability
Explain yourself in your code
Use exceptions rather than return codes, nulls
Configurable

Security

Minimize the accessibility of classes and members
Avoid excessive logs for unusual behavior
Release resources (streams, connections, etc.) in all cases
Validate inputs
Validate outputs
Make public static fields final
Authorize and authenticate

Performance

Avoid excessive synchronization
Keep synchronized sections small
String concatenation
Avoid creating unnecessary objects

Extensibility

Make it easy to add enhancements with minimum changes
Components should easily be replaceable

Scalability

Consider large user base and data
Can be deployed in distributed systems

A few possible checklist item comments:

For naming checklist items, the below code piece complies with the conventions as it uses the solution domains as packaging name.

package com.google.search.common;

On the other hand, the following line would violate the Java package naming conventions since its letter capitalizations and domain usage are inconsistent.

package COM.google_search.common;

Sometimes a very long line of code could be used which usually violates the readability standards.
Or sometimes a variable used in a class scope in a concurrently run singleton Java object violates thread safety and could be a security issue.

Static Code Analysis

Reviewing is viewing the software by reviewer and the reviewers’ own comments and best practice notes will be used as the final output.

There are also static code analysis tools like PMD for Java which generally checks the code for bugs and clean code. This static code analysis, usually attached to the final code review report of the reviewer, is most helpful for an overview of the code.

Sometimes software development IDEs also provide best practices while writing the code. I like the PyCharm IDE’s smart clean code suggestions for Python, for example.

Automated Code Review

Automated testing is subject to measurable reports and static code analysis. Static code analysis tools can be scheduled for code checking and reporting.

Testing the code as a preparation for code review is another process that can be automated and already part of the SDLC processes. Some overlooked issues, like releasing resources and concurrency issues, can be also checked by automated tests and can be reported as automated code review results.

Conclusion

I have outlined the general code review cycle of a project.

Depending on the programming language and the architecture of the software to be developed there can be additional best practices like functional, non-functional, OOP design principles, etc.

Reviewers may extend or lessen their checklist items according to future plans, architecture or review request coverage.

Hot-deploy Java classes and assets in Wildfly 8/9/10

2017-10-27T00:00:00+00:00

Introduction

Java development can be really frustrating when you need to re-build your project and restart a server every time you change something. I know about JRebel, but while it’s a good tool, it’s also pretty expensive. You can use the open-source version, but then you need to send project statistics to the JRebel server, which is not a viable option for your more serious projects.

Fortunately, there is an open-source project called HotSwapAgent and it does the same thing as JRebel, for free (thank you, guys!).

I will explain how to combine it with Widlfly in order to hot-deploy Java classes as well as how to hot-deploy other resources (JavaScript, CSS, images).

Wildfly configuration

Let’s assume that we use the standalone-full.xml configuration file.

We need to use exploded deployment instead of deploying WAR or EAR. You can do this in production as well to allow for application changes with zero downtime.

Start by configuring the metaspace size; we had to increase defaults for our application, but it’s possible that it will be just fine in your case. It’s encouraged that you play with these values after completing all steps.

In:

WILDFLY_DIR/bin/standalone.conf

set:

-XX:MetaspaceSize=256M -XX:MaxMetaspaceSize=512m

so it looks like this:

JAVA_OPTS="-Xms512m -Xmx1024m -XX:MetaspaceSize=256M -XX:MaxMetaspaceSize=512m".

Now, look for the deployment-scanner node in:

WILDFLY_DIR/standalone/configuration/standalone-full.xml

Replace it so it looks like this:

<deployment-scanner path="PATH_TO_DEPLOYMENT_DIR" relative-to="RELATIVE_TO_PATH" scan-enabled="true" scan-interval="2000" auto-deploy-exploded="false" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}"/>

Note:

PATH_TO_DEPLOYMENT_DIR is WILDFLY_DIR/standalone/deployments

RELATIVE_TO_PATH is, as the name suggests the dir that the PATH_TO_DEPLOYMENT_DIR is relative to.

HotSwapAgent installation and configuration

We need to download and install the latest release of DCEVM Java patch from here: https://github.com/dcevm/dcevm/releases. Why it’s needed? It will allow us unlimited redefinition of loaded classes at runtime. This is not possible with the original Java HotSpot VM. Make sure you update to the same Java version that you’re going to use to run the Wildfly server.

Now, download the latest release of the Hotswap agent from here:

https://github.com/HotswapProjects/HotswapAgent/releases

The only thing that you need to do is get the JAR and put it anywhere on your hard drive (I recommend to add it to your Java project).

Ok, great, now just some configuration.

Open:

WILDFLY_DIR/bin/standalone.conf

and add new Java opts:

-XXaltjvm=dcevm -javaagent:PATH_TO_HOTSWAPAGENT_JAR.

What does this do?

The altjvm option sets an alternative Java Virtual Machine.
The javagent is just an interceptor on the top of your classes that allows the HotSwapAgent library to manipulate your code on the fly.

That’s all you need. It’s a good idea to create a configuration file for the HotSwapAgent. This is well explained here:

http://hotswapagent.org/mydoc_configuration.html

Basically create a new file, name it hotswap-agent.properties, set all needed configuration inside and add it to the classpath of the application.

If you use Netbeans, Eclipse or Intellij you should check the HotSwapAgent page for some helpful plugins here: http://hotswapagent.org/mydoc_setup_intellij_idea.html, http://hotswapagent.org/mydoc_setup_eclipse.html, and http://hotswapagent.org/mydoc_setup_netbeans.html.

Application configuration

Now that we have everything in place, I will explain how to put it all together. I doesn’t really matter which build-tool you use (Ant, Gradle or Maven). The process should look like this (you can do it in many ways, in our case, it’s pretty specific as our build process is really complicated):

Build your application and deploy it to the PATH_TO_DEPLOYMENT_DIR in the exploded version,
Create a script that will look for changes in the application directory (this one is interesting: https://gist.github.com/peter-hank/3ecf7fc285ba4b9c50cf8cace1badaf4),
On change, trigger a job that will:
1. Copy all resources like JSP, JavaScript, CSS and copy to the PATH_TO_DEPLOYMENT_DIR,
2. Compile classes and copy them to the PATH_TO_DEPLOYMENT_DIR.

That’s it, after you replace files in the PATH_TO_DEPLOYMENT_DIR HotSwapAgent and Wildfly will do the rest really fast. We have a ton of assets and classes and the whole process takes only a few seconds!

Summary

I feel this process is really worth doing. It doesn’t take a lot of time to configure everything and saves a lot of manual work. Just multiply the number of manual deployments and the number of developers in your team and you understand how much time you lose everyday without hot-deployment.

From now on, focus on development, forget about deployment!

Lastly, good luck!

PKIX path validation failed — Debugging

2017-10-04T00:00:00+00:00

I recently ran into a case working on an application with a PKIX path validation error on a site that had a valid certificate. I was able to solve the issue using OpenSSL to debug.

Typically, the PKIX path validation error arises due to SSL certificate expiry, but I ran into the same error even when the system was configured with a valid certificate. There are two web applications in our scenario, AppX and AppY. AppX uses AppY’s authentication mechanism to allow the users to login with same user account. AppX sends a POST request using HttpClient with necessary arguments to SSL enabled AppY and allows the user to login based on the response.

HttpClient httpclient = new DefaultHttpClient();
// ...
HttpPost httppost = new HttpPost("https://app2domain.com/sessions");

try {
    resp = httpclient.execute(httppost);
}
catch (Exception e) {
    throw new Exception("Exception: ", e);
}

Error

The AppX was isolated to new server and it started throwing PKIX path validation failed error while sending requests to AppY.

Exception: javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path validation failed:
java.security.cert.CertPathValidatorException: timestamp check failed

PKIX (Public-Key Infrastructure — X.509) is standard for key based encryption mechanism. The PKIX path related errors come up due to the failure establishing the connection with SSL applications.

Debug

It is good to identify the root cause of the problem since there are few possible reasons for the same error. Let’s start debugging…

Check the Certificate status and expiration date in your browser The browser reports that the certificate is valid and will expire at a future date for AppY’s domain name. So now on to the detailed debugging using OpenSSL.
OpenSSL validation The openssl tool is a handy utility to validate the SSL certificate for any domain. It reports error with return code 20 (unable to get local issuer certificate) when checking the status of certificate — which is in contrast with browser’s report on the same certificate.

$ echo -n | openssl s_client -CApath /etc/ssl/certs/ -connect app2domain.com:443
...
Start Time: 1482921042
Timeout   : 300 (sec)
Verify return code: 20 (unable to get local issuer certificate)

$ echo -n | openssl s_client -CApath /etc/ssl/certs/ -connect app2domain.com:443 </dev/null | openssl x509 -noout -dates

verify error:num=20:unable to get local issuer certificate
DONE
notBefore=May  4 00:00:00 2013 GMT
notAfter=May 14 23:59:59 2015 GMT

Root Cause

The openssl tool reported certificate details another unused and expired domain. Since this is configured on the same server, it is causing the error in our case. The same scenario happened to the AppX when sending request to AppX. It may have tried to establish connection through the expired certificate. So, the lesson here is that it is necessary to clean up the expired certificates when the connection is established through HttpClient utilities. Also, a specific domain name can be validated by passing the -servername option (for SNI) to the openssl, which in this case reports appYdomain.com has valid certificate.

$ echo -n | openssl s_client -CApath /etc/ssl/certs/ -connect app2domain.com:443 -servername app2domain.com
...
Start Time: 1482920942
Timeout   : 300 (sec)
Verify return code: 0 (ok)

$ echo -n | openssl s_client -CApath /etc/ssl/certs/ -connect app2domain.com:443 -servername app2domain.com </dev/null | openssl x509 -noout -dates
...
verify return:0
DONE
notBefore=Sep 26 11:52:51 2015 GMT
notAfter=Apr  1 12:35:52 2018 GMT

Conclusion

In most cases, the PKIX path validation error comes up when the SSL certificate is expired for the domain name, however, there may be different reasons such as certificate expiry, picking wrong certificate, etc. It is always helpful to debug with the openssl tool to identify the root cause. This specific issue was fixed by removing the unused expired certificate.

JBoss 4/5/6 to Wildfly migration tips

2017-07-31T00:00:00+00:00

Introduction

Recently, we have taken over a big Java project that ran on the old JBoss 4 stack. As we know how dangerous for a business is outdated software, we and our client agreed that the most important task is to upgrade the server stack to the latest WildFly version.

It’s definitely not an easy job, but it’s worth to invest to sleep well and don’t worry about software problems.

This time it was even more work because of a complicated and not documented application, that’s why I wanted to share some tips and problem resolutions for issues I encountered.

Server configuration

You can set it up using multiple configuration files in the standalone/configuration directory.

I can recommend to use the standalone-full.xml file for most of setup, it contains a default full stack as opposed to standalone.xml.

You can also set up an application specific configuration using various configuration XML files (https://docs.jboss.org/author/display/WFLY10/Deployment+Descriptors+used+In+WildFly). Remember to keep the application specific configuration in the Classpath.

Quartz as a message queue

The Quartz library was used as a native message queue in previous JBoss versions. If you struggle and try to use its resource adapter with WildFly just skip it. It’s definitely too much work, even if it’s possible.

In the latest WildFly version (as of today 10.1) the default message queue library is ActiveMQ. It has almost the same API as the old Quartz has, so it’s easy to use it.

Quartz as a job scheduler

We had multiple cron-like jobs to migrate as well. All the jobs used Quartz to schedule runs.

The best solution here is to update Quartz to the latest version (yes!) and use a new API (http://www.quartz-scheduler.org/documentation/quartz-2.2.x/tutorials/tutorial-lesson-06.html) to create CronTriggers for the jobs.

trigger = newTrigger()
    .withIdentity("trigger3", "group1")
    .withSchedule(cronSchedule("0 42 10 * * ?"))
    .forJob(myJobKey)
    .build();

You can use the same cron syntax (e.g 0 42 10 * * ?) as in the 12 years old Quartz version. Yes!

JAR dependencies

In WildFly you can set up an internal module for each JAR dependency. It can be pretty time consuming to create declarations for more than 100 libraries (exactly 104 in our case). We decided to use Maven to handle dependencies of our application and skip declaring modules in WildFly. Why? In our opinion it’s better to encapsulate everything in an EAR file and keep WildFly configuration minimal as we won’t use our server for any other application in the future.

Just keep your dependencies in the Classpath and you will be fine.

JBoss CLI

I really prefer the bin/jboss-cli.sh interface to the web interface to handle deployments. It’s a powerful tool and much faster to work with than clicking through the UI.

JNDI path

If you can’t access your JNDI definition try to use the global namespace. Up to JAVA EE6 developers defined their own JNDI names. These names had a global scope. This doesn’t work anymore. To access a previously globally scoped name use this pattern: java:global/OLD_JNDI_NAME.

The java:global namespace was introduced in JAVA EE 6.

Reverse proxy

To configure a WildFly application with a reverse proxy you need to, of course, set up a virtual host with a reverse proxy declaration.

In addition, you must add an attribute to the server’s http-listener in the standalone-full.xml file. The attribute is proxy-address-forwarding and must be set to true.

Here is an example declaration:

<subsystem xmlns="urn:jboss:domain:undertow:3.1">
    <buffer-cache name="default">
        <server name="default-server">
            <http-listener enable-http2="true" name="default" proxy-address-forwarding="true" redirect-socket="https" socket-binding="http">
 </http-listener></server>
    </buffer-cache>
</subsystem>

Summary

If you consider to upgrade to WildFly I can recommend it, it’s much faster than JBoss 4/5/6, scalable and fully prepared for modern applications.

infoShare 2017 - JavaScript, JavaScript everywhere

2017-05-27T00:00:00+00:00

The last week was really interesting for me. I attended the infoShare 2017, the biggest tech conference in central-eastern Europe. The agenda was impressive, but that’s not everything. There was a startup competition going on and really, I’m totally impressed.

infoShare in numbers:

5500 attendees
133 speakers
250 startups
122 hours of speeches
12 side events

Let’s go through each speech I was attending.

Day 1

Why Fast Matters by Harry Roberts from csswizardry.com

Harry tried to convince us that performance is important.

Great speech, showing that it’s an interesting problem not only from a financial point of view. You must see it, link to his presentation here

Dirty Little Tricks From The Dark Corners of Front-End by Vitaly Friedman from smashingmagazine.com

It was magic! I work a lot with CSS, but this speech showed me some new ideas and reminded me that the simplest solution is maybe not the best solution usually and that we should reuse CSS between components as much as possible.

Keep it DRY!

One of these tricks is a quantity query CSS selector. It’s a pretty complex selector that can apply your styles to elements based on the number of siblings. (http://quantityqueries.com/)

The Art of Debugging (browsers) by Remy Sharp

It was great to see some other developer and see his workflow during debugging. I usually work from home and it’s not easy to do it in my case.

Remy is a very experienced JavaScript developer and showed us his skills and tricks, especially interesting Chrome developer console integration.

I always thought that using the developer console for programming is not the best idea, maybe it’s not? It looked pretty neat.

Desktop Apps with JavaScript by Felix Rieseberg from Slack

Felix from Slack presented and show the power of desktop hybrid apps. He used a framework called Electron. Using Electron you can build native, cross-system desktop apps using HTML, JavaScript and CSS. I don’t think that it’s the best approach for more complex applications and probably takes more system memory than native-native applications, but for simpler apps it can a way to go!

Github uses it to build their desktop app, so maybe it’s not so slow? :)

RxJava in existing projects by Tomasz Nurkiewicz from Allegro

Tomasz Nurkiewicz from Allegro showed us his high programming skills and provided some practical RxJava examples. RxJava is a library for composing asynchronous and event-based programs using observable sequences for the Java VM.

Definitely something to read about.

Day 2

What does a production ready Kubernetes application look like? by Carter Morgan from Google

Carter Morgan from Google showed us practical uses of Kubernetes.

Kubernetes is an open-source system for automating deployment, scaling and management of containerized applications. It was originally designed by Google developers and I think that they really want to popularize it. It looked that Kubernetes has a low learning curve, but devops agents I spoke after the presentation were sceptical, saying that if you know how to use Docker Swarm then you don’t really need Kubernetes.

Vue.js and Service Workers become Realtime by Blake Newman from Sainsbury’s

Blake Newman is a JavaScript developer, member of the core Vue.js (trending, hot JavaScript framework) team. He explained how to use Vue.js with service workers.

The service workers are scripts that your browser runs in the background. Nice to see how it fits together, even though it’s not yet supported by every popular browser.

Listen to your application and sleep by Gianluca Arbezzano from InfluxData

Gianluca showed us his modern and flexible monitoring stack. Great tips and mostly discussing and recommending InfluxDB and Telegraf, we use it a lot in End Point.

He was right that it’s easy to configure, open-source and really useful. Great speech!

Summary

Amazing two days. All the presentations will be available on Youtube soon: https://www.youtube.com/user/infoSharePL/videos.

I can fully recommend this conference, see you next time!

Connect Multiple JPA repositories using Static and Dynamic Methods

2016-11-16T00:00:00+00:00

The JPA Repository is a useful Spring Framework library that provides object-relational mapping for Java web applications to be able to connect to a wide variety of databases. Most applications need to establish a connection with one database to store and retrieve the data though sometimes there could be more than one database to read and write. There could also be some cases where the application needs to choose which database should be used dynamically, based on each request’s parameters. Let’s see how to configure and establish connections for these three cases.

1. Single Static Connection

In order to use JPA the following configurations are required to get the database connection handle and define the interface to map a database table by extending JpaRepository class.

UserRepository.java — this part of the code configures how to map the user table

package com.domain.data;

import com.domain.User;
import org.springframework.data.jpa.repository.JpaRepository;

public interface UserRepository extends JpaRepository <User, Integer> {
}

persistent-context.xml — the dataSourceReadWrite bean class defines the database connection while the entityManagerFactoryReadWrite bean helps to access the database from the base package com.domain

...
<jpa:repositories base-package="com.domain" entity-manager-factory-ref="entityManagerFactoryReadWrite" transaction-manager-ref="transactionManager">
</jpa:repositories>

<bean abstract="true" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" id="abstractDataSource" p:driverclassname="${jdbc.driverClassName}" p:maxactive="20" p:maxidle="20" p:minidle="20" p:testonborrow="true" p:validationquery="SELECT 1" />

<bean id="dataSourceReadWrite" p:password="${jdbc.password}" p:url="${jdbc.url}" p:username="${jdbc.username}" parent="abstractDataSource" />

<bean abstract="true" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" id="abstractEntityManagerFactory" p:jpadialect-ref="hibernateJpaDialect" p:jpavendoradapter-ref="jpaAdapter">
  <property name="jpaProperties">
    <props>
      <prop key="hibernate.use_sql_comments">true</prop>
      <prop key="hibernate.temp.use_jdbc_metadata_defaults">false</prop>
    </props>
  </property>
</bean>

<bean id="entityManagerFactoryReadWrite" p:datasource-ref="dataSourceReadWrite" p:persistenceunitname="readWritePU" parent="abstractEntityManagerFactory">
  <property name="packagesToScan">
    <list>
      <value>com.domain</value>
    </list>
  </property>
</bean>

<bean class="org.springframework.orm.jpa.JpaTransactionManager" id="transactionManager" p:datasource-ref="dataSourceReadWrite" p:entitymanagerfactory-ref="entityManagerFactoryReadWrite" />
...

UserController.java — the userRepository object access defines how to use a static database configuration to fetch the User object record

@Api(name = "User", description = "User API Service")
@Controller
public class UserController {

  @Autowired
  private UserRepository userRepository;

  @ApiMethod(
    description = "Return the user object using the userId in the URL",
    produces = {MediaType.APPLICATION_JSON_VALUE},
    roles = {"read", "write"}
  )
  @RequestMapping(value = "/users/{userId}", method = RequestMethod.GET, produces = "application/json")
  @ResponseBody
  public UserModel getUser(@PathVariable @ApiParam(name = "userId", description = "User ID") Integer userId) throws ServiceException {
    User user = (userRepository.findOne(userId));
    if (user != null) {
    return new UserModel(user);
  }
  else {
    throw new ResourceNotFoundServiceException("UserId " + userId + " was not found");
  }
}
}

2. Multiple Static Connections

In some cases, we may need to connect more than one database in our application. Usually there will be a primary database and a secondary one which syncs data from the primary, most likely as a readonly replica load balancing approach. In this case the application needs to be configure to establish connection with two different datasources.

To achieve this result it’s possible to define ReadWrite and ReadOnly datasources in the spring configuration and then declare the specific Repository classes for each specific datasource.

UserRepository.java — ReadWrite repository definition under the package com.domain.data

package com.domain.data;

import com.domain.User;
import org.springframework.data.jpa.repository.JpaRepository;

public interface UserRepository extends JpaRepository<User, Integer> {
}

UserReadonlyRepository.java — ReadOnly repository definition under the package com.domain.data.readonly

package com.domain.data.readonly;

import com.domain.User;
import org.springframework.data.jpa.repository.JpaRepository;

public interface UserReadonlyRepository extends JpaRepository<User, Integer> {
}

persistent-context.xml — this file defines two different datasources (dataSourceReadWrite and dataSourceReadOnly) while jpa repositories specify the repositories package path

...
<jpa:repositories base-package="com.domain" entity-manager-factory-ref="entityManagerFactoryReadWrite" transaction-manager-ref="transactionManager">
  <repository:exclude-filter expression="com.domain.data.readonly" type="regex"></repository:exclude-filter>
</jpa:repositories>

<jpa:repositories base-package="com.domain.data.readonly" entity-manager-factory-ref="entityManagerFactoryReadOnly" transaction-manager-ref="transactionManagerReadOnly">

<bean abstract="true" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" id="abstractDataSource" p:driverclassname="${jdbc.driverClassName}" p:maxactive="20" p:maxidle="20" p:minidle="20" p:testonborrow="true" p:validationquery="SELECT 1" />
<bean id="dataSourceReadWrite" p:password="${jdbc.password}" p:url="${jdbc.url}" p:username="${jdbc.username}" parent="abstractDataSource" />
<bean id="dataSourceReadOnly" p:password="${jdbc.readonly.password}" p:url="${jdbc.readonly.url}" p:username="${jdbc.readonly.username}" parent="abstractDataSource" />
<bean abstract="true" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" id="abstractEntityManagerFactory" p:jpadialect-ref="hibernateJpaDialect" p:jpavendoradapter-ref="jpaAdapter">
  <property name="jpaProperties">
    <props>
      <prop key="hibernate.use_sql_comments">true</prop>
      <prop key="hibernate.temp.use_jdbc_metadata_defaults">false</prop>
    </props>
  </property>
</bean>

<bean id="entityManagerFactoryReadWrite" p:datasource-ref="dataSourceReadWrite" p:persistenceunitname="readWritePU" parent="abstractEntityManagerFactory">
  <property name="packagesToScan">
    <list>
      <value>com.domain</value>
    </list>
  </property>
</bean>

<bean id="entityManagerFactoryReadOnly" p:datasource-ref="dataSourceReadOnly" p:persistenceunitname="readOnlyPU" parent="abstractEntityManagerFactory">
  <property name="packagesToScan">
    <list>
      <value>com.domain</value>
    </list>
  </property>
</bean>

<bean class="org.springframework.orm.jpa.JpaTransactionManager" id="transactionManager" p:datasource-ref="dataSourceReadWrite" p:entitymanagerfactory-ref="entityManagerFactoryReadWrite" />

<bean class="org.springframework.orm.jpa.JpaTransactionManager" id="transactionManagerReadOnly" p:datasource-ref="dataSourceReadOnly" p:entitymanagerfactory-ref="entityManagerFactoryReadOnly" />
...

UserController.java — in this definition it’s interesting to note the the readonly flag, which will establish a connection with ReadWrite or ReadOnly database, based on that flag value

@Api(name = "User", description = "User API Service")
@Controller
public class UserController {

  @Autowired
  private UserRepository userRepository;
  @Autowired
  private UserReadOnlyRepository userReadOnlyRepository;

  @ApiMethod(
  description = "Return the user object using the userId in the URL",
  produces = {MediaType.APPLICATION_JSON_VALUE},
  roles = {"read", "write"}
  )
  @RequestMapping(value = "/users/{userId}", method = RequestMethod.GET, produces = "application/json")
  @ResponseBody
  public UserModel getUser(@PathVariable @ApiParam(name = "userId", description = "User ID") Integer userId, @ApiParam(name = "readOnly", description = "Param to set data source to read from") Boolean readOnly) throws ServiceException {
    User user = (readOnly ?
    userReadOnlyRepository.findOne(userId) : userRepository.findOne(userId));

    if (user != null) {
      return new UserModel(user);
    }
    else {
      throw new ResourceNotFoundServiceException("UserId " + userId + " was not found");
    }
  }
}

3. Multiple Dynamic Connections

Recently there was an application that needed to choose the database during API request processing. Unfortunately defining multiple datasources and choosing the database based on the hard coded checks in the code is really cumbersome. Instead it’s possible to use JPA Repository which provides a feature to override the database lookup dynamically using AbstractRoutingDataSource when a request is sent to the application.

UserRepository.java — defines mapping to the user table

package com.domain.data;

import com.domain.User;
import org.springframework.data.jpa.repository.JpaRepository;

public interface UserRepository extends JpaRepository<User, Integer> {
}

persistence-context.xml — dataSourceRootDB and dataSourceLiveDB beans defines two different databases. MultitenantRoutingDataSource holds the datasources available to chose dynamically from the code

...
<jpa:repositories base-package="com.domain" entity-manager-factory-ref="genericEntityManagerFactory" transaction-manager-ref="transactionManager">
</jpa:repositories>

<bean abstract="true" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" id="abstractDataSource" p:driverclassname="${jdbc.driverClassName}" p:maxactive="20" p:maxidle="20" p:minidle="20" p:testonborrow="true" p:validationquery="SELECT 1" />

<bean id="dataSourceRootDB" p:password="${jdbc.password}" p:url="${jdbc.url}" p:username="${jdbc.username}" parent="abstractDataSource" />

<bean id="dataSourceLiveDB" p:password="${jdbc.livedb.password}" p:url="${jdbc.livedb.url}" p:username="${jdbc.livedb.username}" parent="abstractDataSource" />

<bean class="com.domain.route.MultitenantRoutingDataSource" id="dataSource">
  <property name="targetDataSources">
    <map key-type="java.lang.String">
      <entry key="rootdb" value-ref="dataSourceRootDB"></entry>
      <entry key="livedb" value-ref="dataSourceLiveDB"></entry>
    </map>
  </property>
  <property name="defaultTargetDataSource" ref="dataSourceRootDB">
  </property>
</bean>

<bean class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean" id="genericEntityManagerFactory" p:datasource-ref="dataSource" p:jpadialect-ref="hibernateJpaDialect" p:jpavendoradapter-ref="jpaAdapter" p:persistenceunitname="readWriteDynamicPU">
  <property name="jpaProperties">
    <props>
      <prop key="hibernate.use_sql_comments">true</prop>
      <prop key="hibernate.temp.use_jdbc_metadata_defaults">false</prop>
    </props>
  </property>
  <property name="packagesToScan">
    <list>
      <value>com.data.domain</value>
    </list>
  </property>
</bean>

<bean class="org.springframework.orm.jpa.JpaTransactionManager" id="transactionManager" p:datasource-ref="dataSource" p:entitymanagerfactory-ref="genericEntityManagerFactory" />
...

UserController.java — this class choose the datasource dynamically based on the request and calls the selected service to complete the action

...
@Api(name = "User", description = "User API Service")
@Controller public class UserController {
    @Autowired     private UserService userService;

    @ApiMethod(
            description = "Return the user object using the userId in the URL",
            produces = {MediaType.APPLICATION_JSON_VALUE},
            roles = {"oms-read", "oms-write"}
    )
    @RequestMapping(value = "/users/{userId}", method = RequestMethod.GET, produces = "application/json")
    @ResponseBody
    public UserModel getUser(@PathVariable @ApiParam(name = "userId", description = "User ID") Integer userId, @RequestHeader(value="X-Database", defaultValue= DatabaseEndpointUtils.ROOT_DB, required=false) String databaseEndpoint) throws ServiceException {
        MultiTenantContext.setCurrentTenant(databaseEndpoint);
        return userService.getUser(userId, true);
    }
}
...

MultiTenantContext.java — this code sets the datasource connection based on the request from Controller

package com.domain.common;

import com.domain.util.DatabaseEndpointUtils;
import com.domain.supplychain.app.ws.exceptions.InvalidDatabaseEndpointException;
import com.domain.exceptions.ServiceException;

public class MultiTenantContext {
    private static ThreadLocal<Object> currentTenant = new ThreadLocal<>();

    public static Logger logger = LoggerFactory.getLogger(MultiTenantContext.class.getName());
    public static void setCurrentTenant(Object tenant) throws ServiceException {
        logger.info("MultiTenantContext setCurrentTenant: [{}]", tenant);
        if(DatabaseEndpointUtils.isValidEndpoint(tenant.toString())) {
            currentTenant.set(tenant);
        } else {
            throw new InvalidDatabaseEndpointException("Invalid database endpoint");
        }
    }

    public static Object getCurrentTenant() {
        logger.info("MultiTenantContext getCurrentTenant: [{}]", currentTenant.get());
        return currentTenant.get();
    }

}

MultitenantRoutingDataSource.java — here there’s the definition which determines how the datasource establish the connection. Specifically it will get the datasource which was set previously based on the request parameters

package com.domain.route;
import com.domain.common.MultiTenantContext;
import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;

public class MultitenantRoutingDataSource extends AbstractRoutingDataSource {

    private Logger logger = LoggerFactory.getLogger(MultitenantRoutingDataSource.class.getName());
    @Override
    protected Object determineCurrentLookupKey() {
        logger.info("MultitenantRoutingDataSource determineCurrentLookupKey: [{}]", MultiTenantContext.getCurrentTenant());
        return MultiTenantContext.getCurrentTenant();
    }

}

DefaultUserService.java — Fetch the user data from the dynamically chosen database.

@Service
public class DefaultUserService implements UserService {

    @Autowired
    private UserRepository userRepository;

    @Override
    @Transactional
    public UserModel getUser(Integer userId, boolean readOnly) throws ServiceException {
        User user = (userRepository.findOne(userId));
        if (user != null) {
            return new UserModel(user);
        }
        else {
            throw new ResourceNotFoundServiceException("UserId " + userId + " was not found");
        }
    }
}

Conclusion

The application establishes a connection with database through any one of these methods, based on the requirement. The single or multiple static connections are commonly used in most of the applications. But when there is a requirement to choose the database dynamically to establish a connection, AbstractRoutingDataSource class in spring framework features a wonderful way to implement the functionality as explained above.

Understanding the Relationship Between Apex and Salesforce

Apex and Salesforce

Apex vs. Java: Key Differences

Why It Matters

How to Migrate from Struts 2 to Struts 6

Getting Ready for Migration from Struts 2 to Struts 6

Config and Code Changes

Servlet API Dependency

Deprecated Sitegraph Plugin Removed

Velocity Plugin

DTD of struts.xml

Class Changes

Constant Changes

OGNL

HTML Escaping

Tiles Support

Conclusion

Getting Output from jps with NRPE

Getting process information with jps

Running jps as root, with output to a file

Conclusion

Related reading

Migration from Struts 1 to Struts 2

A Comparison of Struts 1 and Struts 2

Migration

Dependencies

Request handling

Configure web application

Migrating actions

Action Configuration

Getting started with Java development using Visual Studio Code

Step 1: Install Visual Studio Code

Step 2: Install the extension pack

Step 3: Use the extension pack

Creating a Messaging App Using Spring for Apache Kafka, Part 5

Activation and Login

Activation

Login

Chat Application

Activation and login through the chat application

WebSocket Handshake

Chat in action

What is the point of using Kafka and WebSocket?

Conclusion

Quartz Scheduler as a Service

Quartz Job Scheduler

Quartz Scheduler as a Service

Read and Manage Job Data

Spring Application

Design

Scheduler

Spring Application Startup

JobFactory

Observer Job

KPIObserverJob

Business Jobs

Running this Solution

Extend by Creating a Management UI

Conclusion

Extending Your Jetty Distribution’s Capabilities

What is Jetty?

Know Your Jetty

Extending Your Jetty Capabilities

Build

Compile Classes Individually

Java developer job opening

What you will be doing:

You’ll need professional development experience with:

You have these important work traits:

What work here offers:

Get in touch with us:

Creating a Messaging App Using Spring for Apache Kafka, Part 4

WebSocketPool

MessageHandler

MessageHandlerImpl

MessageSender

WebSocketHandler

WebSocketConfig

MessageService

MessageServiceImpl