Category Archives: Java

ActiveMQ Custom Security Plugins

With this post i’m starting a short series of articles on creating custom security plugin’s for ActiveMQ server (probably the most flexible MOM/messaging solution around; imho).

 

To get a quick overview of how powerful ActiveMQ plugin API really is, let’s start with some basic background information:

  • The flexibility of ActiveMQ plugin API comes from the BrokerFilter class
  • BrokerFilter class provides the ability to intercept many of the available broker-level operations, such as:
    • adding consumers to the broker
    • adding producers to the broker
    • committing transactions in the broker
    • adding connections to the broker
    • removing connections from the broker
  • Custom functionality can be added by extending the BrokerFilter class and overriding a method for a given operation

 

Using the ActiveMQ plugins API is one way to approach broker security; used often for requirements (security, among others) that can’t be met using either:

  • ActiveMQ’s native Simple Authentication Plugin (which handles credentials directly in XML configuration file or in a properties file)
    or
  • JAAS-based pluggable security modules (JAAS stands for Java Authentication and Authorization Service). What is worth mention is that ActiveMQ comes with JAAS-based implementations of some modules that can authenticate users using properties files, LDAP, and SSL certificates; which will be enough for many use cases.

 

OK, having said the above, let’s move on and study following example implementations:

 

 

Resources:

Displaying GIT build number (hash) in your REST API

The product i’m working on currently (a PaaS cloud offering) had a requirement to provide an API resource (GET call) throughout which a user could obtain basic details about the actual version of the API exposed (the api version, build time, corresponding git repo build number (hash id) and the jvm version used to compile the API). Except for the git repo hash part, everything else seemed to be quite easy to obtain. Below you’ll find the solution (step-by-step guide) i came up with.

 

End result (goal):

> curl http://api.my-system.company.com/1.0/
{
  "Implementation-Build" : "2951e7e",
  "Implementation-Build-Time" : "2013/09/17 12:40:02 AM,UTC",
  "Implementation-Jdk" : "1.7.0_15",
  "Implementation-Version" : "1.0-SNAPSHOT",
  "Implementation-Vendor" : "My Company, Inc.",
  "Implementation-Title" : "My System"
}

 

Technologies used:

 

Steps required: 

1. First let’s add the <scm> configuration tag to your master pom.xml file. The connection string represents the repository for which the buildnumber-maven-plugin will obtain the git hash id.

<scm>
    <!-- Replace the connection below with your project connection -->
    <connection>scm:git:git://github.com/mariuszprzydatek/hyde-park.git</connection>
</scm>

 

2. configure the maven-war-plugin to generate project’s MANIFEST.MF file, where the git hash id will be stored. Also, the Spring MVC controller will read this file in order to return its content as a result of GET call.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-war-plugin</artifactId>
    <version>2.3</version>
    <configuration>
        <archive>
            <addMavenDescriptor>false</addMavenDescriptor>
            <manifest>
                <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
            </manifest>
        </archive>
        <warName>1.0</warName>
    </configuration>
</plugin>

 

3. In the <properties> section of the pom we can define the format for the date timestamp that will be returned as the value of “Implementation-Build-Time” attribute.

<properties>
    <maven.build.timestamp.format>yyyy/MM/dd hh:mm:ss a,z</maven.build.timestamp.format>
</properties>

 

4. Next, let’s add the remaining pom sections that we’ll be storing in the MANIFEST.MF file for further read:

    <version>1.0-SNAPSHOT</version>
    <organization>
        <name>My Company, Inc.</name>
    </organization>
    <name>My System</name>

 

5. within the <archive> key of the maven-war-plugin <configuration> section, we need to add additional manifest entries including the one (<Implementation-Build>) that will be generated by the buildnumber-maven-plugin:

<archive>
    ...
    <manifestEntries>
        <Implementation-Build>${buildNumber}</Implementation-Build>
        <Implementation-Build-Time>${maven.build.timestamp}</Implementation-Build-Time>
    </manifestEntries>
</archive>

 

6. Add the buildnumber-maven-plugin itself which will do the hard work:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>buildnumber-maven-plugin</artifactId>
    <version>1.1</version>
    <executions>
        <execution>
            <phase>validate</phase>
            <goals>
                <goal>create</goal>
            </goals>
        </execution>
    </executions>
</plugin>

 

7. Finally, add the <configuration> section to the buildnumber-maven-plugin together with the <shortRevisionLength> key that is responsible for the length of git hash id we want to export:

<configuration>
    <shortRevisionLength>7</shortRevisionLength>
</configuration>

 

 

Now, let’s create the Spring MVC controller that will be handling the MANIFEST.FM file read and returning its content to the presentation layer.

@Controller
@RequestMapping
public class ApiController {

    /**
     * Handling GET request to retrieve details from MANIFEST.MF file
     * @return implementation details
     */
    @RequestMapping(method = RequestMethod.GET)
    public @ResponseBody Map<String, String> getBuildNumber(HttpServletRequest request) throws IOException {

        ServletContext context = request.getSession().getServletContext();
        InputStream manifestStream = context.getResourceAsStream("/META-INF/MANIFEST.MF");
        Manifest manifest = new Manifest(manifestStream);

        Map<String, String> response = new HashMap<>();
        response.put("Implementation-Vendor", manifest.getMainAttributes().getValue("Implementation-Vendor"));
        response.put("Implementation-Title", manifest.getMainAttributes().getValue("Implementation-Title"));
        response.put("Implementation-Version", manifest.getMainAttributes().getValue("Implementation-Version"));
        response.put("Implementation-Jdk", manifest.getMainAttributes().getValue("Build-Jdk"));
        response.put("Implementation-Build", manifest.getMainAttributes().getValue("Implementation-Build"));
        response.put("Implementation-Build-Time", manifest.getMainAttributes().getValue("Implementation-Build-Time"));

        return response;

    }

}

 

 

Hope you enjoyed this post.

Take care!

Redis data sharding – part 2 – hash-based keys

In my previous post on Redis data sharding i introduced the concept of data sharding/partitioning and provided a small Java code example to illustrate the idea. As you noticed i was creating fixed-size “emailbuckets”, containing 1024 emails each. An email address was the key of my hash, while user id was the value. For a shard identifier i used a simple integer value (shardKey) obtained as a result of “i mod shardSize” operation.

Whereas such approach illustrates the concept well, it’s impractical in “real life” applications. This is due to a simple reason – knowing the email address alone (which may often be the only thing you’d know at some point of the app execution flow; for example when you’re using Spring Security and requesting emails “as usernames” during sign-in process), you wouldn’t be able to retrieve the corresponding userId. If you would know the algorithm by which shardKey was generated, in this case – yes – you would be able to traverse emailbuckets one-by-one looking for the appropriate email, but without that knowledge you wouldn’t be able to tell which shard the email address you’re looking for, ended up in.

One solution to that problem is to use a different key for sharding data; something that is computed directly based on the data you’re interested in partitioning. An obvious candidate here is the email address itself. If you’d be able to generate shardKey based on email address you could reproduce the same scenario every time a user provides you with his email during signing in and retrieve his userId (which you could use later on (for example) to further query another hash in Redis – “users:id” – that stores complete user profile).

 

This seems like an ideal task for a hash function… Let’s start first with some background on hashing. According to Neil Coffey’s Javamex article Introduction to hashing:

  • Hashing means using some function or algorithm to map object data (eg. content of a String object) to some representative integer value. This so-called hash code (or simply hash) can then be used as a way to narrow down our search when looking for the item…

 

Also, when you search Wikipedia after Java hashCode() function, you’ll get the following definition:

  • In the Java programming language, every class must provide a hashCode() method which digests the data stored in an instance of the class into a single hash value (a 32-bit signed integer). This hash is used by other code when storing or manipulating the instance – the values are intended to be evenly distributed for varied inputs in order to use in clustering. This property is important to the performance of hash tables and other data structures that store objects in groups (“buckets”) based on their computed hash values.

 

Looks like this is exactly what we’re interested in – …evenly distributed values for varied inputs…, which …is important to the performance of data structures that store objects in groups (shards in our case) based on their computed hash values.

Conclusion: Java hashCode() function is what we’ll proceed with.

 

More from Wikipedia on Java hashCode():

  • Starting with Java version 1.2, the java.lang.String class implements its hashCode() using a product sum algorithm over the entire text of the string.
  • An instance s of the java.lang.String class, would have a hash code h(s) defined by:
    Java String hashCode()

     

  • where terms are summed using Java 32-bit int addition, s[i] denotes the i-th character of the string, and n is the length of s.

 

Now, looking at Java docs on String.hashCode() function we can read:

  • The hash code for a String object is computed as
    s[0]*31^(n-1) + s[1]*31^(n-2) + … + s[n-1]

     

  • using int arithmetic, where s[i] is the ith character of the string, n is the length of the string, and ^ indicates exponentiation. (The hash value of the empty string is zero.)

 

Finally let’s take a look at some Java code of String object showing how hashCode() has actually been implemented:

public int hashCode() {
    int h = hash;
    if (h == 0 && value.length > 0) {
        char val[] = value;

        for (int i = 0; i < value.length; i++) {
            h = 31 * h + val[i];
        }
        hash = h;
    }
    return h;
}

 

An alternative (faster) implementation may look like this (from Apache Harmony JDK):

public int hashCode() {
    if (hashCode == 0) {
        int hash = 0, multiplier = 1;
        for (int i = offset + count - 1; i >= offset; i--) {
            hash += value[i] * multiplier;
            int shifted = multiplier << 5;
            multiplier = shifted - multiplier;
        }
        hashCode = hash;
    }
    return hashCode;
}

what’s the difference between the two above code snippets? As you can see, multiplication can be replaced by a bitwise shift operation and a subtraction for better performance. “(multiplier << 5) – multiplier” is just 31*multiplier after all (however VMs nowadays do this optimization automatically). If you’re interested in good reading on the subject of Binary numbers i strongly recommend Neil Coffey’s Javamex article: Introduction to binary numbers.

 

OK, applying all this knowledge to our sharding code example results in the following implementation:

while(i<1000000) {
    String userId = String.valueOf(i++);
    String emailAddress = String.format("user_%s@mariuszprzydatek.com", userId);
    int shardKey = emailAddress.hashCode();
    redisTemplate.opsForHash().put(String.format("emailsbucket:%s", shardKey), emailAddress, userId);
}

 

Happy coding! 🙂

 

 

Resources: