Tag Archives: KahaDB

MOM and JMS primer (ActiveMQ example)

Java, JMS July 19, 2013 Leave a comment

I’ve been reading recently a very good book on Message-oriented middleware (MOM) broker server Apache ActiveMQ which is an open source implementation of the Java Message Service (JMS) spec. and makes for a reliable hub in any message-oriented enterprise application and integrates beautifully with Java EE containers, ESBs, and other JMS providers.

The book “ActiveMQ in Action” (authored by Bruce Snyder, Dejan Bosanac, and Rob Davies) starts from the anatomy of a JMS message and moves quickly through connectors, message persistence, authentication, and authorization.

Please find below few notes i made while reading:

 

Standard Message Headers:

JMSDestination

  • Header Field: JMSDestination
  • Set By: JMS provider
  • Set mode: auto
  • Meaning: where to send the message
  • Example: queue://mycompany.1.0.notifications.rs.queue

JMSDeliveryMode

  • Header Field: JMSDeliveryMode
  • Set By: JMS provider
  • Set mode: auto
  • Meaning: persistent or non-persistent*
  • Example: Persistent

JMSExpiration

  • Header Field: JMSExpiration
  • Set By: JMS provider
  • Set mode: auto
  • Meaning: when should the message expire (in ms)
  • Example: 0 (never)

JMSPriority

  • Header Field: JMSPriority
  • Set By: JMS provider
  • Set mode: auto
  • Meaning: priority 0-9 (0-4 normal, 5-9 expedited)
  • Example: 4 (standard)

JMSMessageID

  • Header Field: JMSDestination
  • Set By: JMS provider
  • Set mode: auto
  • Meaning: identifier of the message
  • Example: ID:POL-MPRZYDATEK-63606-1370265295842-1:2:1:1:1

JMSTimestamp

  • Header Field: JMSTimestamp
  • Set By: JMS provider
  • Set mode: auto
  • Meaning: when the event occurred
  • Example: 2013-06-03 15:15:27:196 CEST

JMSCorrelationID

  • Header Field: JMSCorrelationID
  • Set By: JMS Client
  • Set mode: manual
  • Meaning: link one message with another (eg. conversation-style msg. ex.)

JMSReplyTo

  • Header Field: JMSReplyTo
  • Set By: JMS Client
  • Set mode: manual
  • Meaning: where (destination) is the reply expected

JMSType

  • Header Field: JMSType
  • Set By: JMS Client
  • Set mode: manual
  • Meaning: used by few vendors to semantically identify the message type

JMSRedelivered

  • Header Field: JMSRedelivered
  • Set By: JMS provider
  • Set mode: auto
  • Meaning: if set to true, it is likely that the msg was delivered but not acknowledged
  • Example: false

* Persistent messages are intended to survive system failures of the JMS provider (the message server). Persistent messages are written to disk as soon as the message server receives them from the JMS client. After the message is persisted to disk the message server can then attempt to deliver the message to its intended consumer. As the messaging server delivers the message to the consumers it keeps track of which consumers successfully receive the message. If the JMS provider fails while delivering the message, the message server will pick up where it left off following a recovery. Persistent messages are delivered once-and-only-once. Persistent messages incur
more overhead due to the need to store the message, and value reliability over performance.
* Nonpersistent messages are not written to disk when they are received by the message server, so if the JMS provider fails, the message will be lost. In general nonpersistent messages perform better than persistent messages. They are delivered more quickly and require less system resources on the message server. However, nonpersistent messages should only be used when a loss of messages due to a JMS provider failures is not an issue.

 

Request/reply messaging pattern

  • is an asynchronous back-and-forth conversational pattern utilizing either the PTP domain or the pub/sub domain through a combination of the JMSReplyTo and JMSCorrelationID message headers and temporary destinations.
  • The JMSReplyTo specifies the destination where a reply should be sent, and the JMSCorrelationID in the reply message specifies the JMSMessageID of the request message.
  • These headers are used to link the reply message(s) to the original request message.
  • Temporary destinations are those that are created only for the duration of a connection and can only be consumed from by the connection that created them.
  • These restrictions make temporary destinations useful for request/reply.

 

Distinguishing message durability from message persistence

  • Message durability can only be achieved with the pub/sub domain. When clients connect to a topic, they can do so using a durable or a nondurable subscription
    • Durable subscription—A durable subscription is infinite. It’s registered with the topic subscription to tell the JMS provider to preserve the subscription state in the event that the subscriber disconnects. If a durable subscriber disconnects the JMS provider will hold all messages until that subscriber connects again or until the subscriber explicitly unsubscribes from the topic.
    • Nondurable subscription—A nondurable subscription is finite. It’s registered with the topic subscription to tell the JMS provider to not preserve the subscription state in the event that the subscriber disconnects. If a subscriber disconnects, the JMS provider won’t hold any messages during the disconnection period
    • Message persistence is independent of the message domain. Message persistence is a quality of service property used to indicate the JMS application’s ability to handle missing messages in the event of a JMS provider failure

 

Four types of message persistence:

  • KahaDB
    • recommended message store for general-purpose messages since ActiveMQ version 5.3 is KahaDB.
    • a file-based message store that combines a transactional journal, for reliable message storage and recovery, with good performance and scalability
    • the structure of the KahaDB store has been streamlined especially for the requirements of a message broker.
    • The KahaDB message store uses a transactional log for its indexes and only uses one index file for all its destinations.
    • It’s been used in production environments with 10,000 active connections, each connection having a separate queue
  • AMQ
    • like KahaDB, is a combination of a transactional journal for reliable persistence (to survive system crashes) and high-performance indexes, which makes this store the best option when message throughput is the main requirement for an application.
    • But because it uses two separate files for every index, and there’s an index per destination, the AMQ message store shouldn’t be used if you intend to use thousands of queues per broker.
    • Also, recovery can be slow if the ActiveMQ broker isn’t shut down cleanly. This is because all the indexes need to be rebuilt, which requires the broker to traverse all its data logs to accurately build the indexes again.
  • JDBC
    • JDBC persistence is definitely not superior in performance to KahaDB nor AMQ
    • the use of a shared database is particularly useful for making a redundant master/slave topology out of multiple brokers. When a group of ActiveMQ brokers is configured to use a shared database, they’ll all try to connect and grab a lock in the lock table, but only one will succeed and become the master. The remaining brokers will be slaves, and will be in a wait state, not accepting client connections until the master fails.
  • memory
    • memory message store holds all persistent messages in memory. No active caching is involved, so you have to be careful that both the JVM and the memory limits you set for the broker are large enough to accommodate all the messages that may exist in this message store at one time.
    • The memory message store can be useful if you know that the broker will only store a finite amount of messages, which will typically be consumed quickly

 

Caching messages in the broker for consumers

  • Although one of the most important aspects of message persistence is that the messages will survive in long-term storage, there are a number of cases where messages are required to be available for consumers that were disconnected from the broker, but persisting the messages in a data store is too slow
  • ActiveMQ supports the caching of messages for these types of systems using message caching in the broker by using something called a subscription recovery policy. This configurable policy is used for deciding which types of messages should be cached, how many, and for how long
  • subscription recovery policies:
    • fixed (memory) size subscription recovery policy
    • fixed count (messages) subscription recovery policy
    • query-based subscription recovery policy – Caches only messages that match the query
    • timed subscription recovery policy (The time in milliseconds to keep messages in the cache
    • LAST IMAGE SUBSCRIPTION RECOVERY POLICY (only last message)
    • NO SUBSCRIPTION RECOVERY POLICY (disables message caching for topics)

 

Authentication

  • Simple authentication plug-in—Handles credentials directly in the XML configuration file or in a properties file
  • JAAS authentication plug-in—Implements the JAAS API and provides a more powerful and customizable authentication solution (JAAS provides pluggable authentication, which means ActiveMQ will use the same authentication API regardless of the technique used to verify user credentials (a text file, a relational database, LDAP, and so on)
  • custom plug-in for handling security (BrokerFilter implementation)

 

Authorization

  • operation-level
    • Read—The ability to receive messages from the destination
    • Write—The ability to send messages to the destination
    • Admin—The ability to administer the destination
  • message-level
    • remote address
    • message headers

 

Hope you find this short summary useful.