Amazon AWS – SQS – Simple Queue Service

Amazon AWS, Others September 19, 2013

Basic information about Amazon SQS Service:

 

AWS Free Tier availability:

  • 1 million requests

 

Developer Resources:

 

Functionality:

  • no limits on number of queues and number of messages.
  • queue can be created in any region.
  • message payload can contain up to 256KB of text in any format. Each 64KB ‘chunk’ of payload is billed as 1 request. For example, a single API call with a 256KB payload will be billed as four requests.
  • messages can be sent, received or deleted in batches of up to 10 messages or 256KB. Batches cost the same amount as single messages, meaning SQS can be even more cost effective for customers that use batching.
  • long polling reduces extraneous polling to help you minimize cost while receiving new messages as quickly as possible. When your queue is empty, long-poll requests wait up to 20 seconds for the next message to arrive. Long poll requests cost the same amount as regular requests.
  • messages can be retained in queues for up to 14 days.
  • messages can be sent and read simultaneously.
  • when a message is received, it becomes “locked” while being processed. This keeps other computers from processing the message simultaneously. If the message processing fails, the lock will expire and the message will be available again. In the case where the application needs more time for processing, the “lock” timeout can be changed dynamically via the ChangeMessageVisibility operation.
  • developers can securely share Amazon SQS queues with others. Queues can be shared with other AWS accounts and Anonymously. Queue sharing can also be restricted by IP address and time-of-day.
  • when combined with Amazon SNS, developers can ‘fanout’ identical messages to multiple SQS queues in parallel. When developers want to process the messages in multiple passes, fanout helps complete this more quickly, and with fewer delays due to bottlenecks at any one stage. Fanout also makes it easier to record duplicate copies of your messages, for example in different databases.

 

Common design patterns with SQS and other AWS components:

  • Work Queues: Decoupling components of a distributed application that may not all process the same amount of work simultaneously.
  • Buffer and Batch Operations: Adding scalability and reliability to the architecture, and smooth out temporary volume spikes without losing messages or increasing latency.
  • Request Offloading: Moving slow operations off of interactive request paths by enqueing the request.
  • Fanout: Combined with SNS to send identical copies of a message to multiple queues in parallel for simultaneous processing.

 

Service interface:

  • CreateQueue: Create queues for use with your AWS account.
  • ListQueues: List your existing queues.
  • DeleteQueue: Delete one of your queues.
  • SendMessage: Add messages to a specified queue.
  • SendMessageBatch: Add multiple messages to a specified queue.
  • ReceiveMessage: Return one or more messages from a specified queue.
  • ChangeMessageVisibility: Change the visibility timeout of previously received message.
  • ChangeMessageVisibilityBatch: Change the visibility timeout of multiple previously received messages.
  • DeleteMessage: Remove a previously received message from a specified queue.
  • DeleteMessageBatch: Remove multiple previously received messages from a specified queue.
  • SetQueueAttributes: Control queue settings like the amount of time that messages are locked after being read so they cannot be read again.
  • GetQueueAttributes: Get information about a queue like the number of messages in it.
  • GetQueueUrl: Get the queue URL.
  • AddPermission: Add queue sharing for another AWS account for a specified queue.
  • RemovePermission: Remove an AWS account from queue sharing for a specified queue.

 

Message Lifecycle:

  • A system that needs to send a message will find an Amazon SQS queue, and use SendMessage to add a new message to it.
  • A different system that processes messages needs more messages to process, so it calls ReceiveMessage, and this message is returned.
  • Once a message has been returned by ReceiveMessage, it will not be returned by any other ReceiveMessage until the visibility timeout has passed. This keeps multiple computers from processing the same message at once.
  • If the system that processes messages successfully finishes working with this message, it calls DeleteMessage, which removes the message from the queue so no one else will ever process it. If this system fails to process the message, then it will be read by another ReceiveMessage call as soon as the visibility timeout passes.

 

 

 

Resources:

Advertisement

Tagged:

%d bloggers like this: