Category Archives: Others

Amazon AWS – DynamoDB – advanced NoSQL

Amazon AWS, Databases, NoSQL, Others September 20, 2013

Basic information about Amazon DynamoDB Service:

 

AWS Free Tier availability:

  • 100MB of Storage,
  • 5 Units of Write Capacity,
  • 10 Units of Read Capacity
  • up to 40 million free operations each month with eventually consistent reads, or 25 million operations each month with strictly consistent reads

 

Developer Resources:

 

Features:

  • Local secondary indexes
  • SSD-storage
  • automatic 3-way replication
  • you pay a flat, hourly rate based on the capacity you reserve
  • when creating or updating tables, you specify how much capacity you wish to reserve for reads and writes

 

Limits:

  • Table name: allowed characters are a-z, A-Z, 0-9, ‘_’ (underscore), ‘-‘ (dash), and ‘.’ (dot). Names can be between 3 and 255 characters long.
  • Local secondary index name: allowed characters are a-z, A-Z, 0-9, ‘_’ (underscore), ‘-‘ (dash), and ‘.’ (dot). Names can be between 3 and 255 characters long.
  • Table size: No practical limit in number of bytes or items
  • Tables per account: 256 per region (default). Increase can be requested.
  • Hash or hash-and-range primary key: No practical limit.
  • Number of hash key values: No practical limit.
  • Hash-and-range primary key: No practical limit for non-indexed tables (otherwise total sizes of all table and index items cannot exceed 10 GB)
  • Number of range keys per hash value: No practical limit for non-indexed tables (otherwise total sizes of all table and index items cannot exceed 10 GB)
  • Provisioned throughput capacity unit sizes: One read capacity unit = one strongly consistent read per second, or two eventually consistent reads per second, for items up 4 KB in size. One write capacity unit = one write per second, for items up to 1 KB in size.
  • Provisioned throughput minimum per table: 1 read capacity unit and 1 write capacity unit
  • Provisioned throughput limits: can request an increase but default values are:
    • US East (Northern Virginia) Region:
      • Per table – 40,000 read capacity units or 40,000 write capacity units
      • Per account – 80,000 read capacity units or 80,000 write capacity units
    • All Other Regions:
      • Per table – 10,000 read capacity units or 10,000 write capacity units
      • Per account – 20,000 read capacity units or 20,000 write capacity units
  • UpdateTable: Limits when increasing provisioned throughput: You can call UpdateTable as often as necessary to increase provisioned throughput. You can increase ReadCapacityUnits or WriteCapacityUnits for a table, subject to these conditions:
    • You can call the UpdateTable API to increase ReadCapacityUnits or WriteCapacityUnits (or both), up to twice their current values.
    • The new provisioned throughput settings do not take effect until the UpdateTable operation is complete.
    • You can call UpdateTable multiple times, until you reach the desired throughput capacity for your table.
  • UpdateTable: Limits when decreasing provisioned throughput: You can reduce the provisioned throughput on a table no more than four times in a single UTC calendar day. These reductions can be any of the following operations:
    • Decrease ReadCapacityUnits.
    • Decrease WriteCapacityUnits.
    • Decrease both ReadCapacityUnits and WriteCapacityUnits in a single request. This counts as one of your allowed reductions for the day.
  • Maximum concurrent Control Plane API requests (includes cumulative number of tables in the CREATING, UPDATING or DELETING state): In general, you can have up to 10 of these requests running concurrently. The only exception is when you are CREATING a table and you have defined a local secondary index on that table. You can only have one such request running at a time.
  • Maximum number of local secondary indexes per table: You can define up to 5 local secondary indexes per table.
  • Maximum number of projected attributes per table (local secondary indexes only): You can project a total of up to 20 attributes into all of a table’s local secondary indexes. This only applies to user-specified projected attributes.

    In a CreateTable operation, if you specify a ProjectionType of INCLUDE, the total count of attributes specified in NonKeyAttributes, summed across all of the local secondary indexes, must not exceed 20. If you project the same attribute name into two different indexes, this counts as two distinct attributes when determining the total.

    This limit does not apply for indexes with a ProjectionType of KEYS_ONLY or ALL.

  • Attribute name lengths: The following attribute names are length-restricted:
    • Primary key attribute names.
    • The names of any user-specified projected attributes (applicable only to local secondary indexes). In a CreateTableoperation, if you specify a ProjectionType of INCLUDE, then the names of the attributes in the NonKeyAttributes parameter are length-restricted. The KEYS_ONLY and ALL projection types are not affected.
    • For any of the attribute names listed above, the name must be between 1 and 255 characters long, inclusive. The name can be any UTF-8 encodable character, but the total size of the UTF-8 string after encoding cannot exceed 255 bytes.
  • Item size: Cannot exceed 64 KB which includes both attribute name binary length (UTF-8 length) and attribute value lengths (again binary length). The attribute name counts towards the size limit. For example, consider an item with two attributes: one attribute named “shirt-color” with value “R” and another attribute named “shirt-size” with value “M”. The total size of that item is 23 bytes. These limits apply to items stored in tables, and also to items in local secondary indexes.
  • Attribute values: cannot be null or empty
  • Attribute name-value pairs per item: The cumulative size of attributes per item must be under 64 KB
  • Hash primary key attribute value: 2048 bytes
  • Range primary key attribute value: 1024 bytes
  • String: All strings must conform to the UTF-8 encoding. Since UTF-8 is a variable width encoding, string sizes are determined using the UTF-8 bytes.
  • Number: A number can have up to 38 digits precision and can be between 10^-128 to 10^+126.
  • Maximum number of values in an attribute set: No practical limit on the quantity of values, as long as the item containing the values fits within the 64 KB item limit.
  • BatchGetItem item maximum per operation: Up to 100 items retrieved, with the request size not exceeding 1 MB.
  • BatchWriteItem item maximum per operation: Up to 25 items put or delete operations, with the request size not exceeding 1 MB.
  • Query: Result set limited to 1 MB per API call. You can use the LastEvaluatedKey from the query response to retrieve more results.
  • Scan: Scanned data set size maximum is 1 MB per API call. You can use the LastEvaluatedKey from the scan response to retrieve more results

 

 

 

Resources:

Amazon AWS – Overview

Amazon AWS, Others September 20, 2013

Here’s some basic info about AWS (Amazon Web Services) in case you’re considering deploying your product in the Amazon cloud:

 

Core AWS components:

 

 

Resources:

Amazon AWS – EC2 – Elastic Compute Cloud

Amazon AWS, Others September 20, 2013

Basic information about Amazon EC2 Service:

 

AWS Free Tier availability:

  • 750 hours of Linux and Windows Micro Instances each month for one year.

 

Developer Resources:

 

Instance categories:

  • On-Demand Instances – pay for compute capacity by the hour with no long-term commitments.
    • Removes the need to buy “safety net” capacity to handle periodic traffic spikes.
  • Reserved Instances – make a low, one-time payment for each instance you want to reserve and in turn receive a significant discount on the hourly charge for that instance.
    • There are three Reserved Instance types (Light, Medium, and Heavy Utilization Reserved Instances) that enable you to balance the amount you pay upfront with your effective hourly price.
    • If your needs change, you can request to move your Reserved Instance to another Availability Zone in the same region, and Amazon will process your request based on available capacity.
    • You can also sell your Reserved Instances to other customers for cash via the Reserved Instance Marketplace (currently, only customers with US bank accounts may sell Reserved Instances).
  • Spot Instances –  bid on unused Amazon EC2 capacity and run those instances for as long as their bid exceeds the current Spot Price.
    • The Spot Price changes periodically based on supply and demand, and customers whose bids meet or exceed it gain access to the available Spot Instances.
    • If you have flexibility in when your applications can run, Spot Instances can significantly lower your Amazon EC2 costs.

 

Features:

  • Elastic Block Store (EBS) – persistent storage for EC2 instances.
    • EBS volumes are network-attached, and persist independently from the life of an instance.
    • they can be leveraged as an Amazon EC2 instance’s boot partition or attached to a running Amazon EC2 instance as a standard block device.
    • when used as a boot partition, EC2 instances can be stopped and subsequently restarted, enabling you to only pay for the storage resources used while maintaining your instance’s state.
    • EBS volumes are automatically replicated on the backend (in a single Availability Zone).
    • they can be used to create point-in-time consistent snapshots of your volumes that are then stored in Amazon S3, and automatically replicated across multiple Availability Zones.
    • such snapshots can be used as the starting point for new Amazon EBS volumes
    • EBS provides two volume types: Standard volumes and Provisioned IOPS volumes.
      • standard volumes offer cost effective storage that is ideal for applications with moderate or bursty I/O requirements.
      • provisioned IOPS volumes are designed to deliver predictable, high performance for I/O intensive applications such as databases.
  • EBS-Optimized Instances – enable EC2 instances to fully utilize the IOPS provisioned on an EBS volume.
    • For additional hourly fee you can launch selected EC2 instances types as “EBS-Optimized” instances.
    • EBS-Optimized instances deliver dedicated throughput between EC2 and EBS, with options between 500 Mbps and 1000 Mbps depending on the instance type used.
    • When attached to EBS-Optimized instances, Provisioned IOPS volumes are designed to deliver within 10% of their provisioned performance 99.9% of the time.
  • Multiple Locations – place instances in multiple locations.
    • Amazon EC2 locations are composed of Regions and Availability Zones.
    • Availability Zones are distinct locations that are engineered to be insulated from failures in other Availability Zones and provide inexpensive, low latency network connectivity to other Availability Zones in the same Region. By launching instances in separate Availability Zones, you can protect your applications from failure of a single location.
    • Regions consist of one or more Availability Zones, are geographically dispersed, and will be in separate geographic areas or countries. The Amazon EC2 Service Level Agreement commitment is 99.95% availability for each Amazon EC2 Region. EC2 is currently available in nine regions:
      • US East (Northern Virginia),
      • US West (Oregon),
      • US West (Northern California),
      • EU (Ireland),
      • Asia Pacific (Singapore),
      • Asia Pacific (Tokyo),
      • Asia Pacific (Sydney),
      • South America (Sao Paulo),
      • AWS GovCloud
  • Elastic IP Addresses – static IP addresses designed for dynamic cloud computing.
    • An Elastic IP address is associated with your account not a particular instance, and you control that address until you choose to explicitly release it.
    • Unlike traditional static IP addresses, however, Elastic IP addresses allow you to mask instance or Availability Zone failures by programmatically remapping your public IP addresses to any instance in your account.
    • Rather than waiting on a data technician to reconfigure or replace your host, or waiting for DNS to propagate to all of your customers, EC2 enables you to engineer around problems with your instance or software by quickly remapping your Elastic IP address to a replacement instance.
    • In addition, you can optionally configure the reverse DNS record of any of your Elastic IP addresses
  • Virtual Private Cloud (VPC) – lets you provision a logically isolated section of AWS Cloud where you can launch AWS resources in a virtual network that you define.
    • You have complete control over your virtual networking environment, including selection of your own IP address range, creation of subnets, and configuration of route tables and network gateways.
    • You can also create a Hardware Virtual Private Network (VPN) connection between your corporate datacenter and your VPC and leverage the AWS cloud as an extension of your corporate datacenter.
  • CloudWatch – a web service that provides monitoring for AWS cloud resources and applications.
    • It provides you with visibility into resource utilization, operational performance, and overall demand patterns—including metrics such as CPU utilization, disk reads and writes, and network traffic.
    • You can get statistics, view graphs, and set alarms for your metric data.
  • Auto Scaling – allows you to automatically scale your EC2 capacity up or down according to conditions you define.
    • With Auto Scaling, you can ensure that the number of Amazon EC2 instances you’re using scales up seamlessly during demand spikes to maintain performance, and scales down automatically during demand lulls to minimize costs.
    • Auto Scaling is particularly well suited for applications that experience hourly, daily, or weekly variability in usage.
    • Auto Scaling is enabled by Amazon CloudWatch and available at no additional charge beyond Amazon CloudWatch fees.
  • Elastic Load Balancing – automatically distributes incoming application traffic across multiple EC2 instances.
    • It enables you to achieve greater fault tolerance in your applications
    • detects unhealthy instances within a pool and automatically reroutes traffic to healthy instances until the unhealthy instances have been restored.
    • You can enable Elastic Load Balancing within a single Availability Zone or across multiple zones
    • CloudWatch can be used to capture a specific Elastic Load Balancer’s operational metrics, such as request count and request latency, at no additional cost beyond Elastic Load Balancing fees.
  • High Performance Computing (HPC) Clusters – complex computational workloads such as tightly coupled parallel processes, or applications sensitive to network performance, can achieve the same high compute and network performance provided by custom-built infrastructure while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2.
    • Cluster Compute, Cluster GPU, and High Memory Cluster instances have been specifically engineered to provide high-performance network capability and can be programmatically launched into clusters – allowing applications to get the low-latency network performance required for tightly coupled, node-to-node communication.
    • Cluster instances also provide significantly increased throughput making them well suited for customer applications that need to perform network-intensive operations.
  • High I/O Instances – for customers requiring very high, low latency, random I/O access to their data:
    • High I/O instances are an Amazon EC2 instance type that can provide customers with random I/O rates over 100,000 IOPS.
    • High I/O instances are backed by Solid State Disk (SSD) technology and are ideally suited for customers running very high performance NoSQL and relational databases.
  • High Storage Instances – for customers requiring very high storage density per instance, and high sequential I/O for data-intensive applications like data warehousing and Hadoop:
    • High Storage instances are an Amazon EC2 instance type that can provide customers with sequential I/O throughout of 2.4 GB/s and provide customers with 48 TB of instance storage across 24 hard disk drives.
  • VM Import/Export – enables you to easily import virtual machine images from your existing environment to EC2 instances and export them back at any time.
    • By importing virtual machines as ready to use EC2 instances, you can leverage your existing investments in virtual machines that meet your IT security, configuration management, and compliance requirements.
    • You can export your previously imported EC2 instances back to your on-premise environment at any time.
    • available at no additional charge beyond standard usage charges for Amazon EC2 and Amazon S3.
  • AWS Marketplace – an online store that helps you find, buy and quickly deploy software that runs on AWS.
    • You can use AWS Marketplace’s 1-Click deployment to quickly launch pre-configured software and be charged for what you use, by the hour or month.
    • AWS handles billing and payments, and software charges appear on your AWS bill.

 

Operating Systems:

 

Databases:

 

Application Stacks:

 

 

 

Resources: