Cassandra Data Distribution using Partitioners

Before wading through the partitioners lets have a look how generally data is being distributed in a Cassandra
cluster. It includes a ring like a topology between the nodes. Here the data are being broken into tokens and are circulated
among the nodes through the Cassandra ring. Data is distributed across the cluster by the value of the token by using the
hash technique. Based on the value of the tokens the data will be evenly distributed among the cluster. The main advantage
of this method is the data retrieval process can be quick based on the token range. Now we can have can see what does a
partitioner role in a cluster in detail.

Partitioners

Partitioners allow how row keys should be sorted and how data will be distributed across your nodes in the cluster. The
read/write request to the cluster are evenly distributed when each part of the hash range receives the same number of
tokens on average. Based on the difference in the hash methods partitioners are classified into three types as below:

• Murmur3Partitioner
• RandomPartitioner
• Byte Ordered Partitioner

Murmur3 Partitioner

This is the default partitioning strategy for Cassandra. It provides fast hashing and good performance. It
uses MurmurHash hash values to distribute the data across the clusters.

Random Partitioner

It involves MD5 hash applied to place the keys on the node ring. An MD5 hash provides a natural way of
load balancing keys to nodes. Each data item is mapped to a token by calculating the MD5 hash of its key
The disadvantage of this method is that it causes inefficient range queries when keys specified in the range
might be in another ring.

Byte Ordered Partitioner

This method involves the distribution of the data lexically by key bytes in an ordered manner. It treats
the data as raw bytes, instead of converting them to strings. This is most likely to use when you want a
partitioner that doesn't want to validate the keys as being strings.

How to change a Partitioner?

So based on the need for the application you can choose any one of the above partitioners. In order
to apply it make the below change in the cassandra.yaml file. This file will be generally located in conf
directory.

So this is all about how a data is distributed among the Cassandra cluster and how partitioner helps more
for the sorting of data & retrieving them. Got any queries about partitioners to comment on them.

Read More

Consistency levels in Apache Cassandra explained

Cassandra is scalable column-oriented open source NoSQL database. It is the right choice for managing large amounts of structured, semi-structured, and unstructured data across multiple data centers when you need scalability and high availability without compromising performance. In this article, we are going to discuss how the read/write operations are maintained in a cluster and various consistency levels in Cassandra & how can they be applied to our business applications.

According to CAP theorem, it is impossible for a distributed system to simultaneously provide all three guarantees:

• Consistency -Every node contains same data at the same time
• Availability- At least one node must be available to serve data every time
• Partition tolerance -Failure of the system is very rare

"Cassandra is typically classified as an AP system, meaning that availability and partition tolerance are generally considered to be more important than consistency in Cassandra. But Cassandra can be tuned with replication factor and consistency level to also meet C.So Cassandra is eventually consistent."

Replication factor:

Before deep diving into the consistency levels its necessary to understand the term replication factor. It describes how many copies of your data exist. Based on the RF & the consistency levels it is easy to design a very good stable architecture in Cassandra.

The below terms explains how the write/read transactions serve its purpose:

Commit log − The commit log is a crash-recovery mechanism in Cassandra. Every write operation is written to the commit log.

Mem-table − A mem-table is a memory-resident data structure. After commit log, the data will be written to the mem-table. Sometimes, for a single-column family, there will be multiple mem-tables.

SSTable − It is a disk file to which the data is flushed from the mem-table when its contents reach a threshold value

Write path in Cassandra:

When a write is initiated its first captured by the commit logs. Later the data will be captured and stored in the mem-table. Whenever the mem-table is full, data will be written into the SStable data file. All writes are automatically partitioned and replicated throughout the cluster. Cassandra periodically consolidates the SSTables, discarding unnecessary data.

Read path in Cassandra:

For any read operations first, the values are fetched from the mem table and then Cassandra checks the bloom filter(cache) to find the appropriate SSTable that holds the required data.

Consistency Levels :

Consistency levels are used to manage the data consistency versus data availability. Below are the various levels of consistency that can be set to achieve the data consistency in the DB:

ALL- Writes/Reads must be written to the commit log and memtable on all in the cluster.

EACH_QUORUM- Writes/Reads must be written to the commit log and memtable on each quorum of nodes. Quorum is 51% of the nodes in a cluster.

QUORUM- Writes/Reads must be written to the commit log and memtable on a quorum of nodes across all data centers.

LOCAL_QUORUM- Writes/Reads must be written to the commit log and memtable on a quorum of nodes in the same datacenter as the coordinator.

ONE- Writes must/Reads be written to the commit log and memtable of at least one node.

TWO- Writes/Reads must be written to the commit log and memtable of at least two nodes.

THREE- Writes/Reads must be written to the commit log and memtable of at least three nodes.

LOCAL_ONE- Writes/Reads must be sent to and successfully acknowledged by, at least one node in the local datacenter.

ANY- Writes/Reads must be written to at least one node.

How to calculate the DB impact based on these parameters?

Its very easy to calculate the DB impacts for any given RF & read, write Consistency levels. For example, say let us set up a 5 node cluster with 3 RF, Read & Write Consistency level as quorum then the impact would be as below:

1.     Your reads are consistent

2.     You can survive the loss of 1 node without impacting the application.

3.     You can survive the loss of 1 node without data loss.

4.     You are really reading from 2 nodes every time.

5.     You are really writing to 2 nodes every time.

6.     Each node holds 60% of your data.

The same cluster scenario with Read & Write Consistency level as ONE will have the below impact.

1.     Your reads are eventually consistent

2.     You can survive the loss of 2 nodes without impacting the application.

3.     You can survive the loss of no nodes without data loss.

4.     You are really reading from 1 node every time.

5.     You are really writing to 1 node every time.

6.     Each node holds 60% of your data.

Thus the Cassandra cluster architecture can be defined according to our own business need with the optimal use of the resources to yield high performance.

Credits: You can use this Cassandra Parameters for Dummies to find out the impact: https://www.ecyrd.com/cassandracalculator/

Read More