Test Driving Cassandra on GoGrid

Posted on

A Cloud of Big Data

An exciting development in the BigData solutions arena of late is hosted Big Data. GoGrid are one of the forerunners in this space with the promise of one-click setup for Big Data technologies such as Apache Cassandra and Apache Hadoop among others.

Recently we have entered into a partnership with GoGrid to help companies who need hosted NoSQL to get set up. If you are interested in a free test of their infrastructure, follow this link and get a generous voucher to use.

What follows is our own practical experience of getting set up with GoGrid and hopefully some useful information about finding your way around your shiny new cluster!

One Click to Rule them All

If you want to follow along just go here and click the Get Started button. By the time you have finished signing up with GoGrid your new cluster will already be set up.

Screen Shot 2014-05-06 at 13.23.26

We logged into a node using the public ip, the root user and the password provided in the dashboard. To check if our Cassandra Cluster is up we used ‘nodetool status’. We got the output below.

root@cassandra-01:~# nodetool status
Note: Ownership information does not include topology; for complete information, specify a keyspace
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address     Load       Owns   Host ID                               Token                                    Rack
UN  10.99.34.4  109.82 KB  34.2%  dc705839-217c-48a1-a1e6-a0906b977ad6  -9160067904041036413         rack1
UN  10.99.34.2  110.27 KB  34.5%  e3e08239-eb60-4cca-aac7-76060265de17  -9216627618871766268         rack1
UN  10.99.34.3  93.11 KB   31.3%  16ea38ee-3ffc-428d-83eb-936c5fe874bd  -9175295726179289665         rack1

Total Time: 5minutes (including registration)

Gaining Access

The default setup assumed we were accessing nodes from within the GoGrid network or from the nodes themselves. If you do want to access a node from an external server you can make one of the nodes publicly accessible. Open the yaml on one of the nodes in /etc/cassandra/cassandra.yaml and update the rpc_address to be the external IP of the machine. Restart using the /etc/init.d/cassandra script.

NB: This is a security risk so for production deployments it is not recommended.

We can now be run cqlsh from a local machine (with the cassandra tarball installed) to the public node for DML operations. We are doing this here to test an application integration later.

Schema Creation

Now we use cqlsh to create a demo keyspace which has data replicated to all nodes and a test table.

First the keyspace:

create keyspace gogrid_test WITH replication = {'class': 'SimpleStrategy','replication_factor': '3'};

And then the test table:

CREATE TABLE userProperties (user varchar, propName varchar, propValue varchar, PRIMARY KEY(user, propName));

Just to check everything is working properly, we hopped onto another node and ran ‘describe keyspaces’ and we could see the keyspace and table had been automatically created on all nodes.

Here is one I made earlier

Just for completeness we ran a sample application against the cluster. The sample in this case was a user properties store we used for our most recent Cassandra Developer Day.

Not a whole lot to see here except the output from gradle. The application populates test properties for a user into the Cassandra Cluster using the DataStax Java Driver. In this case, we are doing so at CL.QUORUM. Given the public contact point the tests ran first time and a quick check showed the data had been replicated to all nodes!

BUILD SUCCESSFUL

Total time taken so far to get a cluster up and an application running against it, about 15 minutes!

Tracing

Next we jumped back on a node and turned query tracing on (literally in cqlsh ‘tracing on’) to gauge the inter-cluster latency.

Select query executed in 3ms or so. That is pretty good.

cqlsh:gogrid_test> TRACING ON;
Now tracing requests.
cqlsh:gogrid_test> CONSISTENCY QUORUM;
Consistency level set to QUORUM.
cqlsh:gogrid_test> SELECT user, propname, propvalue FROM userproperties WHERE user='testUser' LIMIT 10;

 user     | propname | propvalue
----------+----------+-----------
 testUser |  testKey | testValue
 testUser | testKey1 | testValue

(2 rows)


Tracing session: dfe22650-d5d4-11e3-ace8-91692e612816

 activity                                                                                     | timestamp    | source     | source_elapsed
----------------------------------------------------------------------------------------------+--------------+------------+----------------
                                                                           execute_cql3_query | 03:46:41,718 | 10.99.34.2 |              0
 Parsing SELECT user, propname, propvalue FROM userproperties WHERE user='testUser' LIMIT 10; | 03:46:41,718 | 10.99.34.2 |             49
                                                                          Preparing statement | 03:46:41,718 | 10.99.34.2 |            289
                                                               Sending message to /10.99.34.4 | 03:46:41,719 | 10.99.34.2 |            797
                                           Executing single-partition query on userproperties | 03:46:41,719 | 10.99.34.2 |            865
                                                                 Acquiring sstable references | 03:46:41,719 | 10.99.34.2 |            881
                                                                  Merging memtable tombstones | 03:46:41,719 | 10.99.34.2 |            922
                                                                  Key cache hit for sstable 1 | 03:46:41,719 | 10.99.34.2 |           1118
                                                  Seeking to partition beginning in data file | 03:46:41,719 | 10.99.34.2 |           1130
                                                            Message received from /10.99.34.2 | 03:46:41,720 | 10.99.34.4 |             15
                    Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones | 03:46:41,720 | 10.99.34.2 |           2044
                                           Executing single-partition query on userproperties | 03:46:41,720 | 10.99.34.4 |            454
                                                   Merging data from memtables and 1 sstables | 03:46:41,720 | 10.99.34.2 |           2063
                                                                 Acquiring sstable references | 03:46:41,720 | 10.99.34.4 |            462
                                                            Message received from /10.99.34.4 | 03:46:41,720 | 10.99.34.2 |           2160
                                                                  Merging memtable tombstones | 03:46:41,720 | 10.99.34.4 |            490
                                                           Read 2 live and 0 tombstoned cells | 03:46:41,720 | 10.99.34.2 |           2174
                    Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 03:46:41,720 | 10.99.34.4 |            536
                                                   Merging data from memtables and 0 sstables | 03:46:41,720 | 10.99.34.4 |            544
                                                           Read 2 live and 0 tombstoned cells | 03:46:41,720 | 10.99.34.4 |            608
                                                            Enqueuing response to /10.99.34.2 | 03:46:41,720 | 10.99.34.4 |            768
                                                               Sending message to /10.99.34.2 | 03:46:41,720 | 10.99.34.4 |            829
                                                               Sending message to /10.99.34.3 | 03:46:41,721 | 10.99.34.2 |           2927
                                                         Processing response from /10.99.34.4 | 03:46:41,721 | 10.99.34.2 |           2996
                                                            Message received from /10.99.34.2 | 03:46:41,722 | 10.99.34.3 |             14
                                                            Message received from /10.99.34.3 | 03:46:41,722 | 10.99.34.2 |           null
                                           Executing single-partition query on userproperties | 03:46:41,722 | 10.99.34.3 |            331
                                                         Processing response from /10.99.34.3 | 03:46:41,722 | 10.99.34.2 |           null
                                                                 Acquiring sstable references | 03:46:41,722 | 10.99.34.3 |            337
                                                                  Merging memtable tombstones | 03:46:41,722 | 10.99.34.3 |            369
                    Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 03:46:41,722 | 10.99.34.3 |            405
                                                   Merging data from memtables and 0 sstables | 03:46:41,722 | 10.99.34.3 |            409
                                                           Read 2 live and 0 tombstoned cells | 03:46:41,722 | 10.99.34.3 |            462
                                                            Enqueuing response to /10.99.34.2 | 03:46:41,722 | 10.99.34.3 |            672
                                                               Sending message to /10.99.34.2 | 03:46:41,722 | 10.99.34.3 |            715
                                                                             Request complete | 03:46:41,721 | 10.99.34.2 |           3503

Inserting a record executed in about 2ms which is excellent for a QUORUM write. This showed our SSD latency and network latency are low.

qlsh:gogrid_test> INSERT INTO userproperties (user, propname, propvalue) VALUES ( 'testUser', 'demo3', 'val2');

Tracing session: 727fdf20-d5d5-11e3-ace8-91692e612816

 activity                                                                                              | timestamp    | source     | source_elapsed
-------------------------------------------------------------------------------------------------------+--------------+------------+----------------
                                                                                    execute_cql3_query | 03:50:47,698 | 10.99.34.2 |              0
 Parsing INSERT INTO userproperties (user, propname, propvalue) VALUES ( 'testUser', 'demo3', 'val2'); | 03:50:47,698 | 10.99.34.2 |             40
                                                                                   Preparing statement | 03:50:47,699 | 10.99.34.2 |            275
                                                                     Determining replicas for mutation | 03:50:47,699 | 10.99.34.2 |            439
                                                                        Acquiring switchLock read lock | 03:50:47,699 | 10.99.34.2 |            637
                                                                        Sending message to /10.99.34.4 | 03:50:47,699 | 10.99.34.2 |            639
                                                                                Appending to commitlog | 03:50:47,699 | 10.99.34.2 |            643
                                                                        Sending message to /10.99.34.3 | 03:50:47,699 | 10.99.34.2 |            639
                                                                     Adding to userproperties memtable | 03:50:47,699 | 10.99.34.2 |            665
                                                                     Message received from /10.99.34.2 | 03:50:47,700 | 10.99.34.3 |             11
                                                                     Message received from /10.99.34.2 | 03:50:47,700 | 10.99.34.4 |             13
                                                                     Message received from /10.99.34.4 | 03:50:47,700 | 10.99.34.2 |           1631
                                                                        Acquiring switchLock read lock | 03:50:47,700 | 10.99.34.3 |            335
                                                                        Acquiring switchLock read lock | 03:50:47,700 | 10.99.34.4 |            137
                                                                  Processing response from /10.99.34.4 | 03:50:47,700 | 10.99.34.2 |           1688
                                                                                Appending to commitlog | 03:50:47,700 | 10.99.34.3 |            369
                                                                                Appending to commitlog | 03:50:47,700 | 10.99.34.4 |            141
                                                                     Adding to userproperties memtable | 03:50:47,700 | 10.99.34.3 |            380
                                                                     Enqueuing response to /10.99.34.2 | 03:50:47,701 | 10.99.34.3 |            941
                                                                     Adding to userproperties memtable | 03:50:47,701 | 10.99.34.4 |            157
                                                                     Message received from /10.99.34.3 | 03:50:47,701 | 10.99.34.2 |           null
                                                                        Sending message to /10.99.34.2 | 03:50:47,701 | 10.99.34.3 |            996
                                                                     Enqueuing response to /10.99.34.2 | 03:50:47,701 | 10.99.34.4 |            233
                                                                  Processing response from /10.99.34.3 | 03:50:47,701 | 10.99.34.2 |           null
                                                                        Sending message to /10.99.34.2 | 03:50:47,701 | 10.99.34.4 |            301
                                                                                      Request complete | 03:50:47,699 | 10.99.34.2 |           1761

Wrapping Up

GoGrid scores highly on ease of use. In terms of getting a cluster up and running with an existing application in a matter of minutes it worked really well. We noticed there is not much to change on the provided configuration. VNodes are already active and the cluster is running a recent version of Cassandra 2.0.

In terms of performance, the SSD disks and low network latency contribute to fast reads and writes. More testing is suggested on this front, it would be nice to see some performance test figures.

A comment we would make about physical topology is that the Cassandra nodes are set up to be used from within the GoGrid network as the network binds are to the internally resolvable ip addresses. With this in mind, if you are running a PoC make sure to stand up your test api servers in the same GoGrid availability zone to make sure you don’t run foul of longer latencies.

As with any cluster, hosted or not be sure to simulate your traffic model before attempting to go live. Often issues only appear after the fact and again, consider co-locating your servers if possible.

If you are interested in trialling GoGrid for free or have any questions about this post get in touch or just follow this link to sign up for the free trial!