We have tested two MongoDB cluster solutions deployed on Google Cloud Platform: Google Launcher’s MongoDB cluster and our MongoDB cluster Docker container running on Google Container Engine. We wanted to simply compare the performance of both solutions using a standard mongo performance benchmark. What are the benchmark results?

Mongo Database benchmark

The workload compares highly available instances of Google Launcher’s MongoDB cluster and our own deployment of MongoDB cluster docker image on Google Container Engine. The replication technology used is a standard MongoDB replica set with primary and secondary nodes and maximum of one arbiter node if needed (to have an odd number of nodes in total).

We chose the same smaller instances of 2 nodes + 1 really small Arbiter node for both mongo clusters. We picked the same storage size and type. Google doesn’t charge extra for using Google Launcher’s deployments and MongoDB as a software is also for free, so you end up paying the exact same price as you would for buying the cloud computational power from Google Compute’s services directly. For running MongoDB in Docker containers on Google Container Engine (GKE) you have to pay a bit more extra for the Kubernetes cluster service but it doesn’t really matter as the only reason to use this solution is when you are already using GKE in “production”.

Name Configuration HA Disk Price
Google Launcher MongoDB cluster 2x n1-highmem-2 + f1.micro ReplicaSet 2+1 150 GB SSD 188$
MongoDB Docker Cluster on GKE 2x n1-highmem-2 + f1.micro ReplicaSet 2+1 150 GB SSD 188+$

To determine throughput, a mongo-perf framework was used to generate workload from a mongo client in the same zone as the primary database instance. The workload is a set of step load tests that double the number of threads (connections) with each run. The benchmark’s set of tests is a selection of insert and update simple and complex test cases from the standard test package.

Operations per second

Operations per second results show that Google Launcher’s MongoDB cluster is faster than our MongoDB Docker cluster on GKE in most of the test cases. We chose three representative tests to show the results.

mongodb benchmark mongo mongo-perf
Insert.Empty test case simply inserts empty documents into database and lets mongod to create missing _id field which is monotically increasing so the entries are larger and larger than any current entry with more and more requests.

At 16 threads, variance and the potential for replication lag increase, causing both MongoDB clusters’ perfomance drop at higher thread counts. Google Launcher’s MongoDB seems to be more stable with the results here as well.

mongodb benchmark mongo mongo-perf
MMS update workload tests the overhead of reaching into deep right children in complex documents. This one particular test goes 24 levels deep, has 60 elements each and is incrementing 2 fields.

MMS sets of tests were the only tests with stable results where Google Launcher performed worse than our MongoDB Docker. You can also see how the performace goes rapidly down with more threads used due to synchronization overhead.

mongodb benchmark mongo mongo-perf
This test populates a big collection with _id and one numerial field and increments the number. It tests the Doc Validation overhead. Each thread works on its subset of collections.

The third test Update.DocValidation.OneNum shouldn’t cause any synchronization lag between concurrent threads so it shows the replication lag of both MongoDB cluster instances. Google Launcher wins here again.

Running the benchmark

We used the following environment configuration mongo-perf parameters for our testing.

Test environment:

  • Google Launcher’s MongoDB – 2x n1-highmem-2 (2cpu, 13GB RAM) + f1.micro (shared cpu, 0.6 GB RAM), MongoDB 3.4.1, 2x 150 GB SSD – Read/Write 4500 random IOPS, RW throughput of 72 MB/s
  • Our MongoDB Docker on GKE -2x n1-highmem-2 (2cpu, 13GB RAM) + f1.micro (shared cpu, 0.6 GB RAM), MongoDB 3.4.6, 2x 150 GB SSD – Read/Write 4500 random IOPS, RW throughput of 72 MB/s
mongodb benchmark mongo mongo-perf
Google Launcher’s MongoDB cluster deployment settings.

Test overview:

75 different insert and update tests were run with this command. 256 threads was more than both instances could handle to serve so the final benchmark was executed for a maximum of 128 threads. The output is saved in JSON format to a file.

The output JSON file is then parsed in a bash script called parse.sh using jq and saved into a CSV file. The CSV files are then automatically processed and used for plotting using gnuplot software.

The benchmark scripts, tests, environment setting and provisioning, the parser, the gnuplot template and more you can find in this github repository.

Result

Google Launcher’s MongoDB cluster outperformed our MongoDB Docker cluster on Google Container Engine in most of the test cases of the used benchmark. Google Launcher’s MongoDB is probably faster due to no overhead of Docker/Kubernetes and being optimized by Google’s Engineers to run on Google Compute VM’s.

Leave a Reply

Your email address will not be published. Required fields are marked *