PostgreSQL & Datrium - from Virtualization to Bare-Metal with Data Services
While in my experience virtualization has been immensely pervasive for the past 15 years we still see applications that are habitually running on bare-metal Linux hosts due to the added performance, hardware lock-in or sometimes merely because of traditions.
On the other hand, the movement from SAN architectures to HCI is undoubtedly displacing complexity, but for such bare-metal applications, it offers no added benefit because HCI is traditionally built on top of hypervisors and virtualization technologies.
One of the applications that we often see running bare-metal is PostgreSQL, a modern Open Source Relational database, ranked 4th on global market share, and that is traditionally run on bare-metal Linux hosts.
Luckily, Datrium Open Convergence architecture efficiently solve IT teams’ dilemma on how to manage operations when virtualization, containers, and bare-metal applications need to co-exist on a single platform providing all the added management benefits of HCI and the Tier1 performance of SANs.
Getting it out of the way – a few HCI solutions provide NFS or ISCSI targets for applications running on bare-metal hosts, but in that case, the locally-attached Flash or NVMe are not leveraged to provide data locality, accelerate the application, gaining microsecond read latencies. This type of implementation is no different than a legacy 3-tier SAN architecture, adding complexity, and in many cases not taking advantage of table-stake features like deduplication, compression, erasure coding, file level awareness and visibility for snapshotting and replication.
Datrium implements the same architecture and features used with virtualization and bare-metal Linux hosts. We do that deploying a hyperdriver in each Linux host, and this hyperdriver will provide all Datrium features that IT would typically expect in a virtualized enterprise environment. That includes data-locality, deduplication, compression, erasure coding, encryption, file-level snapshots, restores, replication, cloud backup & cloud DR.
The nuance is that the Datrium GUI now can be used from vCenter and in stand-alone mode – that means IT now can look at performance information from Linux hosts from within vCenter.
I will write more on the bare-metal architecture, but for now, I want to focus on PostgreSQL.
I wrote an article entitled “PostgreSQL Benchmark on Datrium – 4.3 Million TPS with 1 GB RAM – and some Frustration!” a little while ago, and then I published another short post on ‘Software-Defined-Storage Architectures, a gift that keeps on giving! PostgreSQL and Datrium’, followed by Is there an Optimization Limit for Applications and Databases? PostgreSQL and Datrium Demonstration.
Clearly, I have been knee-deep into PostgreSQL, but today I want to demonstrate how PostgreSQL perform running on bare-metal Linux hosts with Datrium and comparing it to a virtualized PostgreSQL. For the sake of this demonstration, I kept all PGBENCH testing parameters the same to my previous tests. However, since this benchmark is running bare-metal, the workload will enjoy from all resources available on the server.
When compared a optimized and virtualized PostreSQL (same version):
– Transactions per Second (TPS) increased 90.9%
– Average Read latency remained increased from .3ms to .5ms
– Average Application Write Latency decreased by 650%
* Please see my article “PostgreSQL Benchmark on Datrium – 4.3 Million TPS with 1 GB RAM – and some Frustration!” to understand what vendors and solutions are being compared.
Important notes:
- The virtualized benchmark was using an E5-2698 v3 @ 2.30GHz, while the bare-metal uses an E5-2680 v4 @ 2.4GHz. It’s a newer CPU, but according to cpuboss.com the older processor is more powerful than, the newer one (here).
- Previous tests allocated 16vCPU and 1GB RAM while with bare-metal all host resources were available. That said, I made sure PostgreSQL did not starve VM CPU at any point in time, and on bare-metal CPU consumption was only about 40% at all times.
- The memory could have played a role in reducing latency, but we had a small increase in read latency. As for the write latency, that one dropped considerably, but it is essential to understand that the storage stack was the bottleneck because every write needs to be de-duplicated, compressed and erasure-coded inline.
Here is the screenshot of the pgbench result on RHEL:
Conclusion
I want to finish this blog post with a solid message that Datrium performs flawlessly for virtualized PostgreSQL databases as well for bare-metal Linux while delivering all data services.
However, to demonstrate that it is easy to improve TPS results and show staggering numbers I ran PGBENCH on a smaller dataset with a different number of clients and threads and here are the results…. 88,987 TPS… Incredible! I was not able to find any storage vendor, converged or not, that have provided numbers similar to that even with a small dataset and plenty of memory. So, please be aware of results that storage vendors throw your way. Run your tests and understand your workload.
This article was first published by Andre Leibovici (@andreleibovici) at myvirtualcloud.net