Oracle + Datrium Encryption Benchmark with Near-Zero Performance Impact

This is my 4th article on the topic of Datrium + Oracle, and for this article, I focus on Datrium's ability to provide native FIPS 140-2 compliant software-based end-to-end encryption with near-zero performance impact for Oracle and Busines Critical Applications workloads.

If you haven't read the previous posts here they are:

Datrium AllFlash + Oracle measured with SLOB
Scaling Oracle SLOB to 7M IOPS and 55.4GB/s Throughput with Datrium – and the Latency FAKE NEWS!
Oracle + Datrium provide Enterprise Performance and Granular Protection with ZERO Performance Impact for Millions of Snapshots

However, before digging into the Oracle SLOB results, let me discuss the Datrium encryption because it is a truly unique and much broader technology than any other HCI or SAN vendor on the market.

Blanket Encryption & FIPS modes

Datrium’s encryption is an industry-first, implementing software-based end-to-end encryption with full data reduction (deduplication, compression, and erasure coding), across hosts, host SSDs, datanodes, and also for data in-flight (on the wire) and at-rest.

Datrium’s encryption uses FIPS AES-XTS-256 military grade crypto algorithm and leverage Intel Intelligent Storage Acceleration Library (Intel ISA-L) with Intel AES new encryption instructions set (Intel AES-NI), that provide <5% performance hit at worst case scenario, but in most cases is negligible. The Datrium encryption satisfies regulatory requirements for government agencies, banking, financial, healthcare and other G2000 enterprise customers.

AES – Advanced Encryption Standard specified in FIPS 197.
FIPS – Federal Information Processing Standard.
XTS-AES – Mode of Operation defined in IEEE Std. 1619-2007 and approved by SP800-38E with one additional requirement on the lengths of the data units.

There are two types of encryption to be known:

Approved mode is the FIPS mode that is using the encryption algorithm which is approved by FIPS, but the code path is not getting through the FIPS validation process.
Validated mode is the FIPS mode that is using the encryption algorithm that is validated through the FIPS validation process and certified by NIST.

The FIPS-Validated mode provides excellent performance, but it does impose some additional CPU load, while FIPS-Approved provides minimal performance degradation. Both methods have no impact on data reduction.

The detailed NIST report for Datrium FIPS Object Module FIPS 140-2 can be found here. (Make sure you ask your HCI vendor to show you their NIST certification!)

SLOB

For this test, I am using SLOB (Silly Little Oracle Benchmark) create by Kevin Closson, a tool focused on generating Oracle I/O workloads to stress the storage infrastructure. SLOB provides a multitude of possible configurations, but I decided to use what seems to be the most common configuration used by vendors – 70:30 Read/Write ratio. I used the same testbed and configuration described in my previous Oracle articles here.

VM and OS Configuration

32GB RAM and 12 vCPU
Oracle 12.2.0.1 Enterprise Edition – Single Instance
VMware ESXi, 6.7.0, 8941472
1 x 100GB vDISK for Linux CentOS/7
6 x 250GB vDISK PVSCSI w/ LVM aggregation w/ XFS for the Oracle database
1 Terabyte SLOB database

Hardware Configuration

PowerEdge R930 – E7-8890 v4 @ 2.20GHz / 2016
8 x SATA Samsung GC57 (MZ7LM240HMHQ0D3/2016)
1 x Datrium Datanode F12X2 2x25G-23TB

The configurations above represent in its totality one server, one datanode, and one virtual machine. Datanodes are responsible for receiving write IOs and storing data permanently, while a copy of the data is left, in a de-duplicated and compressed form, in each server for future local data read. Further, this is an old 2016 hardware configuration, like what most organizations would already have in its data centers.

1-Hour Oracle Burn-In with FIPS 140-2 Approved and Validated modes

The picture below demonstrates three 1-hour Oracle burn-in tests, being the first one without encryption, the second one with Approved mode encryption and the third one with Validated mode encryption. At the peak, we see SLOB/Oracle producing 67,020 IOPS with an average latency of 1ms.

The picture represents 5-minute roll-ups for all three 1-hour runs, and as we can see, there are negligible IOPS and Throughput variances - the numbers only vary a little because the SLOB workload isn't static. The VM-Level latency varies a little according to the workload, but the average latency remains steady with a maximum peak of 1.3ms.

It is particularly neat to see that deduplication and compression ratios remained basically the same during the entire set of tests. There are no other HCI solution on the market today, other than Datrium, able to deliver this kind of feature set – full data services with encryption and performance all together.

CPU Utilization

Datrium provides administrators with the ability to enable Insane Mode, allowing hosts to utilize up to 40% of host CPU to improve storage IO operations. In Fast mode (default mode) a maximum of 20% host CPU is allocated to the storage IO stack and data services.

I measured the host CPU consumption during the three benchmarks with Datrium in Insane mode, and the impact was negligible, with CPU consumption steady at ~31%. Please note that this is the host CPU consumption, including the Oracle VM and the Datrium software stack.

Oracle DB

I also measured the number of I/O requests per Second issued by Oracle, as driven by SLOB. The 2nd run with FIPS Approved-Mode presents a near-zero performance impact. On the 3rd run, with FIPS Validated-Mode we see a small variance of 1.8K I/Os. As I mentioned, in FIPS Validated-Mode we expect <5% performance hit at worst case scenario, and here the impact is only 3%.

We say 'No' to Complexity

Please note that the numbers provided above are not representing a peak-performance benchmark, but rather a consistent and near-zero impact of Datrium Blanket Encryption on Oracle workloads. If these were peak-performance benchmarks, there would be few configuration changes to Linux, Oracle and to the File System.

Instead of trying to pursue complex VM and database configurations as I have seen other vendors do, in this Oracle series I have opted for a more simplistic approach that doesn’t involve dozens of disks, LUNS or in-guest iSCSI protocol, or esoteric settings, which in production environments would be difficult to support. I even dropped Oracle ASM disk groups.

Demo

I know most of you believe my numbers and openness, but just to make sure you all believe my words I have recorded a live encryption mode change (from disabled to FIPS 140-2 Approved Mode) where we see near-zero performance impact to the Oracle/SLOB benchmark.

Conclusion

It is evident that Datrium provides consistent performance and near-zero impact encryption along with full enterprise data services (deduplication, compression and erasure coding).

For this Oracle/SLOB benchmark, the CPU load impact across all three benchmarks is zero, and the only perceivable performance impact is I/O requests per Second measured within Oracle with the FIPS Validated-Mode, with only 3% overhead. The FIPS Approved-Mode provided near-zero performance impact or overhead.

The servers and SSDs used are from 2016, and newer hardware would most definitely provide better performance and lower latencies. Datrium customers have been using NVMe drives on hosts and harvesting the benefits of the technology.

When it comes to supporting business-critical applications and enterprises workloads Datrium is miles ahead of other HCI solutions on the market. I recommend IT organizations to request a POC and simply compare.

This article was first published by Andre Leibovici (@andreleibovici) at myvirtualcloud.net