The most recent result was for a four-socket Lenovo ThinkSystem SR950 with 3TB of RAM using a 48TB initial database size. This system had an official result of 11,357.28, which is the highest score ever submitted for a four-socket server. This system has a total of 112 physical cores, so if you divide the total score of 11,357.28 by 112, you get a measure of the single-threaded performance of the Intel Xeon Platinum 8180 processor under a full load (where the clock speed of the individual cores will be pretty close to the 2.5GHz base clock speed). In this case, the result is 101.40 score/core.

Back on June 27, 2017, Lenovo submitted a result for a two-socket Lenovo ThinkSystem SR650 with 1.5TB of RAM using a 28.5TB initial database size. This system had an official result of 6,598.36, which is the highest score ever submitted for a two-socket server. This system has a total of 56 physical cores, so if you divide the total score of 6,598.36 by 56, you get a score/core of 117.83, which is significantly higher than the result for the Lenovo ThinkSystem SR950 configured to use four-sockets (using the exact same Intel Xeon Platinum 8180 processor).

I would attribute most of this difference to the added NUMA overhead from a four-socket system, compared to a two-socket system. Another difference, which probably hurt the score of the two-socket system was the fact that it had to be running on a pre-release version of SQL Server 2017, based on the submission date of the benchmark.

This is just another piece of evidence that even with NUMA, capacity does not scale in a linear fashion as you add sockets to a server. Assuming you can split your workload across multiple database servers rather than just one, having two, two-socket servers instead of one, four-socket server will give you both more CPU capacity and better single-threaded CPU performance even when using the exact same model processor.

I would also argue that you could purposely pick a lower core count, but higher base clock speed processor from the same Intel Xeon Scalable Processor Family to find a sweet spot for SQL Server 2017 usage, where you have fewer physical cores to license, with better single-threaded performance across a higher number of servers.

The post Recent TPC-E Results on SQL Server 2017 appeared first on Glenn Berry.

These are both incredible scores for a four-socket system, both for the actual raw score and from a score per physical core perspective. Both of these tested systems have actual TPC-E scores that rival an eight-socket system with the previous generation 32nm Intel Xeon E7-4870 Westmere-EX processor, while their single-threaded performance (as measured by the TPC-E score divided by the number of physical cores) is also relatively close to what we see in the latest 22nm Intel Xeon E5-2697 v2 Ivy Bridge-EP processors. This gives you the possibility of eight-socket capacity, with close to modern two-socket single-threaded performance in a four-socket server.

Table 1: Recent IBM TPC-E Benchmark scores

As you can see from Table 1, the Intel Xeon E7-4890 v2 processor is a huge improvement over the previous Intel Xeon E7-4870 processor, with much higher overall capacity and higher single-threaded performance. You also get much higher memory capacity and PCI-E 3.0 support with the new processor.

On the negative side, your SQL Server 2012/2014 core license costs will be 50% higher if you go with the high-end 15-core E7-4890 v2 processor. One alternative would be to use the 12-core, Xeon E7-4860 v2 processor or even the ten-core, Xeon E7-4830 v2 processor to minimize your SQL Server 2012/2014 license costs. One slight problem with that strategy is that the base and turbo clock speeds are lower in the lower core-count processors in the Xeon E7-48xx v2 product family, since they don’t have lower core count, “frequency-optimized” models like the Xeon E5-26xx v2 product family does.

Four-socket systems with these new processors are going to be much faster and have much more total load capacity than previous four-socket systems with the older Westmere-Ex processor.

The post Two New TPC-E Benchmark Results for Intel Xeon Ivy Bridge-EX Processors appeared first on Glenn Berry.

Imagine a legacy system that is a Dell PowerEdge 2950 with one 45nm, quad-core, 3.0GHz Intel Xeon X5450 “Harpertown” processor, along with 64GB of RAM. That processor has a 1333MHz FSB and a 12MB L2 cache. It has the 45nm Core2 Quad “Harpertown” microarchitecture, which means that it does not support Intel hyper-threading or Intel Turbo Boost, and it uses the older symmetric multiprocessing (SMP) architecture instead of the newer non-uniform memory access (NUMA) architecture.

Nearest TPC-E Comparable Result for Existing System

There is a TPC-E result from 12/11/2007 for a Dell PowerEdge 2900 system with one 65nm, quad-core, 2.66GHz Intel Xeon X5355 “Clovertown” processor, along with 48GB of RAM. That processor has a 1333MHz FSB and an 8MB L2 cache. It has the 65nm Core2 Quad “Clovertown” microarchitecture, which means that it also does not support Intel hyper-threading or Intel Turbo Boost, and it also uses the older SMP architecture. The Intel Xeon 5300 series is one Intel Tick release older than the Intel Xeon 5400 series, so there is a relatively small difference in their relative performance. This actual TPC-E score is 144.88. The Dell system from 2007 was running SQL Server 2005 on Windows Server 2003.

Comparing that Dell TPC-E system to the existing system, we have to make some adjustments to account for the clock speed difference, L2 cache size difference and the Intel Tick release difference. A 3.0GHz clock speed is 12.4% higher than a 2.66GHz, and I estimate that the combination of a larger L2 cache and the newer Tick release would be another 10% difference. If we multiply 144.88 times 1.224, we get a result of 177.33 as an estimated TPC-E score for the current legacy system.

Nearest TPC-E Comparable Result for New System

There is also a TPC-E result from 11/21/2012 for an HP Proliant DL380p Gen 8 system with two 32nm, eight-core, 2.9GHz Intel Xeon E5-2690 “Sandy Bridge-EP” processors, along with 256GB of RAM. This has the 32nm Sandy Bridge-EP microarchitecture, which means that it supports both Intel hyper-threading and Intel Turbo Boost, and it uses the newer NUMA architecture. It also has PCI-E 3.0 support. The actual TPE-E result for this system is 1881.76. This system is running on Windows Server 2012 and SQL Server 2012.

Since we want to minimize our SQL Server 2012 core-based license costs, we are considering only using one actual Xeon E5-2600 series processor in the new server, possibly with a lower core count. The best choices for SQL Server 2012 are the four-core 3.3GHz Intel Xeon E5-2643, the six-core 2.9GHz Intel Xeon E5-2667, and the eight-core 2.9GHz Intel Xeon E5-2690. These three processors have slightly different base and Turbo clock speeds and different L3 cache sizes (although the size per core is the same) and different core counts that must be accounted for. We also need to account for the fact that we will only have one physical processor in the system instead of two.

With a NUMA architecture in a two-socket machine, you will get quite good scaling as you go from one processor to two processors. I believe we should use an estimate of 55% (i.e. one processor will have 55% of the scalability of two identical processors in the NUMA architecture system). We will have to adjust for the core-count difference in the six-core and quad-core processors. We also need to adjust for the higher base clock speed difference in the quad-core Xeon E5-2643 system.

The two-socket Xeon E5-2690 system has an actual TPC-E score of 1881.76. If we multiply that by .55 we get an estimated TPC-E score of 1034.97 with one Xeon E5-2690. If we multiply that by .75, we get an estimated TPC-E score of 776.23 with one Xeon E5-2667.

If we take the 1034.97 estimate for a single eight-core Xeon E5-2690 and multiply that by .50, we get a result of 517.49 for the four-core Xeon E5-2643. We also need to multiply that by 1.138 to account for the 3.3GHz base clock speed compared to the base 2.9GHz clock speed. This gives us an estimated TPC-E score of 588.90 for a single Xeon E5-2643 processor.

The table below summarizes these TPC-E score estimates.

The post Using TPC-E OLTP Benchmark Scores to Compare Processors appeared first on Glenn Berry.

The TPC Benchmark E (TPC-E) is an OLTP performance benchmark that was introduced in February 2007. TPC-E is a not a replacement for the older TPC-C benchmark, but rather is a completely new OLTP benchmark. It is an OLTP, database-centric workload that is meant to reduce the cost and complexity of running the benchmark compared to the older TPC-C benchmark. It simulates the OLTP workload of a brokerage firm that interacts with customers using synchronous transactions and with a financial market using asynchronous transactions.

The business model of the brokerage firm is organized by Customers, Accounts, and Securities. The data model for TPC-E is significantly more complex, but more realistic than TPC-C, with 33 tables and many different data types. The data model for the TPC-E database does enforce referential integrity, unlike the older TPC-C data model.

The TPC-E database is populated with pseudo-real data, including customer names from the year 2000 U.S. Census, and company listings from the NYSE and NASDAQ. Having realistic data introduces data skew, and makes the data compressible. Unlike TPC-C, the storage media for TPC-E must be fault tolerant (which means no RAID 0 arrays). Overall, the TPC-E benchmark is designed to have reduced I/O requirements compared to the old TPC-C benchmark, which makes it both less expensive and more realistic since the sponsoring hardware vendors will not feel as much pressure to equip their test systems with disproportionately large disk subsystems in order to get the best test results. The TPC-E benchmark is also more CPU intensive than the old TPC-C benchmark. It is essentially CPU-bound, as long as you have adequate I/O capacity to drive the workload.

The TPC-E implementation is broken down into a Driver and a System Under Test (SUT), separated by a mandatory network. The Driver represents the various client devices that would use an N-tier client-server system, abstracted into a load generation system. The SUT has multiple Application servers (Tier A) that communicate with the database server and its associated storage subsystem (Tier B). TPC provides a transaction harness component that runs in Tier A, while the test sponsor provides the other components in the SUT.

The performance metric for TPC-E is transactions per second, tpsE. The actual tpsE score represents the average number of Trade Result transactions executed within one second. To be fully compliant with the TPC-E standard, all references to tpsE results must include the tpsE rate, the associated price per tpsE, and the availability date of the priced configuration.

It seems interesting that, as of early 2013, Microsoft is the only database vendor that has any submitted TPC-E results, even though the TPC-E benchmark has been available for over six years. Whatever the reasons why the other database vendors haven’t allowed any TPC-E results to be submitted by the hardware vendors, there are certainly many results posted for SQL Server, which makes it a very useful benchmark when assessing SQL Server hardware.

The most recent posted TPC-E result is for an IBM System x3850 X5 Server with a 5457.20 tpsE score for an eight-socket system. This system has eight, ten-core Intel Xeon E7-8870 processors that have a total of 160 logical cores for the system. It also has 4TB of RAM and 236 SSDs in its I/O subsystem, using a 22TB initial database size for the test. Looking at the Executive Summary, you can see that it is running SQL Server 2012 Enterprise Edition on top of Windows Server 2012 Standard Edition. This is basically the biggest system (in terms of RAM) that you can build under the current Windows Server 2012 license limits for memory.

It is using RAID 5 for the data files, RAID 10 for the log file and RAID 1 for tempdb, with (236) 200GB 3Gbps SAS SSDs (model # 81Y9956), and (2) 600GB 6Gbps 10K SAS drives. The data files are spread across eleven, twenty-drive RAID 5 arrays (using 200GB 3Gbps SSDs), while the log file is on one, sixteen-drive RAID 10 array (using 200GB 3Gbps SSDs), and tempdb is on one, two-drive RAID 1 array, using 600GB 10K SAS drives). That shows you that tempdb is probably not hit very hard during the TPC-E benchmark!

The post A SQL Server Hardware Tidbit a Day – Day 21 appeared first on Glenn Berry.

What is notable about this is that the 3218.46 score for a four-socket Xeon E7-4870 system is significantly higher than we have seen for similar four-socket Xeon E7-4870 systems in the past. An especially good comparison is between an IBM System x3850 X5 that was submitted on June 27, 2011 and this latest result for an IBM System x3850 X5 system that was submitted on November 28, 2012.  As you can see in Table 1, the newer submission for the same model server has a 12.4% higher score than the older submission. This is for the exact same model server, with the exact same number and model of processors.  The first big difference that jumps out is that the newer submission is running SQL Server 2012 Enterprise Edition on top of Windows Server 2012 Standard Edition, while the older submission is running SQL Server 2008 R2 Enterprise Edition on top of Windows Server 2008 R2 Enterprise Edition.

Table 1: Comparing Two IBM System x3850 X5 TPC-E Submissions

Could this 12.4% performance jump be simply due to the newer operating system and the newer version of SQL Server?  It is very possible that there were some low level improvements in Windows Server 2012 that work in conjunction with SQL Server 2012 to improve performance (similar to what we saw with Windows Server 2008 R2 combined with SQL Server 2008 R2). With Windows Server 2008 R2, Microsoft did some low-level optimizations so that they could scale from 64 logical processors to 256 logical processors. This work also benefitted smaller systems with fewer logical processors.  I think it is likely that some similar work was done with Windows Server 2012, so that it could scale from 256 logical processors to 640 logical processors, so that might explain some of the performance increase. I have some questions in to some of my friends at Microsoft, trying to get some more detailed information about this possibility.

It is also possible that there were improvements in SQL Server 2012 all by itself that contributed to the performance increase. Another possibility is that the TPC-E team at IBM just did a much better job on this newer system. If you dive deeper into the two submissions, you will notice some other differences in the hardware and the environment for the test.  The newer submission is a system with 2048GB of RAM and (126) 200GB SAS SSDs for database storage, with a 13.3TB initial database size, while the older submission is a system with 1024GB of RAM and (90) 200GB SAS SSDs for database storage, with a 11.6TB initial database size. As long as you have sufficient I/O capacity to drive the TPC-E workload, the TPC-E score is usually limited by processor performance, so I don’t really think that the RAM and I/O differences are that significant here.

What do you think about this?  I would love to hear your opinions and comments!

The post Two New TPC-E Submissions for SQL Server 2012 appeared first on Glenn Berry.