Constructing a database benchmark (average load)

Our first new benchmark was custom written in .NET, using ADO.NET to connect to the database. The AnandTech Forums database, which is over 14GB in size at the time of the benchmark, was used as the source database. We'll dub this benchmark tool "SQL Loader" for the purposes of discussing what it does.

SQL Loader allows us to specify the following: an XML based workload file for the test, how long the test should run, and how many threads it should use with which to load the database. The XML workload file contains queries that we want executed against the database, and some random ID generator queries that populate a memory resident array with ID's to be used in conjunction with our workload queries. The purpose of using random ID's is to keep the test as real-world as possible by selecting random data. This test should give us a lot of room for growth, as the workload can be whatever we want in future tests.

Example workload:

< workload>

< !--- A SAMPLE WORKLOAD QUERY THAT RETURNS ALL THE FIELDS FROM THE PRIVATEMESSAGES TABLE RANDOMLY --->

<query>

<code>select * from privatemessages where imessageid = @pmessageid</code>

<type>read</type>

<randkey>pmessageid</randkey>

</query>

<!--- RANDOM ID GENERATOR FOR SELECTING RANDOM PRIVATE MESSAGES --->

<randomid>

<rcode>select imessageid,newid() as pmsgid from privatemessages order by pmsgid</rcode>

<name>pmessageid</name>

</randomid>

< /workload>


A screenshot of the SQL Loader

Test Information

The workload used for the test was based on every day use of the Forums, which are running FuseTalk. We took the most popular queries and put them in the workload. Functions, such as reading threads and messages, getting user information, inserting threads and messages, and reading private messages, were in the spotlight. Each iteration of the test was run for 10 minutes, with the first being from a cold boot. SQL was restarted in between each test that was run consecutively.

The importance of this test is that it is as real world as you can get; for us, the performance in this test directly influences what upgrade decisions we make for our own IT infrastructure.

Hyper Threading and The Tests AnandTech Forums Database Test Results
Comments Locked

58 Comments

View All Comments

  • Jason Clark - Tuesday, March 2, 2004 - link

    Pumpkin, maybe next time there is 4GB of DDR400 ECC laying around we'll give it a run. It wasn't readily available to us when these tests were run. And in all honesty I doubt it would change any numbers by anything more than a percent. The bigger picture would remain the same.

    Cheers.
  • Blackbrrd - Tuesday, March 2, 2004 - link

    It would have been real interesting to see a comparison with the Athlon MP processor platform...
  • Pumpkinierre - Tuesday, March 2, 2004 - link

    #24 Opterons are now specified for DDR 400. So you should have tested them (and all the cpu's) at their maximum spec.. It might have made the difference in the 2way test where the opterons were close but not quite up to the Xeons.
  • Jason Clark - Tuesday, March 2, 2004 - link

    William, all we had on hand were the Xeons tested... we had requested 1MB parts but they didnt make it.

    hirschma
    Tyan S4880 is one that I know of, and the system we used is a reference amd system ("Quartet"). Appro makes a server based on it I think and a few other companies.
  • Jason Clark - Tuesday, March 2, 2004 - link

    Ski.

    Slow DDR333? It isn't slow, and it was what we had and it remained the same across platforms. These are servers not watercooled tweaked out systems running DDR500 :) This is a CPU test not a memory round up guys.
  • Jason Clark - Tuesday, March 2, 2004 - link

    zomg555 broaden your thinking to an IT director who is about to spend 50K on a server that has to last him X years. Do you spend it on a cpu capable of 32 bit only or a cpu capable of 64 bit that is also faster in 32 bit?. Then, look at the cost of each platform as per our cost graphs.



  • William Yu - Tuesday, March 2, 2004 - link

    If you already have 4P Xeon servers, there's no point in switching. But for a new 4P server, the difference isn't just 10%. It's $8800 based on the list price difference between the Xeon MP 4MB 3.2 and the Opteron 848. If it's somebody else's money, what the hell, buy the Xeon. If you have a direct stake in the financial status of the company...
  • zomg555 - Tuesday, March 2, 2004 - link

    I was a little surprised by how close this test was. 10% more performance isn't enough to get most shops to switch from Intel Xeon to a new platform.
    It would have been interesting to see some tests with more than 4GB of memory, though. In these tests, the Xeons weren't paying a PAE bounce penalty, which would be sapping a lot of performance in servers with more physical memory.
  • hirschma - Tuesday, March 2, 2004 - link

    What hardware was used on these tests, just out of curiousity? Where can one get 4-way boards for Operton? I'd sure love to build a monster like that.

  • William Yu - Tuesday, March 2, 2004 - link

    Can you run these tests on the "crippled" Xeons with no L3 and 1MB L3? That would give a good picture for those who currently have Xeons and are exploring upgrades to their servers. (I.e., popin replacement for $$$ versus wholesale replacement.)

Log in

Don't have an account? Sign up now