Test Setup And Methods

So, have you heard about those latest and greatest hex-core CPUs running at more than 3 GHz?  The ones that every self-respecting hack around the globe has in his testing rig?  So have we, but we're not self-respecting so we're not using them.  In fact, as you'll notice below, our testing rig is quite pedestrian – like other things we mentioned upstream, it too is more than enough for the task ahead.



Now, the other big thing for us - outside of getting to play with a new and exciting architecture - is being able to use the new testing suite we've put together for the first time. You know we've been talking about our testing tool revamp for what seems like forever.  We've kept chipping away at it and finally we put together a completely new piece of software for testing modern graphics hardware, and one that we plan to update and extend going forward.

The testing suite is a ground-up design based on DirectX 11 (so much so that we've yet to implement the inferior feature levels, even for the cases where that would be possible), modular, and more extensive in its nature than anything we've used in the past -- definitely not exhaustive though, at least not yet.  It's also written in C++ and being DX11, it's Windows-only for the time being.

All in all, we daresay it's somewhat unique in the particular landscape we're placed on, at least for the time being -- so unique that we ran into a couple of driver rough edges from both camps, as we were beating the driver and API up, relying on as-yet unused features like dynamic shader linkage.  The plan is to release it for public consumption at some point in the future, once we've tidied up some rough edges we don't expect end-users to have to deal with.

Having exhausted geek pride, we feel compelled to remind you that it's possible that, in spite of our best efforts, and the reviews we've put them through, some tests miss their intended target (or get optimised in intricate, unintended ways by drivers). We're ready to accept that, and fix whatever there is to fix. In fact, we're looking forward to trading blows with IHVs over what does what, and we expect both ATI and NVIDIA to have some feedback and input.

You should also keep in mind that the tests are synthetic in nature and quite targeted (we choose optimal strides for our accesses, ensure atomics are uncontended etc.), so interpret them in that key, as a means to unveil how an architecture performs under the best conditions in specific cases, rather than as a decoder ring for in-game performance -- it's not intended to be that, and that's a hairy problem we'll tackle differently, soon.

What you should take home is that, like always, there's no absolute truth and what we state or show is in line with our best current knowledge, and our reasoning based on data we have available at this time, both of which may be incomplete or just plain wrong -- we always try as hard as possible to lower the probability of those two evils manifesting, but it will always be there.

And now, after hundreds of chicken years of writing and countless cups of coffee, not to mention some 2000-ish words of wordy innuendo, it's time to go out on a date with Slimer!