Feature Tests

Given the nature of the testing Beyond3D carries out the 'Feature Tests' in 3DMark can often be more important to us than the actual game tests, as they should single out individual processing elements. However, in this instance I can't help but feel Futuremark missed the boat slightly.

Initially I had been concerned over the Vertex Shader test. Using the Troll scene it is still dependant on Pixel Shaders to render the scene, which could leave the possibility of the scene being fill-rate bound, rather than Vertex bound, which is what we would want to see from a Vertex Shader Test. From our testing we can see that all the plots on the fill-rate graph are relatively straight, indicating that they are all Vertex bound, with the possible exception being the 9500 at 1600x1200, as it has an unusually high Vertex throughput in comparison to its pixel fill-rate.

The PS2.0 Pixel Shader Test possibly isn't quite as useful as a pure Pixel Shader test as it should be. If we take a look at this fill-rate graph, taken from our Radeon 9500 PRO review using the 3DMark2001SE 'Pixel Shader' test you'll note that at the lower resolutions the it scales up quite rapidly, with most boards plateauing quite a lot by 1024x768, which is clearly indicating that these boards are fill-rate bound by the shader calculation -- this is what we would want from a pure Pixel Shader test. With the new version's Pixel Shader test the fill-rate graph indicates that all the boards, including the 9500, are much less fill-rate limited as they are still scaling up through all the resolutions, indicating that in parts it is either bound by the system or the geometry rates. Although the test is good to see some pure DX9 PS2.0 / Procedural Texturing, the fact that it isn't purely limited by the Pixel Shading limits what its able to tell us about a boards Pixel Shading performance alone.

As for the Ragdoll test, I am still scratching my head as to whether this is particularly useful or not to me.

Beyond3D's position over the past few years on versions of 3DMark, while perhaps not rigidly adhered to, is generally well known to be that we do not advocate the use of the final '3DMark' score in reviews, but that does not prevent us from using the benchmark.

As a 3D technology site we like to test various aspects of the hardware we are reviewing, going beyond just gaming benchmarks. 3D cards are always advertised with plenty of new features to entice potential users with the promise of what's to come; however, with game development times such as they are it's virtually impossible to actually test the features that are being sold through gaming titles alone, leaving consumers none the wiser as to whether these features are worthwhile. This, of course, is where specific tests and synthetic benchmarks step in. At this point in time 3DMark03 is one of the most strenuous DX8 tests and has some of the only true DX9 utilisations to date which really makes it a requirement right now.

So Beyond3D's policy will not change for 3DMark03. We will not advocate the use of the final score, but we will be using specific tests as we see fit. It's quite likely that, for the short term at least, we may continue to use specific tests from 3DMark2001SE alongside tests from 3DMark03.

To avoid any queries, the fact that I am personally credited in the 3DMark03 titles has nothing to do with Beyond3D's opinion of the product. That is there as Beyond3D beta tested the product prior to release and we provided some similar feedback to what can be found above.

• Please feel free to comment on this article here.