Hindsight and Foresight

How do you feel about 3DMark05 as a project?  Do you think it achieved all of its targeted goals to within reasonable parameters or are there aspects about it that, in hindsight, you would have addressed differently?

I think 3DMark05 was (and still is) a great product! It introduced a lot of new approaches and techniques, and also worked as a showcase what games will look like when developers will be able to use SM2.0 to the same extent as we were. 3DMark05's goal was to benchmark SM2.0 hardware, and I believe it did a great job at that. Of course there are things that could have been done better, but then again, we are very happy with how 3DMark05 turned out, so there's no use to start speculating “what if...”. We know that there are some shortcomings in 3DMark05, and they have been addressed in the next 3DMark.

3DMark05 scaled to being CPU-limited with a 7800 GTX in SLI mode - did you expect that to happen so early or was this part of the road plan that has the next 3DMark appearing when CPU-limit cases become more frequent?

In all honesty, no we didn't. We know that the industry is moving at such a pace that it is very difficult for anyone to keep up, but we didn't expect 3DMark05 to become somewhat CPU-limited so quickly. It is worth to note that this is only the case with high-end dual graphics card systems. The number of users with 2 high-end graphics cards in one system is very low, but it is still a user group which is important to us. This is an issue we have focused on for the next 3DMark (one of the shortcomings in 3DMark05 I mentioned earlier), so we have been working very hard to make the graphics tests as GPU/VPU bound as possible. I know that this bit of news is something some users are not happy with since they want all tweaks (CPU, memory, etc.) to affect the 3DMark score, but that's also something we have taken in count with the next 3DMark. Like I said earlier, the next 3DMark will be a bit different.

Quite a lot of comments were made about the perspective shadow mapping techniques used in 3DMark05, with many of them concerning the approach to having an even testing field for the benchmark.  Looking back, do you feel that it was still the right decision to make or would you choose a different route if you had to do it again today?

We stand behind every decision we did in 3DMark05, and wouldn't change them even if we could. We had strong and valid arguments why we made 3DMark05 the way it is, and those arguments haven't changed. We strive to do what game developers will do in games, and for the techniques we used in 3DMark05, we believe we made the correct choices.

The next 3DMark will of course sport new features and improved techniques (such as a new dynamic shadow technique) but the reason is not that we think that 3DMark05 had flaws; we just want to get the visual fidelity to new heights, and we are pretty sure that games will start to support the same ideas as we use in the next 3DMark. They seem to work extremely well in all tested scenarios, which mean they should work fine in games too.

3DMark05 uses LiSPSM (Light Space Perspective Shadow Maps) and we are now seeing games getting released using similar techniques and approaches. With the next 3DMark, we will take the use of real-time dynamic soft shadows to a new level. I don’t want to reveal too much, but I have witnessed quite a lot of people being more than impressed when they have seen what we have been able to do with the shadows in the next 3DMark. I want to stress that we will continue to push forward the use of dynamic soft shadows for every pixel in the scene. In 3DMark05 I think we did a nice job, but in the next 3DMark we will take the shadows to a totally new level in terms of quality and usability. As games start to have more destructible and dynamic objects than ever before (thanks to ultra fast graphics, CPU’s and feasible physics), the approach we have in the next 3DMark is very feasible and suitable for games. It simply works. I wouldn’t be surprised to see games shipping in 2006 using similar techniques/approaches as we do in the next 3DMark.

But new shadows are not the only thing we have been working hard with. We have also been looking into bringing something completely new to the table. Improving and extending existing techniques is one thing, but creating totally new ones is a whole lot more difficult.

There’s a bagful of new effects and techniques we have been working on for the next 3DMark, but I don’t want to hand out all the treats we made. J You’ll see it all when the product ships.