The answer : Compression… VTC!

If something is too big and uses up too much memory then there is always a way out called compression. NVIDIA probably did the same math as I did and realized that implementing simple uncompressed 3D textures is a big no-no. Doing this would be like implementing a useless feature, unless memory prices drop like a rock. Now, the issue with compression is, as always, how do you do it ? 

Texture Compression is not new, we already have various formats for compressing 2D textures. We have S3TC (DXTC), FXT1 and VQ (Vector Quantisation). The first two techniques have compression ratio’s of up to 8:1 while VQs compression ratio is resolution dependent (the higher the resolution the higher the compression, goes up to 25:1).

There are some rules we need to keep in mind when selecting a compression technique for textures. The main issue is that this we need real-time access to these textures, so any technique we use must have a very, very fast de-compression system. This means that complex, yet very powerful, systems like Wavelets or JPEG are out of the question. They compress with a fantastic ratio, but the decompression is slow. Another issue is that texels must be individually available, so we can’t use a technique that requires the decompression for the whole or a larger part of the texture. So we need direct fast access to very specific texels using our technique.

The existing techniques are based on 2 principles, well actually one principle known as a look-up table. But there are 2 different approaches to it. The first approach used by S3TC and the more advanced (and visually better) FXT1 spilts the texture in blocks, 4 by 4 texel blocks. For each of these block a small lookup table is made, this lookup table contains 2 color value (so an RGB(A) color). The individual texel colors are defined by a mix/blend of these 2 base colors- which blend is determined by some parameters. More details about this technique can be found here.

VQ works differently but is also lookup table bound. Instead of creating small lookup tables for small blocks in the image, a global lookup table is used. Actually the best way to explain this is as follows. Imagine the texture map as big puzzle existing of small 2 texel by 2 texel ( or even 3x3) puzzle-pieces. Using clever maths you can find puzzle pieces that are almost identical, so instead of storing all the puzzle pieces, some can be used several times. So the basic idea is to create a pool of puzzle pieces (lets say 256 pieces) that are used to form the final texture. The compression lies in the re-use of the same puzzle piece at various locations. This principle is used a lot in compression of still images and even video. Videologic's PowerVR chip supports this technique and its used with great success in the Sega Dreamcast console. You can read more about VQ here.

Now, both these principles can be extended to 3 dimensional textures. The S3TC, FXT1 principle of small zones with a small local lookup table can be extended to small cubic zones inside the 3D texture. Each small cube gets some base colors and the lookup value give each cubic texel a color based on a blend of the base colors. The problem with this is that you need to find some setup which has a good byte size. S3TC and FXT1 use block with a total size of 64-bits (or even 128bits), a cubic extension should maintain this size or move to 128-bit/256bit. These values are needed to make efficient use of bandwidth (read in 128-bit or a multiple of that at every clock tick). Finding such a setup should not be too difficult.

VQ can also be extended. Instead of 2D puzzle pieces we can use small cubic volumetric puzzle pieces. The same idea applies: re-use of the puzzle “blocks” delivers the compression.

At this moment it is not known how the VTC compression works. Beyond 3D will try to find out how it works and a detailed analysis of the quality of this technique will be provided as soon as possible. 

One thing to keep in mind is that a massive compression ratio is needed to make 3D textures usable. The extra dimension does introduce more relations between neighboring pixels, these added links can result in better compression. Feel free to speculate about techniques and ratio’s in our forum.

Another solution to the problem is the use of procedural 3D textures. Personally, I prefer this solution since procedural textures hardly use any memory and have an almost infinite resolution. Sounds cool, no? So what is it...

Procedural textures are a bit like mathematical textures. Instead of providing a true map saying this pixel has this color, you provide a mathematical description of how a texture looks. For example, a checkered texture can be generated very easily using mathematics (think about it). Now, procedural textures on their own are worth a full article (and we plan to do this soon) so I won’t go into details. Suffice  to say, almost all natural textures can be generated using mathematics, even the stripe patterns of a zebra or tiger can be generated using a mathematical function. The infinite resolution is a result of the mathematical basis of this system (e.g. a sine has infinite accuracy too!). The low storage comes from the fact that you only need to store the small program that mathematically describes the texture (this is much smaller than true pixel stored textures). This technique is used successfully in 3D ray tracers. Actually, procedural textures is what make movies like Toy Story 2 look so cool.

The final and easy way out is, of course, to revert to using low resolution 3D textures only. The big question with this solution is: "Is it visually acceptable?".