Futuremark 3DMark Vantage Review
Reviewed by: ccokeman
Reviewed on: April 28, 2008
When the subject of video card performance comes up, the questions that are asked (here lately) are how does it do in 3DMarkXX or how does it do in Crysis? The reasoning is that the Futuremark series gives repeatable results for a valid comparison across platforms and video card manufacturers' products. In the past seven years, Futuremark has released a benchmark roughly every two years, starting in 2001. Of course, we all have used the older benchmarks and have seen scores blossom to unheard of levels as computing technology has improved and scaled much higher. On the other side of the fence, you have the fact that each successive benchmark initially brought the latest video card technology (and later processor tech) to its knees. Guess what things have not changed? The newest from FutureMark is 3DMark Vantage. This benchmark is specifically designed for use with Windows Vista and DirectX 10. So what's new with this version of 3Dmark, you ask, besides the obvious? Pretty much everything. There are two new game tests, Jane Nash and New Calico. There are two all new CPU tests that have been designed around a new "Physics and Artificial Intelligence-related computation." CPU test two offers support for physics related hardware. Four preset performance levels that can give a wide array of official scores instead of the one official setting used in the past. These are just some of the new items. Follow along as I take a look at the latest from FutureMark.
First off, you have to have the hardware to even run 3DMark Vantage. The minimum requirements are pretty steep if you have last year's technology, but are about on par with what the current crop of gamers and enthusiasts should have.
The minimum hardware and software requirements and recommendations are:
- Processor: Requirement: SSE2 support, Recommendation: A dual-core processor with performance equivalent to Intel Core 2 Duo E6600, AMD Athlon X2 6000+, or higher
- Graphics Card: Requirement: Fully DirectX10-compliant graphics hardware
- Display Device: Requirement: Capable of 1280x1024 resolution, Recommendation: Capable of 1920x1200 resolution, Needed to run all the presets
- System Memory: Requirement: Windows Vista minimum requirement, Recommendation: 2 GB or more
- Hard Disk: Requirement: 1 GB of free hard disk space
- Operating System: Requirement: Windows Vista with Service Pack 1
Now that you know a little about the program and have seen the hardware requirements, it's time to look a little closer at the application. First up, the Main Configuration tab. In this section, you can choose the GPU and CPU tests that the game runs, as well as the preset performance option using the drop down menu.
The Options menu is where the detailed configuration of the tests can take place. The Test Options tab is where you you can choose the resolution, performance preset, anistrophic filtering, texture quality and more. Each item with a drop down box is adjustable to configure a custom test or to use the preset option box. Each of the presets are shown below, as well as a custom test configuration.
Additional items under the options tab are the Game Test, CPU Test and Feature Test tabs. Under these sections you can turn on and off the tests under each tab for a custom configuration. Each Game, CPU, and Feature test includes a description of what the test is designed to do or test.
The System Info tab provides a detailed look at the devices and drivers installed in the current system.
Scoring is handled a bit differently in 3DMark Vantage. There is the CPU score and GPU score, as well as several sub-scores. As the performance level of the preset goes up, the graphics score has increased weighting to that score based on the preset. Therefore, each preset can only be compared to similar scores (i.e. Entry scores can only be compared to Entry scores, Performance to Performance and so on). Custom tests are comparable only across a test setup that meets the exact same criteria. Each score is identified by a letter in front of the score. This letter denotes the type of test run, E for Entry, P for Performance, H for High and X for Extreme.
Now that we have an understanding of what kind of setup is involved and how to navigate through the menus, we can look at the tests to see some specifics on how they are being set up.
There are two all new game tests in 3DMark Vantage, Jane Nash (Game Test 1) and New Calico (Game Test 2). Each uses a different testing methodology to generate the score. In Game Test 1, Jane gets caught in the wrong place and successfully escapes via a boat sponsored by, you guessed it, Sapphire!
Graphics Test 1 Jane Nash:
"The Jane Nash test scene represents a large indoor game scene with complex character rigs, physical GPU simulations, multiple dynamic lights, and complex surface lighting models. It uses several hierarchical rendering steps, including for water reflection and refraction, and physics simulation collision map rendering."
This list of features applies only to this scene.
- Lots of static objects
- Lots of complex dynamic skinned objects
- Cascaded shadow maps using PCF filtering
- Very few instanced objects
- No ray-marching (volumetric) effects
- Cloth simulation
- Anisotropic materials (math-heavy)
- Hierarchical rendering passes to render water reflection and refraction
Here are several screenshots following through the benchmark. These are all on the Extreme preset.
Graphics Test 2 New Calico:
"The New Calico test scene represents a vast space scene with lots of moving but rigid objects and special content like a huge planet and a dense asteroid belt." The following feature list applies only to the New Calico test.
- Almost entirely consists of moving objects
- No skinned objects
- Variance shadow mapping shadows
- Lots of instanced objects
- Local and global ray-tracing effects (Parallax Occlusion Mapping, True Impostors and volumetric fog)
Again, we follow along as the world below is attacked from far above.
Let's move on to the CPU and Feature tests.
CPU Test 1 AI:
The CPU tests have both been re-designed from the ground up. CPU Test 1 consists of mapping routes or flight plans through the maze of obstacles and gates to plan the most efficient routes. These artificial intelligence calculations are run in parallel using multiple CPUs and threads to do more calculations.
CPU Test 2 Physics:
"The Physics Test features a heavy workload of future generation game physics computations. The scene is set at an air race, but with an unfortunately dangerous configuration of gates. Planes trailing smoke collide with various cloth and soft-body obstacles, each other, and the ground. The smoke spreads, and reacts to the planes passing through it."
Feature 1 Test Texture Fill:
"This test draws frames by filling the screen rectangle with values read from a tiny texture using multiple texture coordinates. The texture coordinates are rotated and scaled between each frame."
Feature Test 2 Color Fill:
"This test draws frames by drawing a rectangle across the screen multiple times. The color and alpha channels of each corner of the rectangle is animated. The pixel shader is pass-through. The interpolated color is written directly to the render target using alpha blending. The render target is in 16-bit floating-point format, currently the most relevant format for HDR rendering output."
Feature Test 3 Parallax Occlusion Mapping (Complex Pixel Shader):
"This test draws frames by rendering a single rectangle (two triangles) on screen, seen from an animated camera position. The pixel shader uses the Parallax Occlusion Mapping technique to simulate complex geometry under the surface of the rectangle. Heavy ray-tracing operations against a huge depth-map determine the actual intersection of the view ray with the geometry. Further ray-tracing determines visibility of that point from multiple animated light sources. Finally, the surface is shaded using the relatively complex Strauss shading model. This test represents a very complex, heavy pixel shader, containing massive amounts of texture reads (ray-tracing) and dynamic flow-control (ray-tracing, looping over multiple lights), as well as traditional lighting calculations (Strauss). All the geometry on screen is rendered on just two triangles, and simulated entirely in the pixel shader."
Feature Test 4 GPU Cloth:
"This test features physical simulation of cloth on the GPU. The simulation is performed as a vertex simulation using a combination of vertex shader and geometry shader stages, with several simulation passes needed for each simulation step. Stream out is used to cycle the cloth vertices from one simulation pass to the next. This test stresses the vertex shading, geometry shading and stream out features of the hardware."
Feature Test 5 GPU Particles:
"This test features physically simulated particle effects on the GPU. The simulation is performed as a vertex simulation, with each vertex representing a single particle. Stream out is used to cycle the particle vertices from one simulation pass to the next. There are hundreds of thousands of particles in the test, all individually simulated, and colliding with a height map. The particles are rendered by expanding each vertex to a full rectangle in the geometry shader. The test stresses the vertex shading, stream out."
Feature test 6 Perlin Noise (Math-heavy Pixel Shader):
"This test features multiple octaves of Perlin noise evaluated in the pixel shader. Each color channel has its own noise function for added computational load. The Perlin noise function is a standard building block of procedural texturing approaches, and is very math-intensive to compute in a pixel-shader. This feature test emphasizes the arithmetic computing power of the graphics hardware."
To test 3DMark Vantage out, I will run the test setup through each preset to generate a score for comparison. Per our normal testing, all settings will be left at factory defaults to minimize the variables. Driver version 174.74 was used throughout the testing.
- Processor: Intel Q6600 Core 2 Quad 266x9
- Motherboard: Abit IX38 Quad Gt
- Memory: Mushkin XP2 8000 Redline 2 x 2GB 5-5-5-12
- Video Card(s): Asus EN8800GT TOP
- Power Supply: Mushkin 800watt Modular Power supply
- Hard Drive: 1 x Seagate 320GB SATA
- Opticals: NEC DV5700
- O/S: Windows Vista Ultimate Edition
Comparison Video Card:
As expected, the 9800GTX finished ahead of the 8800GT, but not by much. The margin between the three cards is not that large and came as a surprise. The 8800GT and 9600GT are both factory overclocked cards and that could have had an impact, but the cards and scores are the end result of the testing, surprising as it may be.
There are three versions of 3DMark Vantage available at this time, Vantage Basic, Advanced Edition and the Professional version. This covers the spectrum from the home user to the professional. Each version comes with a price tag and a defined set of features. Yes, you read that right. It will cost a little bit to play this "game." There is a free one time use version, but further use will require an activation. Let's look at the pricing structure and what you get for those hard earned dollars.
3DMark Vantage Basic Edition:
- This version will cost you $6.95 (download price)
- Unlimited number of test runs using a single preset setting
- Network connection required to view results
- Licensed for non-commercial and personal use only
3DMark Vantage Advanced Edition:
- This version will run $19.95 (download price)
- Unlimited testing, access to all presets and custom settings
- Network connection required to view results
- Licensed for non-commercial and personal use only
3DMark Vantage Professional:
- This one will set you back a few pennies at $495.00/seat (download or CD-ROM)
- All Advanced Version features
- Technical support
- View results without network connection
- Benchmark automation with command line scripting
- Licensed for full commercial use
Futuremark has done it again. The company has effectively come out with a benchmark that makes some of the latest hardware feel oh so inadequate. The upside to this is that it is upgrade time, and this is just another reason to do so, if for nothing more than bragging rights. This time, I think Futuremark has come up with a better mousetrap, to coin a phrase. The preset feature allows all levels of players (enthusiasts, benchmarking gods) to step up and take a swing for the fences to see how well their systems perform. There is the entry level setting that should allow any of the $150 and up cards to run at the lowest settings while having the higher end presets to tax the big boys up top. This preset tiered structure provides four levels to get official scores rather than just one level as there has been in the past. This makes it appealing to not just the hardcore benching crowd since all scores count, but to the wider base as well. Is that enough to actually have a wider base pay for that privilege? Only time will tell if that is the case. With pricing for the non professional maxing out at $19.95, the $6.95 option may be the best for the non hardcore group as the professional pricing is just beyond the cost range for anyone but that level of use.
If you would like to try out 3dmark Vantage you can do so here.
- Provides a tough test
- Interesting benchmark
- Preset options
- Works with physics cards
- One time use for free version