Welcome Stranger to OCC!Login | Register

Serious Statistics Review


Vulkan - Hatsheput:

As I suspect this is the API most people will be interested in, I will cover it first. Vulkan is a low-level API being developed by the Khronos Group, the same group behind OpenGL. It borrows a lot of code from the Mantle API AMD created that seemed to set the industry on the path to low-level APIs. Unlike DirectX 12, the dominant low-level API today, Vulkan is cross platform so we can see it being used on more than Windows 10 machines. As low-level APIs offer reduced overhead and greater access to a system's hardware, they can enable significant performance gains, especially on machines where the CPU was a limiting factor and if it is a multicore processor.

First up, the course graphs:


As you can see, we had some outliers that expanded the y-axis out a fair bit on the first and third run, while the second kept within the minimum scale I set. Still, most of the data for all three runs kept around the 8.333 ms mark, which corresponds to 120 FPS.

The averages for these runs:

Average Frame Time (ms) Framerate (FPS)
Run 1 5.986 167.05
Run 2 5.945 168.19
Run 3 5.963 167.69

Averages are nice, convenient pieces of information that are easily interpreted, but do not do the best job describing data, so here are the percentiles:

Percentiles 0.1% 1% 99% 99.9%
Run 1 4.183 ms (239.05 FPS) 4.486 ms (222.91 FPS) 8.025 ms (124.62 FPS) 9.665 ms (103.46 FPS)
Run 2 4.209 ms (237.57 FPS) 4.477 ms (223.34 FPS) 7.884 ms (126.85 FPS) 8.930 ms (111.98 FPS)
Run 3 4.229 ms (236.45 FPS) 4.449 ms (224.77 FPS) 7.904 ms (126.52 FPS) 9.089 ms (110.03 FPS)

A quick scan of these data shows us the game kept between 124 FPS and 222 FPS most of the time.

Now here are the percentiles that match up with 16.667ms (60 FPS), so you can see just how often the performance dropped below that point:

  16.667 ms / 60 FPS
Run 1 0.011972224%
Run 2 0.001982278%
Run 3 0.005964333%


I could give you the percentiles relating to the other points I mentioned before, but considering we are already at fractions of a percent, I see little reason to.

With these points covered, time to take a look at the QQ plots:


Before getting to the frame time, frame rate, and display time graphs I want to share the difference data. These are percentiles for the difference between consecutive frame times:

Diff Percentiles 0.1% 1% 99% 99.9%
Run 1 -2.405036 ms -1.112860 ms 1.120000 ms 2.485668 ms
Run 2 -1.935110 ms -1.054550 ms 1.071550 ms 1.969885 ms
Run 3 -2.004812 ms -1.090030 ms 1.095000 ms 2.067842 ms

This data can be interpreted to mean that if one frame in the first run had a frame time of X the next frame had a frame time of X – 1.112860 ms to X + 1.120000 ms 98% of the time. The smaller these values the better.

Now we can get to the frame time graphs:





These final graphs show a different record from the OCAT data; MsBetweenDisplayChange. As I mentioned earlier, I have changed the x-axis scale to count in ‘frames later,’ corresponding to my 60 Hz monitor.

With most of the frames being sent faster than my monitor’s refresh rate, there is great potential for screen tearing, as the frame is updated before the GPU finishes sending it off.

  1. Serious Statistics - Introduction
  2. Serious Statistics - Software/Tools
  3. Serious Statistics - Testing and Processing
  4. Serious Statistics - Vulkan - Hatsheput
  5. Serious Statistics - Vulkan - Dunes
  6. Serious Statistics - Vulkan - Thebes - Karnak
  7. Serious Statistics - OpenGL - Hatsheput
  8. Serious Statistics - OpenGL - Dunes
  9. Serious Statistics - OpenGL - Thebes - Karnak
  10. Serious Statistics - DirectX 11 - Hatsheput
  11. Serious Statistics - DirectX 11 - Dunes
  12. Serious Statistics - DirectX 11 - Thebes - Karnak
  13. Serious Statistics - V-Sync Tests
  14. Serious Statistics - Conclusion
Related Products
Random Pic
© 2001-2018 Overclockers Club ® Privacy Policy
Elapsed: 0.1635591984   (xlweb1)