Welcome Stranger to OCC!Login | Register

Serious Statistics Review


DirectX 11 - Hatsheput:

If you took a brief look down before reading this (and you can do that right now if you want) you will notice that there are four course graphs being shown, instead of three. There is a reason why I did an extra run on Hatsheput with DirectX 11, but I am going to leave the explanation until it appears in what I am already presenting to you. This decision, like including the fourth course graph, is to reinforce the point that what I discovered is actually not an issue. It is a rather interesting quirk my testing exposed between these three APIs, or at least their implementation in Serious Sam Fusion 2017.







Averages are next and I will not be including those for the data from the fourth run because I had to change a setting when I did it, making it not-immediately comparable to the other data in this article.

Averages Frame Time (ms) Framerate (FPS)
Run 1 7.164686 139.5687
Run 2 7.158765 139.6892
Run 3 7.045851 141.9269

If you did not already see it in the course graphs you can see here, and in the percentile table next, that DirectX 11 is outperforming OpenGL.

Percentile 0.1% 1% 99% 99.9%
Run 1 5.211 ms (191.92 FPS) 5.509 ms (181.52 FPS) 9.615 ms (104.00 FPS) 12.326 ms (81.13 FPS)
Run 2 5.259 ms (190.16 FPS) 5.505 ms (181.65 FPS) 9.544 ms (104.78 FPS) 11.985 ms (83.44 FPS)
Run 3 4.992 ms (200.31 FPS) 5.301 ms (188.64 FPS) 9.414 ms (106.23 FPS) 11.387 ms (87.82 FPS)

Comparing these numbers to those I got for Vulkan, the newer API is still in the lead, but DirectX 11 is not far behind. Now to see where 60 FPS and other framerates occur in the data.

  16.667 ms / 60 FPS 20.000 ms / FPS 33.333 ms / 30 FPS
Run 1 0.011946289% 0.007167774% 0.002389258%
Run 2 0.00477270% 0.00238635% --
Run 3 0.016441574% 0.009395185% 0.007046389%

Now for the QQ plots that so neatly show percentiles.


Time for the difference data, where we can see the trends were nice and smooth for these runs.

Diff Percentiles 0.1% 1% 99% 99.9%
Run 1 -4.237772 ms -1.572480 ms 1.597480 ms 4.145284 ms
Run 2 -4.038194 ms -1.502940 ms 1.544970 ms 4.234947 ms
Run 3 -3.655507 ms -1.505270 ms 1.539540 ms 3.582697 ms

With this we can get to the frequency plots.





By process of elimination the quirk I encountered must appear with the display times, and indeed we see something different was happening with this data (making me very glad I decided to look at these numbers).


These look nothing like the display time graphs for Vulkan and OpenGL, and if you know why, good for you. Before getting to that, here is the display time graph for that fourth run.


A close look at the first three graphs tells us the frames were being synchronized to my monitor’s refresh rate, and if you remember what I wrote way back on the third page you also see that frames were being dropped, a lot of them. Now v-sync was off, so they should not be synchronized with my monitor’s refresh rate, but Microsoft made a decision concerning Windows 10, and that is at play here. I am not sure why Vulkan and OpenGL were unaffected by this decision, but DirectX 11 running in Borderless Window/Fullscreen is being subjected to the Desktop Window Manager’s forced double buffering. The setting I changed for the fourth run was to turn on Exclusive Fullscreen, which means DWM is bypassed, so frames can be sent to the display as fast as they are completed, just as the fourth display time graph shows.

After discovering this I decided to do some v-sync tests, the results of which will be after I finish with the DirectX 11 data, but I did not alter my process for the Dunes or Thebes – Karnak runs. I still did these in Borderless Window/Fullscreen to maintain parity with the Vulkan and OpenGL tests, and the fact that the performance numbers were not affected means that the data is still viable.

  1. Serious Statistics - Introduction
  2. Serious Statistics - Software/Tools
  3. Serious Statistics - Testing and Processing
  4. Serious Statistics - Vulkan - Hatsheput
  5. Serious Statistics - Vulkan - Dunes
  6. Serious Statistics - Vulkan - Thebes - Karnak
  7. Serious Statistics - OpenGL - Hatsheput
  8. Serious Statistics - OpenGL - Dunes
  9. Serious Statistics - OpenGL - Thebes - Karnak
  10. Serious Statistics - DirectX 11 - Hatsheput
  11. Serious Statistics - DirectX 11 - Dunes
  12. Serious Statistics - DirectX 11 - Thebes - Karnak
  13. Serious Statistics - V-Sync Tests
  14. Serious Statistics - Conclusion
Related Products
Random Pic
© 2001-2018 Overclockers Club ® Privacy Policy
Elapsed: 0.1654551029   (xlweb1)