NVIDIA doesn’t trust DX12 benchmark to be accurate

Microsoft has been talking about DirectX 12 (DX12) for quite a while now, and with Windows 10 released it should bring "console level efficiency" to PC's. It's been suggested that DX12 will be 50% – 70% more efficient when it comes to rendering speed and CPU consumption than its predecessor DirectX 11.

According to DX12's first benchmark, it definitely improves AMD performance, but for some reason bogs down NVIDIA's top-of-the-line $999 Titan X and 780Ti, although findings weren't consistent across all NVIDIA GPUs or CPUs — they were for AMD.

DX12 was tested out in the Alpha build of Ashes of the Singularity, the game is based on developer Oxide Games’ own Nitrous Engine, which is designed to take advantage of lower level APIs like Mantle, DirectX 12 and Vulkan.

NVIDIA has responded to the benchmark saying that it cannot be trusted, the game is still in tests and has yet to be optimized. Furthermore, NVIDIA and Microsoft have been working very closely to achieve the best support for DX12, they've even powered the major public demos for it.

NVIDIA's Brian Burke responded to WCCF with the following statement, a statement that has been refuted by Oxide Games (to a certain extent):

This title is in an early Alpha stage according to the creator.  It’s hard to say what is going on with alpha software. It is still being finished and optimized. It still has bugs, such as the one that Oxide found where there is an issue on their side which negatively affects DX12 performance when MSAA is used. They are hoping to have a fix on their side shortly.

We think the game looks intriguing, but an alpha benchmark has limited usefulness.  It will tell you how your system runs a series of preselected scenes from the alpha version of Ashes of Singularity. We do not believe it is a good indicator of overall DirectX 12 gaming performance.

We’ve worked closely with Microsoft for years on DirectX 12 and have powered every major DirectX 12 public demo they have shown.  We have the utmost confidence in DX12,  our DX12 drivers and our architecture’s ability to perform in DX12.

When accurate DX12 metrics arrive, the story will be the same as it was for DX11.

Oxide games does not agree with NVIDIA, on their blog they posted that the benchmark is valid regardless of whatever stage their game is in and that the bit about MSAA in untrue:

Fundamentally, the MSAA path is essentially unchanged in DX11 and DX12. Any statement which says there is a bug in the application should be disregarded as inaccurate information.

It should not be considered that because the game is not yet publically out, it’s not a legitimate test. While there are still optimizations to be had, Ashes of the Singularity in its pre-beta stage is as – or more – optimized as most released games. What’s the point of optimizing code 6 months after a title is released, after all? Certainly, things will change a bit until release. But PC games with digital updates are always changing, we certainly won’t hold back from making big changes post launch if we feel it makes the game better!

To be quite frank, none of the statements really matter. The only thing that matters is DX12 is still new and there is some fine tuning needed on both sides. AMD however, can hang out and live in their dome of "everything works as planned."

It is absolutely not a surprise that DX12 is working well on AMD GPUs and CPUs, it's Microsoft developed and its the Xbox One's GPU… Of course they would make sure it works.