Instead, they maintain the same versions of software on Windows as well until enough time has passed and enough updates have piled up that they do finally re-test everything
I’m not that involved with their testing procedure but doesn’t that put newer cards at a disadvantage?
They lack any sort of driver optimization if the release drivers are never installed.
That’s a good point. I went back to the video to rewatch it, and turns out I totally missed where they said they only freeze things during a testing phase, then unfreeze it after they’re done and allow updates to commence as normal.
They mentioned that due to Linux receiving more frequent updates often with meaningful performance improvements, they’ll have to throw away older data and re-test more often on Linux, as Windows doesn’t really change much in performance between updates. So I would guess that they would use release drivers with new cards, and likely would only re-test their entire suite if the release driver also gave a big performance boost on older cards.
I’m not that involved with their testing procedure but doesn’t that put newer cards at a disadvantage?
They lack any sort of driver optimization if the release drivers are never installed.
That’s a good point. I went back to the video to rewatch it, and turns out I totally missed where they said they only freeze things during a testing phase, then unfreeze it after they’re done and allow updates to commence as normal.
They mentioned that due to Linux receiving more frequent updates often with meaningful performance improvements, they’ll have to throw away older data and re-test more often on Linux, as Windows doesn’t really change much in performance between updates. So I would guess that they would use release drivers with new cards, and likely would only re-test their entire suite if the release driver also gave a big performance boost on older cards.