ABSTRACT
In Chrome 28 a bad WebGL bug-crbug.com/259994, which intermittently broke Google Maps and many other WebGL applications-shipped to Stable and, embarrassingly, the issue was reported to our team primarily via Twitter. The bug slipped through automated testing for two principal reasons. First, the majority of the automated testing in the opensource Chromium project, and the Chrome browser built on top of it, occurred on virtual machines, which did not exercise the GPU-accelerated rendering path that is taken on most end users’ systems. The few machines with physical hardware and real GPUs had not received enough attention from the overall team and were not reliable enough to detect intermittent failures. Second, there were not enough “try servers”—the banks of machines that developers use to certify their code changes during development and checkin timerunning on physical hardware with actual GPUs. The effect was that it was impossible to detect potential breakage to Chromium’s graphics stack ahead of time, even though there were dozens of developers working directly on this code.
