Ask people on the streets in Germany who they think is the most famous magician ever, and the name you would probably hear most is David Copperfield. Almost everyone knows him and is fascinated by his magic tricks. A specialty of Copperfield's is to make things disappear—a train, for example. Imagine an old train wagon from an Agatha Christie movie on the stage, a giant white bedsheet wrapped around it. Then, some flashing lights and effects, and when you refocus your attention—the train wagon is gone. Just disappeared!
Alexander Dobrindt is Germany's transport minister. But just like Copperfield, he has a talent for making things disappear—high carbon dioxide (CO2) emissions, in particular. Shortly after it became known that Volkswagen used illegal defeat devices in some of its diesel vehicles, in September 2015, Dobrindt’s Ministry for Transport (BMVI) commissioned a series of laboratory and on-road vehicle tests. In total, 56 Euro 5 and Euro 6 diesel vehicles were tested. The Ministry published the test results for nitrogen oxide (NOx) emissions on a Friday afternoon in April 2016, as I described in an earlier blog post.
Those 56 vehicles were not only tested for NOx emissions, though. The tests also measured CO2 emissions. But the Ministry kept those CO2 measurements strictly secret. It was not until November 2016 that the CO2 results were leaked to the press and published in an article by the German news magazine Der Spiegel.
The results published by Der Spiegel were surprisingly bad, even knowing that new cars—under real-world driving conditions—emit about 42% more CO2 than advertised by the manufacturers. But the measurements on behalf of Dobrindt’s Ministry were not carried out under real-world driving conditions; they were carried out under laboratory testing conditions, following the official New European Driving Cycle (NEDC) test procedure. And yet some vehicles in these tests (e.g., an Audi A6 2.0L) exceeded the manufacturers' advertised CO2 emission values, supposedly obtained under the same conditions in the laboratory, by as much as 36%.
The following months turned out to be a busy time, with the BMVI arguing with the Italian Ministry for Transport about the alleged use of an illegal defeat device for NOx emissions in some Fiat vehicles and with cities across Germany threatening to ban diesel cars in the near future. The focus remained on diesel NOx emissions throughout—almost as if a giant white bedsheet was wrapped around the suspiciously high CO2 measurement of the BMVI a few months earlier.
The sheet has lifted now, and the high CO2 emissions are gone! On June 26, the BMVI finally published the CO2 measurement results we had been waiting for. But in the interim the Ministry re-tested 29 of the original vehicle models. Only a subset of those results, data on 19 out of the 29, was published (data for the other 10 models will be published later, in a second report, according to the Ministry). For 12 of those 19 models, we can compare the initial test results, as published in Der Spiegel in November 2016, and the new test results now presented by the BMVI. As Figure 1 shows, for all of these vehicles the gap between official type-approval and CO2 levels measured on behalf of BMVI during the retest is significantly lower than it was during the initial testing. Instead of exceedances of up to 36%, the maximum divergence is now 4% (a Smart Fortwo) and “less than 4%” (an Opel Zafira). In many cases, the vehicles show lower CO2 emissions during the retest than according to their official type-approval value. Those are impressively good results—and strikingly different than the initial results from 2016.
The BMVI report is quite vague about the reasons for the striking differences between the two measurement campaigns. One key difference is the procurement of the test vehicles. During the initial test campaign, the test vehicles were obtained “in such a way that an influence from the manufacturer on the vehicles was not given”—meaning that the vehicles were purchased or rented from an independent vehicle dealer. For the second test campaign, the vehicles were provided by the manufacturers. In some cases, the testing itself was even carried out at the manufacturer’s facility.
Another key difference is that during the first testing campaign, the technical institutes carrying out the tests on behalf of BMVI decided how to test the vehicles within the boundaries of the NEDC test procedure. For the second measurement campaign, the manufacturers told the technical institutes exactly how to test. As we know from our previous analyses, there is a long list of loopholes in the NEDC test procedure that manufacturers can exploit during testing: turning off the air conditioning, raising the ambient temperature to 30°C, not following the NEDC speed trace exactly, optimizing the inertia class of a vehicle . . .
The BMVI report provides an example of the influence these test parameters have. For the Audi A6 3.0L Euro 6 vehicle, during one of the tests the driver was told to follow precisely the NEDC speed trace, whereas in another test the driver was told to use all tolerances possible (±2 km/h and ±1 second). In addition, the ambient temperature was increased from 23.9°C to 26.3°C. As a result, the CO2 emission level measured dropped from 161.2 g/km to 151.8 g/km—a 6% drop, which is surprisingly high.
The Audi A6 3.0L vehicle provides a good example of how the CO2 measurements themselves were optimized during the second test campaign (Figure 2). While the average test result for that vehicle model was 180 g/km during the 2016 measurements, it fell to 169 g/km—a 6% drop that exactly matches the single test-to-test change noted in the preceding paragraph. Not only that, but during the original testing campaign, the test result was compared with an official type-approval figure of 133 g/km. This is where the 35% gap comes from (180 g/km measured vs. 133 g/km type approval). For the second test campaign, a type-approval value of 172 g/km was chosen for comparison—the highest CO2 emission value that you can find among all Audi A6 3.0L vehicles offered on the market. In other words, BMVI tested a vehicle version with 29% higher type-approval CO2 emission level and then measured 6% lower CO2 emissions. This is how the overall gap decreased from 35% to –2% (169 g/km vs. 172 g/km).
For most vehicles though, the main difference between the original and the retesting campaign by BMVI comes from exploiting loopholes in the testing procedure itself, rather than redefining the reference point. This is shown in Figure 3. As discussed above, in the case of the Audi A6 3.0L the absolute CO2 level was 6% lower during the retesting campaign than during the initial measurements. But for some vehicles, such as the VW Golf 1.6L, the differences in testing results were up to 25% lower—an impressive illustration of the amount of gaming that vehicle manufacturers can also do during the official type-approval testing.
The BMVI report concludes by stating that “the current regulation . . . allows manufacturers too much of a tolerance regarding the boundary conditions for testing.” It is a pity that the Ministry just repeats this typical defense offered by the manufacturers, instead of trying to do something about it. The recent report, for example, does not include any further measurements that could help to rebut allegations that some manufacturers might be applying defeat devices not only for NOx but also for CO2 and fuel consumption. The report does not even include the CO2 results measured during on-road testing, as it did for NOx. That rabbit is still kept deep in the Ministry’s hat, even though it would be of crucial interest to consumers—who mostly care about the real-world daily performance of their vehicles, not any magically low laboratory test results.