This morning Angus Deaton and Bettina Aten released an NBER working paper (gated, sorry) about understanding changes to international measures of real GDP and poverty that occurred following the release of a new round of price indices from the International Comparison Project (ICP).
Price indices? Methodological nuance? I know, ideal subject matter to drive my web traffic to zero.
For those of you still here (thanks mom!), the paper by Deaton and Aten is a great chance to understand where comparisons of real GDP across countries come from, and to highlight that these comparisons are inherently imprecise and should be used with that in mind.
The basic idea of the Penn World Tables, or any other attempt to measure real GDP across countries, is to compute the following
$$ RGDP_i = \frac{NGDP_i}{PPP_i} $$
where ${RGDP_i}$ is the real GDP number we want, ${NGDP_i}$ is the nominal GDP reported by a country, and ${PPP_i}$ is the "purchasing-power-parity" price index for that country. While there can be severe issues with the reporting of nominal GDP, particularly from poor countries with a bare-bones (or no) national statistics office, the primary concern in these calculations is with the ${PPP_i}$.
Think of ${PPP_i}$ as the cost of one "bundle of goods" in country ${i}$. So dividing nominal GDP by ${PPP_i}$ gives us the number of real bundles that a country produced. If we do that for every country, we can compare the number of real bundles produced across countries, and that crudely captures real GDP.
The ICP produces these measures of ${PPP_i}$ for each country. I'm going to avoid the worst sausage-making aspect of this, because it involves lots of details about surveys to find prices for specific goods, how to get the right "average" price for each good, and then how to roll those back up to ${PPP_i}$ for each country. The important thing about the methodology for computing ${PPP_i}$ is that there is no right way to do it. There are methods that might be less sensible (i.e. let ${PPP_i}$ be the price of a can of Diet Coke in a country) than what the ICP does, but that doesn't imply that the ICP is correct in some absolute sense.
It also means that the ICP can, and does, change methodology over time. The paper by Deaton and Aten works through the changes in methodology from 1993/5 to 2005 to 2011 and how we measure real GDP. The tentative conclusion is that the 2005 iteration of the ICP probably was over-stating the ${PPP_i}$ levels for many developing African and Asian countries. From the equation above, you can see that over-stating the ${PPP_i}$ means under-stating real GDP. So in 2005, we were likely too pessimistic about the economic conditions in a lot of these developing countries. Chandy and Kharas found that using the 2005 values of ${PPP_i}$ implied that 1.215 billion people in 2010 lived below the World Bank's $1.25 per day poverty line. Using the 2011 values of ${PPP_i}$ instead, there are only 571 million people living below $1.25 per day. That's a reclassification of some 700 million people. Their domestic income stayed the same, but the 2011 ICP suggests that they were paying lower prices for their "bundle of goods" than we assumed in 2005, and hence their real income went above $1.25.
But as I said before, these are tentative conclusions because there is no way of knowing this for sure. Deaton and Aten's conclusion is that the 1993/5 and 2011 rounds of the ICP seem more consistent with each other, and 2005 looks like an outlier. So just to keep things comparable over time, we should probably avoid the 2005 numbers. But again, who knows. It's quite possible that mankind's true welfare is measured in the number of cans of Diet Coke that we can produce.
Measuring real GDP or global poverty levels is - to put it kindly - a fuzzy process. There is not the right method for this. As you can see, the measurements can be pushed around a lot by differences in methodology that are inherently trying to make apples-to-oranges comparison (I mean that literally - how do you value apples compared to oranges in national output? What's the right price? It's different in Washington, Florida, and Wisconsin. So how do you compare the total "real" value of fruit consumption in different states or countries?).
The implication is that we shouldn't be asking real GDP measures or poverty line measures to do too much. For really crude comparisons, real GDP from the Penn World Tables is fine. The U.S. has higher real GDP per capita than Kenya, and the Penn World Tables pick that up. Is it a 40/1 ratio? A 35/1 ratio? A 20/1 ratio? Not entirely clear. Different methodologies for computing ${PPP_i}$ in the US and Kenya will yield different results. But is it really important if it is 40/1 versus 20/1? In either case, it is clear that Kenya is poorer. We can go forth and try to explain why, or make some policy advice to Kenya to help close the gap, or go to Kenya to work on interventions to alleviate poverty there.
Where these real GDP comparisons, or poverty line counts, should not be used is in finer-grain comparisons. Is Kenya's real GDP per capita lower or higher than Lesotho's? According to the Penn World Tables, in 2011 Kenya's was lower. But should we do any kind of serious analysis based on this? No. The difference is as likely to be from discrepancies in how we measure ${PPP_i}$ for those countries as from real economic differences in capital stocks, human capital, technology, or institutions.
Real GDP comparisons are best thought of as similar to baseball stats. The top career OPS (on-base plus slugging percent) players are Babe Ruth, Ted Williams, Lou Gehrig, Barry Bonds, and other names you might recognize. Players like Albert Pujols and Miguel Cabrera are in the top 20, giving you a good idea that these guys are playing at a level similar to the greats of all time. You can't use this career OPS to tell me that Pujols is definitively better than Stan Musial or definitively worse than Rogers Hornsby. But career OPS does make it clear that Pujols and Cabrera are definitely better than guys like Davey Lopes, Edgar Renteria, and Devon White (and distinguishing between Lopes, Renteria, and White is hopeless using OPS).
The fact that ICP revises the ${PPP_i}$ values over time doesn't make them useless, just as OPS isn't useless even though it ignores defense and steals. But you cannot ask too much of the real GDP measures that are derived using them. They are useful for big, crude comparisons, not fine-grained analysis.