For months markets have run on the expectation of a V-shaped recovery out of the Covid-induced global slowdown. Recent monthly business surveys have projected a sharp rebound, with the upside surprise boosting equities.
These numbers are a mirage. Lurking behind them are methodological shortcomings that investors are often willing to ignore in their thirst for a singular data point that captures the trajectory of an entire economy.
The problem is illustrated by the recent misreads of monthly surveys conducted in China. Over the last two months analysts have concluded that China was seeing a V-shaped recovery and that its rebound was ” durable and broad-based,” on the basis of the Chinese government’s official surveys as well as privately produced purchasing managers indices. Meanwhile, a survey conducted over the same periods by China Beige Book, the firm where I work, not only showed an economy contracting from a year ago, but seeing just minimal improvement over a historically weak first quarter.
How could two surveys of the same economy point to such different realities? Explaining the difference requires a careful examination of what these polls measure and how. It also forces us to contend with quality issues in existing business surveys, which are only exacerbated in times of rapid economic change.
Monthly surveys of businesses are tracked by investors and policymakers to get a real-time look at changing economic conditions. These polls are conducted by government agencies like China’s National Bureau of Statistics and the U.S. Federal Reserve as well as private firms such as the Institute of Supply Management and IHS Markit. The PMI, which is based on interviews of supply-chain managers, is used most commonly.
These surveys ask companies about changes in business conditions compared to the previous month. It is hardly surprising that 100% of PMIs improved in June as countries emerged from lockdowns. Business activity recovered simply by virtue of economies reopening. But these surveys cannot speak to how the business cycle may have changed compared to the previous quarter or the same period last year — what official economic data measure — because they don’t poll those questions.
While the Chinese government’s PMIs showed a sustained recovery, when we asked firms to compare their business performance to last quarter and last year, we found signals of the economy’s uneven rebound. For example, firms said industrial output grew over last quarter, but year-on-year production was still contracting by double-digits.
China is not an outlier. ISM’s June manufacturing production index surged back to growth, although U.S. manufacturing production was contracting. The U.K. PMI also overstated the strength of the economy in the second quarter, while the recovery signaled by the June Global PMI contrasts starkly with another economic statistic: 93% of the world’s economies are currently in a recession.
These polls are also structured to only provide directional data — whether conditions are better, the same, or worse — and therefore offer no insight on the magnitude of expansion or contraction. As a result, these monthly numbers can exaggerate the strength or weakness of an economy.
The mis-signaling of these polls is perhaps best illustrated by the Empire State Manufacturing Survey’s six-month outlook index, which was higher in June and July than at the height of the recovery from the Great Recession! New Yorkers hardly need to look out their windows to know the future of a state with over 25,000 Covid-related fatalities and a 15.7% unemployment rate is not so bright.
Several PMI surveys do not publish underlying data used to calculate the sub-indices comprising the headline index. This is especially true for China, where the statistics bureau does not report the proportion of firms seeing conditions improve, stabilize, or worsen. This matters because the underlying data would likely reveal a remarkable story. Our latest results found 30% of firms reporting outright contraction in sales nationally versus 33% seeing expansion. Among firms with declines, over a quarter saw sales fall by more than 10%. In other words, a sizable chunk of the economy was in serious agony in the second quarter. These granular and quantifiable growth measures cut against the bullish outlook catalyzed by single-digit PMIs.
Many business surveys also suffer from important methodological challenges. Most crucial is a lack of transparency on survey methods. For instance, ISM does not publicly release the sample sizes of its surveys. While China reports sample sizes for its PMIs, it offers no data on sample composition. It is well-reported that the official PMI is biased toward large, state-owned firms. Alternatively, the Caixin PMI focuses on small and medium private enterprises.
Most enterprise polls rely on repeat surveys of firms in a panel, rather than either randomly selecting respondents or at least drawing in fresh respondents each cycle. This, unfortunately, magnifies the issue of nonresponse bias that surveys suffer from intrinsically. While this approach may be sensible at a regional level when sample sizes are sufficiently large, repeatedly polling a few hundred firms nationally is at best a cohort study disguised as a representative survey of the entire economy.
Enterprise surveys are a crucial tool for successful investing but being smart consumers of such data is a prerequisite. Portfolio managers must carefully evaluate the methodological approaches of different polls and weight their role in models and investment decisions accordingly
For its part, the business survey industry is in dire need of standard-setting, including guidelines on transparency — especially for market-moving indices — and best practices. Counter to election surveys, where pollsters are excoriated for misses, corporate surveys have no such repercussions and to detrimental effects. It’s time the data-collection world outlined a “gold standard” for business surveys.
©️Newswires
