Constructing Market Data Thresholds: The Approach Reviewed in This Essay and the Fed Proposal
We base the thresholds we review in this essay on the Federal Reserve proposal. There are some differences between the two.
First, thresholds in the proposed rule would also rely on an additional type of market data, subordinated debt. We do not have ready access to such data for most of the firms in our sample for the earlier parts of the time period we examine.
Second, the proposed rule relies on a different threshold regime. The firms in the proposed rule would breach the threshold if the median value of the signal over the trailing 22-day period exceeded the 95th percentile of the five-year range. We do not have daily observations across all of our measures in the time period under review. It appears that this difference may not matter. We compare our results using end-of-month observations along with the requirement that two consecutive observations breach the 95th percentile to the threshold system in the proposed rule when we have daily data, and the results are almost identical.
Third, the proposed thresholds also included measures that controlled for overall market effects by subtracting the median of corresponding changes from a larger peer group. Because of data limitations, we have omitted such measures from our discussion.
Finally, the proposed rule also sets forth a threshold regime based on fixed thresholds. The thresholds we use change over time as the distribution of historical observations evolves. We do not back-test the fixed thresholds. We assume the Federal Reserve would have altered the fixed levels annually to ensure that the thresholds identify more potentially weak firms and fewer potentially strong firms. We cannot make these adjustments in our analysis.
A fixed threshold regime could perform differently around crises periods than the varying threshold regime we use. The fixed threshold regime may remain in breach long after the worst of the crisis; the absolute level of the market data may still be above the fixed threshold level. A varying threshold may be harder to breach after the worst of the crisis. Almost by definition, the worst recent experience used to calibrate the varying threshold has already occurred, making it difficult for additional observations to exceed the 95th percentile.
In contrast, a varying threshold regime may be breached more easily in a pre-crisis period. Recall that a firm breaches the varying threshold we review if it exceeds the 95th percentile of historical observations over the last five years in our example. A firm could breach such a threshold even if the absolute level of the threshold would not strike an observer as worrisome.
To explore the performance of fixed versus varying thresholds, we alter the threshold regime we review in this essay as follows: Instead of using varying thresholds, we run the same back-testing experiment, but use the fixed thresholds discussed in the Federal Reserve’s proposal. The thresholds in that proposal are set at CDS levels above 44 bps, EDFs above 0.57 percent, implied volatilities above 45.6 percent and marginal expected shortfalls greater than 4.7 percent. The proposal notes that these levels were chosen after considering the trade-off between early warning and potentially false signals.
Relative to our varying thresholds, fixed thresholds provided less of an early warning, but remained in breach status much longer. Specifically, from January 2007 to July 2007, the varying thresholds had breaches for all but one of the failed firms and 14 of 23 of the other large banks. Under the fixed thresholds, there are three failed firms and three large banks with breaches during this same period. From August 2007 to November 2007, all of the large banks and failed firms had threshold breaches when varying thresholds were used. In contrast, six of the 23 large banks and seven of the 10 failed firms had breaches when the fixed thresholds were used.
By October 2009, more than half of the large banks were no longer in regular breach status under the varying thresholds. Under the fixed thresholds, there were only six firms not experiencing a breach as of June 2012.
