Words matter: Mining comments for COVID-19 nuggets

As the Minneapolis Fed has been regularly surveying businesses across the Ninth District about the economic impacts of COVID-19, the response has been overwhelming. Many firms are eager to share their experiences during this unprecedented shock.

Even more impressive than the thousands of survey responses has been the amount of extra information businesses have been willing to share in the form of comments. Respondents to the most recent survey, conducted at the beginning of April, submitted more than 4,300 comments to various questions, totaling almost 59,000 words, or the equivalent of about 120 single-spaced pages.

Parsing all of this text can be time-consuming, but technology can help summarize major themes exponentially faster than doing so the old-fashioned way (namely, reading them all), while finding additional nuance from businesses beyond responses to multiple-choice questions.

To illustrate, the figures below display relative word frequencies among the collective comments on each survey question. These aren’t the most heavily used words; those were words like “business,” “money,” and “employees.” However, those words tended to be fairly similar across the comments for all survey questions. (Methodological note: So-called stop words like “the” and “but” are also eliminated as part of the frequency analysis.)

Relative word frequency

Rather, the terms displayed in these charts are the ones that show up most on a given question in comparison with the others. For example, the word “expansion” popped up more frequently in responses to a question about supply chains and capital expenditures. These comments were from businesses reporting that they were putting growth plans on hold. One example was a midsize Minnesota manufacturer that reported it had pulled back on planned plant expansions.

This computer-based approach helped point out the unique frequency of the word “clinic” (and related words) among comments to a question about lost revenues. The news of a jump in unemployment insurance claims among health services workers was initially greeted with some surprise, given expected surges in hospitalizations. But survey comments revealed some interesting effects on non-essential health care providers.

“We are a dental clinic and had to shut down our practice except for emergencies only,” wrote a North Dakota respondent. Numerous other dental care providers across the district reported similar closures. The word “clinic” also got a boost from nonmedical businesses in one area in particular: Rochester, Minn. “Mayo Clinic has canceled all non-essential appointments for our guests,” wrote one food and lodging business there.

Other comments illustrated the indirect impacts on other health care providers from cancellation of non-essential medical services. “Because elective surgeries are canceled and many medical clinics are providing emergency care only, the number of patients referred for physical therapy has dropped tenfold,” wrote a Minnesota contact. Reports like these from physical therapy clinics were one reason the word “therapist” also popped up frequently, as were comments from massage therapists.

The high frequency with which the words “furlough” (as well as “furloughed” and “furloughing”) appeared in response to a survey question about employment effects provided some detail about firms’ staffing decisions. “Any cuts will be furloughs,” said a large Wisconsin retailer, one of many respondents reporting that they want to keep employees available to return once business as usual resumes. “Our plan at this point is to furlough our employees with the thought of bringing them back,” wrote a Minnesota construction firm.

These comments revealed a variety of other employment strategies. For example, a small North Dakota manufacturer said it was “furloughing employees but allowing them to work up to the max allowed by unemployment.” Meanwhile, a large Minnesota manufacturer said it was “cutting hours across the board by 10-15%. No furlough at this point.”

However, not all mentions of furloughs were hopeful. “We cannot pay our staff with the uncertain end date, and it is better for them and us to furlough them,” said a Minnesota health care provider. “However, if this drags out longer, we may need to reduce staff permanently.”

Mining comments also provided some insights into the anticipated effects of state and federal emergency aid programs. As the survey results indicated, respondents were optimistic on balance about the prospects for aid, and comments were generally consistent with that—for example, among commenters who mentioned “PPP,” an abbreviation for the Treasury Department’s Paycheck Protection Program.

“We have approximately 60 eligible businesses for PPP and [the Small Business Administration’s Economic Injury Disaster Loan] in our town,” wrote one North Dakota nonprofit. “The influx of forgivable loans is going to be a huge boon for our town.” Businesses also expressed optimism that the program would help them retain employees, such as a Minnesota bank reporting that “the PPP program will allow us to keep 100 percent of our staff and their income levels.”

Comments also betrayed some of the skepticism and frustration businesses are experiencing with aid programs as well, especially among smaller operations. The word “confusing” appeared in the top 10 relatively most frequent terms for the survey question about emergency aid. All of the businesses using the term reported having 10 or fewer employees.

These insights reflect just the tip of the iceberg from the thousands of comments received on surveys so far. More analysis will be forthcoming as the Minneapolis Fed sifts through them, and as more flood in with future, ongoing surveys.