COVID-19: Data Uncertainty and Effectiveness of Interventions

It definitely is. The early evidence we see in terms of deaths and hospitalisations is way above anything in the past records, even if we go back to the 1918 epidemic it seems comparable. You only have to look at what happened in Iran or Italy – their health systems have been completely overwhelmed. And that has an impact. It is not only people with the disease that are dying, it is also others who should have been receiving care. It is also having to make decisions on who you treat and who you don’t. Furthermore, the damage is there not only in healthcare, but also, for instance, in the economy.

In our study (Lai et al. n.d.b) we tried to understand, which types and combinations of non-pharmaceutical interventions had the biggest effects, and the role of putting those interventions in place in different time periods. At the time of the research – back in February – only China had a major outbreak, and we were working to build a model of what was going on.

The results may help public health leaders to optimise the set of interventions needed and the timing of their implementation. Our findings show that case detection and isolation of confirmed or suspected cases, if put in place as early as possible, are effective, as is scaling up the testing. In the countries that are the most successful in controlling the disease, testing is widescale.

Unsurprisingly, however, the biggest impact was from all these interventions combined. Social distancing and the travel bans added to that had a major effect. Our estimates show that the outbreak in China in February would have been almost 70 times greater if they hadn’t put in place any of those interventions. Other modelling studies on policies in the UK and the US show similar results.

How to ensure these interventions are in place early is now a major concern for low-income countries. The reason for implementing such measures in the richer countries is to protect the health systems from being overwhelmed, to ‘flatten the curve,’ ie to not get above the capacity of the health systems and minimise the amount of excess deaths. For low-income countries the challenge is that they already are reaching the limits of their health systems. This means they need financial support to be able to scale up testing and isolate cases, just as other countries have done. Some of that support is coming through aid agencies, but the capacity is just not there, for sub-Saharan Africa in particularly.

The recommendations here would be similar as what has been recommended to high-income countries and what many of them didn’t put in place as early as possible because they didn’t have the time to prepare. The situation is very similar – it is known that the outbreak is coming, so to reduce the impact on their health systems, low-income countries should ensure best possible testing and isolation.

In such countries, there are some positive aspects, eg their populations are far younger and the numbers of elderly people, that are really driving those excess deaths, are much lower than, say, in Italy. But on the other side there are many negative aspects with regard to their health systems, many underlying heath conditions, malnutrition across the populations, or the fact that social distancing is just something that many cannot do in low-income settings.

It is interesting to see. Every country is following a different path, experimenting. The UK, just like Sweden now, had started out with slower, ‘wait-and-see’ approach and then suddenly switched. Sweden already has a much higher death rate per population than the surrounding countries that have put in place stricter measures. So it wouldn’t be surprising to see Sweden following them soon. Each country is ending up in a very similar situation. The question now is when they come out of this situation and how.

This is a very difficult decision. It is about finding the balance between the safety of the public and protection of economy, and I wouldn’t want to be a politician at the moment to make that decision. If we just end the lockdowns, we are risking a massive resurgence of the disease. Across the world, there is still a very small percentage of population that have had this virus. This means the potential for spread and overwhelming the health systems is still there. We’ve already seen this with Hong Kong in particular. They put in place strong measures and then took some of them away. This led to a massive second wave coming in, and they had to tighten the measures again.

If you are considering taking away some of these interventions, you have to make sure that the testing and the monitoring is very strong. Then you can start to gradually lift up some of the restrictions, but at the same time be ready to put them back if there is any threat of resurgence.

First of all, we don’t yet know about SARS-CoV-2 immunity – if people are immune and how long that immunity lasts. Second, with herd immunity we need 60-70% of population to actually get the virus, and we are far from this yet. Plus, if we do get there, but too fast, then we will end up with many deaths and overwhelmed health systems.

There are, indeed, massive uncertainties about the data on COVID-19. We cannot really compare case count across countries – it scales with testing, the more you test, the more you find. For example, in the UK many deaths that occur in care homes are not included in the regularly reported numbers. Yes, there is a very large proportion of elderly with underlying health conditions, but there are still quite a lot of deaths in younger age groups. It is all a balancing act again – trying to understand what the fatality rate is. We keep learning, and so many studies are ongoing.

On the one hand, it is great that scientists can put out their research so quickly now. The way everything is changing so fast has had a big impact on our way of working. We had to really adapt and do things as quick as possible, otherwise they become outdated. There are analyses that we have planned and wanted to do with the same rigour as usually, but we had to find ways to get results out quickly so that they are valuable to act upon. Take our study on interventions – it was online in early March, but it is still a few weeks until it is finally published in an academic journal. With things moving so fast, by that time it wouldn’t have been so useful. At the beginning of February we put out another COVID-19-related study (Lai et al. n.d.a), an analysis of the spread down in China based on the air traffic data. It caught a lot of attention, but now we may not end up publishing it in a journal because things have change dramatically.

The information should be made available as early as possible, but the question is how accurate it is. The academic journals that do peer-review are trying to keep up, but many of those reviewers are already overwhelmed with their own work. It all comes down to responsibility of the scientific community to highlight and point out problems with papers that have been put out there as preprints with no review. I do see a lot of that happening on social media and news articles – when something has a bit of a headline, but the study hadn’t been reviewed and there are major problems with it. I am hopeful there is a kind of self-correction mechanism in that, but it is still a concern as spreading unverified information can be dangerous.

In general, I am impressed by how quickly the scientific community has put together a lot of work, and has collaborated on making the results open – that hadn’t happen so much in the past.

This outbreak is showing how we all are really connected, and the influence it has. The unprecedented speed of the spread has become a problem for so many countries. I have done many studies in the past on population movements, how diseases move around and the speed it takes them to spread. COVID-19 has broken all records.

It has also highlighted the challenges in the regions where countries are strongly connected, Europe is an example here. Right now we are trying to adapt the model we used in China to European settings, looking at how connected European countries are and what that means for strategies. If one country completely removes all of its interventions, it is still strongly connected to the rest of Europe – so how much of an impact that has for other countries in terms of the second wave that we anticipate will happen in all countries at some point.

In discussions around the coronavirus there have been two broad categories about the use of mobile data. First, the one we are more focused on, is about broad dynamics of how populations in general move around. There is no need to look at individual level data. It is similar to traffic statistics you see on Google maps, ie aggregated and anonymised monitoring of mobile phones that telecom companies are using routinely. These data are quite valuable in understanding the flows between the cities, or the general changes between mobility patterns.

Second is the ultrasensitive data, the kind that Singapore had used, ie contact tracing and surveillance of individuals. Here the real discussion should be taking place about the sensitivity of these data and the role of government in working with them. This can be, on the one hand, a very valuable tool if we are to get out of the lockdowns, but on the other hand, it is bringing up questions about governments tracking our movements.

At WorldPop, we’re a group of about 30 researchers at the University of Southampton. For the last 15 years our work has been focused on integrating different data sets, mapping populations and their characteristics, mostly in low and middle-income countries where there are no strong systems for collecting data regularly on populations. We integrate household surveys and satellite data to try and build up that picture of where the populations are and who they are.

In this outbreak, the data on populations’ age structures in particular have been used by many groups and organisations to see which areas have higher rates of vulnerable populations, eg males, elderly, etc. These can also be integrated with some other data that we produce – on poverty, on malnutrition – to identify underlying health conditions and how those may vary within countries.

The World Health Organization has been in the news a lot, for various reasons. They are relatively small funding-wise, but have a lot of influence, particularly in the low-income countries, which rely on their support and guidance. They are also influential in instructing or supporting countries in data collection and coordination.

I for sure trust WHO more than many governments.I know many people from there, academics who are very rigorous, robust scientists trying to deal with all the politics across the world. This is not an easy task, to coordinate and provide guidance for setting standards across different countries where you have multiple types of politicians, all with different agendas. We have been part of coordination calls that they have put together for modelling groups. For us, it is very valuable to hear all those groups, and it is really only WHO that has the influence to bring all of them together to discuss ideas, models, data. So WHO does have quite a valuable influence, but it is more about guiding governments and setting standards, not so much about implementing interventions themselves.

Who was the first infected person is something that would be there in the months to come, when they start assembling the genetic data and comparing those with viruses that are circulating in the wild, to better understand where they actually came from. It may be the case that we will never know who was the first person. It is hard to be able to definitely say when the symptoms can look so much like influenza – and many people have influenza that time of the year.

I have had quite a few experiences recently, with journalists in particular trying to get a story out of the work that we’ve done, eg about ‘had China acted three weeks earlier, it would’ve stopped the global pandemic.’ We cannot support that without data, and considering how much has been going on and how many political decisions have been taken – it’s hard to say.

Clearly, some government or a newspaper may have an agenda to make a story out of some data. It is always a challenge to communicate something where there is a lot of uncertainty. We are trying to produce the best that we can, yet nothing is set in stone. It is still our responsibility to communicate our outputs, and it sometimes means building that uncertainty into decision-making. Nobody wants an uncertainty, however – they want a number to make a decision on.