Podcast: The complexities of infectious disease modeling

Published on October 15, 2020

 

The following is a transcript of an episode of From the Front Row: Student Voices in Public Health, the University of Iowa College of Public Health’s student podcast. This week’s episode features Dan Sewell and Caitlin Ward from the University of Iowa Department of Biostatistics. They share their thoughts about working on the COVID-19 modeling app for the state of Iowa and the complexities of infectious disease modeling.

Stevland Sonnier:

Hello all, and welcome back to From The Front Row. My name is Stevland Sonnier. If this is your first time listening to us, welcome. We are a student led podcast that focuses on major issues across the field of public health. Today we’ll be chatting with two of our colleagues at the College of Public Health, Dr. Dan Sewell, an assistant professor in the Department of Biostatistics, and Caitlin Ward, PhD student in the Department of Biostatistics. We’ll be discussing the released COVID-19 modeling work done by the College of Public Health, as well as the importance of infectious disease modeling. Can you both introduce yourselves, your roles at the College of Public Health, and how you became involved in the CPH COVID-19 response team, and Dr. Sewell, if you want to start off first for us?

Dan Sewell:

Sure. Yeah. So my name is Dan Sewell. I’m faculty in the Department of Biostatistics. I started out getting my PhD in pure stats, but was motivated by the interesting applications in biostatistics and hence moved in that direction. So I became part of the COVID-19 response team back in March when various faculty across campus, primarily in the College of Public Health, but also in Carver College of Medicine, we were volunteering ourselves to help IDPH with their COVID-19 response. So this involves understanding what models and forecasts were out there, taking their more granular data and making more Iowa specific forecasts and projections, and importantly, trying to understand the effects of various non-pharmaceutical interventions in Iowa. So that’s sort of how we got involved.

Caitlin Ward:

Yeah. So I’m Caitlin Ward, I’m a PhD student in biostatistics and my dissertation topic was actually infectious disease modeling. So when the pandemic started, I thought this is going to be really interesting to see the different modeling approaches that come out of this. Dan had actually proposed creating this application to help display the results from his models, exploring how these non-pharmaceutical interventions impact our trajectory. He reached out to the graduate students in the department, so I volunteered at that point because I wanted to see more what the modeling team was doing and also help out, given that I had some expertise in the area already.

Stevland Sonnier:

It’s a lot of really tremendous experience and really timely opportunity for everything to come along and be able to it as a student and as faculty as well. Kind of taking us back in time, we go back to April, which sounds so long ago at this point in the timeframe of COVID-19, but the CPH COVID-19 response team had developed a free modeling tool to assist in guiding the community response to the coronavirus pandemic. Can you talk about the impetus for the project? We had mentioned about granular data already existing. We had common models that were already out there, like the Institute for Health Metrics and Evaluation at the University of Washington. How did this effort differ compared to those other existing models?

Dan Sewell:

As you say, there were a whole bunch of different methods, different websites that people could go to to see projections. The IHME was certainly the most widely referenced in mainstream media and elsewhere, still is one of the most widely used forecasting sites. One thing that these forecasts and all these online tools that were available, one thing that it was missing was the ability to accurately evaluate the what ifs. So they were all focused on what will happen in the future. We were focused less on that and more on the, what would happen if we were to say, relax the lockdown on such and such a date, or implement a mask mandate here and what would happen if we reimposed these are bar closures at this time and released it at this other time?

Dan Sewell:

So it was all hyper-focused on decision-making. What would happen if we went in this route versus this other route? So that’s really how it differed from everything else that was out there. One other thing I want to say about this is that these non-pharmaceutical interventions, such as social distancing, quarantining, wearing face masks in public, all of these things are intricately tied with interpersonal contacts. So try not to get too much into the weeds on these models, but there’s always a balance of various aspects, computational efficiency, can we actually compute these models and get results in a timely fashion? There’s a balance between making sure that we’ve got a reasonable representation of the disease characteristics. One facet that sadly gets neglected too often in these sorts of models is interpersonal contact patterns being accurately represented. So my background is in network analysis and looking at how, in this case, how individuals contact each other. So we spent a good amount of time making sure that that aspect was modeled very accurately. So that, when we’re about these non-pharmaceutical interventions, we really are getting an accurate estimate of what their impact might be.

Caitlin Ward:

Yeah. The only thing I would add to that is when we were first starting this modeling, we really wanted it to be Iowa specific. So the IHME model is, now it’s countrywide, nationwide, but at the time we didn’t have a stay at home order, but we did have lots of closures, but their model didn’t reflect that at all because we didn’t have an official stay at home order. So our model was able to really be Iowa specific in counting for the things that we know since we live here.

Stevland Sonnier:

I think that’s an important distinction too, is when you have something national like that, it does miss those non-pharmaceutical interventions that Dan has mentioning about how that varies state to state, depending on if there is stay at home orders or mask mandates or whatever have you with it. When we’re talking about these models coming out, we were talking about what would happen or what could happen. One of the statistician adages I’ve heard before is all models are wrong, but some are useful. How do you really navigate this concept when you’re delivering information to policy makers and the public, especially when we’re really at a point where there’s an emphasis on public health to champion clear communication efforts?

Dan Sewell:

Yeah. So the statement, all models are wrong, some are useful, this statement is absolutely true, but it’s also really horribly misused and oftentimes I think misunderstood. So I think in the age of COVID-19, I’ve heard people use this phrase more often in response to muddling results that they don’t like more than, here’s a model that I don’t like. There’s good, solid, scientific reasons for why I don’t think this model is good. So the reason why it’s true is because infectious disease and the way that it spreads is just insanely complex and you cannot possibly accurately model everything. I don’t know, for instance, how often your grandma goes to the grocery store, when she goes, how many other people will be there, what surfaces she’s touching, who else has touched those surfaces? The transfer efficiency of virus from that service through hands, then from hands to mouth. This sort of thing. You have to simplify at some point in time.

Dan Sewell:

So there are some simplifications that are okay and some are not. Some really will lead to erroneous results. They’re just simplifying things that really need to not be simplified, but other simplifications are perfectly fine. The results don’t change based on going to a less granular level. So one example of this is in the modeling work that I’ve done alongside Caitlin and a bunch of others is, so here we built a contacts network between all the three plus million Iowans and the network, the specific network that we used, really didn’t make any difference whatsoever so long as certain network properties were maintained. So we did a sensitivity analysis on our modeling results with thousands of different networks and it made absolutely no difference whatsoever, again, as long as certain network characteristics were maintained.

Dan Sewell:

The big problem is that these simplifications are not readily accessible to a lay person. It really requires a pretty in-depth understanding of these models and what’s going on behind the scenes. So I think this is why it’s just so very, very important to have a solid faith in science by the public. Scientific process is pretty rigorous, and it’s pretty good at self-correcting. To be perfectly frank, scientists can be pretty ruthless when we find errors in other people’s works.

Dan Sewell:

So I think the safe thing to say is that if you are looking at some of these online forecasting results and you can find a citation for where this modeling work is published and it’s been peer reviewed, then you can have a fairly reasonable degree of faith in that work. We as scientists love to be cited, so we’re going to make it as easy as possible. So if our modeling results are in fact published and peer reviewed, we try to make it obvious. So if that’s not there, that would be the time when I would suggest you start being a little bit more skeptical, but if a forecasting result is peer reviewed, published, I think it’s safe to say it’s going to be reasonable.

Stevland Sonnier:

I know you mentioned the complexity of disease transmission, especially in this case right now, we’re having a novel virus that we’re dealing with. Caitlin, when you’re going through your dissertation process, like you said, you’ve had some previous experience in this area. How did your work as a student prepare you to come into this area of disease modeling?

Caitlin Ward:

The work that I’ve been doing for my dissertation was really helpful in thinking about how these models are set up and how to effectively display and communicate the results from these types of models. Since I’ve been familiar with the output and the types of summaries that we can make, I think something that’s really critical with the communication piece of these models is people look at the forecasts, maybe forecasts that were made in April or May for like around this time. They say, well, they were way off. These models must be terrible.

Caitlin Ward:

Being able to communicate there’s so many things that we just don’t know what’s going to happen. In Iowa, we’ve seen easing restrictions and then adding restrictions and then easing restrictions again. There’s just no way to account for that when you’re projecting so far out. But I would say with the modeling that we’ve done, we really tried to focus on the trends. So we say what happens when you implement a mask mandate or something to that effect and we can see the case count decreasing over time, and we’re less focused on the actual number that we’re getting from our output and more focusing on the trend that we’re seeing there.

Stevland Sonnier:

Kind of following along with that communications piece that you’re reiterating, you had helped lead the user interface and Spanish translation for this, and these usability features are critical for folks who might have difficulty engaging in technology, or they might have English not as their first language. What was your approach to these areas because they are so complex? What feedback did you receive from users or other policymakers?

Caitlin Ward:

First to talk about the Spanish translation, that was another graduate student’s idea. We felt like it was really critical to add that piece in because we saw such a disparity in incidents and mortality for minorities compared to a white individuals. So making our app really accessible for the people that are being affected the most was really important. That was a huge effort from a couple of graduate students in our department, Felix [inaudible 00:13:35] Rodriguez and Elaine Hernandez, who volunteered their time to translate multiple iterations of the application. In terms of usability, we went through lots of iterations with that as well. We got a lot of really good feedback from people in the College of Public Health, both in the Department of Biostatistics and not in our department. It really helped us refine our things. So as Dan was saying, these models can be so complex.

Caitlin Ward:

There’s so many parameters that go into these. So what we really wanted to do was balance that complexity while still allowing users to vary important parameters and see the impact of different scenarios. So what we ended up doing was picking two specific interventions to really hone in on, which was social distancing and mask wearing or PPE efficacy. Then the users could play around with the efficacy of those interventions, as well as the date of implementation. We really tried to simplify the user interface as much as we could while still allowing users to see lots of different scenarios and lots of different projections.

Stevland Sonnier:

That’s a really important piece because you want to make sure that there are boundaries accessible to everyone, but then also be able to interface with it at multiple levels, whether you’re going to be a lay person, or if you’re a policy maker trying to make a decision about what could these scenarios actually look like? How should we respond?

Caitlin Ward:

Yeah. It was really helpful to have feedback from some non biostatisticians when we were creating the app, because we’re so close to it that it’s like, oh, this makes perfect sense. They’re like, we’re not sure what’s going on here.

Stevland Sonnier:

It is always that idea. I’m always amazed at folks who can translate into the design piece because I’m very much so in that similar boat. I’m great at making this data appear and this is all fantastic. Then when someone can workshop it and say, have you thought about this?

Stevland Sonnier:

It’s really great when you have that interactivity. Within this concept of scenarios that we’re talking about, at the beginning of the semester, we had a big rise in cases. Dr. Sewell, back in June, you had noted that if Iowans were willing and able to wear PPE, like face masks, we could potentially offset this rise in cases. We’re talking about this idea of non-pharmaceutical interventions, whether social distancing or face masks. We know that this past Sunday, Governor Reynolds allowed for bars to reopen in Johnson County? What is your hope for patrons and business owners given that we saw this large increase of cases at the start of the semester?

Dan Sewell:

So there’s no question that had Iowan’s behaviors not changed back in March, that we would have seen pretty catastrophic results. Fortunately, we did have a statewide lockdown and people did start wearing masks more and more, and it made a huge impact on the number of cases in the state. The surge of COVID-19 at the beginning of the semester, this was a really foreseeable event. I don’t think anybody in infectious disease was caught off guard by this. This was really not surprising. It was disappointing of course, because what our modeling results based on the data have shown definitively is that behaviors matter. We would have hoped that college students coming back would have taken that message to heart, but that message clearly is not communicated strongly enough. So we did see the cases rise, and so there was some panic from a lot of different corners in response to this surge.

Dan Sewell:

So I had some conversations with some of the leadership at the university about whether we should or should not keep the campus open. They were talking about bar closures, trying to get a citywide closure on the bars and other similar type facilities. I did express the expectation that closing these bars would bring a dramatic decrease in cases. That is in fact what happened. So I guess what my hope is, to finally get around to your question, I guess what my hope is is that sudden drop that corresponded so tightly with the closure of bars, that that is very clear in people’s minds. That it’s very clear that, so long as we can actually act responsibly, we can keep large portions of our society open. We don’t have to have these really strong and severe lockdowns on different aspects of our society. If nothing else, in the bars self-interest to make sure that there’s not a second spike once the bars reopen.

Dan Sewell:

So if nothing else, I would hope that the owners of the bars, the employees, and the students and others who are visiting these bars will behave responsibly just so we don’t have to close these places down again. But there’s an obvious, more strong motivation in that if there’s another big outbreak, then that means it’s not just the people in the bars who are getting infected. It’s then those people going out and infecting others and the people that they subsequently come in contact with. There’s just large downstream effects that I’m just not sure how much is in people’s minds. I think that it’s important to bear those in minds. So my hope is that, simply put, people act responsibly when they go to visit the bars and we don’t have another surge in cases.

Stevland Sonnier:

One of the areas I do want to bring to light is, what do you think the most pressing issues are in your field? You’ve mentioned that folks might not have information downstream. Are there other areas that folks should be focused on, especially when it comes to infectious disease modeling or biostatistics? What are areas that people might be missing or not thinking about off the top there?

Dan Sewell:

Well, I would say that the most important issue by far is just the understanding the behaviors of individuals matter. They matter a lot. So we’re at well over 200,000 deaths in the United States alone. If that doesn’t get people’s attention, I just don’t know what will. It’s a big deal. We all are in this together. Most importantly, the behaviors that an individual does is not self centered. It’s not, I’m wearing a mask because I’m afraid of getting COVID-19. It’s not about the person doing these things, social distancing or wearing the mask, it’s always been about protecting other individuals in our community. You think about when people are most symptomatic, it’s right before you start showing symptoms, if you ever do in fact show symptoms, which means it is literally impossible to decern when you were at your peak infectivity versus you just perfectly fine.

Dan Sewell:

So if you are not constantly wearing a mask when you’re around other people, you could be infecting all sorts of other individuals who then go on, as I mentioned before, with the downstream effects. They go on to infect maybe the clerk at the grocery store, and then that person comes into work the next day and starts infecting other people. Your behavior is not about you, it’s about protecting your community. So I think that understanding is the most pressing issue.

Stevland Sonnier:

I think that’s an excellent point about understanding the effects of what this can be. There’s multiple news stories constantly where someone will hold a wedding and then folks will be infected who weren’t even at the wedding. Those occurrences are very commonplace. I think it’s a little bit of an out of sight, out of mind characteristics sometimes, but I do appreciate the idea of it. It is a collective effort. We are all in this together. We all want to get through this together. We would like to see this pandemic end rapidly. Part of that is doing our part to protect our fellow citizens. Caitlin, I want to turn to you and get your thoughts into as well for our listeners. What do you think is the most pressing issue in this field of infectious disease modeling or within the field of coronavirus since it’s so impactful at this point?

Caitlin Ward:

Yeah. So definitely I agree with what Dan was saying about thinking of individual behaviors. One other thing, particularly with modeling that has been a big issue with COVID is thinking about the data generation process and data quality that we have available to us. So our models can only be as good as the data that we have. Particularly with COVID, we’ve seen asymptomatic cases. It’s hard to get data on them. The data that we see is going to be positive tests over time, which brings up a whole other issues with tests aren’t perfect either. So just looking at positive tests is really not giving us the whole picture. So some things that I’ve seen, I know in Indiana, they’ve been doing some statewide prevalence surveys, so they just test a random sample of individuals in the state and they get some information about this asymptomatic issue, and things like that can be hugely beneficial for our models and allowing us to make accurate modeling of the epidemic.

Dan Sewell:

I could add to that. Just because I think that Caitlin’s bringing up such an important point in talking about data transparency and data availability, so this is something that’s been difficult for a variety of reasons in the state of Iowa, certainly not the least of which is insufficient funding to IDPH. I think that the difficulty in getting reasonably transparent data to important decision makers that, think county supervisors, think school boards, leadership at universities and colleges, they all need to understand the state of the disease in their area. I would just hope that this really illustrates the need to have a strongly funded department of public health that can deliver the necessary information and can have the staff on site to be able to make best use of this data and to make sure that everybody knows exactly what is happening on the ground.

Stevland Sonnier:

Yeah. The idea that this is a big moment for public health, the time to shine, this is showing really where field can be going towards, what we can do when we have the proper data, but really making the call too that we need funding for this. You think of the idea of you don’t put gas in the car, it can’t drive. You don’t give funding to public health, there’s not a lot of expectation for us to do these really important work of infectious disease modeling, if we’re doing community behavioral health and [inaudible 00:00:25:09]. all of these things that we’ve talked about today are really critical, but they can’t happen without funding.

Dan Sewell:

Yeah, I think it’s another point that illustrates this is the fact that it took a bunch of faculty members from, as I said at the very beginning, College of Public Health and Carver College of Medicine who just simply volunteered our time. It was a lot of time. We were all fairly well working round the clock morning, day, night, weekends, just all the time working on this. It was all volunteer work. There just simply wasn’t the personnel that was already in place to do that work. I’ve had a number of contacts with individuals across the state who have put in countless hours of just volunteered unpaid time simply to help make things more transparent in Iowa, to help decision makers have nice dashboards for data, say, at the school district level, this sort of a thing. It’s been a lot of volunteers stepping up to the plate in this time.

Stevland Sonnier:

Yeah. I think back when I’m reflecting on the timeline of this pandemic, there has been a surge of community work. For most of it, we’re concerned about individual actions and how they may affect things. There also is that balancing aspect of, we have so many people who are willing to give back to our state and give back to our country by developing and putting in hours to protect other folks and their efforts should be rewarded and recognized too, for what they’ve done. I would like to thank both of you for your efforts and for your team’s efforts too, as well. We are all very fortunate to be part of a college that is helping to lead, especially in the response to these big issues. When we’re talking about these areas, whether it’s infectious disease modeling the coronavirus pandemic, or just how the pandemic has played out as a whole, what’s one thing that you thought you knew, but were later wrong about?

Dan Sewell:

One thing jumps to mind when you asked this question and that is, if we rewind the clock to May, we started to see cases going down, and this was really perplexing to me at the time. Because when these cases were going down, Caitlin and others on the team, we were looking at mobile phone data that was showing that people were quarantining less. So it was this very odd scenario where, based on the quarantining levels, we would have expected cases to go up. Yet here they were going down. So then I started looking very seriously at seasonality of other human coronaviruses. It was really shocking, two things, one that they were so strongly seasonal and two, they were almost identically so. We’ve got four other major human coronaviruses that pop up year to year. They’re all very low cases to no cases in the summer months and very high cases during the winter months.

Dan Sewell:

So at first I was thinking that this was probably the best explanation for why cases were going down, which is a little terrifying to think that’s the full force of COVID-19 we hadn’t even seen yet in the United States. That would come in November and December, but then as the summer months went on, we did not see, of course, a dramatic drop in cases. It’s been pretty steady with some ups and some downs. Now I wouldn’t go anywhere near so far as to saying that I don’t think that SARS Cov2 is going to be seasonal. It may or may not be, it’s simply too early to tell. All of the ups and downs that we’re seeing is because of all sorts of different factors, people’s behaviors, the George Floyd protests, for example, students coming back all sorts of different things.

Dan Sewell:

If you look back in history, this is not atypical for novel pathogens when they’re first introduced into fully susceptible populations. It tends to behave fairly erratically. But after some further study, I think that the best explanation for why cases started going down in May and why they haven’t just really taken off since then is because Iowans have been wearing masks so much more than they did in the first couple months. That was something that at the time we didn’t really have any data on. So we didn’t realize that it was happening. We go to the supermarket and see, oh, well maybe there’s a few more folks wearing masks than I saw last month, but but since then, I think that that is actually the best explanation for why we’ve been able to dampen the effects of COVID-19 in Iowa. So hopefully, hopefully I’ve been completely wrong about the seasonality of SARS Cov2. We’ll see it, I guess, in a few months.

Stevland Sonnier:

Caitlin, if you want to take that question too. Across the scope of the pandemic, what’s one thing that you thought you knew, but were later wrong about?

Dan Sewell:

Caitlin’s never wrong.

Caitlin Ward:

I’m wrong all the time. I really like your example, Dan, though. Because we did think it was going to be really seasonal. So I had totally forgot about that. For me, there’s a lot of things I could say about how things have gotten so political surrounding public health response and science in general. But what I will say instead, something that I was wrong about, I guess, when I started studying infectious disease modeling two years ago, I never in my wildest dreams would have imagined a global pandemic breaking out and being on this type of modeling team response team where we’re making daily, weekly updates, working so hard on this volunteer effort. Never, ever would have imagined something like that. The models that I’ve worked on before, we’ve taken a year to come up with a model. We come up with something we feel really good about, a really good model, and it takes that long. So to be coming into COVID and having to do things way faster than that, I guess that’s something that has definitely surprised me, just how my dissertation has gone and my time here.

Stevland Sonnier:

I think it’s an excellent point, going back around to the how much effort, the level of effort from all of the public health community. What an excellent opportunity as a student to really put things into practice, albeit not the most optimal of circumstances necessarily? I do want to thank you guys both for coming on today. This has been really insightful discussion and I’ve really enjoyed getting to hear more about what an important issue we are dealing with, and what the folks at our College of Public Health are dealing with. So I want to thank you for your time and for your efforts for the people of Iowa. Like we said earlier, these models are being used. This is data that’s empowering people and all of your efforts contribute to that. So thank you again for your time today.

Caitlin Ward:

Thanks so much for having us.

Dan Sewell:

Yeah. Thanks for those words. We appreciate it. It’s been fun.

Stevland Sonnier:

Thank you for tuning in and big thanks to Dr. Dan Sewell and Caitlin Ward for chatting with us today. If you enjoyed this episode, please share it with your colleagues. This episode was hosted, written, edited, and produced by Stevland Sonnier. You can find our team’s work on iTunes, Spotify, and SoundCloud. We’re on Facebook as the University of Iowa College of Public Health. If you have any ideas for our team or want to connect us with speakers, you can reach us at cph-gradambassador@uiowa.edu. Stay safe and stay healthy out there.