This transcript is auto-generated and may contain inaccuracies.

Dina Hofer:
Well, welcome again. My name is Dina Hofer. I am with Guidehouse, and I am an Associate Director for our public health informatics team. Today, we’re going to be discussing enterprise evolution considerations for enhancing public health practice through shared data solutions. I think we all know and have heard the critique that public health can be very siloed, especially when it comes to data and our data systems. But as we have made progress with data modernization, and those efforts have moved forward, enterprise data platforms have been key to that success, and today we’re going to hear some really great stories about shared data solutions and learn more about how that works within our public health agencies.

So, session objectives today, we’re looking to understand the key stages and components involved in the implementation of an enterprise data solution. We’re going to analyze the unique strategies and challenges encountered during Enterprise Data Platform implementation, and then finally recognize that we have organizational change management and the important role that plays in an enterprise data solution success.

Obviously, we just finished up. We’re working through our welcome right now. We have a series of jurisdictional spotlight presentations, a really great team here today to present to everyone in the room and online. And then for the last 10 minutes or so, we will have plenty of opportunity for any questions. As we work through the presentations, save your questions for the end.

We have five speakers today. First up will be Rebecca White, who is the DMI portfolio and project manager at the Massachusetts Department of Public Health. Then we have Matthew Buck. He is the director of data modernization for the Michigan Department of Health. We have Sophie Rand and Chip Ko from the New York City Department of Health and Mental Hygiene. And then finally, today, Will Wheeler from the Department of Public Health Informatics at the Santa Clara County Department of Public Health. And I will now turn it over to our first speaker, Rebecca, to kick us off.

Rebecca White:
Thank you very much. Good morning, everybody. My name is Rebecca H White. I am representing the Massachusetts team here today. So, for starting out with our shared data solutions. We really needed to convince folks that there is something in it for them, and we wanted to try to bring those wins back early on.

And so one of the first things, well, actually, I was going to set the stage a little bit for you, just to explain a little bit about Massachusetts. So we are a very decentralized organization. Bureaus and offices have a lot of autonomy to not only manage their own data, but often their own data systems. And so the data modernization team is located in the commissioner’s office, so centrally located, but trying to provide services and a solution to meet the entire department, which has operated very much on their own, you know, agendas and strategies.

We also have around 1500 employees in DPH Central, but there are also four public health hospitals, and about 1500 employees are there as well. So we have a very large and varied workforce. So, looking at our very simplified data modernization vision, as I shared before, most folks have their own data systems and data solutions, and so putting it into one place, into our enterprise data platform, is a big endeavor for us. So we’re going to be taking disparate DPH data sources, our national and public population data sets, as well as data from partner agencies, putting it into that centralized platform and creating faster and more holistic analytics on the end.

So, although we wanted to create the centralized platform, we did not fully understand the data and technology needs at the Bureau office and program level. And the Bureau office and program are kind of long, so I’m just going to stick with bureaus, calling them bureaus. So what do these bureaus and offices even have? Data Wise, we don’t really know what is important to them. Once again, we don’t really know once we started this out, and what challenges or unmet needs were they experiencing that we could try to help meet as a centralized team, and with all this information, what do we actually bring onto our platform first?

To try to start out by answering those. Questions. We went through a data discovery process. It was a four-meeting series endeavor where we had one intro session, two deep dive sessions to be able to explore the bureau and office and how they’re set up, as well as the data assets that they have and the data assets that they would like to have, or that they do have, but they currently have challenges in getting them. And then lastly, it was a live review of the opportunity snapshot, which was our deliverable post these Data Discovery sessions.

So we had three different objectives, which was to understand the key data sources within each bureau and office, consolidate the process flows and identify where we the DMI team, can step in and support help data and help data and help support data management and validate and complete Bureau and office snapshots and a Data Project inventory. So, in addition to understanding what data they wanted and needed, and what they most used, we also compiled a list of the data inventory, or the data that they actually had, and they were the stewards of, and the deliverable from our Data Discovery sessions was this opportunity snapshot. It was broken into three sections.

We did a program brief that summarizes the bureau and office mission, structure, and function. Not only are bureaus and offices very decentralized and have their own mission and vision, but they also have very different structures within them. So just creating a high-level understanding of whether they had a data team that was, you know, off in its own little unit within the bureau and office, or if their data folks were spread throughout the bureau and office.

Secondly, we understood the high-value data for that Bureau and office. So, not just the data they held; it could be data they held, but it’s also data that they’ve recognized other folks in the department hold that would be high value to their mission and vision. And then lastly, data-related opportunity themes. We heard a lot of different data management needs and stories throughout our data discovery process, and we wanted a way to capture those without, you know, recognizing, well, recognizing that our first mission is going to be getting data into the platform so then we can actually help manage it.

These are just kind of like keeping an eye on the prize here. Through our data discovery, we have now completed what is important to the bureau and office, and some of the data that they actually have. And this is a picture from our high-value data add piece of the opportunity snapshot. So this is an example from the Bureau of Family Health and Nutrition.

They identified birth deaths, fetal deaths, and linked birth deaths as one of their high-value or data that would be of high value to them and their strategic priorities. So it dives into detailed info on how and why they want the data specifically, not as much about the data set itself. And so with that, we also learned that there were five other bureaus and offices that identified this as some high-value data. So we have stories and use cases for each bureau and office around how this set of data would be useful to them. So now we have a repository of use cases we can pull from when we’re looking at different datasets.

Along with that, I mentioned how we also inventory just the data that they have. We also identified the data stewards and validated that list of data sets. Post our opportunity snapshot publication to start registering the datasets that the Bureau of Office had. So by registering the datasets, we’re starting to create an understanding of what our bureaus and offices actually have. So currently, we have 186 data sources registered. That is not how much we actually hold, because as you’ll see at the star at the bottom four bureaus and offices have not yet registered anything. So this is an ongoing project, and once they register it, we now have the data steward contact.

We also have a lot of other contacts as well, including their IT and technical support, and so once they fill out that baseline, or once they fill out the registration, then we can get some baseline information, which we have used to create the prior our prioritization rubric. So this was going to be a way. So now we have understood what the bureaus and offices want and need Data Wise, and what they have through the data inventory, and finally, we wanted a way to look a little bit more holistically across the department to understand what would be most beneficial to the department to onboard early on in our data modernization process.

So for our rubric, we have three different sections, and we use the DPH data inventory surveys. To be able to get this information. We wanted timeliness because, of course, you know, sometimes we have very emergent needs. However, we wanted most things not to fall into that timeliness category. We wanted most of our data sets to be calculated on that value-add piece, meaning that we have identified that they have really high aspects that support our data and our strategic priorities as a department.

So by doing this, this has turned out a little bit different in practice. It would have been really nice if we had had an overall project prioritization rubric, rather than just a data set onboarding prioritization rubric, because now, that we have data in the platform, how do we manage what is actually important, like, do we onboard more data, or do we turn to more analytics use cases or operationalize some of those things that we learned from the high value data?

So some of our next steps are going to be continuing onboarding to the EDP, but as I mentioned, we need to show that value back to bureaus and offices and how the information they gave us through our data inventory and data discovery process can benefit them. So we’re going to be completing the registration of all data sources and baseline surveys, and then I’m just going to jump down to number four. We’re going to publish a static version of the DPH data inventory, which will be available internally. And then next fall, we hope to have published an interactive dashboard for the DPH data inventory for internal staff use, so hopefully by adding some teasers about what they could access once it’s all available. So next up, we have Matt from Michigan talking about organizational change management.

Matthew Buck:
Good morning. I’m going to start my own clock here. To keep myself honest, I have a 37-slide deck for you. We’re not going to get through that. So, suffice it to say that if you have questions afterwards, please email me. Reach out. I’m assuming that the slide deck will be socialized more broadly, but if not, I’m happy to share it too. So I’m going to get through some core content, hopefully plant a seed. Unfortunately, what I’m not going to be able to cover is really sort of this specific example of how we’re implementing against this model in Michigan, but I’m going to provide some overarching guidance.

So a few disclaimers. I’m not an organizational change management expert. I just play one on TV, so take what I say with a grain of salt. This is organically learned within our jurisdiction. This is not some academic experience that I’m presenting to you. I did not receive any direct funding or payment in order to do this. This is just like I said. I’m an unpaid actor, but so if you’re willing to throw a few bucks my way, please let me know.

This is organically developed within my jurisdiction, and we feel it’s important and working well for us. So we wanted to share more broadly what we have learned, which has not been a linear path. When you look back retrospectively, obviously, things look linear, but that has certainly not been the development of our OCM process, and we have a lot more to learn as we go. So this is where we’re still sort of in the initial foray of how to proceed.

And I strongly recommend, if you have a chance, grab this QR code, look at ASTHO, OCM training resources. I’ve tried to adapt a lot of what they have within the structure. So you’re going to see some references to the ad car model, which is awareness, desire, knowledge, ability, and reinforcement of change. And in the appendix, when I captured the snapshot of what we’re doing in Michigan, I tried to frame it. To frame it within that ADKAR model so you could see how it actually works in implementation.

A brief note on what we do at MDHHS: we administer programs across a broad range of health and human services to meet the health, safety, and prosperity of the residents of the state of Michigan. Key takeaway here, we’re a super agency. We have Medicaid, child welfare, and Child Protective Services, among others. In addition to public health, we’re a $38 billion agency. We represent over half of the state government within a single department, institution out of 13 within the state, and public health is a very, very, very small part of that.

So, from an organizational change management perspective, we have an uphill battle, like we are swimming upstream to get recognition and acknowledgement within our leadership space, because they have so many competing demands that they’re trying to prioritize. Right? So how does organizational change management fit into that so that you can get that leadership buy-in when there’s so much else happening within the same organization?

Organizational change management is the structure that we use to prepare people for change. By and large, this has been around for about 100 years plus or minus. It’s strongly applied within the technology space these days, and it prioritizes the people who will be impacted by the change over some specific mandate or agenda that somebody might be pushing. So it really tries to manage the disruption, that disruption potential that can be the impact of the result of change in any organization. This quote was from a member of my team when I was asking folks to review the slide deck before we send it up. He noted this in the comments, and I said, thank Nick. Thanks, Nick, that’s a great comment. I think I’m going to put it in the presentation itself.

And what he was really framing here is that within a massive bureaucratic organization like MDHHS, large-scale initiatives are painful, and we are slow to change. It’s like turning the Titanic on a dime, right? Like, we just don’t respond, and there are a lot of competing demands. So how do you manage that change with people who don’t have an incentive to change, and sometimes are actually resistant or hostile to the change itself, because they want to remain within the status quo? So this really positions us as an organization, and I think of public health as its own enterprise, within the broader enterprise, in how we move this forward?

So you can ask yourself some core questions, like, why are we innovating? What are we innovating for? Where does that need to happen? Those are good questions. They’re good starting points. We took a little bit more of an introspective approach, thinking about what our motivation is. And for us, when we think about our public health data strategy, we’re really going upstream, right? You think about a lot of what’s happening within DMI, within PHIG. There’s a lot of focus on things like ECR and fire, and these are really important topics. They’re, they’re going to be super innovative for public health, but they touch on a small amount of what public health actually does, right?

When we think about going upstream to other determinants of health and promoting health equity. You know, no case report is going to tell you about the lifespan history of somebody that you’re trying to help or impact, right? So, really important technologies, but they do a small bit. And when I think about what the opportunity is for public health, we have to go beyond that trope where the COVID-19 pandemic taught us blah, blah, blah, blah, blah, right? We hear that all the time, but nobody actually tells you what the COVID-19 pandemic taught us. What we learned, at least in my experience, is that, yeah, we could have done better at case investigation. We could have done better at surveillance. We had antiquated systems. We all know that, right?

But where we really failed was the stuff that passed just under the radar. It’s when the Executive Director comes to you and says, I need to know what proportion of people with disabilities are vaccinated against COVID. And you have to take a team of five people over the period of several months, where every week, they churn through data and manual exports, because nothing is interoperable and nothing is ready to measure who is a person with a disability. First and foremost, how do you define that? And then how do you measure that across an organization and even at a super agency like MDHHS, where we have behavioral health and disability services within my own agency, I don’t have access to those data, right?

So this was an incredibly manual-intensive effort that lasted for months on end with a team of people that were pulled from their regular work to be dedicated to this effort that is not anything that ECR is ever going to fix, right? So that’s really the focus of what we’re thinking about in MDHHS for data modernization. So we’re applying a diffusion of innovation theory model, where diffusion of innovation theory is a well-founded theory that looks at how to promote changes across people. It is specific and measurable. It’s evidence-based. It’s a good go-to if you want to adopt something rapidly and know that it’s well-founded and embedded. And you can apply the Ad Card model, which is that, you know, building awareness across the teams that you’re trying to promote change with, trying to promote a desire within those teams, those teams that otherwise don’t have any innate desire to change. How do you build the knowledge and the capacity to change, and then how do you reinforce that over time?

And this slide, if there’s any takeaway that I want you to walk out of this room with, it is this slide, where you are targeting change is not at the end. It’s not the 100% adoption. If you look at the adoption curve through innovation, the Innovate Diffusion of Innovation theory, you really want to get to that early majority, and we’re presuming a normal distribution here, right?

But you start with your innovators. It’s the smallest, two and a half to 5% of your population, the people who have a predisposition, they really like change. They like being innovators. They like taking risks. They then bring in the early adopters. And when you get through early adoption, and you really start showing progress, you get the buy-in from the early majority. And if you can get to that point about the initial 18 to 20% the evidence shows that there’s a snowball effect that it just keeps moving. So it’s the momentum through that, that early adopter phase, that you really want to focus on,

Perfection is the enemy of delivery, right? Sometimes, good enough is good enough. Some of you might remember, years ago, there was a certain bridge of a digital variety. There we spent years, and I mean years on, trying to test ECR within a few, few small cohorts. We focused on the edge cases, right? And we really specified the implementation pilots down to the nitty-gritty. And as a result, it took three-ish years plus or minus to really get it launched at one or two sites, right? Perfection can be the enemy of delivery. Sometimes you just have to get through the 80%; you’ll deal with the 20% later.

So organizational change management is the end goal in many ways, and that’s kind of heretical to say, I think, when we’re focused on technology, but at the end of the day, all of your plans lead to organizational change management, because in order to lead that change, it implies a set of missions, visions, goals, objectives and priorities, right? Which implies a strategy, which implies that you’re then building the capacity to make decisions, which then means that, well, you have to have a governance body to actually implement those decisions, and that governance body has to have something that they’re implementing against. So that builds the implementation roadmap.

So everything works towards this organizational change management structure, where at the end of the day, if you’ve played all your cards right, you’re setting yourself up for success. As a super agency, where a large proportion of our business is focused on the human services side of the house, they are rooted in the defensive data capabilities on the right-hand side of that page; it’s about reporting to CMS. It’s about preventing fraud. It’s about showing that you’ve done your due diligence.

Public health is on the left side. We’re on the offensive data capabilities, and both of those are rooted in a shared set of foundational capabilities. So our goal at MDHHS is to build, right now, through our organizational change management, build a coalition of the willing, willing that’s focused on reinforcing those foundational capabilities and moving towards more data democratization on the left-hand side, through those offensive data capabilities. So OCM has to be baked into your model. This is our implementation plan for technology on the left. OCM on the right. It is just as important as the technology that we’re moving forward.

If you have any questions on this, I’m actually out of time. I’m getting the stop sign right now. But if you have any questions about this, email me. I can, sort of, I can walk you through the logic behind why we structured it this way, and also think about a maturity model. We’re using the CMMI model here to go from an initial level of maturity up to optimization. With that, I’m going to skip everything else. If you have any questions, here’s my email: buckm@michigan.gov. I’m happy to share more anytime anybody has any questions.

There are some appendix slides in the shared version of this that aren’t in the master copy here. So there’s more content. And I am introducing, yeah, it’s our colleagues from New York City, Sophie Rand, and Chip Ko, who I believe are virtual today. I’m not sure how, okay, all right, I’ll leave it to you. That’s awesome. Thank you.

Chip Ko:
Good Morning, everyone. Just checking that you can hear me. Thanks so much for having us today. We’re excited to talk about our progress in developing an enterprise data platform at the New York City Department of Health and Mental Hygiene. My name is Chip Ko, Senior Director of Integrated Surveillance and Data, and I’m joined by my colleague, Sophie Rand, our enterprise data platform product lead. Today’s agenda, we’ll talk about the data Modernization Initiative here at the New York City Department of Health and Mental Hygiene, and then go into our enterprise data platform, or EDP, as we might call it. Throughout the presentation, we’ll go through some key components of the platform, discuss the gains for the agency, and then do a couple of use case spotlights to show our progress.

All right, we are so grateful for the support the public health infrastructure grant has provided to our DMI progress here at DOHMH, and I just want to acknowledge that a lot of this work would not have been possible without the funding. And so, as we all probably know, the data modernization portfolio is quite expansive. And so it was really important that we defined our data modernization mission at DOHMH in New York City. And so that is the data Modernization Initiative, a strategic effort to build a modern, secure, scalable, and interoperable public health data infrastructure that ensures timely, high-quality, and actionable data for decision making, operations, and public health action.

Our goals for DMI were one: to modernize the foundational data infrastructure and platforms. Two, to improve data integration, interoperability, and data quality. Three, to enable advanced analytics, AI, and data science capabilities. Four, to strengthen our data governance security and privacy frameworks, and five, to support faster, smarter, and more equitable public health decision making. So our overview today will include a high-level understanding of our EDP bill and also our approach to taking on this very big endeavor.

Our approach, as we’ll demonstrate later in this presentation, is really to take use cases from across the agency that do a couple of things. One, add crucial data sources to our enterprise data platform to expand our data catalog, and two, enable platform capabilities to move this work forward. So again, with the concept of data modernization being quite expansive, it was important for us to first define the scope of work for our core DMI team, or the one team, as we call it, versus the scope of our work for our programmatic analytic teams that will be actually using the platform.

So we define the DMI core project on the left as the foundational infrastructure services, systems, and processes that ingest, store, manage, govern, secure, integrate, and share the data. This is mostly the build and maintenance of our EDP and associated data governance processes being developed in parallel. So, where the DMI is applied is what we call it. We define this as the applications, dashboards, analytic products, and processes built on the core infrastructure to support public health programs and functions. So essentially, applied projects where the programs really still retain business ownership. The remainder of this presentation will really focus on the DMI core work of building the enterprise data platform, and so I’ll turn it over to Sophie.

Sophie Rand:
Thanks, Chip. Just a quick sound check. Yep, great. This slide provides a high-level overview of our EDP architecture and the users of our platforms. Each of the bolded symbols here on the left and in the upper right represents our primary users. On the left, we have data analysts accessing data assets to create analytics and visualizations, and on the right, end users of the data products they create. So our agency leadership program, subject matter experts, and data stewards. I’ll do a brief overview of some of the components represented in this slide. Oops, sorry, from left to right, here, outside this black box, which represents our cloud, enterprise data platform, we have GitHub.

GitHub is integrated with our EDP, and it’s our source for storing our analytic and our infrastructure as code, code base, which are used by data analysts, as well as by the EDP administrators who are maintaining the infrastructure of the platform itself. We have, on the bottom left, ingestion systems representing the tools and systems used to ingest data into the platform. Those go directly into our data lake, which have a medallion architecture with raw data landing in the bronze and then data traveling through silver and gold layers as they go through various levels of cleaning, standardization, validation and curation, transformation, analytics and engineering tools are available for analysts and users to access those data which are governed that access is governed by unified access control to ensure that data are protected and only accessed when needed, both by end users and by EDP administrators and engineers.

We have, of course, our data catalog where all the data are stored as well as metadata, and through that data catalog, our leadership, SMEs, and data stewards can view information about their data assets, including access patterns. Then, of course, the dashboarding and reporting tools, which pull from data from the data catalog. We also note here platform services part of our EDP architecture are services like geo coders and person matching applications that transform and combine the data and a port out to other DOHMH systems, the EDP is the back end to other systems within the agency. I’ll go through just a couple of anticipated gains for our users, secure collaboration. In our current state, we have data and code intermingled, causing some trade-offs between security and collaboration. In our current EDP environment, we have code and analysis separated, ensuring a more secure and collaborative environment, more modern development, data development practices, so data pipelines are sort of modularized so that we can control data quality, data curation, etc.

I’m going to move on to the next slide, just because I see we’re closing in on time. Maybe I’ll jump straight to a use case spotlight to use the time a little more efficiently.

So I’ll just go over one of the use case spotlights here. This is for our Vital Statistics Program. This centers on an in-house-developed system called Mortals, which predicts the provisional cause of death for all deaths in New York City. Typically, it takes three to six months to establish an official cause of death for any given use case, but programs need to have a sense of death trends much sooner than three to six months, in case there are any anomalies and patterns that need more immediate public health attention.

To solve this problem, Data Scientists in our agency developed a system called Mortals, which uses machine learning to predict the underlying cause of death from historical death data and clinical text associated with the death records. So we took on mortals as one of our first use cases. It enabled us to flush out many important features, such as data masking, synthetic data creation, segregation of data between environments, provisioning clusters for machine learning, connecting EDP as the back end to other tools, dashboarding, using Power BI and other dashboarding tools, etc. I’ll hand it back over to chip for, I guess, to wrap us up because we’re looks like we’re at time.

Chip Ko:
All right, we currently have over 100 use cases for our enterprise data platform submitted by various programs across the agency. So our next steps include finalizing a prioritization framework that evaluates the impact on the agency, feasibility, and value to the actual EDP and value to the stakeholders. From there, we’ll be able to identify our next set of use cases, and then future releases to the EDP will incorporate new features like bi-directional data exchange with external entities. There’s our contact info if you have any other questions. And I see our time is up, so I’m going to pass it over to Will Wheeler with the County of Santa Clara.

Will Wheeler:
Great. Thanks. My name is Will Wheeler. I’ll be presenting on behalf of Santa Clara County on our data platform, which we have called gathering, uniting, and analyzing data for action, learning, understanding, preparedness, for emergencies data platform. We started with the word Guadalupe and filled in the acronym behind it. We had sort of colloquially called our platform either AWS, which is the host, or the data lake, for a number of years. And we wanted it, I mean, it includes a data lake, but it doesn’t encompass the whole thing. So we needed to come up with a different name. We wanted to keep a body of water in the sort of theme, and so we went with Guadalupe, which is a river that flows through Santa Clara County, and right behind the public health building, there you go.

So today I’m going to talk a little bit about the challenges that motivated Guadalupe, or we also call it “Lupe” for short, our data strategy, briefly a high-level overview of our implementation, and then how we’re leveraging the platform. And I’ll give a brief overview. Santa Clara County is South Bay County, so just south of San Francisco, it encompasses San Jose and some cities up the peninsula, as well as far south as Gilroy. We have about 1.7 million population and about 700 staff in the health department. Our organization centralizes the epidemiology, informatics, and evaluation resources into a science branch, so we’re centralized there, and we serve the other branches.

So what motivated our data platform? What motivated Lupe were challenges in scale. And this is, of course, coming out of the COVID pandemic, we were limited because all of our compute power was on premises, and to be able to scale both up and down, we wanted to move to a cloud, which would allow us to do that. We had challenges in standardization and workflow. So, as you know, in an organization, if you have lots of segments of that organization, they will figure out different ways of doing things that are not similar to others. And so when you need to bring people all together, they have different ways of doing things, and that makes it more difficult. So we wanted to standardize the way that people work as much as we could, as well as the workflow and data flow. And then we also wanted to have the ability to leverage automation, often with on prem commute compute, you have to have people there in the office or on their laptops clicking a running person or some button to execute things, and we wanted that to we wanted to automate the pipelines so that they could run on them on their own.

And so what sort of guides us are both the CDC health data strategy as well as our own Santa Clara strategic plan, which may be too small, so I’ll read it to enhance the breadth, timeliness, accuracy, accessibility and transparency of data to better understand current health situational awareness, as well as develop the staff capacity to use those data to build policy and action and improve our ability to use environmental health contextual data. And so how that has informed the structure of Guadalupe. This is a very high-level architecture we have. We endeavor to connect any source of data directly to Lupe, rather than having, you know, emails, people collecting data, storing it on a shared site, and then moving it up. We want to make those direct connections with the data brought in. We store that in a data lake, which is sort of the bronze stage. It’s in its native format. And then we have pipelines taking that data, transforming it, matching it across data sources, so that we have a master person index, and we have an ability to look at people across data systems.

We standardize the data to some degree in that sort of silver stage where we if we want to do some quick analysis. We can use that sort of staging or silver layer, and then we have the gold layer, where everything is fully transformed and standardized across data sources. And then we can serve those data to both public purpose-built applications that are within the data, the AWS entity. We also have Power BI that connects for our public-facing dashboards and reports, as well as a posit server that’s actually hosted on AWS so it has direct access to all those data. And then we’re building out our generative AI create tools and other applications using Bedrock and Q, which are AWS stack, and then ultimately, the purpose of all of this is to allow people to connect to it and make that easy.

And so sort of our, our, our strategy for Lupe is sort of five points. And this is a living, living strategy. I would call it. We don’t try not to think of data in silos. So it’s not tuberculosis data; it is a contribution to public health data. And if we start there. That becomes our motivation, and we sort of try to treat the data differently, not as the parts, but as the sum of the parts. This is borrowed from Chris Baumgartner in Washington. We lower barriers for bringing data into Guadalupe by, you know, focusing on making the direct connections to the data sources. We make data and insights from data easy to use, access, and customize for the public health staff and the public. We want to be able to use this platform to leverage new technologies. So right now, everybody is the shiny object is AI, but in five years, we want to know whatever is the shiny object, then we want to be able to leverage that for public health. And then embedded in all of this is continuous improvement and the process of processes and tools.

Okay, this is oddly placed. I just wanted to note that we have built some internal applications on the AWS stack. Those can both be served data as well as go back into the data lake, as well as. As all of our vendor-built applications are used in the procurement process, we now say you have to be able to send data and receive data, so it needs to be fully interoperable. So, how we’re leveraging Guadalupe, I have sort of three use cases. This one is especially in an emergency situation if our data sources go down. So if the state of California systems go down, we still want to be able to case find and capture the data that we need in order to make those policies and situational awareness decisions. And so we have two real-time or near-real-time data sources that we collect directly, and those are the syndromic surveillance data sources, as well as ECR.

We send that through a match process, and that should gather roughly 90% of our cases that our state partners would be able to find in an emergency, and then the rest of the process remains the same. People will still access data in the same way, and we’re motivated by this. Our emergency response activities shouldn’t be that much different than our routine, routine activities. If we can get those to be similar, then an emergency becomes well easier. Seems like a pipe dream, but at least more straightforward.

We’re also, as I mentioned, building purpose-built applications. So, this is really, we have a highly skilled epidemiologist who knows how to program in Java, so we are taking advantage of that. So what we have, what we’ve built, is for the Harm Reduction Program. They’ve gone through about a five-year cycle where they haven’t found a software solution that really fits what they need. What they really need is a customer interfacing application as well as a supply inventory, and those to talk to each other. So what we were able to do is essentially build the prototype while talking to the program and say, This is what we built. Is this right? Does this seem like what we need to fix? And we went through those quick iterations and were able to do that on the AWS platform.

And then we have shiny applications. This one we built in about two weeks, again, for the Harm Reduction Program. They wanted to be able to see where overdoses were happening, so we gathered some EMS data, and we’re able to produce that for them, because the data is, I mean, that goes back to our low barrier for data coming in. That data was available to us. So we, we were able to make this fairly quickly. And then we also had heat deaths and heat emergency room visits based on essence or syndromic surveillance data, a dashboard we built for that.

And then, as I mentioned, we are, we have a fairly restrictive generative AI. We have a low risk tolerance in the county, I would say, so there has been, there’s a long process in order to approve generative AI projects. We’ve recently gotten four through, so we’re very excited about one of them, which is a partnership with AWS, let’s see, cloud innovation creators, something. It’s a Cal Poly kick, which is a partnership that AWS has, and they’re developing a method for parsing ECR messages. We’re excited and happy to take your questions. Thanks.

Dina Hofer:
All right, we do still have time if folks, if there’s anyone in the audience that wants to raise a hand and ask a question of our panel, definitely open to that anyone also who is participating via zoom, we are open to questions there as well, see if we can get things started. Oh, got a hand over there.

Larissa (Audience):
Hi. I’m Larissa. I’m from Harris County, a public health county that has Houston in Texas. My question is for Will, so when you said purposeful applications, so one just had a conversation yesterday about. Generative. Ai, I’m trying to actually buy some qualitative analysis software, and had to have a conversation with our IT person about their risk tolerance for generative Ai, so top of mind for me, my question for you, is it really resonated when you said, we’re doing purposeful applications.

We have seen similar success, like the programs that are delivering the community-facing services, like they want what they want, and they need what they need. Because half the time when we go in, and we look there, it’s, you know, cobbled together spreadsheets or some software that they were using literally 20 years ago, and they’ve kind of jury-rigged to work. And so we want to help them. And we come in, and we’re able to, you know, utilize dashboards, curated reports, performance insights, and that kind of thing. And it really gets our foot in the door and gets the early buy-in, which we need.

But what is always in the back of my mind, and what I’d like to know, what you’re thinking about, is 10 years from now, when we’ve been doing that, do we just have a whole bunch of disparate dashboards? I don’t know. How do we be mindful of not like they need, what they need, and sometimes it’s extremely unique, and we can’t stamp out the same dashboard for everyone. But also, how do we not, in 10 years, end up with just like 1000 Power BI dashboards, and no one knows how to navigate them, except the dude that left five years ago. You know that kind of situation?

Will Wheeler:
I don’t have, we don’t have a magic answer. We certainly see that we have a lot of Excel spreadsheets. We just did a data inventory, and I can’t remember the number of sorts of applications, such as Excel, that we’re using. So it’s definitely something we’re working through. I think the way that maybe the two approaches that we’re taking are strong governance around application building, around data and sort of around activities, so that any new request is balanced with, you know this would take, this would cover 80% we have already built it for another program. It would cover 80% of your needs. And then with the 20% would you be able to figure out something else to cover that the other, the other possibility is taking those two programs and saying, Okay, you’re very similar needs, and maybe there’s a third or fourth program that are we anticipate is going to be similar. So we built an architecture that accommodates all four while being flexible about the last 20% for each program.

So I think that’s possible, too, and that sort of gets to my second tool, which is the agile project management process we’re using, where we ideally engage the whole organization in the discovery of what needs are. They get to hear what other program needs are. And the process of having those conversations, it sort of makes that 20% a little bit easier to communicate why. You know, if you’re a program and you know you need exactly what you need, but you’re in a conversation hearing about another program needing, you know, slightly different things. It just it makes that sort of meeting in the middle a little bit easier to tolerate.

Dina Hofer:
And I think we have a question from online.

Audience Member #2:
This is a question from the virtual audience for New York City. Can you provide additional details on the machine learning model? What information do you feed into it, and how accurate is it in determining the cause of death?

Sophie Rand:
Sure, I don’t have evaluation metrics top of mind that can try to pull them up while other questions are getting answered. The Model reads in historical death data, so trends in causes of death over time, as well as weather data, since we know seasonality and hot and cold can impact deaths; those are the primary two sources. And then obviously the death, the, you know, the new death record itself, which includes data, EHR data related to the death. So, clinical text is processed. There’s an NLP component to the model. As well as ICD codes associated with the case of death.

Dina Hofer:
Okay, we have another question.

Hanford Lynn (Audience):
Hi, I’m Hanford Lynn from Guidehouse. So, we’ve heard from state health agencies, counties, and cities. This is a question for everybody, I guess. Can you talk a little bit about how all of these efforts are being coordinated, if any, so that whatever, let’s say, New York City is learning can be translated up to Albany for the state, or whatever Santa Clara is learning can be shared across multiple counties across California. What, if any, kind of organizing factors are occurring at all levels, from state to county to cities?

Rebecca White:
I can start, so I’m Rebecca H White from Massachusetts. We currently have an ongoing project with the Office of Local and Regional Health called the metric project. In Massachusetts, we have 351 cities and towns, which all have their own local boards of health. So it’s a very, very massive and disparate organization. And through our ARPA funds, they are creating a platform that will help with licensing investigations, etc, but then they are going to have 14 analytics use cases to help further the public health foundational capabilities and to help those local boards of health who only have like half time epidemiologist, provide some analytical power to their work.

So the DMI team is heavily coordinated in that. I would say I spend about half of my time right now working on the metric project. I joke that we are the Parent Project, and we are at the whims of our very wealthy child, because they have, they have the money and the resources right now, so we’re kind of following along with each of those use cases. They’re going to be bringing on around 70 data sets to be able to support those.

We’ve also asked that they bring on all of the data and talk with each of the programs to ensure that not just the data that they need for the dashboard can get onboarded to the EDP, but that the data that the bureau and office need, or that the department needs, can get onboarded as well. So we’re coordinating very heavily with them in hopes that, yeah, our programs will have the data that they need, and so will the office of local and regional health. That’s one way.

Matthew Buck:
There’s a conflict in how to respond to that question, and it’s embedded in the nature of how public health operates, which I think a lot of people are intimately familiar with. I would say there are really three avenues in response to that. There’s a funding component, a policy component, and an organic Community of Practice component, in terms of how you coordinate across those to meet that need that you’re describing.

Some groups, and I’ll call out Sarah Shaw from PHI in the back, who’s done a fantastic job in organizing the data modernization learning community through PHII. And I call out Sarah specifically because one thing I’ve noticed in that group that I haven’t seen elsewhere is that there isn’t an offloading of the intellectual labor needed to conduct the work. Oftentimes, people convene a group, and then they’ll put the onus back on you to figure out how to operationalize things. And that group has done some really good work to show us. They did an assessment across 12 different jurisdictions for enterprise data platform development, which, from my perspective, really helped me understand that what we’re trying to do in Michigan is actually the same calculus and the same thing that everybody else is doing. So that organic sort of resource sharing was really helpful in helping me understand that we’re not on the leading edge, but we’re also not behind. And there are a lot of others who have a similar shared experience.

There’s a policy question around: how do you make sure that this flows up and down? It’s really limited by public health codes and different levels of Federation of public health activities across different jurisdictions. I would note that all public health is local. It’s important to remember that, and within our jurisdiction as a decentralized Public Health Authority, we see our role within data modernization as a convener of the solution, right?

So I’m not innovating just for new technologies within MDHHS, but how do I make those available to the broader public health ecosystem within my jurisdiction that includes local public health, tribal partners, community organizations as well, and that is rooted in the Public Health Services Act at the federal level. And then if you think about the way that’s written towards the federal government, it charges the director, the Secretary of HHS, to serve in that convening role at a national level.

There is a funding gap that I would certainly encourage CDC and other federal partners to look at. There’s a lot of good effort within PHIG, DMI, ELC, and other funding resources to say, you know, go forth and modernize your systems, but there isn’t that synergy behind how do you do that in a shared way. There was a missed opportunity, I think, in PHIG where we could have said, Yes, assess your data systems, and here’s the methodology, here’s the modality by which you should assess those systems, so that we can do that in a shared way across the country, so that we’re all undertaking the same kind of assessment in a comparable way, where we can measure against each other, understand what the shared opportunities are.

Unfortunately, what it’s resulted in is each jurisdiction figuring out how to do that on its own. So there is a gap there that I think systematically we have yet to figure out how to address nationally. But we can also do that organically. And if you look at what Josh Weimer has done in Missouri, in adopting the HIMSS DHI tool, we piggybacked on that in Michigan; we’re also implementing the DHI tool because it gives us a degree of comparability. So there are ways that we, as individual jurisdictions, can also take on the onus to fill in that gap where we see it.

Dina Hofer:
All right, any other questions? Oh, okay. Not another one online. Great.

Audience Member #2:
Another online question for Rebecca or others for data discovery: what did you find worked best to identify all data assets and gather detailed information, including surveys, interviews, and combinations? How did you combat data owners, who can tend to be territorial over their data?

Rebecca White:
We have not found the answer to territorial data owners, but I think one of the big things that we found is being able to provide that opportunity snapshot back to them and also back to the others in the organization. It’s been really, really useful for onboarding folks across the department, because they have an automatic way to understand what each bureau office is about in terms of getting the actual data sets.

We have one business analyst who is, she will, she will keep going. And I think that one of the really strong things that she did is that she just, like, kept asking, she kept asking the question in kind of different ways. So, like, what data does this group use? Like, how do you use that? What kind of questions are you answering? Etc, and then she was just trying to be really broad. Like, we’re not, like, I think that the data inventory that because it’s going into REDCap, it’s the data itself isn’t actually going anywhere. It started from a very like, low hanging fruit place, like, we’re just trying to learn about you. Like, what can you tell us about you? What can you tell us about your data?

And also, we just tried to cast our net very broadly. Like, I think that there are, as Matt said, the ECR and the MIS and those big systems are really important. But we really wanted to ask, you know, deep into their their systems as well, what are those small like, one off questionnaires that you had that you put to your, you know, program around, you know, Sexual Violence Program administrators or something like, what are those things as well? So I think just showing interest in those small-scale things has really helped build trust and partnership with those data owners.

Chip Ko:
I also have something to add from New York City. I think the importance of developing a strong data governance framework also helps data stewards feel more comfortable with sharing data in a shared data platform, too, because we’ve definitely come across territorial data owners as well.

Dina Hofer:
Absolutely. All right. Well, we are basically at time, so I want to give a big thank you and shout out to our presenters today. We are actually butting up right against a break. So I know everyone shared contact information. We have three of our speakers here in the room, so take full advantage to you know, walk up to them and ask any additional questions.