This transcript is auto-generated and may contain inaccuracies.

Whitney Magendie:
Good afternoon. Thank you all so much for joining us. I know everybody is exhausted at the end of this. We’re so out of shape for in-person convenings, right? It’s like all these people that we have to interact with in person, and it’s been a long time since we’ve done that.

My name is Whitney Magendie. I’m the Lead Advisor for Public Health Infrastructure at PHAB. I oversee the strategy and implementation of our PHIG infrastructure grant, and I’m super happy to be with you today for Tools You Can Use: Resources and Strategies for Leveraging Data to Inform Workforce Planning and Pathways. We are going to hear from folks from three states who are leveraging data to better understand the complexities of the public health workforce opportunities, to plan for a pipeline moving forward, and to better anticipate the challenges lying ahead in workforce issues.

I’m going to introduce our speakers now. They’ll go up in kind of a streamlined fashion, and then we will have time for Q&A at the end. First, we’re going to kick off with Scott Murakami, Hawaii’s Workforce Director and PHIG PI. Then we’ll turn it over to Azalee Hoffbauer and Steve Holloway from the Colorado Department of Public Health and Environment. Aza is a Public Health Workforce Analyst, and Steve is the Health Access Branch Director. And then we’ll conclude with Dr. Emma Spencer, who’s the Division Director for the Public Health Statistics and Performance Management Division at the Florida Department of Health. Without further ado, I will turn it over to Scott.

Scott Murakami:
Good afternoon, everyone. I started off with this cartoon here. I really like it because it gives you a very good idea of the breadth of labor analytics.

So here’s this guy right carrying the billboard, going, I’ll work for cheap, and telling the statistician-looking guy, that you know, there just aren’t any jobs, so I give up. And the statistician tells him, Well, congratulations, you’re no longer unemployed, and that is true if you are working, if you are not working and don’t have a job, and you give up actively looking for a job, after four weeks, you are no longer in the unemployment count. You are now considered the discouraged worker.

And the reason for that is that there are six levels of labor unemployment. They call it the alternative measures of labor underutilization. And people don’t realize that. They’re like, Oh yeah, our unemployment rate is 4%, and everybody else is working. That is not true. And so the question this breeds is, like, how do you use this data? How can this data help you? Labor analytics is very helpful because it can tell you for a specific geographic region, what the jobs, the number of jobs that are there, what the median income is. It can give you demographic information and migration patterns on who’s coming into the state and who’s not. It can be used for a particular occupation, you can drill down within a region on occupation and find out what knowledge, skills, and abilities are needed, and it can help you better understand the education pipeline that will feed those occupations.

So what I’m going to show you today is a tool that we use in all of our planning, and it’s not just for public health. So the tool is used statewide by the University of Hawai’i. It is a tool that has a subscription fee, and our Department of Business, Economic Development, and Tourism uses it. So our State Economists also uses this tool. But the tool consists of two types of data, and in general, labor Analytics has two types of data, right? Structural data, which is government data, that’s reported out quarterly or monthly, and I’ve listed some here for you. The other one is big data. So the issue with government data is its suppression, right? Just like PHWINS, they have suppression if it gives any one business a competitive advantage over another, they will not share that information. And the other thing is, for big data, it’s large volumes of heterogeneous dirty data that have to be pretty much aggregated, collected, scrubbed, and then normalized so that you can actually use it. So the tool we use helps us quite a bit in doing in in planning for our workforce.

So we share this slide kind of regularly throughout our presentations, right? This looks at the first portion of it, the workforce planning aspect of it, and really, this is a quick example of how our process works. So we start off with what our immediate need is. So last year, before the legislative session started, we reported on all of our vacancies. And one thing that was pretty clear was our entry-level office assistant positions, which are very low-level positions. Those positions were, there was a high vacancy rate with it. There were, like, 91 vacant positions. So whenever we have a situation like that, we start asking, like, what’s going on here? Right? Are these? What are these individuals getting paid? And as you can see, there’s a range of salaries listed here. And if you start at the very level, you will be below the ALICE Threshold in Hawaii, meaning you are living below the paycheck-to-paycheck level.

You guys know what the ALICE level is. So ALICE stands for the, see, I’m going to blank out here. It’s Asset Limited, Income, Constrained, Employed people. So there’s a federal poverty rate, right? Everybody has an established federal poverty rate. But the ALICE population looks at what it actually costs to live in Hawaii for a single adult or a family of four, and that population, anybody below that, is living paycheck to paycheck, so they start becoming a vulnerable population for us.

So we do some general analysis of each occupation with higher vacancy rates. What we then do is we try to figure out what’s going on in this market, who we are competing with, right? What are who’s feeding it, and what are other people paying for it? And to do that, we use a proprietary tool called Lightcast. So Lightcast is actually an economic modeling software that does forecasting. It does not do predictive analytics, right? It’s just straight-line forecasting and provides a good baseline under the standard economic assumption that all things remain equal. But what it does is give us a very good baseline.

So what I wanted to do is, if I don’t mess this up. I’m sorry if I messed it up in advance, but I’m going to just do a really quick live demo on it. So this is our Lightcast software. It’s a subscription that we pay for through PHIG, but it allows us to do quite a number of things. So under the region, we can actually identify what the economic situation is, what the general economic condition is like for a particular location. So this is and I’ll show you a couple of them if we have time. So this is Hawaii, right? Our popular shows our population size, the total number of jobs, the median household income, and data about our gross domestic product and trade balance between imports and exports. And of course, Hawaii, we’re going to have a negative trade export, right? Export, because we are always importing fuel. We are importing food.

So, I loaded a copy of some reports into the attachment so you can kind of take a closer look. But what this does is it tells us for Hawaii, right? What are the major industries? What are the occupations that are competing for these positions? What are the overall population characteristics for the state? So it’s, it’s pretty interesting. You can drill down to the zip code or metropolitan statistical area level. So, for example, right? Let’s see, this is why. I don’t want to scroll around too much and get everybody all dizzy, but it’ll tell you how many people in our workforce are retiring soon, and it’s based on Census Bureau data, right? So it’s the people who are 55 plus. So they qualify for some of them, and will qualify for retirement. It shows you where the talent works and where they live, but it also shows you migration data, right?

So this is our state migration data. We get a lot of people coming into the state from San Diego, and a lot of people leaving to go to Las Vegas. So you know why they’re coming from San Diego? Because we have a shipyard. Right? And that’s a lot of people who are coming from the San Diego base that, I guess, used to be systems in Hawaii. So it tells you where people are coming from and where they’re going to it shows you how our industry stacks up in Hawaii. And what I really wanted to show you is who the biggest employers are. Let’s see, and this is a pretty interesting slide. When we look at a particular occupation, you will actually see what institutions have completers that are feeding the data into that occupation. So this data comes from the Higher Education Council’s IPEDS data, Interdisciplinary Postsecondary Data System, I believe, or Integrated Postsecondary Data Educational Data System.

So it pulls all this data in, and what it tries to do is remove suppression from any data points it has. And the way it does that is it uses a proprietary biproportional data algorithm. So it’s basically, think about it this way. It’s basically playing a huge game of Sudoku with the data, trying to figure out, so what are the seed values? Right? We’ll plug it in and see if it works.

So that’s kind of a general look at what the data does. What’s really cool about it is that you can actually look at it by an occupation. So for I’m going to show you a real quick example, because I think I’m running out of time, but let’s see. So this is, I think I said it. So these occupations in Hawaii are the ones that we are struggling with now. So it will give you a very good profile of it. There are 6,822 people working in this field. Now the compensation rate is $50,005.81 at the median compensation rate, right?

Here’s what really helps us. If you scroll down to the bottom, it’ll tell you. It tells you all the common skills that they need, the foundational ones, but it tells you what your educational pipeline is. So to build a program that we talked about this morning in the morning session, if you were there, that’s why our contract, or MLAs with Leeward Community College, because they’re the ones putting out the completers in this area.

I’m going to stop it there, because I think I’m running out of time. I got a few other slides, but I’m going to skip down to the next presenter’s slides, and I’d be happy to answer questions. Oh, well, you’re wrong. Oh, oh, well, okay, all right, so you’re still stuck with me.

Look at that, so this is a real quick summary of what the data tells us, right? And the neat thing is you can do it for any with our subscription, we bought it nationwide. So you can do it for anyone. We can do it for any state, any geographic, any zip code, any MSA, but this is the summary data that shows you in a real quick snapshot. So it helps you better understand the market that you’re in. And the reason for it is that it’s very competitive in Hawaii, right? We have a very limited, limited labor pool. If you look at our census data, our working-age adults are leaving, and we’re gaining a lot of the elderly population. So that makes you worry, because it’s like, who’s going to take care of these people, right? But where we’re heading with this, where I’d like to head with this, as a possible TEP for us, is, I’d like to see how we can use this as the baseline, add additional data to it, and see if we can start forecasting some of the social determinants of health right, because that would give us a more forward posture on where our work or workforce should be.

So I’m going to stop it there, even though I probably have some more time, I’m going to yield to my colleagues, because that was a lot of information I’m happy to share with the tool more on an individual basis with you if you’ve got questions, but yeah, that’s our that’s what we are using to help us create our workforce plan to understand who’s competing with us for the labor supply that we need. So thank you.

Steve Holloway:
Thank you. I’m very interested in seeing more of that, that dashboard and software, perhaps on a break. Colorado. My name is Steve Holloway, and I am from Colorado, and I direct the Health Access Branch at our State Health Department, and we have been working on a very similar kind of strategy and solution for understanding clinical workforce for the better part of a decade.

Most of this presentation is going to be delivered by my friend and colleague, Aza, who is a young, newly minted MPH student, and I will briefly put in a plug for the essential nature of creating a platform for the next generation of public health informaticians and workers to do this kind of work, because it doesn’t happen fast.

In sitting through these sessions, I thought that every project that I’ve heard is kind of dependent upon three things: authority, funding, and time. Three things that we often have a pretty significant shortage of in the kind of public health work that we do, I think the nature of our success and our strategy, frankly, is a couple of things. We were too dumb to know how hard it was going to be. We borrowed lots of authority from other parts of the state system, and along with that authority, we borrowed effectively funding. There’s lots of investment in data that is essential to understanding the public health workforce, but it exists in silos. It’s a cliche in meetings like this to hear about data in silos built for a single use, use case, and we know that it answers lots of questions, but it’s kind of locked up, and it’s structured strangely, and we have to reach in and get it and make sense of it.

So a lot of our strategy is really about simply accepting the landscape for what it is, which is siloed systems, unstructured data, differently structured data, and how do we make leveraged use of that information to tell an important story about the workforce. We began with health systems, healthcare delivery systems, so payers, facilities, and licensed professional clinicians, and, through that work over many years, we were really set up to accept work under PHIG to expand that effort into understanding the public health workforce.

Public health workforce is much more complex to understand than the clinical workforce, because clinicians, by nature, are credentialed and licensed, and there are certain regulatory data sets that you can always go to and learn something about that workforce. Public health is a much more diverse field. It requires many more data points to understand each worker’s capacity. We also have less standardized nomenclature for how we understand who is a public health worker, and what does it mean to be an epidemiologist, and what does it mean to be a community health worker, and what are the constellation of credentials that go along with those titles that can tell us something about whether or not a community has the resources and capacity that it needs to respond to routine public health challenges, but also equally essential those emergency public health challenges. So with that, I’m going to turn it over to my colleague Aza, who’s going to talk about a public health taxonomy design and how we are developing a very similar strategy for the public health workforce that we’ve applied to the clinical workforce to hopefully create a sum that is greater than its parts when it comes from those data sources. So I’ll introduce my friend and colleague, Aza. Thank you.

Azalee Hoffbauer:
Thank you, Steve. I appreciate it. Yeah. So to get started, I’m Aza. I use they, them pronouns, just to make sure that’s clear. And I just want to start by thanking Steve and my other colleagues, Shanna Wisdom and Tamara Davis, for their support in this work, because, like Steve said, it’s taken a lot of time, and it’s often time that Shannon and I, when we reflect back on the work that not a lot of other people working in public health get to just sit down and try to think very broadly and kind of brainstorm, how does the public health workforce actually look? How does it actually function? How can we actually describe what we do on a day-to-day basis in a way that makes sense across the board, across agencies, across the infrastructure?

So in all this time that we’ve been given to do that, we have created our own way to process this data and have created a taxonomy that organizes programs, capabilities and roles across the public health system. And that work has really built on years of effort to better understand and classify the public health workforce. So now that we have built it and have used it in a couple different places, what we want to do now is show it to everybody and hopefully engage you all in hope in starting to use it in your own places, so we can continue to learn how a standardized workforce data set for public health can help us in the future with many, many different things, because now, hopefully we’ll be able to have just data to use moving forward that’s more standardized than we have ever had before.

So to get started, if I asked everybody right now, let me click if you could compile a list of everyone contributing to maternal health in your agency by 5 pm tonight. Could you do it? Could you identify everybody who’s been trained for contact tracing? Well, the reality is that the workforce picture in public health is very fragmented, right? Each organization uses its own job titles, programs, has its own tasks, and we all use different payroll and HR systems, and even sometimes our public-facing data and information doesn’t line up with our internal systems. So practically speaking, the data that we do have to use for work, for our workforce data needs, is working job titles and job classifications, and none of that actually tells us how the work is done every day, and none of that is standardized across agencies, right? A program director at a local health agency and another one can have completely different capabilities, tasks, and none of that is defined by them being called a program director, program manager, right?

So the easy solution may sometimes seem like mandating some sort of universal system for doing that. But Colorado is decentralized by design, like many other states, and even in centralized states, many systems still lack the granularity to answer unit-level, person-level questions about the work that we do every day. So the other one that I want to mention is that the workforce surveys that we take year after year also don’t provide that individual-level visibility that we need for day-to-day planning, and it’s often suppressed, as Scott mentioned, too, a lot of those different things. So we don’t necessarily have any systems to mandate that we use to better define our workforce.

So, without a shared framework of some kind to describe our program’s tasks, responsibilities, and capabilities, our workforce will continue to remain invisible and therefore under-resourced. So our solution to that is called program capability mapping, which is simple, scalable, and grounded in the real work of public health, as we have operationalized the Foundational Public Health System Services Framework and have mapped over 300 different job types across different settings, from state epidemiology departments all the way down to small local health departments.

So this isn’t just a spreadsheet exercise; it’s more of a schema that aligns people with the programs they serve and the actual capabilities they use day-to-day. So this common language transforms that often abstract concept of the public health infrastructure into something that is tangible and that can be visualized, mapped, and strategically supported.

So on the slide here, we can see that it’s centered around those Foundational Public Health Service areas and then those Foundational Public Health Capabilities. And the goal really isn’t to label people, but to better describe the work that we do. So overall, we have built this way to organize data of about ourselves and our agencies so that we can better understand the technology that we use, the policies that influence our work, and the resources we provide to our communities. And this core organization is what we need to be able to move public health into the future.

This slide is busy, so I’m going to sit on it for a while so you can look at the text, but this is just a closer look at that mapping. Obviously, that mapping is held within us in a spreadsheet, and it’s a lot harder to try to show you a spreadsheet, right?

So, I tried to break it down like this, but we start with the Foundational Public Health Service areas, including, let’s look at just the top maternal, child, adolescent, and family health, environmental health, access and linkage to care, right?

So we start there, and then each Pro, each of them, has program and subprogram areas nested within them. For example, WIC is nested within maternal and child health. Obviously, SNAP is an access and linkage to care. We have WISEWOMAN in chronic disease, injury prevention, and behavioral health promotion. So there are hundreds of programs and sub-programs, if not more than hundreds. It is all nested within those different foundational service areas.

And then we move on to Foundational Public Health Capabilities, and those include the assessment and planning, communications, organizational competencies, and emergency preparedness response. And then we also added additional important services there to ensure that important fields like clinical, technical, community, and education services are all included in those Foundational Public Health Capabilities, and then occupational descriptions are what is nested within those capabilities, and these provide a everyday vocabulary, such as epidemiologist, community health worker, policy analyst and preparedness liaison, so that teams can then identify and recognize themselves within this map. So any of these occupational descriptions can, and then the foundational service areas and programs can be combined with each other to describe the work that we actually do.

So here on this next slide, I’ll walk us through a quick example with this like an arrow graphic on the top. So, we know how you start with the capability in program mapping is you begin with basic HR data, which doesn’t have to come from HR. It can be just a roster, really, if you have a person’s name and then their job title, that’s you can start with that to do this mapping.

So let’s use Clark, for example, who works at a local public health agency, and their job title is case investigator. So now we want to move on and connect them with their service area. Well, Clark works with the community within the communicable disease prevention service area, and now we connect them to the program areas that they might work in. They might have worked in multiple program areas, and Clark specifically works in and supports COVID-19 and Measles response, which are within the respiratory program area that is nested within communicable disease prevention. And then we add the capabilities that they use. Clark. Clark uses the assessment and planning capabilities and does data collection and data distribution within their role as a case investigator.

So that’s program capability mapping. We’ve mapped the program that they said works in, and then we’ve mapped the capabilities that they use to do that work. So now we can create a number of different, more descriptive, detailed role descriptions for Clark, like even just a COVID-19 and Measles Case Investigator, which easily describes the work that Clark does to a whole different level by just having that little bit of data. And that’s just one way to use that more standardized and detailed data, which is to give them a more detailed job role, right? And what that does overall is give us this massive grouping of data that can be used in the future for whatever we might see fit, right? So within different teams, you might use this program capability data to do workforce assessments, right? To do workforce planning in the future, to see where gaps are. Why do so many people work on the same program yet they’re all doing only one capability? What’s going on there? Why is it like that? Right? So now we have data that prompts questions that we can then try to answer.

So that program and programming capability mapping allows us to switch from using basic job titles to using detailed work descriptions for our workforce data. And we piloted this framework at the state and local levels, and I do want to say it was messy for sure, but that was really because we were trying to integrate differing data, HR data, right? So it’s coming from all these different places. It looks completely different, and where it got messy was when we were trying to put it all together. But that’s where this whole framework comes out, is that no matter what source of data I’m getting it from, I can put it through this mapping, and then all of a sudden it is this standardized, very easy, very understandable data that I can use and put into other things to use.

So something I also want to know is that a single person can often will span multiple programs capabilities, right? Clark doesn’t have only one capability, right? It’s probably that Clark has a whole bunch of them within their role as a case investigator, and they also might not just be a case investigator, let’s say Clark is the lead case investigator. That opens up a whole other grouping of capabilities that this person probably has with supervising, management, and administration, right? So that within this program capability mapping, it captures that reality and that people can have multiple roles, and this allows us to show that, and it makes those invisible roles that some aren’t typically tracked into something that we can actually see and therefore analyze in the future. And when you map at that level, it breaks down agency structure into seeing the actual underlying public health infrastructure and why we have gotten to this point in our workforce, and how we have gotten here with how many roles and different capabilities we all probably have.

The last thing for this slide is that framework. This framework doesn’t require any new software or tech or anything like that. It’s just a common language that anyone can use at any scale to understand their workforce and their infrastructure. It really can just start with one simple Google Sheet, an Excel sheet, which I’ll show you a little bit later, how that looks, and it can be super simple. You can just go map yourself today, and it’ll show you a bunch of insights into how much work and capability you probably carry.

One thing is that we often treat workforce data like a tech problem, but what’s actually missing is a shared understanding. So most of our data in our daily work is actually in our daily work, right? It’s in job titles, org charts, funding codes, and it’s all in these different siloed systems. But as that data currently stands, none of it really helps us see how people are actually working together to deliver public health. Without this simple shared structure, we can’t effectively use our existing sources. So this programming capability mapping process provides us with that shared common language to describe the real infrastructure, and clarifies how to describe infrastructure in a way that reflects the reality and not just these organizational boxes.

And the implementation is a lot simpler than most people expect. You pull your HR data. It doesn’t have to be HR data. Again, it can just be like a roster, something like that, and with your job titles and basic demographics. And then you will add the nine fields aligned with our foundational public health taxonomy. And then you will collect additional data about what programs people work in, what capabilities they use with whatever methods best fit your team, right, to not add any extra burden. You know, an email might be all that your team needs to get this data really quickly. For other people, it might be a little bit more intensive, but that’s where that, you know, unique adaptability with this framework comes in handy. And we want to encourage making sure that you’re using your current tools, right? Whether you use an Excel group or you, as you use a Google platform, whatever works best for you, you can do, and even with just a handful of staff, mapped patterns really do start to emerge quickly, including duplication gaps and siloed teams. And just as important, you’ll be able to start seeing where some data systems, as well, aren’t being used for planning and where people are improvising to fill systematic, systemic gaps, and therefore you’ll be able to recognize those larger issues that might be happening within your infrastructure.

So when we’re trying to talk about, like, what this dataset will be able to enable for everybody. There are a lot of different options, right? There are some of the things that we’ve been able to see, which is that at the staff level, it gives you a lot more visibility. You can finally see what people are doing across programs, including temporary roles, cross-change staff, and those, like other real duties assigned that we all have. And you can measure capacity by capability, not just by title or FTE. And then at the systems level, you can see how your infrastructure actually functions, where roles connect to funding, policy, and platforms, again, identifying those systemic issues.

And then at the community level, it creates transparency. It helps you describe and explain your workforce to partners and funders, and show how your people actually support critical programs in an easy, digestible way, and not just by title, but by actual capability. And again, there’s going to be many use cases that teams will find for themselves, just by starting to use this framework with their own data and being able to see, like, Oh, we’ve never had this data to look at before, and now I’m getting all these ideas of what it could be helpful and useful to use.

And we know that workforce like sharing workforce data can feel very risky, and that hesitancy is very valid, especially where trust has been broken, but the reality is that we share much more sensitive data on a daily basis, whether that’s through our personal accounts, on our phones, even with our workforce, with our employers. And therefore, workforce data is no more private than the data that we have on our social media accounts, really. And when we don’t collect that workforce data, or we can’t use it, then we just continually become more and more invisible, and we lose out on funding, staffing, and planning support, because we can’t show what we do. And what started as a workforce enumeration pilot, we’ve obviously created come into this idea that it’s something bigger. It’s a foundation for organizing public health infrastructure as a whole, because we don’t need perfect systems or complete alignment to get started. This is just a common, scalable language we can start using, even if it’s just individually, to begin to understand our infrastructure and then hopefully move forward into the future, using it more consistently across the board.

And then. We all know public health is changing very fast, right? The politics, the funding, the leadership. And right now, we can’t wait for the perfect conditions to understand our workforce and collect workforce data. In all honesty, we’re probably in one of the most dire places for accurate workforce data for what our workforce does, because I could not be here tomorrow, technically, right? Like, how many people are in a similar place of maybe not being having their positions tomorrow, right? So, when is the right time to collect what our workforce does? Then now, when we are hoping in the future, if something happens, we can bring them back, right? If we can’t bring people back, if we don’t know what they were doing, and if our whole system collapsed, what are we supposed to do?

So by using a framework like ours, it makes it a lot simpler to describe the work, right? It makes it standardized. It’s classified, classified in a very easy to understandable, understand way, because it’s based on this model that’s already been widely accepted, right, the Foundational Public Health Service model that we all widely accept, and that’s why we used it as the main foundation. And now, with everything that we’ve covered with data, schema, mapping, everything.

It really all comes back to one truth that I want to make sure we recognize: people are the infrastructure. It’s everybody in this room. Public health doesn’t run on software. Love, love old software. People who like sponsored this and all the money there. I appreciate it. I appreciate them, you know. But if it wasn’t for us here right now, we wouldn’t have any public health at all. We are the infrastructure. So when we can’t describe what we do day to day and to other people, and show that, then there’s no way we can actually get the support we need, no matter what that support looks like. So this framework really allows us to do that easily, and it doesn’t force any org chart. It’s not some one size fits all platform. You can start using it with one person, one team, and scale as you go and see how helpful and useful it is to you, and then tell us if it’s not, too, please.

So yeah, this initiative really just serves, I think, as a big step toward recognizing what we need to be doing as a workforce together to raise ourselves up and bring ourselves into the light of what public health is. So lastly, just to kind of conclude mine a little bit, we built this framework to be shared, to be tested, and to be adapted, and it really is ready now. So the schema, the tools, templates, and guidance are all freely available. No logins, no software, no approvals needed at all. You can start this afternoon if you want. You can just add those nine fields I talked about, there are nine columns, into a Google sheet and start mapping your way.

And what we need right now, from all of our PHIG partners and all of our other PHIG peers, is really people to start who see the value in this and bring it into their organizations, agencies, and people willing to try it and offer feedback so we can keep improving. And our goal is really to create this shared understanding and a common scalable language to organize and understand public health infrastructure at any scale. And now that we have that common language, it’s here, and it’s ready to be used. So please start where you are, use what you have, and help us build something better together.

If you’re interested in co-designing, piloting, or aligning your work with this framework, the QR code is on the screen. It will direct you to our Google folder, which is 100% open and available and contains all the details you need to get started. And please don’t hesitate to reach out to us, if you like, after the session or via email too. We’d love to collaborate or even just get your feedback as well. The taxonomy is something that we have gotten a lot of feedback on from people in Colorado, but it’s not something that we’ve been able to get a lot of feedback on from people outside of Colorado, which would be wonderful, because I’m sure that there are program areas that we did not include. So even just being told like, Hey, you didn’t include this and then we’ll add a new row would be awesome. So thank you so much, and I’ll hand it off.

Dr. Emma Spencer:
Great. Hi. Good afternoon, everybody. I’m probably going to take a little bit of a different track. I’m going to talk a little bit about how we’re leveraging data from our recent assessment and building this repeatable, replicable framework as it relates to data literacy in the Florida Department of Health.

This is a framework that can be used for other workforce areas, not just for data literacy, but we’re really trying to leverage the information from this assessment to plan for having a more data-centric, data literate, and competent workforce, and then create specific trainings and leverage the pathways for our specific data personas that we identified through this assessment.

I’m Emma Spencer, I’m the Division Director for Public Health Statistics and Performance Management ofthe Florida Department of Health, and thanks so much for inviting me to speak today. So I’ll just quickly go over a few things. So we will I’ll talk about the project overview, describe our competency and skills framework, the assessment design, and the gap analysis results, and any next steps that we have, and then the further application of this replicable framework that any of the tools that we have, maybe you all might want to use one day.

So, just a little bit about the Florida Department of Health. We are an integrated public health system. So we have one state health office in Tallahassee, Florida, where I’m situated, and then we have 67 county health departments that serve 23 million people in the state of Florida. We, like I said, we’re integrated. So we have a state Surgeon General, and then all of our policies, procedures, and everything like that are kind of passed through the state health office all the way to the local county health departments, which provide those local services that are required by statute.

So we embarked about maybe 18 months or a little bit more, on a data modern part of data modernization, workforce development effort. If we’re modernizing our systems, we need to really invest in our workforce. And so we identified a kind of project that we wanted to facilitate over the next couple of years, and this was the first phase of that project. And so we wanted to do a skills assessment, which would analyze our existing infrastructure.

We would use position descriptions and identify our data workforce or the functions around the workforce to use data on a daily basis, conduct focus groups, and then launch a data literacy assessment, which would really help us to identify any gaps in our current workforce, and then identify any training recommendations for our workforce as a whole. And this was an enterprise initiative, so we’re looking at nearly 13,000 workers at the Florida Department of Health.

We weren’t focusing just on data users, like epidemiologists, analysts,and data scientists. We were focusing on the enterprise as a whole and the whole department. And eventually to develop a training plan to address those training needs that we would identify from the gaps, deliver a very specific training around data literacy and other specific trainings for particular personas that we identified, develop a human resources toolkit that would help supervisors to identify their next data users or hires around specific data functions that we identified, which would also include identifying key performance indicators and other competencies around specific personas, and then develop an evaluation strategy around that training plan.

So this is our main kind of framework for what we wanted at the continuum of how we did this assessment to really focus on modernizing our workforce as it relates to data literacy. So we took a lot of the information that we already had on hand, like I said, and we identified some personas and used our position descriptions to identify specific functions.

One of the interesting things that we have is that we don’t have a position description for an epidemiologist or a data scientist right now, so we had to really dig into the data that we have to identify specific personas around data-related functions. We looked at different department levels. So we looked at the divisions that we have in the department. We looked at our 67 county health departments and offices, and looked at the three different tiers, so entry-level, mid-level, senior level management, and then we did a whole assessment to identify current skills and competencies. And then focused on three areas.

We conducted interviews and focus groups to build out the understanding of the current state, essentially, of our skills and competencies, and then start building out those personas, of which I’m going to talk about in a second. And then we did conduct 22 focus groups across the whole agency. There were over 196 individuals who were participating in those focus groups, and they came from various backgrounds, from epidemiology, leadership, county health department directors, and just a workforce development and human resources folks who helped us to determine those relevant functions and use cases associated with any data literacy expectations that we had.

Then we did actually conduct an assessment, but the assessment was built upon the understanding from what we gathered from the interviews and the focus groups, and then we used that. We had two reports released. One was the analysis report and the findings from the gap assessment, which really helped to identify those training needs and also what was urgently needed, and helped prioritize some of those issues that we identified through the assessment. And then the three main deliverables that we got out of this, and I’ll talk a little bit about these later, but we really focused on identifying a specific foundational data literacy, 101 course, and then training pathways based on specific personas. And then also that HR toolkit that I just mentioned, which provided interview questions and preferred skills, knowledge, and abilities for various different data-driven positions, as well as position descriptions that we may not have had previously. And also created a smart expectation bank for those specific data roles.

So I’m going to talk a little bit about how we built out the competency and the skills framework, and we identified that we used a lot of industry standards. So you’ve probably seen the Public Health Foundation’ss keep Core Competencies for Public Health. You’ve probably also seen the CSTE Applied Advanced Epidemiologist Competencies. And CDC has a data science one too, and then there were a few others that we used. So we took some of those industry standard frameworks, and we refined them based on the needs of the department, as well as any input that we received from focus groups and the interviews, and we identified these six main data-related compass competencies throughout the agency.

So the main competencies that we found that relate to data literacy were data management, data interpretation and visualization, security, privacy and ethical data use, public health informatics, research methods, evaluation, and advanced analytics. And then with each of those competencies, there’s a set of skills. I’m not going to read all of these, but these ones relate a little to the data management competency. And so we built out this, this whole matrix to, you know, identify the inventory for these specific skills. So we also wanted to identify proficiency levels to see exactly where people were, you know, so we have to understand where you are, so that we know where you can get to.

And so through this assessment. So we haven’t actually done the survey yet. This is all pre-survey. This is building out the framework. And we identified these four proficiency levels: acquiring, applying, guiding, and shaping. And they included a progression in certain behaviors, skills, and any use of tools and technology as you go through this continuum of competency levels. And so I’m not going to read everything that’s on the screen, but in essence, acquiring is that basic knowledge, so you’re at the beginning and the understanding of any techniques and concepts as it relates to data and the use of data, and this can be from any level and any type of position. All the way to shaping, which is that you become a recognized authority and that you are an expert, and you are able to shape the direction through specific strategic insight, adding any value to the use of data, and making those data-informed decisions across the agency.

So I talked a little bit about you probably asking, why? What’s she talking about when she mentions personas? So we have three that were identified. We needed to create personas because, as I said, we have an agency of more than 13,000 individuals, and we needed to group people together so that we could really build out the framework. And that’s why a lot of the information and data we gathered through our focus groups and key stakeholder interviews, and any data we had on hand. We basically were able to identify three categories to classify our workforce, which were based on any characteristics and needs across the agency, based on their data interaction and usage.

So we identified a data consumer function, which is kind of the main user of data. They might make decisions when using the data, and they can create value for critical objects around data and in their everyday roles at the agency. We also have a staff development function, because not everybody is a true epidemiologist or a data analyst, but you’re going to use data. I mean, we just heard two really great presentations on the use of data for staff development and understanding the roles of the workforce as it relates to data. So that was another key persona we identified.

And then we also identified the data providers. So those are the ones who make data available to the data consumers through various different avenues. And as I stated, the main reason why we developed these personas was to enable the clustering of the workforce when we were able to create these training pathways across such a large agency. And then the course, as stated previously, is aimed at entry-, mid-, and senior-executive levels across various folks at different levels of the agency. And you know this, this also, like I said, really is going to help us improve our learning experiences if we’re able to tailor it to those specific personas by classifying people into these three major buckets.

So, throughout this assessment, we were able to identify the functions and align proficiency with each. We identified 45 data-related functions, and then we also have a functional use case that goes with each one of those. And we were able to do that to help us capture how employees across the agency interact with data across various roles in the agency, and to provide examples of the functions. Obviously, we have accounting and finance. We have our administrative and office support, clinical and community health communications, customer service, data architecture, governance, etc.

We also, as part of this, this huge matrix, which we’re quite happy to share, we developed this proficiency expectation matrix, which was really to help kind of clarify where our expectations were for these specific data-related functions across the three levels that we talked about, and also include their competencies, their proficiency levels, and then have the example use cases associated with those. And this was largely driven by our stakeholder interviews and focus groups, and any other data to help kind of create this, this matrix. And it was also going to be the basis for helping us following the assessment, to be able to tie everything back from the assessment to this proficiency matrix.

So I’m going to talk a little bit about the assessment, which is primarily the survey. This is all part of the same framework. So the survey has six major sections. The first one is primarily the introductory section. It’s trying to identify who you are and get a general awareness of your understanding of data literacy and governance, and then the next set of sections are really looking to determine what organizational level you fit into. Are you mid-level? Are you entry-level? Are you senior management? And as you fill out the survey, we funnel you based on your answers to the next set of questions. So if you were senior management or executive, you would be routed to a set of specific shaping-level questions first, so that we could kind of understand our our executive level leaders also shaping the strategic direction as it relates to making data decision, data driven decisions, and so we wanted to have that first, and then they would go on to do the standardized assessment.

And so everybody received the same set of standard questions. And that helped to assess whether or not individuals were at their applying, acquiring, or guiding levels, and then also assess some of their proficiencies based on some knowledge, knowledge-based multiple choice questions that we had in the survey. And then, once you are finished with the standardized portion, you would be directed to a final section. And this is really getting those open-ended responses that we would be able to do any kind of sentiment assessments on as well, which helps us to understand people a little bit more if we’re able to ask those open-ended questions. And so again, in this area, we were able to ask anyone to get any general perceptions and attitude around data, and that helps us to build out the data literacy, understanding, and where we’re at with the baseline.

So how were these scored? So we took all of the scores from the assessment, which were really great responses. We got over 5,000 as I said, this is an enterprise initiative, so it was nearly a 40% response rate, which was pretty good for the agency, especially considering the survey was only open for less than three weeks. And so it was, it was really good to get that level of response. And we did have a lot of communications go out with it, and we did provide FAQs about how the data would be used and what it was for. And so we basically were able to identify and align the data that we had with the proficiency matrix and group, as I said, these areas together, and we’re able to do this by division, by office, and by county health department. And so each individual office division would end up with their own responses, so that their training could be developed that way.

Okay, so what can we take away from some of these Gap Analyses? Well, for the department, we identified that most of our executives met expectations for shaping proficiency level, but our mid-level supervisors demonstrated some gaps, particularly in performance improvement, supervisory, and accounting areas, as well as in data management and advanced analytics. So we’re bridging the gap by implementing targeted training. I said itis at the division and county health department levels to address these particular skills.

So next steps? Well, we already have our roadmaps for all our personas built, and we have a foundational data literacy training course built, tested, and approved to include in our employee onboarding. So when you are new to the Department, you will go through this data literacy training, and then we do have mandatory trainings annually, and so this year, it will be part of our annual mandatory training for all agency personnel. So that’s actually quite a big win for us. I think we’re going to be excited to see the data from that; there are tests and scores, and then we’ll evaluate, and then we will repeat in the next year. And then we also have our toolkit is all ready and built, and we’re happy to share that with anybody who is interested, and we’re excited to kind of get this trait. The next step in this project’s next phase is to really implement the training paths for those different personas.

And then just one last thing. So I said this was a repeatable framework, and I know I spoke a lot about data literacy. But we can apply this beyond data literacy specifically, and it offers opportunities to kind of span across various entities of different sizes. The framework is pretty simple, you know, we define our focus area, and we can use those industry-specific competencies to build out the assessment and the proficiency matrix, and then do the skills assessment, analyze the information, develop that gap analysis, and then work to build your training paths based on that. And so that’s it for me, and thank you so much.

Whitney Magendie:
Thank you so much to our presenters for today. We’ve got a bit of time for Q&A. If folks in the audience have questions, just raise your hand, and my colleague Anthony will run the mics for us.

Arnaldo (Audience):
Arnaldo from the Massachusetts Department of Public Health, Workforce Director. I have a question for Colorado. First of all, all presentations are impressive, but the work that you described sounds massive. I have a question related to how you develop the capabilities, very specific to Colorado, that, in itself, already feels like a big project, right? Are you able to share them, and how did you collect and assign those capabilities, right, to each one of the roles, and who in the organization, right, you tackled to do that work, like, was it like leadership? And how did you do that? So the capabilities list, and then how do you assign it to roles?

Azalee Hoffbauer:
Perfect. Okay, so the capabilities. So Colorado in itself took the Foundational Public Health Services model and their capabilities model right, and then it kind of adapted it a little bit for themselves, and that’s where that original set of foundational capabilities came from, like the assessment and planning, communications, all of that. So that was a framework that was already created from Colorado, and then we also assigned CALPHO capabilities to it. So that made it even first. Oh, yes, CAL, do you know that?

Steve Holloway:
The Colorado Association of Local Public Health Officials.

Azalee Hoffbauer:
Okay. Yes, all the abbreviations and that sort of thing. So we used both of those capability frameworks to create to implemented it into the framework. So those have their own sets of like, toolkits and stuff that they’ve created on their own for those, so that is a great reminder to include that in our research, inside that folder, to explain how those capabilities were chosen, and everything, because that wasn’t our work. We saw those, and we’re like, oh, perfect. We have identified capabilities for public health workers that are foundational. So we’ll make sure to include those in the taxonomy.

So we’ll make sure to go back and like, cite where those came from and how those worked out. But a lot of them, in how I understand it in the taxonomy, is like with assessment and planning, we have multiple different CALPHO capabilities then. So it goes, assessment and planning is a Foundational Public Health Service capability. And then within that, you have another set, which is the CALPHO description capabilities, and there are multiple of them. And then all those lead back to the roles that those are connected to, right? So, and then you have the overarching, so it’s all nested in it all together, and that’s what leads you to your general occupational description. So an epidemiologist is in assessment and planning, but it’s not in the Clark’s example, which was the data collection and distribution. And an epidemiologist does assessment and planning, but does it specifically in analysis and interpretation, right is where, like an epidemiologist, will fit in, but they also fit in generally to that assessment and planning section. So it’s, yeah, it’s very all nested in it. But thank you for bringing that up, so we can get a better description of where those capabilities have come from and why they were identified as foundational. So thank you.

Jae Basiliere (Audience):
Hi everyone. All three of your presentations were so great. I was frantically texting ideas to my colleagues while you were all talking. It was really inspiring. I’m Jae Basiliere, they, them, pronouns. I’m the Workforce Director for Vermont, and I actually just have a comment for our Colorado friends. I mean, first, Steve, A+ mentoring. I love how you were just like, I’m gonna let my colleague who is new to our field and just did the work, show it off. I loved that. I just wanted to let you know that, like many jurisdictions, we’re trying to contingency-plan right now for what a devastating loss of funding might do to us, and we have been using your framework, but not in the way I think you designed it or intended it to be used. So I’m really excited to go back and be like, Hey, this is what they intended us to do with this. Let’s try that. But yeah, it made it all the way to the East Coast, and we’ve found it really productive. So thank you.

Azalee Hoffbauer:
Thank you for using it.

Ari Whiteman (Audience):
Hi, I’m Ari Whiteman. I work at ASTHO. I’m a Senior Advisor for the data modernization workforce. I had a question for Florida, so I guess a two-part question. First, how were the actual trainings made? Like, who did the actual trainings to create them? Like, were they PowerPoints? Were they videos? What do the actual materials consist of? That’s number one. And number two, we’ve actually had a few TA requests at ASTHO about how we can improve jurisdictions’ ability to use data in storytelling, which I think is like the next step, once you understand the data, how can you use it to tell a story? And I was wondering if you thought about sort of going that one step further to try to potentially train, I guess, maybe even a subset of your staff, to once they understand the data, what do we do with it? You know, how can we use this to tell a story?

Dr. Emma Spencer:
Yeah, thank you. So the first one is our foundations of data literacy courses created in articulate so it’s just a software, AI software, and that’s it’s dynamic. So there are prompts and questions that people can click around in the training. And so that’s one of the tools. For the other training, we’re still building out many of the main training pathways. We have the pathways. But we also looked at it because we can’t augment every single thing that everybody would need. So we looked at Udemy, and we looked at LinkedIn Learning, and we looked at other kinds of free training that were available, and put those in the curriculum as well. And then the core pieces will be the ones that we’re developing in phase two of this project. And again, we want those to be more dynamic, so that people can actually learn and not just, you know, ignore, you know, have a kind of a webinar going on. The second piece about storytelling is actually really interesting, because when we built out the foundations of the data literacy course, one of the things that we looked at was another competency: communication. So we didn’t necessarily see storytelling. Well, I think we did, actually, but we want people to be able to tell the story with the data. So it’s more work to be done, but we did consider it in the foundations of data literacy calls.

Whitney Magendie:
Any last questions? Thank you to our presenters. Thank presenters. Thanks, everybody.