June 16, 2026 DMPS Board Meeting & Work Session

001[music] >> I would like to call the school board meeting for Tuesday, June 16th, 2026 for the Des Moines Public Schools to order. Ms. Rios, would you please take the roll? >> Yes. Ms. Alonso? Ms. Campos? >> Here. >> Mr. Dix? >> Here. >> Ms. Hopkins? >> Here. >> Mr. Howell? >> Here. >> Ms. Maritano? >> Here. >> Dr. Mayberry-Maze? >> Here. >> We have quorum. >> Thank you. The first item of business is the approval of the agenda. May I have a motion and a second to approve the agenda? >> Mayberry-Maze moves. >> Campos seconds. >> We have a motion by Mayberry-Maze and a second by Campos. Is there any discussion or does anyone have any questions? Board members, please cast your votes. >> [snorts] >> Mr. Dix? >> Yes. >> The vote is

002approved 7 to 0. Next on the agenda is public forum. Miss Hopkins. >> [snorts] >> Thank you. >> We provide speakers at the forum an opportunity to address the board for up to 3 minutes regarding items that are not on the board agenda and up to 5 minutes for an agenda item. If anyone wishes to speak during any portion of the business meeting, please email [email protected]. The same speaking protocols apply to all individuals who wish to address the board. Speakers will be called individually and we ask they state their name and zip code prior to providing comments. Students and minors do not have to provide their addresses. Guests of the board are encouraged to speak to matters related to the programs, services, and operations of DMPS. Please remember that Iowa law prohibits us from discussing

003specific employees or their job performance. We ask all speakers to avoid references to personalities or character attacks as those type of comments do not contribute to the purpose for which this public forum exists. The use of inappropriate language will not be tolerated and individuals choosing to do so will be gavels out of order and asked to excuse themselves from the premises or escorted out by district security. As a reminder to the board and public, the board will not engage in a discussion with public forum speakers as it is our obligation to discuss only matters posted in the public agenda for the sake of transparency and opportunity afforded to all community stakeholders. Matters requiring attention will receive appropriate staff and or board follow-up at a time after the meeting. Thanks for your participation. We appreciate your

004feedback and input. Thank you. Miss Rios, do we have any speakers for the public forum? >> Yes, we do. First up, I'd like to call Britney Ann Roseboom. >> My name is Britney Roseboom, zip code 50316, and I'm here today. I got backed up, so I missed some board meetings, but now I'm all caught up, and I noticed something that I would like to lift up. Sorry. I am hearing consistently something across meetings. Multiple groups are asking for the preservation of services that directly impact students' safety, stability, behavior, and learning outcomes, >> [clears throat] >> as well as the safety of our staff. Increasingly, these conversations and proposed changes are also affecting some of our most vulnerable students, including those in day programs and students with 504s and IEPs. >> [clears throat] [snorts] >> Teachers

005are asking you not to eliminate essential services. Students are asking you not to lose teachers they rely on. Mental health professionals and community partners are asking you not to dismantle programs that, in some cases, have supported students for over 40 years. And many of these supports are provided at low or no cost to the district and are provided through volunteers and community-based services. I operate under the belief that all of you want the best for our district, students and staff alike. >> [clears throat] [snorts] >> And tonight, I want to extend a direct invitation to the board and interim superintendent to consider participating in the trauma-informed training I previously shared. These decisions sit at the intersection of education, mental health, safety, trauma, and classroom functioning. And I am concerned that without a shared understanding beyond

006a baseline of the interpersonal and regulatory challenges our community faces, even well-intentioned decisions >> [clears throat] >> can unintentionally misdeliver realities of students, staff, and families. >> [clears throat] [snorts] >> We are asking to be heard because schools are more than budgets and buildings. They are our communities. And I do want to close by saying thank you. I genuinely appreciate your service to this district, and I hope the community input you are receiving continues to be fully considered as you move forward. If you would like some business cards for that uh class, just let me know. I have some. Thank you. >> Thanks, Britney. >> Next speaker is Daisy Campos Gaitán. Next speaker is Ashley Crawford. Next speaker is Zeeshan Badani. >> Hi, Zeeshan Badani here. Zip code 50314. Um I was going to come

007tonight to talk about uh my status as the Iowa Democratic Party disability chair, and uh this weekend at Iowa Democratic State Convention, we passed an amendment to say that voucher schools, charter schools shouldn't be able to kick out students that have an already established IEP or 504 for any reason defined within those plans. Um which I do want to talk about, but I had some community members bring up uh some transportation issues, and I asked my wife, the principal of Hoover, about it. She's like, "You have no idea." So, I looked into the DART online boards and commissions and stuff, um their changes, and I've been kind of following it along by way of DART emails since the reimagining process started, as well as uh being a part of the Engage Des Moines cohort that

008just ended a few weeks ago. And I've also checked out the lovely book, Human Transit by Jarrett Walker, which is uh this author is the guy that runs the consulting group that DART hired to do their reimagining program. And I'm just finding a lot of issues with the DART plan it where it relates to the Des Moines Public Schools contract with their school tripper routes. The old routes had local dedicated routes like 14, 15, you know, five, all this stuff. And then they had these school tripper routes, often designated with like a letter letter, so like 5 AB, you know, 14 AC. And these routes, you know, were part of the 2023 contract contract for 5 years between DMPS and DART to provide, you know, direct drop-off from students uh for students from DART stops

009to the you know, the front door of uh middle and high schools. I've been asking around, going to open houses, um you know, emailing DART. And it seems like the assumption is that the DART changes that happened due to reimagining get to preclude the contract and say we're not taking these kids to school anymore. Uh when asked about, you know, the 80% service level compared to last year, that's not a reduction in service level. It's a reduction in the number of stops they're doing. Um from an IT compliance tech support side for my clients, you know, if we fire some guy that's not performing well and we're saving money internally, that doesn't change the level of contract or our service level agreement that we provide to our vendors, right? Like we still have to show

010up even if it's the most inconvenient thing because it's part of the contract. And the fact that there are now stops instead of dropping kids off right in front of Hoover and Meredith, of school uh two schools that share one building, um the closest stops now are at the Casey's on Merle Hay and Donut Hut on I believe uh Douglas Avenue, which are about 0.7 mile walk from the schools. Like a middle schooler walking 0.7 miles to their middle school in addition to whatever changes are from their local DART stop, from their house. Um, the running patterns, like the cycling of the scheduling. It doesn't make sense to me. And again, I troubleshoot IT all the time. I troubleshoot RFPs and and the cybersecurity policies. And it's not always like a malicious thing or a

011or a hack or a bug. It's just like a feature gone wrong, right? That we need to redeploy or do an update on. I think some of the stuff in the Dart plan needs an update. It doesn't even match some of the best practices in this book. Um, Dart services served us like 32 million rides last year. The Des Moines Metro is >> 30 seconds remaining. >> uh a quarter of a million people. And over a year and a half, they only got like 1940 responses. When we have about 450 kids from Hoover and Meredith who are no longer able to get to their school. My concern is for the attendance rates, the the results. What What does it mean for our students when it's easier to take a bus to go to a job

012than it is to high school? Um, that's my concern. I'd love to talk to anyone further about this. It even conflicts with what the book's best practices are. So, again, I really am looking forward to the enforcement of that contract with Dart. Thank you. >> Thank you. Next speaker is Caitlin Hanson. >> Good evening again. Caitlin Hanson, 50237. Um, I I am a uh focus um special education case manager. And uh I am here again tonight um just to lift up again that um Des Moines Public Schools has an amazing group of really experts within district-wide programs. Um, staff that have been around for 30 years, 20 years, staff that have retired and then come back to be program subs because they have truly believed in our different programs purposes and missions. Um, and that

013uh uh listening and hearing um, obviously are two different things and that uh these experts that we have um, are listened to and not only that, but um, their perspectives are taken into account in our planning, [clears throat] especially around Reimagined, especially around um, the changes that are coming to the district-wide programs. Uh there's, you know, obviously a difference between collaboration um, in meetings versus presenting options. There's a difference between counseling and psychotherapy. Uh there um, is uh difference between preserving programs versus asking the same number of adults to serve twice as many students. Um, and uh there's a difference, speaking of Dart, between access right outside of the door versus crossing Douglas by the VA. If you know, you know, um, especially in the wintertime, that's not a great intersection. Um, and again, I

014just bring this forward because I truly believe that everyone in this district is working for what's best for our students and that um, we all want to do what's best for our students. So, thank you. >> Thank you. Madam Chair, those are all the speakers. >> Thank you. Next on our agendas are district recognition. Superintendent Smith, will you introduce the recognitions today? >> Yes, good evening everyone. Uh I'd like to begin tonight by recognizing our state champion Hoover Huskies BOYS SOCCER TEAM. >> [applause] >> UH HOOVER HIGH SCHOOL CAPTURED THE 2026 Iowa Class 3A State Championship, becoming the first NPS high school to win a boys state soccer title. The Huskies showed incredible determination, rallying from a two-goal deficit before winning a dramatic shootout 4-3 to bring home the state championship trophy. >> [snorts] >>

015While coach John Rubino and his players are competing with their clubs teams this summer and unable to unable to join us tonight, please join me in congratulating the Hoover Huskies on an unforgettable season and a historic state championship run. There we go. Hoover Huskies, good job. >> [applause] >> I'd like to think that my soccer skills had a lot to do with that, uh but we'll just keep that going. I'd also like to recognize Roosevelt High School team who qualified for the state tournament this year as well. That made DMPS one of only two districts in Iowa with multiple state qualifiers in soccer in the soccer's state championships. >> [snorts] >> Um next, I'd like to recognize Chad Donahue, welding instructor at Central Campus. Chad was recently selected as one of the just 50 finalists

016nationwide for the 2026 Harbor Freight Tools for Schools Prize for Teaching Excellence, one of 50 nationwide. >> [snorts] >> This prestigious award honors outstanding skilled trades educators in public high schools across the United States. Chad is now in the running for one of 25 prizes totaling $2 million that will be awarded this fall. His selection as a national finalist reflects both his dedica- dedication to students and the high-quality career and technical education opportunities available in Des Moines Public Schools. So, please join me in congratulating Chad Donahue on this remarkable achievement. >> [applause] >> And finally, I'd like to recognize our communications and public affairs team. The team recently received a Golden Achievement Award for its work in engaging the community during the development of reimagining education and directing communication efforts leading up to the successful

0172025 bond referendum. >> [snorts] >> This recognition reflects a tremendous amount of planning, outreach, and collaboration across the district. So, congratulations to Phil, Amanda, Adam, Kyle, John, Amy, whose work earned this recognition. We also extend our thanks to our partners at CISO for their support throughout the process. Also, please join me in congratulating our communications and public affairs team for this national award. >> [applause] >> And that concludes tonight's recognitions. >> Thank you. We will move on to consent agenda. As reminder to the board and public, the board will not engage in discussion or deliberation with the speaker regarding comments made to agenda items. Discussion and deliberation will remain among board members at the board table with speakers' comments informing the board's discussion, deliberation, and determinations as deemed necessary. Ms. Rios, do we have any

018speakers? >> No speakers. >> Thank you, Ms. Hopkins. I believe you have the consent motion. >> I move that the board approve the consent items in accordance with the recommended action for each item on the consent agenda, including bills previously authorized, certified, and approved for payment by the board secretary in the amount of $3,306,259.77. >> Is there a second? >> Second, Mayberry Maze. >> We have a motion by Hopkins and a second by Mayberry Maze. Is there any discussion or does anyone have any questions? Board members, please cast your votes. Mr. Dix. >> Yes. >> The consent items are approved 7 to 0. Now, we will move on to our other items. The first item E1, consideration of sealed bids opened and reviewed by the secretary of the board and the municipal advisor. Superintendent Smith,

019can you please introduce the matter? >> I recommend the board consider the sealed bids opened by the secretary of the board and municipal advisor for the school district. >> May I have a motion and a second to approve this item? >> Hopkins, so moved. >> Campos second. >> I have a motion by Hopkins and a second by Campos and I will turn it over to Shashank for discussion. >> Good afternoon and congratulations. I think this is the single largest bond that this district has issued, a hundred million dollar bond. And Suzanne, who is the PFM partner, can confirm whether I think among the school district this is probably the largest bond. >> This this year, yes. >> This year, yeah. >> So far this year, yeah. >> Absolutely, yeah. We went through an S&P rating

020process. Our bonds were rated A. And the total interest cost for us is 4.03. Suzanne will share more details. But it's an interesting process as I had mentioned to a few of you. So in the morning at 10:00 both of us are sitting in her office over there. Seven bids come in. At 10:00 the bids open and at 10:01 we had all the bids over there. So we're looking at all the details trying to make some decisions. And then finally we decided to move forward with Bank of America Securities. Suzanne, please feel free to add >> Yeah. >> to it. >> Yeah, so we we use parity internet bid system and it's a blind reverse auction system where the bidders compete against the each other and they have to provide the lowest TIC to win

021the district's bonds today. And we received seven different bids. They can submit and see if they're winning or not, but they don't know by how much and they can pull and resubmit, pull and resubmit. And so our goal with the competitive sale is to get three bids and you received seven bids today. And it is the largest names in I think he's kind of set the market today. You were one of the largest deals in the market today as well nationwide. Um, the winning bid came from uh Bank of America Securities out of New York, New York. Um, they bid a loan. Um, the second place bid came from Truist Securities, which I think is a holder of your safe revenue bonds. So, I'm sure they're very disappointed they didn't win today. Um, with just

022two basis points or 0. 0.02% difference between first place and second place. So, very close range from beginning to to second place. Um, and then we had a bid from Jefferies LLC out of New York, New York, Morgan Stanley & Company out of New York, New York, Robert W. Baird out of Milwaukee, Wisconsin with a 32 member selling syndicate um, out of uh everywhere from Missouri to Minnesota, Virginia, Illinois, Tennessee, Florida, Colorado, New Jersey, and a many other states probably uh 30 almost 30 different states. Um, second to last place was J.P. Morgan out of New York, New York and last place was Wells Fargo out of Charlotte, North Carolina at 4.19. So, beginning to end just 0.16% between first place, last place. Everybody wanted to win your bonds today. So, that results um the

023winning bid um was uh slightly cheaper than what PFM had anticipated. We like to plan conservatively. So, in the budget we uh we um we we structured the bonds to match the $1.99 tax rate that you approved in your fiscal 27 budget, which allowed us to even have more savings. Um, we put a a little bit more principal in year one and two reducing the final TIC to 4.02% today, which saved over a million dollars in interest for the district and the taxpayers. Very pleased with the results today. And happy to answer any questions anyone might have. Mr. Dixon, any questions? Okay, board members, please cast your votes. Mr. Dicks? >> Yes. >> I think that was a yes. >> That was a yes. >> Okay, thank you. The item is approved 7 to 0.

024The next item is E1A resolution directing >> [laughter] >> the sale of general obligation school bond series 2026. Superintendent Smith, can you please introduce the matter? >> I recommend the board approve the sale of the bonds. >> May I have a motion and a second to approve this item? >> I'll move. >> Hopkins second. >> We have a motion by Howell and a second by Hopkins. Is there any discussion? Do you have any additional? No? >> Uh no, this just directs us to tell Bank of America that they won, they'll start ticketing, and then at your next board meeting you'll authorize all the issuing documents. The big large stack of documents. Um and then you'll get bond proceeds at the at the end in in August. >> July 29th we get the $100 million bond.

025So between now and then is a holding period. So we got today what they call as a million dollar What is the term used for the million dollar? >> Par amount. >> No, the million dollar the amount that you received today. >> The the principal the principal amount of the bonds. >> Yeah. So so a million dollars we received today basically confirming that they are interested in moving forward. The good faith deposit. Thank you. We received the million dollars I think around 3:45 p.m. is when we received that. So that basically means that Bank of America Securities is moving forward. And we should get the full amount by July 29th. >> Yes. >> Very good. Any questions? Board members, please cast your votes. Mr. Dicks? >> Yes. >> Thank you. The item is approved 7-0.

026Congratulations. >> so much. Thank you. >> Next, we move on to our first review for future action items. The item F1 board policy updates series 5 and 600 are available for viewing. Next, we have our information only items. G1 is a list of new hires and G2 is a list of bills paid. And now on the agenda, we have items of privilege. I would like to ask if our board members would like to share anything. >> I would just offer a reminder to all of our recent high school graduates that the federal application or free application for federal financial student aid, the FAFSA, is open until June 30th. So, if you're thinking about college or industry verified credentials or certificate programs or going directly into workforce and looking for funding opportunities, the FAFSA is open

027and there's still plenty of time to get prepared, to figure out a plan for what's next. So, hopefully all of you are thinking through that and thinking about it and there's still resources within the community that are willing to help you. >> Superintendent Smith, do you have a report this evening? >> Uh just one. Just want to acknowledge and recognize and celebrate Juneteenth as it's coming up here in a couple of days and the district offices are closed in observance of Juneteenth. So, let's make sure that we are out actually in the community serving one another, loving one another, and helping out our neighbor. Let's make sure we get that done. >> Wonderful. Thank you. Thank you for joining us this evening. We greatly appreciate you all. Without objection, this meeting is adjourned at >>

028[sighs] >> 5:54 p.m. And the board will now move into an open work session to receive a presentation from our data team, Joe Sturgis and Kaylee Coster, Dr. Kaylee Coster. So, I'll turn it over to you, Matt. >> Um Thanks, everyone. Um just want to make sure uh Kaylee and Joe, are you on the call there? >> We are We are online. >> Thanks so much. Um yeah, so tonight um there's a a slide deck with um quite a bit of information. And the information tonight and the purpose of tonight is just a preview for the board um uh some research um and and again sort of going through a self-examination of our current um SOFG model, talk a little bit about um board governance in general, um about some research behind that that uh

029Dr. Custer has done, as well as lift up and preview for the board some potential models moving forward um as you get ready for your June 26th retreat. And so, just wanted to put this in front of you um so that we can have more conversations about it headed up here for the next couple of weeks before we join um on June 26th. So, without further ado, Kaylee and Joe. >> Thank you so much for that. Um so, our presentation today is called data governance and recommended next steps. Um Kyle, next slide, please. Um a little bit about who you are here. And I realized as I was reviewing this this afternoon that we broke the cardinal sin um and we put an acronym on the very first slide. Um but my name is Joe

030Sturgis. I am the director of the assessment data and evaluation team. Um and with me here is Dr. Custer, who is one of the amazing program evaluators on my team, who just defended her dissertation. Um and so, I'm going to pass it over to her to get us started tonight. And then you'll hear a little bit more from me later on. >> Hello, everyone. Um thank you very much for the warm welcome. Uh I'm I'm super excited to be here with you tonight. Um and um before I preview topics here, I want to tell you just a couple of of things about myself because um I have worked previously with a few folks in the room, but actually um I'm probably not a familiar face or name to to most folks on the board and

031so I want to offer just a little bit of background about myself that I think is important context for why I do this work. Um, so I have just recently finished a PhD at the University of Iowa in education measurement and statistics. Um, and while doing that worked here at DMPS. Uh, during the five years that I was in that program, really the thing that kept me motivated and the thing that was at the center of of my why for being there was my identity as a practitioner researcher. And so, um, I I really pursued that opportunity to, um, train within myself, you know, cutting-edge methods and capacity and, um, rigorous research practices, but instead of applying those elsewhere, bringing those back to a practical context and to a K-12 context, um, and offering the

032results of that work to decision-makers and to folks who are working with kids and families every day. And so, I I want you to know that that's kind of how I approach this project and, um, why I I want to be doing this work and and I'm really hopeful and optimistic that by sharing a little bit about the work I've done, um, you'll find value in it as well. So, thanks, Kyle. You can head to the next slide. So, uh, the presentation today is roughly organized into three main sections. We're going to start by talking a little bit about the empirical literature, um, in two main areas. The literature on the relationship between school boards and their ability to impact student outcomes and what the literature has to say about goal monitoring and governance. Then

033I will share some very high-level takeaways from my dissertation. Um, I do want to be very honest that I'm going to attempt to summarize, uh, 175-page paper in a in a very timely manner. And so, I'm going to stay, uh, at the kind of very high level and share some of those most relevant, uh, headlines, but I want to assure folks that are in the room that, uh, if there's anything you want to know more or you want more technical detail on, I'm I'm super happy and excited to engage in that conversation and would be eager to provide more information um in another conversation or in a follow-up uh in writing via email, something like that. So, uh we'll do our best to keep uh the information at a at a helpful level, but happy

034to dig into the details anytime. And then we'll close out with a little bit of um some recommendations. We're excited about what this work could mean for us moving forward, and so we have some ideas to share with you that were really energized by um and we're hoping to start a good conversation about next year. We can go to the next slide. Thanks. So, in the effort to keep things at a high level, um I've used this sort of Q&A format throughout the presentation so that after you walk out of this, if you're just kind of wanting to hang on to the the big headlines, the gist of what we talked around, hopefully that can be summarized for you in this kind of Q&A format. So, we'll kick off the first section with this question.

035When we go to the empirical literature and we look at what research is telling us, um what does it have to say about whether school boards can really impact student outcomes? And unfortunately, the answer is there's honestly not a lot there. Um this area faces a lot of challenges um to to high-level empirical research, and the next slide summarizes a few of those. Uh this is a a high-level quote from a literature review done in 2002, and so I want to be really clear that school board governance and school board work in general is not an area where we suffer from a lack of writing. It's an area where we suffer from a lack of empirical work. And so, there are lots and lots and lots of models and ideas and recommendations and things that

036are out there, but they are primarily authored by practitioners like former superintendents, former board members, professional associations, things like that. And they're almost universally based on individual experiences, anecdotal evidence, and things like that. There's really a dearth of um designed high-level research studies in this particular area. And the There are several barriers to that. Um the first here it um that it's challenging to operationalize variables in this area. It's It's hard to to sort of measure what it means to be a a high-quality school board, right? What does that mean? Um it's It's hard to operationalize that and grapple with that. Uh there are some challenges with case study selection procedures that we'll dive into a little bit next. Um and then school districts are really large, complicated, and multifaceted systems. And so when we

037look at improvements um or challenges, it's quite difficult to feel really confident that the school board's action is the variable that we can hypothesize is responsible for change. And then the final thing um I I borrow a phrase from a a book by this name. Um you know, local control of public school boards is really a tradition that goes all the way back almost to the original founding of this country. Um and and the that value of local control has resulted in the fact that we have about 10,000 school boards, 10,000 different democracies that are all running in their own tiny little way. So even simple structural things like how boards operate, how they are appointed or elected, terms that they serve, things like that aren't standard across American school boards. And so when we

038think about doing empirical work, it's also really challenging to have sort of a unified concept of what a US school board is. So, with all of that in mind, I'm I'm about to share some themes that do come up in the published literature, but I want to be really clear about framing those themes. Um there are published studies in this area, but the studies are observational, not causal. And so, what I mean by that is folks are looking at districts where there are high-performing outcomes, and they're observing what are school boards in those spaces doing. Um there's not a design that would lead us to be able to kind of prove, or in some cases even theorize, that one is driving the other. Um so, these are these are high-level observations about what's happening in

039some of those high-performing, or conversely low-performing, spaces around the school board table. So, the first theme that shows up in the published literature is um from self-reported data that school boards talk about having a higher level of focus on student outcomes, or specifically designing some of their um agendas and and meeting practices to focus on decisions with instructional impact. There's some um some studies that that sort of observe that higher-performing districts have school boards that that do those things and make those priorities. >> [gasps] >> There are a couple of um interesting studies that talk specifically about board member capacity, and I'm using that term kind of generally. Those individual studies come at that idea in slightly different ways, um talking about uh board member knowledge, confidence in their role, or confidence in their ability

040to have the impact that they desire to have. But that idea of um board members really feeling like they have the knowledge and the skills and the abilities to show up in the way that they want to. There's um some observational work that's done in that area as well. There's also some work done um in what I'm calling work norms. Uh again, [clears throat] there are some some more specificity showing up in individual studies, but this idea that the way that board members work together and the types of relationships that they have, the collegiality of meetings, solutions orientation, the degree to which school boards are speaking with one voice in public spaces, there's some work done in that area with a specific kind of nuance in a 2017 study that suggests that in spaces where

041there are negative effects due to challenging relationships, it's possible that those negative effects are magnified for urban school boards specifically. There's also some work done on that really zeros in on the relationship between school boards and superintendents. Um and some mixed work that specifically comes at that from the idea of role clarity. So, there's three studies that really argue for very clear and delineated roles with superintendents, and then there's some work done by Mary Del Gardelle that argues the opposite, that we should have really integrated um relationships between a superintendent and a school board. And on all of the studies that that discuss role clarity have different perspectives about exactly what those roles should do and exactly who should do what. So, there's quite a bit of discourse in that area of the work. There's

042also a couple of interesting studies that look at the impact of board member demographics, that folks who are coming with different backgrounds, whether that be race, occupation, or political ideology, bring those perspectives to their work. And then also some some work that's coming specifically from the kind of political science corner that talks about turnover on school boards, and specifically noting negative relationships when turnover is politically motivated among school board members. So, the next um area of the literature that I'm going to attempt to summarize um for you is what the empirical literature has to tell us about um goal monitoring and specific board governance models. And unfortunately, the answer there is there's even less um work available there. And and I mean that I I I don't mean to be sort of flippant or tongue

043and cheek about it, but I I do just want to to highlight that um this is an area where there's not a lot of high-level scientific empirical work that we can go to when we're reviewing um some of the the evidence for board governance. Uh we're going to talk a little bit about specific governance models where there there is some published work. Uh I'm going to start by talking about other governance models that are not student outcomes-focused governance, and then I'll finish with the work that is most closely uh related to student outcomes-focused governance. So, there are a pair of studies that uh take a look at the National School Board Association's key works. Um they present some limited evidence uh specifically calling out positive relationships between certain indicators in that framework on certain outcomes.

044Um the Iowa Association of School Board has a a model that that talks about key areas of performance. That's coming out of their landmark lighthouse studies that they did. Again, those um note observational changes in specific subject areas in a in some grades at some um pilot districts, and some later work that uh reviewed that same model found a minimal to moderate difference in descriptive work and self-reported perceptions from boards. So, so mixed and limited evidence for for both of those specific models. The uh balanced governance approach is uh put forth by Allsbury, and there's been a pair of studies done there as well. Um again, I've listed out some of the indicator specifically, but but that's a quite a large model with a a number of different indicators. And so, there were some some

045positive relationships between or strong relationships between certain indicators, certain features of that model, and certain measures of student outcomes or achievement. Um and the second study kind of zeroed in further yet on some of those features of that model. In terms of what the literature has to say about student outcomes focused governance, we get that answer primarily by looking at work that's coming out of Texas. Texas um implements a model called the Lone Star Governance Model model, which is nearly identical to student outcomes focused governance, as evidenced by the fact that the coaching certification for those two models is nearly one-to-one, um which is a very small additional kind of Texas-specific um quiz that has to be passed to be Lone Star Governance certified. Um there are some limited results, um descriptive data coming out

046of of that [clears throat] model there. Um the Robinson study uh talks about self-reported data coming from school boards that implement the Lone Star Governance Model. Um folks reported that they did have an increase in student outcomes focus, but there was also a a frustration with the perception of inflexibility in that model. Uh and in Texas specifically, the Lone Star Governance Model has some state requirement um kind of implications, and so there was some resentment of of how that model came to be implemented there as a result of those those contextual factors. And then the final study, the Ramos study 2002, is specifically zeroing in on training, and um that folks reported positive outcomes and and a greater um sense of understanding the Lone Star Governance Model after training. So, the headlines at the end

047of the literature section, I think are really summarized by this specific quote from a literature review. And so, if I were to sort of pause here and talk about what this means for DMPS, I think that there have been lots of conversations about what governance looks like and and the direction that DMPS wants to go in our governance model. And unfortunately, the answer is that we're not going to find kind of a cut-and-dry solution to that in the literature. It just simply doesn't exist because of the challenges we've described in doing empirical work in this area. Um it's also not as simple as just kind of swapping our model one for the other because there's not overwhelming evidence for any other model, either. And so, what that means is that we're going to kind of

048have to figure it out ourselves. >> Yes. >> I'm sorry to interrupt. We're having volume issues. If you could give us a minute. >> You are fine. Thanks for letting me know. >> Kyle, are you able to increase the volume? We're not able to use the control panel up here. >> [clears throat] >> Testing, testing. Kaylee, would you test for me? >> I sure would. How about now? How's your volume level? >> You keep talking for me. >> I sure will. Um it is great to be here with this evening as a disembodied voice coming from the ceiling. Perfect. Thank you for Is that better now? >> Yep, I think we're good to Thank you. >> That was Of course. No, thanks for interrupting. It was probably good I could take a take a sip

049of water and catch my breath. Would you mind, Ms. Maldonado, telling me sort of what you last heard um so I can pick up from from the last um information that you received? >> We are right at the headline aspect of this. You were getting ready to quote um Honig um and in terms of the ambitions for school boards and the expectations upon them are not evidence-based. >> Thank you very much. So, Superintendent Smith said it right there. I I think for me, when I come down to bottom line, what does this mean? That quote really captures it for me. Um we have a lot of ambitions and aspirations for the shared work that we do together. Um but the the the kind of bad news is that as we're sitting here and grappling with,

050what does that mean for our governance model? What are the things that we want to take forward as practices? The answer to that question is not going to come from the empirical literature. It's just too challenging to do work in in this area for all of the reasons that we've just discussed. Um it's also not as simple as kind of switching from one model to another model. The other available models also have mixed evidence. Um there's no sort of coalescence behind any single model. And so, what that means for us, in my opinion is that the the grappling and the work is our work to do. And it's our opportunity to chart a path forward with great intention, with great collaboration, with our earnestness and cooperation together. And a little preview, some some things that

051I think are exciting is with a great desire to study ourselves like we've done here and to continuously improve. So, with that in mind, we pivot to my dissertation work. So, we're going to talk a little bit about what my dissertation has to tell us about past SOF G board monitoring practices. And I'll give you the tiniest description of the work that I did. The work is a pair of analyses that are roughly aligned to our first and second school board goals. So, the first paper it deals with elementary data and the second with middle school data. Both of those papers took a longitudinal approach, so they include several years of data, second through fourth grade and sixth through eighth grade. And both of them are running multi-level analyses, multi-level structural equation analyses that are

052studying differences between students and also differences between schools. So, with that in mind, I'm going to try and highlight a few of the most relevant pieces of that work. We've talked a little bit about how this area is really dominated by what they call gray literature or opinion writing. And the the good and the bad news is if empirical research about the things we've already talked about is extremely rare, technical guidance for goal monitoring is virtually nonexistent. And so, I was really excited to take this opportunity in my dissertation work to study the work we've been doing and to take a look at our practices and build our our technical understanding on our own setting, not on some other space in some other high-level abstract kind of way, but on what's actually going on here

053at Des Moines Public Schools. So, what's on the slide right now is a really a high-level summary of student outcomes-focused governance and specifically the monitoring and accountability section. So, folks at the board table are really familiar with the the whole SLFG rubric. I know that there are many elements of that rubric and many elements of the framework for this work. I'm specifically taking a look at the monitoring and accountability work. Um and and roughly, at a very high level, the theory of action there is that if the school board sets student outcomes-focused goals and then they have expectations for the superintendent to give updates and to drive that work, and then the school board also focuses their own attention and resource allocation on that, then through that whole process, um the the goal outcomes are

054going to improve. And underlying all of that, this idea that uh the evaluation of the superintendent is based in this work and that um board monitoring kind of functions as micro evaluations of school system performance and therefore the executive performance. And so, I want to be very transparent that my work is zeroing in further yet to the initial steps of this process. And the argument I'm basically making is we've been doing this work in the way that we set it up has some kind of certain assumptions or or um interesting things baked into it that we haven't yet had the opportunity to empirically test because um we haven't we haven't had the opportunity at this level of work. And so, um for me, this is a really exciting opportunity because when we know more, we

055have more context, we can um know better, do better, and um interpret data and and enrich the information we have for decision-making. With that being said, there are limits to this work. Uh so, there are lots of big and good questions that I know folks in in uh in the room and around the board table have asked and have started to grapple with. And I want to be really clear that there's there's a lot of questions I cannot answer because of this work, but I believe that the information I can provide provides great value so that as you have these conversations, you have access to that information. I also want to be really clear as we launch into findings here that I'm not viewing this as any sort of a gotcha or accountability model. I

056really think that this is an exciting opportunity to to deepen our collective knowledge and and to kind of to learn new information and therefore have new opportunities to to take that information forward. So, without further ado, back to the original question, what did we find when I did my dissertation work? The first major finding has to do with the relationships between the universal screening assessments that we're using as interim goal measures and the ISAT ELA assessment that we're using for the goal measures. And the finding there is that there's a significant relationship those two things, but it's important for us to remember that that relationship is not perfect. So, on the next slide you'll see some some simple pie charts. These summarize that the whole circle summarizes all of the student variance. So, every single

057difference between all the students that are in in the sample in ISAT outcomes. Then the colored-in portion, you can sorry, go back a second, Kyle. The colored-in portion there summarizes the proportion, the percent of that variance that is explained by prior years of the FastBridge the FastBridge assessment. And so, for the elementary paper, about 56% of the total student differences in grade four ISAT come from or are explained by second and third grade CBM reading. And for the eighth grade ISAT ELA assessment, about 71% is explained by sixth, seventh, and eighth grade A reading. So, what does this mean for us? Um, the good news is that the evidence that supports the hypothesized relationship that we already knew to be true between FastBridge assessments and ELA outcomes. We've done correlational work in that area. We

058knew there were relationships between those assessments, and we find again further evidence that that relationship exists. But, I also want to lift up to this audience and to, um, all important stakeholders that are looking at this information, that we have to keep nuance on our discussions of that relationship. Um, the relationship is not one-to-one. The ISAT assessment tests a much broader group of skills, a much wider range of skills than than either of the screeners, and that's by design. Um, and so, I think that in any any framework that we adopt, there are, uh, risks associated, unintended risks that that are associated with any decision that anyone makes. And I think it's important to lift up that, uh, one possibility of using screeners as interim goal measures is a risk of over indexing to screener

059skills, and potentially losing sight of a tuning to the larger specifications of ISAT. Um, and I would summarize that by saying, you know, moving screener results or improving screener results are not in and of themselves alone going to be sufficient to drive ISAT results. There's a relationship there, certainly, but we have to keep a nuanced perspective on what that relationship looks like, so that we're attuning to both of those things. Uh, another finding has to do with, um, the the structure of our interim goals. You may remember that in both the goal one and goal two areas, we have interim goals for younger grades, but we don't have an interim goal for the same grade that the goal measures. And there's a finding that that's actually not really a big deal. Um, on the next

060slide, there's a a bar graph that summarizes the results of a model in the middle school data where our assessment practices actually allow me to run a model that includes just 6th and 7th grade data, and then another model that also includes 8th grade data, and I can compare them to see what extra information do we have when I include that extra 8th grade information. And the answer is about a percentage point and a half. So, the the kind of practical considerations that we relied upon to make that decision are totally valid, and we don't need to worry about that too much. A major finding has to do with cohort differences. So, we have static goals that look at or expect improvements year over year in the same grade level. And in the elementary data

061specifically, we've been administering assessments for long enough I'm actually able to analyze 3 years of data for seven different cohorts of students. And the finding there is that there are are really widespread cohort differences that that really suggest we should be cautious when we're forming conclusions based on year-over-year changes. We're going to dig into that. So, what you see here on the slide, on the left side here, are predicted CBM reading outcomes by cohort. And I want to start by acclimating you to just kind of the shape of the lines. So, across the horizontal axis here, you see F1, F2, all the way through F6. F stands for FastBridge, and the number stands for occasions. So, F1 is the first occasion when students take CBM reading at the start of 2nd grade. That runs all

062the way through F6 at the end of 3rd grade. And so, the high-level takeaway here in terms of the shape of how students grow on CBM reading is that students increase their CBM reading words per minute during 2nd grade. That's where you see the first vertical line. Then, between F3 and F4, that summer, students slide backwards a little bit over summer. And then in third grade between F4 and F6, students grow again. So that's generally the shape that we see in that CBM reading data. And you can see the differences in cohorts roughly following a rainbow order so that the oldest cohorts are red and and the most recent cohorts are purple. You also see cohort differences in ISIP ELA outcomes, predicted ISIP ELA outcomes on the right side of the slide. And I'll note

063that there is a year of missing data here due to assessment cancellations during the COVID-19 pandemic. So you can see that just based on cohort, student outcomes are different. And and I'll note here that in the model, all of the variables are being held consistent. So all of the other moderators and student characteristics are totally held consistent here. The only difference in these predicted outcomes are cohort differences. Whenever we talk about cohort differences, a lot of times the next question out of folks' mouth is, well, the pandemic happened. And it sure did. And I think it's really important that we pause to acknowledge that and to say that the pandemic disruptions, which are really, I mean, foundational disruptions to the ways that we delivered educational services and that students experienced educational services, that's a part

064of these differences, but it is not the whole story of cohort differences. So before we go to the next slide, I want to just tell you what you're looking at right now. The parts of this table that are already filled in, it's all pre-pandemic data. So um each row is a comparison between a cohort. So the first row are the differences between the cohort of students that were fourth graders in 2019 as compared to the cohort of students that were fourth graders in 2020. Um the first column shows differences in where students started in second grade. The next column, differences in their rate of growth over second grade, then differences in their change or decline over summer, then differences in how they grew over third grade, and then finally differences in their ISAT scores. In

065bold uh numbers here are statistically significant differences. So, even before the pandemic happened, um there were widespread differences between groups of kids for no other reason that we can account for in the model besides the fact that they were different groups of kids. So, the next slide then adds the the information from the pandemic. So, I've tried to to show it's challenging when we're looking at cohort data to kind of explain where each cohort of kids was during that time. But, I tried to draw red lines to show you where the interruptions happened, and the red text specifically are results from the 2020-2021 year, which I I sometimes call the hybrid year. I mean, that was the year when some kids were in person and some kids were online, and and we were switching day

066That was really the the entire year where that's that educational service delivery model was was really rapidly changing. And so, you can see that there are some initial very large cohort differences that happen um right before and after that pandemic interruption. Um but, then even after that happens, incremental changes of a couple of words per minute here and there but that are statistically significant continue to happen in both positive and negative directions across cohorts. And so, I would really summarize that on the next slide um in in in images as well. So, this is the exact same data we just looked at but summarized um in in a in an image. So, I've pulled apart the pre-pandemic cohorts um on the left side and the post-pandemic cohorts on the right side. The the green line

067is is kind of the uh the the cohort that experienced COVID right in the middle of of right over the summer. Um and so, you can see that uh cohort show up on on both of the graphs, but it's the same kind of information showing up in a visual way underscoring the fact that there are are big cohort differences both before and after the pandemic. Cohort differences also differ across assessments, and so what I mean by that is that it's the story's not as simple as just saying, "Oh, well, you know, this group of kids was really high and this group of kids was really low." It's It's not that straightforward. Um and and an example of that here is that on the left side of the screen, the purple and green cohorts are relatively

068lower compared to some other cohorts. On the right side, they're relatively higher um in the ISAS data. I want to be very, very clear this is a simple observation from this analysis, and it's it's kind of way too early to hypothesize about why that might be. I I just really want to call out for you that um even the direction of cohort differences can look different and the pattern of differences can look different across different as well. So, the headline here is that we've got cohort differences nearly everywhere across the model, um but they really don't follow a discernible pattern. Um this model includes um moderation effects for participation in ELL services, uh students who have IEPs, and students who qualify for free and reduced lunch. And so, even after we control for those differences

069uh that and after we control for um cohorts that would be before, during, or after the pandemic, um those differences [clears throat] aren't explained by any of those variables. And so, we're left with sort of the conclusion that groups of kids are different from one another, and that we're we're really at risk of potentially coming to some difficult conclusions if we're comparing year-over-year changes without any context about the cohort or without any additional context to help us grapple with what those changes might mean. And if we wanted to feel pretty certain about success or failure, you know, multi-year changes would be the kind of thing that would make us feel more confident about coming to the conclusion that our work was a success or a failure. And I do want to say this is another

070area again where the SLG model potentially puts us at a little bit of risk because the goal monitoring reports are you know, evaluative activities. And so when we're engaging in those evaluative activities, it's important that we have access to other types of information so that we can add quite a bit of context around the year-over-year changes. In and of themselves with that degree of cohort difference, it's challenging to understand or to speculate about exactly what that might mean and feel really confident about it. >> [gasps] >> I I also want to highlight a couple of findings that have less to do directly with board monitoring and the board goals, but that I think you'll find particularly compelling because they're they're themes that come up often. The first is that we spent a lot of time

071looking at the CBM reading outcomes in second and third grade. On the right side of this slide, I want to contrast that a little bit with the model predicted outcomes for a reading in middle school. And really the takeaway here is the pattern of change is quite a bit different. And so we have less data at school. Those assessment practices started later. So we have only one cohort of those students in this analysis. But we see differences across different schools. We see different students growing at different rates during different periods of time. And so um there's some opportunity to kind of further explore what the pattern of change looks like in middle school in that data set. The other important takeaway here that I think is really interesting. I mentioned earlier that without going too

072far into the technical weeds, this is a multi-level analysis, which means that students that attend the same school are nested within schools, and we're able to to run a student level and a school level model that actually measures um to what degree differences between students based on their individual characteristics drive outcomes, and to what degree features of the school are having an effect on all students who go to school there regardless of the the students' own personal characteristics. And so, um one byproduct of doing analysis like that is that you're able to describe what percent of total differences have to do with things that are different between students, and what have to do with things that are differences between schools. And and what we find is really good news for DMPS in the sense that

073the vast majority of differences are driven by student differences. And so, another way to say that is you know, the same profile of student isn't sort of showing up to one school and having fundamentally different outcomes because of the school that they're at versus if that student showed up to another school. Um the percent of variance that's explained um by student level differences is a little bit lower for ISAS, but but generally I think we should take this as really good information about equity of outcomes across the schools in our district. And then the last um finding that I want to highlight is a big big win, and my hope is that everyone who hears this can shout it from the mountaintops because I think it's just a tremendous and exciting thing. Um in the

074elementary data, we because we have longitudinal data, we actually have several years of information about the the programming and the services students qualify for, and we're able to look at differences between students who received those services for all three years, second, third, and fourth grade, versus students who never received those services, versus students who received those services for some of the years, but not all of them. And so, for ELL services specifically, the vast majority of students who sometimes received ELL services during the those years, 98% of them, they sometimes received services because they exited. So, they took the Alpha 21 test and demonstrated English language proficiency, and therefore didn't receive ELL services as a result. And those students outperform not only their peers that are in ELL services, they outperform non-ELL peers. They outperform

075students who speak English at home. And so, I think that there's a variety of things that are really interesting about that. Finding number one, it's a huge celebration for our ELL programming and for our multilingual learners. Just a a tremendous thing to highlight. The other thing that's really interesting, if we hypothesize that English language proficiency was really the only gap, when students who speak another language at home demonstrated English language proficiency, we would expect them to converge or match their peers that already spoke English at home. But, what we see instead is that they leapfrog those peers and they outperform them. And so, what that might mean is that our students who speak English at home also have language needs. It's just a language need that we are not as attuned to because they don't

076qualify for English language screening under the same way that multilingual learners do. So, I think that's a an interesting thing to highlight in terms of the work that the district's doing to have language-rich classrooms not only for multilingual learners, but for all of our students. And the other finding that I want to highlight here is is a additional takeaway that comes from the multi-level aspect of the analysis. Not only are there significant effects for students based on whether they receive ELL services and the extent to which they um receive ELL services, there are also significant school effects. So, what that means is that at schools where more English language learners are exiting, every student in those school schools has higher outcomes regardless of their own um uh experience or qualification for for um ELL services.

077All students are doing better in those schools and so those spaces that are language rich and have high language um are benefiting all of our students. So, uh in great summary, I won't read all of these, but these are sort of the high-level headlines um that that I would share in terms of most relevant findings from my dissertation work um and again, I hope that they um provide a lot of value as as you guys continue to have conversations about um where you're heading next. I do want to, like any responsible researcher, be very transparent about the limitations of my work. Um the largest limitation is is the um the first bullet point there. Because of the the size and scope of the work, um I started with a restricted sample. And so, students um

078who are in this analysis are students that were nested within a single school. And so, what that means is if students changed schools during those years of study, so 2nd through 4th grade or 6th through 8th grade, they were not included in the analysis and and I describe there what percent of available data therefore was included. Um I think the the percentage of available data there is still very is quite high. We can feel really good and really confident about the quality of this analysis and its applicability to our district broadly, um but I want to be transparent that because we were start with this is the first time we've done work like this, we started with a restricted sample um to to test those assumptions there first. Um I also want to be clear

079that the cohort work is limited to elementary because we don't have um as many cohort we don't have any other cohorts um only one cohort that's available in middle school. And the analysis of including that additional goal year data is limited to middle school because conversely we don't have an assessment set up that um allows that in elementary. And [snorts] then um finally want to be transparent about about the fact that neither FastBridge nor Iowa testing programs reports conditional standard error of measurement. So, for the real technical people in the audience um what that essentially means is is we don't have uh an estimate of the error when those test scores were calculated during the measurement process. And so, I can't correct for for any of that when I use those assessment results in a

080broader project like this. And um that's a limitation both of this paper and um this this field in general. That's something that um behavior science research can can stand to work on. So, with that uh I'm going to turn it over to Joe who's going to share a little bit about what this can mean for us moving forward. >> Doctor >> Um Yes. >> Hey, sorry. Uh question from online. This is Patrick Dix. Um just want to clarify your last slide. It was not one school. It was simply students who stayed at one school. >> Yes. >> And I just in reading it, I just want to make sure that like the public understands you didn't just look at one school, you looked at students who stayed at one school. >> Thank you for clarifying

081that. And yes, I can I can reiterate that. Yes. So, um it was students who stayed at one school during the time of analysis, but all schools all elementary schools all 30 elementary schools and all 11 middle schools are included in the analysis. Um and there's approximately depending on cohort about 2,000 um students in any given cohort in elementary. So, it's quite a large sample size of students that are included. >> Thank you. All right. So, now that we've learned a lot um and we've had the opportunity to to dive a little bit into what what Kaylee's been able to bring to us, what does this mean for us moving forward or what could it mean? Um and I want to frame these next slides around this particular piece, which is that we have a

082little bit of an enforced opportunity to try something a little bit new um if everybody's on board. So, there is this natural opportunity to pause that's coming up to us because of the four things that you're going to see see on this slide. And we're going to talk a little bit more through each of these as we go through. That new knowledge obviously that we've learned from Kaylee um but especially those first two, I want to give you a little bit more context about what we're talking about there with our assessment changes. Um so, as you know, our board goals one and two are focused around ISASP or the Iowa Statewide Assessment of Student Progress. This is the annual assessment given to third through 11th graders in ELA and math um and then also uh

083specific grades for science. Up until this year, um so for 2019 when it was rolled out to 2025, there were three levels um that a student could achieve based on their scale score, which was advanced, proficient, or not yet proficient. So, for example, fourth grade ELA, if you were between 350 and 413, you were not yet proficient. 414 to 477, proficient. 478 and above, advanced, right? Um what was announced in earlier this spring was that they were going to be new standards coming out set by ISASP that have instead of a three-level model, a four-level model that you see there on the screen. Um we do not yet know where these scale scores will fall out for each of the grade levels and each of the areas. Um this uh in fact, as far as

084I'm aware, I believe the DE is working on it with the teams um, from folks from across the state um, as well as Iowa testing right now. Um, so we will know more in August about what these two things might look like. Um, but we the Iowa testing has made it very clear in their directions that they do not believe it would be in any way appropriate to compare the old standards, the advanced proficient not yet proficient to these new standards um, because each of those scales score points for those four levels are all going to be set independently of what was previously in place. Additionally, FastBridge, which may sound familiar as what's used in as many of our interims, um, announced uh, Renaissance announced uh, that FastBridge would be those assessments would also have

085new cut scores to be reevaluated for FY27 to ensure that they're in alignment again with those standards set by ISASP. Um, we don't yet know when we will be getting those new scale scores. That has not yet been shared. So what does that mean for us and our current board monitoring system? Um, so as you can kind of see on the screen, we tried to color code it because this the the the letters got real small real fast. Um, but we have a lot of things that under the SOFG model that asks for us to have at least um, two to three years of data or at least two to three data points available that just will not be available next year for us to report to you the way that we've been reporting it

086to you. Um, there are some additional notes on the slide that you can see um, about some other potential changes that could be coming to those different metrics um, as the district undergoes our comprehensive data system evaluation over the course of the next year. Um, so what we wanted to pose, and as I was working with the team earlier this spring, one of the things as we learned about what Kaylee has learned, knowing what we know is coming about i-FAS, um, knowing that there have been changes kind of across the board, um, what if we took this pause or this reset up as an opportunity, um, to dig a little bit deeper, um, to understand, um, in a much more thorough and deep way all of these different things that are going into not only

087your board monitoring and how you monitor our system, but also how we as a system internally at the district use, um, that guidance and, uh, build on those systems in our processes. So, I'm back now to make what we're calling the pitch portion of the presentation, and I'm really excited to say that we presented this this kind of pitch to Superintendent Smith, um, and he he's hardly endorsed this project, and I'm really excited about it. Um, I won't, uh, read to you this definition of program evaluation, but I want to ground us a little bit in the work that program evaluators do, um, and the value that I think it provide provides. I'm really passionate about program evaluation because I think it's the natural complement to research. Research has a lot to tell us about

088the big picture and the best practices and the generalizable things, but program evaluation is boots on the ground. What's going on here? Let's dig in deep here. Um, and I think that that, um, our expertise has a lot of value here. So, on the next page on the next slide, I want to, um, show you this image here. This is, uh, a kind of high-level summary of of a logic model, which is a common way that program evaluators organize the way that a program works. We talk about, you know, what's the what are the resources that we've got? What are we doing? Um, what do we see as an output as a result of that? And then, you know, what are outcomes and what are or of our long-term outcomes? Um, right now the board

089goal monitoring process spends a lot of time talking about those outcomes. Um, and I colored red uh sort of the the output section and the bigger data story. A lot of times we spend time talking about some of these things, but we're sort of missing There's a big hole in the middle. Uh the next slide shows you an example of of what I'm talking about. So, this is a topic that I think comes up um has come up in past board reports quite a bit, you know, this idea that we have a resource, we bought a curriculum, you know, we did PD, um and then the question is like, great, did the vast scores on our interim goals go up? And and to answer that and to start digging into that, there's this big gap

090in the middle of all of these questions, um and and there's there's more questions yet, right, to dig into um that that we really kind of have to have some information about in order to be able to tell this bigger data story that connects the work we're doing with the desired impact in student outcomes that we want to have. And and program evaluation is a discipline, um is a field that I think has a lot of expertise to offer really tremendous value here to tell this bigger story and to dig into some of those those um gaps to help us understand our system. So, you might be wondering, great, Kaylee, you're a program evaluator. What have you been doing this whole time? And so, I I want to call out um that for us, this

091is uh what we're what we're proposing is a big shift in our focus. Um, my title is program evaluator. And historically, our field really looks at programs. I mean, that is really the unit of analysis. And what we've actually proposed to Superintendent Smith instead is what is being called a problem-bound evaluation. Um, and so, instead of rooting our work just on one specific program, one specific thing that's happening, really rooting our work in the challenges that we're grappling with and seeking to understand those challenges and the work going on there, which is a really fundamental fundamentally different way of of practicing evaluation work. I will not go into all of the components of this graph, but for folks who who might want to nerd out about program evaluation methods, there's a full citation here. But

092I do want to call out that this is exciting. I'm excited about it because it's a new area in our field, and this is a new model that's being proposed with a lot of enthusiasm and excitement about about applying evaluation in this way and to to take practices from complexity science and implementation science and what we know about leadership in large systems and to merge those things with evaluation methods to to do the work at this problem level instead of the program level. And so what what we're coming with you today is excitement around doing a problem-bound evaluation of K-8 literacy and want to share really how we came to that focus. We went through a formal process for intaking and and kind of considering different evaluation topics for this work and and very quickly

093literacy emerged as the top priority. And I won't go through the whole list of things we considered, but I want to highlight for this audience the depth of alignment to the to the board goals, not just these board goals, but past board goals as well. We've been doing literacy work for a while and also the importance that we heard from our community and from parents, um particularly when the board went through the listening sessions as a part of their their prior goal-setting process. So, we're really centering this work in this problem statement. This idea that we we all know that our literacy outcomes are not yet where we hope they would be. We've got, you know, examples of great success. We've got big wins, but as a system, we're not yet where we had hoped

094we would be, where we want to be. And that's despite the fact that we have a several years long history of really diligently putting effort towards improving in this area. And despite that, we're we're hitting up against this problem of of of having the outcomes that we hope we will we would be seeing. And so, we're going to talk a little bit about what I'm going to turn it over to Joe to talk a little bit about how this new project that program evaluators are tackling my interface with board monitoring particularly next year. >> Um so, again, this is kind of the second part of the pitch. But what we're hoping that we can do and what we're hoping to present to you today is a plan that gives us the opportunity to honor the

095voice of the community. Like Kaylee said, we've seen this a lot over the past >> [clears throat] >> over the past many years. I think the whole eight years I've been in the district while providing high-quality and relevant information. And on the next slide, you'll kind of see that summarized in that that answer Q&A style. And then on the next slide, what we're kind of to get to the nitty-gritty of this of where we are right now and our recommendations as a as a kind of an opening an opening pitch to you guys of what could what could this look like. So, as you know, as many of you know, what we're currently doing is that you receive between two and three reports per goal and guardrail and they're interims of of moderate depth depending

096on depending on how they're written. So, and they come in anywhere between two and 12 pages. There were over 20 reports over the course of this year. You have not yet received all of them. And there were over 35 in FY25. And going back even farther, you see those numbers continue. Um, what we're recommending is kind of splitting this off into three different versions of um monitoring reports. Um, and we've labeled those informational reports, um progress and findings of the problem bound evaluation, and then continuing to report on that goal three process, um and the progress that's happening there. Um, there are two functions in these three kind of models that we want to make sure that we're hitting. The first is transparency. We know um we don't ever want you or anyone in the

097community to feel as though because we're changing our monitoring process, um that we're not being transparent or that we're trying to hide our data. We actually see this as an opportunity to give you more. Um, but then at the same time be more in-depth and intentional about that highest leverage piece and that most important piece, which is that K-8 literacy that we've been working on. Um, brought to you by the superintendent. Um, so I'm going to go through kind of those three different versions that I talked about just so you can kind of get a sense of what we're thinking about. Um, informational reports um would be built to provide the board and the public data by demographic groups and grade levels that identify themes and trends in the data as well as potential outliers.

098These will be high-level information across a range of data sets. Our current list that we are working through um would be FastBridge, certainly reading and math across the system, ISASP, chronic absenteeism, the Panorama survey. So, that data that you're used to seeing in goal and guardrail reports, um but a little bit more broad um across our system and across different areas. Um, the evaluation and progress findings presented to you by Superintendent Smith would then be built around how are we doing on our evaluation activities? Where are we in these different processes? What are we finding? What barriers are we hitting both at the school level and at the district level? Um how our findings are then turning and impacting district actions and how we're making decisions at the district level to support schools. And then

099of course, what we're what our evaluation will tell us about um our what our future planning for those evaluation activities will look like. Um and then I didn't put a slide in here for three, but then we'll continue that goal three monitoring just to continue to honor that work with the data that we have available with map um and the the goal three work. Um there are some pros and and what we're calling considerations to to what we're putting what we've put before you. Um we believe that some of the pros are that you will if through this work be able to receive a deep understanding of very specific outcomes. Um through those informational reports, you'll actually get assessment much faster um because you won't have to wait for an entire board monitoring process or

100a board monitoring um process to be written. We can get that assessment data to you much more quickly after the actual benchmarking period occurs. Um and then of course, build knowledge uh throughout the system about literacy um as we think about action planning and goal setting. The other side of that of course is that the in-depth reporting will be deeper but on a smaller quantity of things. Um and of course, we're potentially trying something very new. So there will be opportunities to iterate and improve throughout this process as we work through it. Um I we made it at exactly an hour. So we have if if we want to keep to that time frame, we have a few minutes for questions. Um we're also happy to come back another time. >> But I'll I'll start.

101Did you want to say something? >> No. No. No, please go ahead. >> I have so many questions, so I'll try to keep it brief because I'm sure we'll go into a lot further detail about this, but I love I love everything that you guys have done. As a new board member, I've been trying to wrap my head around how our role as board members impact student outcomes. So it's really interesting to see that there's really not a lot of data to support that it does. But I think it's an opportunity that we can be collaborative and brainstorm, and I like how you put that we can kind of uh navigate our own path to finding success with it. Um I'm curious if when you talk about the problem-based um outcomes or what we're trying

102to find out can it be more than just the literacy or the math scores? Can it go deeper into something such as athletics? What is the problem? How do we find a solution to it? How do we work together as a board to come up with a solution? >> Yeah, so I think there's kind of two levels of the answer to that question. Um the first is uh we would love that. I would need several more staff members. >> [snorts] >> So I apologize. I can't hear you very well. >> Oh, I apologize. Uh how about now? >> Nope. >> Now? I am talking now. >> go. Now we can hear you. >> Yeah. >> Okay. Um uh I apologize. I'm going to I'm going to answer a little bit, and then I'm also going

103to let Haley answer a little bit. Um but I would say we I I we we would love that. Um I would need several more staff members. Um kind of as we've built out this process and what we think that K-8 literacy pro pro problem bounded evaluation is going to look like it's going to take a significant amount of time um processing, building, uh, engaging with community members, uh, people in across the district, going and and evalua- and seeing and evaluating different things. Um, we really had to be careful to narrow our scope to a very specific area that we thought would be the highest leverage, um, just because of the amount of of time and and capacity it's going to take out of the two program evaluators that are on the team. >> Okay,

104thank you. >> Yeah, I I have nothing to add on that front except I would say, um, that if you're grappling with your role as a as a board member, the good news is I think so are all of the other board members, right? Like that's that's what the literature has to tell us is like folks are asking these questions and they're grappling with things. Like that's a really normal part of the experience and so, um, I'm super excited that we're having these conversations, um, and that we can continue having them. Um, it's it's exciting for me that to to be invited to speak, um, at a forum like this and and I'm I'm really thankful for the opportunity. >> Very impressive. Thank you. >> Um, I'm going to be a little transparent. Um, I

105am probably wanting to, um, go back to this later, uh, after I go through the slides again. Um, you did a good job, Kaylee, but, um, for some of you that English is not really their second language or first language, um, it was kind of hard to follow you. So, I will probably reach out to you, um, or other board members and try to get with you and, um, after I'm done looking through this, I'll probably have more questions. >> I I would love to to be helpful in any possible way. So, looking forward to that collaboration. >> And I'm right there with you. I, um, need a little more time to digest and would like a little more time to digest this. I think it tells us a lot of some of the things

106that were intuitive for a lot of us, but um this helps me understand that honestly many of the things that we were told over the last few years about research and this and that being research-based um may not have been as conclusive as they were painted to be. And so I think that gives me a lot of pause about um how we move forward. And so I just want to understand it before we do it. >> Yeah, just echo thank both for the [clears throat] presentation. It's very helpful, very interesting. Can you maybe expand upon the problem? >> Can't hear you. Sorry. >> How about now? >> Yep. >> [clears throat] >> Thank you. Can you guys Kaylee, can you please maybe expand on you shared a lot of excitement about the problem-bound evaluation and

107that shift. Can you maybe kind of go back to that and help help me understand or help us understand what's so exciting about that and if there are examples of other areas, boards, or non-school board areas where that that model is gaining traction, why that's so exciting in contrast to the program-bound. >> Yeah, sure. I'd be happy to talk about that. So let's use an example. I'll contrast sort of the two models. So so traditional program evaluation of a program is sort of the unit of analysis. And programs obviously can come with various degrees of scope, right? There's complex programs, there's kind of straightforward programs. But in general you think about those as as a project that's bounded by the length of the program. You'd really be trying to understand, you know, let's use a

108tutoring program for example, right? What are the tutoring activities look like? Who's doing them? What are they hoping to have the impact. You'd be talking to folks, you'd be saying at the end of the day, yes, this this tutoring activity is is working the way it's designed or it's not working for the following reasons. Um they're less flexible to a larger context because they're looking at the scope of that program. And so when we set and we and we kind of grapple with like what are the biggest things facing our work and literacy emerged as that priority, and we could do a program evaluation of so many that are literacy activities, but but really I think there's an opportunity to grapple with that larger query, which is we've been doing certain things and they're not

109yet scaling up to the impact that we've hoped they had. So the the problem bound evaluation really takes the practices of program evaluation and applies them in what Rohanna and Christie call cycles of inquiry and and they're much smaller, more frequent cycles of inquiry that are then directly fed back into the organization. So we're we're learning in sort of an iterative continuous improvement type of model from the discipline of program evaluation that then is directly influencing actions that we're taking, you know, not at the end of the program at the end of the year or at the end of a three-year program, but really um in a much more frequent cycle um to have a a cycle of learning um that is more adaptive to the needs of the organization. >> So that help? >>

110Yeah, so thanks Kaylee, this is Matt. Um I just maybe want to add to that with the program bound evaluation model, if we utilize our current reality, is that we would present on I would present on information and data that actually was three to four months old and we're already three to four months past all of that. Um what we're talking about is the potential of moving towards that um cycle of inquiry where we're the data is right now and it's live and what we're learning about that and going to do about that in the coming months based on what we're learning. It It's much more formative and iterative. >> Thank you. >> Yeah, great question. >> I would Can I add one more thing, too? I think it also allows us to potentially examine

111um larger-scale assumptions that are features of our system and not just features of a program. Um and so there there may be assumptions about how things are working across our system that go beyond the scope of a single program, a single thing that's in place at any given school, but but are are larger and would be an opportunity for us to dig into some of those deeper matters as well. >> It's on. >> It is. >> Should be. >> Yeah. >> You tested it. >> Can you hear me? Yes. >> Yes, I can. >> This is This is Maria. I um just want to thank both of you. It's very encouraging and it's also very validating uh for me as someone that has been sitting here for 5 years and just having difficulty with uh

112the model, with SOIFG, and uh being I think uh very um clear about uh why I thought it was a very limiting model um in a lot of ways. And so this is very encouraging. Uh it's very exciting and I absolutely love what you said about what you have on one of your screens or your uh slides to better understand differences in test scores. We need to look at why students are different from one another. And to be able to be more intentional about that, I think, really speaks to our commitment um as a district to do better for our students. So I'm excited to hear how this evolves. So, thank you. >> Thank you. We're really excited, too. And and I I'm excited to be in a space where there are opportunities to to

113study ourselves and to learn more about our practices so that regardless of which model you you choose to continue forward and you're you're doing so from a place that has access to information. >> I have a few questions. I will try to limit them, but first I I want to echo the board's comments and say thank you for the research, Dr. Custer and Joe. For I think this is extremely important and similar to Maria, I had similar questions when I joined the board, not just about the evidence of SOFG, but about the the peer-reviewed empirical research of it and if it was sound. So, I I really appreciate um you affirming some of those assumptions. The first question that I do have is around kind of that opening of your dissertation, the coming to the

114question or coming to the problem, and the question being whether school boards can impact student outcomes. Dr. Custer, I'm curious to know if the scope of your research was solely around monitoring of goals because what I feel like I've heard in chapter five is that the answer is no, that school boards cannot impact student outcomes, but I don't want to generalize that by using information that's solely around monitoring if there are other ways that school boards can impact student outcomes. I don't know what those are, but one example may be something as simple as school board members visibility in the schools may impact student outcomes and or recognizing educators or there's lots of other things. So, my question is was the scope of this research solely around monitoring or was it a broader question around

115school boards in general and their ability to impact student outcomes? >> That is an excellent question. And it really tracks the the summary of the various drafts of my chapter two [laughter] because because I I really started with a focus on on on monitoring in that process. And the reality is there's not the literature is so limited that I then began to zoom out and to say, "Okay, well, let's look at the literature on governance." And then that literature was limited and I said, "Okay, well, then let's let's zoom out some more and really examine the evidence on sort of the foundational assertion of student outcomes focused governance, which is that student outcomes don't change until don't outcomes change and the change begins at the board table. And so so that that scoping out of

116the literature review really happened because of that limitation. I would say in in answer to to your question, the the literature does not empirically say that that school boards can can drive student outcomes. I would hesitate to say that there is no relationship. It's just not an area where we can prove that with a causal with a with a research study that's designed that has strong causal evidence because of the nature of the work. There are lots of folks that are putting forward hypotheses about about what that looks like or observations about what that looks like. And I think anyone who sat in a leadership knows that your work matters. And and so I I don't want to leave anyone with with that conclusion in any way, shape, or form, but I do want to

117kind of set appropriate nuance around the empiricism of of those findings in the published research. >> Yeah. Now, I certainly appreciate that. And on a side note, I appreciate you using a 0.01 p-value instead of a 0.05. >> [laughter] >> The second question that I have is around, um, the So, obviously, you spent an entire semester or year, possibly, researching and doing this. My question is for your findings or or your recommendations for us moving forward, um, how will the board know whether your new reporting model provides any better information than what we're currently using unless another doctoral student goes through and chooses to research this this research this again. >> Yeah, I think I think my answer to that would be we'll have to remain committed to to inquiry. Right, and to studying ourselves.

118I mean, that's why I'm passionate about program evaluation and why at the end of my doc studies I'm not I'm not pursuing a position in in in academia. I'm I'm trying to stay in a practical setting to do this kind of work here. I think we'll have to continue to ask those questions and to study ourselves knowing that we'll never have every single answer, but as we do that we'll gain more and more information and be able to to adjust to the new information that we have. >> Right, thank you. >> But also I would encourage anyone who wants to go back and get their doc degree to there's there's lots of opportunities to research in this area and I'd love to encourage you to to choose this as a as a place to focus

119that work. >> [laughter] >> I second that. And then my last question is really around I know there was a lot of recommendations that you had towards the end, but if there was a single most important change that we as a board could make as a result of the findings, what would that be? >> I think we have to find a way to grapple with cohort differences and I want to be very clear and I make a big point of this in the manuscript as well that I'm not saying static goals are bad and cohort goals are good. Cohort goals have their own limitations. So the the cohort for one for one the cohort is always changing, right? Students are coming and going all of the time. There's lots of complexity around cohort goals as

120well, but I think we have to to >> [gasps] >> to work on coming up with some sort of a of a strategy that helps to provide some cohort context to static information and and I think we'll have to collaborate and iterate through what that looks like, but I think that's that's probably the number one thing giving the the degree of those changes across various components of the model both pre and post dating COVID. That's That's one of the the major takeaways. >> So, thank you so much. >> Could you help me understand a little bit better what you mean with a cohort? I understood Patrick's question for clarification, but would it be one school in one district? Are you saying all fourth graders in multiple schools? Yeah, so I'm >> Sorry, go ahead. >>

121No, I just think you can help me understand it a little bit better. >> Absolutely. So, what I'm saying with cohorts are students that are in the same grade but different years. So, an example of that would be students who were in fourth grade in 2019. We compare those to the fourth graders the next year. Those are different groups of students. They're fourth graders, but last year's fourth graders and this year's fourth graders are different groups of kids. And so, there are differences between those groups of kids that can't be accounted for by anything else in the model other than the fact that they're different groups of kids. Does that help? >> It is, but would that be school by school? For example, fourth graders in the downtown school would be compared to fourth graders

122at Jefferson? >> So, there we could we could look at cohort differences cross-referenced with school. The cohort findings that I presented there hold district-wide. So, as a district, the the group of fourth graders last year versus this year, they're different from one another in where they start the year, how they grow. There's lots of differences across the board um that aren't just explained by school membership. School membership has its own features in the model, but separate from school membership, there are just differences in the groups of kids year over year um that that nothing else in the model having to do with ELL, IEP, or free and reduced lunch services, or or school membership totally explain. >> So, district-wide? >> Yep. >> Thank you. >> All right. Thank you so much. Um some things that

123stood out to me um in particular are just the way that you framed your presentation um Kaylee and Joe. I the ability to make it so positive and take the word cons and make it consideration. Um it it those little things um bring me a great amount of joy and fill me with uh the baseline knowledge that that shows that you are working on this continuous improvement cycle and focusing on the positive. So, thank you for doing that. Another thing that really excited me was the findings that you had with our EL our students who are English learners leapfrogging um our students who are non-English learners. Um that was incredibly exciting and I think certainly something to celebrate and I think that we saw a little bit of that in a recent monitoring report, but

124this exemplified that uh year over year. So, I appreciate you lifting that up. I am curious um what is the I know a traditional program evaluation cycle, but can you shine a light on what you would consider this problem um bound evaluation cycle to be? >> That's a great question. To be totally transparent, it's a new practice. Um and so um I think that will depend a lot on the scope of what we find out. So, so right now um we're really focused on the first phase of that model, which has to do with understanding the system. So, for example, we've been knee-deep in um about six years of board reports understanding, right? What have we been doing? What's what's the history of this work in our district? Um what's the system? Who's involved? What's

125going on? What should we be attuned to as we start this work? Um when we uh those of us who are are traveling for a conference right now, get back to Des Moines, um, next week, we've got meetings scheduled with district stakeholders, um, to start interviewing folks about, um, their assumptions, their work in this area, and then we'll we'll intake and and make some program evaluation plans and be able to provide a much more detailed picture of our year one work later in the fall. Um, what I can say that, um, we can commit to and that we want that we have committed to are our quarterly updates that are open and transparent about about what we're finding and what we're doing. I think if we, um, tackle some really, really big things, this the

126scope of that work is going to differ, um, quite differently than if we identify some some very, uh, well-defined discrete focuses, um, it when we'd be in a position to give you findings and information, um, that's kind of more finalized on a much rapid pace and at more rapid pace. >> Okay. Wonderful. Thank you. I do have some additional questions regarding your recommendations. Um, so I will share those with, um, you and Joe and as well as Superintendent Smith, and I would like to request board members, um, any additional questions and inquiries that you have, if you can share those within the next week, um, to Joe and Kaylee and Matt. Um, and then we will plan on revisiting this topic on the 26th during our board development. Um, so Matt, I'll turn it to

127you to wrap us up. >> Yeah, I just want to thank you again, uh, Joe and Kaylee from joining us from afar. Um, um, again, wonderful presentation and thank you for the continued work that you, um, uh, apply to our school district and service to our students, staff, and our family. So, um, great job and thanks again, um, for being a game changers. And then also just, uh, thank the board for just, um, allowing us to preview, um, uh, some modeling and some things that we're actually working on. Uh, and as well as just, a again note and underscore tonight was really not about making any sort of decisions. It was really just about putting information in front of the board for you to consider as we're heading into a retreat here coming up here

128in a couple of weeks. While we certainly may have recommendations and we will also follow up on any questions that you've got leading into that that retreat date. So just happy to be able to have the conversation and put things in front of you so that we can grapple and debate together and move forward with the best plan possible. So thank you. >> Thank you. Have a good night everyone.