OCPS | 2026-05-19 School Board Work Session

001Heat. Hey, Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Hey, hey, Heat. Heat. Heat. Heat. Well, good morning everyone and welcome to the Orange County School Board's work session for this morning. And the purpose of our work session is to discuss program evaluation update. The public will not have an opportunity to speak during this work session for several reasons. One, no action will be taken and there will be no open public comment during which the public would normally have an opportunity to speak on other issues that are relevant to the board. I want to thank those who are here in the room for joining us. Uh, as a reminder to our board members and the public, our board members will not be engaging in outside electronic digital communication

002during the meeting so that we can focus our full attention on the matters before me. To my fellow board members and district staff, when speaking, please make sure your microphones are turned on and speak directly into them so our YouTube and Orange TV viewers can hear you clearly. Again, thank you all for participating this morning. We will now begin with a moment of silence and then I'll invite you to join us in the pledge of allegiance. >> And now for the pledge. I pledge allegiance to the flag of the United States of America and to the republic for which it stands. One nation under God, indivisible, with liberty and justice for all. >> All right, Dr. Vasquez, I will turn the meeting over to you at this point. Uh, thank you, Madam Chair. This morning

003we will um hear from our research and accountability uh team. Uh Dr. Border is uh leads uh that that department. We have for a number of years um done program evaluation both on process and on uh programs that we have purchased in order to ensure number one um that they are being implemented in the manner that they were intended to. Uh additionally to make sure that our money is giving us the return uh that we desire based primarily on on outcomes. And so as a result of the work of the program evaluation team, we have made recommendations to discontinue uh certain programs uh in the past, we also have uh identified areas where implementation was not ideal and we were able to make those adjustments. Uh and so I'm eager for the board to hear

004um their update uh for the programs that have been evaluated and then what is to come. Dr. Border. >> Thank you, Superintendent Vasquez. Good morning, Chair Jacobs, members of the board, and thanks for the bianual opportunity to present to the board on our program evaluation updates. I'm joined today by several members of the research ma ma man measurement and strategy office, including Dr. Sasser, senior executive director, Mr. Ben Allen, senior director, and Dr. Shiao Jang Sun, director of research. At this time, I'll turn the presentation over to Ben Allen. >> Thank you, Dr. Border. Thank you, Dr. Border. Uh, and good morning all. Uh, today we will provide an overview uh, of today's presentation as well as the purpose and process behind the district's program evaluation efforts. Our last schoolboard program evaluation presentation was in

005December 2025. We'll talk through the three main points today before opening the conversation for discussion. First, we'll review the purpose and process of program evaluation. Then we'll highlight outcomes from recent evaluations and discuss the ongoing queue of work. And finally, we'll have our space for conversation and discussion. As a brief recap from from previous presentations, program evaluation is a formal process of reviewing programs within the district with several purposes in mind. Making data informed decisions, fostering improvement, and allocating resources. OCPS uses two general processes for conducting program evaluations. Internal evaluations are conducted by members of OC the OCPS research department. External evaluations are conducted by contracted vendors. A member of our OCPS research department serves as the primary point of contact for framing the external evaluation, reviewing documents, and communicating with the vendor. One of

006the most important steps in evaluating a program's effectiveness is making sure we're comparing students fairly. Students who part who participate in a program are often different from those who do not. For example, they may have different academic performance or demographic backgrounds. Because of this, simply comparing all participants to all non-participants can give a misleading picture of the program's impact. To address this, we use a matching approach called propensity score matching. This means each student in the program is paired with a similar student who did not participate. We match students based on key characteristics such as prior achievement, grade level, and background factors like economic status, English learner status, and disability status. By doing this, we create a comparison group that more closely resembles the program group. This helps us be more confident that any differences

007we see in student outcomes are due to the program's impact rather than pre-existing differences between students. While this approach isn't perfect and can't account for every factor, it provides a much more reliable comparison than looking at unmatched groups. And the results that we'll be talking about today use an example of this approach in several different instances. Today we will present information about program evaluation findings for the following programs. 3DE by Junior Achievement, the Rosen preschool, Rosen postsecary, the comprehensive assessment system or CASS experts, VPK outcomes, uh, VPK impact on kindergarten outcomes, and then SAGA. Uh, I'll now turn the presentation to Dr. Shagang Sun to discuss findings from these program evaluations that were conducted either by OCPS research department staff or in collaboration with external vendors. >> Thank you, Ben. Uh, first, sorry. Thank you,

008Ben. First, we're going to discuss the project highlights for the 3D program evation results. Um, introduction. OCPS initiated the 3D by junior achievement program at Oakidge High School during the 201920 school year, expanding to four campuses, including Okoi, East River, and Innovation High School. in by 2425. The program engages students in a hands-on real world challenges developed with business partners. Students work collaboratively to solve challenges with guidance from teachers and industry professionals. Student course grades are then determined by their performance across core competencies. The high school office serves as the sponsoring department for this program. This evalation assesses the program's impact on student academic outcomes during the 2425 school year by comparing 1,2 3D students to a statistically a matched control group of 102 non3D students. Because the analysis is based on a non-experimental matched

009comparison design, the evaluation results should be interpreted as associative rather than causal. Major findings. Findings number one, the evalation revealed that 3D students achieved significantly higher GPA than their nonparticipating peers. The 3D cohort maintained a 3.9 weighted GPA compared to the co control groups 3.4 weighted GPA. Findings number two, participation in the program is strongly associated with career technical engagement with three C 3D students earning an average of 0.94 industry certification course credits compared to the control groups 0.37. Findings number three, both the treatment control group achieved a 100% graduation rate within the senior cohort. Findings number four. While 3D students recorded fewer average absences and earned slightly fewer dual enrollment credits, these differences were not statistically significant. Recommendations for this evaluation. One, strengthen supports for identified student groups. provide targeted academic and instructional support

010for ELLL, SD, and racial ethnic student groups as these students demonstrated comparatively lower GPA and assessment outcomes within the 3D cohort. Two, develop consistent expectations for program delivery. For example, teacher training, industry partner engagement, and instructional practices and monitor implementation to ensure fidelity across all participating schools. Next, we're going to present the findings from the Rosen Preschool Education Program Evaluation. Introduction. OCPS initiated the Rosen preschool education program at OCPS academic center for excellence to provide a vital early childhood educational foundations for historically underserved student population conducted in alignment with the OU signed between the Rosen Foundation and OCPS assesses the program's impact on student academic outcomes and behavioral metrics across yearly cohorts dating back to the 178 18th school year. The analysis compares students who attended the Rosen preschool to two distinct comparison groups. The

011broader OCPSA's comparison group and the statistically matched district comparison group. The student services department has recently transitioned into the role of sponsoring department for this program. Major findings. Findings number one. The evalation revealed that Rosen students achieved competitive academic readiness upon entering the K12 system. Across the eval across the evaluated kindergarten cohorts, Rosen preschool students entered the year with the highest baseline scores in both mathematics and early literacy compared to their campus and district peers. Two, throughout their kindergarten years, the Rosen cohort maintained an academic advantage. They consistently achieved the highest end ofear average scores in early literacy and demonstrated growth in mathematics outperforming the matched district comparison group. Findings number three. As students progressed into primary and intermediate grade levels, their academic performance leveled out to match peer groups while average score fluctuated ma

012numerically yearover-year on primary reading and mathematic assessments. Differences in mean scores between the Rosen OCPSAS and district cohorts were not statistically significant. Therefore, Rosen preschool students performed at a statistically comparable level to their peers in later elementary grades. There was not a gap between Rosen preschool participants and the comparison groups. Findings number four, attendance and behavioral data indicated that localized campus challenges may impact the Rosen cohort. Rosen students recorded higher average yearly absences and suspensions than the match the district cohort. But these figures closely mirror the broader OCPS ace campus averages suggesting these are the challenges that are not unique to the Rosen preschool participants. Future directions for this evaluation. Number one, future evaluations of the Rosen program will continue longitudinal analysis to assess long-term outcomes such as the maintenance of early literacy gains into

013reading proficiency benchmarks and subsequent middle school transitions. F uh direction number two, tracking current primary and intermediate cohorts through secondary education is a critical next step to build upon this initial findings. This will provide a broader understanding of whether the strong academic advantages observed in kindergarten can be sustained over a student's entire academic career. Next, we're going to discuss the Rosen Scholarship Program at Johns High School evaluation. Introduction. The Rosen scholarship program at Johns High School was designed to provide a vital support and post-secary education opportunities for a student population. This relation assesses the program impact on high school retention, post-secary enrollment, and longitudinal cohort attritioning through the 22, 23, and 24 high school graduation cohorts. The student services department recently transitioned as the sponsoring department for this program. The analysis tracks the comprehensive student

014journey to identify critical transition points and systematic barriers by evaluating the attration funnel beginning with the eligible 11th graders and follow them through high school graduation into their post-secary careers. The evaluation highlights enrollment trends and long-term retention challenges. Major findings. Findings number one. Scholarship eligibility requires students to reside in the specific geographic zone for grade 11 and 12. For the three cohort years of 22, 23 and 24, 162 students were eligible for the scholarship at the beginning of the grade 11. By the time of graduation after accounting for student changes of address 126 students were eligible and 78 students received a scholarship and enrolled in post-secondary institutions among those enrolled. Most did so immediately after high school. However, fewer than 30% of the students remained enrolled in post-secondary institutions after for at least two consecutive

015years. Findings number two, student enrollment across post-secondary institutions has become more varied over time. While early cohorts were concentrated at single institution, more recent cohorts enrolled across a broader range of instate universities. Historically, black colleges and universities and state colleges. Findings number three among eligible students who did not secure the scholarship commonly observed factors included incomplete applications, missed deadlines and missing documentation. Changes in residents outside the zone John's high school attendance zone were also associated with reduced program entry. Findings number four, monitoring of eligible students has been a challenge and district staff has worked with school and program staff to mitigate the issues. Future directions for this evaluation. Uh one, streamlining the initial application process is a critical next step. Providing dedicated administrative support for document gathering will help capture eligible students who are currently

016falling through the cracks due to missed deadlines or missed paperwork. Two, addressing the critical gap between high school graduation and the second year of college will be essential to improve long-term retention. Three targeted advising and continuous support systems for alumni during the early college years. Tracking this cohorts through high education may provide a broader understanding of how to translate high school graduation success into sustained degree completion. Four, we are hiring a program specialist to support families and students throughout the entire Rosen program as students progress through schools and beyond. Next, I'm going to discuss the program in relation of the CAS expert program introduction. OCPS launched the comprehensive assessment system CAS expert program in the 2223 to support the transition of assessments to Florida assessment of student thinking progress monitoring. CAS experts serve as instruction

017leaders facilitate datadriven discussions and act as liaison between the OCPS assessment department and schools to strengthen data literacy among educators and families. The assessment department serves as the sponsoring department for this program. CS experts are selected annually by each principal as a leadership opportunity across the district. These roles are held by assistant principles, instructional coaches, and other school-based leaders. In partnership with Hangover Research, OCPS evaluated the program through interviews and surveys culminating in a capstone report that synthesizes synthesizes insights and themes from CAS expert projects to inform district leadership. Given that the program is managed within a closely related department, measures were implemented to reduce potential conflicts of interest throughout evaluation. Major findings. The following are the major findings from surveys and interviews with school administrators, district administrators and cash experts. Findings number one, roles

018and responsibilities. Cash experts serve as leaison between the district and schools guide data literacy and engage families through assessment presentations. Responsibilities vary by school based on principal's priorities. Findings number two, professional training. Cast expert training is highly valued and well structured. Foundational sessions and advanced skill naps were offered to cast experts. There are strong preferences for virtual and flexible formats of the training offered. Findings number three, data use and instructional impact. The program impact improves data literacy, informs instructional decisions, and reduces stigma around data use. Challenges include variability in impact on the underutilization of resources due to time constraints. Findings number four, positive impact on data culture. Cast experts strengthen communication and make resources accessible. School schools embracing cast experts show stronger data literacy and deeper analysis beyond surface metrics. Recommendations for the cast expert

019program. Recommendation number one, integrate h responsibilities into existing school structure. Embed cast expert activities into established workflow such as professional learning communities, grade level planning meetings or MTSS discussions. The this approach could address scheduling conflicts in short cast task aligned with instructional priorities and making datadriven practice as part of uh routine collaboration. Recommendation number two, continue offering flexible professional learning options. Respondents appreciate recorded sessions, modular training and asynchronous learning opportunities for CAS experts. Prioritize short focused module that uh allow staff to engage without sacrificing classroom or coaching responsibilities. Maintain virtual and inerson options to increase accessibility and reduce time related barriers. Recommendation number three, strengthen administrator capacity and engagement. Provide tactic training for principles on system principles on the cast programs goals, resources, and implementation strategies. Equipped leaders with practical guidance on integrating CAS activities

020into campus structure, prioritizing he impact tasks, and supporting cast experts in balancing dual roles. I'm now turning the conversation back to Ben. >> Thank you, Dr. Son. >> Thank you, Dr. Sun. Um, I will now turn to our partnered research spotlight, which is our VPK and kindergarten outcomes. Handover research collaborated with the research. Is it is this is this better? >> Little better. Look at this. I have I don't know if my mic is better or if I'm just closer. >> We'll try this one. Is this any better? to get the mic even closer. >> Okay, here. >> Much better. >> There we go. Okay. >> I need one of the fancy mics from up there, too. >> Did Did we get Did we get new mics? >> Nope. I think we're okay. >> We

021just swapped. We're good. Um so in the VPK on outcomes uh and for kindergarten outcomes this is a project where handover research collaborated with the research department and completed the impact on VPK on kindergarten's outcomes and analysis report. The study examined the relationship between participation in the 2425 OCPS voluntary pre kindergarten or VPK program and students beginning of year kindergarten academic outcomes for the 2526 school year. Specifically, the analysis compared kindergarten students on the Florida Assessment of Student Thinking, Progress Monitoring One outcomes in STAR Early Literacy, and Star Math for students who participated in the OCPS VPK and those who did not. Using descriptive analyses, propensity score matching, and logistic regression models, the study assessed whether VPK participation was associated with improved early academic performance after accounting for student demographic characteristics. findings. First, students who

022participated in OCPS VPK outperformed non-participants on all kindergarten fast PM1 outcomes with higher rates of meeting grade level standards in STAR Early Literacy and Star Math and higher identification rates as probable readers. A student is considered to be a probable reader if they achieve a unified scale score of 852 or above on star early literacy. After controlling for demographic factors, VPK participation was significantly associated with improved early literacy outcomes, including a higher likelihood of meeting grade level standards and early literacy and being identified as a probable reader. After controlling for demographics, uh, VPK participation was also associated with improved STAR math performance, though the magnitude of the effect was smaller than literacy and was not statistically significant in matched comparisons. OCPS VPK participants were more likely than non-participants to be economically disadvantaged English language learners

023and students of color, indicating the program's reach among underserved populations. For recommendations uh from the report were to expand participation in the OCPS VPK, particularly among economically disadvantaged students, English language learners, and students of color as participation is associated with improved kindergarten outcomes for these groups. The second recommendation was to strengthen mathematics readiness components within the OCPS VPK curriculum to better support early numerousy development. As far as future directions, handover research will collaborate with the research department to complete a phase two of this analysis focusing on the impact of VPK participation in the 2526 kindergarten PM3 outcomes. And finally for this our last program evaluation spotlight uh we are bringing the Saga program evaluation. Uh the Saga education algebra 1 highdosese tutoring pilot program is designed to integrate Saga trained tutors directly into classrooms to

024provide intensive face-to-face smallroup instruction using the Saga mathematics tutoring system and virtual platform. This evaluation assesses the program in program's impact on student academic outcomes and teacher satisfaction during the 24-25 school year using a mixed methods approach. The study compared 657 students in the saga treatment group to a control group of 657 statistically matched uh students who did not participate in the program. The high school office serves as the sponsoring department for this program for the major findings. Firstly, when both groups began the study, uh they had statistically comparable baseline mathematical achievement. And by the end of the school year on the algebra 1 end of course assessment, the Saga treatment group's mean score was not statistically uh significant over the control group. Program dosage levels did not significantly predict student success as students receiving

025the recommended 60 or more hours or a partial 30 to 59 hours showed no statistically significant performance difference compared to the control group. Teacher satisfaction was overwhelmingly positive with approximately 70% of surveyed teachers reporting being extremely or somewhat satisfied with the program tutor support and tutor knowledge of the Florida's best algebra 1 standards. As far as future directions, uh OCPS retired the SAGA program um from its offering of academic supports at the close of the 2425 school year. Uh and this resulted in a total cost savings of approximately $1.3 million. Looking at our queue, uh the research department is actively working on many more program evaluations. These include city year uh collaborative classroom or sips genow pick read and roll uh we continue working through the Rosen preschool and postsecary uh SAT ACT impact on

026student outcomes the ACT non-ol reportable assessment program as well as the travel and leisure preschool and post-secary programs. These pro projects are in various phases and some of them are annual evaluations, some are based on memoranda of understandings uh and some are based on district leader requests. At this time, I'll turn the presentation back to Dr. Vasquez for any comments or discussion. >> Uh thank you so much. Uh needless to say, there were um some uh highlights, but there was also some data that um took us by surprise. And so um I really appreciate all of the time and effort and the conversations that were had as a result of this with with cabinet as we continue to find the best supports uh for our our children and for our staff. And with that, I'll

027turn it over to the chair. Thank you. Thank you, Dr. De Dr. Vasquez. I I I agree with you and I think this was extremely worthwhile to go through this process. Let me begin with actually the um the whole board is lit up. Some of you are lit up two or three times. Let's start with member Bird and thank you chair and thank you um Dr. Border and your team for um again providing terrific analysis of um some of our programs. And I I thought what really struck me about this group of assessments was um it has painted a picture of for me it painted a picture of exactly what um we've heard Lyft Orlando talk about numerous times to us. the whole cradle to career idea that it's not just um putting in one

028program in seeing success rate there and then letting that go. And it's not just getting them to college, getting them into college or providing them the scholarship money to get accepted to college. It's that mentoring that takes place and you we've heard them talk about this that mentoring that takes place that first year in college and that second year in college and providing the supports that they have at home that keep them engaged that allow them to remain engaged in their college life. Many of the kids that you know we give that initial opportunity to their home lives are their families are still struggling. So when mom and dad need help paying bills, they leave college or they stop going to college because that's not the priority in their family. So what this did for

029me was paint a picture of okay, I think we're on the right track with where we're going with Lyft. This further proves that it is not just about one program. It is not just about um helping preschoolers get ready for school. It's about continuing that um support in the community and the I thought that the Rosen preschool evaluation was extremely valuable that we saw this huge boost compared to their peers which would be expected with the struggles economic struggles that that population goes through right but then it levels out um and they're not making significant they that prep that they had when they were four years old didn't necessarily result in higher scores later on in elementary school and higher levels. So that tells me that that we that it's not just what we're doing

030in those classrooms, it's that whole support system. As a board member, this all is making me more concrete in my belief that we are on the right track with the partnership that we made with Lift Orlando and that we need to continue to look for ways to boost the entire wraparound service that we provide for our families and our kids. because no matter what we do in that classroom for those eight hours a day, the other 16 hours a day are not spent in our classrooms. And so we have to um build those partnerships. I I think this is this is great data for us as a board as we continue to make those big shifts and kind of how we're thinking about where we want to invest our money and um the programs that

031we want to continue to build on the partnerships that we want to go out into our community and gather. I think um I appreciate I just I I cannot even tell you how much I appreciated this input here. Um, my question real quick about I only have one question that's with the OCPS VBK. Um, and I realize that we just really checked the kindergarten scores this year, but is it possible that we could do kind of a long-term study like we did with the Rosen to see how those those scores also um turned out in the long run. I'd love to see if the scores remained high or if they leveled out in that case, too. Y so that is uh certainly something we can put into the queue. Now once we've identified the students

032who have completed OCPS VPK, it's much easier to track them through the system. So that that kind of first most difficult part is is done essentially. Uh so that's a project that the team can take back to to see um how they can continue that on a longitudinal basis. >> Okay. And and my guess my guess would be that if um as you said that in this study that partic these kids particularly have you know are ELLL students or students economically disadvantaged that we are again going to see probably the same thing would be my guess. Um, and you know, I just I just really want us to continue looking for ways that our community can help us um with that whole big picture of how we can help our families so that they can

033support our students. Thank you. >> And chair, if I may build upon that, thank you. Thanks, Miss Bird, for that. Um, I just want to take a moment also to thank the Rosen Foundation and Travel and Leisure for providing um our neighborhoods in Hungerford, Paramore, and Tangelo Park access to um early childhood education and then also post-secondary opportunities. We'll continue to evaluate um all of those programs annually. Um the Rosen team and uh Frank Santos are highly engaged in this data. They're meeting with us regularly as well and looking at how they can use the information moving forward. Uh they've added supports at UCF and Valencia. Um so hopefully we will see um that data continue to grow in that area. And also there's a program specialist uh that will be part of this process

034moving forward. And the purpose for that is to monitor the students uh post um once they exit the uh prek and enter kindergarten and make sure they're successful on their journey through 8th grade and then also to continue to be connected with the counselors at both Dr. Phillips and Jones High School um when it comes to the Rosen program to ensure the students continue to stay on success um throughout. So those were a couple of iterations that came through those conversations and so I just wanted to to add that. Thank you. >> Thank you, Dr. Border. All right. Uh let me call at this point on member Felder. >> Thank you >> again, Dr. Border and your group. Uh thank you for this presentation. I wrote down this is quite telling uh the information a

035couple of things I just want to mention um with the Rosen uh VPK my first year on the board in 2020. I was invited to go to the VPK I think was opening uh first day of school or whatever. And that if you go in my office that that picture of me holding this little girl that is one of my former students baby and when she saw me and she said Miss Falderart you know and the little girl just ran to me and I picked her up and I got a great tour of what they were doing with the Rosen and I am so thankful for what OCPS is doing for our children and our people in marginalized communities. We're in an uphill battle. And as I'm looking at reading the stats that you have

036gathered in the data and I was very concerned when I would hear that the children were getting the scholarships, they were going to school but they could not stay. They were not staying. and my experience. I can't even imagine my first year at Spellman in August and then October I'm home. I I just I can't wrap that around my head. I'm expected to be there. I went in the fall of 74 and I'm coming out in the spring of 78. Uh and so there was seemingly as uh uh board member Bird was saying because there was an large amount of kids of students not staying. So when as member Bird was saying as they came in and I fought them so hard but nevertheless when they came in and they were making helping us understand

037that we really do need this wraparound service and when traveling leisure came I remember Mr. Rosen talked to the gentleman said, "We need you to come in and help." And of course they did with the uh Hungerford uh elementary. Um and so you said something about making sure because I wanted to know, we looked at the data with Rosen Scholars at Jones. Are we having some of the And you don't have to answer right now because I want to say a couple of things, but are we having some of the same issues uh uh with the students at Dr. Phillips that get on the Ross and scholarships and when they go to school are they is there some issues where they're not staying not not able to stay and um the other thing is I

038guess I'm going to have to have some more and it's okay town halls because either get on a radio show or something because I think people need to understand the fight. I don't think they understand the struggle of what we're doing as a district. we prepare them, but we need the parents or the guardians to continue to encourage. But then again, we do understand when you have this economic concern in our communities, am I going to eat a next meal? Am I going to take this money and finish send, you know, paying for the transportation to get me to school or to uh pay for uh my books and my parents? You know, it's it's just a hard hard decision to make. But what I I I do want us, and I don't know how

039that can work, superintendent, because I know our teachers and our counselors have so much that they're that's on their plate, but we've got to do something about that gap. We've we've got to, I guess, be more diligent in in following. Again, that's where Lyft comes in. But that's just at one school. We need that kind of thing across this district because we got communities I'm sure in uh member Bird's district uh member Douglas's member gallows member far we got marginalized communities all over this district but people are struggling and they need to help but I appreciate I so appreciate what we're doing as a district and I I don't want to put them out on front street but I wonder if everybody else in the rest of the state doing what we doing. I'm serious.

040The struggle that we're doing to try to make sure that when we put out programs to improve learning and educational uh experience for our children, VPK up from cradle to college to career. I wonder who else is doing that because this is the only way that we're going to fix this concern that we have in our state as it relates to educational pursuits. So, in closing, I just want to say um it's a lot. I'll probably come back again, but I'm grateful for this data. I'm going to do my part as a board member to talk to some of my schools, the principles, counselors, and see what other kind of things that we need because outside of liver, there are other people in the communities that come in and help because that there needs to

041be followup cuz there are children who get to school and their parents are situation moved in the dorm with them. That can't happen. There are children I know that went to school on the scholarships and they were being asked and harassed to come home because they needed to put money put money in the house and that's a real that's a reality that we dealing with. So um thank you so much and I'm going to hold on to this and I appreciate your work. So in my question about Dr. Phillips and the Rosen program, are they having the same concerns? Chair. >> Yes, Dr. board. >> Thanks board member Felder for that question. Um we'll we'll provide you the Tangelo data for Dr. Phillips. Uh we don't have that with us right here, but we'll we'll

042provide that information. And I also neglected to share uh earlier board that we are um working with the William Julius Wilson Institute. Uh we have anou and um one of their new models uh like Lyft did, school districts are now serving as backbone organizations for cradle to career work. And so we have anou with the William Julius Wilson Institute uh to further this work in South Apopka and and Anglewood. And the goal is that in Anglewood it would cover the entire Colonial High School feed feeder pattern. And so that work continues uh to progress and uh we will be looking to leverage partnerships um across the city and the county to further the cradle to career services um for students and families in both uh neighborhoods. And I want to thank you. I'm so proud

043of the VPK uh school that's in my my district. I went there to visit and I saw those children three and four reading. So that just did my heart good because we had an issue about Richmond Heights whether or not we were going to close it and all of that. I'm glad this district I wasn't on the board then. they came up with a great idea under Cat's uh jurisdiction when she was uh Cat Gordon was the uh school board rep and we made that a VPK that you know I'm so happy that we have that VPK school in in my district and those children are achieving and doing very very well thanks to our director there uh Miss Maria Barington but I appreciate the district understanding the need to start the reading at an

044early age and thank you for the presentation. Thank you, Chair. >> Thank you, Member Felder. Member Gallow. >> Thank you, Chair. Um, and thank you all to this great presentation. I always look forward to these presentations because it shows us what what's working, what's not working. It shows the public the thoughtful intentionalness that we put into really reviewing um the programs in place. and then eliminating those that we don't find of value that haven't lived up to the promise. And so I think that just speaks to our efficiency and our willingness um to make sure the dollars that we we're spending are well spent and they're making a difference with the students. So I uh or academically with the students. I appreciate that. Um the 3D I'm a big fan of Junior Achievement 3D. I

045just we have it at East River. Um, I've been over there and I just I love what it does um with the students. It's I I This is probably not a good reference, but I equate it to like FFA because I love FFA because it builds leaders. It builds um the students ability to debate, to talk, to look people in the eye, um to do research. It's just it's such a great program. And and for me that's what 3D is just without the animals. They they really do teach leadership and those leadership skills that you know so many um people employers want to see in our students with I do have a question. So the 3D students earning the uh 94 industry certification in a year. I'm I'm curious why not why not one like

046what happened that it's 0.94 is that an average? Why not why do they could they not get a whole credit? >> Yep. Uh that is an average of the credits attained in courses aligned to industry certifications. So it's not an industry it's not a whole counter value. It's you can get an average there. >> It's an average. So you have some that did and then some that didn't quite make it. Okay, that makes sense. Thank you. And then um on the the um the preschool, I'm not going to echo everything that my board members have said. Um I agree with them. The the cradle to college is so important and the research is very clear that prek makes a difference and it has across the state. I was just on a call last week with

047um the Council for Early Grade Success because I sit on that council and they were really breaking down the VPK with the new scoring and the kindergarten readiness and then they took it a step further and broke it down to third grade because we have the know they actually looked at it and maybe we could do the same thing in the know you um know your school portal like all that information is in there and so what they found what what we know after three years. I think it's three years of really doing the PM1, PM2, and PM3 in preK is the kids regardless of backgrounds. If they are VPK completers and they go into kindergarten, they are like 60 to 70% more likely to be kindergarten ready. And that's even with the summer slide,

048they're just more apt to be kindergarten ready. And when we followed that data from kindergarten to third grade, what we found or what the state found, this is statewide, not Orange County specific, was that 70% of third graders were proficient, which means they scored a level three or higher on on the fast test on the the the third PM. So, we know we know if we can get them and teach them early, regardless of of their zip code or where they live, they can be and will be kindergarten ready. And they and it's it's it's taking it with them to third grade. and they're much more successful in third grade, which kind of backs what we've been saying for years now, is that we need to invest more money in VBK and preK so that

049we're not spending it in third grade to retain or to remediate students. Um, I am interested, as member Bird said, to follow that to 8th grade like because it seems like even with the data that we the increase in data for literacy in third grade, we we tend to lose them in middle school. we lose that that data dropped significantly in eighth grade. So, I don't know if we could look if it it would be easier if we could look through the know your portal um to see how to expand on that to really see does that do those gains stay with the child as we move forward because what we're seeing in third grade is really really encouraging and I hope um that the legislature is paying attention because I do believe the money

050that we spent on the front end will save us a lot of money on the back end and it's much more fair to the child to get them up front than to remediate or hold them back in third grade. So, I I love that information and I love that what we're showing with our data really is mimicking what is happening across the state. Um, and then just going on to the the CAS experts, do what was the participation rate? Do we have a CAS expert at every school? What what is the participation rate within that category? So yes, each school has an identified CASS expert, at least one. Uh some of our schools may have more than one. Uh and then they occur at different levels. So the first year that someone's going through the

051program, there's a series of trainings that they would attend. The second year there's a more uh tailored set of trainings where they don't necessarily attend every session that's offered for CAS experts. Um we can get specific completion rates, but participation rates tend to be very high with that program. Um and the other part of that program is providing resources not just for CAS experts but for uh all district employees. So those are through toolkits through uh resources and materials that are ready for staff to use and we have substantially high use on those resources as well. >> And when you were when you did your deep dive on this did do do people enjoy being the cast expert? Is this something they enjoy doing or do they feel more like it's something else that's put

052on their plate? >> So that is a research finding. I would this is one of those separation pieces because I actually supervise both the assessment and the research department. So I don't want to speak for the research findings. I can speak for the the programs itself. Um the I would leave it for um either Dr. Son or for the report to talk any more about the specific findings in the in that particular program. >> Yeah. based on the survey results from cast experts and from principles as you can fi um find it from the findings. Number three, the feedback from principles and cast experts are saying the program improves data literacy, informs instructional decisions and reduce stigma around data use. So there's overwhelming positive feedback about the program. Thank you. So, they're enjoying they're enjoying

053being trained and on and and we're showing >> that it's helpful. >> Yeah. >> Okay. All right. I think that is all my questions. Um so, thank you very much. >> Thank you, Member Galla. Member Vanos. >> Thank you, Madam Chair. I'll actually just start where member Galla left off. um for the CASS program. What is the impact on student outcomes? I would say that the CASS program is one of the several different initiatives across the district that leads to increased student outcome. So when we look across multiple years and even kind of where we're looking at PM1 and PM2 this year, we see outcomes continuing to increase in our reading and our mathematics outcomes. Um so that's one of the general impacts. The studies that uh that conducted here did not look directly at

054the program to student impact. Uh it's more of an indirect effect that it's part of a whole system around data literacy analysis supporting teachers to to understand the information and make instructional decisions based off of that information. >> Okay. Thank you for that answer. I um I guess I would like to see the comparison of like what things are we doing that are contributing to increasing scores because when you don't have a you know a direct study it's you could just say any number of things are contributing to it. So, I'm I'm actually just curious. I mean, maybe there's not a way, and maybe you can answer this for me, but is there a way to really tie down this program, this, you know, these experts, this software thing? Like, because I I sort of

055feel like we just sort of are throwing everything and we and it scores are increasing. So, we can kind of say, well, everything is playing a part, but how do we really drill down on specifically what is working? So one of the challenges for that type of research design with this program would be that every school has a cast expert and every school is participating in the training. So there may be some differentiation in terms of how much training each cast expert is participating but they are generally well attended. Uh so there isn't really a control group that would be naturally apparent in there. So it's that would make it kind of difficult to say as a result of this program we see these impacts in schools that are highly implementing versus low implementing or

056implementing versus non-implementing um just as a design challenge in that study. >> So this didn't start off as a pilot program. It's just everybody got it all at once. No, it started with everybody. It it really came out of the transition from FSA to the fast assessment and needing to have a lot of information provided to schools on a quick turnaround basis. Uh so we took a a full-in approach on the implementation of that. >> And I understand why that decision's made, but um but it it it does make it hard to like truly evaluate any one thing. So we'll just leave it leave it there. Um thank you for that explanation, though. Um, for the for the um for the preschools, I am a huge proponent of um supporting high quality child care, high

057quality early education. Um, I think we need to expand it, make it available to more people. I like that OCPS is is doing that. But I will say when I read these two evaluations, it it makes me and and what what I'm seeing is that students are yes, more kindergarten ready, but if ultimately the benefits even out over two or three years, and and I'm glad that member Gallow um brought in the third grade data because I do think that is important. But I I think again I I support just early child care support and you know expanding that to families. I think it's good for our economy. I think it's good for our families. We need affordable options um so that people can continue to work and their children are in a safe place.

058And so I support that. But um you know I guess just whether a child learns to read in kindergarten or second grade could just be the developmental you know piece of being a child. Not everybody learns at the same time. So, I guess I'm um unsure of exactly the academic benefit if it all kind of evens out after >> Yeah. May >> second grade. >> Uh that's exactly the conversation that we had. Actually, it was the most surprising outcome of any of um the program evaluations. we expected to see that um they would continue to achieve above their peers. And so that is um part of what we're trying to peel back. And what is it that is happening uh after kindergarten that is causing them to even out with others in similar uh demographic?

059um neighborhoods. Uh so that when I said there were some surprises, that was the one that I I don't think any of us expected to have that outcome. And with the amount of money that they are investing in the program, we need to be able uh to dig deeper and identify what are those factors that are contributing to students um leveling leveling off whether it's attendance, whether it's the rigor of instruction, whether it's um the level of intervention that they need to continue to receive above what others may be receiving. So, you're you're right on point. >> Okay. Well, thank you. I because I'm I'm was wondering if I was the only one who was seeing it that way. Okay. That Thank you. That makes me feel bad. >> We actually Well, we won't at

060cabinet. It was a very long pause like wait, is that what you're saying the data is showing? And then of course the followup was um with um uh the Rosen uh folks for them to also work with us on identifying what is causing that because they're doing this. >> Okay. Thank you. and and I appreciate member Gallow's comment and maybe there are structures in place at the state level that that that we can learn from because if if they're seeing improvements through third grade like that would be great to know what what's happening there. So, not to take anything away and like I said, I I support it. I think we need to expand um for for so many reasons, but I I was concerned actually when I when I read the preschool evaluation. So,

061um, sorry, let me Okay, my other question was with the 3D program, love, junior achievement. Um, just a question when you're talking about the GPA, if if a student participates in the three um, DE program, is is there like is that a weighted course or not? >> We would have to double check that. I I would want to confirm that before making a statement on that one, but we'll double check to make sure if it's weighted and if so, to what degree it's weighted. >> Okay. Thank you. Because it's hard to to know whether the the weighted GPA is just because the class is weighted or not. Okay. Thank you for following up on that. And then also with um under the 3D again, I think it's a great program. Um you're talk and the

062last recommendation is developing consistent expectations for program delivery and monitoring implementation. Would that come out of the OCPS budget or is that junior achievement monitoring or like how would we actually make that happen? Or maybe that's not your job but maybe you're just your job is to give the recommendations but okay. >> Yeah. So for that is where we generally defer back to the sponsoring department to make decisions about how implementation is impacted by the results. So we can certainly talk about the evaluation findings, but it's it's really the sponsoring department that would make those decisions as far as what happens next. >> Okay, that makes sense. Thank you. I think I just have a couple more questions. Um preschool Oh, going back to the Rosen preschool data. Um, in the last major finding, um,

063when you talk about the yearly absences and suspensions as being greater than the matched district cohort, I'm curious if you if that data has changed since that school opened. So for example, when the students were previously going to I think it was Howard and maybe another school, did we see that same kind of discipline and um absence data at that time or has that have absences and discipline increased since students started going to that school? We'll have to go back, Miss Vanos, and look back from the beginning um to see if those if that trend has continued uh since ACE opened or not, but we can do that. >> Okay, thank you. And I think, oh, and then just um on the Okay, this really is the last question on the uh the Rosen scholarships.

064Again, I mean, when I when I look at the the information on the students who missed the scholarship deadlines because of incomplete applications, missed deadlines, missing documentation, um this really hit me hard. like this saddens me because the money is there, the opportunities are there and it's I feel like these are factors that we can really help students with. Um particularly with our guidance counselors, with our you know college and career um I I feel and I I don't feel that this is the only example. I feel that this is a gap that we really need to I I just hope we continue to focus on um because this is is is an easy opportunity that's being missed for administrative issues. >> Yeah, thanks board member Vanos. So that that will be a focus moving

065forward. Um the Rosen um team has an individual who was helping support this as well as there's um a guidance counselor um specifically uh for these students at Dr. Phillips and Jones. However, by adding the program specialist person now that will be interacting on a regular basis with all entities, um I believe you're going to see um those numbers improve and that's certainly the investment from the Rosen team as well is that they believe that that will certainly impact the data that you're seeing. >> Okay. Um thank you for that. And I think, you know, I know that maybe not every person who um applies and and gets in and starts a you know, college uh or or a community college two year, four year degree finishes. But I do know there's research from the

066Helios Helios Foundation that even starting even if you don't complete, there's a benefit there's a benefit to students there. And so I just think even just getting students across that line and getting them to start a program, even if it's not finished, is important. Um so I wanted to say that. And I also just wanted to I know we we've talked a lot about Lyft but also and travel and leisure but also shout out to Elevation Scholars that does a really really good job of helping students through the entire process of applying to colleges um and then staying with them through their four years to in to kind of help them with these issues that maybe do cause some students to um to have to to leave a program. So they weren't mentioned so I

067just felt like we needed to shout them out and and maybe we can learn something from their their template as well. So, thank you so much for the presentation. >> Thanks, board member Vanos. >> Thank you, member Vanos. Member Ferrant. >> Thank you, Madam Chair. >> Okay. Awesome. I love these um presentations where we can really do a deeper dive into the programs that we're using and if they're helping us, not helping us, and then how we can kind of pivot or uh remove them maybe altogether or continue them and and flourish. um 3D Junior Achievement question with this. How many schools do we currently have um the junior achievement in? I know Oakidge has one and then I think member Gallo said that um East River had one. >> We had we had four

068in the study. So it was um Okoi, East River Innovation, and Oakidge. >> Awesome. Thank you. Um, what does it cost the district to have this program? >> So, the cost in the 2526 school year for 3DE was approximately $590,000. >> Awesome. And it provides teachers or what is that going towards? >> Um, those that's just the budget number that I have. I believe that's general program expenses, but I'm not sure exactly how that those funds get broken down. >> Awesome. >> Yeah, I believe there is a coordinator um position, Miss Fant. Um and then also, you know, students that are in the 3D program are engaged in a lot of uh case study work. And so they're really um getting the relevant instruction related to the business world and lo a lot of local

069businesses um tied to the curricular experience. So it really takes the learning experience and and integrates it for the students. >> That's awesome. Uh something I would love to see even for next time too when we um look into this is the students feedback and how they feel about school because when I've been at Oakidge for instance and um spoken with students they said it really helped them want to even be at school. They enjoyed it. It was uh it made sense. It brought real world problems and issues that we're going to deal with and help them to figure out how they could solve it or how they could you know um if they enjoyed something then well what certifications could you get in order to um you know head in that direction. So really

070bringing the real world into school. And a lot of times when I'm talking with students, they say if they say they're not really interested in coming to school, you know, they don't like being at school. Well, a lot of times that's because they're in a class that they say, "When am I ever going to need this uh in the real world?" Um, I'm going to throw one out there and maybe there's stuff that we'll use this for, but trigonometry for instance, the vast majority of people are not using trigonometry in your everyday life. Um, but the way that 3DE and Junior Achievement really brings the real world to the classroom, I love it. Are we doing this in ele any elementary schools or not currently? >> Uh, no. Um, and I do want uh to

071clarify there are four schools in the pro that were part of the evaluation, but we have we've expanded that. Um, and I don't recall if we're at nine high schools currently. Um, and so we are it may even be 11. Uh, we are slowly increasing um the number of students for the various reasons that um have been mentioned. It is definitely something that our students uh enjoy the method of instruction, the case studies. Um they go on field trips. Um they get to hear from um different businesses and uh really they're starting to tie in also entrepreneur a lot more entrepreneurship uh modules into that. Part of the um cost that we have does include personnel, not the teachers, but personnel, and it's also the curriculum that 3D um has. So, we currently have um

072nine, and I think we're adding Colonial Wakiva for next year as well. Wow. Excellent. Um excellent. I know. I was a little concerned. I'm thinking, man, we only have this in four of our schools. Um, I need to get this in, um, you know, something I would love to have more of in district 3 and really seeing this, um, flourish. I love the We could talk about 3D all day, but um, I really love what they do with elementary students as well. Um, and even something I think would be cool to have their um I don't even remember what it was called, but I know they have it in maybe in Atlanta and where they have uh the different businesses and the kids come and they actually get to pretend like they're in a town

073for a day and they get assigned different duties and um job descriptions and there's a mayor and a head of this, you know, you work at Starbucks or you work at the Home Depot and then they are using what they've learned in class and in a real world setting and how that um I feel like those are the um the experiences that kids leave school and will remember for their whole life. So, okay. 3D is awesome. Love, love, love the Rosen preschool. I had a lot of the same questions that mine was why did why do they level out and not continue to see growth? So, I think that's been answered. um and really seeing how we can um just continue that growth. Um and I think that'll be something we need to just discuss

074as we move forward. Um my question for the the scholarship with Rosen also was answered. Um, okay. So, Cass expert, I heard one um answer was that each school has a CAS expert. Is that a identified teacher or someone within the school already or is this an extra position? >> The CAS expert is someone already in the school. they get identified by the principles actually right around this time of year for the upcoming school year. >> Okay. And does this actually cost us something? Are we giving like a stipen to anybody for doing this or >> So the costs associated with this are for some of the professional learning that the teachers go through. So if those are occurring after hours, some of them have tasks associated with them that that teachers complete after the

075workday. So they receive compensation for that. Uh we also have a small team in the district office that supports this work. Uh so the that's kind of the the budget for that program. It's it's really the professional learning time as well as the district support. >> Is this the program that just to try to tie it all together? Is this the program that I think you guys were discussing at one point where it really breaks down the data and helps them understand a get a deeper dive which you all designed inhouse. So yes, that's one of the the tools within the CASS toolkit where they have a training developed for the CASS expert to give to their staff. And then there's a series of supporting resources which would be spreadsheets, documents, uh step-by-step directions of

076how to access information through performance matters and other platforms. Uh that's just one example. Um and I said substantially high earlier just to give a number on that. We've had over 76,000 views on that toolkit in the last two years alone. So, um, it's very widely used by by staff across the district. >> Awesome. Awesome. Um, do you know what it costs the district to utilize this? >> Yep. The total cost uh on an annual basis is $579,000. >> 579,000. >> Yes. >> Perfect. Okay. Thank you. Um, I do think it is very beneficial to be able to get that deeper dive and I think it is helping our principles. I have a principal at a middle school in district 3 who is really digging in and utilizing this and helping to um push the

077level and they're seeing a huge growth in their uh students academic uh growth which is why we're here. Um the VPK so OCPS VPK do how many not every school has a VPK correct and is it only certain schools can have it or is it I don't remember I think it's the all day preschool or VPK that you can we have um full day VBK in all of our title one schools uh we have expanded both half day and full day at some of our non-title one schools uh we are exploring uh this upcoming school year having uh the full day where parents pay a a small portion uh of the fee for them to attend full day. >> Okay. Wonderful. Wonderful. I love it. Um I can The other day I was um honoring

078Miss Aienza from Weine Guard Elementary School. She does the VPK program. Um she actually just sent me um an email regarding the and I won't be able to find it now but um for the fourth year in a row they received a national recognition for the growth in the work that um is happening within that VPK program and these children they're seeing that anybody who is coming out of that VPK class is scores the highest in their class even up to fifth grade. I would love to see, as some have mentioned, how that um translates into middle school or high school. Um but, you know, anytime we can help kids get uh a leg up and um really help them focus. So, I think in those classes, I really want to make sure that um

079those teachers are very well trained in the science of reading and helping those kiddos uh with that. But also then um as we've seen with math, you know, what we can do to to help that if it's, you know, more of the hands-on um math manipulatives that would help the kids. I'm not sure what would what we could do, but I appreciate the information. And my last question I believe is on the algebra and um if we are removing this program because it clearly was not uh doing what we were hoping it would, what would we be doing moving forward to help um kids who are struggling in that algebra classes? Do we have anything um kind of on the horizon or not quite? >> May Oh, I'm sorry. >> Go ahead, Dr. Asmus. Uh

080so one of uh just like uh being proficient in third grade is a key indicator in reading of how students are are going to progress uh algebra is a key uh uh determiner of how students are going to succeed in high school. And so for those students that are taking algebra in 9th grade, not all because some have special circumstances. They may have come from a district that didn't offer algebra in uh middle school. The the vast majority of students taking algebra in 9th grade are uh want are lacking in some very basic math skills. They don't know their times tables. Uh they don't have uh numerousy or um knowledge of some of the basic uh foundations in math. and and so we've tried a variety of um techniques. We've tried the tutoring. We've tried

081uh we have smaller class sizes where teachers are able to give more individualized instruction. And we've made some small gains in that area. But what's really going to fix our algebra problem is the best tier one instruction in elementary so that they are leaving school with those foundational skills. Um, sometimes it's very sad when you're walking into an algebra class uh and they cannot explain to you the concept of of multiplication or um when you're dividing fractions. Why is it that the number, you know, is larger as opposed to smaller when they're doing it with whole numbers? Um, they literally don't know their basic facts. If you're asking them what is 7 + 7, they have to count on their fingers. And so there's some I mean significant deficits. And so the vast majority of

082students that are on track are taking algebra in middle school. Uh so that's a long answer, but really we have to fix tier one instruction so that they're leaving elementary school with those core foundational skills. um and then they will be able to be successful in those higher level mathematics. >> Awesome. I Okay, I agree. Um and I think and just like with reading where we're seeing kids like the one in five kids have dyslexia or um some sort of reading uh difference where they are needing that uh you know explicit instruction. We also have the kids who have discalculia or um a a math um you know difference that's they need a little extra help with. So, um I really love hearing that we have we're going to focus more on that and I

083think by us focusing and being very intentional, we're going to see a better uh result in the end instead of getting to 9th grade, 10th grade, kids are not doing well in algebra and now they're they're stressed because can I graduate or not? Um so catching them earlier it will be better. I appreciate it. Thank you guys again for um all your expertise. >> Thank you, Member Fer. Member Douglas, >> uh thank you for the analysis of these programs. I really appreciate that and you're always doing a great job. Uh thank you. Um I just wanted to uh piggy back on what um member Bird said earlier about the mentor the mentorship of our students. I think it's great to have wonderful program for our kids and that they they excel but the goal is

084to make sure that they continue in their education and they go further and unfortunately um due to um problem at home or any other problem that students are dealing with they drop out and I I I just want to uh to bring that community together again. Is there a possibility? I know OCPS is really great in thinking out of the box to help our students and we have different organization in our community that will be willing to step up and be uh the the the the mentor for these uh men and women that in in high school uh that needs it that that push that um and I was thinking of maybe maybe church is not but I was thinking of church. I was also thinking about different organization like the Rotary Club or uh

085or any you know any great organization. The other day I was invited to um an organization called uh Empower Teens. It was absolutely amazing. I mean these young men and women were holding themsel like they were already entrepreneurs. Uh to learn how to shake hands, how to you know to to have high contact, how to to be involved in the community and do things for other people so they know how to be servant for the community. It was just amazing to be in that room. And among them there was a student from Colombia that was a salutarian of um uh Windermir High School from Colombia. Okay. I mean it was amazing the job that they've done in that little community for the students and they want to enlarge it. They want to increase it. They

086want to make it even better to bring more students to be involved in that organization. I don't know if it's a good time to say it, but I think we need more of of this type of organization just like I think Rosen was a pioneer in doing that in our community coming in and say I want to do something for the kids and uh and we have other organization that want to follow through and I need we need to look for them because they are there and they're ready to work for our students. Um the other thing that I want to say is um uh 3D uh thank you for the recommendation to provide targeting uh you know academic and instructional support for EL. I'm an EL kid. I'm just saying. Okay. Uh because those

087students are very smart. They they just need a different approach to learning. That's it. a little tweaking of learning and they they they will fly. So, thank you so much for considering uh making it better. So, my question is how what do you have in mind to make it better to include those er um and to make them more successful? Do you have >> Yeah, thanks board member Douglas. Um so I'm trying to stay in my lane as a supervisor of program eval here but we we provide the recommendations um and then there'll be um various teams who uh look at those recommendations decide whether to go forward with them and then determine the implementation uh process. But we can follow up with you on um what specifically uh those sports will be. but it

088will include looking at the data, looking at the GPA um and monitoring those students uh throughout their their coursework. But we'll follow up on that. >> Yeah, I would love that if we have the followup of this meeting to see the difference um and the VPK uh ex you know um to expand uh among economically disadvantaged student. What would you do to make VPK accessible for those student? Do you have anything in mind right now? >> Because May I so we do have VPK at um all of our title one schools. We work very closely with the early learning coalition. Uh we uh are looking also for uh expansion. So at schools, title one schools where they have um the need for more than one classroom, we've been able um to provide that. And this

089this year we have ventured out into uh the model that allows uh families in non-title one schools to pay a small fee to be able uh to attend. What we need uh to open a unit is to have enough parents that will uh pay for the teacher and the paraprofessional. The funding for VPK is different than our K12 funding. Uh we uh are paid by seat time. So the children if they don't attend if they only attend 15 days in the month, we're only going to um get paid for 15 days during the month. And so having u families that um are are interested in having their child attend a prek uh in our schools is something that we are looking forward to um expanding. Additionally, we um have uh I believe it may be

090East River and Lake Nona that have expanded um they go down to um age three and the more we are able to expand that model uh the more we'll be able to uh provide those services for our children. >> That's wonderful because that was also one of my question because now kids at 3 years old are ready to go to school. So, and parents are ready too also. Anyway, I won't go further from this one, but you know, some parents would like a little breathing, just bring the kids to school a little bit and take it, you know, but uh thank you so much for the work that you do and for uh the service that you're doing for our community. Appreciate that. Thank you. >> Thank you, Member Douglas. Member Salamanca. >> Thank you,

091Chair. Um a couple of things. Um so for cost, so given that we can't really do a control group, a non-control group like for that design like what what was our goal on evaluating? Was it mostly evaluating on is this liked by folks and felt effective or I guess I mean there's other programs that we can evaluate a lot more with the control group. Yeah. So, this would fall into the category of an implementation evaluation. So, we're looking at how well it's being rolled out across schools, looking for variance in the ways that it's being run. >> Got it. That was implementation. All right. That's right. Um, okay. And then for SA, um, looked like the outcomes of that were not ideal, but it seemed that it was wellliked. So like when we are call

092it removing a program and in this case it's great to hear that we got one million plus back um in the budget like how are we communicating to this to staff. I feel like staff sometimes gets like teachers or whoever is using this sometimes gets frustrated when they're like I really like that program and suddenly it's gone. Um and they don't get a clear understanding of the why it's gone. But in this case, like there's programs that they like that are just not effective. >> Uh so when we um first of all, when we start a program, a pilot, uh we make sure to communicate to uh those individuals involved that that's exactly what it is that it's a pilot. There is no guarantee that um it will it will continue uh with Saga and

093then I'll defer to Mr. Martinez, but I believe that was also communicated with them. Part of the issue was being able to attain the the tutors. Uh we worked um closely with the University of Central Florida to get a pool of individuals that could serve as tutors for the students. Um but once the decision is made to discontinue that communication is done um at the at the school level uh and I will confirm that it was it was funding that the reason was given and uh and with all of the programs that we have done away with it we we have tried to communicate as effectively as possible identifying the rationale uh whether it's the program evaluation or whether it's a loss of dollars or if it's a grant that you know this is for

094a short period of time and if we are not able to find additional dollars once the grant expires that we're not able to continue with it. >> Got it. Okay, that's helpful. Yeah, I think in general it was just when I saw the saga and it said teacher satisfaction was overwhelmingly positive but the dosage were not really significant and the control group and the mean school were about the same. So, I uh I just I worry when we introduce things and then we take it away and folks like it and we just don't communicate that like not only is it funding and yeah, it's great we got $1.3 million back, but it's also effectiveness and outcomes. I think that we do get a lot of um questions about, you know, why do we have these

095programs if they don't work? Well, here's an example of a program that is liked but doesn't quite work and we should communicate that and just have transparency around that. Um okay so that was that one. Um adjust the cause one. Um on the VPK one um super interesting um because this was actually one of the areas that I did one of my thesises on. It was VPK um and it was this big um Head Start study that was the National Head Start study. So, it's not unique to OCPS, but Head Start National Program shows that all of the gains students have in the VPK setting also wash out by third grade. Um, and that was seen across the nation. And so, it's not unique to OCPS. Um, and there was like three hypothesis as to

096why that happened. one was the skills you learn in prek um are much easier to measure and close the gap on versus some of the skills you learn later like vocabulary depth or executive function and so you start seeing these gaps on different skills and that was like I remember like one of the most likely one. Um the other one was actually redundant instruction and it was these students that enter strong from VPK um are ahead but then they enter and their peers are behind and they spend almost a whole year catching everyone else up and these students are just sitting around and so they kind of just wash out all their gains they had and disengage from like the process itself. And so that was I remember the second one and the third one

097was more just straight up the quality of instruction in K through three. That was like the third kind of hypothesis of the students that did have very high quality instruction after VPK after very high quality VPK continued those gains. Um and so I remember that I guess to like obviously we're seeing the same thing that nationally Head Start saw. I think every VPK program has seen this. Um, one of the things I also like to think about, even if call it these VPK students, um, and very disadvantaged students, ELLL students of color, um, are kind of at the same level as their peers by third grade. We don't have the counterfactual. So, there's a world where those students would be behind if they didn't have EPK. And so, I think it's a win. I don't

098think it's unique to OCPS. And then I do think that there's a lot of actual academic research and hypothesis as to why this happens and some of that we can address and some of them is a little bit more systematic and difficult. But um what keeps me warm after I did my thesis was the counterfactual which is they're not behind they're just about the same. Um and then last one was huge shout out to Elevation Scholars on um I think they have one of the best highest quality programs on preparing students uh to apply to college to think about college to think about funding of college and then supporting them post enrollment. Uh I think I've talked to um uh Jose about this, but the program has uh repeatedly offered resources and their time to

099come in and share those resources so that all of our students have access to the same uh type of call it materials or work with OCPS in a more formal way to work across more than just our title one high schools, which I think would be amazing. Um I would love to see that happen because they really have incredible work. Um, obviously they're uh have constraints. They're a very small team. Um, but I will take all of the materials they have because they're very high quality. So that's just my two sense there. Thank you. >> Well, um, thank you, M member Salamanca. H, great job. That is No, I don't need to. I I half of it I wrote. Um, no, I lived. Let me put it that way. Um it's so interesting that that

100you would that two out of three parts of your hypothist or yeah I can say this um were either things I were was thinking or my own life experience. So, um, yeah, I I was, um, sitting here really struggling at first with the VPK issue and then I started thinking about how different it is the skills of a sprint runner versus the skills of a marathon runner and totally totally different skill set. um it being being able to start fast and continue for a short period of time is very very different than than being able to stay in for the long run. And not that these skills um necessarily translate to that. But I do think that um that kids that are that are learning um decoding and really focused on that and then we

101kind of drop out of that engagement when they're working on comprehension and more difficult concepts. They're two different two different skill sets. And so when we later on we measure everybody by the same skill set, we're we're not seeing the advantage that we actually picked up, which basically tells us what we have already concluded. Maybe that's the why for what we've concluded. And that is you have to stay the course. Um you can't you can't say good, you're there. You did good in third grade and now we're going to walk away and you'll always be a good third grader. So um that's kind of the obvious takeaway from that. The other interesting um uh issue that you raised um was something that I lived and that was um we moved when I was entering high

102school and in um middle school what was then called junior high school. I was a really really good student in math in particular and um and and in most subjects and we moved to Jupiter and the education level was much lower. Um we had these books called SRO's. I I think that's probably before your time too. SRAAS. I thought >> I thought it was SRO. Yeah. SRAAS. Yeah. And so instead of evaluating where was I, they started me at the beginning of that. And so I had to do each one of these workbooks and and before I could advance the next and I kept saying, could could we could I just jump in the middle and figure out where I'm supposed to be? No, you have to go through this process. And so I went

103from being an A student. Yeah, you're looking at me like that's what I was thinking. And I'm sorry, I was I was just in ninth grade and I thought this was crazy. Um, but I I went from being an A student to a C student pretty quickly. And from there I transitioned to a I call it um a work release program. Um, but uh it's what it felt like to me. Um, where I went to school I think like I think I had to take like one or two classes. Um, and the rest of the time I worked because there was no value in sitting in the classroom. And my mom was just she wanted me to get a tutor. She thought I needed serious help. She thought the curriculum was just too hard. And

104and I kept trying to tell her it's not too hard, it's just stupid. So, um, there is that that factor that if if our students aren't challenged and if we've got these kids that are in VPK and and we've pushed them ahead and now we put them at a level that it's no longer challenging and no longer taxing, I just I think it's kind of human nature that you're bored and then you tune out and and then you get behind. So, I think that that's another really important factor in this is making sure that regardless of where our students are that we continue to challenge them. So, uh I I didn't do the thesis, but I completely agree with it. I would love to read it, but like I said, that that last part I

105I I lived it. Yeah. So, um, but, um, I think this is in incredible work and I I do want before we wrap up, I do want to say I don't know how many school districts put this effort in, but I continue to be amazed after seven and a half years at the amount of effort that goes into understanding what's happening in order to better our efforts to um, challenge our students to make sure our students come out as prepared as they can be. Um, Dr. Vasquez, I know I sing your praise all the time, but this is another area where you keep coming back to us with data that is extremely helpful in making decisions that data that that we up here aren't thinking to ask. Um, and I doubt that very many school

106districts have the kind of not just leadership, but dedication and understanding that you do, that your team does again to make sure that we're not we're not overlooking something that we're digging really deep and working really hard and it's really really impressive. So on that, nobody else is queued up. We're going to um end now unless you have any final closing comments. Your team's wonderful and you're amazing and this was extremely helpful. And we're going to take a break now for at least how long? At >> at least 10 minutes. >> Okay, we'll say 10 minutes and we'll see you back here in 20. >> Thank you. >> You know how this is going to go. Let's call it 10.