001Good morning. It is Tuesday, September 30th, and I'd like to call this regular schoolboard workshop to order. Uh, please stand for a moment of silence followed by the pledge of allegiance. >> Thank you. I pledge allegiance to the flag of the United States of America and to the republic for which it stands, one nation under God, indivisible, with liberty and justice for all. [Music] >> Good morning, fellow board members, Mr. Deputy Superintendent Natalie, uh staff, and the public. Appreciate you guys being here today. Um, as we get started, we do actually have a public input card today from Miss Elizabeth Emery from LCA. if you'd like to come up. >> Hi, good morning board deputy superintendent. Uh we're really excited to hear the status of the audit this morning. We had some great conversations with
002our audit folks. Uh so we're excited to see what they're going to bring today. We hadn't heard if the audit for instructional materials did actually go out. So I'm hoping that we'll get an update on that today. And I did also want to thank the board for your kind words and closing remarks as well as the superintendent. I know our educators really appreciated that at the last school board meeting. Uh hearing from you all really makes a difference. Um and as an alumni and Lake County school educator and taxpayer, I really appreciate it myself. Uh and I appreciate all you do for Lake County schools. I know that you all deeply care about Lake County schools and I've seen that firsthand working with you all and and the collaboration, the work that you all do.
003So, I'm I'm grateful for that personally. Um, but not all 2900 instructional employees know that, right? So, um, I think the best way that you can communicate your belief in them is by showing their value. So, I'll ask that you consider that today as as you deliberate. Um, and we're excited to looking forward to bargaining this afternoon at 4:30. So, thank you all. >> Thank you, Elizabeth, and thank you again, Mr. Matias, for that suggestion on including LCA in the audit. I thought that was very timely. All right, that is the only other card or the only card we got this morning. So, we're going to move on to item 3.01, the internal audit report. RSM, if you'd like to come up. >> Hi everyone. So, we're here to deliver two of our three reports
004from our audit plan that you all approved. So, we're here today to present our procurement competitive comp procurement and instruct instructional materials. We are still vetting the cyber security maturity assessment. We've v vetted it with management. So now we need to vet it with the superintendent's office. Um and then we'll be brought bring that final report to you all as well. So I have here today Jack Hughes will who will go through our procurement report and also Ava Dorisami who you're all familiar with for the instructional materials. It was very important for us to take our time with these reports. These were our first reports you know with Lake County schools for some time now. I have with us Jennifer Murtha who leads our government services industry um for for our firm but we're talking
005about when the last time she was here probably 15 to 18 years ago. So she was she's the one um that was leading this when we were last with Lake County Schools. So it was really important for us to make sure that we got to know management management got to know us and we talked to you when we presented to you all earlier um last year. We talked about the importance of building trust as we go through this process as well. Internal audit, sometimes it can be very intimidating what what we do and how we audit, right? So, we wanted to make sure we were building trust with management, really listening to them, understanding them, trying to understand their processes and what works best for Lake County schools. That was very important for us as
006we were going through this process. So, we really wanted to take our time to make sure that we were listening to management and really making sure that we were getting to the right issues and what's important to them. So, we're happy to be be here today and present these reports to you. I'll turn it over to Jack. >> Good morning, everyone. Thank you, Cler. Can I be heard? Okay. Yes, sir. Mic's working. Wonderful. I'll walk us through our competitive procurement report. uh just to give a high level kind of overview of the structure of our report. We do have an executive summary on page two that gives a really 50,000 ft view of the structure of our report. Uh we have a background which is pretty comprehensive beginning on page four that gives another overview
007of the different processes related to competitive procurement including some of the procurement types that are leveraged by the district. Uh we have an objectives and approach section which you'll find beginning on page 8 before getting into our observation matrix on page 10. I'll very quickly move into going through our observations unless there are any questions about the rest of the structure of our report. >> Forward. Any questions? >> Please proceed. >> Wonderful. Our first observation titled procurement compliance is broken into two pieces uh with subsections there formal advertisement and procurement documentation. As it relates to formal advertisement, what we noted is that all formal solicitations are posted to the online vendor link system which does facilitate a very broad outreach to uh a range of potential vendors. However, there is an existing board policy PO6322
008which requires construction related solicitations to uh advertise that solicitation in a newspaper of general circulation for a period of time. What we noted during our review is that there is no established process in place uh to advertise those competitive solicitations in the manner prescribed currently by board policy. And we did see a couple instances in our sample selection related to those competitive uh solicitation construction uh solicitations. For procurement documentation, we noted one sole source procurement did not retain a finalized executed vendor uh contract within the procurement file. Uh without that complete documentation, the district may be unable to support some of those uh purchasing decisions. So that was the risk identified as it relates to some of that procurement documentation. I believe yes >> I had a quick question if we can go back there
009about the newspaper advertisement. Is there anything that is statutory or state obligated for us to follow that or is it just something through board policy that maybe captured came over from Nola that maybe we don't have to >> Yeah, great question. There is existing statute that does provide guidance on the advertisement of those of that type of solicitation. I think when we coordinated with the legal council, the interpretation was that that statute didn't necessarily apply to the district, which is why we cited board policy. >> So, it's just a board policy issue. >> The board policy issue. And within our recommendation, we do say that we recommend that either procurement reevaluates its procedures and establishes a process to adhere to that board policy or consider making a recommendation to the board to change that policy
010to reflect its current practice of advertising uh using the vendor link system. >> Okay. >> Any questions about observation one? >> Nope, I don't think so. >> Perfect. Observation two titled selection committees split into three subsections. Uh subsection number one selection committee representation. We noted that the selection committee membership often is primarily composed of the department or enduser group representation and we did not necessarily identify evidence that representatives with financial expertise or with that financial background uh provided evidence or provided insight as uh to those evaluation committee members. And so we're not necessarily saying that it did not occur, but we were unable to verify evidence that that financial expertise and insight was solicited by the evaluation committee. As it relates to the selection committee uh composition itself, we did note that members of the
011public may occasionally be involved in the selection committee uh process for those solicitations. Currently, the committee relies on conflict of interest self-disclosure forms, but with members of the public, it may be difficult to identify whether to identify the accuracy of those self-disclosures and that the current process did not necessarily include a robust type of evaluatory procedure to attest that that disclosure was accurate and upto-date specifically for members of the public. And then finally, as it relates to construction related competitive procurements, as it currently exists, vendors are shortlisted by that selection committee, those short-listed vendors then fully present to the board and the board is ultimately tasked with scoring those vendors and making the final selection uh and award decision. In our experience working with school districts, that is a unique uh type of relationship here
012at Lake. um you're one of the only ones that we work with that has that type of structure. And while it's not inherently incorrect or wrong, I think there are opportunities to improve efficiency by leveraging the selection committee and that shortlisting structure. Um and so within our recommendation, we do have two bullet points, three and four. If the board wishes to maintain its current eval role in selecting those short-listed vendors, they consider implementing a conflict of interest at testation to provide for additional disclosures and transparency as an award an awarding entity. Uh, alternatively, the board could revise its current role uh, such that the identified selection committee as it as it exists currently continues its current role but also performs uh, an evaluation scoring of its shortlisted vendors as a two >> part of this
013packet is is the document you're reading from. >> Uh, that is page 13. I'm on page 13 in our recommendation. the observation itself uh primarily on page 12 there. It's this >> uh it should be the competitive solicitation report. >> Is yours is yours not more detailed? >> Oh, you're reading it right now. >> Okay. >> I have bits and pieces highlighted. Absolutely. >> Sorry. Thanks. >> Any questions? I know that that's a threepart kind of observation with the selection committee and then the board role. Any questions, comments or concerns about the content here within? >> So, so the conflict of interest right now people are self-declaring if they've got some sort of conflict of interest >> common and in line with best practice >> I would think. So, so short of us hiring a
014private investigator to find out whether or not they were accurate or not, what are other districts or other entities doing that >> I think that's what we want to highlight. there is an inherent risk especially with members of the public. It's a little easier um with the existing procedures in place with district employees to determine their background and potential uh conflicts there. But as it relates to members of the outside public while they may have a breath of expertise, it's more difficult to identify their background and and potential conflicts which is >> all right >> the inherent risk there. >> I think I heard you say inherent risk but it's a best practice. best practice to perform some type of attestation coupled with a robust background check if applicable. And what we see sometimes is
015Sunbiz for instance um research there different companies must disclose you know voting personnel and and key stakeholders within that um I believe that's currently being performed by management. We just want to highlight that risk with the members of the public. Let me ask Ali. Are they are we requiring them to sign like a form one? >> Well, not form one. We have a testation form. It's a seven pager that goes through the rules and all that. And we have them sign those forms. All all evaluators >> and generally speaking, correct me if I'm wrong, but the only time members of the public have been involved were in like for example Minola Horizons when Minnola had a couple members um involved in that process and other projects of that variety. But the the general procurement process
016or short listing doesn't include members of the public outside of staff. >> Yeah. Only in constru we include the public in construction projects. >> Uh other everything else is >> okay. Any other comments regarding observation two? Perfect. Observation three then on page 14. In our review, we identified one vendor profile which was missing an updated IRS form W9. We did note that while that form was not up to date, no payments to that vendor had been remitted since the expiry of that W9 form and through our review we noted with discussions with management that efforts had been made with that vendor to obtain an updated form. Um, so the expectation there should be an updated form on file. We were unable to identify that but management is currently in the process of getting an up-to-date
017form and no payments were made in the time where we did not have up-to-date documentation. And the only other comment I'll add here is for almost every single competitive procurement internal audit that we've done. There's a vendor documentation observation. In our experience, there's normally like five to six bullet points of things that are missing. So having just the one W9 absent in this case was a very positive story. Um it's rare that we see something that's this clean. >> Absolutely. So all in all pretty good result there as it relates to some of that documentation. Our last observation titled procedure manual. We identified several critical procurement activities that are being performed but do not have a documented procedure manual to accompany it. Procurement Services has noted that they are indeed in the process of finalizing
018a comprehensive procedure manual uh intended to supplement existing board policies. Based on our experience within our recommendation, we've aggregated some of the the key points as it relates to competitive solicitation processes to consider as part of that manual. And I believe management intends to publish that and finalize it within the coming months. And Ollie, I believe that's still correct. >> Perfect. So I think that's all to come pretty shortly here. >> And just to summarize, you we talked about this in observation number three. We've do procurement a lot for our school districts, even for our other government entities. This is a pretty clean report. Um we were pretty pretty pretty pleased with what we were seeing. Um so you should be very you should feel comfortable too on your procurement staff and what they're doing,
019the controls they have in place. We take a very broad risk-based approach on how we approach this, the risks that are associated with competitive procurement, what controls should be in place, and really getting in there and digging and testing and just making sure that controls are operate operating the way that they're intended to operate. So, this was a pretty clean report. >> Thank you, Ava. >> Board, any comments on this one? >> Yeah, I'd just like to reiterate what Ava said. Olly, thank you for the great work that you and your team continue to do. Um, going back through this, we've got an outdated policy for us to update on our end. Um, and we need to look at how we continue to do the construction part on our end, but not for what you
020guys do day in and day out there and going through the rest of it. But excellent work per usual and just want to thank you for what you do. >> All right, next. Yep. Next one. table. >> All righty. Okay. You should have in front of you I don't know that this Okay. Thank you for whoever is manning the report in the back for me. Um okay. I will be walking through the instructional materials utilization internal audit report. And um first before I get started I know that we have the entire team um behind me and they have done a phenomenal job in getting us the documentation that we've requested. Um I know there was a lot of documentation that we requested and they were probably the most organized that I've ever seen documents come
021to us. Um and just overall absolute pleasure to work with. So we were we were really excited to get into the details of this uh report and similar structure as what Jack mentioned for competitive procurement. We always have an executive summary at one-stop shop for kind of if you have 5 seconds that's the place to read. Um background on page four, that's where it starts, has information on the adoption process, forecasting, some roles and responsibilities, some financial data, regulatory requirements, all just intended for the reader to have some context because instructional materials is such a broad subject area. So, we wanted to kind of set the stage before we get into objectives and approach, which is on page 12. And as you recall, the information on this page should look very familiar. It's um a
022direct reflection of our statement of work, but as a refresher, our goal here was to take a look at the control structure around um how instructional materials are ordered, what volume is purchased, how need is calculated, um what are the inventory management procedures that we have in place spread throughout central district offices and decentralized school sites, and how we're adhering to policies regarding instructional materials purchase and utilization. So that was big goal and stepping into the observations next which start on page 14. You'll see the results um in our observations themselves. Starting with observation one. This is related to instructional material inventory control. Overall we found some inconsistencies and where material is stored. um how material is checked out for use, how inventory plans are documented and submitted, how inventory is received, the procedures that
023are like I mentioned previously very briefly briefly handled decentralized at each of the school locations. There's a lot of responsibilities as you can imagine related to textbook and um supplemented material use at the school sites. So just some areas where the actual use, checkout, tracking, monitor procedures can be tightened. Um I'll talk a little bit later in a a subsequent observation about some of the expectations for textbook managers um and how detailed those expectations are. This observation is intended to highlight some of the areas where we can tighten up those controls. And I'll also mention that this is based off of our testing of five school sites. And we take a risk-based approach always when we do our internal audits. We don't want to I think a good analogy is we don't want to have
024the doctor go to the healthiest patient. >> We want to be able to take a look at areas where schools or textbook managers or personnel can be struggling for any reason. um and then start there and make sure we can tighten controls to address any issues that we may find. Um not necessarily that these are issues. I'd say they're really just opportunities to tighten up those controls. Um and so the selection of those school sites were based off of one analysis of financial data. Who is where is the spend going? Where are the materials going? And then also some of the discussions with management. Um so I wanted to make sure that that's clear in the in the get-go that this is not a sampling of all schools. This is a sampling a risk based
025sampling of five schools. >> So let let me ask a little bit more about that because I think I my understanding is you kind of went back and forth with management about so the the what ended up here in the report in terms of of what line you printed um honing in on this one. We judgmentally selected five inherently high-risisk schools. Were all five of those high-risisk schools? >> My understanding was one of them was an exemplar, right? >> Yes. >> So why why is that grouped in with the high-risisk schools? We wanted to make sure that we also could take a look at, okay, we have some schools that are maybe considered high-risisk. We also have an example of a school that's doing well so that we can address if there are any inconsistencies
026in how schools are operating. It's clear um and kind of be able to do a compare and contrast. So why wouldn't you write that in your report >> that there is a >> because you grouped them all together as five inherently high-risisk schools but we know one of them is an exemplar and to illustrate your point that why would the doctor go to the healthiest patient that's exactly what you did but you didn't call out that that's what you did and and then I guess the other part of me is wondering more about the random sampling. So if you target specifically the high-risisk schools we already knew that there are challenges or or obstacles at those particular schools. So when we look at here and we see three schools didn't do this or four schools
027didn't do that, it doesn't I don't know that that gives me a a sampling of the entire district. I don't know. Right. I know at these five schools we already or four schools that we already knew about. >> Right. We can't take a look at every single school. >> No, but isn't there a sampling that goes into auditing? >> Yes. Yes. Absolutely. And taking a risk based approach. I think that sentence our goal was to say this is how we approach sampling. We can absolutely clarify language and say four high-risisk schools and one school that's exemplary. I think that's a a verbiage update, but our intent is to highlight that these schools were selected intentionally. But to your point, your point is good. Yes. To mention that they're selected intentionally. This is not just some
028random random sampling. We can't say, well, three of the schools didn't do that, so 60% of the schools across the district are also in the same boat. No, you specifically targeted four schools that you knew were had some challenges. One school that was an exemplar, but all five of those got grouped as high-risisk. Um, yeah, >> I I 100% see what you're saying. Yep. Absolutely. >> I'm just trying to call it out how it really is. That's all. >> Yeah, absolutely. And I agree with you. We're not intending to say we took a look at five schools and you can extrapolate that to the entire population. That's why we wanted to say high-risisk. But to your point, we absolutely can clarify the language that it's for high-risisk one exemplary. I think that's a simple update
029to the language. >> And who picked the schools? >> We we began our sample selection by doing some data analytics on overall purchasing trends. And what we noted, all five of those schools are represented as big purchasers or big, you know, recipients of instructional materials. And so while we understood with management that a selection of those five had difficulties in managing inventory or communicating with the district, all five could be considered high- risk based on the amount of materials being received on an annual basis or at least through the duration of our audit period. I guess where I feel disappointed on this one is that because your sampling is not very wide or random. We just have no idea if the challenges are isolated to these three particular schools that the management already knew about
030and called out to you to feed it to you in the audit or if it's more widespread across the district. >> I don't know. >> Our recommendations are geared towards the entire district. So on the sub >> we don't even know if it's a problem at most of the other schools, right? based on the audit and and how you approached it. >> I mean, this is our this whether we're looking at something like instructional materials, whether we're looking I'll give timekeeping as an example. We can't go to every school, right? We have we have to pick a sample and part of our process that like like Jack said, we looked at the um the spend. So, technically, even the one that does things well, it does have high-risisk components because of its spend. Um, but
031yes, we can still modify that sentence. But even back to let's say if we were doing timekeeping, we would still select five schools and we would ask work with the district. This is what we're seeing with different classifications. This is who we think have more higher risk because of the different attributes. Who do you think you need more handholding with? Who do you think does things better? We want to make sure that we're getting a mix, whether it's three that need more handholding or two that do things better or four that need more handholding and one that does things better cuz then you can kind of see whether or not some of those controls are operating as intended, you know, throughout the district. That that's just that's how we do our judgmental sample so we
032can make sure that we're getting to the cusp of the risk and seeing how we can make sure that the risk is being controlled. I I think we'll just have to disagree on on how you pick a sample because I I recognize that you're not going to go to every single school, but I think it's also challenging to for us to know are these isolated to these three three schools that were handed to you or is it more widespread like I think in auditing you pull kind of a random sample and figure out but that's okay. I I want I'm ready for us to go as a district and look at these other schools. >> Yes. But who's got the labor or time to do that is my wondering because they're a department of two
033and I think what as I read through this this report a lot of your observations are what's going on in this school setting. Um and and that that department I don't know that has any supervisory um powers over the textbook managers and so I don't know I don't know what you know >> well that's isn't that a discussion internally for us >> perhaps perhaps >> I want to go back to the core so I understand from 30,000 ft you looked at the spin and then from that spin you looked at the largest elementary and the largest middle and the largest high school. Is that how you got this random sampling? >> Correct. Well, I wouldn't call it necessarily random. It was still judgmental, but we started with that spend and to get diversity across different
034school types, we made sure to include an elementary, a K through 8, a middle, a high school. We then took that sample. We had pre selected five. We went to management and said, "These are schools that we see that may be inherently high risk." We actually had pretty considerable overlap when we wanted to get their input from the process owners to say, "Where do you see controls may break down? where which schools have difficulties or different unique challenges that stress the controls that are currently in place. And so we had pretty significant overlap with our originally identified selection of schools which gave us comfort to proceed forward with that selection. >> If I may, um I think it's extremely important that we hear your report. Um I don't think this is the time for us
035to start trying to defend what you're presenting to us. I think we need to hear what you're saying with an open mind. And then once you've done that, if we see some what we might consider biases or other things, you know, certainly there's a time and a place to bring that up. But I can attest to the fact that the way you chose your sampling is very similar to when DOE comes in for ESC files or they randomly pick schools like that. So this is >> this wasn't random. They were handfed the schools, >> but it was based on criteria. >> Okay. >> And that's the same way ESC does it. That's the same way Title One does. The >> rest of the report. >> I think that's what we should do. >> I'm happy
036to get into observation number two then. Okay, that's on page 16. This observation is related to material forecasting and ordering. Um we did find that there are certain criteria that the department um recommend is used in the forecasting process. That could be full-time equivalent student counts um data within focus, proposed order numbers submitted by the textbook managers, um actual course enrollment. When visiting uh our sampled schools in person and speaking with the textbook managers, we ask questions. How did you come up with this order? um what type of information are you looking for? Um and we did find that there is some inconsistency in what data points are being used? Um and then in the subsequent bullets, you'll see that we did also take a look at what inventory was actually on site and tried
037to see if there's any surplus inventory. Um we did find that one of the schools had ordered double the amount of materials needed for a semesterbased course. um in discussing that further with that particular site, it wasn't identified that a single set of required course material could be used for both semesters. Um and I I'll talk to a little bit about that in the recommendation in a moment. And then the four remaining sampled schools, they had some surplus order quantity that ranged from 10 to 29%. Um of course, we do expect some surplus um to make sure that there's no instance where a child doesn't have their textbook when school starts. Um so this is just a matter of again kind of tightening those controls related to forecasting and ordering. And you'll see the management
038action plan um does talk about collaborating with school leadership to confirm the course offerings as part of the forecasting and ordering process to get those sight specific data points. Um coordinating with school schedulers regarding course enrollments. Coordinating a date for master schedule submission in conjunction with textbook ordering. So again, that's another instance of getting an an additional data point before submitting the forecasted order. Um, and continue to implement a staggered approach to ordering. That's something that the team had implemented um, fairly recently as well as staggering the order so that if there is ever an instance where we need to pause on an order, we are able to do that. Um, their estimated completion date is August 2026. So, in any any of those schools where they had seemingly overordered um was there a declining
039enrollment in any of those schools that that might have contributed to that or um or was it just that they're kind of getting carried away wondering about well maybe I'm going to have a lot of kids who move out or you know they take their material with them and I can't pass it on to the new kid who comes in or or they >> Yep. Great question. So for as it relates to declining enrollment, um I would need to look into that or refer to management, but I know the school sites that we talked to uh in these instances and ask like, okay, what does the surplus inventory come from, they were also the same school sites that indicated they were not necessarily speaking to each of the instructors about what they need, what courses
040being enrolled. Um, we did have an instance where a textbook manager was having that direct communication with instructors um, and saying, "What do you need? What courses are being offered? Can we use this material also for this course? Does that make sense? Did you like using this material last year? Yes or no?" And then having a more tailored order. Um, and then there were schools that didn't do that. And it was strictly just based off of, okay, how many kids do we think we're going to have? Um, so same schools that we talked to had those same struggles as the ones with the surplus. But again, as it relates to the enrollment, I would have to look into that or deferred management. >> Yeah, I'd be I'd wonder about that because I think part of
041the, you know, why we're not set up for success in this just by nature of the game that we have to play is that we pick a number. Let's say a thousand kids are going to take this course and when we adopt the the thing we say they're thousand we're going to place an order for thousand books and then we're locked into paying for a thousand books for all five years of that contract. Uh but if the enrollment goes up we'll pay extra. If the enrollment goes down we're still paying for books that we're not going to use. And so my wondering is is some of this inventory related to that? I don't know because you know we've already paid for those books because we >> I don't know. >> No. Nope. That makes total
042sense. and and that was one of the discussion points that we had with with management on there um related to timing. Textbook orders are submitted in March prior to finalizing student enrollment or July and August. So with that timeline, there's absolutely risk for us to submit an order. Then we get the enrollment data and we're like, okay, wait, that was too much. Um so I think that's the management team has already identified that staggered approach to ordering to try and assist. And then also that coordinating the date for the master schedule submission and including that the curriculum department as part of that process I think is another identified area where we might be able to alleviate that. Right. >> It seemed like to me because you just were as you were giving articulating what one
043school does with the coordination, the communication with the teachers and all the departments that must be your exemplar school I would sense and >> so that seems like that would be best practices but it's not in here as a recommendation >> unless I'm missing. >> We have uh enhance additional collaboration between textbook managers and instructors to obtain additional insight and to order preferences. it it goes on. That's a third bullet point in the recommendation. Um >> to your point though, we did want to address that recommendation, but still leave it broad enough for management team to decide what works best. Um once we actually start implementing, you may need a pivot. Um so we tried to include the language I just mentioned in bullet point number three, but also wanted to make sure we gave
044some wiggle room. >> It reads a little different than you said it. I like the way you said it better than >> All right. Was anyone taking notes on what I said? Go back and let it >> recording. >> Thank you. >> Yeah, it seems like that's really a tough job is managing >> textbook managers have a lot of responsibility and so we wanted flexibility in that recommendation to acknowledge that they're balancing a lot of workload while giving management the opportunity to explore what opportunities to improve the process may look like. And for my edification, is that a position at a school? >> One of my questions I think when we first met was I was interested in knowing who is doing that job at the schools. How many of them are assistant principles? How
045many of them are loadbearing instructors? How many of them are non-loadbearing instructors? Did you all ever come to a conclusion on who these people are, how that comprises, and just just to get a sense of how much they've got on their plates? because I you know when if you've got a lot on your plates it's hard to manage all this. >> I think management management might be able to add in can you need to come up to the microphone. >> I'm gonna have to bring my person there. >> That would be you, right? So that's good. >> So um I do want to acknowledge that Kelly Dodd and Iana are here and they are the masters of this piece. I came into this obviously very late in the audit process, but I can answer that
046question. >> Okay, >> there are 22 um textbook managers that are APS, assistant principles is their primary role. There are nine that are coaches, some kind of instructional coach at the school level. There are five that are deans and eight that are media specialist. So while somebody does have the role of textbook manager, it is not a job in by itself. >> Yeah. So >> they have many other roles as well. So, I think it's important to kind of just process things through that particular lens, you know, and that they've got plenty of instructional duties throughout the day and and being the leaders of their schools. Then they're also trying to like be warehousers at the same time, but it's a lot of hats. >> Is there a stipen or anything tied to that assignment?
047>> Not that I'm aware of. >> There's not. >> No, sir. >> Okay. >> Any other questions for Dr. Felner or M? >> That was that was great data. I guess my question would be is there any um indication that housing it within the media specialist works better than the assistant principal or vice versa or any of those other roles like did we find any efficiencies with who was managing the textbooks or is it really just approach of how the school was doing? >> I don't think that we have data to support any opinion. >> Okay. I mean >> who was the exemplia specialist? >> Traditionally it's given to an assistant principal. uh I had it as an assistant principal, but you have other targeted people who help you like the grade level chairs or
048the department chairs and they actually do the bulk of the work. The textbook manager him or herself isn't the person who's physically doing the majority of the work. >> I do want to reiterate just for the to be fully transparent, the exemplar is a um a smaller elementary school of ours. So the the difference in the amount of materials could influence possibly >> but I thought we took the highest risk highest purchasers did >> a few minutes ago. Didn't we say that? >> Mhm. >> We wanted it could be not highest when we look at all schools but when we're taking a look at school blocks >> it's the highest of the elementary schools. >> It's one of the highest. I would need to confirm if it is the highest, but we certainly don't include
049middle or low. >> Miss builder, do you have >> um I know that the school was Seminal Springs Elementary School. >> That's not enormous. >> That was the exemplar or that was that was the exemplar >> included. Okay. And they were not included in your >> they were they were >> y >> as one of the larger schools. >> That's the model. to to clarify are >> I'm afraid you missed the discussion about sampling random sampling and whether it was truly random or not and and that methodology but >> miss Fner to clarify are we >> smaller school by enrollment number or smaller school by spend >> uh by enrollment number >> so that that might be the point of clarification while it may be smaller by enrollment by spend on instructional material it was
050one of the higher represented elementary schools >> that doesn't make sense to me >> me neither right if they're an exemplar and they're a small school then they're probably not overordering Right. So why why are they so high in volume if they're lower in >> because they may choose to buy other instructional materials that some schools don't purchase. >> So that you're talking about the amount of money they spend, right? >> Yes, ma'am. >> I need to correct myself. Dr. >> I I think my comment to this would be at this point I mean I I agree with what you're saying, Mr. Dodd. If it would make you feel better to make it say selected schools instead of sample schools, then that can I No, I don't mean that. >> I'm really ready to get
051past that. It just it just keeps coming back to this that this sample right looks wonky. >> But I think the challenge is we're looking at five schools out of time and a cost basis. So if we had 10 or 15 and can get a true sampling what's going on trying to get some best practices of what is working in one maybe or what are some areas to improve in the others. I I don't take any of this as critical or like we're looking at staff saying they're doing a poor job. I think this is it's a large line item. We're going through challenges in budgets and trying to see where we can tighten as much as possible and know that we're going to have some declining enrollment in issues where we're going to over
052orderer in certain areas, but I don't take any of this as we're overly critical on staff or thinking schools aren't doing a good job. Just trying to get a baseline of where we are today and how we can improve. >> No, and I I agree with that statement. Certainly, we want to we want to improve. Um I'm I'm just I guess eagerly awaiting to hear where the cost savings are in your audit because as I read through, I thought the ideas didn't come from you. They came from all the brilliant people in that department. And so you I'm just wondering did you all approach this from a compliance piece or from a efficiency savings side of things? >> Both. And I will mention the inventory control procedures. While while that is the backend, the materials
053already been ordered, it directly feeds into how we are approaching the next order. So we take a look at it from compliance with what are the established policies and procedures. what is in compliance with what we heard the expectations are and then also while we're walking school sites while we're having discussions with technical managers and the team how can we make this process better more efficient have less surplus etc so I would say it's from both lenses >> you answered it very well >> okay >> um I can move to observation number three >> yes please thank you sorry >> observation number three is instructional material utilization and reporting. Um, again, this is one of those, to your point earlier, this is one of those areas where we can establish some stronger controls as it
054relates to tracking what materials we're using in order to make informed decisions into what materials we need. So, down the road, that can potentially be a cost savings. But, I'm getting a little ahead of myself. What we found here is that um there's not currently a formalized process to monitor the use of our adopted digital instruction materials across grade levels and subject areas. Some of the providers that we have um at the district offer a dashboard that says here's how much uh what your utilization has been for a particular digital product. Others don't have that. Um the district does have a report called catch on or a system called catch- on that can track utilization. It was established for other reasons but it could potentially be leveraged to track utilization of digital materials as well.
055And the recommendation there is to start getting an idea of how much of our instructional materials that are purchased. Um, some of them are bundled and they come for free and you may not want to decide to spend the time or energy looking at those and I think that's very understandable. Some of them that we may be paying for whether it be supplemental or part of the larger bulk purchase purchases. Um, if we can get an idea of how much they're being used, how much are being logged on, what the data suggests for how often a child is actually interacting with that product, that may help drive some data informed decisions on what we actually need to purchase moving forward. Then the second part of this observation is related to supplemental instructional material. And in
056that instance, we did find that while there may be some spend by school reports, there's currently not a process to review those reports to identify um any trends in what's being ordered from a supplemental perspective. So again, this is outside of those bulk purchases that we know are going to occur, but may be occurring at individual sites, but seeing if there are any trends in um how we can group purchases together, potentially save um by getting discounted pricing, avoid unnecessary delivery costs, um identify any opportunities uh to scale that type of data. I think the data likely exists, but it's not part of a regular review process where we would be able to identify that opportunity real time. So recommendations I think are are pretty straightforward. I kind of explained them in that um in
057my summary just then. Management action plan is to use that system that I mentioned earlier catchon um for monitoring digital license utilization for schoolbased for digital based courses. Also going to be using Destiny that's an inventory system for tracking consumable materials at the school and district levels. So I think again both of these systems have been in place. We're just talking about using that data in a more intentional way to look at what we're doing next year or the next adoption cycle. Similarly, uh management stated that there are plans to increase communication with the school leaders regarding those specialized courses being offered at the school site um and coordinate around school-based supplemental materials. Um and all of those materials are all of those actions are something that's going to be implemented on an ongoing basis.
058One of the questions I'd also asked in our pre um audit interview was about some of the delays and sometimes getting those in um digital licenses activated. >> Sometimes a good amount of time goes and I was wondering if is that an IT challenge or is it a publisher challenge or is it kind of combination of both or did you all did you all look at that? Yeah, I catch on that system that I mentioned earlier could potentially track, hey, here's where we're getting the actual folks logging in and here's where we report it. That's not >> No, no, I don't think you're understanding the question. We're we're like our our students or teachers don't have access like it hasn't it hasn't activated yet. So, of course, Catchron's going to say they've got zero usage.
059you know that I know sometimes there are courses and for whatever reason the digital access doesn't go live until maybe you're weeks or months into the school year and and I don't I don't know much about that process. Certainly that is way over my head. Um and so that was that's why I had one of the that was one of my questions going into the pre- audit but it doesn't sound like you all had a chance to look at that one. >> We talk about it with the textbook managers on site to try and get an understanding of where that potential delay with digital licenses could be. Um, and there wasn't any cited anecdotes that would say it was a publisher issue. Um, so I do think setting up the information that we talked about
060earlier in the recommendation and the management action plan will help identify when that is occurring and what the issue is. >> Okay. >> Observation number four. Um, here we did uh take a look at what are the expectations. I've talked a lot about compliance to expectations. Um, what the expectations are for with textbook managers who already have a lot on their plate. Um, and we wanted to take a look, okay, what is the documentation for succession planning or to teach a new textbook manager or to teach someone new in the curriculum department um, how to most effectively do their role. Um, and we did identify just some instances in where there's some key activities that are not yet written in a in a detailed procedure. I think overwhelmingly and you mentioned this earlier u Mr.
061Dodd is that the team is extremely knowledgeable. Um losing any team member for any reason would be an absolute loss of institutional knowledge and there's so much work that's being done behind the scenes that's not necessarily documented. If for any reason uh we always use the the um the situation if someone wins the lottery and they leave then that individual who comes in to take their place would have a lot of learning to do. So we just want to make sure that there's specific re written guidance so that that person can kind take off the ground running. Um and the team has indicated that they will be developing um roles and responsibilities specifically for the instructional materials and media services manager in August 2026. questions, comments. Okay, I will talk next about process improvement opportunities.
062So, process improvement opportunities are a little different um in that they're not necessarily reported observations. They're more just instances where we identified something could be considered to tighten something up. Uh whether it's a process or maybe we have something in place and we would recommend considering going about that process in a slightly different way. Um the first one here is coordination of excess inventory. Um we did find based off of our discussion with individual school sites that schools with that surplus inventory are rarely communicating their availability uh of surplus inventory outwards. Um so in many cases the surplus inventory that we identified on site it had just been sitting there for a long time. Not necessarily there was no indication that everyone knew that that surplus inventory existed. And if there's a school that has
063an unmet need, we would want to just enhance that coordination. We do know that the um curriculum instruction and assessment department um does coordinate the use of surplus inventory, but of course that's an additional task. Uh that may not be done consistently or at a regular cadence. Um, so we do recommend considering uh and implementing a a centralized surplus reporting procedure so that schools can kind of take ownership of that surplus inventory that exists and distribute it outwards if necessary. And then the last process improvement opportunity is material adoption feedback. Um, right now there is certainly a strong culture uh in practice of engaging with teachers in the instructional materials adoption and ordering process through the use of surveys. Um, we do note some opportunities to engage with those educators post adoption to see what
064was working, what wasn't working, what data can we get that might be able to inform us in the next adoption cycle, potentially identify some cost savings if there's something that we don't need to order, if we can pivot in any way because there was maybe a lot of supplemental material ordered instead. So here just recommending that we enhance those survey procedures and maybe adding some post adoption data points. I will pause that that was the entirety of the report but now that we've gotten through all the observations I'm happy to answer. Oh, I'll also mention there's survey results that start on page 23. Um that's essentially a raw data export of these survey results that we received. Um, based off of the responses received, we didn't notice any significant trends that indicated a strong opinion
065one way or the other. Sometimes we we issue surveys and there is 80 to 100% of one response all the way down. And to us that indicates there's obviously a strong opinion somewhere. Based off of the responses that we receive, they were pretty mixed, which in most cases is a positive story to tell. Um, but we did include that. Uh so you can take a look at the data itself and again we don't manipulate any of that data. It's it's simply just a export of of what was received. >> Um okay. >> Is there a standard like the completion rates at 8% and it >> is there a standard that you guys typically that makes data meaningful? This seems really low. >> When when was it sent out? >> Yeah. I I was going to
066ask that as well. >> I August 8th I believe it was. We delayed, as you may recall, >> the first week of school >> we and we had left it open for several weeks after that, but we delayed initially because we didn't want to send it out during the summer and only receive responses from >> certain instructors that may be online working and completely omit the others that didn't. So, um, we delayed starting the survey and then we left the survey open as long as we could in order to present this on time. Um, >> it looks like looks like I I see it in the box here, 8th to the 15th. So they had a week >> uh they were advised of the survey. Oh yeah. Of the 15th you may need to >>
067So they had the the first week of school and the first weekend of school to to do the survey. That's probably why they got 8%. >> Well and I wonder also did it go out to teachers specifically like the list serve is it was it sent to teachers in the district? >> Yes. >> Correct. Yes ma'am. >> And we coordinated through it to push out that notification out. Did you guys communicate with LCA at all to help spread the word about the survey? >> We actually met with LCA before sending the survey and got their feedback as well, >> right? Not feedback about the survey, but like they can sometimes help at the school site by saying, "Hey, there's a survey coming out." So, people are kind of overwhelmed with the start of school or
068pre-planning. Was that coordinated or just the input on the survey that the data in the survey >> and to know it's not spam, right? I mean, if I hear it from LCA, then I know, oh, this is pretty legitimate. This is actually specific to Lake County. And oh god, >> I believe the link as well as additional follow-up reminders to complete the survey were sent out over the duration of that piece. Uh, yes, correct. Through email >> and that would be a great comment for LCA. I'm not sure if you had done any additional communication outside of what we did through it. >> Okay. I didn't want to speak on behalf of you. >> Thank you. >> Well, I'm not sure that they were aware that the survey was going out. Right. I am interested,
069Elizabeth, and if you don't mind coming to the podium hearing, now that you've sat through the presentation, I do want to make one comment. This would have been really nice if we had this in our hands before sitting here and trying to absorb all this. >> Yeah, I agree. >> This was our understanding was that this was sent out with the agenda a week in advance. >> Wasn't >> I saw it yesterday. >> I don't know which day it hit board docs. Yesterday was the first chance I had to to look at it, but um >> Okay. Okay. That could be a we'll double check our communication. Our understanding is that we sent it out at least a week in advance to make sure that everyone had time to read, but if we need to
070maybe send it out two weeks in advance or coordinate a different way, we're happy to do that moving forward. >> So, so now that you've heard the presentation, do you >> what do you think? >> Well, to Dr. Luke's point, I think we would have loved to know when it was going out so we could elevate it and let our teachers know. I mean, you all know the first week of school is crazy. They get tons of emails. So, if there was some way to communicate that to get better survey data, uh I think that would that would be great and we would love to help support that. Um I think like Mark Dodd mentioned, I I am curious about some of the other schools and and I think there's definitely a lot to look
071at here. It's a very vast complex piece. Um but I think it is something worth looking at and and trying to make some adjustments. So, um I'm not an expert in this area. I think obviously there are some folks in the room that could probably speak to more so but um from what I heard today those are my quick takeaways. >> Thank you. >> You're welcome. >> Dr. Felner, you have anything to add after hearing the >> or Miss Dodd or >> I have a question if I may for Mrs. Dodd. Do you ever have communication with instructional leaders in the schools as to what the textbook managers like for example with the survey that went out were the textbook managers notified and maybe this is part of yours that this is going out and
072so you need to encourage because they're the people that have the direct communication with the teachers. They're also the people who have the authority to uh put some teeth into it. So, do we let them know that this is going out and these are the dates and you need to announce it, you know, in faculty meeting or whatever? >> Great question. And yes, ma'am, we actually shared it at the textbook manager meeting that the audit was beginning also that there was a survey coming so they knew and um we shared that survey through focus so that it would go directly to the teachers as well. Um I think it was they weren't sure when the survey was coming out because again we shared that at the end of the year at the textbook manager meeting
073and we did not meet again before the survey came out. So, they didn't have the exact timeline for the survey, uh, but they knew that it was coming. They knew to expect it. >> All right. Thank you. >> Yep. >> Do you have any other comments? You want to >> um I would just say that it was it was a pleasure working with the audit team. Um it was tricky for me and I feel like um it was tricky in the in the respect that it was select schools and those schools were identified as schools that um are have more difficulty getting to textbook manager meetings or and or submitting paperwork that we needed. Um so it was not a surprise to me that maybe they had more inventory. It is a regular procedure that
074we talk about in textbook manager meetings about looking at your counting your materials before you order each year. Um, and I I want to say that our textbook managers really do a great job using the email lists um to reach out to other textbook managers when they're short materials and navigating when they do have surplus sharing materials. Um, that is like a very busy place at the beginning of the year and they know to reach out to Iliana and I if we um if they cannot find those materials they need um and we try to really quickly resolve that issue. Um and we actually Destiny has a new system. Sorry I'm a little under the weather. a new system that allows for us to track consumables that they came out with this year. Um so
075when we rolled out the science adoption this year, the consumables are now being tracked within Destiny. So we know at any given time how many um consumables they have in their in their possession and we plan to do that with each new adoption uh as we roll out those adoptions. That's going to really help with the inventory management that the auditors mentioned. Did you say that some schools don't show up when you have a >> Yes, it is tricky because again this is not like a specific job. It is a role that they have. So, um I know the question was asked earlier about positions and I think that it is more challenging often for an AP to get to a meeting than it might be for others. Although media specialists do a great job
076because they are very knowledgeable of destiny which is how we track textbooks. Um and but they do wear a lot of hats as well. I think all of these individuals wear a lot of hats. Um, and I understand that they are feeling very stretched. So, I don't think it's intentional that people miss meetings, but I think that they're feeling like they have a lot going on at their school campus and sometimes they can't get away that the meeting information is still shared with the textbook managers following the meeting. So, they can definitely go through and look at the the documents they missed and that kind of thing as well. >> And is the meeting in person or is it? >> Great question, too. We have two in person. We have the beginning of the year
077in person as we talk about new adoptions and all the rollouts. Uh the second meeting of the year is um virtual so they can come meet um and and tune in with us with a variety of options for that meeting. The third one is getting ready for ordering. Um and that one is also in person and then the last one is also uh virtual. Uh and I wanted to mention too real quick if it's okay. The timeline for ordering is tricky. I know that one of the things that was mentioned was a proposal for maybe moving that back a little bit. The hard thing with that is that we can start placing orders as of April 1 with FSBD. Um, and we really have to start placing orders as of April one with FSBD in
078order to get materials to the warehouse to barcode them in early July and then get them to schools for the first day of school. Um, there it's like a line at Disney. The earlier you get in line, the better spot you get and the earlier you're going to get your materials. So many things even as it is and I think we're one of the earliest orders um are backorded. So we had to wait for materials and it and it creates more um time on our on our warehouse guys hands to to ship materials after they arrive later. multiple trips, that kind of thing as well. So, we really do like to get the orders out as soon as possible. And uh we try to delay anything that we can that like anything that's digital only
079materials. We try to wait for those orders so that we have more finalized numbers once enrollment is complete and scheduling is complete at the high schools. >> So again, this is educating me. >> Yeah. >> So at the school level, they aggregate what they're going to need. >> Yes. >> They feed it to you. >> Yes. there is a Google doc and it's a Google sheet and they input what they have what they need for each subject area for each course um and they subtract out what they currently have in their possession so they give us a final order for each thing and they share it on a Google sheet we use that Google sheet to go through and place the school's orders but first we compare it with growth planning numbers and also our
080focus enrollment numbers but um again there since we we often don't have like a final date of for enrollment so we're watching to see when seats are filled at schools to know exactly what the enrollment Um, and it has been great. They have set up like my own little special focus report that can show me I can search by course and I can see how many seats are filled and how many seats are awaiting or requested. So I know I can kind of tell um when it's closed out or when it's not. I can see the percent finalized of a school for their their scheduling. We and again we've done this for years but we we do elementary orders first because they're easier. Middle school orders are due second. High school orders are due at
081the end of March. So, we try to build in more time for high school because we know it's trickier for building schedules. I know, you know, if I'm in first grade, I'm going to second grade next year. So, >> and you're placing that order your department. >> Yes, we place that order. We start placing those orders right at April. >> All the schools and then trying to get >> Yes, we place all those orders. >> Totally. >> Yes. >> Very good. >> Awesome. >> Sounds like a lot of work. >> It is. We have a great team >> and a lot of money. >> Yes, it is a lot of money, too. >> Your team is three. >> We It's It's the two of us. >> Two. >> Yes. >> Okay. Thank you so much.
082>> Thank you, Kelly. >> Can I go I'm just going to go back to my original question. What typically, and this is for me going forward and understanding your audits, what is your percentage response rate that you consider statistically impactful for the data to make to be meaningful? It >> It's a great Sorry, it's a great question. Obviously, we would prefer as high a completion rate as possible. >> If you're looking for like a statistical confidence number, that's something that we could give you. Um I'm unfortunately not I can't do that in my head but you can calculate it. Claire, did you wanna >> I was just gonna say it it's the survey was more forformational purposes forformational gathering um with in the the few instances that we do send out surveys as part of
083our internal audits. Again it's for moreformational purposes so not from a statistical perspective etc. I just I I that gives me just concern because some of the conversation that we're having, I know it's informationational data, but it's informationational drives decisions and funding and and procedures that we need to have in place. And it it seems I I can see benefit in this. I don't mean to I'm not trying to be argumentative. I just want to know like when you're looking for information from a district that's this size, is there a typical threshold that you look for for the information to be indicative of the district? Usually sample size. like we like it when it's over 50% because that kind of gives us more of a a better basis, but this one was pretty low, but
084typically once we get above 50%, it does give us a better kind of insight and be able to give um a better basis of what it is that we're looking at. >> Right. So, is there a point in your um in your procedures that you say, "Okay, we're going to try this again." Because the survey rate was the return rate was so low. >> We we went back and forth. when we did receive when we did receive the results cuz when I when they first presented it to me I was like wow this is pretty low from what we've seen you know elsewhere again the few instances that we do send surveys as part of the audits that we do because typically it was just pertaining to the audit the process the risk controls that
085we're testing um so I talked to the team about what was the communication what were the reminders all of that and you know and they it went through the process that I would have expected to kind of go through you know from that communication lens and it just this is just where it ended up at. >> Okay. >> And you know cuz because I think there's potential here of I'd love to know the answers to some of these questions but you know it's hard to put a lot of confidence in 8% return rate like you know for instance 30% of people who responded said they occasionally monthly or never use district supplied materials. I I don't know as or 67% of people are purchasing additional supplemental instructional materials to complement what we're providing. 30% aren't
086even using what we what we're providing. Why? Um 82% of the people are using their personal funds to pay for these materials, you know, and you look at it here and it's going to online teacher paid teacher sort of content. It just makes me wonder, you know, and I mean, are there some things here from a from a curriculum side of things that we need to address? Or maybe not because 8% return rate, you know, so like I'm not going to jump out being alarmed at these, >> but it's it's a wondering I'm I'm left with, >> right? We we were left that same wondering as well when we received it and we didn't want to not include it because we talked about this. We said we were going to do this. So, we didn't
087want to keep those results from you all. So if I can make a comment, I think part of this is also an issue of timing from we finally got on board with RSM in what March or April. We started this as one of our first audits out the gate in April, May. This is one of the three that we targeted is let's get started and then it was a matter of okay in hindsight we started at the end of the school year. They did the bulk of their work or started to then starting in the middle of the summer and then we got to the survey part and okay let's wait for teachers to come back and be able to get some results and it's like okay we need to report this audit back within
088a reasonable time frame. So, I don't think any of this was purposeful. I'm not saying you guys are saying that, but it's also a standpoint of I don't think 250 responses is nothing. I wouldn't make big changes off of this data, but I would certainly agree with 80% of teachers are purchasing some materials on their own. I don't think that's out of the ordinary. I don't think somewhere within a standard deviation of 30% of teachers aren't using their instructional materials. I don't think that is out of the ordinary. I'm also not looking to make big decisions on this. But I think what point I just want to drive home is first and foremost, Kelly and I, thank you for the work you do. It does not go unnoticed on us and I don't think anything
089in this report is negative to what you two do or what our textbook managers are doing. I think we're wearing a lot of hats. We're doing a lot of roles and it's going to be incumbent on our school leaders to try to levy out this workload a little bit more and get some more hands involved to be able to do some of these areas in inventory control. Um again I agree the selected group is small but if we're looking at these in bulks of access and security controls or storage practices or end ofear documentation inventory plans etc. I think it's good information to be able to take back to our principles and APs and the rest of the folks that are wearing these hats as textbook managers and try to clean up controls where we
090can to try to make these materials go further. Um, so I just I don't want it to get lost that this is overly critical on anybody or that RSM didn't live up to what we told them to do from inventory controls. I think it's a matter of here's where we are, here's where we can improve. We can always do better communication on both ends. We can always get more responses, but I think this still leaves us with information that we can take back to do a better job at the board level of div resources, making sure we're keeping an eye on instruction materials to control the big cost of it. So, I I think we're just getting very emotional in certain ways in this and I don't want that to get lost from the overall
091message of you guys first and foremost do a great job and so do our textbook managers, but how can we improve in meaningful ways or control the the overuse or the digital utilization or whatever else and just get those easy wins to make these dollars go further. So, I just felt like that needed to be said. >> So, the teacher survey that does that belong to us that if we wanted to send it out again, we could rather than pay y'all. I I would I would be inclined for us to do that because I think we >> I think we could send it out if we felt like we needed another one on our own and look at the data ourselves without involving y'all. >> Sorry. >> The the the my other takeaway is is
092um that I want to be quiet for a minute and listen to Mrs. Cunningham who had to do this for years and we heard of three different groups that handle books right now. Should it be that or should should it be the media specialist or the AP? >> I think that different personalities in different schools do things better. That's why I think it's vital that we involve the instructional leaders at those schools because if it is the mindset of the principal this is important and they remind their textbook manager you got a meeting today and they receive the notifications right Mrs. Dodd that there's going to be a meeting the principal does so they can remind their people. Um my take on this is I've had some great textbook managers in my career and I've
093had some not so good ones. I've had some that I had to ride all the time and others that just did what they were supposed to do. Um, I think the beauty of this report is I'm going to be honest with you, I'm not surprised by anything you said. I'm not even surprised by the number of people that completed the survey. I think there are lots of things that contribute to that and the timing of it certainly had to do with it. But I also think that people think things are important if the people who do their evaluation think that they're important. So if we decide that textbook managers need to do the responsibilities that have been assigned to them, then we need to communicate that to our leaders. This is an important job. We
094expect you to make sure that your people get it done. Um, I have been a textbook manager and I will tell you it's not an easy job. Um, but it can be done. I've been a textbook manager in a high school and a middle school, so I get it. Um, but I think overall I share some of Tyler's sentiments. I appreciate the value of the work that you did. Um, I think it definitely needs to be considered and we definitely need to think about uh doing some things to beef it up and make sure some of these things happen. Um, I don't have a concern about the sampling because I have been through a lot of audits, um, title one, um, ESC and all those things and they generally choose the schools the same way
095you did. Um, and usually you get similar results. You're going to have outliers that are great places and they don't get checked. And so, you know, you get a bad report or what could be viewed as a bad report. But basically, the bottom line is the things that can be changed, they need to be changed in the schools where they're needed. And if there's a place where people are doing an excellent job, they need to be congratulated and told that they're doing an excellent job. Um, I appreciate the information. Um, I I think that's the way it was supposed to be and I think you did what we asked you to do. So, thank you. >> Does that answer your question, sir? >> Yes, ma'am. >> Okay. >> Did y'all come up with any dollar
096amount that you could see as a savings if we implemented? >> We talked about this whenever we first met you all. We when we're talking about internal audit, what can internal audit bring to the table? And I believe it was you that asked the question about return on investment. And typically, you know, it's hard to to put a dollar around return on investment for internal audit because what we're doing is we're mitigating your risk. We're mitigating your exposure, making sure that your controls are in place, your processes are designed effectively, and your controls are operating effectively. So that return on investment is always a hard number to to to put put together. But if we're buying in surplus range and if you but if you but if you look at some of the observations and
097the recommendations there there's some data points that need to be in place to be able to forward think and be able to understand some of those costs as well. So you know I think it was observation number three when we're talking three or I believe it was three when we were talking about some of the data points on the digital side and being able to determine the use etc. So that's just as an example of it's hard to hard to put together that co for this particular example. It was hard we weren't able to put together cost savings because we need some of those data points as well in order to be able to do that. >> Any other comments? >> All right. Thank you guys very much. We appreciate it. >> Thank you. All
098right. Is that all you guys have for us today? >> Well, we had also um I believe the package that we had sent last week included the top 10 that we had uh previously reported out to you all. >> Yes. >> Um you know, we got started late in this fiscal year to for these three projects. So, if um you wanted to discuss today the the top 10 that was issued in December of 2024, we could start talking about potential topics for this fiscal year so we can get started sooner so we're not doing them all at one time either and spread it out more for management, district management, etc. Um, so we're happy to talk about it today or we're happy to talk about it at another time. I don't know if I'm the
099minority opinion on the board, but I'd rather hold off on that because I'm I'm honestly interested in knowing which other sort of internal options, internal audit options exist and I'd be certainly very interested in having the superintendent's opinion and direction on those areas as well on on which areas she thinks are of particular importance for us to look at. >> Sure. But >> like I said, I could be the minority opinion. So, Dr. Luke, >> Mr. Cunningham, >> I don't have a problem with that. >> Mr. Matias, >> you don't have a problem like >> that. That's what I'm Yeah, I agree too. Especially with the superintendent being out. Let's schedule another time to get together and go over that information and um and regroup on that one. >> Sounds good. >> Perfect. Thank you
100guys very much. We appreciate it. >> Thank you. >> All right. Any board member comments? >> I do want to say thank you to Srass Bay Elementary School. We spent the day there yesterday. Um it was a professional learning day for our semester 2 students and they got to walk classrooms and the things that we saw um in that school. The instruction was amazing. So I looking at you so that you know that um anytime a school is willing to open their doors for learning. It's always um just incredible partnership and the learning that we saw um was fantastic. Our focus was on mathematics and Esau and um we really got to see some great things and some great strategies being used in every classroom we walked. It was it felt very um urgent. the
101instruction felt relevant. It was standardsbased. Um, kids were engaged. It was great discussion for our students. I think my favorite part of the day was after the instructional walkthroughs, um, the instructional rounds that we w we walked classrooms, I had them identify what were some best practices they saw in mathematics and what were some best practices they saw with Esau learning and where those over overlapped and we made a really good list. And then they had the opportunity to do like a fishbowl experience with students where we had small groups of kids come in and they taught while their partner observed and then they switched roles and we had another group come in. Long story longer um when we were going over what they did and the best practices that they implemented in their lessons,
102I was able to go back to that list and identify areas that they did not use in their lesson that we saw teachers using. And I felt like that was pretty impactful for them to go, "Oh yeah, we did identify that. Those are some things we could have done." which again anytime we can anytime we can challenge practice and push teachers to be better future teachers to be better it's a win for our students. So I just wanted to say Mr. Loisen and both of his APS Mrs. Torres and Mrs. Estro were involved in the planning of that day um and the setting up the walks and the teachers did a phenomenal job. So we we appreciate the partnership and preparing teachers for Lake County. So thank you. >> Thank you Dr. Luke. Mrs. Cunningham
103>> um I guess I'd like to say this as a final comment about the audit. um you know, anytime you audit something, especially if it's forensic, if it's after it happened, um you're going to find stuff. Otherwise, what's the point? And that's the only way you can improve. Um so, I think it's important that people go into those with an open mind and understanding that if the auditor does their job, they may find something that's not totally pleasant. And as a result of that, it's incumbent upon the person that was audited to come up with a plan to do better. Um, I can tell you that textbooks and textbook inventories have been uh a challenging item in Lake County for at least the past 35 years. We've lost a lot of money on textbooks. We've
104lost a lot of textbooks. I'm going to share a personal story with you that broke my heart. When I first got elected to the board, I went to a school and it will remain nameless. I went into their textbook room where they keep textbooks and I found materials that I ordered as a principal, supplemental materials that I spent title$1 for about $45,000. Those books were still on the shelf, never been opened, never been used. And that I'd been gone from that school like five years. And I remember thinking to myself, how can this be? This is a failing school. They got some great reading material on the shelf. And when I mentioned it to the powers that be, they said, "Oh, really? We have that?" So, I guess what I'm saying to you is it's
105important to recognize when someone points out something. Maybe we just need to come up with a system to make sure it's better. Um, there are lots of things that deserve attention. Textbooks certainly is one of those. I think Mrs. Dodd does a great job. I think I'm sorry, the young lady with her does a great job. I think the curriculum department does a good job. I don't think because you find something in an audit that it means that anyone didn't do their job or that they're not doing their job well. I think it just means that we're not perfect and we brought in someone to point out those things to us and it's incumbent upon us to work to make them better. So that's my philosophy about life. When things aren't as they should be,
106you just work to make them better. So I challenge all of us to let's do that with this audit and any other audits we should or might receive in the future. >> Thank you. >> Well said. >> Chairman, may I really quickly? Yes, ma'am. I did want to apologize. It was a very very important call that went over time that was allotted with the Florida Department of Education. So, and I didn't get the materials and that again, this is new for us. It's the first time you guys have come back with an audit. Um, I got the email yesterday. So, I was kind of like, what is this? But I did I'm sorry that I missed the conversation and it was a really important phone call, but this was important as well. So, I did
107want you guys to know that it was I tried to get it done. So, >> Mr. D, >> my greatest takeaway from these audits is just really how great our people are. That while certainly you always have the opportunity to do things better, I don't think anybody does it better than what we do. When we looked at that procurement audit and just how forwardinking we were on instructional materials, I would I would put this up against any other district in the state, any one of these, and say, "Yep, >> I think I think we got something that's worth bragging about." Um, and then the other thing that's just really worth bragging about, I'll piggyback on Dr. Luke's comment about to the PL day at Sawrass Bay. It's also a great bit of gratitude to the
108district and all these principles who have, let's say, a a paraprofessional who's working in their school full-time, but releases them to go get this professional learning. Uh, that uh is just going to pay dividends on the back end for those people and their growth and the impact that they're going to have on students. Um, and you know, fulfills these students dream of becoming a teacher and getting that degree. So I'm just really grateful and thankful to all of these people who make that partnership possible. It was a you're right very very wonderful day, >> Mr. Mth. >> So one of the concerns I always have is and I didn't take anything negative to be quite honest to to um Mrs. Cunningham. It's a learning experience and you guys are like the internal revenue when they
109come to audit my company. You know they're going to find something, right? it's just expected and so but then it's what do we do with these reports afterwards and that has been an ongoing concern when we had our internal auditor that was staff to now what are next steps for the district to look at how do we support u these two procurement functions and and uh so if we take this today and we gosh we spent two hours that was nice and you guys are all warm and we're doing a wonderful job we're missing a great opportunity in my opinion that uh for us to actually be matter. So that's where my head is. >> Well said. Um yeah, that I think I've already said what I wanted to say on this one. Again, thank
110you Kelly and Iana. Thank you Ali when he was here. Thank you RSM. Um and I agree, Mr. D. We certainly have wonderful staffs in both of those departments and throughout this district. Um and as Mr. Mati said, looking forward to using this information to make us um even just that much more better. Um so thank you all for that. Um had one comment that I want to make. had the opportunity to attend the uh Lake Economic Area Development annual dinner last week with Francis and Mart and the CCR team where the business community recognized the school district for our A- rating um and all the work that we've done with CTE programs and growing that throughout the county. Um so just wanted to share that with the board and with staff that that is
111not going unnoticed in the business community and as growth continues to come to Lake U more of the high quality smart growth that we all hope to get. um when people look at how are your school systems performing, what that means when you choose to move your family, uh we are certainly moving in the right direction. That is helping Lake as a whole. Um and that was shared um with us last week. Um so just want to pass that on back to you guys as well. U Mr. Deputy Superintendent, do you have any comments? >> Uh just like say for the two groups that were subject to the audit, um I concur. Great job. Um I was especially impressed with the action plan steps that were put in place immediately with instructional materials. obviously taking
112it as an opportunity and while you you guys can't be perfect, I know that you are perfectionists and um and I know those actions plans that they put in place, they will definitely um move forward on those and we'll continue to learn and grow and getting to work every day with procurement and the purchasing process. I can um say Ali and his team, they're they're great also. They are always looking to improve every single day. So, I think they'll use this audit as a tool and um and we'll continue to get better and I agree, Mr. Dodd. I think we are the best. So, that's all I have. >> I'm sorry. I had one more comment. >> Go ahead. >> So, and the fact that we're the only district that interviews the last three um
113it has worked extremely well for us. So, you may want to carry that. >> Walton does it as well. One of the two that I know of. >> You may want to carry that forward as how great we are. >> That's it. Any other comments? >> All right. Uh thank you everybody. We are adjourned.