2026-05-07 Board of Education's Audit Committee

001Here we go. Okay. This this public meeting of the board of education's audit committee for May 7th, 2026 is called to order. Good afternoon and welcome. Thank you all for being here. Today's agenda includes a discussion around the district's annual bond performance audit. Then we'll have a status update on the implementation of the recommendations included in the student body funds audit. Then we'll take time to reflect on the audit committee's work for the 2526 school year and discuss some potential updates to the district's audit policy that our internal auditor is working on. We're also going to discuss audit committee membership and our internal auditor will be providing us a status update on the office of internal performance audits activities. Finally, we will have an opportunity to discuss any requests for future audit committee meeting agenda

002items. So, let's begin with introductions. And I think we'll start with the person who's on Zoom. >> Hi, that's me. Um, I'm Christy Split. Sorry to have to be virtual today. My daughter has a music show starting at her school at 5:30, although she'll be up closer to 6:30. So, I will be um I try to kind of be off camera because I don't like being this like face behind everybody the whole time. Um but I'll turn my camera on for discussions and if I'm planning on saying anything. Um so, thank you. Um I don't even think did I say who I am? I'm Christie Split. I am um uh the board member who lives in zone one. >> Thanks, Christie. Um let's start with um our council this end. Yeah. Hi, I'm Andrea. Captain,

003internal auditor. >> Janice Hansen, internal performance auditor. >> Evan Cole, district student council representative. >> Uh Virginia Lefort, a board member who lives in zone 5, which is northeast Portland. and I also chair the facilities improvement and oversight committee. >> Stephanie Englesman. I'm the board member in zone 6. I live in southeast. >> Hi, Patty Sullivan, chair of the audit committee. I live in zone 3. >> Hi, Caroline Zavikovski. I'm a community member on the audit committee. >> Jennifer Samuels. She, her. I'm also a community member on the audit committee. >> Michelle Morrison. I'm the chief financial officer and staff liazison to the internal performance auditor in the committee. >> Okay. Thank you for introducing yourselves. Um next we have public comment. Miss Powell, do we have anyone signed up? All right. We will continue.

004So, first we have the annual bond performance audit and I'm going to turn that over to our senior internal auditor Janice Hansen. >> Thank you very much. Um, so today we actually have three subcategories that we're going to talk about as it relates to the annual bond performance audit. Um, we're gonna have a brief status update from our external contracted bond performance auditors who are here logged in um, virtually to give us that quick status update. And then we'll have an opportunity based off their status update to have a quick conversation about scheduling um, for when their audit will be ready to present. We will also discuss the status of the outstanding bond audit recommendations. And lastly, we'll discuss the um bond auditor's contract. The current bond audit team is in their last year of

005their contract. So, we are working on doing an RFP to solicit um proposers for that service. So, first I would like to introduce um Kathy Brady and Lynl Louu. Both are here from Showberg Shank. I'm going to let them introduce themselves. Sorry. SEC. Um so, I'll turn it over to you first. >> Wonderful. Thank you. Good afternoon, Chair Sullivan and members of the audit committee. Thank you for having us here today to provide an audit status update. Uh my name is Kathy Brady and like Jenna said I'm here with Lanlu from our firm Shoberg Eva Shank Consulting. It's a mouthful. Uh we currently are and have been your external bond performance auditors since 2018. So in terms of status of the seven-year uh year 7 performance audit, we kicked off the engagement in October 2025

006with an entrance conference with the uh basically the bond delivery team uh the senior chief of operations management from the prior office of school modernization, Presidio staff as well as staff from um other bond areas, the office of technology and information services, office of teaching and learning and the center for student excellence. So, uh, between October and December of last year, we conducted what we call our reconnaissance interviews where we meet individually with stakeholders, including the same bond delivery, um, people that we met with at the entrance. But we also meet with individual project managers, construction managers, and support teams, procurement, finance. Um, as you probably recall, we also met with some board members, uh, members of the BAC, uh, as well as district executives like the CFO, chief of staff, legal, um, as well

007as your internal performance auditor, Janice. So we do that because these interviews help us um understand the current environment, see what's changed since the last time that we were on site, helps us assess risk um and just gather insights and perspectives, all of which um inform and refine our audit scope. Uh speaking of the scope for year 7, our audit scope focused on bond status uh in terms of schedule, scope, and cost for all the bond years 2012 through 2025. Um as well as we reviewed contingency management and usage uh practices. Um also as we do with each audit cycle um those efforts involve reviewing many many documents uh that that uh the district is kind enough to pass on to us um including project files and public testimony uh board and BAC meeting packages,

008financial records, um and other source documents like that. Um we also validate data uh reported publicly to the board and your committees and the BAC. Uh and definitely we reconcile the the projects and works that have um work that has been done and completed against blond bond pledges. Uh we also look at testing of fiscal controls, different types of expenditures, um bond activities, contracts, that sort of thing. And then finally, as part of our audit scope, we always follow up on the implementation status of prior audit recommendations and we analyze and verify any reported actions that are taken by PPS. Um some of the prior outstanding recommendations were in areas such as um CMGC contracts, bond staffing, equity, uh business equity, workforce equity, career learning um as well as for the center of student excellence.

009So we conducted analyses in those areas as well. So currently we are uh wrapping up our audit fieldwork as we speak and we hope to have an internal draft audit report um to Dr. Franco and the Rise team by the end of this month, May, early part of June, after which we'll hold an exit conference to internally discuss the draft. And that's done so that we can consider PPS perspectives, which we try to do along the way as well. um and to get any additional information uh before we request and receive PPS's written responses that are part of the audit get attached to the audit and they respond to any new um year 7 audit recommendations that we might have. So currently our goal is to we'll get that report like I said end of

010May, early June and then the goal would be to finalize the report by the end of June, early July. Um there's been a lot of change and a lot of new players in place that have not gone through this audit cycle yet. So there might be some slight movement within that schedule, but we remain hopeful that you should see a final report publicly released within those timelines. Um a little bit on the reporting process. We provide the final report to PPS and they are responsible for the actual distribution of the report and posting the report on the uh PPS website. Uh it's only after that report is public that we are often asked to present the report to the BAC or to various board committees. We present it to the audit committee before the facilities

011committee and even at times to the board. So unfortunately today we can't share any uh preliminary results or recommendations as the audit process is still considered draft and confidential until the report is final and that's because analysis is ongoing really till the end of the audit. So results can change if we get additional information or if we in our internal QC process we identify any modifications that need to be made. um we do that all the way up till that final report is is issued. So protecting that confidentiality is very critical to us um to really ensure the accuracy to make sure no premature misleading information is released before we look at all the evidence and also to make sure that we discuss things with PPS so that we can get their perspective and that

012u we've considered all relevant information. Uh but once it's final, we're open and able to publicly discuss the results and recommendations. So with that, uh that concludes a quick status. Um thank you for your time and also we're more than happy to answer any questions about the audit process. >> Does anybody have any questions for them? Thank you so much for your status update. I really appreciate that. >> No problem. Thanks. Okay. All right. Well, then just at a very high level, just to briefly talk about potential scheduling opportunities. So, based off of those comments, it sounds like we need to start thinking about roughly a July meeting to for scheduling to be ready to receive that final report. We've talked about potentially a July meeting earlier in the fall and we've really struggled to

013try to nail down a July opportunity. So, now that we are getting closer, we probably want to make some um real plans on trying to figure out a meeting opportunity. Um any thoughts about potential meeting opportunities in um July? any major traveling that we might need to take into consideration. >> Well, summer in general is going to be hard to get everybody. >> Yeah. >> Um so I guess when we finally know we can do it. >> Yeah. >> Just going to have to put out some feelers and get a date. >> Okay. So, please anticipate hearing from me in early July about getting a date really um finalized to deal with that. And also, one of the things that Chair Sullivan and I have talked about is doing our best to try to get

014a joint meeting, especially with our BAC members. Um, when we watched the BAC meeting in the past where the bond auditors have presented to the BAC meeting, their questions were so good that I just was like, "Oh, I wish we all could have benefited from that rich discussion." So, um, there might be some reach out to the BAC to try to facilitate something. I heard that the BAC may have a already be planning a potential July meeting and if that is already on their plan, there might be an opportunity there and I will be talking to Director Split a little bit more about how we might be able to arrange that. And it looks like her hand is up. So, um, go ahead Christie. >> Thanks. Um, I just wanted to suggest that we do

015it virtually. So that way regardless of where people may be, they could still participate. Um the barriers for participation with like travel >> um are reduced if we do virtual. Plus it reduces everybody's need to travel around Portland during rush hour um at a time when they might not have the same like child care and stuff. >> Yeah, that sounds like a great >> great idea because to get VAC and facilities and us together would be practically impossible. But yeah, good idea. And then you can be all over the country and vacationing and still take a break from your vacation to watch >> a meeting which would be very fun. >> So thanks Chrissy. Good idea. Okay. So the next of the four elements within this category that we're going to talk about is the

016status of the outstanding bond audit recommendations. And today we have Sarah Norman here which I see she's just popped up and she has provided a um a slide as it relates to the status of those the implementation of those recommendations. So I will let her go first and then um but Sarah sorry I didn't have a chance to talk to you before the meeting but just do the first slide and then we'll talk about the um RFP afterwards. I figured I I read the agenda so I figured uh no problem. Um hi everybody. Good evening. Uh thank you for inviting uh me to join um the meeting today. And I think I need to share my screen so you can actually see the slides. So give me just a second. I just got to get

017it coordinated here with technology. So hopefully you are able to see my slide. If I hope you guys tell me if you can. >> No. All right, perfect. So when um Presidio uh was contracted and um started working with the office of school modernizations um we started with a total of 81 open audit items. Um and so it was it was a a big task to kind of wrap our heads around it, read the previous audits, um really understand where the office office stood and through that really working with um um the staff there with the RISE team. We have been responding to the auditors with additional information um and just working on that list to get things closed closed out. Uh we have reported to the FIOC on a couple of items that we've

018you know that we've dug down into and reported where we are with it. Um, and to date we are sitting at 21 open audit items with nine of them actually having to do with business equity which are um, recently legal just issued as of yesterday a revision to a um, administrative directive that we will be looking at and then updating our um, equity practices within the program to that. And then an additional four outstanding o um audit items are actually from year four um regarding the CBSE which again some of them are kind of overcome by events because of where we are with the CBSE um and now named the Gryair Center. So we are just continuing we've made leaps and bounds. I mean, we've we've made this a focus within our office and we

019are working to close out these audit items and satisfy um our external auditors. Um we continue to work with them um and answer any remaining questions that they might have. And then we're also reviewing the open audit items that are still currently open to make sure that they are still aligned because some of them are quite old to make sure that they are still aligned with PPS's priorities. And so in doing that um that kind of provides the update on where we are on audit outstanding audit item reconciliation. Are there any questions? >> Yeah. Hi. I have a question about that last item about confirming the items are still aligned. Will you speak a little bit more about who's who's involved in that and what's that process and how does that get communicated out once

020it's settled? >> Yeah. Um, we actually have to we actually have a tracker that we put all of our information into that we share with leadership which we we report directly to John Dr. John Franco who is a COO and then that is then communicated to externally to our um to our external auditors where they actually review and say whether it is complete or still open or in progress. And so that's the kind of the process of closing items out. I think there's a step in here this um in the meeting materials there's this great grid that I think >> I'll talk about that next step >> because there there's a board step in there as well. >> Yeah. >> Can can I ask so I was going to ask about this and maybe I

021don't understand what this is in comparison to what >> Yes. So in the fall in the audit committee, we talked about what did we want to do with as it relates to the outstanding bond audit recommendations. Um it is technically in the audit committee's charter. Um we our is this committee is the only committee that has the written charter authority to follow up on the outstanding any outstanding audit recommendations. So when we talked about this in the fall, there were 43 at the time outstanding bond audit recommendations and um I worked closely with Director Sullivan and and one of the priorities that she wanted to emphasize is do our best to try to limit the number of different committees our staff have to report to. So what I did is um I which is this

022is the document that you guys all have in your packet. This it looks like this. So what I did is I went into all of the outstanding bond audit recommendations and allocated them out to different committees and I talked with the chair of each of those committees and asked if they would be willing to follow up on the implementation of those recommendations in their committees. And um so as part of that when we talked about diving it up and asking other committees to follow up on it, one of the requests was at towards the end of the year we circle back and see how it went. So this this document is for us to discuss how it went um as it relates to the assigning out of the committees because we probably want to discuss

023based off of the success we've seen whether or not that is um what the audit committee would like to do moving forward or whether or not um perhaps the audit committee would like to go ahead and and continue the followup. um in the audit committee as opposed to um asking the other committees. However, you know, it is nice because the FIO committee, a lot of the rise staff already are reporting to their committee. So, it's really nice of the audit committee to to kind of let not add an extra committee if at all possible. But I do want to kind of go through a couple of these and talk about this how successful we were um and how that relates to our work moving forward. Go ahead, Jennifer. >> I just want to bounce back

024to the question that I asked because I if you don't mind I want to check in here. Um I think that um Sarah that was about confirming that last bullet is confirming that the open items are still aligned. Is that was I reading that correctly? And then who has the authority or charter? This is something I just honestly don't know. Who has the authority or charter to assess whether those open items are still priorities? >> Ultim ultimately because they all have to do with the bond oper the majority of the our external auditor is uh auditing the bond program. So ultimately that falls under the direction of our um chief operations officer and of um obviously Michelle Morrison should it have something to do with bond sales or something of that nature. And so um

025both of them I mean we're review everything that we are presenting and and um giving to our external auditors are then either it's either accepted and they agree that it is completed and closed. All of that information is reviewed by those PPS employees. >> Okay. So, I just want to I'm just trying to get clear on that. >> It does feel to me a little bit like that. Um, and again, bear with me, but that does feel to me a little bit like it is saying who is getting audited gets to make the call on whether the open items are still relevant. That's the piece I'm I'm I'm a little stuck on. Thank you. to cl pot potentially clarify. I don't want to put words in your mouth, but I think you might be specifically

026asking as it relates to the response to the recommend the district's response to the recommendations whether or not the district concurred. Typically, the term concurred has been used by our the team to say yes, we agree with the recommendation and we'll implement it. um versus concur with comment is another um kind of category that staff have responded. So they'll add say concur as in yes we agree and then the comment kind of clarifies a little bit as it relates to what they're going to implement. I think you're asking um whether or not the bullet point confirm items are still aligned with the PPS priorities. Specifically, Sarah, are you guys going to be going back and looking at the initial response and perhaps saying we no longer concur? Is right. That's correct. >> Correct. >> So,

027there are some that are just overcome by events. I mean, and Janice, I'm looking at what's posted in the board book, and this is this this is actually not our tracker um for the bond audit that we're using. We actually have one that's been going back and forth with our external auditor. And so I have one that I I'm happy to pull up with our current one if people if everyone wants to look at it that shows that whether the PPS what the PPS reporting status is to the external auditors and then what audit verification status they are still looking at. But yet what I mean by that is some of the items that are from year four just as an example um um established a framework for CBSC management and staff with clear roles

028and responsibilities defined authority and accountability. We've already done that and we've provided that and it's overcome by events because now we have a building. So the a lot of these a lot of these audit items that were done in year four are kind of at that point in time and they and they've and everything's kind of moved past. So that's what I mean by aligning the audit item with where we currently are with progress of the program and and PPS priorities. >> Did that answer your question? >> I think I think so. If if if I'm not mistaken, weren't some of these more kind of having to do with policy? >> Yes. >> And we we're I'm gonna kind of talk through some of that. Um so let's do that next. >> Well, can I

029ask just a quick follow-up question? So I was counting and I appreciate Sarah that you have a different tracker which you're willing to share out which I would love to have shared out because I was going through counting the you say there's 21 open on the form we have in front of us. There's 40 still recorded. Yeah. So I it sounds like your tracker is different because of these circumstance and time and um event thing. So if you can share that with us that would be helpful if that's what's going out. >> Sure. I mean and it's also updated because it's it's a working document that we're working with our external auditors on and so as we deliver information to them they are going through and updating their responses to us and what we need

030to respond to them on. And so our so our audit log is active on a weekly basis as we are delivering additional information. So I don't know there's no on the copy that I'm looking at there's no date and time andor revision number on this document that's in board book. So I don't know how recent this report was was made. So but I'm happy to share out our full report that has the comments on it from not only PPS but also from our external auditor. >> That'd be great. Thank you. >> Yeah. Is it possible to have Janice be part of that um that um line of communication so she has visibility to what's going on in real time? >> That's it's no problem. We can copy her on all the emails. Sure. >> Thank

031you. >> Thank Thank you so much for that. Um because I did share this document with you Sarah and G >> back in January. I've seen it. >> Yeah. >> Um okay. So now let's just talk a little bit about um how things went this year for us. So what again I am not privy to the current status that Sarah is um but sounds like I will in the future. So this document could potentially be way better updated moving forward. Um but for today the purpose of today's meeting what I did is um I put in the comment here to say who kind of we referred the um the recommendation to and whether or not there's been some followup. So if you look um and on the far right hand side I know it's very

032small so you can barely see the numbers but um for example um number two you can see like that one was discussed in the FIO meeting on March 17th and I went through the presentation from that meeting and found the what staff reported in their presentation to the FIO meeting and for that one staff for reporting the implementation is complete. So I would say we were very successful on um on that item specifically. Um one of the things that um we asked Director Lefort to be here for today to join in this conversation is um just to kind of talk about how it's gone in her committee. And one of the things that she and I have worked out over the course of this year is as she's doing her agenda setting, um when she

033looks at how much she has going on on the agenda, she'll shoot me an email and say, "Can you give me your top five um or three depending on the month um priorities for us to follow up in the committee?" And as you can kind of go through these um comments, you'll see um each each of the recommendations that she's been able to follow up on and the outcome of that follow-up is listed in the comments there. One of the areas that um I'm not sure we've been very successful at is as it relates to the recommendations that were referred out to the policy committee. Um there's quite a few like very specifically the recommendation is um bear with me PPS board. So this is recommendation number four. So it's on that front page of

034that document. Uh the PPS board should revisit its equity and purchasing and contracting policy. Um there's really nothing staff can do. Um we really need the board to take some action to implement uh this recommendation. Um so we probably need to talk about um potentially how can we follow up on that in our committee. Um, also there were a couple of recommendations as you kind of go further down. Um, specifically as it relates to the ones for the C, um, the Grace Adair Center for Excellence, um, we didn't really have a committee this year that was designed to follow up on those recommendations specifically. So, I sent an email to our board leadership and asked um you know for some suggestions on how to follow up on that. Um and I just I haven't gotten

035much um haven't been very successful in that. However, I did talk to Virginia director Lefort last week and she agreed to pull those into the facilities committee. So I think um the ones that were identified related to the Grace Adair Center for Excellence she's planning to follow up on. So there's some give and take like we can potentially rearrange how we're trying to follow up on these. I just wanted to give us an opportunity to talk about it. Um, how successful we were and kind of come up with an approach for moving forward. That's all I have to say and I'll let you guys ask the questions from here. >> Yeah. I' I'd just like to first comment on how um Director Lefort took that on and most of them had to do with your

036committee and you did it and um really appreciated it because I think it saved staff. They didn't have to come here. They just went to one meeting. Um, so that really that part of it worked, but some of the things it seems like they're going to have to come back to us and we're going to have to follow through. I think that's my thought, but I want to hear from the committee as a whole as to whether or not that makes sense or if you want to give it another an alternative, some other way of doing it. But that's what I was thinking. But I want to hear from you guys. >> I can I >> So Janice pulled this together. I don't know, was it summer? Was it summer, fall? It was a while

037ago. And sat down and and um >> was able to walk me through what was initially felt very overwhelming. And I thought the and I still believe that part and parcel it to our committees like this was extremely effective for me as a committee chair. It was really great to just have a checklist of all the things that we could bang out in the committee and we did. We still have a little ways to go. We're really close. Um but I found it incredibly helpful and useful and um strategic. So I really appreciate all of the effort that went into this. This was a lot and so I just actually want to say thank you Janice. You made my job a lot easier. So, >> appreciate it. Uh, one other thing. Um, the TLE committee

038had two recommendations and, um, I kind of missed a couple of my meetings with, um, director um, Chase Miller, sorry, um, director Chase Miller, but we're back on track and we talked just two weeks ago and I gave her everything that she needs to follow up on those in the TLE committee. and I sounds like that is her plan. So, um we might have some additional success still yet this year in the TLE committee as it relates to the two that were assigned to them. >> Can I can I ask something? So, um I sit on as you know I sit on the policy committee and um I am surprised to see so many of these assigned to the policy committee because that I know they we haven't we haven't seen these in the policy

039committee except um perhaps number four because as I understand it the policy committee is working through every single policy. So presumably this one will end up being worked through, but the rest of them uh maybe I've missed something, but um I note that you said you emailed in August, which is quite a long time ago. Has there been any additional reach out from you to policy chair? Yeah. And I um I I certainly sent the email to the chair of each committee um initially and it's really the chair of the committee's um each chair of the committee kind of has their authority to kind of set their agenda for their meeting. So I can only suggest and and ask and um it and certainly sounded at the beginning when I mentioned it. I did send

040Michelle, our director to pass, an email and then I followed up with her in person. Um, and it sounded like there was going to be some movement and I'm sure she has other priorities. I'm not sure everything that's going on in the policy committee right now, but this just doesn't seem to have had movement. Um, so we got to figure out some way to make some movement on here. And um, also side note that um, Miss Director Depass and I have just gotten back into regular check-in meetings. Um, so I'll have a more of an opportunity to remind her more routinely about these outstanding recommendations as we move forward. Um, but there has been most of the year um where I really haven't had that opportunity to really check in with her. >> Okay, that's

041helpful. Thank you. Um I suppose so we had um you know I don't know the reasoning behind it but there were two policy committee meetings um in the fall I think that were cancelled because we didn't have anything on the agenda. So I think it would be great and it sounds like you sent the email in August and had a little bit of followup but not much. I think it would be great to send just a reminder email that this is assigned to the policy committee and perhaps include the whole policy committee as well. Got it. You know, because things can fall through cracks and get overlooked, but it sounds like you've been doing some followup and if we want to keep it assigned to the policy for at least the remainder of this calendar

042year, but again, I don't know. So, if you think time is of the essence and that we should pull it back to audit, I would also support that because we're here and you know, we can work on that and you come to this and then staff doesn't have to go as many places. I see director split has an idea, too. But that would be I'll leave it up to you whether you think we should pull it back. I'm happy to obviously I said on both votes so I get the information either way. >> Okay. Director Scott. >> Yes. Thank you. I was just going to say that um I think that using the term assigned just is like maybe a little strong and that might be part of the difference in how things have been

043handled by different committees. um you know, one committee can't assign work to another committee. The emails were coming from Janice. So, it's more like she was just trying to like parse out the work, not assign it. And so, I just think that I just want to be cautious in um feeling like it was a dropped ball or something she was assigned and didn't do. It's more like it was suggested and maybe she just decided that this wasn't the thing she wanted to work on this year. So, um, I just wanted to like make clear that we don't have the power to assign anything from this committee to another committee. >> Thank you for that, Christie. Words mean a lot. >> And oh, I also I also just wanted to say that I'm open to like

044whatever solution we want to arrive at to keep things moving on getting this backlog done. think it's really important that and I and I think it's um pretty clear to me that um Sarah and her team are ready to make sure that we're doing things according to the recommendations of auditors so that we are having best practices. Um so I'm really appreciate that commitment. Um and I also just she has her hand up anyway. I'm sort of curious what your thoughts are about how this was sort of divided up so that we could get through the list in like the most efficient way possible. Um, so I'm open to your ideas, too, if other people are hopefully open to at least hearing them and dismissing them or not. But I think it would be good

045to know just how you think it would work best to kind of process and get through these before we get a whole new list of possible recommendations that we need to add to the queue um after this summer. So um yeah, open to your ideas, but also very open to just do whatever um out of that of this meeting. >> You know, we're Thank you. And if I could just speak really quick, I I just I I want to just say that our office and the sta I want to recognize the staff has made an incredible effort since we were contracted to reduce the outstanding audit items that have been listed from the external auditor. So I I just I I want to take a moment to recognize the feat that the staff has truly

046and the work that they've put in to do that. Um and that will continue. Um, as we get year as we get year 8 that's going to come in, we're going to have items that we're going to review and we're going to either agree or or show the process that we have in place in order to in order to comply with our external auditors program and um we will continue to do that. It's been great to talk about the audit items in the FIO meetings and and the board me I actually I think I presented on I think I presented on audit items at a board meeting at some point. Um and so I you know we're here to serve. So however the board members um and directors decide to direct us to do that

047we are happy to do it and we are here to comply with our auditors and provide clear clear answers on what is happening with um with the bond program. I mean we not only have our ex we not only have our external auditor we have our BAC meetings where we have an accountability committee there and um and we report monthly to the board board of education and so we we will continue that moving forward. You know, I just want to say just follow up on that, Sarah, I do want to tell you that I appreciate how you really jumped right in. You know, we started these lists. I think we started knocking these out in December. Um, which is, you know, right? I mean, you've been with us for a while, but it was right

048when we really, you know, were were getting into the contract. So, I just want to tell you that I appreciate that you never ever tried to put off the audit items that you jumped right in and always had a response and a complete answer. So, I just want to thank you for that. I think it would have been really easy to try to uh push it off a little bit and that didn't happen. So, thank you. >> I just can I I just thank you director split for that. Again, 9 8 months in I was looking at year one from 2018 and assuming that someone had assigned I don't understand the process yet. So, I I that makes me sort of think that maybe we do just pull it back to audit if it is

049originating in an audit committee rather than sending it out. we just keep it here. >> Well, one opportunity that we're about to get in July is um we will hear feedback from the bond auditors because they mentioned in their scope they follow up on all outstanding recommendations. So, we will get a really great status. I mean, it's within the bond auditor's um purview to decide whether or not whatever actions staff have taken actually um address and have implemented the recommendation and in their scope and we'll get that they cover that each year. So the 43 outstanding bond audit recommendations that were included at the end of the year six report that we got last year, last summer in June. when they do the report roughly this summer and July, if they have received evidence of

050the implementation, they will sign off on them and we will be left with just a list of things that are still outstanding. So, um, we wanted to talk about it here and I got your feedback and how about we see how it goes. We'll we'll keep the assignments to the committees through the rest of this academic year. um try to get the last couple implemented and we'll see how it goes with the bondage report and then in our first meeting in September, we'll decide for the next year if that's okay with everybody on either separating them out or we could separate some out to the facilities committee and keep the rest. Um you know, wherever we're finding the best success that works the most. um efficiently is the best opportunity that we could get to

051get them implemented. Does that sound like a really good idea for everybody? >> Yeah, because we'll have more information then then we we can make that decision. >> Um did you get your question answered? >> I think so. >> Jennifer, >> did any >> I think I understood your question. You you heard like who decides if something is it is essentially relevant or not relevant anymore? I heard your question. I'm not sure I heard a direct answer either. I think is you know we probably want to follow I mean how do we close things out if it's when we say not relevant is it because it's old and this might be a question for Sarah. Is it just because some of these were real old and you know they I appreciate that you captured every

052single one. Um, are we at a point where, you know, we're only addressing 2017 forward, um, or 20 I think we still have some stuff from 2017 that we want to cover for sure, but is it like 2020 forward? Um, >> well, I think that is the decision of the bond auditors. >> Okay. So if in fact um the RISE team has said this recommendation is no longer relevant then the bond auditors in their year 7 report in July will say the district has decide not to implement and they will say either we reaffirm the outstanding recommendation meaning we still think it's relevant to implement or they won't, but that is under their sole purview. They get to decide whether or not um the there's still relevance to a recommendation that may be old and

053staff may feel is no longer relevant. Ultimately, they get to decide to make that final. >> Okay. Yes. >> Yeah. I appreciated your your question and >> it was still a little bit in the air for me, too. So, thank you and thank you, Janice. Okay, sorry that actually took a little longer than I was hoping. So, we're running a little bit behind, but the last element um as it relates to the out the bond audit is bond performance audit is discussion about the RFP. And um Sarah did a second slide for us just to give us a brief update on where we are with the RFP for the bond performance audit. I did. Sorry guys. I'm trying to find my microphone button now. Okay. Uh can you guys see the slide? I hope it's

054large enough. Um so we are in the midst of developing the RFP for the bond performance auditor. Um we are collaborating with the senior internal auditor um financial operations manager legal council and purchasing and contracting for input um of the subject matter you know subject matter experts um for this RFP. The RFP is still in review and not yet published. So it um limit limits what I can share. Um purchasing and contracting is scheduled to publish the RFP in midJune. Um proposals will be evaluated based on the criteria and scoring within the RFP and the awarded contract will be for one a one-year term and renewable for four additional years which um follows the same um status that we've done in the past with our external bond auditor that the the district has done in

055the past with the external bond auditor. Are there any questions? Um just a quick clarification on that last bullet point. Um it's been the practice to do our first term as a 4-year term and then renewable for four additional years via one-year contracts. So maybe it's just um if we switch the four and the one then it probably makes sense. But I just want to clarify um my understanding that the first part of the term will be a four-year term. I will clarify that. Um I will clarify that for you Janice because I think that um I'm not again I want to clarify that with purchasing and contracting um because I think we there was a bond recommendation to have a bit more flexibility with the our bond our external auditors and so I just

056want to clarify that but I'm happy to come back to the committee. question. >> Um I will add that I have been included very and they have um purchasing and contracting has been working collaboratively and um asked me for my input quite pleased with how well it's going. Um, so the next thing for the audit committee to talk about as it relates to this topic is as it relates to the panel that will be scoring the RFPs when they come in. Um, in when we did the financial statement contract, um, we had director Sullivan and one of our community members, Caroline and myself serve as three members of the scoring panel along and then PPS uh, staff assigned two staffers to oops, sorry, also sit on that panel. And as um Director Solomon and I

057have been talking over the course of this year, she has mentioned that she would be willing to give up her spot on that scoring panel potentially to um director Lefort or potentially director Split as they um director Split may have more relevant experience being the BAC liaison and vice with V director Lefort. So, um, we are starting to get close. It probably going to happen over the summer. It sounds like we're getting really close to getting that, um, RFP out. So, um, we don't have to make a final decision today, but, um, just wanted to throw it out there that we will need to talk about that over the course of the summer, who will be serving on that panel. Um, and one other um, opportunity we may want to consider is seeing if one

058of the BAC members would be interested in being on the panel. Um, just to have a variety of experts. So, um, you all want to talk about that now or do you want to wait until over the summer, but it's going to be coming sooner rather than later? I think we should probably wait since we're behind already. >> Okay. But >> can I Yeah, I just I I think I'm Oh, Sarah, it looks like Sarah wants to >> I just want to say I just want to say that uh the panel has to be set by purchasing and contracting and we have to make sure that whomever's on that is is is approved. And so I just want to be careful with kind of I don't know if BAC members can be on that evaluation

059committee. Um just I just want to make sure that we run all that through legal and purchasing and contracting. I'm not saying no. I just want to make sure we double check because normally purchasing we provide options to purchasing and contracting on who can be the on the evaluator um committee and then they reply back to us on what that committee size is and what that makeup is. So I just I just wanted to say that. Um and that's that's it. >> I'm glad you said that because I think you've now confused me even more. So my original question was going to be it sounded like Prescidio was going to be in charge of deciding who audits Prescidio. >> No. >> Okay. But then you just said >> it has nothing to do with it

060has nothing to do with it with Presidio the bond. I just want to but yes our department writes the RFP for for this procurement. So we have we have consulted with PPS legal Janice uh TJ has been in um communication with Janice to add to that and then it goes to purchasing and contracting for final review and that's kind of where we are right now and then we will make Janice is included in this will make recommendations to purchasing and contracting for like the evaluation committee for the proposals that will come in for external auditors. to serve to serve the district. >> And at that that's very helpful. Thank you. So then at that point anyone who's working on implementing bond work is not on that committee. >> We do not. No, we don't want

061to be on that evaluation. >> That's what I figured you'd say. Okay. So that's when director Sullivan um you the other people who have in the past been on the panel to >> correct >> and every and again this is so outside my wheelhouse so I use the wrong term sometimes but everyone who submits a proposal in response to the RFP is >> given to the committee there aren't there's no weeding out that happens before everyone they get to see all the proposals that are >> sent to see all the prop they get to see all the proposals and they get to evaluate and rank every single proposal that is submitted to the And then if you want to hold interviews, you're you have that opportunity to do that as well. So >> no, you

062see every single one. >> Thank you. >> Okay. Any more questions about bonds? Okay, I think it's time to move on. Um so next on the agenda, it's a status update on the implementation of recommendations included in the student body funds audit. Chief Morrison, would you please introduce this topic? >> Thank you, Director Sullivan and Janice for your support as we have been focused on this work since the last report was given last year. I do want to give some gratitude to our controller. She has been with us for about a year as well. That is Natai Nicicolesu. She has been fantastic in this work and making improvements all the way along. So I know that in the packet there was a a grid with some very tiny print uh that we have been working

063from in terms of the original um recommendations um an initial response and then the status of implementation and I'm not planning to go through them one by one. I would rather just give you kind of the highlevel um where the gaps were and the steps that we've taken to close those gaps. All right. So, we have there was a real uh emphasis on a need for in-person training onetoone both as staff turn over throughout the year and not just one-time training. Um we do have uh one FTE support person um that that is out in schools now working with our uh bookkeepers and our business managers managing those student body funds. Um but we've really increased our standard of training and made it more available to folks. Um and we do come out on request.

064We also have offered training for supervisors who are which would have be our administrators that oversee this work to make sure that there's a level of accountability as well. Um we did add uh our annual pepper training which is our platform for um it's kind of asynchronous. people can access that when they're available. And for anyone who has access to EPES, which is our software that we use for SPF accounting, they have to have this training completed um before they can gain access to that. For our pecards, we have uh an infraction model which kind of gives guidelines and training on when you need to do what by and how you need to proceed and if uh trans transactions occur. uh outside of that framework, they begin a series of infractions which can eventually lead

065to um discipline or or termination of the PE card or whatever. And so we've incorporated that same framework into uh infractions for when we notice that things aren't reconciled in a timely way, deposits aren't happening in a timely way, and oversight isn't occurring as it should be. So that gives us a a kind of a standardized framework for not just training but for following up with additional support as needed. Um we do hope to incorporate functions into the new ERP. Uh you've heard a lot about that. It's going to have a lot of great functionality. some systems can be set up as a little like mini version of it to um provide support to our schools that have their own independent student body fund accounts. So, that's good news coming. Um and we're also offering

066additional support for SPF uh users and folks that access that. Even though we're taking reductions in the business office, we're reworking some of the things that we do in order to bri provide a better level of support and oversight to our schools. So that's probably those are all my talking points. Um I can try and answer questions. Netatai is also here to answer questions if they go beyond my level of understanding. I don't I don't think I have any questions more remark like I'm um pleasantly surprised. Is it is it usual I haven't seen these obviously they didn't see this one last year. Is it common for a district to agree with so many of the recommendations and then work towards implementing those? It's very impressive to see agree agree agree and then taking steps

067to implement the recommendations. >> Yeah, we definitely want to improve our work. So in prior districts that I've been in, there's actually the third party independent auditor does that work and then we implement those recommendations. So student body funds, they are an area of risk that concern everyone involved with that. So we're yeah, we're happy to do the work. >> Thank you. and student body funds. Just so I'm clear, those are things that's basically kind of like cash on site or then like probably student stores, yearbooks, events, like game game admission, things like that. Okay. >> Yeah. Cash handling, online stores, things like that. >> Great. >> Okay. Um just quickly uh typically what I try to do in advance of the audit committee meetings is review the supporting documentation that's provided. And what I'll

068do is go in and confirm once I get the evidence um and then gray them out so everybody can see that they are grayed out and completed and confirmed implemented. Um, and uh, Chief Morrison's team did send over a couple of documents this week and I just didn't get a chance to review them and confirm the implementation. So, what I will do is over the course of the next week, I will send out an updated template with graying out the ones that I was able to confirm. So you will have an updated um list acknowledging the hard work and the ones that we were able to confirm are implemented. I don't want to wait for a whole another year to show you that we confirm those. So I will do that next week and get

069an updated email sent out so you'll be able to see which ones I was able to confirm. >> Okay. Any other questions? Um yeah, just regarding the student body funds audit, um as a like club as a leader of several clubs at my school which fund raise, um you know, the process for accessing some of those student body funds is notoriously difficult because of you know the like various various forms of people that we have to go through. Um, is any of the training that's being given to these people on site to like, you know, is any of that trying to streamline the process of accessing those funds? >> I can get more information about that. I haven't asked that specifically. We were more focused on controls, which would probably be the opposite experience for

070you. >> Exactly. We It's intentional. >> Yeah. It's It's never our intent though to to be a barrier either. So, I'll check in on that training and and with my team as well. Good question. Um, any others? All right. Thank you, Chief Morrison. Um, we're going to go on to the next agenda topic. Um, now we're going to reflect on the audit committee's work this year and review the district's audit policy and charter, which I know Janice has been really working hard on to make it very workable for for her and for the district. So, Janice, would you please introduce this item? >> Yes. Thank you so much. Um so I wanted to take an opportunity towards the end of our year as we've been together for a whole year to talk about um you

071know how things have gone in the audit committee and provide each of you with an opportunity to give me some feedback on what things have been going well and potentially feedback on things that are not going well that you would like to see some changes I'm always receptive to input and I actually really appreciate it. One of the things that I heard a lot from some of you or most of you this year is that you really appreciated the um document that we created for the work plan that laid out all of the agenda meeting uh all the meetings and the agendas for roughly for the entire year. So, you kind of had an idea of what we have planned for the year. And so, I will keep doing that, but I also wanted to

072start with an opportunity for any other feedback because I just haven't heard much. Um, so I just want to make sure and give that opportunity for feedback. Um, other than the one work plan document, are there any requests or preferences or thoughts uh that you want to share? Well, I'd just like to share how I appreciate how you go over things with us individually and really explain what's happening because without that, I might just be staring blankly at some of this stuff. So, I just I really do appreciate it. And as you said, you know, I really like knowing what's happening through the year. I like that we set up um every was it every Thursday? Yeah, we >> got it there and we can always cancel it, >> but it's there. Um so that

073really worked well this year. I appreciate it. So, it sounds like we're happy with having monthly scheduling next year for an entire a monthly meeting with the option to cancel that meeting if we don't have anything necessary. Um, but it would be there every we would have an opportunity to meet every month. When I look at um the things that I feel like I need to bring to the audit committee, um I'm really I really think it needs to be quarterly. Um but I really appreciated the opportunity to have that meeting monthly and this year seemed to work out well. We canceled a couple because we didn't have any um topics specifically. So, I am totally supportive of that um concept for next year. Um sounds like a good plan for everybody. Yeah. Um, one

074other thing that I did bring um that I wanted to ask for feedback on is in your packet there is what a one page um that is a um sorry the matrix it's a audit committee matrix. Yep. Uh you have it that's the correct one. Yep. That's it. So when I have been re doing some research on different audit committee charters, I found a charter that had this matrix. This right here um inside the charter. So what I did is um I took the template from the example charter and I popped in just the top two two or three responsibilities of the audit committee. So y'all could look at what the charter would look like as it applies to PPS. I didn't list all the responsibilities, but I wanted to give you an idea of

075what the matrix would look like as it applies to PPS and and the work that our audit committee does. So, I'm looking for feedback on whether or not you guys feel there's value in having that matrix with identifiable deliverables being included in the charter. Um, perhaps it's just an idea. Um, I thought it was really cool when I saw the the charter with the matrix and so I wanted to share and see get your thoughts. Um, I don't have a final charter ready to present um for tonight, but I am still doing a little bit more research. So, this is the chance to give me, hey, we really like this, then I'll keep going with this. And if you think it's a little overkill, I'm happy to um, you know, move on without without. This

076is a little hard for me to to read, but I do like the idea of knowing >> what we have to do. And unfortunately, what we have to do isn't always scheduled scheduled the way we want it because they can't present when we want it. >> But something like this is okay. But I'm having trouble reading this actually. >> Okay. Um I will sit with you next week and kind of walk you through what that um what what the matrix kind of give you a better understanding of how it would work. Let me borrow this or sorry I have a whole stack and then I can't find this one. Um but just at a very high level you know the audit committee responsibility the top one listed is the review um uh the charter for

077the audit committee you know so what are the how would the audit committee accomplish that objective that's what that second column is for what's the deliverable tied to that responsibility and then how often does that responsibility ility need to happen. So on the example of the charter um it's really best practices to reconsider it every 3 years. So it would only need to be reconsidered every 3 years and in the matrix we would put when the last time it was reviewed so you would have an idea like well it's been three years so we know this needs to happen this year. Um, but that's high level, but I'll give you more details. Um, next week, any other thoughts? >> I was going to say I really appreciate this because I like road maps and um,

078as I'm learning realizing when things have to happen annually or when things get slipped in. I just really like a a road map of knowing when we're going to look at something and then also being able to sort of check off like, did we accomplish this? And I see like, you know, one of them is review annually at one of the October, November, or December meetings. And so then if we have this roadmap and there's no check off by the time we get to January, there's sort of like a an uh oh, we've got to do this immediately because we >> people are busy, things get really stacked up on a on an agenda. And so we know, oops, >> let's make sure that that's top of the agenda in January because we didn't. So

079I I like this is what I'm trying to say. Thank you. Yeah. Means we don't miss anything. >> Yes. >> You know, I I kind of thought this might be a good tool um especially as we transition to hopefully not next year but in the future for a different audit committee chair. This might really be helpful for them to help organize. So I think I hear what you guys are saying and so I will move forward working on this and and present um something along these lines whenever I have the charter updated. Um okay so that was just one piece. The other part um and also just keep in mind you're welcome to give me feedback at any point. So, just because you didn't say anything to me today, please feel free to send me

080an email if you have any thoughts you want to share um because I really appreciate it. The next part of this topic that I wanted to talk to you guys about is the audit committee or the audit the performance audit policy. So, I was really excited to see the board retreat earlier this week where the board had an opportunity to get together and talk about um values and as it relates to policy writing. Um, and I was really excited because it was so timely and I feel like it really sets up the um, lays a good really good foundation for having this conversation about um, the auditing policy because it is one of the 10,00 based off of the discussion in the board retreat that um, the board is or the policy committee is going

081to focus in on these 1,000 series utilizing some of the techniques that you learned in the board retreat and I am very excited because it's the timing is just really good. This never happens. um audits are almost always, you know, a year behind, but it just would really laid a good foundation for engaging in this conversation. Um because that happened just this week, I would like to um have an opportunity to put together a little bit better of a presentation um where I am able to tie the values that were discussed to the proposed updates. So, I wanted to just ask you today for some feedback on three key categories that are relatively new to the policy. Um, so in your stack of documents, you will see a redline version of the policy. And if

082you flip in that document to page nine, at the bottom of page nine, this kind of starts um a new category and this is the category is risk assessment, three-year audit plan, and annual work plan. Um, so what I will be proposing um is transitioning to a three-year audit plan um and risk assessment and the annual work plan being based off of the three-year plan. So let me just give you an idea of what that means. So for bullet point A, it says every 3 years, the office will require an enterprise risk assessment be conducted. Um, and it should identify, assess, and prioritize and respond to opportunities and threats that affect the achievement of district objectives. Um the enterprise risk assessment can be conducted internally or based on a recommendation from the auditor and budget

083constraints the office may contract um with a third party to complete the risk assessment. Um, in our meetings for most of you, I've had an opportunity to get show you an example of an enterprise risk assessment that PPS has received in the past and I showed you what that looked like. Um, and I have another example, a more updated example of what um, a risk assessment with the three-year audit plan would look like for us to conduct internally. So, in your packet, there is a section called additional documentation um, part two. In there, yep. Um, there is an example of a three-year audit plan and risk assessment that you guys can take a look at to see what that would look like and we'll give you a better un highle understanding of an example of

084what I am proposing. So, I'm sure not all of you have had a chance to um read that three-year plan in detail. So you are welcome to reach out to me incrementally and give me that feedback. If you have any questions or thoughts about that category or any concerns about u me proposing a transition to that, please let me know. I would really appreciate it. The second category that is relatively new is on page 11 and that is the external audits and review section. Um so under the proposed language um whenever the district is having any type of audit being done um either internally or by externally the departments under audit will need to consult with me to um share like the scope of what the audit is and also the outcome of the audit.

085Um, a couple an an example of when this happens is OD will sometimes or oftentimes actually audit compliance with some of um like special education um amongst other programs. And um it is really of value for me to know what they're testing and what the outcome of those audit results are. It will really um impact my ability to do a formal risk assessment because I will have better insight into the individual departments. Um, so that's my I mean there's lots of thoughts that went into why I'm proposing it, but that's kind of overarching. Um, so it's a newer category and so please read that category in a little bit more detail and if you have any thoughts you want to share with me as it relates to that, that would be very helpful. Ideally, I'll

086have an updated proposal uh proposed audit policy for the audit committee along with a presentation um in our next meeting. And ideally hopefully the audit committee will be able to vote to refer this policy out to the policy committee for the next steps. Um, and I am I I talked to Director Sullivan about this and um, we were both kind of excited just how timely that conversation was. So, I I really want to capitalize on that and keep the momentum because it was such a relevant conversation and it will really help us have this conversation. So, I'm going to do my very best to get an updated uh proposed policy that we can refer out in our next meeting. >> Um, I have a question. I'm sure somebody else does too, but so when you

087talk about the three-year plan, because now we sort of give you ideas for an audit and you know, when I was first on the committee, I thought, oh, that that'll be done in a year. Then I realize you have to figure out how to do the audit, right? Takes about a year to do that. So how would what the board or the audit committee and the board is suggesting, how does it fit in the three-year plan? >> So um in lie of doing a risk assessment, I have really taken used the opportunity because I haven't done one and I've been super transparent about I need to do one and it needs to get done and it's required. Um but in lie of that I have been um asking for suggestions a little bit more openly

088and actively. Um, I sent a list to each board member saying, "Hey, give me your thoughts." And I used all of the input that I got to um as a and not a substitution, but kind of a as a little bit of a risk assessment since I didn't do one. Um so moving forward the risk assessment will drive the audit topics that the audit committee will want to select and I will do um you will see much closer of a proposal um from me as it relates to it will be less asking for audit topic ideas and more about me talking to each of you about the risk assessment and how that risk assessment should drive the audits moving forward. Um, and that's the way it's supposed to be. I just needed a way to

089um, you know, generate areas of risk of concern from the variety of stakeholders. Um, so that's the way I have been doing it. But moving forward, you will see a little bit of a change as as it relates to that. The proposal that comes to the audit committee will be driven by the risk assessment. You will still get to give input and collaborate with the audit topic ideas as will all the other stakeholders. I meet with um other board members as well as PPS staff leadership um to get their inputs as well. So there still be the same level of input opportunity, but you will see the proposal being provided based off of risk assessment and not so much just on the ideas that I have gathered based off the risks that you guys have

090we have talked about >> or the even sometimes it's not even risk we're thinking about as much. It's just something that's bothering us. >> So yeah. >> Okay. Thank you. Um, any other questions for Janice about this? >> I I just have one question. Again, this is my I'm so new to I feel like I'm new when I'm all these questions I'm asking, but um I am new because I've done this before. Um, why? So, I noticed the documents too are Seattle public schools and I am curious. I imagine that Seattle Public Schools internal audit work plan is based probably at least in part on the equivalent of the Washington School Board Association andor the equivalent of Washington OS's and rules and laws and os. So why Seattle and what are best practices as recommended

091in Oregon? Yeah. >> Is Seattle sort of leading the way in? So tell me why Seattle. >> Um thank you for that. So, I used to work for Seattle um in their internal audit department before I came to PPS. So, I am so lucky to have a resource in my colleagues that remain at Seattle public schools to talk about audit topics. Um and really what the proposed also in your packet you will see a audit report that was done by an external audit firm of the internal audit function at Seattle School District. And I really appreciated that audit report because um it really it you know in a small internal audit department often the recommendation to be more efficient is to get more staff you know whereas this really focused in on how to be

092an effective get the best out of the department of the size that it is and we are of a same size currently. The they have two my they have their um chief auditor and one additional auditor. They also have a full-time FTE auditor who is bonded and focuses solely on auditing bond funded projects. Um but other than that we are very similar in size. That is my experience. Um, audit com the three-year plan that I gave you an example of that is the standard is audit standards and it's not isolated to a state per se. So um the founding principles of how it was generated would be applicable for everyone. Of course, there are some regulatory differences. And even now, I'm still shocked at I keep finding things that I thought were standard practice, but

093then I realize, oh, that was only in Washington. In Oregon, it's not an expectation. So there are some differences but the driving factor is audit standards for um the expectation of what is being provided and not necessarily in state specific and the reason why um I use Seattle and I have I'm part um of a quarterly group that meets together the council of great city schools all the internal auditors ers get together once a quarter and we talk about whatever we want to talk about and it's super um it's great to have that opportunity and there are not very many departments of this size. So when I was doing my research I did try to look specifically for school districts that are of our size. It really makes a difference. Like in our last meeting,

094um, one of the audit executives was talking about how meaningful it was to lose like two employees, um, two of their team and they had a team of 30 >> 31 50,000 students. >> They had 31 auditors and they were losing two. I That's just a different perspective than our perspective. Um, so I did try to find school districts where we were of similar size. You're welcome. >> I have a couple questions. Thank you for this. Um, my biggest question is just what kind of timeline are you looking for for feedback because I definitely think I'm going to want to spend some time with this. Um, could we please say two weeks? Ideally, um I need to I would like to um present the updated policy for the audit committee in our last meeting of

095this year. And at the very end of our meeting agenda, we do need to talk roughly about scheduling the um June 18th meeting. That is what is already on the calendar for our final meeting for this year. Um and I think we might need to do a little bit of rescheduling. We've had a couple of um committee members mention some challenges. So, if we update that um and extend it maybe a little later in June, then we could go an extra week. But how much time would you like? >> I mean, as much as I can get, but I'll make it I I'll make it work. Um but I was also wondering you know if you have looked at other examples besides Seattle public schools um I you know and even outside of schools maybe

096there are a lot of other very small audit functions in counties or cities >> in universities I found. So um you know I think having a really strong >> um policy for this office is going to be really valuable. also finding those best practices, those other examples. Um, >> great. I will bring some additional examples next time. Thank you so much for that. But I think you'll also give me lots more feedback in the next couple, >> which I appreciate very much. Like, you guys do not understand how nice it is to have another professional auditor to brainstorm with, and I just really appreciate that. >> Can I ask a question? That's a perfect segue to my question. You mentioned that um did you say that Seattle has an internal bond auditor that auditor that

097solely focuses on bond funds? >> Yeah. >> Is that common across just I don't know if you know the answer this >> because I'm I'm I'm curious um because it is you know we have as much you know more more money in bonds than we do in our general fund. Um, yeah, and hearing that makes me wonder um if that's something we would want to consider as a board as as a position that's funded with the bond. Um, in in part because you know we have restructured the department and we are working with you know an owner's rep that's you know essentially embedded with the department. I've wondered like where does the information live? Does everything live on our systems or is it also on outside systems? you know, how you know, where is the

098district's information? So, that's crossed my mind. So, I just wanted to just flag that as something um you know, I would be open to considering. >> Thank you for that. I had a feeling um that this question might be asked and so I just did a little bit of preliminary research um and unfortunately I didn't look to see how common it is. So I I can't answer that element of it, but I did look into confirm like yes, in fact it is 100% bonded. Um and I gained a little bit of an understanding of what exactly a bondfunded internal auditor would look at. So I can speak a little bit to that, you know, in as we move forward. Um, I have been thinking about this a lot because one of the things you may

099hear me say recently is when there we do unprecedented things, we need to do have unprecedented oversight. When we do new activities and delivering three brand new schools all kind of at once is relatively unprecedented for our school district anyways. And so the li the risk associated with that activity is also unprecedented. So um I am open to that conversation. Maybe not right this minute, but I am open to it and I've been thinking about um how much over how much of my time that I use mo monitoring that activity to identify risks um and whether or not that's sustainable over the course of three years. I will um be super transparent and say that I have been very mindful of the bond related activity over the course of this year with intention because of

100the staffing changes. Um and so I've been paying careful attention watching all the BAC meetings um reviewing all of the documents that are available. Um, so but that is not something I would like to do long term and ideally as you know the BAC updated their charter that's going to be coming back to the facility committee and they are willing to take on a little bit more responsibilities which is awesome. So when things kind of get into a cycle like um for example Sarah said she'd share her documents with me that will make it a lot easier for me to understand and monitor the level of risk associated with that activity. Um right now in the past few months I just haven't had that purview. So um it is possible that it is feasible for

101me to monitor the level of risk at a high level. Um but I need a little bit more time. But I'm well I'm open to the conversation in the near future. >> Well, you gota do two jobs. Doesn't seem quite fair. Uh, I would like to uh reiterate how often Janice says how much she appreciates you too. I hear it all the time. So, um, so thank you, Janice. Um, do you are there any do you want to say anything more or are there any more questions? >> I think that's all I have as it relates to this topic. Yeah. So what we want to do is get feedback look at this so that when we work on the policy in policy committee we can agree that it's what we'd like them >> ideally we'll

102have the core content referred to the committee the policy committee from the audit committee and based off the conversation I think I'm very close to being able to tie by the proposed changes to the values that were discussed in um so I think it will do well once it gets to the policy committee there might be some word smithing and stuff but I really think this updated version will align very well with a lot of the things that were talked about. So once we pass it on to the policy committee, I think it has um it will be successful. Um based off of what you you guys were talking about in your retreat, >> um I'm seeing the the lawyers sort of directly behind this way. Um before we work on it and give you

103>> feedback on it, >> has this gone through legal? Can it go through legal before we start feedback so we don't >> do feedback and then have to have something undone? >> So what I have been meeting regularly with um Liz Large in our legal department. I've provided her and she and I have gone back and forth multiple versions of the policy. I just had a quick chat with her be at 3:30 today, right before our meeting, and she has done an another review of the version that I'm showing you today that she has done another review and given me feedback. So, she and I are going to meet next week to go over that. Also, I have been working with Chief Morrison to get some input from her also. and she also let me

104know today that she has had an opportunity to look through the version the last version that I've shared with her. So I think next week will be busy for me but also a good opport I mean I'll be collecting that input. Um but Liz and I have gone back and forth a couple of times already. Um, and so it's I'm trying to capture in that uh PPS staff input um as part of the um the proposed. So whatever input they may have, if I can reconcile it and just, you know, then I will before it even comes to the policy or to the audit committee. And if there's feedback that we don't necessarily reconcile, I'll just keep the comments in and we can talk about it in the audit committee. >> I think um the

105draft you have in front of you is still very iterative from the legal team to answer. I mean, that's really the question. That is not a final legally approved. Um, not that it's ours to approve, but I want to be really clear that it's still iterative and there's still um feedback and work that needs to go into it and reconciliation. So, when you get it, assuming you get to the policy committee, I think that's what I heard the plan is, >> there would then be the staff memo and the staff review and the input and the legal the legal analysis that goes with it. So what you have is not yet that the same thing that happens with all the other policies that you get at the policy committee. Does that make sense? >> It

106does make sense. I'm just wondering the order of spending time reviewing it before that and whether >> if we want to talk about it here, which makes a whole lot of sense. >> So excuse me. Can it can it do an analysis, come back to us so that we're not reviewing something that then hasn't gone through legal and then we can send it to policy. I Yeah, I just >> my suggestion is that at the policy so it's a question of how the board is going to want to handle policies. I mean this is a section one just like all of your other policies. Um and so we have deliberately I mean intentionally and appropriately so wanted you all to do the board work that you did at the retreat. The the dynamic or the

107protocol or uh the rubric maybe is a better word that that you all use for the first policy would be to start from the policy itself apply your values and your board uh impromater for lack of a better word then that's goes to staff to then iterate from and give back to you. So if you follow that that's a that's a completely different approach than this. So it's a question of how how you want to handle it. And with section one I understood the my understanding potentially flawed but is that the board wanted to could do it to take that approach with all of section one for each of them. They are so substantively representative of the board. Um and so if you do that it would be to take that back then if you

108treat it like the other one the other was a ones in section one would be to start exactly like you did with the board retreat on whatever 1.010 or whatever that was. So however however it's decided to keep doing section one. My suggestion would be to do that and that doesn't mean this discussion can't happen but it would start from the baseline of what was there. what are you trying to accomplish applying your values and then iterate from there and in that step excuse me in that step there would be a legal review does that make sense it's a little bit of >> um >> this is this is in a different posture for lack of a better word than the rest of section one so it's a question of how to integrate that and

109integrate what the board wants >> that's very helpful and it it is such a different posture because the the experience for this is here, >> you know, and rather than asking you to whoever it is to come to the policy and present to us. So, I understand why it's here. I just want to make sure that we're reviewing something, not giving feedback and then having it completely undone because maybe it doesn't pass must >> well and I think I think more importantly the understanding you all had as a board about how you would approach this sections this section being section one how do you want to integrate that how do you want to do that and how do you want to take the feedback from this committee as one of those but overlay the values

110with it though I mean it's kind of been a wonky situation, wonky posture for lack of a better word. And you're right, you have a bunch of experts here. How do you then incorporate that? Um, there's lots of ways I can see doing that. But given the retreat, my suggestion just as a staff the staff person of the policy committee is that you follow the process that you all identified for the rest of one and do it the same way uh and iterate from there with the with the with the input of this group after that or as part of that maybe is a better way to say it. Does that make sense? And not that you have to agree with me. I just want to have made sense. Okay. So the parts to clarify

111what I would prefer some feedback on are those three categories that I mentioned are relatively new to the policy. If you have a chance to look over those in more detail, maybe not so much the word smithing, but conceptionally if you feel good about it or if you have any questions, that would be the element that I would appreciate. some those three categories that I mentioned are the areas I would really appreciate feedback from you guys on um specifically sooner rather than later. >> Could you repeat what the three are? >> Yes, I can. So at the bottom of page nine is the three-year or the risk assessment three-year audit plan and annual audit work plan. Um so that section the next section is the external audits and review section and the which is on

112page 11 and the third section is the implementation plan and validation. Um we already have an implementmentation plan that we do and utilize. This is just really formalizing the process in our policy, but it's still new to the policy. So, >> that was it. The implementation plan and validation is the third category on the >> bottom of 13. So, not indepth feedback, but conceptual feedback because you know that would be very helpful. So when I bring the updated policy conceptionally, I've addressed any conceptual concerns in the proposal at our next committee meeting. >> Sorry, I have one more question. I guess um the timeliness of this I like understand the idea of wanting to do this in the next couple of weeks, but like we're in the big budget session. We've got a meeting next

113Tuesday that we're prepping for. Then we've got the budget >> listening session. Then we've got all um is there a is there a specific reason to get feedback to you in two weeks so that we can do this in June or >> could it be? >> So the audit committee technically ends um before it's reconvened. So the um you know when we laid out the agendas for the year, you know, Director Sullivan and I did not realize that this was going to be on the agenda for the board at this time. Um but I just feel super lucky that it happens to have been. So it really led to kind of it will help support the conversation. Um, so I do think it's timely, but I see your point. Um, and the the I don't

114want to say urgency, but I will say that um, it did take a lot of my time to get to this spot. And I would like to not lose that momentum. And also, I really like to capitalize on the momentum of the discussion in the retreat. It just really resonated with me and I think it will set this conversation up so well. Um, while it's all fresh in our heads, it would just really So, that is the only real rush. Um, I will just acknowledge if we go probably more than two months, I really have to refresh my memory on where I am at to be able to speak to it um very well. So, if we don't do it within two months, it it would really be a loss of some efficiency on my

115part. Um, just to answer your question, >> thank you. Well, and you're talking about a agreeing with the concept, not with exactly how it's it's written. >> Okay. Um, we are way over time already, so we're going to hurry it along here. Um, Janice, thank you so much. I know that was a lot of work and a lot of good good ideas there. Um so now we are going to talk about um a committee membership and Janice you wanted to talk about um referring our two community members for next year because I think they've agreed again. Thank goodness. >> We're always glad. Um yes, thank you so much. I did put together a few talking points to talk about the value of consistency in audit committee membership. Um, but I'm going to pass over that

116those talking points um because we're so late right now. But I wanted just to mention that I hope each of you um takes the time to chat with me separately in the future about some of the research that I did as it relates to the value of um m being consistent um in the leadership of an audit committee meeting. But to get to the exciting part of today's agenda, um you know, Director Sullivan has reached out to both of our community members. Both of our community members terms end the end of June of this year. So the policy allows for reappointment of the community members if they're willing to serve for another term and they have both acknowledged that they are. One element that I would like to propose um is staggering their term. Um,

117and so I met with both of them earlier to ask for their thoughts about potentially staggering the term and they were welcome to that. And Jennifer mentioned um this if we one has a lesser stagger and Caroline >> two and three. >> Yeah. And Caroline was willing to take on you know the extra year stagger. So, um I was looking right today and I um for the policy that says what the terms are and I couldn't find it for today. So, I'm really sorry. I um but I do we do need to have the audit committee refer our two community members to the board um for reappointment. So, that what is what does they say? Two years. for two years and may be reappointed at the end of their term. Is that not >> Yes,

118that is in our audit committee policy. But I think there is another policy that speaks to community member um terms more broadly. So, I just want to double check that before I confirm, but it is my hope that we will um refer Jennifer for two years and Caroline for three years, assuming I don't find anything that says two years the max. So, um that is the request for today. Okay. And we don't do formal voting, but can we just >> Sorry, one more thing. I'm so sorry. I forgot. I wanted to um mention our student rep. So, we have been so lucky this year to have Eban. Um and he is a junior >> and in chatting with him, he has mentioned he's willing to take on another year. and I am so excited about

119that because I've been working with him on our building security audit. Um trying to let him give me some a lot as much as he's willing to give me input and thoughts on the student perspective and he and I have been working on a survey. So it would be really great if he was able to stay for another year and kind of see that audit through fruition. Our policy does allow us to have two student reps. Typically what we allow the student rep to the board to assign um committee assignments to the students. Um, so that is the practice that we have been doing and I would like to keep that and let our new audit student rep to the board have an opportunity to assign another student if there's one interested and we can

120have a second one because our policy does allow us to have two. So, if there is another student that would be interested, I don't ever want to pass up on the opportunity for a student to get to talk to a student about auditing. It just warms my heart. Um, so we could have two, but I just also wanted to mention because he's a junior and he has said that he's willing to I will kind of send an email to the student rep, the new student rep. When I ask for a new student member, I'll also mention we'd be happy to have him back. >> Yeah. But we can't quite well vote him in yet. >> Yeah. We don't need to vote in relationship to that >> because I asked Janice about that. We want you

121in. >> Yeah. >> As long as you're willing to stay. >> Thank you. >> And I remind me, did you come in no November? Was it when did you first start coming? >> November was my first meeting. >> That's what I thought. Yeah. And so, um, I love the idea of you staying on because then we could just keep you now instead of the new student rep getting, um, their feet under them and then finding other students who are interested and that perhaps the student rep, the new student rep could find another junior who would be interested in perhaps being the senior the following year and then we would have sort of continuity of someone for two years, >> right? Yeah. Yeah, and that's what we're doing with the DSC like positions for grant. Um,

122at, you know, last week's DSC meeting, we did discuss that we're going to, you know, try and get these things set up sooner because the reason that I started coming in November is because we didn't really set up board positions until October. Um, but that process should be accelerated next year. So, you guys should know before the first meeting of the year. >> Good. Okay. So um this is very informal but can just can we have a thumbs up for referring these two appointments? >> Yeah, you can. >> Okay, >> double check with Christie to make sure cuz she's a member. Is she Papa? >> She I think her daughter's performing. >> Okay, no problem. >> Yeah, she definitely planned to be off probably by now. So um okay, next is for Janice. you're going

123to be talking about your um audit update and status. >> Yes. Um I'm so sorry we're running so late. Um I know in our last meeting I mentioned today maybe a two-hour meeting, but I tried very hard not to have it be a two-hour meeting. Um so just very quickly uh in the stack of documents I provided the office of internal performance audit status update and I will be very quickly go over that and you mind if I still thank you. Um so I just wanted to give a highlevel overview of where we are on our audits. Um right now we we as in Mary Katherine has just let me know that she's wrapped up on the fieldwork portion of the measure 98. Um so what that means is I will start that review process

124and writing the report. Um and right now I think we are on track to have that report ready to come to the audit committee in our June meeting. The next audit that is of our focus is the building security audit. Right now we're in the planning phase and like I mentioned I have been making meet working with our student rep to make sure I get some student feedback and input on the work that we have planning. But we are really close to wrapping up that planning phase. And then I'll be turning over like the fieldwork piece of it to Mary Katherine. Um, and we will I mean there's so much work you have to it will be a lot but the wrapping up of the planning phase is getting really close and we'll start into

125the testing phase um very soon and we are tentatively planning to wrap up on that audit in the fall. one element um you know when we go out to inspect the actual schools sometimes it's nice to not have kids there so you know we can get see where the cameras are loc and all the things but it's also important to do some touches um some inspections when students are in school. So, um, the first couple weeks of school, we'll do those final site visits and then wrap up. So, early fall is what we are hoping for. Um, then the CTE audit, this I mentioned to you guys back in the fall that the scope that was previously approved has been outdated. So, I have been waiting until we get a little bit closer to actually

126starting that audit to rescope it. So, I uh will be working on that rescoping of the audit over the summer. And I'll bring it back this updated scope to the audit committee just for review um before I actually get started because I am going to modify the scope just a little bit. and then that will wrap up all of the audits that have been previously approved. So that is a really good opportunity timewise for our new three-year plan. Um, also one of the things that we did in the fall is talk about future audit topics and um, the audit committee we talked and narrowed down a nice long list down to three topics and for those three topics I have been studying those departments um, gaining information about what's going on and preparing to propose

127a scope of work as it relates to those three topics. Um, and we are not we we do not have the capacity to get started on any of these audits until next year. So, we have a little bit of time, but it is um I in the three-year audit plan example, I walk through a couple uh with a couple of you guys like how one of these three how these three topics could potentially be popped into what that plan would look like. So, I don't think we'll lose any of this. Um, I still see these topics coming to you guys as a proposal for an audit in the fall, but it will be part of a three-year plan. Does that make sense? >> Were we sticking with three or were you deciding going to pair

128it down to the two? >> Only get to pick two. You're right. I'm so sorry. I forgot. I'm going to propose three, but you only get to pick two for inclusion into the audit plan. >> Okay. Thank you. I Does anybody have anything else to bring up? We need We need to change the date. We need to change the June >> 18th date. If you have your calendars, can you look and see if >> So, I looked before I came down for the meeting and it looked like there may be an open opportunity on June the 25th, which is the following Thursday. It didn't look like there were any other um board activities. Um I I checked with uh the board manager about graduations because I know that's a busy graduation. Sounds like that season

129will have wrapped up by the 25th. So that was my thought. I just want to throw that out there as a suggestion. Any other time will also work, but that's what I checked beforehand to see if it would would work. >> I know you often have Thursday problems, but is that over? I don't have I don't have my Thursday. Um, let me see if I can look really quickly. You're right. I have the conflicts have all which I missed again this evening have all been double. Let me see. Um, >> I um I'm I'm gone the last two weeks of June. So, I was going to miss the 18th anyway, but um the 25th also wouldn't work. I'm >> okay. So you could zoom from your tropical island possibly. Possibly. Got a pretty packed agenda

130there. Real commitment. Uh Patty, you're so good. Um and also I think Michelle had another uh other topic that she wanted to mention before we wrap up. I don't want to forget to give her the opportunity to bring up her other topic. >> Oh, okay. Yeah. Just Yeah. Sorry to slip this in under the wire, but my staff brought it to my attention that I need to probably share with you guys. Part of uh some information that we're going to bring to the board uh on June 23rd is going to include a request for appropriation changes, a potential supplemental budget. And the reason for that is because um as we have a very tight budget this year, if we exceed our major fund and fund uh function appropriations by more than 10%, we could um

131draw a finding another audit finding unless we bring appropriation changes in advance. So it's it's complex. I wish I had more time to share with you about it and I'm happy to send up a like a follow-up email to kind of show the timeline and process, but I just want to share with you as audit committee that that will be coming to the full board. And so I was hoping to have more details for you on the 18th, but it sounds like that meeting is being moved. >> Oh, more details on the 18th. >> Yeah, I thought that's when the 25th. Um, >> well, ideally you should you should have it before the next board meeting. >> Okay. >> Is that something that we got a finding on related related to last Okay. >> Just

132I thought it was so I'm so glad to hear that you're being super responsive. Yeah. >> Yes. Yeah. >> Awesome. >> Okay. Well, an email would be great. >> Okay. I'll send more information. Thank you guys. >> Thank you. >> And as always, I'm a resource if you need to talk it through. Okay. Does the 25th look okay? >> And it sounds like the 25th is not going to work for Caroline. Um, >> yeah. Well, we >> I try my best to accommodate our community members where we can. Um, I know that technically our kind of committee work wraps up in the end of June, but we could potentially do early um early July if uh I'm here every day. Um, so just whatever works for you guys, but we really do need to get

133one more meeting in the before kind of we wrap up. The one of the key elements we need to talk about in our last committee meeting of the year will be the policy of course we just talked about, but also my performance evaluation. That is something that the audit committee gets to provide a lot of input and it's you the draft is done by the chair of our audit committee and she likes to get input from each of you. So I want to make sure that everybody knows that will be talked about. We that needs to happen by the end of June. Um, okay. And I can probably do a summary email if we end up not meeting until July, but it needs to formally be submitted, completed, and signed off on by um, Chair

134Sullivan by June 30th. So, if we don't have a June meeting, I'll have to send everyone detailed emails about how to get you input. >> Well, we'll probably do that anyway. >> Yes. So, so they could get the input. I could sign off. >> Yes. >> So, we could still have the meeting in July. >> We could. >> Yeah. >> But I just want to make sure that that is something that kind of we talk about in that final meeting. Um, but I will get some emails sent out next week to make sure each of you have an opportunity to provide Director Sullivan with some input. um we'll probably follow the same process that we did last year. And I know some of you weren't around last year, but I'll go over the process. And

135she sends this out to all board members, not just the audit committee members. Um all board members get equal opportunity to provide input, but um as the chair of the audit committee, she gathers all of that information and puts it together for us. >> And it's only audit committee members that usually respond. So there you go. That is also true. >> But you can't >> good. >> And I'm I'm realizing we don't have director split for choosing a new date. >> So maybe we should not do that now. >> Shall we mail for a new date? >> Yeah. I think everybody knows how just how hard it is to get us all together to come up with a an alternative date. So I will start come up with one >> right away. doesn't have Christie

136if and if it doesn't work for her then we'll >> okay >> email so let's see if at le least we can get one for this group >> y >> so how about July 2nd >> that is the conflict that I have every Thursday >> oh that's Thursday no I just looked it up I don't have anything on the 25th but it's right here in my calendar on the second >> okay the Second, third, the first. >> How about Wednesday the 1st? >> Um, I think we just got a request from Roseanne to put off talking about finding a date. So, I'll send some emails out. It was a good idea to try to get us to agree while we were here, but we will um I'll send some emails out tomorrow and follow up

137next week. >> But are we officially canceling the 18th? >> The 18th is officially cancelled, >> I think. So it would be my request and anything else for the good of oh my gosh after 7 o'clock. Anybody else? Meeting adjourned. >> Thank you.