001Okay. Hello everyone. It is 4:30 and I would like to call the audit committee meeting to order. Thank you everyone for being here. My name is Patty Sullivan and I am the PPS board member representing zone 3 and I am the chair of this audit committee and I'm excited to welcome our student representative to the audit committee. Thank you for your willingness to serve on this committee. I don't want to put you on the spot but I guess I'm going to. Would you like to introduce yourself to the committee, talk about the school you're from, and anything else you'd like to share? >> Um, >> yeah, push the uh green. There we go. >> Sure. Um, I'm Evan Cole. Uh, I go to Grant High School, and I'm one of the two uh DSC representatives
002for Grant. I don't think there's anything I'd like to say at the moment other than I'm grateful for the opportunity to work with all of you and I look forward to the year ahead. >> What year are you, Greg? >> Uh, I'm a junior. Yeah. Well, thank you for being willing to join us. Okay, starting at the legal end, would you please um introduce yourself? >> Yeah, sure. Hi, I'm Andrea Shears, one of the uh attorneys in the general counsel's office. >> I'm Jennifer Samuels, she her. And I'm a community member um of this audit committee. Hi, I'm Caroline Zavikovski. She, her, and I'm also a community member on the audit committee. >> I'm Stephanie Englesman. I'm the zone 6 representative on the board of education. >> And I think uh Director Split, do you
003want to introduce yourself? >> Sure. Can you hear me? >> Yes. My um curse of the hour commute to get to this meeting has uh happened once again, only this time self-inflicted as I got on I5 South in downtown Portland for some reason. Um but I'm on my way back. Uh and um I am the zone one representative which is in Southwest Portland. I will be there in about 20 minutes. >> Okay. Thank you Christine. Thank you for making it online. [clears throat] >> Oh yeah. No, I mean goodness gracious I can't believe I did it. and I'm Janice Hansen, the district internal auditor. >> Okay, thank you everyone. Um, Rosanne, do we have anyone signed up for public testimony? Okay, let's get started. Um, I'd like to start by giving an update on a
004couple of things we talked about in our last meeting. First, I reached out to the chair of the board to talk about scheduling a July audit committee meeting. He said that this was discussed at an agenda setting meeting and it was decided that it would be best to have the board audit the bond auditors present to the entire board and the BAC at the board work session in July or August. I ran this idea by our internal auditor and she agrees that this is a good plan, the best way to get us all together. Secondly, on Tuesday of this week in the regular board meeting, I provided the full board and management the recommen recommendation from the audit committee to follow up on the outstanding bond audit recommendations to be done in the committees identified
005by our internal auditor. I will be coordinating with the chairs of the other committees to bring regular updates on the progress. I was also happy to hear director Split who is the board's liaison to the BAC and director Lefort the chair of the facilities committee facilities improvement and oversight committee are they're both are coordinating with each other to work on the recommendations that were assigned to both the BAC and the facilities committee. Um, also director Split, you mentioned, and we didn't follow up on it, about working together on the BAC charter. Is that something you'd like to say anything about? >> Um, I'm I'm happy to. Um, the the bond accountability committee is working on an update to their charter to add in obligations related to the 2025 bond passage, and there were some recommendations
006in the bond audit about that committee's charter. So, I thought it would just be helpful for the board andor audit committee members to just have some awareness of that process unfolding and have an opportunity if there's anything in the charter per that bond audit that we want to make sure they get into the um charter before they adopt it. It just made sense to me to be able to kind of cross that item off the list of the bond audit um to-do list since they're updating the charter anyway. >> Okay. Does that do the trick, Patty or director? >> Yeah, we'll we'll figure out how to work work on that together. Good. Um, we also have a status update on the bond audit recommendations on our agenda for the January [clears throat] or February audit
007committee meeting. So, we'll get to check in on that progress then. Okay, let's see. Next on our agenda for today is to discuss any updates to the 202526 audit committee work plan and meeting agendas document. Janice, could you go over those updates with us, please? >> Yes, thank you. So, like we had in our first meeting, I provided the audit committee's work plan and proposed meeting agendas. So, you have that document. Um, if you flip to page two, I wrote completed over the agenda for our last audit committee meeting because I love to check things off my list. Um, but you'll see for the dis the November 6th meeting. Um, originally we had a little bit different of an agenda, so I just highlighted at the bottom what the changes were for today's agenda. And
008if you flip to the next page where it lists the December 4th meeting, we are planning just a slight modification to the agenda for the December 4th meeting also. So those updates are highlighted for you below as well. Those were the two changes that have been made so far to this document. So everything else is still the same as we talked about in the last meeting. >> Okay. Thank you. And I'm glad we added that other December meeting. >> Yes. So we don't have to go too long. Okay. Um I think >> I haven't heard anything new about the um see Director Pat Michelle is online. >> Yeah. Oh, Michelle, sorry. I didn't see you up there with >> with your name. Okay. Yes. Do you have anything to report? >> I we have not
009received any additional information. So, we'll be sure and send an update when we have one. >> Okay. Good. So for now, we're going to continue moving forward with the current plan for a virtual audit committee meeting at noon on December 17th. >> 17th or 18. >> Yeah. For the purpose of receiving the presentation from the external auditors, we're apply >> Is it 17 or 18? >> I think it's 17. >> I just wrote >> Oh, no. I wrote 18th in the meeting agenda. I'm so sorry. It is the 17th. Good catch. I'll get that updated. >> Oh, okay. Great. Thanks for checking. Um, don't want to lose my notes. Okay. Um, okay. Next item on the agenda. This is the important one for today. Um, this is the status of the implementation of the contracts
010audit recommendations. Thank you, Dr. Adams. really glad to have you here to talk about this. Janice, could you provide us a little background information to prepare us for this conversation? >> Thank you. So, I just wanted to provide some context and framing for the discussion. The contracts audit was completed back in 2020 and that was a really long time ago. It feels like forever at this point. Um but the purpose of the to that contract man or the purpose of the contracts audit was to focus in on management of personal service contracts related to student-f facing services directly. Although the district manages a wide variety of personal service contracts um about 400 um in total, when you narrow the scope to um personal service contracts that are for studentf facing services, that number comes way
011down. And there is only a total of 51 personal service contracts that are active um in the school district for student-f facing services. Of those 51 contracts, 23 of them are managed by Dr. Adams and his team. So I was recently asked um about the other personal service contracts that are not managed by Dr. Adams team and why it appears that Dr. Adams and his team are solely responsible for the implementation of these recommendations. And I just want to be clear that that's not the case. Um the there are other contract managers throughout the district that have a personal service contract or maybe two. However, there is not any other personal contract manager or team throughout the district that manages more than two or three. So, um when we think about um capacity, you know,
012the office of internal performance audit, we're very small. There's just the two of us. And so um given our limited capacity, we must prioritize following our follow-up efforts on the contract representing the largest portion of the district spending, which in this case is Dr. Adams team. So I just wanted to put that out there. Um it was not our intent to make it seem like um Dr. Adams team is solely responsible. there are other contracts that are out there. Um, I just wanted to explain that his team manages 75% of the total dollar amount of all district spending for studentf facing service contracts. Um, and so that is why at least from my position, we have focused in on on working and asking Dr. Adams to come to um the audit committee to provide updates.
013However, I do want to mention that um if district management would like to bring other contract managers to the audit committee to talk about how they've been implementing th those recommendations, I don't think we would say no. So, uh maybe we'll hear from other contract managers in the future. Okay, Dr. Adams goes to you. [clears throat] >> Good evening. Um, committee members, um, Chair Sullivan, it's good to see you all and be with you. Um, for those of you I haven't met, my name is Renard Adams. I come from Baltimore, Maryland. I started my career as a one-toone aid working with a student with multiple disabilities. And my first educational training was as a special educator. I was a middle school special ed teacher, um an elementary assistant principal of a comprehensive high school in
014charge of the special ed program and then a high school assistant principal um at a all special needs high school before joining Baltimore County and becoming a coordinator of special education compliance and instructional support. That was I've always been a data guy, but that was sort of my life before I was assigned to be in data offices in um my previous um experiences. And I share all that to say I'm very familiar with audits. As a special educator and someone who's in charge who was in charge of district-wide compliance, I and someone who worked in a school, I was used to auditing IEPs of students in the district and having the state come in and audit us. Um, I did want to I appreciate um Janice's explanation and clarification because we have had media outlets
015assume that this contract this contract conversation is about all PPS contracts and it is not. Um, I wasn't aware that the little over $7 million that I'm responsible for was 75% of the cost. I thought there were some higher areas in other with other chiefs, but I'll um love to see those data. Janice, um, what I do want to share before I get into sort of the back and forth with the audit recommendations is I appreciate this audit occurred before I arrived. I arrived in 2021 and I've had these contracts under my purview for about two years now. And in that time, I feel like we've made tremendous progress um because we've checked a lot of things off of the list in the two years that I've been overseeing these contracts. One of the ongoing
016concerns from staff members and from community partners is that the audit only included them. These are all black and brown facing organizations and there's I wanted to just share and name an ongoing concern that it appears as if we might be PPS might be holding community partners to a higher standard and a more intense amount of scrutiny than we do other contractors or sets of contractors. Um, I don't know, Janice, if anyone was going to willingly come and testify here except me. I'm not I'm I'm good with it, but I don't know that we'll have other um particular contract managers signing up for that. So with um that said, I wanted to just walk through um if that's possible the um current recommendations that have not been marked as met and just respond to um
017>> the office of internal performance audit comments. Um we're working hard to close this audit. Um hey, welcome Director Split. Um, and we want it to be closed because we want to be about the business of actually um, spending our time observing the services that our students are receiving in schools and not necessarily in all of the paperwork. Um so um the first recommendation talked about performance reviews and um we did perform performance reviews and the board received that memo and the most recent comments from the office of internal audit talked about the performance view reviews not occurring before contracts were reawwarded and I want to explain why that was one I want to explain that our performance review of contracts is not limited to just the deliverables in the contract what we mostly like
018to focus on is whether or not partners are actually delivering on the services they say they're going to deliver on for students across our district. And we do that by site unannounced site visits and communications with our building principles. And you'll notice if you had a chance to read through the memo without um before clicking all the many links um we have three we had three uh partners that we worked with last year that were not working with this year. One of those partners uh unfortunately went out of business. The other two were non-renewed based on the fact that they were not able to start service provision for students. And so there were no services provided. we did not make payment and we decided not to continue working with those partners again. Um so we
019will continue to do those performance reviews. Two things about those. Um when we look at some of the outcome data that partners list, it really takes us until we have all of our end ofear data to really see if they met those targets and standards. And then what also happened, we have greatly reduced capacity. One of the departments that we basically eliminated through last budget cycle was the department of research and evaluation. So the racial equity and social justice team who are not researchers um have a certain amount of capacity to analyze those goals. And so what we're going to do moving forward is make sure that the performance outcomes that partners put in their scopes of work are within our evaluative capacity to make sure there isn't any sort of disconnect there. If you
020have questions at any time, just stop me because you know I'll steamroll and just talk. So you talked about put in your valued capacity. So what's that actually mean? >> It it means we will still have partners that have outcomes in their scopes of work in their contracts and we'll make sure for example here's an example I'd like to give um because the office actually identified this. A partner may have had third grade reading increasing third grade reading as a goal. Well, as you know, we don't get those data until well after the school year is over. And um my the resj unlike the research team does not have individual access to individual student data in that way. So it became an unfortunate casualty of our reductions and as it hit against our current capacity
021to examine goals like that that were related to OSAs. Uh yeah, that was a question [clears throat] I had because in reading through some of the narratives, it sounded like a frustration with some of the people was not being able to get that information. >> Yes. >> And is there any way to have it so that they can >> Well, I'm I'm not we aren't allowed as a district to release the information until it until the embargo lifts. >> Um which is a challenge. You know, the the data are quite delayed. So what we will do um the other this hits on another topic, but what we will do is uh continue to align to the district continuous improvement plan as our schools do because we want everybody to sort of be rowing in the
022same direction. And then we can think about um what are some leading indicators of those areas of the district improvement plan just as we talked about when we had the world cafe experience with the board. What are those leading indicators we can look at to see that um progress is being made so we can look at that? >> Yeah. And share that with Yeah. >> Yes. >> My button. Um, just quickly, I was thinking about what you said about you don't have the data in time, but it seems that the first time would be delayed because you don't have the data in time and then every other year would be delayed, but it would still be on a rolling basis that we would see the right like what am I I I'm so new to
023this. I still haven't quite figured out >> what this is. Honestly, I'm sure I'm trying to figure this all out. Um, but if I'm thinking about it just in my own world, the first time you do something, even if it's in October and the the contract renewal was June, but you didn't get the data till October, so you do it in October, it's delayed for that first time, but then the next year you would have the same, you know, you would have on a cycle that's always two months after the contract renewal, but for the next year's contract renewal, you will have seen that maybe either things are slipping or things are going the way. >> We can we can certainly look into that. One of the things I also wanted to clarify is our
024partners have aligned to our district plan um per our direction. However, their role isn't typically academically focused. It's culturally specific mentorship support and wraparound services for students which hopefully um which are the strategies that they employ are um on our website and those are all researchbased and evidencebacked. And hopefully those strategies do things like allow students to see and interact with people who look like them and which then encourages them to attend more and be in class more. And so we're looking at those types of things. Um and certainly uh take your um comments, Director Englesman, into cons we'll take your comments into consideration when we think about how we may um give the board timely information and have it be thorough and complete because our effort was try to get the report here as
025soon as possible because we typically had provided the memo in June when the contracts came. However, the contracts were approved for four years and so we just had a little bit of a things weren't aligned in the way that I would most hope that they would be this for this cycle. >> I'll weigh in on this too if that's Oh, you please go ahead. >> Oh, thank you. Uh I had two quick things. I just um uh I was concerned to hear Dr. Adams, your comment in there about um the organizations who are primarily black and brown organizations, not being curious or not or feeling like they were potentially targeted. And um I'm going to respectfully say I'm just going to give a very quick personal uh I had been running an organization who was
026not under your department and I personally did participate in the um in the review of contracts in 2020 with an organization that that doesn't uh fall under your purview as a contract manager. So I wonder if there's some repair that needs to happen there around making sure that everybody that it is transparent and everybody does know what went what went on. Secondly, also as a person who's been running running nonprofits, I feel like I have a lot of firsthand experience with people asking uh for us to report data on which we have no over which we have no control. And I wonder if um if there are going to be changes or modifications going forward, if this is something where we can look at um particularly working with the organizations around uh both the organizations
027and your department around um what data can we provide, when can we provide it? So it's calendared out and nobody's ever surprised about when or when we do or don't have it. and what is realistically authentic data that we can use to report on um that is meaningful because we we know these organizations are doing good work. How can we point to it? What can we point to to um um pro prove it might be the word I'm looking for. What what can we point to to ensure that we all understand the good work? >> Sure. I I appreciate that. Um, Miss Samuels, I that that has been the purpose of the narrative port narrative reports from the partners, but we will certainly I appreciate um the idea of having sort of a data calendar
028because I like that and um providing that to partners and having conversations back and forth with them. Um my I'll be completely transparent. People lose their jobs in school systems because they harm children and because they mess with money. I am committed that neither of those things will happen on my watch. And so my focus has really been are the dollars we're spending being spent on the services that students are actually being provided in buildings before, during, or after school because I think that is my utmost responsibility to be a fiscal steward of taxpayer dollars. So I just wanted to put that out there. Um happy to do all of these little adjustments as well. Um, but really want to have the team focused on making sure students are getting what they need. Um, for
029if I could move along for recommendation 3B, there was a recommendation to use um, a board cover memo that I believe was created by the purchasing and contracting office. um we modified we created something that was modified off of that and we feel like it still meets the need of having a performance review and so we believe that we've actually met this recommendation just not in the exact dotting of the t dotting of an eye or crossing of the tea that the office of internal performance a audit suggested and I don't expect us to resolve everything during this conversation ation. I just wanted to there are things that um I wanted to share with you all and um the office and I can continue to meet back and forth and come to resolution. It's quite
030um normal for auditors and audites audites to go back and forth to find resolution. I I agree that that is often the case and I don't want to make a big deal about the formatting of the template per se. Um, but I do want to mention like one of the key areas that I thought was missing from the performance review was a description of any changes that were made to the services provided um, based off of the review and evaluation. So, I do think that there really does need to be a spot in there where you you are prompted um as part of the template to enter that information to say, "Wait, so now we've assessed the goals and this is the progress we made and these are the tweaks that we think are relevant
031for this coming year um to do even better." >> I I appreciate that. And um as I mentioned our evaluative process is more is mostly about are they performing the services the the partners have their service packages that are aligned with our strategies and we try to ensure that they're doing that and that is mostly how we evaluate them. >> Can I ask a question about that? Um, one of the things that I've been thinking about in looking at these materials is like especially on these ones where we're having issues with getting the data in order to measure, right? Because if you look at the memo that you sent, there's um goals where we have like report backs and then goals where there's just kind of silence on how we did, right? Because as you
032put it, the data wasn't there. We talked about that. But to me, a smart goal, the word at the M is measurable as everybody knows. And um it strikes me as like we have smart goals that aren't actually measurable. So they're what SART goals and that's not quite right. So I I love the idea of talking to the the nonprofits about what data we can get. But then you just said something that just leaped out to me, which is like you're you're wanting to evaluate them on something a little different than the smart goals, which is are they actually doing the things that you want them to do, the scope of work or process or whatever you wanted to call it. Why not just make the smart goals align with what you want to evaluate
033them on instead of what you think? >> That's a great question. So >> um the >> the equivalent of what you're suggesting would be um to write a school improvement plan that didn't talk about necessarily student-based outcomes, but talked about what principles and teachers might be doing. Right? And I think we want to what I mean we could certainly think about that but we tried to get our partners and this was our first year because that's our first district continuous improvement plan to get them to align with that. And so I'm not saying that there isn't room for improvement. Um I'm just acknowledging that we believe that having everybody attempting to row in the same direction is really important. >> Yeah. I don't disagree at all, but in the same in the same conversation, you've
034mentioned that most of these aren't meant to be like academic programs. The academic outcomes aren't necessarily what you're looking for out of these programs is what I heard you say before, but then we're judging them in some cases have smart goals that are directly tied to academic performance. And so, I'm just saying like I just feel like we should figure out what do what do you and your team want to evaluate them on and then have the goals match that. Um, and the I totally love the connections to the district continuous improvement plan. Um, that's great, but I don't want us to like be so tied to that that we're >> like overengineering a system where we continue to have these blanks where we're not having the data to actually evaluate. Um, so I just
035I just hope that you'll consider just what is it that you actually expect these people to do and have that be the goal. And I also think there are some, you know, there are some data that are out there like the I just it's literally sitting in my inbox for me to complete, but I still have issues with the name of the climate survey, the student success survey, >> successful school survey. God, one of these days. >> Um, which I am like so excit like I can't believe I can't remember the name because I like think that that's to me like so important. Like to me that's that's it. especially with these kind of programs like sense of belonging for these students is what we're trying to get um here so that then they want
036to come to school and care and stay and so um so anyway I think I just urge your flexibility in um what maybe existed in the past with the smart goals and what we can match up with what we can do given existing resources what actually want to track on what data we actually have um yeah I just want to open the door to to that being that aim so that way we're not making extra work or >> making you look at something that doesn't seem important to you or your team. >> Understood and duly noted. I appreciate that. >> And following up on that, I did notice a lot of the narratives, not well at least a few said they really wanted access help with their surveys. I don't know who want >> Yeah,
037we can follow up on that because um the team is the team's a really responsive team. >> Yeah. and we do have um some staff that could help them with their survey construction. And so I wasn't aware until I read the report that we had this ongoing need, but now we know we can address that. >> Yeah. And maybe they should have told you. And also uh following up to that you want um partners to have goals that are written as a theory of action and that maybe those goals could have certain percentages. >> Yes, they can. >> And that that might help us to understand. >> Yes, absolutely. >> And how give an example of that. How would that >> um if we provide an affinity group to a group a culturally specific affinity
038group opportunity for groups of students who are Latino and Latinx Latina um they will report an an increased sense of belonging in school >> and we could then as measured by students we could we could have a survey we could do a focus group there's lots of way we lots of ways we can get at that information and we can help partners do that. >> Okay. And I know it's it's difficult to measure all these things. >> Yes. >> This isn't a math score, a math test. So I know it's hard. >> Yeah. Sorry. >> I'm sure this happens. Um but do you at what point when you're seeing some of these percentages percentages in here? Um is there followup to try to figure out why? Um, for an example, a few of them I
039looked at had sort of a higher behavioral overview uptick from the previous year, [clears throat] which to me suggests less about the student population and more about maybe the external things like what's going on externally that maybe is driving something in the school or is there >> I is there a problem person in the school that needs to be looked at like I mean so does these numbers to you lead to addition We absolutely have conversations with partners [clears throat] all throughout the year and then we bring our partners together uh >> let me not misspeak three times a year first quarter, second quarter and third quarter for conversations, information sharing, best practice strategies um and teaching and learning. So we do both of those things. >> Okay. Thanks. I noticed there was a meeting
040I think last week and it didn't happen or cancelled. >> Sure. That was the racial equity and social justice citizens advisory committee. >> Okay. >> That committee um requires a quorum. It's a 15 member committee. When we were able to confirm attendance, we had three or four people attending. And the business that the committee needed to undertake at the first meeting was electing a chair and a co-chair. So, we are working feverishly to reschedule that meeting. Would that be good meetings for us to observe just to see what they discuss? >> It it could be. Yes. What we um what what the if I'm remembering correctly, what the charter of that committee says is they will bring forward recommendations to the board. Um so we'll work on that, but we need to [laughter] get the
041the the minimum amount of people in this space so we can actually conduct committee business. Okay. Um there was a re there's a recommendation around contract man there's several recommendations that touch on contract management. Um I want to be clear I was the contract manager when I first assumed these contracts because I wanted that kind of oversight over them. Um I am I am not and have not been the contract manager uh for some time. So I have a staff member that's a business operations analyst that is the contract manager. That staff member has a supervisor or director of the department and contract management conversations occur during their one-on-one meetings and with my one-on-one meeting with my director. So those are things that are happening and we worked really hard because there was also a
042a prior uh recommendation about um con there my understanding was when the audit was completed in 2020 there was someone who was identified as a contract manager who expressed they didn't know that um so there um which was surprising to me when I when Janice and I had that conversation and so what we have done and what I've worked with HR on is there's contract contract management language described in the job descriptions of um my person who is the contract manager and it is um she is aware that she's the contract manager and she's doing that work because I see the flow of the approvals and the emails and things like that. Um there was also uh uh a notation that the co the performance reviews weren't uploaded in cobblestone. We were always intending to
043do that. We wanted the board to have the information before we uploaded it into the system and those will all be uploaded by the 15th of this month. There is um this one was a little curious to us recommendation four and I'll explain why. Um when we uh brought forth the contract information and memo in June of 2024, each of the contracts contain smart goals. And I I know you remember that because I remember specifically Director Sullivan, you and um Chair Chair Wong commenting on that. and I and the team remain a little confused about why we weren't marked as having met that standard when we met it. I get that we've made a shift towards the district continuous improvement plan. Um and that some of the goals as theories actions don't have percentages although
044they're still written in a sort of if then this will happen. And so um we will ensure that there are percentages in the goals written as a theory of action or their counts whichever one seems more appropriate. But we um are just a little confused that you know having done that um and met this in 2024 why it wasn't marked as such. >> I have a question. So, if you've complete it, but then the next time you don't do it, is it still completed? >> I'm not used to an audit that continues to look at the same thing over and over again after it's been met. >> so you feel like once it's done, >> and it wasn't it wasn't an intended backslide. We >> wanted to align with the theory of action and the
045district continuous improvement plan. We did not have intention to not have smart goals or goals that didn't have every component of a smart goal because the goals written of theories act as theories of actions can have that. Um it's just it was the same as the I believe the um timing of communication with the last audit committee meeting where everything looked as though it was outstanding and these were things that we were pushing through the process to get to the board's attention. Right. So I again just we can talk offline about it or or or online but I just the team is just really curious about that. Um but we are that being said we will make sure that um for next cycle goals written as a theory of action will contain data that we
046can provide or we can access and we'll have the percentages in all of them to make sure that this is met. I'm guess is a misunderstanding about we're used to smart goals and then the area of action was was hard to relate >> sure >> as clearly to it. >> I I I accept that and will own that. So, um I could have in hindsight, which is always 2020, sometimes it's 2010, [laughter] you know, um I could have provided information to let you know something different was coming that had a different format so that there it wouldn't have been possibly the shock that it was. So, I'll I'll I'm able to able and willing to own something when I do it. Um >> I'm going to ask a question this direction really quickly. Does it
047because I can't remember off the top of my head. Does the district's continuous improvement plan include smart goals? >> Yes, it does. >> So there are goals with which specifically with which to align >> there are >> smart goals. >> Mhm. There are only in four areas however right. So in for the district continuous improvement plan there's attendance um literacy mathematics and nth grade success. So, a um partner may be working on um strategies that may improve attendance and they also may be work on strategies that might improve sense of belonging which isn't captured in the district plan, but it's still something very important. If I may, I think um for me, this is obviously brand new, but I like the the smart goals and the percentages because even if >> a plan says raise
048attendance by 50% and we raise it by 4%. We're still seeing that like Dr. Armstrong was talking about at the most recent board meeting, >> the importance of seeing the upward trajectory is not to change um like really important board goals, not to lessen them because we didn't hit it right away, right? >> But to keep those high because we have such high hopes and high achievement goals for all of our students, keep them high and then just keep sort of plotting along. And so for me, at least seeing the percentages and seeing the smart goals linked to everything is important because both when I'm seeing numbers that aren't going the right direction to be able to ask, is your team investigating why they're not going the right direction? because it could unearth other issues
049that need to be addressed that are outside of this but still important to this. But then also seeing things going in the right direction is the positive work we want to see happening and it may not it may not overnight get everything we hope for, but it we're still seeing it go. So >> sure, I I understand that the board has strong interest in this and we'll make it happen. You know, this was our first time with the shift and it obviously didn't happen in all the cases. So, you know, again, we'll own that. um we pointed partners to our plan, we asked them to do something and um it didn't always happen in each case. So again, I'll own that. Um there is um the recommendation number eight around deliverables. So, we recognize this
050issue um with um having the quarterly data submitted and the reports sent in um and we'll just continue to work on it is is what I can tell you. Um you know, it is um it's important and we want to make sure it happens. I want to do a quick jump back to the recommendation for um Dr. Adams is what does is your team getting what they need to assess whether the partners are I don't know exactly what terminology we're using but does is your team getting the data and information that they need to assess whether partners are meeting the basic standards of the work that they're being contracted for? Do you have what you need? >> That's a great question. I so appreciate it. [laughter] Some days I feel like I do and some
051days I feel like I don't if I'm being honest. Um what I would say is we perform announced and unannounced visits and observations and have consistent communication with principles to observe and make sure partners are um providing services to students. Um so in that vein we go out and get the information that we need and we have capacity to do that. Although um the contracts I we have I have staff members for whom this contractual work isn't the oversight isn't their work but they help us with you know it's sort of like when we do school visits it's it can be an all hands-on deck experience from the academic team in the central office. It doesn't matter you know I'll go in an English classroom I'll go in a math classroom. I'm not in charge
052of English and I'm not in charge of math but I was a teacher. Um, so I believe we have what we need and I sometimes wonder if our partners have what they need in order to provide us with some of the information. And so that's a question that we can um that's a conversation we can have with partners. Um we have a quarterly meeting coming up pretty soon. Um, and we can certainly ask that of them to see how we can improve and partner better to make sure everybody's getting what they need and you all have clear communication um, and information around how things are going. >> Yeah, I did get a feeling I think I said it already that sometimes they don't know where to get what they need. >> So that would be
053helpful. I mean it and [laughter] it it takes a lot to work with us. Our partners have to go through all the work to get badges. They get all their staff have back as as it should be. Their staff have background checks. They have to complete trainings in our pepper system just like an annual every year. you do, you know, I get this ping in on July 1st with my laundry list of trainings that I need to do before October 31st or else I won't be able to get any work done. >> And if we don't get them done, it actually reminds us like if I didn't do one on time, it pings you. >> It reminds you and your supervisor when they're when when they're past due. Um so um we are working we
054work really hard to get them set up with BPS emails and all those things. Um and we can certainly I'm about continuous improvement so we can get better at that. Not a problem. Um there are a couple other um recommendations around contract management. Um again, and I just note that those responsibilities and the workflow um to make sure that our contracts are managed properly are covered in um one-to-one meetings between supervisors and staff. And so I'm not sure because those agendas sometimes have sensitive information on them um beyond, you know, an update on contract implementation. I'm not sure how to meet this audit recommendation other than to say this is actually happening. Um, so again, Miss Hansen and I can go back and forth and come to some kind of resolution on on those items
055that touch. Can I ask can I ask a question because looking at just the recommendation itself >> it and I see the request is provide a narrative describing but so forgive me >> but I think that it says like to achieve this we consider updating position descriptions which you've said is done and then include an evaluation of the performance of these responsibilities and annual performance evaluations. I'm a state employee and we have like a really prescriptive annual performance evaluation that is like based on our job description and annual work goals. I'm assuming PPS has something similar. >> Yes. >> So I feel like if it doesn't seem like you necessarily need we necessarily need to get in the weeds on the one- on-one meetings, but more just having it just saying, "Yeah, we went ahead
056and their annual performance evaluation now includes >> this." Am I missing something? I feel like I feel like and and that has happened um in in terms of >> um >> describing and providing feedback about employee performance for contract management in um their annual evaluations has occurred. >> Great. >> So I I just want to maybe I need to type that up for Janice or whatever I can do. >> We have to confirm we have to see evidence to confirm the implementation. So I would just need to take a look at those things >> like the form or whatever. >> I I don't have the form. the form um the form does not have a line that says contract management and then comments. Um I can share the form we have I've shared the position
057descriptions that have the contract responsibilities in them. >> Is it hard to add it to the form? >> I I would have to talk to our our HR team. >> Okay. >> Because our our um evaluation is not set up um in the way you describe, right? Um, it is not for central office staff. It is not a list of competencies with comments for each. >> Okay. >> Is is what I would say >> like less linked to like as mine's linked to my job description and goals for the year. >> Yours is more just everybody gets the same form kind of thing. >> Yes. >> Okay. >> That's my understanding. >> That is tricky. Maybe we could. >> I'm on a plate here also. >> And you know, director Sullivan does my performance review
058each year and so I'm pretty sure we all start out with the same template, which I mean is pretty blank in the template. Um, so what Patty and I have done is taken my position description and each of the key responsibilities and put that into there and done evaluation on each of those. Um so like I at least in from my experience there's opportunity to customize it um depending but I'm not familiar with how it works for other classes of employees. I'm unrepresented. Maybe that makes it different. So I can update my understanding um on the other uh classifications to see if there's similarities or if it it is very um structured. >> My understanding it was the same as mine though. >> It is it is the same. My my team are also unrepresented
059employees >> um as am I um obviously. So it is >> as am I. >> Yes. Um it is uh at the moment it is manager discretion what they put into the evaluation and what they monitor. We don't have at the moment um structured competencies again that come from our department of um human resources for example. Uh so uh I can certainly continue I I guess you know Janice and I Janice we can have ongoing conversation about what would it take because even if I put something in someone's evaluation I couldn't necessarily share that with the office of audit right so I just have to think about how do we have this closed because my people are managing the contracts money is not being wasted they know they're managing the contracts we're talking about the
060contract management so I just need to know what we need to do in order to close these couple that have to do with um the contract management. >> I'm committed to meeting with you very soon to be much more prescriptive. Thank you for that feedback. >> I have a just question that came up, pretty simple one. Um, when I'm reading the reports, I'm not sure when I um read about attendance and disciplinary if that's like for the whole time they're in school or just for meeting with the group they're in. That's like >> Oh, it's it's usually um for the time they're in school. So for if it if it's attendance, it's not attendance at say the group the afterchool group. >> Correct. It's it's attendance at school >> in their whole school. Okay. >>
061All right. Um is there any any more? Yes. Sorry. >> I have a quick comment and question. um just say for me what is most important is you know making sure that the issues that came up in the original audit are being addressed and I don't think it needs to be you know exactly specific one way or another but it sounds like maybe there's some disagreement on whether some of these are meeting those issues or not which you know we can all talk about but um you know I think we all want it to be realistic and feasible right >> um and I had a question about um you said you do site visits which is great. Uh do you have the capacity to like do site visits for all of the >> We do.
062That's amazing. >> We do visit all of the partners. >> Yeah. And that is that something that you have documentation of that you can use towards some of these recommendations perhaps. >> I I know we keep a we've kept records of um when we visited and what we saw. I just >> I haven't been deeply involved in the process, but I do know what happens because I sometimes get the invitations to tag along. >> Okay. >> So, I I will look into that for sure. >> Thank you. >> Absolutely. >> Is there anything else? >> I was going to bounce off. I think it was something that you'd said um about making sure that you mark down or somehow put down what worked and didn't work and analyze why. >> Mhm. >> Because you could
063learn other people could learn from that too. >> Yes. Um and I think is there a place for that in >> we can make a place for it. >> Yeah. >> You know this is this is when I mentioned earlier like we modified the performance review template we can or in the partner narrative that we want from them we can ask them to describe what worked, what were some barriers, what were some successes. We can certainly >> Yes. >> Yeah. because especially when it's hard to have a certain percent of growth or not in the work that's being done that would be really helpful. >> Sure. I mean it's just like I think I think of our partners as overall supporting student and school success. Right. And so what I know from being in a
064turnaround school as a elementary assistant principal, I mean the school has been taken over by the state and then I was told to be there. [laughter] um as happens in those situations in other states um people are working really hard and the data don't always move even though you're working really really hard and really really smartly. So um we will continue to work with the partners on the narrative format to see if we can gather information around what's feeling really successful and what's feeling like barriers so we can look at patterns. Um, I'll share just because I I thought it was cool. Um, going into last year when I still had the research and evaluation team, I asked them if there was a way to create like a Yelp review review for every partner. So
065they created something and principles were able to rate the partners and leave little reviews and we used that as a part of thinking about whether or not there's another data point when we thought about whether we wanted to renew or not because we do want um there to be successful relationships between our building leaders and their staffs and the and the community based organizations and their teams. Um and so I just wanted to share that because I thought it was pretty cool. That's a really cool idea. Is have you thought about I know it's a brand new idea. Very cool idea. Having the students also do I mean all these kids love this online stuff. >> That is true. I [laughter] hadn't thought I hadn't thought about that. >> Hadn't thought about that. But um
066>> maybe it's too much information but um it's a really cool idea. Thank you. >> Thank you. I appreciate as always the opportunity to come and talk about the work that we're doing as I shared with our data. I think some really great things are happening here um in in the district and I'm just still so happy to be here four years later. Um missing the sunshine a little bit but still happy to be here um and and doing good work on behalf of our students and staff. So, thank you all. You all have a great evening. Can >> I say one one quick thing? >> Sure. >> Um Oh, go ahead. You want to say that? When when I started in July, I just sort of met everybody and I didn't really have an
067opportunity to understand people's specific backgrounds. I knew everybody here must have an incredible background because they're here and they're doing the work. But tonight was the first time I heard of your extraordinarily long and noble background >> and my father-in-law spent until he retired his whole career in a special ed classroom in Illinois. And so I just wanted to say I'm impressed and very grateful. So >> thank you. I actually thought I was going to spend my entire career in special ed, too. [laughter] And then a superintendent I worked for decided otherwise and here I am. So I'm glad >> serve at the pleasure. This is by I'm just grateful. >> Yeah. And I just wanted to say thank you and again I hope the spirit as you two move forward is how do we
068make sure that the process we're developing is one that doesn't create extra work but just becomes something that fits what's already happening and holds us and our partners accountable on behalf of our kids. Yes. So I do encourage you to find if there's that moment where you're thinking this is just extra work like then okay what do we do to make it so that we're actually just holding accountable to the work that's happening. I think that to me is um something I hope that you too will keep in mind. >> Absolutely. working on this because um we're all you made a comment earlier about wanting to be in this for the kids and um you know that you're always going to do the right thing. Um and I super appreciate that, but you still have
069to have processes to back that up. That's just the way it goes with public service. So, thank you so much for the two of you for working on it. We really appreciate it. >> All righty. Next time we see you, these are all going to be green. Actually, we're not even have to talk about it. [laughter] >> Thank you, Dr. M. Um, does anyone else Well, before he leaves, anybody else have something to ask [laughter] Dr. Okay, I think we've we've done it. All right. Um, what else? One last thing I want to confirm that our next audit committee meeting is scheduled for December 4th at 5:00 p.m. Janice let me know that Jennifer will not be able to attend. Sorry for that. >> Be attending virtually. >> And what? >> And I have to
070attend virtually. And you will be virtual but virtual is fine. >> I will be >> not as fun but um so um yeah but yeah Janice also Janice tells me everything. Janice says they have a meeting so that uh um Jennifer will know what's going on. And I want to thank you all. Well, that was a great conversation and it's a hard one because we know that Dr. Adams is doing the very best he can do, but we want it to show so that everybody knows that and I really appreciate everybody's comments. Very well done. And I'll get a meeting scheduled with Evan. So, um, we'll do some onboarding so he'll be fully prepared for our next one to chime in. >> Quality control. Great. [laughter] >> Okay, this meeting is adjourned.