Virginia Board of Education Work Session 3/26/25

001e e e morning um I would like to call this meeting to order apparently we were having um some technical difficulties on um the audio side of of this recording video YouTube um link so I would like to call the meeting to order and offer uh superintendent glixon an opportunity to um say a few words hi good morning uh it is an honor and a pleasure to be appointed by Governor yunan to serve as the superintendent of Public Instruction here in the Commonwealth uh for those of you I have not yet met I'm very excited to work um with our students families teachers and Educators uh and this is truly one of the best Boards of Education in the country and so we're really looking forward to keeping students at the center of our work

002and I will give further remarks tomorrow uh at the business meeting and want to turn back uh to the president so we can continue with today's agenda thank you uh with that I would like to ask Mr uh Mr Hansen if he would please move us into Clos session thank you madam president I moved at the Virginia Board of Education I move that the Board of Education convene in a closed meeting for the following purposes discussion or consideration by the Board of Education of information relating to the denial suspension or revocation of teacher licenses subject to the exclusion in subdivision 11 of section 2.2 3705 point2 pursuant to section 2.2 3711 A40 of the code of Virginia and consultation with Council regarding specific legal matters requiring the provision of legal advice pursuant to Virginia code

003section 2.2 3711 A8 of the code of Virginia and the legal council to the board of education and staff whose presence will Aid in the board's consideration of this matter participate in the closed meeting is there a second second um Miss Kilgore hi Mr Hansen hi Mr Rotherham hi Miss Dr Northern hi uh Miss Holton I and chair votes I uh we will now move into closed session thank you e e e is it mine we convene today in a closed meeting pursuant to a recorded vote on the motion above and in accordance with the Freedom of Information Act section 2.2 3712 of the code of Virginia requires that the board certify that to the best of each member's knowledge only public business matters lawfully exempt from open meeting requirements under this chapter and only

004such public business matters as were identified in the motion by which the closed meeting was convened were heard discussed or considered any member who believes there was a departure from these requirements shall so State prior to the vote indicating the substance of the departure that in his or her judgment has taken place the statement of the departure will be recorded in the minutes Miss Kilgore H Mr Hansen hi Mr rham hi uh Miss mcferson hi Dr Northern hi is Miss Ashton still with us hi can you hear me yes gotcha thank you Miss Holton I and chair votes I okay um the chair will entertain a motion to issue a license in case I and chair votes I uh the chair will entertain a motion to issue a license in case 202- mov Miss Kilgore

005H Mr Hansen hi Mr Rotherham I miss mcferson hi Dr Northern I miss Ashton hi Miss Holton I and chair votes I um the chair will entertain a motion to issue a license in case 202- 06 move second Miss Kilgore hi Mr Hansen hi Mr Rotherham hi Miss mcferson hi Dr Northern hi Miss Ashton hi Miss Holton hi and chair votes I the chair will entertain a motion to issue a license in case 2025-the so so moved okay Miss Kilgore Mr Hansen hi Mr Rotherham hi Miss mcferson hi Dr Northern hi Miss Ashton hi Miss Holton I and chair votes I um the chair will entertain a motion to deny a license to Wendy Lee weer here miss Kilgore H Mr Hansen hi Mr Rotherham Miss mcferson Dr Northern hi Miss Ashton hi Miss Holton

006I and chair votes I the chair will entertain a motion to revoke the license of Alyssa Michel outley is there a second okay Miss Kilgore hi Mr Hansen hi Mr rham hi Miss mcferson hi Dr Northern hi Miss Ashton Miss Holton hi and chair votes I okay thank you so much um does anybody need a quick break or do you want to move ahead into our um first item on the agenda yeah five minutes okay yes so we'll take a five minute break here please thank you e e e we are moving to item C first um Dr Myers is here to present um item C to us so Dr Meyers the floor is yours right thank you for being with us today yeah absolutely thank you um good morning president crey and members of

007the board I'm here to talk to you about the waivers that have been granted to several School divisions due to an unprecedented year of natural disasters and weather throughout Virginia and especially for the Western and Southwestern portions of the state in fact the only time people on the VOE data team that have been around for a while can remember this happening was back in 2011 when the earthquakes hit Louisa and surrounding counties um for reference there was one wave due to a tragedy at the end of the school year in 2022 23 at one school and in 2324 there were none so it gives you an idea of of the amount of waivers that we're used to getting um I also want to start out by pointing out um that on the the majority of

008the schools you're going to see in this presentation um in the past during adverse times have really shown that um that they were trying to get kids back into school and and we saw that during the covid pandemic uh uh where those schools were the first to return back to school so I do think it's important that their past um motives getting back into school uh can play into this presentation um on slide two you will see uh I think it may be helpful to also ground this conversation in Virginia code that states that a school must meet a minimum of 180 teaching days or 990 teaching hours for each School the code also states that if due to a state of emergency severe weather or other emergency situations that the division May seek a

009waiver from the board of education for the 180 or 990 hours provided they have exhausted all avenues to make up missed instructional time um the code also states that the board of education has delegated its authority to review and approve the length of school term to the superintendent of Public Instruction the code goes on to say that to receive a waiver that the School Division must submit a request uh that request must be signed by the superintendent and also the board chair of the local division they must also include in that request um how many hours they're trying to get waved and any evidence that they have as to why um which often includes pictures um news things that have come out in the news that type of thing um moving to slide three you

010will find that on on September 25th uh Governor yunan issued EO 37 declaring a State of Emergency due to Hurricane Helen as we know some divisions in Southwest Virginia experiened similar catastrophic damage as we saw in W Western North North Carolina like in Asheville I think everybody remember those remembers those pictures these seven divisions were hit hard as seen on slide four um and as a result formerly requested a waiver to the 180-day 900 90h hour requirements due to not being able to run buses and open schools for extended periods of time it's important to note that um these divisions saw catastrophic Devastation to infrastructure that included roads Bridges water systems and the power grid on Slide Five um you will also see that winter weather has played into this this year we have had

011uh unprecedent not just the snow but it was really the cold weather that followed the snow that didn't allow the snow to melt that kept this problematic for many divisions and in some places it was the same divisions that had problems with hurricane Helen so you will see on Slide Five that Governor yunan again declared a state of emergency on January 3rd and February 10th because of snow and extended periods of cold weather um these divisions also requested weather waivers for their 180 day 990 hour requirement on slide six the vdoe has traditionally collected at the beginning of the school year how many days and hours that each school is in session the vdoe has also collected during that time when a School Division starts when they have winter break spring break as well as

012the end of the year I think we also collect when they have first and second semester um so that's what we've traditionally collected and um quite frankly it's it's done its job we've had so few a waivers that we haven't needed to know day in and day out exactly which schools we also don't have an automated data system that collects that um at this current time but um due to that and knowing how many of these waivers superintendents began reaching out about this the team um went on to scour the internet to find when um schools were open and closed in January and February and came up with this heat map and um so while it's not exact uh we we do believe it was an exhaustive search to find out when these schools were

013open and closed and this is just to give you an idea um it's also important to note here that it was during this time that the governor's transformation office was also working to coordinate resources across the state um to help these School divisions and uh actually superintendent glixon was was was involved with that and knew quite a bit about that so I don't know if if you be able to share a little bit more about those actions I just want to recognize uh vdm as one of the critical partners and VT um being able to have a daily heat map as to which schools were closed you could quickly see the regional impact and we were better able to Target resources and supports and so one of the goals here was how do we help

014raise awareness that one of the goals of learning where there is an extended school closure allowed us to better collaborate across state agencies and in partnership with local communities to make sure that they had the supports that they needed and this is sort of a cumulative map of what those daily heat Maps looked like so that we could better see where the impact was as you can see it very much varied by regions obviously later in February um uh on the Eastern Shore and Virginia Beach area was heavily hit with snow as you recall from those uh wonderful pictures uh but also different Pockets throughout those months had different um extenuating winter conditions thank you um just moving forward to slide seven um these are these are pictures these are evidence of uh what these

015areas look like um I also wanted to just take a moment I think many of you know I served as a division superintendent for a number of years in Virginia and I will just tell you and you you all can probably imagine this um weather was the one of the most challenging things that you just could never win on quite honestly and I found that usually if I tried to bring students into school because my original career started in Colorado we didn't have snow days so this was a new thing for me and when I tried to bring students into school um when you know I felt like it was safe but you know there was a little snow I got a lot of push back a lot of community push back on that kind

016of thing but if I kept the kids home I never really got much push back which was always a little a little strange to me but it was the reality of it and it was a lot of 3:30 in the morning um wake up calls trying to make these calls so when you have this cold weather uh in Southwest Virginia we had some dirt roads in the county where I was there were two machines Statewide that could be used to clear some of the roads on um on these uh is what I was told at the time and so you had to wait your turn and it was really hard sometimes to get buses down some of these communities 10 20 30% of their roads that they go down um may not even be paved

017so it it is really challenging I will tell you just for a Future Part in the last slide on this presentation the second most challenging thing that I dealt with in this world was air quality and we'll talk about that towards towards the end here um you will see on slide eight Miss hton can I just go back and ask question back to the the the heat map slide I'm just curious uh what's going on in I guess that's gin patan Amilia uh flu the red the red highlight on the Central Virginia piece I'm happy to answer that uh so one of the things in as we discussed this presentation for the board is that there is a significant difference between you know from County to County City to City on whether or not those

018localities are responsible for their own roads so for example you have henrio County that is responsible for their own roads where you cross the line into Guin County and vot is in charge of that um vat does a a very good job of clearing the main roads um but based on how um the School Division operates their bus bus system you know where they don't have stops like henrio County or the city of Richmond does um they essentially go from driveway to drive driveway on the rural um roads that are often the last to be cleared um and that's that's generally what the difference is is there um and so really you know the purpose for this presentation and superintendent glixon may want to add a little bit to this but it is to Resource

019this information for the state to be able to be of assistance moving forward a little bit better to say where do we need to deploy our resources to help ensure that school divisions are able to operate just how they're supposed to which is having kids in school so um you know that that is really the Highlight here is how do we serve as a resource in a liaison to get those things moving to get kids back in school and I will just build off of what Dr Meyers said since uh the school divisions are not required to loop with the state when they have like a weather related delay we were able to Leverage these heat maps with VM and vot in a more intentional way um and being able to partner with school divisions

020of for alternative bus routes or other Pathways that you may need what are the most immediate next set of roads for us to deploy snowplows and get B dot um into your area so that schools can reopen and so but again um that was based off of sort of nightly data polls with press notifications press releases um the local media Outlets who were quickly reporting which school divisions had notified them on a potent a potential School closure or weather delay so that then we could triage but usually it was still a day lag time right because a lot of those notifications came in the evening but that was our our best attempt by doing these heat Maps um which was an unprecedented process and so we just wanted to share with the board that was

021a new development as you heard we just haven't had that kind of weather impact um on the local communities in many years just ask a followup yeah so uh those divisions are not listed as I as far as I can tell on the list of uh divisions that have asked for waivers Etc um do we know is that are they extending the school calendar how are they dealing with it such that they're not having to ask for waivers um Miss Holden the the answer to that is every School Division has a different instructional day and so I will tell you in the jurisdiction where I was superintendent if I recall correctly we had 1,30 1,040 hours built into our instructional calendar so you divide by six hours that gives you the number of days that

022we could have beyond what so we had that number and I remember us being a little higher than some of the surrounding because we also went 180 days so we had more than some of my surrounding Partners in those counties and so we could withstand we had a little more leeway if we had more extended snow days I will also just tell you that I um was next to Handover County and there would be sometimes Hanover County would be out for days and we wouldn't be out but one or two days even though we shared a border because if you look at Handover with up in kind of the northern Fredericks touches Fredericksburg versus where I was touching nearly Williamsburg um you just had big differences in how the snow came down so there's lots

023of factors that go into these divisions may have had they may be read on here for having a number of days but they've got a lot of instructional hours built up that they don't have to seek a waiver and you will see here in a slide um on slide nine that's coming or yeah slide nine that's coming to you there are a few more that have submitted waivers that have not been approved yet so we'll talk about that in just a moment and I I mean I think the heat map is very useful and I see yes that is a a great Improvement but but it does beg the question and you know 10 to 13 is a lot of days to miss over over uh two months but if they've got that many days

024built in power to them they could be going extra time too so they know they don't need to seek a waiver um and and most of these School Division they found other Solutions that's right that's exactly right I can just speak for having worked in Guin County and been on the calendar committee for so many years knowing that our situation was um not generally unique across the state but for the Greater Richmond area um we always built in really way more days than we ever needed to um and so that to answer your question generally speaking those days are already built in can agree to that too also serving on the calendar committee for many years my school system um and in Southwest Virginia U although Scott count is not only is because they missed

025a lot but they were lower elevation so we didn't miss as much obviously as uh some of the neighbor counties but you learn your school system learns to plan ahead and and we're have extra and we went extra hours every day anyway which is what you're I think moving too so that's all about they probably done ready to move on okay um on slide8 you will see that um what we have received and have has been approved for these School divisions for waivers due to weather um as I said they almost all School divisions will move to a one 990 hours instead of 180 days once they're less than those 180 days they have the option to use 990 hours so what they start seeking waivers for is a number of hours that they can

026have below the 990 threshold to be waved due to weather but it's a little hard sometimes to know how many hours we all work in school days right not school hours so I did try to calculate on there approximately how many days to give you a better reference that that may be I used a 6-hour school day which is barely typical across the state but not it may not be exactly 6 hours which is why I have a little footnote there that says um days may not be actual that's using a 6 hour calculation to give you an approximate number of days and the only other element I'm going to flag there is some of the superintendant on this list flagged for me they intentionally sought a waiver after their experiences with the hurricane in

027October and some of the early winter weather um haven't used the waer but wanted to plan ahead because they weren't able to predict what was coming in the spring and didn't want to be under that threshold of the 180 days and so I think that's also an important caveat yeah we'll be able to calculate at the end of the year how many of these actually used waivers and I know some of them will not use the waiver that has been given um because they are extending school days 15 30 minutes and that adds up a lot over the course of several months um had to do that in the past as well and and so they really are trying to get to the 990 but I think they were try they saw this so unprecedented

028this year that they they wanted to have some kind of security if they were not able to get to to 990 and I I would just say I think that we're seeing a lot more proactive work on this these days I mean Debbie you probably um would agree it we used to be very reactive um and scramble to add days at the end and switch our calendar up and do all this stuff you know that parents just really um were hit hard with you know with extending a school day and and accommodating for for child care or you know extending the school year as we go into April or something like that now it's very much you know built into the calendar um and or said that if we get to a certain place in

029our closure um in our closures for the year we will have to do X so please do not plan a vacation or you know whatever it is um so early that because we may find ourselves in a position so I I do just want to um give a shout out to the superintendent across the state that have really changed kind of the trajectory from that reactive state to a more proactive State because that's that has been a big change in the last 20 years on slide nine um as I alluded to a few moments ago there are uh some other waivers that are sending in a pending folder that we have and these are the school divisions um that I I think are really again as president Cy just said trying to be proactive um

030on this it will be interesting in the end to see how many of them actually need these waivers but they have similar evidence and requests that pre previous School divisions that you saw especially on the weather slide a little earlier um and so they they will need to be considered um for uh for the superintendent to make decisions on here in the near future but we wanted to make sure the board was was plenty aware of that slide 10 um wanted to finish up here with a couple of uh a couple of counties with uh specific situations and but Canon is is a um one that we're kind of doing a spotlight on because while they've faced a lot of adversity this year um and talking with them what they they're going to every length

031they can right now to protect every minute of instructional time and uh again having been a former principal I can tell you that there are a lot of hours in a day sometimes in a school year that that get put to other good things I'm not saying that pep rallies and and other fun things you've got to have things that make school interesting for students but there there's a real focus in this school division right now and I think many of these School divisions they know that their students have missed um I've never met a superintendent that didn't want to try to get students to school there are just sometimes real risks and potential for lawsuits and all kinds of things that dictate that they that they can't and so this county is really protecting

032that time right now they're going extra time um they're creating um every couple of weeks they're creating a additional days of school by going this extra time and and really trying to make up that lost learning from missed days slide 11 I'm G to finish um with with Covington and um wanted to talk a little bit about that because there has been uh some some news generated around that especially in Southwest um Virginia and I actually had a chance to um superintendent and I had a chance to meet with um it's it's actually alagan Highlands um and cumington middle school that's all a little bit um not confusing but new to us because they are the first school division to consolidate in many many years in Virginia um and this is the first year of

033consolidation which they said added to the complexity of all of this because as you can imagine some people love the idea of consolidation and some don't in these small communities particularly and so it just added to the complexity that they explained to us um but we did have a chance to meet with them to really get a sense of what was going on and and um I learned that they had an extremely challenging situation that coming to Middle School um they said that on January 31st multiple people reported uh an unidentified odor uh and and what compounded that problem was there were a number of people indicating that they felt ill and were treated for being ill at that same time so as any responsible administrators would do in a building um they they had

034to clear people out and determine if it was safe or not and what they did is they um they did send students home and they they ended up working with vdh and VM very quickly to try to do an initial assessment and understand what was going on they were not able to determine any immediate uh reason for this but again you can't just send someone back into a school if you aren't completely sure so um Covington hired procured a um a company to do with with Professional Engineers to do an environmental study to determine if this building was safe um to come back into and um again the weather during this time made it difficult for this company to complete um in the time frame that they had hoped and also just the nature of

035this testing you're depending on labs and other places to bring things back it's not as fast as any of us would like and so they iMed imediately moved students to online virtual learning and the difference about the online virtual learning they did is it was synchronous and oftentimes had students in the classroom that could get or no that was a different school that did that but they went synchronous um and immediately did this teacher teaching students in real time as opposed to as we saw during Co a lot of learning was more asynchronous the teacher might come on for five minutes so I think they were doing everything they could to try to keep students um learning and then eventually that study did come back and they determined that the building was safe they did

036a lot of deep cleaning um and uh they did bring students back to school on um March 18th and um I I again want to just highlight they they told us that there was a significant pressure within the community um because they were afraid students might be sick to not bring back students back to school um so there were a lot of people that were a little fearful and scared of of that building and remember I told you weather and air quality were two of the more challenging things that you deal with because they they do create a lot of and rightfully so a lot of fear and and concern so um the other really cool thing that Covington did um that that we found the superintendent and I found is they created a website

037a link on their website and I've I've linked it here in this presentation for you um and if you open it up be careful because I mean you can but there's a video that comes up by the superintendent it's a four or five minute video and don't don't push play because we'll we don't want to do that right now but if you have a chance um after the meeting check out her video it's very heartfelt to the community and very she explains exactly what's happening they also were super transparent about the timeline FAQs they published the reports that were done by the Professional Engineers so they really tried to promote transparency which anytime you have a really unfortunate situation like this transparency is one of the best things you can do in a community just

038so that everybody's on the same page so we did want to um congratulate them and and share that as a model of what others can do to try to be transparent with their communities so with that said I'm happy to answer any other questions that you may have about um 180 990 waivers that we call them um happy to answer questions um Dr Myers I have a question is there a means by which doe could maybe put something on their website or something that is really direct for school divisions to just press a button and say hey we need help with State resources to you know really motivate vot or motivate resources to you know get to our locality we need help doing XYZ because I'm I I just imagine you know looking at the

039pictures that you all provided in this presentation um you know there are a lot of things that are going on in the minds of everyone in um local government to include superintendent and just local government in general around how do we get our community moving dug out um you know the the floods in Southwest Virginia due to the hurricane you know I can't imagine just kind of the overwhelming nature is there a means by which we could put some kind of thing on the website rather than having a superintendent have to email somebody who has to chain it out 20 times to 25 different people you know is there something we can do to quick start that yeah um the superintendent I actually discussed this and um absolutely I mean we we can we collect

040lots of things from school divisions um what anytime we do something like this I think it's really important that they see this as something to help them and not as just one more burdensome thing to have to do when you're in the middle of I mean I used to work very closely with the County Administrator the sheriff the I mean I worked you know we're all working this locality very closely together but to have some kind of resource to a bigger State because the county only has so many resources so to have um that connection I think is a great idea um because at the end of the day everybody I talked to dealing with these weather waivers and and hurricane waivers and everything else um that they all want their kids to be back

041they want them in school they don't want them home parents don't want them home they want them in school learning we all have the same so any resources that we can coordinate and I think that's been a big Hallmark of governor yunan too so far um what I've seen is that there is this real push to try to coordinate services to help um those throughout Virginia and uh so yeah we can we can look at doing something like that um I'm hopeful that we're through the worst of this year but there are tornadoes and other things that happen and so yeah thank you question um Miss Merson can you hit your green button please thank you from the time that a waiver application or request is made approximately how long does it take the divisions

042you know the state to make a decision about approving the waiver well that that can depend and I'm sorry to give you kind of a a wishy-washy answer like that but um I will say it depends because because what happens is some school divisions will send in very specific information and they will have really good evidence and it's pretty easy to turn that around um fairly quickly you have to remember this has only happened typically once twice maybe three times at the most I can remember one of other time before the pandemic there was a fire in York where the superintendent at that time was dealing with a single school and a single waiver the last time we've had one in the last several years was due to a school shooting at the end of

043the year and there was a waiver asked on that that I'm not even sure if they used or not but so this year was very very different and so it took a little longer because you know you had to make sure you were doing everything with policy you had to make sure that um all the evidence was clear and really good and and sometimes there's some back and forth about the evidence because there's just not quite enough and you want to really look at every single one and make sure that they that the superintendent is that that the local superintendent and the state superintendent are both making sure everything has been exhausted before you grant these waivers because being out of school we've learned with chronic absenteeism and other things is one of the most

044most uh detrimental to student learning and so um it's a big decision but I do think it's been done with with care and caution and these are just have been really difficult decisions this year so to answer your question I think within a couple of weeks something can be approved with that but um it may take a little longer with some back and forth um on that and it Miss um is there a pending waiver request with respect to the Covington Middle School I see there's alagan you mentioned but I don't the same yeah I think that is I think that's alagan I would I would have to look Miss Holton to to know we we have we have a document that has so so I understand the difficulty in the Covington situation and I'm

045sympathetic to what we're all dealing with with aging buildings and difficult uh environmental conditions and on the other hand we do know plenty of school divisions in a lot less than six weeks have managed to get kids back up again you know the fox School here in the city of Richmond people were back students were back in class in I think a couple of weeks in an alternate location in the Damascus Washington County some of the hurricane damaged they had schools back open much quicker than six weeks through extraordinary efforts um but I'm just you know so Co Covington was dealing comington Middle School dealing with a difficult situation but I wonder if there were other options than the remote learning and in particular if we are evaluating a waiver request from them is there

046any relevance of the law that the general assembly passed in its wisdom in 2021 even before Governor Yan's term started if I recall correctly requiring inperson instruction with limited exceptions and maybe some of those exceptions apply I don't know but wouldn't that be something would want to look at yeah part of the waiver evaluating a waiver request with respect to comington so I I wholeheartedly agree with what you just said because the most important thing is that students are in school and anything I say is a little like Monday morning quarterbacking right now right I I lived these positions and I know how tricky it can be so I want to preface that that I was not there so I I can't answer that question exactly but if you were to ask based on my

047experience um first of all I highly doubt they knew it was going to take that much time and so next thing you know one week turns into two weeks and then at what point you think you're going to get this back and you're going to be able to get kids back in So finding a large enough place and venue and getting buses and doing all of that you really need to know you're going to be out for an extended period of time otherwise there's way too much to have to coordinate to do all of that and my suspicion is that time just kind of Multiplied on it and I also would hesitate for us to compare a small community like this to the city of Richmond because the city of Richmond has far more resources

048than um you know buildings and various places now again I'm Monday morning quarterbacking I don't know what happened exactly in that Division and um but I I will tell you in talking to their their school board as well as their their superintendent um they are passionate about learning and this was a big big thing for them um and they also wanted to highlight that this consolidation made this even more challenging because there were a lot of people with differing opinions about this this whole thing making this just a super challenging event for this community so well I appreciate your admonition against Monday morning quarterbacking you're absolutely right about that and local superintendents and local school boards have the hardest jobs in the world absolutely but um but I thinking about what our responsibilities are our

049meaning both the department and the board if uh Covington is not meeting its legal requirements in light of the inperson school requirement what are what is the board I mean you don't need to answer this today but we should be thinking about what's the Department's responsibility what's the board's responsibility and if they are asking for a waiver request then that's in our court um they may not be asking for a waiver request because they may be counting those remote days as school days and they may be allowed to under one of the exceptions in the the code the law that we're but somebody ought to be looking at that and I don't know I don't know whether the board has any responsibility in this area or whether the department has my guess is if there

050is a waiver request that applies to Covington that would be something that the department would want to consider not Monday Morning Quarterback but going forward and you know then helping them think about what are their options if summer school or extending the school year what what other options yeah so I I think that you know this is another place where this this uh you know opportunity for some kind of something on the website to be able to get this information from you know the school division to the department to say hey guys flag we need help this is what we're living under I'm not I'm not excusing what occurred because as you all know I I had shared um some communication around this that um was very concerning that the board was not made aware

051of this um at the time that this was occurring because I do agree with you miss Holton this is a critical situation you know I mean understandably when when roads are washed out that is a totally different ball game we when we don't have Bridges um uh and and understandably uh you know the the consolidation of the two counties that's another thing just kind of compacting this issue I would just um I think one of the things that I would like to know for certain is what is the communication to the community like there um you know I is that an are there effective means that that the School Division has to communicate with um parents families to ensure and to assure them that their students are not missing so much work that they are

052you know I'm thinking about graders in middle school going to high school are they going to be prepared um are we is our our responsibility is in adequately preparing students for their future um are are we living up to that are we holding them to that standard have they told us that they're extending the school year all of those things um because we can't just say You' not been in school in six weeks how are we addressing making that time up um so yeah that uh superintendent glixon yes thank you I think one of the things that was clear from the Covington Middle School discussion that we had with the superintendent and a few of her governing board members was that um there are also some lessons here for others to learn from and we

053have a lot of best practices coming out of the pandemic both in Virginia and nationally including small learning communities flipping into Partnerships with public libraries and other spaces that have great track records where kids were able to get in-person highquality learning experiences with students and from a Department's lens at least from what I've seen on the website on day six we have not yet put that information for some of those Innovative spaces how you might approach this and so we did speak with the superintendent about it might be a great sort of lunch and learn webinar opportunity of in hindsight what do we wish we knew and therefore from the Department's lend how do we help them think through some of these Innovative strategies where it's not road closures and weather related but it may

054be one of these unique circumstances um while also trying to prioritize State resources to expedite some of the review and analyses processes but and we better arm our school divisions with here's some of the ways to think outside the box on different spaces that may um get you into in-person learning quicker um because we have so many illustrations of that from successful school districts and school divisions across the country Over The Last 5 Years um we as the department can definitely be better leveraging that and making those resources available I just want to iterate that I mean it I think all of us are saying the same thing right in different formats um but in my view it should be like exceedingly exceedingly exceedingly one more time exceedingly rare that we would actually allow a

055waiver at this point in time given everything this National experiment we none of us wanted to have with Co but it should have taught all of us um how to transition quickly to Virtual we know that kids don't learn as much online or virtual but obviously it's it's better than not doing anything at all um and I would say that part of this discussion about having this extra time baked in like that should be a requirement like it shouldn't be allow that school districts would not have baked in I don't know how many days I'm looking at these waivers here I mean up to 10 days I I don't know like what's your worst case scenario that could happen in your school district weatherwise and then build in two more days beyond that um because

056I just we all know how important this is and and waivers on top of what we've already all been through is just crazy so let's let's figure this out let's build it in let's not allow them not to build it in anymore because um we just can't we can't afford this anymore Mr Ram yeah Madam president I uh we are all saying the same thing I too rise to say the same thing um uh I agree I agree with my colleagues I really appreciate um uh what the interim superintendent shared about how they're thinking about it I would urge us also you know the at the very early days of the pandemic um uh I was curious about like states that have experienced natural disasters more on the regular so for example Florida with hurricane

057season and some degree of disruption to normal school operations is is almost they don't know quite where it's going to be but it's almost guaranteed that somewhere in that state and so um I did uh some stuff with Jeb Bush who as it turns out was like oh yeah like we actually know how to do this because this tends to happen every late summer fall for us and so I'd also urge us to think about are there resources nationally whether that's you know the Chiefs or nasby or whether that's getting together with doing peer-to-peer with other states because there and you know as as obviously like we're under a wildfire watch I think today like the climate is changing and the sorts of things that are that that affect us are changing it seems like

058there's probably a fair amount of peer-to-peer learning that we could we could also do elsewhere because I I I recognize we all want to keep every kids in school I think it's obviously a shared goal but you know there are circumstances that are going to come up and despite our best efforts there are going to be disruptions um and it may not be even something weather related like a gas Le like we're talking about in Covington so I just feel like we we we it behooves us to have an urgency about having kids in school but also be realistic with as many schools as we have uh around the Commonwealth we're gonna have to this will come up we're gonna have to be creative and try to be effective Miss mcferson um can we also

059do a little bit of a deep dive into their performance and how it has been and their absenteeism rates so that we can just kind of see where they are and then when we talk about the resources you know under you know Readiness and things like that you know more tutoring being offered um could we just have something at a later meeting that maybe deals with some of those issues so we can see it in perspective yeah the only thing I want to caution to that is out of every School Division in the state this will be the hardest to do any kind of um long long longitudinal look because they're a new school division this year so you don't have you don't have anything to compare to previous years so it's it's going to

060be a little bit challenging with that there's still some analysis I think you could do but I just want to point out that that would be a little bit challenging and I I also just want to reiterate with with them because I know they're watching um they're online watching us right now and I do want to say that if they missed I don't know how many days but let's just say they missed six weeks comes out to 30 days um of school they did go online a lot of that with synchronous learning they also have extended their school day to make a lot of that time up um and so I think in the end their time out will be as minimal as they could have made it I really got that sense from them

061so it wasn't like hey we just missed 30 days oh well that really has not been their their kind of mode of operation at all so I I do want to give a shout out to them because they've they've gone through a lot of challenges over the last um you know couple of months and uh it it it is an unfortunate situation and I really do believe this is a great way for schools to learn um you know instead of waiting till the till the accident or tragedy happens to to to be pre-planning so I think all your ideas are wonderful and and look forward to our teams trying to help them the best that we can thank you very much Dr Myers appreciate it and so now we will move um back to item

062a ladies do we need need any transition time or okay all right we're ready good morning president greasy superintendent glixon and board members um you have in your packet or you should have in your packet a presentation um so I'm going by the PowerPoint presentation right now you also have a kind of a roll out plan um but what I'm looking at now is the presentation um so good morning again president chisty superintendent glixon and board members we are pleased to present with you a status update on the release of the history and social science instructional guides and professional learning plan for educators uh you know me Michelle Wallace assistant superintendent of instruction and I also have with me today the director of humanities Dr Angela bird Wright and the coordinator of history and social

063sciences Miss christania Brown I would also like to give accolades to the vooe history team members Andrea Emerson and Brandy McCracken who have worked behind the scenes to create our instructional guides along with christania finally would like to recognize countless Educators Virginia Museum Specialists and many his history external Partners from across the Commonwealth who have viewed the instructional guides over the last several months and offered and provided insights as to how they could be improved by providing feedback and considerations as we make the history and social science instructional guides the best in class going to your second slide you'll see a timeline and the timeline gives you an idea of the work that has taken place to develop the history social scence history and social science instructional guides um as you're aware the history and

064social science standards of learning were adopted by this board in April of 2023 during the summer through the fall of 2023 the instructional team at vdoe composed of the four different content areas developed an outline of how all instructional guides would look based on um the some common components for teachers um and and then some specifics for each discipline and they sought feedback and approval provided by agency leadership at that time from December 2023 through February of 2024 draft instructional guides were created for each course for feedback back from agency leadership last year from April to June the history instructional guides were completed and sent to copy editors in preparation for publication for the field it was also during this time that the history staff sought feedback from the field and also from external history

065Partners as you know Virginia is rich in history and there are many historical organizations that create resources so it's very important to get their insights into to uh how their resources are going to align to our standards of learning and thus the instructional guides are important for them to create their resources as well so it's kind of a partnership that we have with our our external history Partners in in creating these during July the history team developed a rubric that we would use to vet those resources that external Partners may be creating to ensure alignment between their resources and the 2023 board approved standards during the fall months VOE completed edits to the instructional guides based on feedback from the field and from agency leadership and in late November into December we began releasing those

066instructional guides to the field we then experienced a pause in the release of guides and recently the history staff have been working to add linked resources into the instructional guides which will assist teachers with primary and secondary sources as well as common resources that teachers will find useful as they implement the standards what you see today is only the beginning after the final release of the history instructional uh instructional guides in May the team will spend time with Educators in the field and also with external Partners to continue adding vetted resources to the documents those edit edits will be communicated through the field through email and through our Virginia education updates we anticipate being able to update resources initially J in January of 202 after that first semester of feedback from uh the field and

067then again in July of 2026 and then we anticipate those would be updated annually until until New standards are passed the team today is going to share with you details about how we've engaged with Educators and external stakeholders and provide a glimpse into the changes that have taken place to the instructional guide since their first uh release began in November and early December and then we'll also provide you with a Time line for future release of the guides so now I'm going to turn it over to the team members first to Dr bird Wright thank you Dr Wallace and good morning president Cy superintendent glixon and to our board members multiple external Partners so you are now on slide three so multiple external partners and division leaders have provided input feedback and support on the

068development of the instructional guides you can see beginning spring of 24 so between April and June of 2024 Regional meetings for history and social science division leaders were held so over 40 external Partners I repeat that over 40 external Partners were involved to include the Virginia Association of Museum Educators as well as multiple cultural groups and the Virginia um social science leaders Consortium in summer of 24 so between July and August of 24 surveys were also provided to teacher leader groups to inform and to support and answer questions about the instructional guides as well as expectations for implementation as we move to slide 4 you will also see beginning spring to present members of our team serve on the V a 250 steering committee and that's focused on three areas make history come alive spark

069a civic's Renaissance and celebrating democracy as you know established by the general assembly in 2020 va250 served to commemorate the 250th anniversary of the American Revolution the Revolutionary War and the independence of the United States in the Commonwealth of Virginia so through this work partners of va250 have provided our staff with academic resources and support for teachers and for students these resources will continue to be incorporated into the instructional guides where aligned to the standards of learning in Fall of 24 so beginning on December 3rd the history and social science program provided a webinar to over 350 division leaders curriculum leaders and school leaders about the purpose and the appropriate use so that the structure the content the context and how to implement the instructional guides with Fidelity as well as integrity at this time

070I'm going to turn it now over to christania Brown our history and social science coordinator who will talk about the gradual release of the instructional guides as well as the implementation for professional learning opportunities thank you so much good morning good morning president crey superintendent goon and members of the board thank you so much I am very honored to be able to present this information to you um you should be on Slide Five and on Slide Five you will see the Civics and economic instructional guide shown on the left is what was released November of 2024 it if you remember in August 2022 our team discussed with you the dynamic and interactive elements that are now embedded in the enhanced version that you see on the right the purpose and goal of the instructional guides

071is to include access to primary and secondary sources some of my favorite things and resources that are directly aligned to the standard our major components of the instructional guides these instructional guides are documents that will continue to grow that's very exciting for us and as we update and work with our external Partners such as the museums where they have the opportunity to Linkin resources and in exhibition Dr bird Wright talked about va250 there are so many museums that have exhibitions for va250 and we would love to link that into the instructional guides so they're digital collections that many um museums put their collections online due to covid so we now have access to so many primary and secondary sources this document will serve as a Model A base a springboard our local divisions have to

072develop their own locally developed curriculum this is a starting point for them and as we at the department continue to um add in links we will overc communicate those changes to the field as the previous curriculum Frameworks were in place for almost 24 years it is imperative to understand that the instructional guides for history and social science basically Elevate those curriculum framewor Works through the skills and practice bless you through the skills and practice Educators have an opportunity to find um ways to build reading skills writing skills communication skills critical thinking skills and stay true to the social studies discipline as you remember when we worked on the standards from kindergarten through government every standard starts with the student will use history and social science skills to get into the content the instructional guides are

073not interdisciplinary documents or curriculum but they do support the autonomy of the Local School Division to develop and continuously build their local curriculum as we've kind of discussed earlier the instructional guides will be interactive this is probably one of my favorite Parts about it as each locality develops their own instructional excuse me local curriculum through professional learning opportunities and professional development they will be able to interact with one another and share resources and lessons and because we have this as a model they can link it into their own local locally developed curriculum through these conversations and interactions educators will be able to again share lessons resources assessments and anything to consider that is aligned to the standard that type of networking and collegiality is something that we're really trying to help build not only only

074within a school not only within a division across the region and across the Commonwealth in each instance the instructional guides will also support the goals set forth by other divisions offices and programs within the um Department of Ed for example those schools that are working with the office of school quality one of the first documents that the office asked or let us see your local curriculum so they have it they have this base they can add in what they need to to fit the needs of their students their teachers their communities it's really great for example e students they can add in look at this document how can we best support our e Learners special education Advanced learners if you would please move to slide six the slide that everyone's been waiting for excuse me

075so this is how we're going to release the interactive instructional guides the first four I'm not going to read it to you but I will describe for you the first four you'll see an aster there that asri denotes that those guides are already online but they're not linked as of yet so they will be the first four that we re-release with links excuse me starting April 21st you'll see that we'll have kindergarten grade one and World Geography what we tried to do with the schedule is to balance it between secondary and Elementary this will go from basically April 7th through May 12th all of our linked instructional guides will be online available for our teachers schools communities so further support um to further support the implementation of the 2023 history and social science standards of

076learning professional learning sessions will take place in person and virtually um it makes it easier virtually but it's a deeper conversation when we're in person so we will try to do as many of those as we possibly can to really sit with teachers to help them understand because they're all looking for what's familiar that 24 year old curriculum framework that's familiar so what we'd like to do is to help them understand the instructional guides and how it has elevated the curriculum framework so we want to help them understand the guides navigate the changes of the content and the expectations there with the standards that you all adopted in 2023 there were some changes in expectations we need to make sure that our teachers understand those changes additional professional learning will take place starting fall of

077202 and go through the winter of 2025 possibly on through the winter of 2026 with sessions to focus on inquiry based learning practices scaffolding and questioning reading for investigation writing arguments having that evidence to support a claim and checking for understanding those are the big buckets that our professional learning will fall in so at this time I'm going to turn it back over to Dr Wallace for any questions that you may have thank you so much thank you Miss Brown we appreciate you guys being here um um you also should have a copy LinkedIn of the um sample Civics instructional guide I have a printed copy if anybody needs that I just have one printed copy because they are rather long um they are best used electronically one to interact but also as chrisa described

078so that school divisions can download them and integrate them into their own curriculum and local resources a lot of our divisions put things now in Google or on their own platforms and so it it's really great to have um resources in a a word format so that they can interact with them easier uh are there any questions off the bat here miss hton well first just uh thank you um um Miss Brown uh Dr Wallace and your staff for the good hard work you've done on this over many years now um um I um and I know that uh I'm Delight not surprised but delighted to see that you've done such inclusive work including the uh social studies coordinators and um our um Museum Partners at secretary gadera and I had the both were at

079the uh Gayla at the um uh Museum of History and culture this weekend and they made an announcement about all the work they're doing to put out more Civics materials um for you all and our schools to be using and it it just it's a fun to see that happening all across the Commonwealth and I know y'all are working hard with those Partners which is great um just understanding where we are you you know we've had there's a number of letters in our public comment folder folks um concerned about the timing that that these standards aren't already that these guides aren't already out and how that will Implement affect their ability to implement next year and I understand that's why you're putting in all this information about timing but I want to just understand the

080guides were all written and I I don't mean to be backward-looking but just understand where we are now the guides have been have been completed for several months now all your doing at this point is adding in the links to resources is that accurate that's correct the um links that are aligned so we're trying to focus on primary and secondary sources we don't really want to write down till we meet with our external Partners get into lessons and things like that but we want to for example and Civics and economics is a great um course to start with because we have the national archives we have the Library of Virginia and um the Library of Congress and we can when we're asking teachers to teach students um American constitutional government they need to have those

081founding documents that led to the development of the Constitution so we want to make sure that they have access to it if we're asking in the skills in practice we're asking teachers to reference text from these documents we don't want them to have to go and find so it's there um so right now that's our focus with primary and secondary sources once we get into meeting with our external Partners um such as the museums they have um over the years especially due to covid um they have put together so many great uh materials that are gerain to their collections we want to make sure that they are aligned to the standards before we link them in so those aren't linked in yet those are not linked in yet all right so so far it's just

082the primary and secondary yes primary sources really yes U um but just so I understand the guides that were written and done according to this timeline that we were shown done back in November and December and I've um a number of a us on the board have been sent copies of draft guides th you're not rewriting those guides at all there's no language changes the only change at this point is you're adding in links yes okay and um uh so if folks are concerned about the timing delay the the the content of the guides isn't changing it's only the links that are getting ated now is there a version you know back in November December did you have versions of these documents with links or the is that yeah no we've recently um you know

083just given nod to to back in August of 2022 when the standards were initially kind of uh the board was looking at how the standards would be written and and what would be included in them and then also with um you know discussions of of the board at the time of the approval of the standards we've kind of gone back to that thought process that we would include those Prim Ary and secondary resources to make it easier for teachers to access those resources um particularly some of our um School divisions that have high turnover in teachers um this makes it very easy for teachers to just access that was the original plan back in August of 22 the plan you were operating under in November December of this year did not include links and now

084you've reverted back to the plan to include links okay so but there's no rewriting of the guides um and in that reference I have a question that may be more eple and maybe the superintendent glixon is the better one to address but I'll leave it up to you all but the the prior superintendent made reference in a public uh statement um three or four weeks ago u a department was put out a statement about the guides were being um not released because of an additional review in light of Federal Executive to ensure compliance with Federal Executive action and U superintendent glixon knows she and I've had a correspondence where I raised first with Dr Coons and then with uh superintendent glixon concerns about that language because as I think we all know uh the law

085the federal law is very clear that the US Department of Education and federal entities have no legal role in the content of what is taught in our schools and this is all about the content of what is taught in our schools and it's my understanding that the guides are merely implementing the standards that this board adopted so I that language was a red flag to me I'm concerned about what does Federal Executive action have to do with our guides or our standards and the right answer should be nothing uh and so I asked uh superintendent glixon if she can help us understand what anything is CH I ask now what if anything is changing about the guides because of a review or what what does Federal Executive action have anything to do with these guides

086and what impact is that having on the work that the staff is doing sure thank you uh so first I want to give a nod to what an exemplary job the board did I was not yet in Virginia and so uh we truly have some of the best in-class uh history standards in the country and you all set guiding principles forth that clearly have been a focal point around the country both with the va250 um the convening that's happening right now with other states from around the country um convening and talking about Civics resources our work of the Department's team is being spotlighted I was getting a lot of grief why is the team not here I said because they're educating our Board of Education uh and so we'll be getting them uh all of

087those resources um and that to be said there is one Federal Executive action that occurred on January 29th that has a large focal point on teaching history it talks about the importance of the 250th anniversary it talks about a lot of the work that frankly Virginia has been the leader on for almost two years now and that other states are coming to calling trying to learn from and we wanted to make sure that there was alignment with the clarity that really matches up well with the Virginia's guiding principles which is nonpartisan factual resources and the true telling of History which is what this team has really prided itself on and we wanted to triple check the resource links that you heard from christania and the team of making sure that they are uh primary and

088secondary sources that really help guide our Educators around scking to the facts of History so no standards or instructional guide language has changed solely adding links as of former history teacher that is absolutely critical I uh they I was teaching in the early days of Google and that was not very timely or productive and so wanting to make sure that those were highquality links and very accessible to our teachers in the field Miss hton so U I'm familiar with that um presidential executive order you're you've uh referenced um I'm satisfied that the standards that we adopted absolutely Ely U um embrace the role of of educating our students on on a patriotic education um as they should that's a role of public education um I'm also quite clear that that executive order has no legal

089impact on our it should have no impact on our guides uh because our guides are interpreting our standards and as a subsequent Federal put out the FAQs that the department of Ed put out specifically about that order specifically clarified that Federal Executive action has nothing to do with the curriculum or content that is taught in our schools so in light of that my question would be is there anything about federal act executive action that is influencing the choice of the links that we are using now member Holton no thank you that's what I wanted to hear um I would personally like to say thank you so much to this team because um we did one of the very first things that um when superintendent glixon was appointed to serve uh in this capacity jumped immediately

090into this to write the ship um and to execute fully and completely the will of this board um that was clearly communicated uh in April of 23 and beyond that um these resources or these instructional guides should be dynamic should be resourced well um with those primary resources and secondary resources um we have no place to write curriculum for the local school divisions that is something that the local school divisions have the autonomy to do there's nothing to say that we couldn't provide you know here are some Exemplar ideas that that but that is not the primary responsibility of the department we've got to get our primary responsibilities out of the way before we take on any secondary um uh peripheral type work that could yes be useful for many local school divisions that is

091not an argument that I'm I'm fighting here my my um statement is just to say that these resources need to be there these instructional guides need are and will continue to be specific to One content area because that's what the field expects um having served as a Civics and US history and government teacher for the time that I did um these resources would you know it definitely is going to cut down on the time that teachers have to spend um digging and sifting through determining what's factual what's not factual um because there is a lot out there in on the internet and having a source that is credible at the department to provide those those resources um is just so deeply meaningful to the field and um I hope we'll think about this for other

092content areas as we move forward that would be my desire so um anything from anyone else Dr Northern I just Echo um just the usefulness of having links and well vetted links and the whole nation has been um working on Civics education for The Last 5 Years and so we have just an abundance of information like the you know it's just the the land of milk and honey here right with resources um so I appreciate that you guys are taking your time um and looking through them and being very conscious about which things are linked to as a resource um because it's just an embarrassment of riches which is very nice um and there are so many award-winning you know organizations that have been given all of these accolades um for the things that they've

093pulled together um the only thing that I would just put on as another option as we roll out this PD um and work with teachers there are multiple ways that we can organize um how we deliver this amazing instruction right and one way is just to lead with some of these primary and secondary sources that you guys have already linked to so I would just encourage us as we dig in and roll up our sleeves with teachers and offer all these PDS uh sessions that we also just think about how we can um use and C really Center these primary and secondary sources um that we're that we're linking tother comments Mr rham I want to say as well um uh this is great work some of the early versions I think were they were

094very utilitarian they helped people understand a crosswalk between the previous standards and the new standards but they didn't provide these resources and we need to make sure we're supporting all our teachers not just the ones who are going to be power users because they've been doing this for a long time we our younger teachers and our newer teachers um uh and so I think this is great I just want two things that my colleague Dr Northern just said that I want to underscore one is I I think uh well vetted I don't think we can I'm not going to say it as many times as she said uh her word earlier but like I don't think we can underscore that enough like the there there are lots of resources out there on the internet the

095the line between embarrassment of riches and just plain old embarrassment is a is a real one and I think it really is going to be important um in my experience like the stuff like that that that Dr Holton was talk was talking about Miss hon was talking about gets the headlines but like this is not a problem of whitewashing or woke washing it's a problem of quality and and we took a huge step forward and quality with our standards and we need to make sure we continue that because you know as Mark Twain teaches us like you know misinformation will be halfway around the world you know while accurate information is still putting its shoes on and it is just and so I just cannot enough especially at a time like this when everything is

096is is hopped up and I think we've seen examples of that not just nationally but here in the Commonwealth of resources that weren't well vetted created confusion got people spun up and um so much work has gone into this these really are when I'm with colleagues around the country people take the time to read them are like that these standards are are excellent as's a few of us us who are still um four of us uh who are still on the board from the time we did that it was it was a long challenging process but we we came up with a great product and I think that that is the best way we can we could continue to honor that so I just want to underscore uh just how how important that is and

097how I just think we we have much less a problem of ideology in general than we have a problem of just bad information and sloppiness that then gets weaponized in these political context so I would just urge you the the we can't say well vetted uh and and and be careful enough and I really like the new model though I think it's going to support teachers it's a step forward we are overdue on this but I think we hopefully we will get some Grace because although we're overdo what we're delivering um is going to be uh valuable and useful so thank you other comments Miss mcferson question yeah I want to thank the staff also for the hard work that went into this but question um what type of feedback will we get after you

098do the um the roll out from May 25 through the fall 2 you planning to bring back you know different reports to us regarding what your findings were whatever criticisms might be still linging as a result of the material I'd be interested in knowing our updates happy to provide an update at a later date of what that feedback back is um we're anticipating that as teachers that you know as we're working with teachers through professional development um we typically do a survey at the end of those sessions to see you know what is it that was beneficial and what is it that they still need um and then how can we improve and so we did that over you know last summer when we worked with educator groups as well as with our external stakeholders

099and that went into informing some of the questions that you might see under the skills and practices or the um content that went in like underneath the standards to help teachers understand what they are so some of that was already worked in um so we're we're anticipating that they're going to love that the resources are right there at their hands but we're happy to provide an update at a later date I'd appreciate that thank you absolutely superintendent glixon as increasing members of the board really quickly uh we were on with some of the principals Association members and uh vas and all the superintendents just yesterday and one of the items that's very critical uh under the department moving forward is making sure that we have those very clear feedback loops not just for history uh

100but also for math Ela the vaa uh and science and computer science and so uh we definitely have emphasized that and we'll be continuing to update the board including at our next meeting in May about um feedback we're receiving over the next few months so that you can be better informed on what's working really well what they want more of and where there are areas for improvement Miss Kilgore probably the person who's most recently left the trenches uh in the classroom I want to thank you can tell that you're not just doing a job but you're very passionate about what you're doing you're very excited about what you're doing and that trickles down on down and thank you for all you're doing and on behalf of all classroom teachers who you know whether it's history

101or whatever but for you all history one we just appreciate all your hard work and you know researching making sure it's valid making sure the links work all that helps so much and I think your enthusiasm will trickle down to their enthusiasm and on our students so thank you so much thank you I will say the team is amazing so they're a hardworking team they do they have have gone with many changes over the the time so we're we're um they're very enthusiastic about the work and they love working with teachers I will just say one one piece of feedback that I think we need to make sure that we're communicating too is I think this these guides are going to be really helpful to parents at home and so ensuring that parents know that

102these are available um is something that I think uh is going to be helpful to them as well you know because everybody's pulling their hair out at you know 30 minutes before bedtime over some kind of major situation but now we've got to find the right link to you know the Declaration of Independence or some something like that that so I think we just need to make sure that we're not only sharing this with the field in terms of our teachers and administrators and um curriculum directors but that we're also embracing these for the purposes of you know extra supports for parents and families at home too so what a what a great idea I remember those nights I just had one not long ago yeah oh yeah it's fresh in my mind this week

103thank thank you everyone thank you yes uh is there anything else here now my child's embarrassed she's not watching so it's fine we appreciate the board's support and superintendent glixon support thank you very much thank you all so much we um we just are so thankful for your leadership and your expertise um as we move forward so thank you thank you very okay we are moving to Item B good morning do you need a couple of minutes to set up or you good okay if we could take a five minute break please thank you e e e and we are here um with Dr cic did I say it right yes you did thank oh good um and uh on to the presentation about setting performance standards and so Dr CAC without further ado please

104proceed we're excited thank you so much for being with us today thank you president grey superintendent Del all members of the board um gosh it's an honor to be here um and it's it it's been stressful being here because um on short notice I'm filling in for a colleague I really respect Dr Chris domaleski who I know has addressed to you before um so I'm in his shadow and I'll hope to do as well as I can um but thanks for the invitation to provide additional information on a a really important you know um the setting of Standards is like a critical aspect probably the most visible in this whole process you've done a good job setting uh content standards and now it's the performance standards that we're we're we're talking about um on a

105new generation of um Virginia um assessments so um you have my slides I'll try and um walk you through this the um the the first slide says some background um it's actually number slide number two um it it looks like a long brag sheet of my accomplishments um but but it's not I'll tell you what it is so first of all um uh the first one says I spent five years as an elementary school teacher um so I kind of know a little bit about what happens in classrooms not so much today as 20 some years ago or 30 some years ago but um uh I I did live in classrooms and so I I have a I think a grasp of what Educators face um and I have a little bit of grasp what

106you face I was on a local Board of Education God bless you for doing this um I got to tell you it was in my case essentially non-compensated and it was on your mind all the time you know you you go to bed and you're thinking about this stuff so um it's it's service out of your hide that I think not enough people really acknowledge and and thank you for doing it um uh I got a PhD in educational measurement blah blah blah um uh recently uh separated from the University of North Carolina Chapel Hill um separate not for cause I mean it was like um I decided to leave the the uh the the teaching profession after about 30 years um um I've been a member of the national ass assessment governing board so

107I know a little bit about Nate um I served um as a chair of their committee on standards design and methodology so I know how they set standards and and that kind of thing um I've been the president of my professional association the National Council on measurement and education and um perhaps most importantly I am your servant on the last bullet I'm I'm a member of um you have a technical advisory committee for assessment and our job um I look at it as a service profession literally like you want to accomplish something it's our job to help you accomplish it in the most accurate defensible technically appropriate way um and to get your job done so so we all serve you okay um and then pictures of my books um on standard setting um please

108purchase them in hard cover they a little better little better royalty um uh and now just a a quick review of of things I know that you've you've heard about um in previous board meetings uh Chris has presented or or that you've um encountered in other contexts but um basic vocabulary we have content standards these are just set of statements that say these is what this is what students should be taught in certain grades and subjects essentially is what contents are content standards are you have performance standards these are the levels that we expect people to achieve in order to be classified into some category those categories we call Performance levels I'm zipping through this slide I'm sorry I hope um I'm I'm I'll try and pay attention to numbers I'm on seven now um

109uh performance levels these are just these short little labels I think you use below basic basic proficient Advance um there are labels that describe those categories and then there's elaborated descriptions we call them performance level descriptors um these are sort of longer much longer um descriptions of what it means to be in a category um I believe you've got distributed um certainly you probably have it as a as an electronic version but for our purposes today where we'll be doing a little bit of an activity you have a paper copy as well so um you can see that um these columns blow Basics excuse me I don't have a paper copy of anything I'm sorry you you'll have them I'll just give you it thank you um good and so if you look down there's

110like four columns and each of the columns describe what a student would be doing at that level what their performance would look like at that level um and we'll come back to the um in in just a few moments okay um the next um slide slide nine um is a sort of a definition of standard setting and basically if you think about um standard setting what this definition here means is that um we have in our minds an idea of what it means to be proficient or what it means to be Advanced but we have to translate that Concept in our minds into a number on a score scale and you got to take what this means and say say well that means how many should they get right basically um so uh there are

111a lot of procedures for doing that and that's been some of my work over the years is developing and evaluating such procedures okay um slide 10 um is the outline for really the rest of the presentation it talks about five key steps in doing standard setting there preparation panel selection orientation training an actual meeting where people get together to do the work and then there's documentation evalu ation and review all right so um preparation um I I believe um some of this has already been presented so I I I'll go quickly um but please you know stop me at any point and ask for you know additional information I'm happy to be interrupted in any way um so preparation for the actual standard setting event is well underway all kinds of things have to happen

112um some of the things that are happening are and have hap have happened is to identify an appropriate method you know there's like I said a number of different procedures that one could use um so a method called the item descriptor matching method has been chosen as appropriate um those procedures are being fine-tuned right now um the plans have been reviewed with your technical advisory committee uh we've seen several iterations of it we've given comments and there's been responsive um actions taken based on our input um identifying and recruiting participants is ongoing now I believe so um getting the best people you can to make these decisions is really a critical element uh materials and resources being prepared and then there's all kinds of logistical stuff that can really make or break a standard setting

113you know you have to a venue that that's accommodating to people so they can have the kinds of discussions that they they need to have as part of this and um okay um slide 12 panel selection so um in order for the results of standard setting to be critical credible you really have to have well-qualified people doing the work okay so there's a um a real effort ongoing right now to um recruit um people who are educator experts who know the the the subject matter reading math science they they know it very well um they're familiar with not just the content of of the discipline at the grade level um but the the standards the the content standards that we just talked about um but also what students are like you know what what what

114what are reasonable expectations for students how can we challenge them um what are their um developmental characteristics at a grade all kinds of things so you want people who really are are expert Educators involved um and have had experience with Virginia's students um and not just students generally but the diversity of students that exist you know you want people who have had experience with English Learners you want people who have had experience with students with special needs you want um people who have broadly representative of the Commonwealth okay um uh there's a bit of a typo here or maybe it was a thinko on my part um the last bullet says bwe has established targets for representation um I don't think that's quite accurate um they will establish targets I just am not privy to

115the finalized versions yet because I think there's still some um some some something things in play to figure out what should the final uh targets be but um if you were to imagine a sort of table it would it would have um in the left-hand column it would say you know uh how many male participants we want to have how many female participants so maybe it's like uh 55 female percentage 45 male percentage in the whole panel and you you establish those as targets so the people after the fact can then look at those percentages and go well they said they want a 55% female but they only got 52 you know and you can sort of go down each of those columns the strata the demographic strata and you can say how well did

116they meet their targets so um uh that's sort of critical information at that that last stage for evaluating the process um so uh targets are being established for representation um percentages of expert Educators education leaders high red reps uh business um U all kinds of um Strat and I'll get to a little bit more of that on the next slide but um this this just not just something you pull out of the air like what percentage do you think we should have um uh there's work going on uh right now to identify what are other states uh practices what are considered best practices for how many um El um teachers should be involved for example U what's the right percentages for this and that as well as the nape has pretty wellestablished percentages for um

117categories when they do a standard setting panel um slide 13 is more information on that um the specific demographic categories and appropriate percentages are being finalized uh Target percentages will be established for each stratum and the stratum typically the strata typically include what you might expect sex ethnicity years of experience experience okay you can read that yes if you would you prefer us to save questions to the end or now it's good thank so Madam president sorry U so this slide 13 does suggest you strata such as sex ethnicity Etc uh why is it important that we have that kind of demographic diversity uh on these panels well um the the performance standards will apply to all students and you want um representation on the panel of people who have experience with all students so

118if if um and I'll take a a crazy example but um in many states there are geographic regions or division types right and if you put a panel together of all uh persons who came from like say your key urban districts I'm not sure what those might be Virginia Beach Richmond I'm I'm out of Virginia geography now sorry but um if you put the the lot of people would say wait a minute there were no folks from Urban dist or Suburban districts like ours there's no uh folks from districts uh we we serve a highly rural population had none of our uh teachers involved there'd be a lot of push back against those standards people say that you didn't represent us in that so um I imagine just like you went through when you approved

119the standards we just um talked about uh Civics and economics or something um you want a diverse representation of input on that and that's the same thing you want in your demographic characteristics on a standard setting panel does that answer your question yeah and if I'm a follow-up question so uh so yeah that does help so it helps with the credibility of the final outcome to have diversity Geographic ethnic uh all of those factors on your standard setting committee will we at some point see uh the targets and the U how well you referenced folks being able to see both the targets and how well we met those targets at some point as part of this public process I think there'll be two points and I'll defer to to any staff that will um be

120able to tell you precisely when that that that second column of the table you know the target percentages um uh will be available um and then you at a second time point when the process has been completed there will be a technical report that's delivered that has that table with the final column filled in how many did we get so yes you will have that at your disposal at two different junctures thank you yes Madam president thank you yeah what this why we're pausing on this I think we you diversity is important diverse groups tend to be able to check each other and and and all that I mean that's that's well established I actually think we've had a problem of diversity however in how we've approached this over the years so something I'm really

121excited about is higher education Representatives folks like that because what you often you'll you'll hear you teachers you'll hear High School teachers say by the time I get these kids they're not prepared right and they're not having a conversation with middle school and Elementary School about what the standards are you he College professors be like the students can't do the work they can't do what I need them to be able to do in this class and and it's like well why don't they talk to each other there aren't opportunities this process is actually the opportunity to articulate that and talk about that so I think that that Viewpoint diversity and getting those people from different points in the system is really essential to getting to a good product rather and and traditional Ally and and

122these people matter a great deal but traditionally we've limited it more to grade level teachers grade level content instructional Specialists and so for all the talk about diversity there's actually an insularity in how people are going to approach that how they're going to think about that so something I'm just extremely excited about I think will be a big step forward is just a broader conversation about what do kids need to know and be able to do at different points to be successful at that next point and that's sort of more vertical uh alignment I think is is really important for students because otherwise they're flying a little bit blind and they're not they and any of us who have taught at the state universities you'll talk to students they get there and they're like I

123didn't know what the I'm not prepared for these expectations and these may have been students who were regarded as being very high performing previously so I just want to like just double click on that because I think it's an enormous opportunity for us and to think about diversity in a in a in a broader frame that will really benefit uh will really benefit students s I would say um everyone in my field really respects that point um um for example the national assessment of educational progress I believe their percentages are roughly 70 and 30 that's the first cut they do 70% Educators familiar with students at the grade level but 30% non educator experts I mean these are people from business and industry like you just said people from higher ed other people coming in

124who say these are our expectations and and uh the second part is is sort of baked into our process which I don't think I have a slide on that but um the different U grade levels will talk to each other there's a point called articulation which is I think a term you even might have just used a vertical something or other um and horizontal as well so that um people at one grade were able to say you know you're your standard seems like it's a little out of you know with where we're trying to get to and students at this really need to know that and there'll be these uh talks among panelists discussion is a huge part of this process so that um there's Clarity of expectation AC Cross grade levels um that's uh

125really integral to the process I hope that's helpful Miss mcferson yeah I'm I think I'm jumping the gun a little bit because it's coming up on the next slides but I did want to say that what's going to be really really critical in the process is the selection of the panel and I think that it would be helpful to the board that somewhere in that process we kind of get an idea of how many people have been selected and where they're from so that if we believe that there's deficiency especially for people that can deal with issues such as diversity and economic um impact issues from communities that need to be represented this time because a a clear example is in the area history so many people just don't know history as it impacts diverse

126communities and we going to have to dig deep a lot of people that are content oriented or content so-called experts are really not as learned sometimes as we think they might be and so we're going to have to really dig deep into professionals that know where we have made mistakes in history and that can talk about those types of issues that come from the community that are not often heard so I'm just as you go through panel selection I just want us to be mindful that at some point the board may have to use all of their its manpower to try to help staff and the superintendent to really look for those resources in the community that that can make a difference and they're not always the people that get selected for everything yeah um

127if if I can throw out some numbers so that you have sense of the scale of this I think there's G be upwards of 200 and some people involved um I think each of the grade subject now we're just talking about like fourth grade math um will have probably 12 or so maybe up to 15 um persons on there so that the the bigger that group is it's advantageous because then you can get better depth and and diversity of perspective um short of um uh individual interviews like actually interviewing a person who wants and that which would I think be time prohibitive um but there will be um screening uh along the lines of there will be um in most instances one of the stratum that is is is used is years of experience in

128education so you'll have um people who are just in their first five years um and that's a different perspective than someone who's been as you alluded to around for 30 years and has seen evolution in the content that's being taught at that grade level so um um uh I would be surprised if years of experience was not you know made made the cut as a a demographic stratum um but uh again um at some point I think you're going to be provided with that information early on so that you can see how the the panel selection is going Madam president uh and this may be U uh beyond what you're in the loop on up but we were're told at the February meeting and I know we've had change of leadership so it may be

129the information needs to be updated but as I understand it we were told that we would be sent the application link that that teachers and others were being invited to participate in these panels so that we could help spread the word and get it out there I think that that we we the schedule we were that was shared with us in February I thought that was going to come out in mid-march with a end of March response date U so that the application process was supposed to be over by the end of March and I'm guessing that we um had to make some changes alterations to that schedule and I don't know whether the department is ready to update it but I hope that it's the case that we are still going to be sent

130the application link when it is made available so that we can help spread the word superintendent glixon president Cy member Holton yes that's the plan we've been looking at the application we wanted to account for those additional uh participants that were seeking outside of the traditional um education uh sector and so um the team was working on finalizing those external parties so that they're not excluded or prohibited when they're coming from highed business parent groups and so uh we will get those links out as soon as possible I am the holdup so that's a project for later today they sent me the both of the final drafts good thank you um slide 14 um talks about um orientation in training um the way I understand the process is that an initial orientation will take place

131for non-educators who might not be as daily immersed in content standards and life in classrooms and all the kinds of things um that that the educator experts will will have some familiarity with so there'll be an orientation um for them um and they'll just find out about content standards the the process their role as a panelist and then how to prepare for the actual meeting um slide 15 a little bit more on orientation and training um there a training then will happen in common for everyone to get everybody on the same page um people will assign to these groups I as I indicated previously um probably about 12 participants in each grade subject level group they'll review the performance level descriptors that's the document that that was distributed just a moment ago um and then

132they'll receive some instruction on how to do this method the item descriptor matching method that is their training okay um slide 16 is actually about the task itself how do you do this um item descriptor matching method well um you've got descriptions of four categories below basic basic proficient Advance the task of panelists is going to be to look at questions from um the grade and subject level and to say for for each question which descriptor is this a best match to like this question something that's really I'm gonna speak a little bit loose here but is this is this question really not um terribly complex um it's sort of the the entry level of some standards so it's really we'd expect students at a basic level to to answer this question or does this

133question really match better the language in the advanced level so you're asking people do something really complicated in this question so I'm going to match it to a different level and we're actually going to do that task in just a second um or or a part of it um one of the uh important things um here is the bullet number three where it says they're going to discuss their judgments and and refine their ratings over uh three different rounds we call them rounds they're iterations where people will make a judgment then they'll be presented with some information um and then they'll U have opportunities to U revise their judgments um I've done this kind of thing before and the facilitating is is really a critical element here you if I were facilitating us let's say

134we were a standard setting panel um and one person in the in the group said oh this is a below basic level item and everyone else said no this is Advanced we'd call that person out and we'd say you you call in a in a friendly way okay right you you establish a lot of Rapport among group but you say Okay um pretty much all your colleagues said advance and you said below basic and you talk about why you why you said below basic and maybe somebody who labeled it as advanced could you tell us why you said it was Advanced and because people feel collegial in that situation uh one person will say well you know what um that's just not in our curriculum we don't I wouldn't expect any of my students to

135really this is okay um and another person will say yes but this is and there'll be this excellent discussion that happens and oftentimes um either someone will go this direction or someone will go that direction or the whole group will go you know what really it seems like this is proficient it seems like it's kind of on the borderline of of two different things so um uh that's probably the the most important reason to do this massive effort of bringing in hundreds of people to a venue and getting them face to face so they can have these kinds of conversations I I think that was a point um the previous folks just made too um don't want to do this virtually you want to get people together to have these professional conversations they're so important

136um so then people examine the results um across the grade levels um this is um in response to board member Ro yeah I'm sorry I can't my vision is just pathetic um uh uh this this is that sort of articulation people get a chance across grades and to make adjustments to talk to each other about what their expectations are okay so here's the here's the meat the guts the inside the box of this item descriptor matching method um on slide uh 17 it says for each item they answer a question like this um and actually um I believe this is as close as I can get to the final wording of the the final wording is always you know tweaked just a little bit but but this is I believe the final wording of the

137question they're going to be using which performance level descriptor most closely matches the knowledge and skills required to successfully respond to this item right which which descriptor most closely matches the knowledge and skills required to successfully respond to this item um the red is just a little cute way I had of highlighting it's called the item descriptor matching method right so it that this defin sort of just highlights what what's going on okay um so now we're going to do it um literally I'll ask you to do it um first people will come become familiar with the plds now I've I've sort of given this short shrift but a large part of that first training is like getting deep into the document you have in front of you okay um really understanding what these performance

138level descriptions say and knowing the difference between what is efficient what is Advanced performance and so forth okay so uh they become familiar I I know you haven't had time to get thoroughly familiar with these but let's pretend we have okay there's the draft that that you have okay this is slide 19 yes I apologize slide 19 is just a clip of a portion of of it not not the whole thing slide 20 says if you just would even look at that first page you should notice how the the expectations the complexity of the PLS um the language there increases as you read across the page um oh let's see here and the uh uh I'm looking under estimation and for below basic it says compare fractions decimals percents written in scientific notation you move

139across identify examples of subsets of the real number system reading further classify numbers belonging to the subsets of the real number system and finally justify the order and placement of real numbers on a number line including positive and negative square roots of values up to 400 you can you can kind of get a sense I um did lesie malun talk about this at all how the how the things um progress in complexity and in demand and in challenge as you go as you read across this page okay um so the columns represent what a student in a category would do the the reading across the page illustrates the increase in in challenge as as you're you're uh categorized into a below basic basic profish or advanced level okay um all right so um I've never

140done this this the first time I'm doing this activity I hope this works I I I tried to figure out something would I'm gonna ask you to answer this question which performance level Des scripture most C clearly matches the knowledge and skills required to answer this item correctly okay so here's an item um this non-secure so we're not showing you it act that is an actual test item it's just not one that's going to be on an upcounty form so it directs the student to look at these angles and asks him this question which two angles are complementary um now I had to go back and review this because I don't remember what a comp I thought I was like oh that's a nice looking angle no it's it it it means they sum to

14190 degrees okay so uh which two angles are complimentary and I think the what think I can do this I just said immediately when I saw math I'm out well no you can do it you can do it I thought complimentary Su to 180 so I got it wrong I was I got it wrong all right take the calculators out go ahead um so um uh flipping back just a few slides not literally but thinking back few slides you can see how important it is to have really expert Educators at a grade level who deeply know mathematics because you can't have the the eighth grade teacher go I'm not really sure what a complimentary angle is right you they have to really know their subject matter and probably have to know what the kids came

142in with from seventh grade and what they're going to be needing in ninth grade okay so you've got a bunch of expert folks who don't blanch at all at this right they go the answer is C it's p andq okay all right now my question is let's use the pl document which you have in front of you and this I'll just have to be quiet for a minute because you're going to have to flip around and find this use the pl document I'm on slide 23 to identify which performance level descriptor most closely matches what we're asking kids to do in that in that question okay so I'm just gonna step aside some M there'll be a musical interlude I want I won't call anyone out for an answer oh go for it were you

143see what I see where okay good good warmer warmer I would say that that would be a fail B thing because it's just describing it or recognizing it and defending yeah I agree fail basic but I think it that the answer is pretty explicit in it is embedded in the performance level descriptor which says fail you're at fail basic if a student knows enough to know how to recognize complimentary angles then they are fail basic but they are not fail proficient what page in the do page three of the hand page three yeah and and you're in the fail basic column okay y um all right whereas the P proficient would mean you have to um know a lot more about angles than just um all right recogniz um here's here's an example on on

144slide what would that be um 24 on 24 um off to the side uh it says from page three of the math plds complimentary angles are addressed at this basic level and a a hypothetical panelist might have circled that one recorded it somehow um because they think it best matches that um the performance descriptor says um students will just have to recognize and determine vertical adjacent supplementary and complimentary angles that's what they have to do at that at that level okay now as you might imagine people won't agree on this I I try to pick one that I thought was straightforward enough but there'll be some hard calls where people won't always agree and we'll come back to what we do when people don't always agree um uh and so one more just for practice

145um uh another non-secure item angles C and D are supplementary the measure of D is 72 what's the measure of C I have no idea what the ter oh you think we would we would fail eighth grade we would fail eighth grade nap n when you're finished put your pencils down and just do silent reading and so our question is not what's the right answer but what level performance yes yes um let's see right answer it might be 108 so I I'll jump that I think that this one likewise is on page three of the handout okay and that it's under the past proficient measure describing and using angle relationships determine the measure of unknown angles so this is a so Department staff I I think she's two for two and we have a panelist

146um but but my answer would be that I really think that so who how do how do how how were these panel the proficient level descriptors the plds who wrote them and how because the answers are embedded in there the the answers for these were sort of embedded in there okay some of them are I Pi the softballs that were the easiest and cleanest for our our so can I just add so having served on standard setting committees myself in the past um it's often way more ambiguous than this um and a lot of the times the discussion centers around um uh vocabulary within the question so I'm just going to speak for history for example um the vocabulary level that is introduced with in the question is often what is is the question for

147those who are sitting around the table it's not necessarily the concept as much as it is the complexity of the the other vocabulary that's part of that question or part of the answers to that kind of um I guess for lack of a better term stump the the group as far as should we be looking at the past proficient or past advance for this particular question and that's really where that consternation comes in from from my experience it's never necessarily been the exact um uh you know for this for this example you know the skill that's being asked to to be um achieved in the question it's more about the sort of the the extra part of what's going on in the question yeah sorry m in this particular examp example I don't think that

148past proficient it accurately describes what the question was because it wasn't unknown angles it was two known angles and with two known angles past Advance describes what you're solving more so than past proficient because you weren't solving for an unknown angle you were determining what the relationship was so know so that's the kind of conversation we would have if we were on the exactly exactly I think U president C you hit this nail on the head um you can write an item to a standard that's at a pretty low level of complexity or something that's a very high level of challenge so the complexity of the item as you indicated um maybe to some extent the vocabulary as you mentioned that's that's involved the number of steps one has to engage in in order to

149solve the problem might might be an issue maybe there's some other knowledge has to be brought to bear um particularly like in English language arts you know you have to bring something else to Bear to answer this question so um there the the homogeneity of the four choices we've all taking tests right and when the four choices are really similar it's a much harder question than when one really sticks out and the other don't don't seem to be be relevant so um a lot of things contribute to the complexity and the and the difficulty and the challenge of a of a question and so um as you just indicated and saw I think um it's not always super clearcut every question wasn't um modeled on exactly the language that you see here so there's not

150a always a perfect match to that um they try as as as best they can that in writers to make sure they're writing items to the the performance levels but um there will be ones that just are Ed cases and you have to you know kindy figure out where where it should be classified okay um so may I Madam president so uh regardless the the the the per performance level descriptors that content on that page that puts things in these different categories in the different columns is incredibly important yes um how was that and that and that content been developed already for grades three through 11 math and reading and in other words that's already I would have to defer as to the final status um of the performance level descriptions for what we use

151in the standard setting is has everything been finalized everything has been finalized to a mode because we do at the end of standard setting we will if the panelists come back with certain concerns we leave ourselves that little bit of room to make adjust but they're ready and I take it that means staff is whose the doe staff has been drafting these or where did the content come from and it and and and I'm just seeing the column headings um I mean what's fail versus pass isn't defined by nape but below basic basic proficient Advance those correspond to nape so does this correspond to nape question because we we got to measure the instance as you say you're going to have inance a seventh grade math teacher whose experience may be that when they get

152to e8th grade they're clearly not here at all they couldn't answer say 75% of these question but where does that fit now with the fact that we raise our stands you follow me right and so how so where did this content come from and how much is how's it connected to npe good afternoon my my name is Jason Ellis I'm the director of the office of assessment um so I think I can provide a little bit of a little bit additional information to to help the conversation so the pl start um looking back at the current performance level descriptors when we go to new item review each summer and so new item review incorporates information from the new and upcoming standards that will be implemented this year so last summer when we had new item

153review we actually took the current plds to meaning the ones we' used in the past yes ma'am okay and we we took those PS to that committee and at that committee after they reviewed what the new items will look like for the new test then teachers make suggestions and recommendations on how we might want to adapt those to the new standards and then you are right it is agency staff the office of assessment content experts as well as the instructional office content experts that build those PLS together um utilizing all the information they have about the standards and about the performance U for students and those in the previously um and that's how we developed the new standard or excuse me the new performance level descriptors this nape is built into the standards of learning

154I'm sure I know you all are very familiar with that and the crosswalk between our standards and nape standards and such the plds are built off of our standards so it's kind of inherently built into our plld that nape exists in there as well because our standards are written that way does that help a little I'm I'm still scratching my head over some pieces of it but um well here here here if I if I may ask another question and if it's better to wait until later in your presentation that's fine but the just looking at this example here I'm on slide 24 U uh you know we've talked about angles the proficient then goes on to say not only this is eighth graders not only should be able to describe and use angles but

155should be able to determine the surface area and volume of square based pyramids with and without diagrams and and cones and cylinders you know it's some pretty complex stuff um and my gut reaction is that a student who can do everything in colum three proficient would be a student who who would correspond to nape proficient where the language is Mastery of complex content for that grade level um but I'm not sure that all students who we would think would be passing 8th grade math would be able to do all of those things so uh the student you know we want all students to be able to master complex content but do we really want to declare all student students who haven't mastered complex contact but have mastered basic information for a grade level do we

156want them to be considered passing or do we only want students to be considered passing when they have mastered essentially the equivalent of of nape proficiency and so you know this these columns where we're labeling basic as failure rather than as pass U essentially is suggesting that we're only G to pass on our s so's the students who would also pass npe proficiency mastering complex levels is that built into the plld columns in some sense um so this is a difficult question for me to answer because um maybe if any staff want to jump in because it sort of um jumps over the fence out of my yard into somebody else's yard like my yard is the technical work of how this is is done and what's not my yard is is policy right so

157if you've chosen uh to implement a new set of content standards s sols that are um more rigorous more line toate more line with the levels of Mastery and and I'm GNA say this Loosely so forgive me if I get it wrong but you have in effect said we want passing to mean this well what we said is we our our content standards are based on the goal of students mastering complex material at that grade level but that we haven't yet made the choice about where to draw the PA I mean where to draw the pass line is what this process you know basically what's the grading scale well not all students are going to be expected to know 100% of what's in all of our standards so what's the grading scale is the the

158question you know is are are we on an ABF scale or are we on an abcdf scale so to speak and and only a students are going to be considered passing or you know so that's really the process we're going through today and tomorrow the decision we're being asked to make tomorrow and ongoing through to when we adopt the cut cut scores are grade scales basically um is the way I understand them teaching at the higher level I and tell me if I'm help me understand if that's not right but determining what's passing and what's not you're absolutely right is a policy decision but it's not implicit in the standard document itself we've already adopted standards rigorous standards Al land with name but the determining what qualifies as a pass versus what determined a fail

159is still to be done I think let let me let me take a whack at that and um that the questions you're asking are the past advanced level of me so they're very challenging um first of all I I don't think there's a direct correspondence between what you're seeing in this performance level description document and abcdf Grading um um as you might imagine and I've seen this I've done research on this there are districts who students are are getting as's and B's yet they perform very poorly on a state assessment just because of the grading standards in in play in that District so um so I don't think there's a direct correspondence there the second thing I would note is that um what standard setting is going to do is it's simply going to draw

160operationally the lines between those columns right right it's going to be here's the cut score that separates people in that First Column from people in that second column that's that's what the standard effort's going to do and then third U and this is where I I wish I knew more about this the whole history of this process but um you could I believe choose to label that First Column fail below basic and you could label that next column pass basic I mean that's a policy judgment that goes far beyond and so I think that's much more useful than thank you than my ABCD so throw out my ABDC analogy but is it fair to say that these plld standards are aligned to the nape categories of basic and proficient so you know um I I

161believe that when the uh new generation of Assessments and the new items were written and the Educators were involved in that and these performance level descriptions were crafted that there was um clearly a an anchor in what nape is doing a onetoone correspondence I don't think exists like you can't look at this for geometry on this uh eighth grade thing right and and say what is being done at the performance level in eighth grade geometry for basic for example is the same as what Nate says not on item by item but overall the concept would be a student who would score at the basic level on nape would be get would understand the items listed under in the second column here whereas only the students who would score at a proficient or higher level on

162nape typically on average would understand these more complex surface area volume square base pyramids etc etc etc not and not answering to this specific chart but to the pl's overall I'm gonna have to say more or less as is my answer to that one and the reason I'm saying that is because also um as I heard listening to the previous presentation um local control um states are are implementing their own curricula and so forth their own standards there's not a onetoone correspondence between what Virginia has decided to do like you've got PLS and you've got content standards you've got Frameworks and curriculum documents things like that nape has something they call their Frameworks for mathematics and they've decided what what they privilege for the so so something that appears in nape as an example as

163an example of what proficient performance looks like in Geometry might not appear on Virginia's and vice versa is that does that help yeah more or less I get it thank you thank you okay Dr Northern so there are 3,000 release nape items and I asked this of Dr domaleski last um last time and he wasn't sure um and just correct me if I'm wrong but my understanding is if we use some of those 3,000 release nape items as part of our calibration with our own panel wouldn't that help us to Anchor and figure out whether our expectations were aligned with some of the um questions that Miss Holton is asking are exactly the right questions um because to me without using some of those release nape items we we we really can't answer with authority

164whether this column is aligned to proficient or not yeah um it is possible Loosely to do that and um for example um youve probably heard of the smarter balanced assessment Consortium I mean they try to do something like that in their standard setting but um there's a bit of a difficulty because nape has a scale with numbers on it and Virginia has a scale with numbers on it there's no Direct correspondence between those two scales so a nape item at a certain level is hard to exactly say where it would fit in in Virginia now you can say because nap has done this work yes here's an here's an item that um we would expect students at the proficient level to answer correctly right you can seed those items in to um an activity to

165to see that people are calibrating that item the same way they're calibrating what they're calling Prof that that is possible to do um it's I can say it's loose because the scales aren't perfect like I can't tell you the 100 degrees is the same as 212 you know I I can't do that um but um there are ways that you can seed these into the process um a couple reasons why people don't always do that even though they want because first of all you've already done some things to to align well to npe second of all nape items are often different looking so panelists go oh you know and they can tell which one's the n i mean you really don't want them to do that right um and there was a third thing I'm

166I'm forgetting oh and it makes the whole process longer this is already you're asking teachers for like two or three days out of their their lives in a busy time of the school year and so the more items people are trying to keep these items that have to make judgments on as as as brief as possible to get the information we need so that's a complication too but no the answer is yes it can be done to give you a sort of a rough guideline just a quick followup Dr Northern so when um you know we come out with that big that big report that sort of shows how the state tests map up to nape that monitoring NCS report that they come out with every year I mean is it possible and I don't

167know I'm just throwing it out there is it possible for us to get the data from them where they've actually when they've aligned and put our our test on the nape scale can we actually see what decisions they made in when they did that analysis I mean I'm just trying to figure out how we can because if we're going to you know we want to be able to communicate right what we're doing here um and in a in a very um detailed manner relative I mean other than we we think we're at Nate profession or maybe we are or maybe we aren't um so I'm just trying to figure out whether some of those underlying data that's already been done um would help us to figure out and using that report as a as a

168guideline because they had to put it on the same scale right they had to do that equa percentile whatever they call it yeah so so um yes in fact um I I was recently looking at um Virginia's results on what they loosly call zipper plots it's it's like a a plot that shows where your state it's a nape State mapping is the official name for it but it's like um where does your States what you're calling proficient how does that relate to what nape calls proficient and the states are all in order and it looks like a little you know graph like that with everybody on there um and so you can say um our proficient is sort of at the high end of basic or our proficient what we call proficient is at the

169middle level or and and also our level of proficient is well beyond Rhode Island or or well I gu well sorry to pick on Puerto Rico but they usually have the low lower performance so we're far above that one but we're we're pretty below like I say Rhode Island or something so um uh yes it's possible to to incorporate that information um and it's uh it's entirely possible to seed in like I said some of the n n police items you refer to um but I would say hold on for a moment because it in in a couple of slides down the road I'll show you how nape has also been um referenced and used as anchor for this process that might um give you some comfort that it's going to be NP likee in

170its end result if I hate to do this but we have some um dignitaries that are here for us to have lunch with um our teachers of the year so if we could hold for about 45 minutes and come back or 30 minutes and come back at 1115 would that suit you I'm at your pleasure okay and then uh so so we'll do one 130 then and if you would like to join us we would love it so for lunch yes if unless you don't unless you have plans for spicy chicken oh good good good we're ready then so if we can do that we will take a break and come back um promptly at 1:30 thank you e e e I think we are ready to jump back in here um thank you all

171and so we will just go right back to where we uh left off here and I believe we were on slide 20 starting slide 28 is that right um I think we maybe 26 or somewhere in that vicinity yes thank you um the math quiz is just about over so um uh the uh I don't think we got to the the answer um this is a hypothetical panelist on page 27 they thought that particular item matched best with the proficient level okay so um couple things about this process the item descriptor matching process you can imagine the Intensive study of the plds you have to do first to make sure you're very familiar with them you'd have an advantage obviously if you were an educator at this grade level um but then looking at a

172test question sort of analyzing it going what is this calling for and then looking at the performance level descriptions and say where is this a the best match that's going to take some time okay and it's going to take some expertise and so those two things are built into the process time and expertise in terms of the panels okay on to slide 28 the actual meeting continued um here's what it's kind of G to look like okay so off to the side there you see um my best attempt to create a visual of um what's called an ordered item set or an ordered item booklet and um this is a a group of test items that is placed in order from the easiest one to the hardest one if you imagine it was a book

173for example the first page of the book would be a question that is just the easiest one in the group you know the next one a little bit harder a little bit harder all the way to you get to the last page in that book that single page is the question that was the hardest one in a test or it's the hardest one in that set of items Okay so they're they're already in difficulty order and how do we know well we've tried these questions out with students we know that only uh 10% of students answered this question correctly at the very far end of the booklet but at the very beginning we know that 90% of students answered this one correctly so we've got a good sense of what the difficulty order is okay

174we then ask people to answer that same question you just answered which was um let me read it verbatim which performance level Des scriptor most closely match just the knowledge and skills required to successfully respond to this item okay so um they'll look at a question and here I've just done sort of an exerpt you can see in the red box this is easiest to hardest you know the easiest items are back there up the top near number one number two hardest ones are down there at the bottom 31 through 50 or however many there are in a given test and so um then for each item and page in this case is really the same as item because it's going to be one item on each page they're going to make judgments that look

175something like this I'm now on page um I think it's 28 I don't have it numbered um uh 30 not 30 thank you um they make a judgment that that might look something like this this person for example said um item 25 they thought best matches the plld for basic they'll make this judgment for every question in the ordered item set all right now once they've made these judgments if you page over to 31 you're going to see a little gray area I've shaded what happens when you're making these judgments is you begin to encounter a Zone where you're not quite certain which level it matches best I'm calling this an area of transition right if you can see the first 26 they said basic excuse me 20 they said basic 21 basic so forth

176a lot of basics in a row uh oh and 24 they said they think this matches is the proficient descriptor but then they kept looking and they said oh well the next one matches basic though and then the next one match proficient so you can see where they kind of um encountered some turbulence in their judgments you know like and so um after that point i' I've shaded up through about 28 but if you see uh is it 29 30 31 they began to say well the rest of these These are more challenging these are all in the proficient zone so um this area this area of transition becomes pretty important um the the cut score likely will be somewhere in there right the cut score defining the difference between basic and proficient will be

177somewhere in there okay um uh in subsequent rounds give me let me add one more sentence here in subsequent rounds they won't have to start at the beginning of the book again or the beginning of the set again like I know question one is going to be below basic I I'm I'm not worry about that one my focus is going to be and every panelist focus is going to be on these areas of transition that'll be where we discuss things that'll about where we have these professional conversations about why do you think it's there let's look at the descriptor and see what we find there and so forth so um please questions in order to be considered proficient proficient um I'm I know how you've labeled your fail below basic fail basic pass proficient pass

178advance um that's not as common a language to me the adding of the passing and failing um so I'm I'm much more conversent with the performance levels um but like I said before you you could L label the first three levels failing if you wanted to by board policy I suppose but but what you've done I think is called the first two more like fail okay I think what we have have always done in the past and maybe this maybe the doing something different this year but there is a level that we Define as passing the so okay and so and that's your third level you have to get to the third level to be students are considered passing or not at some level and that's kind of one of the questions is what is

179how does this translate to those levels but and and passing or not matters for accountability at the school level in the lower grades and for graduation in the Upper Grade so past I hear you you're translating basic versus proficient but one of the things I'm trying to understand is how does this translate into our decision on a cuts which this board will make ultimately this summer presumably on a cut score that determines the difference between pass fail and fail and pass in advance we have we do have an advance category as well yes the process that I'm describing now is going to give you three cut scores for each grade subject it's going to you give you one to separate level one and level two it's going to give you one to separate the second

180level basic and proficient and it's going to give you one to separate the third and fourth levels proficient and Advance this level yes exactly exactly yes this would this is an individual panelist and it's hypothetical right but this is an individual analysts zone of sort of uncertainty they they they're starting to move out of where they think the what the basic student will do and into what they think the proficient student should do but they they're a little bit uncertain in that zone until they get more consistency later items 29 30 31 and so forth okay Miss mcferson Bas Merson we're kind of looking for is a crosswalk between past fail basic proficient and the other category eventually we'll need a we'll need a crosswalk and then I guess the question is how many fail

181Basics you follow me you can have that crosswalk will probably tell us that how many fail basic you can have in a category how many proficient you need in order to get into the cut score I'm assuming that at some point we get that huh okay so library closes at nine o'clock tonight um so I'm I'm gonna give my my my best attempt that um a cut score here would not actually be set at say 25 questions right because your test is scored using what's called a compensatory model meaning you can get some Advanced questions right right that count toward your score but if you don't get enough questions right of any flavor you're still not going to be proficient right so it doesn't matter how you get whatever the cut square is it doesn't

182matter how you get there you you you can get there by a constellation of a lot of different kinds of performance across all this content does that make sense okay um okay uh next slide is 33 so um now I'm going to just layer on a little bit here we we looked at one transition area and I've Illustrated one on this slide um this slide is my attempt to illustrate that there's actually going to be three of these areas of transition so I sort of flipped the axis here now we've got the easier to harder going um horizontally easiest questions on the end question number one is the easiest let's say hypothetically 50 questions 50 is the hardest and so these items are in in difficulty order and here's a first transition this is a

183transition a panelist might have identified because they became uncertain about where the the borderline is between below basic and basic so you can see that gray area they got some bb bb and then a b and then another BB and so this is my area of transition for that um if this person was going to recommend a a page where they would say this is the break point this person might say and I'm now on to page um 34 in the slide deck page 34 just drops in where that person might have said here's the middle of my zone that's what I'm going to say is about my break point between the low basic and basic um this person's uh cut would have been around page 13 or 14 we'll say uh if you go

184to the next slide we see the transition between basic and proficient that same panelist had to make this judgment where's my zone of uncertainty for the uh transition point between basic and proficient um click once and you'll get this you know roughly the panelist will say if you go on to page 36 um they'd say oh my cut would be around page 30 okay um click once more to page 37 they're going to have another transition between U proficient and advanced another sort of gray area and um click once more and you'll see oh well my my best estimate is about page 45 is the transition or the breako between um proficient and advanced okay so they've got these three areas of transition and this would be something the panelists would then talk about where

185was your area where was my area where how do you see this question how do you see that question okay let's go on to um 39 what do they they then do with this information well um they discuss their areas of transition um with other panelists they get normative information on where they stood with respect to everybody else you know U my my zone of transition was about here yours was there um my middle of my you know that the red arrow that I had on the previous slide mine was here where was yours that kind of thing um they'll get some consequences information if we took all of the groups data and compiled it together and found sort of a a general area of transition and put a put a point in the middle

186of that general area of transition we could then compute what would be the predicted impact of that if we use that point as the cut score what would that mean if it meant 20% of the students are CL classified as below basic 35 are basic 25 are proficient I'm not going to add this up right some percentage Advanced um that information will be provided to panelists so that they know the consequences of their decisions for their recommendations okay um finally and this is sort of a um a way of taking your advice and bringing it to bear on the front end this is something something that's being used in this procedure and it's called establishing reasonable ranges for the panelist to consider okay um what is a reasonable range and how does it instantiate your

187your policy advice um I have spoken to boards and other groups before where a panel has come up with a recommendation um and the board doesn't like it that's a problem that puts you in a very awkward situation of having to say no to what a group of experts content experts just spent three days Ming over and working hard on and saying no we think it should be higher we think it should be lower we think it should be different in some way we want to adjust it and so um it it's I would say in the last five to 10 years become reasonably common to get policymaker guidance on the front end so that so that people know when they're making judgments um here's a reasonable range in which my which the the the

188board or the entity has said these judgments should should focus okay so um there's this is a new term reasonable ranges and I want to talk to you about where these reasonable ranges come from okay so a reasonable range if you go to page uh 4040 in the deck um a reasonable range is defined as an external databased source that can be um analyzed or sources that can be analy ized so that we can have a basis for comparison or an anchor to something that is the policy guidance okay so in in this case the plan is calling for three bits of information um to be triangulated in a way to converge in a way take these three bits of information and put them all together to try and figure out what is um a

189a reasonable place for the basic below basic cut to occur all right and basic proficient and proficient Advanced okay so the first one here says um One external bit of information is previous so performance standards that's my understanding and this this we've we've talked about this in your advisory committee meetings um you have said um old standards are not good enough we want to increase the rigor we want to increase the challenge we want to increase performance across state we want to set a bar higher than it was before so a floor if you will for the reasonable range is is the past you can't go below there at all I mean as much as a panelist might want to say I think the standard should be here that's not your policy guidance your policy

190guidance is we're ramping things up so that goes into the mix in this analysis of constructing the reasonable ranges can't go below here for sure okay that's number one number two is nape information now we talked about this before lunch but Virginia's nape results were looked at for the available grades that's four and eight reading and math for the years 2024 what was performance like on nape in those years okay and not just for you but the uh the analysis included looking at other states what are um uh I I I don't think there is a term for this locally I'll make one up but it's called like high aspiration peers right you have States you look to and go we want to be at least there okay we have these high aspiration peers we

191want to be in the range of that level of npe performance okay so that's Source number two nape information Source number three is something called scale anchoring and this is a procedure that's just being done as Source number three where content Specialists sort of tried out this procedure this item descriptor matching and they said um where are these items looking like they match okay so that they could get a sense of do we have um items in the range that are challenging do we have a range that captures where the policy guidance has said we need to be so there's no real formula for putting these three together you can't like just add them up and divide by three or something this is a a really sort of nuanced judgment procedure that says you know

192given all this information we want to be higher than we were before we want to be close to nap we want to be in this aspirational peer you know given all that what would a reasonable range for your judgment look like so I've tried to illustrate that for you um let's go back go forward to slide 44 which is a repeat of a slide you saw before this is where this hypothetical panelist had this zone of transition you see it goes from about let's say page 11 to 16 or something that's their that's their zone of uncertainty or their transitional Zone they said it's there um but if you click once more I've superimposed over it a yellow zone I'm calling now again this is hypothetical I I don't have data on this yet I

193can't tell you what this is going to look like but let let's say a hypothetical reasonable range from analyzing that data from before that I just showed you from those three sources let's suppose that reasonable range went from page 16 to 23 panelists will have this information okay I want to highlight that this reasonable range let's say it goes from 16 to 23 um that right yeah um this isn't like handcuffs on your content Specialists this isn't to say you can't go above 23 you can't go below 16 right but the conversations will focus around we want to be here this is a reasonable range in which the cut score exists so setting you as an individual panelist setting your cut score down near 10 11 or 12 really isn't hitting that aspiration okay and

194so the person who's recommending that might might give good reasons for it and that would be a part of the discussion but panelists need to know your policy guidance in some way and this is one way of communicating it saying look we've analyzed a lot of the things the board has said is important and this is what the reasonable range would look like so ideally you know give a lot of consideration to your transition Zone but also give a lot of consideration to this reasonable range because that's where the board is pointing and saying we want to be here do does that help questions I believe it's a collaboration between the department and the vendor um the the content Specialists on both staff and yes are sort of doing their own version of the ID

195matching Etc and that's an anchoring yeah I gotta tell you I've been sort of impressed I think they're doing their own version of everything like as a dry run of everything ahead of time to make sure these procedures work and and stuff like that so yes they are doing this ahead of time to make sure it's it's gonna work some combination of do and pear and then can I move on Madam president please um so the now on slide 45 that hypothetical reasonable range so this is for each grade and subject somebody subject and cut score and yeah and and yeah so each graded subject is going to have three dividing lines yes around which somebody is going to determine a reasonable range yellow piece to provide to the uh panel members at the appropriate

196time correct and who is doing that so a couple things um one three for y and M rates three through eight maybe only two cut scores for Science and right end of course end of course okay yeah um who is doing that uh the department in consultation with the contractor they have content Specialists who are doing a trial run of that that was information Source number three of the three that are going to be thrown into a pot mixed up to come up with these reasonable ranges and you had one other question and I missed it well that leading to the next question so those reasonable ranges something that's been been decided or is being decided but it is based on an underlying policy choice about I'm hearing I heard you say back in the

197beginning the whole point of the reasonable ranges is to communicate to these panels what the board's policy gu ma' right yes ma'am so even though the board is not going to be on each test each um we're not going to be deciding those reable raos but it is our guidance to the staff that's going to help help shape those reasonable ranges yes um I if if this this is a pretty loose analogy but I imagine something like the board is pointing and saying we want to be there yeah and the staff and the vendor are working to figure out how to get there right so what I we I don't think we've made that policy decision yet but we I think the discussion has been leading towards what I think the board means to be

198saying we want the policy to align our proficient P proficient rates to align it more or less to Nate proficient whereas our current uh our past uh cut scores have aligned more to make Bas yes that to me that is an important policy decision that should be explicit and U now I'm this is not question this is discussion for the board but I think that's something whether or not we think that's the right choice we should be explicit about it and I think that that's a decision right right now the decision item for tomorrow is written as just asking for um direction to use the ID matching process but if it is the Bo's will that we align our cut scores that our panels align our cut scores reasonable ranges around Nate proficient I think

199we should be explicit and transparent about that and encourage us to include that in our motions tomorrow and then I think it has consequences and rep cussions that we should discuss but we shouldn't just it's not like it's the staff is just doing it through magic they're doing it because you all are telling them to do it um yeah can I respond real quickly just um I can say having attended now probably three of these technical I'm a recent addition to the technical advisory committee and at every meeting it's been made abundantly care to us what this board's vision is and that you want to be more nap likee and that you want to raise the level of challenge from where it's been that's that's been abundantly clear I think that's been made abundantly clear

200I I I see the contractor in their face their sensitivity um to this I mean so everyone I think is on that same page and that's how this this got developed the the one caution I would give if I might be so bold as to U caution around policy language you might craft tomorrow when I'll be out of town um you could right now simply say well have n standards adop you won't even need you won't even need a method you could just go boom and we're done I mean that but but the problem there is nape has a different framework than you have nape has different contact Cent than you have nape has different history than you have nape has different item formats than you have there's so many differences that to just map

201over which is what those State mapping studies do to just map Over N to yours and say we don't even need a standard setting is I would say is somewhat ill advised right and so the the language that people are using and it's kind of Wee but it's like in in quotation marks like we want NP like standards in other words we want to say that's our aspiration but because there are so many things that differ how we get there might have to differ also is that make sense yeah so that's I'm using your language from earlier of nape more or less yeah but nape npe like works for me too and I just to be clear I'm not um I personally am not 100% on board with that approach I think there are reasons

202to do it we've discussed it amongst ourselves and I think there are some downsides but regardless I'm going to be encouraging the board to think about repercussions throughout the system so I mean what we're talking about doing given where our scores have aligned with nape in the past is we're going to be doubling the number of kids who fail our SOS you know but right now our Nate proficient rate is around U between 30 and 40% is our where we line up with Nate proficiency whereas our so Pass rates have been much more in the 70 to 80% range so we're and and so the argument is that our so cut scores are too low and I get that they're some of the lowest in the nation and it may well make sense to adjust

203it but if we're going to double the number of kids who fail then that has all kinds of repercussions amongst all the most important perhap or the most front and center the first is how we communicate that to parents the transparency of the change we're making so we're not saying kids are suddenly twice as many of them you know we're not saying kids are Dumber we're trying to communicate to parents something about we think we were telling your you your kids were doing fine and we don't think that was right so we're trying to challenge parents and students and teachers um with this messaging tool of telling now that twice as many students are failures but we do need to deal with the repercussions of that including for all kinds of things like graduation rates

204you know so um right now we're saying 70 80% of our kids are passing and if this plan goes through as Express as as intended then we're going to be saying next year that only 30 to 40% of them are passing and uh what are the repercussions of that throughout the system I think should be looked at up front before we get too far down this line well I I can't say right now what the percentage will be until the panelists make their recommendations and then the well that's just if we go with NP life exactly like if you go exactly like n you could probably freak it pretty well and even if it's not exactly NP it will be in the ring reasonable range of napes so I can tell you it'll it'll double

205our failure rate because you can't get in the reasonable range of Nate proficient unless our students are suddenly more brilliant than they were M I have some experience with this I'll just maybe in terms of the communication which you're you're highlighting now um I helped the smarter balanced Consortium and they would go into a state and a state would adopt the smarter balanced assessments and their levels just plummet right and so um to their credit they had in place a very um wise communication plan to say exactly what you just said which is your kids didn't get Dumber all of a sudden we just implemented a lot more rigorous standards and when you do that you know when you say the bar is here not as many people are going to get over it until

206they've had the instruction and the Intensive focus on those new standards that that that that warrants so um it is a communication plan I think is is really critical here and even then you're not going to get over it because as you know with Smarter Balance Etc that have switched to those uh much tougher grading scales their kids might get a little better over time um maybe because of maybe in spite of the change in the grading scale but or maybe because they've done other things like what we've done here in Virginia with the literacy act which I think will improve scores but you know if we look at the wildest the most most ambitious Improvement we might hope for on say our literacy looking at what happened in the Mississippi miracle if our our

207kids would go up 10 or 11 points but we would still be looking at failure rates in a completely different category from what we've got now so we just need to think about the common Communications plan and then the repercussions for all the other aspects of our system that depend on those Pass rates like graduation Mr ran yeah I I completely agree um with Miss we need to think about repercussions and so forth that we need to be very thoughtful about this this is a this is a c change but I want to back up I'm pretty sure as C you said a minute ago we'd be telling that our students that they would be failures I assume you misspoke because that is well we' be saying they failed yeah or we might be saying

208we might be saying we failed them as a as a well their test score was a failure I agree I agree with uh there there's a couple of things I want to just make sure we're clear on I I agree with we we need to be thoughtful and so forth here and if we start having a more honest accounting for Where We Are we're going to see changes we and we need to we need to talk about that and and and that's going to be a communication challenge we need to be thoughtful I think it would be very irresponsible though for people to immediately pick up that the board is telling kids that they are failures and I mean I can't speak for all my colleagues certain failures but that they have failed the test

209and that the system is not and I think we need to start with okay what are the skills that you need as best we can tell in a fast moving changing economy what are the skills you need to be successful and I I will bring the humility to it we will get some of that wrong of course because the economy changes and so forth and all we can do is is the best we can and we're saying these are the skills that you need we need we need to get you there we just came from a lunch where one of our regional teachers of the year was talking about the problems we have just with expectations and and and this teacher was saying that even just in the system the way people talk about and

210perceive kids so I think by by lifting that and talking about that and then it is incumbent upon us to provide the support to the system something and and so we talk about Downstream repercussions I think that is a very important one okay what are we then going to do differently um but we can do this I mean I I I completely agree I think our literacy work if we can if we can keep going supporting it you're seeing states that are moving quickly on the npe that seems to be a common I'm I'm leery to especially with a psychometrician staring me down I don't want to ascribe causal uh things to the nape but there seems to be a pattern that states that have done that their scores seem to be going up they

211were more resilient in the pandemic all of that but you also see Florida like their scores went up uh under Jeb Bush's leadership North Carolina scores went up under Jim Hunt's leadership these were things they did so nape nape is not the weather there are things you can do and you can see improvements and we've seen different approaches and so forth that that have done that I think if we if we do that here and then that's the last thing I think we need to be very careful with our language like we are trying right now directionally we are not where we want to be relative to The Nap you look at the the chart that goes up sometimes it shows where we are you just you just spoke to that but I think we

212want to be very careful that we don't send the message that we are just linking explicitly and we're overlaying the nape because that will create all kinds of confusion we've already heard we wouldn't even be spending time on this if that was our if that was our intent we are using the nape as a barometer because it is at least for now if we can keep it it is a fantastic National barometer and very useful um but we're doing our own uh we're doing our own framework within that I think it's very important that we communicate that or we're going to get like all kinds of confusion and people are going to I think that the nape standards are something the our standards and these things so I think I completely agree on the communication

213point across the board but it behooves us to be very specific about what we are and are not are not doing here I just wanted to to note those things for the record mad president Madam president so one one thing I do want to be sure and say is I am completely supportive of the rigorous content standards that we have adopted across the board um and we want every kid every student to master complex material uh at the the complex material that they should be taught at their appropriate grade level the question is do we want to say as our current uh cut score are soon to be former cut scores say that a student who has mastered the basic information for that level but not the complex matters for that level do we want

214to say that that St student fails and hold them back from a diploma for instance um and that's going to be thousands of students unless we address repercussions for instance in our current requirements that students pass s sols if we change the cut score dramatically and don't make other changes to our system then we're going to have thousands more students not graduating not getting diplomas and they're going to be those students who are in that range between those who have basic who've mastered the basic skills for their grade level but not the complex matters for their grade level which is essentially the difference between aligning with Nate basic and aligning with Nate proficient and and uh you know if we're going to do that then we just need to think about those repercussions and those

215kids in the middle who we want to learn all of those complex skills but but recognizing that there may be some who get to the basic level but not to the complex level what are we going to call them are we going to call them graduates or not Dr Northern so I agree with a lot of what my colleague is saying here um I don't think we've made the decision that we're going to ramp up 40 points overnight and I'll just speak for myself personally where I am um not having an opportunity to speak you know openly with with the board on this topic but other states have done a more gradual ratcheting up you know most of them actually I'm I'm quite uh close with Dave drisol in Massachusetts and that's been that was

216their plan for many years that they would and they communicated that as we've all said communication is key like we don't want to be perceived as you know we're just you know ramping up the score and now all these schools are going terribly like that's of course not the message that we want to be sending so I do I agree that we need to think about the ramifications of this I'm more for a deliberate ratcheting up and not an overnight ratchet up um I do think that is disorienting um to all of our schools and the folks are earnestly um working to to uphold these higher expectations and get where we need to get um so I welcome that conversation because I think it's the one that we need to be having I don't know

217if it's something we need to decide today I've been struggling as we've been um talking to Dr CAC here of you know when should we be talking about um these these sort of nape like barometers do we want to kind of go through this first do we want to like I I I feel like the chronology of this decision making is very important and I personally don't have in my mind like when when these things should be made because we don't want to be perceived as oh now we know what it's going to look like so we better change right all these cut scores because we we we've got an idea of what we wanted to look like like that's not the right approach right and the right appro and we also don't want these

218terrible downwind effects overnight where we can't Supply the supports that we need in a timely fashion like we all want to do um so anyway I I just feel like there um I'm not disagreeing with what you're saying Miss Holton at all um and I I do think that that's a conversation the Board needs to have and that we need to communicate very transparently to all the teachers and Educators and folks who are working so hard in the state to to to to meet the aspirations we have for our kids M Miss mcferson right I I share the same concerns about visually what it's going to look like to the public when the information begins to come in how we as a board will be able to communicate our desire for um The increased rigor

219versus what may be actually happening in the field and how we going to impact students you know because this is a massive change so I I like to think that maybe a task force should be convened even if it's just a small group of us to begin to talk about communication how we will communicate to the public exactly what is happening as soon as we get the information from the bendor and also how we going to deploy the resources the the financial resources that will allow for the increase in the tutoring and the other things to try to begin to prepare their students and I think we got to get in front of that and I just wanted to make that as a recommendation I know that the staff is I'm sure you're looking at

220that um Madam superintendent but anything that we can do to be helpful with that just let us know madam because I think it's important we going to need to be able to communicate and we going to need to be able to deploy resources as quickly as possible yeah I definitely think that that that um when the decisions are the responsibility of the board it's also um somewhat the responsibility of the staff to do that communication so it wouldn't be us directly doing that communication um but I I know that they're prepared and at the ready to handle that uh Mr Arham thank you m president Mr K I'm sorry I thought you were finished I apologize um uh just two contextual two contextual points I agree with Dr Norther we're not GNA like jump we're

221gonna have to put this in I think we should be transparent about it so people understand where we're going and where the and where the targets are I think that's very important but yeah you have any system like I haven't heard any calls we immediately shock the system however the contextual issues here we cannot get away from I would encourage people go take a look at the percentage of Virginia kids who are eligible for free and reduced price lunch who are at proficient or higher it is in the teens any meaningful change here is going to cause uh is going to have whatever you want to call it repercussions consequences and I I I feel like there there's a there's a um H there's there's sort of a presentism that takes over when we have

222these conversations in terms of how they're reported out in the media and so forth and just where we are any meaningful change going to have consequences because we have a way to go and I would urge everyone to go look the Nate report cards they're accessible again at least for now they're accessible they're online you can you can see them I'm sorry Dr Northern um uh they are they're accessible they're online and I would urge people to take a look so you can see the Baseline that we're starting at particularly when you disaggregate if we talk about hispanic students black students low-income students the gaps that we have and and so it's going to be an issue and then that just bring to my second point you know in his uh letter from birmingh jail

223Martin one of the things Martin Luther King talks about is like nothing's ever well timed and people would always tell him when they were going to do an action well it's just not the time just be patient I feel like we have the same problem it is never the time to raise standards we always get up to the cliff and then there's a reason why it's just not the right time something's not right politically or the economy or how are we going to do this and I would just suggest it we we need to be providing the support to make sure our Educators can succeed but it is the right time to substantially raise the bar I don't know how you can look at either our current performance or the changes and so forth that

224are happening in our economy and what kids are going to need to be successful and not see that this is the time where we need to both uh demand more and then we also need to support and and and invest more I think I I don't know how anyone can read this moment uh that we're not at that place thank you Mr Hansen yeah yeah I'd just like to agree with Andy's um comments we have been working this for quite some time and I just you know we've had a lot of conversations about um uh you know trying to help close the achievement Gap try to help close the um learning loss that we've had and you know we've just have got to have this system in place we've got to be working towards this

225and you know we don't know where to priori ize our resources if we don't uh have this process put into place and again I just have said this probably 20 times from this Podium that we need to be focused on those kids on those kids that meet it need it the most and on those kids where it will matter the most and so I just um appreciate um obviously the importance of communications and transparency but transparency is is important for families to understand where we are and we just heard from four of our teachers of the year I think across the board of um you know different issues with each of them but you know from you know what they've experienced since Co and I think we are already in a ditch but we're in

226a bigger ditch right now and we've got to get ourselves um upwardly mobile and again to Mr rotherham's comment about we have a lot to do just to get us recovering much less getting ourselves positioned for the future and um so I I just really hope that we just continue to really make sure and I just really appreciate the um thoughtfulness that is going into this uh approach um and it is going to be one that's based on uh best practices based on uh research based on uh science and um I'm grateful for the great work that's going into this by the the teams that have been assembled by the uh Folks at the department and our outside partners and I I just am incredibly grateful for the work the leadership and I really have

227taken this today that we're getting to the right place and I just think we've got to keep going there and all these other issues this is the Department's job I think as the Madame President just said about communicating and again transparency is there's different versions of transpar or needs of transparency one is of what we're uh doing here but also the most important transparency is for parents and we're not going to be able to get where we need to be if we don't provide that to them and uh so I just I appreciate the conversation that's going on here but I just I want to make sure we're not um you know trying to you know deflect or uh Kick the Can down the road or any of these things we've got to keep moving

228forward and I think moving forward um progressively M mcferson then miss Holton I think we all agree that clearly we're raising his standards I don't think there's any issue about that I think everybody agrees about that I think we just need to be realistic about the fact that there are going to be the repercussions we need to be as best prepared as we possibly can be and that we are going to need to deploy the resources where they need to be in advance as well as during and afterwards so I think we're saying the same thing um and how quickly we get there is going to depend upon a lot of things but I don't want anybody to be living in a fantasy world that there isn't going to be some Fallout and I want

229to make sure that the students that need our attention the most are going to get it because we all want to get to that higher level as quickly as we can get there so misson two points that the the I do think we all agree that we want high standards for all our students I do think I have a different perspective from some of my colleagues here on in this regard is that I don't personally think that changing the grading scale on its own is going to make kids smarter to make that happen we have to invest in high quality teachers and curriculum and extended school hours and tutoring and all the other tools that have been proven to actually help students learn more and we've done some of that the literacy act for example

230I was thrilled that we heard from the regional teachers of the year that the bipartisan efforts to get cell phones out of uh schools is is helping so we're doing some of the things that work but that's what's going to make kids smarter um the the other point I wanted to come back to was I I appreciate Dr Northern coming about uh maybe we want to ratchet up over gradually over time and when do we need to make these decisions I do think that we don't have to make final decisions right now that we will be you the board maybe after I roll off this summer we'll be making the final decisions for this round of cut scores but tomorrow we do need to make some kind of final decisions about what we're telling the

231staff to do and I think the message the staff is getting so far is that we're telling them to set reasonable ranges aligned to npe proficient that's what the governor frankly announced three years ago and that's the message the staff has heard that's what they're embodying in their uh pdls the the detail descriptors that's what they're going to embody in the reasonable ranges they give to these panels unless we tell them something otherwise so nobody's telling them to ratchet it up over time we're telling them align more or less to n proficient which would be a dramatic change from where we are I mean that's the whole my colleague's whole point about what you call the honesty Gap is that we're going to go from 70% Pass rates down to 30% Pass rates and so

232if we're going to do that in a if we mean to be giving the panels guidance to do something different than that we should be clear on that tomorrow but if that's the guidance we're telling the panel to do work in a reasonable range around Nate proficient then you know we we just need to be explicit about what we're telling them to do and then be ready to Deal address the repercussions so I hate to be such a downer with a technical presentation and a Policy important substantive discussion breaks out um so do you want to go back to this or yeah okay okay um what you're talking about is all like super important and and there are there are models for use for communication I would say you know I can point you to

233some of those and and the best time is to start early when you're changing standards to start early when you know there's going to be a shift um I would say there is precedent for phasing things in in a number of states where they wanted to say we're going to get here it was like they were hiking from the bottom of the Grand Canyon and instead of going all the way to the top they stopped at what that little Waypoint is you know halfway to get a drink of water because you got a long road ahead of you so there's a lot of precedent for that um and I would say uh Looking Back Now at slide 45 that yellow band there which we're calling reasonable ranges is that nape informed reasonable range right okay

234I mean it's not let me make this clear it's not an exact number if you wanted exactly to be nped you could do that without a panel you could do that without any procedure just map it over and done right but because there's so many different moving Parts in the system you want to give people some some area to operate and so this this reasonable range was informed by the three things I just mentioned on the on the previous page okay um so this slide 45 only illustrates one of these it shows one um panelist potential zone of transition but they also get this information on the reasonable range and that's their check like are you in where we're headed right are you in this reasonable range if you're not let's talk about why you're

235where you are but but this will be a a guiding um bit of information for panelists okay uh 46 um again the reasonable ranges are provided for all three cut scores you just saw one on that previous Slide the panels will do that for all three cut scores and they'll have multiple rounds to discuss and ask questions and and Converse with their peers and review and make adjustments if if they desire but that last little um M Dash says but not required right [Music] um panelists really don't like it when you tell them what they have to do because if you say you have to be in a certain place they say why did you invite me here right so you have to rely on their professional judgment and some of the panelists I'll predict

236somebody might be outside a reasonable range over here somebody might be outside of reasonable range over there but you're going to take the collection of all 12 to 15 people in the meeting of their judgment as the group's recommendation so it it it it you have handled a diverse panel for a reason right um if you just wanted one person to sing the song you could have just had a one person panel I suppose but there'll be some there'll some be some diversity of final judgments um um many I suspect will be within the reasonable range some I suspect might be out of it um but um that's that's a process where I think you have to respect individuals different perspectives on their task okay let's see page 4 47 um looks like it's all

237over um we're going to wrap up the meeting provide you with a full report on what happened it'll like we mentioned earlier be that table of demographics how many of these were invited how many actually showed up um was the procedure followed there's usually a um an option in here to have some kind of independent Observer often it's a third-party um research entity often it's a a member of the ad the technical advisory committee often it's a somebody who can just be there say you know this process went as you planned it or as is most often the case stuff happens on the ground you don't expect right some panelist has to leave for an emergency how did you react to that how how did you respond to that um so if there were deviations

238from the intended plan what were the deviations and did they seem appropriate and so forth it'll all be documented in a final report which will be produced for you um the technical advisory committee will review the process and make recommendations along the lines of this looks like it went as as it was intended to go and then the department will make final recommendations to the board for year review and decision making um let's see well I made it to slide 48 which is my favorite slide um any any other questions I'll be happy to answer anything about standard setting on the uh slide 47 the documenting the full report would that include for instance what were the reasonable ranges what was the reasonable range information that was provided to the various committees for their respective

239U assignments I can't say for sure um is that typically made public uh so um to I don't I don't know if I can answer that is there a reason not to make it public I don't think so all those committee members have it so and they're not under any kind of well they might be under an NDA not to theyd be under an NDA as to the content question well content no but also U we we we advise them talk all you want about the procedure and what you did but don't talk about specifics for example if a if a panelist goes out and says oh the cut score for it Advance is going to be between 46 and 47 right well that likely is wrong because it's not exactly the page number where

240they said it on it's the mapping of that page number to the ability level it corresponds to so they're probably going to be wrong if they said that they will be wrong if they said that second of all they're jumping your judgment right you haven't approved this yet so if they go out and they say it's this you say no it's not and then everybody's surprised when the eventual cut score that you approve is different from what they walked away with so there's a lot of things about the specifics of the process that may be subject not would be advisable for sure I I'm a double minds of this I'll tell you what the two minds are one is I don't know if it'd be advisable to give them like specific information about the reasonable

241ranges between page this and page that because Pages aren't the metric in which we set a cut score we set the cut score and the metric of the ability the underlying ability distribution that is on but the reasonable range information that's provided to the standards committees uh well what's gonna be provided to them is is is that essentially that yellow box says here's the reasonable range here and so I wouldn't want those page numbers exactly that's one mind of mine to say no that wouldn't be a good idea the other mind is I really endorse this notti of transparency I've heard a number of times and the more you can tell people what we did the more they go well that makes sense right and also I think it will inform I mean how whether

242you do it in the page numbers or translate into proficiency percentages or what but I when the board evaluates I mean everything the panels do comes to the board by way of a recommendation and the board should have as much background I mean yes when we've had it in the past we've had the art the original committee recommendations we've had the psychometricians analysis we've had um um last round last year's you know we've had a fair amount of information provided to the board such that we then make a decision and to me the reasonable range information would be an important piece I don't see any reason why um certainly in an executive session you couldn't be told the exact numbers of of what reasonable ranges were in terms of probably the ability metric well I

243I'll argue with you there just that if we're going to consider it actually I think it probably we would have to look to see if it came under some exception but my guess is it wouldn't that we would have to either do it in public session or not I may be wrong but yeah um I I would urge caution and there there's too much opportunity unless it's an in-depth discussion too much opportunity for misunderstanding what those reasonable ranges represent in terms of that they're given to panelists in terms of a a Zone in an ordered item set which has really not a perfect relationship to the underlying ability metric this this item response Theory Theta metric that everything is done in under under the under the hood so um I would be cautious in terms

244of the misunderstandings that would would take place there so but let me the second thing um you would you would absolutely get the predicted impact you would absolutely get um what percent is likely to be classified as below basic and basic okay good so we we could look for ourselves to see how that that's a non-starter that's got to be there yeah so we would get predicted impact based on this year's test performance on if you adopt this cut score this is what the percentage outcomes are going to yes exactly Mr R yeah I was just going to say I mean I feel like the we the materials that we're going to get in our meeting will be open the public can see those and so I feel like we can reasonably commit that with

245the caveats we need to protect IP and we need to protect there certain aspects of this we we can be maximally transparent within those parameters so people can understand the decisions that you know the they can at least understand the the information that we're we're using to make the decisions whether they agree with the decisions or not and I would just say um you know sort of in closing um I I think uh my my evaluation of this is that you've got a good process that will get you to where you want to go um I'm confident that those yellow zones represent Nik aspiration you know and I'm confident that this is a process that Educators for whom standard setting is not their first job right I'm confident a process has been designed that they

246can understand and Implement so they can bring their expertise to bear in a conscientious way that results in something that they can stand behind in fact what you'll have in your final report also is a survey of all the people who participated asking them what did you think about this did you understand the process do you stand behind the cut scores you've set and that sort of thing so um uh you will get a a lengthy and comprehensive summary of everything happened in order to inform your judgment further comments or questions Dr Northern I just want to say thank you what an informative session honestly when I first saw the slides I'm like what's he going to say differently that Chris already told us um but you told you added a whole new layer to

247the conversation um and got us talking and and really substantively so thank you for a wonderful presentation appreciate any other comments here questions thank you so much for being with us today your business thank you very much um and I think that ends us for the day um we will see you back here tomorrow at 900 a.m thank you e for