State Board of Education Meeting November 8, 2017

001All right, the State Board will convene with a flag salute. I pledge allegiance to the flag of the United States of America and to the republic for which it stands, one nation under God, indivisible, indivisible. All right, I have some announcements to start, and I will begin with welcoming somebody new to the State Board of Education staff. Kiyomi Meeker will be joining the staff as for project director of the public charter school program in October where she came from and she will be working with Carolyn Pfister and Donna Neville in the State Board of Education office to help us meet program performance measures developed by California and the US Department of Education. So please join me in welcoming as a member of our important team. Please stand up. I'm very happy to announce the appointment

002by the Governor Brown of Vice President Eileen Strauss to represent the State Board of Education on the Education Commission of the States. She'll enjoy that. For the meeting today, Spanish interpreter services will be available on today, Wednesday. There is one public hearing item on our agenda, item 18. That's tomorrow. There are three items on our agenda that have an addendum, item 1, item 2, and item 8. The consent calendar and waiver consent items are on our agenda and there's a handout on the table in the lobby listing them. Regular consent items are 9 through 14. Waiver items are 1 through 14. I want to remind board members to remove items from the consent calendar by Wednesday noon today. notify Executive Director Karen Staff Walters if you wish to do so. You have your electronic folders

003with the letters. We read those carefully. We have an electronic agenda that can be accessed on the web. I want to remind speakers that they should sign in at the table just outside the room and sign up on the pages for the item number for which you want to speak. And please announce yourself and identify yourself before speaking to the board. Item 19 is general public comment item, not for items not specifically on the board agenda. If anybody wants to submit additional materials at the meeting, please give 25 copies to our front desk here with our two representatives at the front. The we have additional TV monitors in all of these Rooms to my left and if we have overflow we can you can see the board meeting on the television there finally Individual speakers

004will be limited to one minute each for public comment for all items Wednesday and Thursday for items three and four today And only for those items today, a group of five speakers may sign up together and designate one speaker who will be allocated a total of three minutes for the group. So with that, we're ready for the superintendent's report. Good morning President Kirsten, board members, good to be back with you. Had a number of things that occurred over the last month. The toughest issue to deal with was the fire, the devastating fires in various parts of the state, Anaheim Hills, and then of course Sonoma and Napa counties. We're working diligently to get students back to school. It was an incredible, devastating, terrible tragedy that occurred. I heard on the radio early that Monday morning

005that there were some fires. I turned on the TV and I could see that this was going to be a pretty horrible... tragic situation. We ended up having 390,000 students at one time out of school because of the fire risk and as well as the toxic risk from air pollutants. So we worked hard to get the, our main goal was to get the schools back open again, make sure they were clean and clean up the toxics along the way. 350, 650 schools closed. I called early on Barbara Nemko from Napa and Steve Harrington from Sonoma counties and their offices of education got involved very quickly and efficiently. There's such teamwork between the emergency response teams and the educator teams throughout the areas that were affected by the fires. We continue, I asked early on to

006the superintendents of the other counties, what could we do, CDE, to help. We continue to offer assistance to educators in Butte, Lake Mendocino, Napa, Orange County, and Solano County. As well as Sonoma these again displaced hundreds of thousands of students Sonoma County Superintendent Steve Harrington was tireless and coordinated the flow of information services to hard-hit districts again Visiting the sites it was just unreal just going in a day or two after Fire had devastate areas in some areas. It was just straight devastation others some homes were spared and there others went up in smoke In one area in Coffee Park in Santa Rosa, 800 homes burned, but the elementary school, Schaefer Elementary, survived almost unscathed. So there was a lot of ironies as it went forward. We immediately brought in, with this top priority, getting

007schools back to normal. We did a survey. Superintendents were calling, how can you get a hold of portable classrooms? The state used to have a program where we had a fleet of portable classrooms. ready to deploy but that program long ago went by the wayside so it was really it's been a scramble to get enough temporary classrooms 43 people perished in including kai shepherd and his sister cressa students at eagle peak middle school in redwood valley up in mendocino county again the damage is staggering and the road to recovery will be a long and hard road and we'll see you know 8,900 homes burned. That was students affected, but also teachers and classified staff throughout the region lost everything you can imagine, all your personal belongings and history. So I was pleased again and amazed

008at the teamwork that occurred. There were so many acts of heroism, so many acts of taking risks to help others, neighbors helping neighbors. CHP in the first few hours with their helicopter fleet came in and evacuated 41 people who were in harm's way. We had the Sonoma County Developmental Center. And had the CHP and school buses helping evacuate 200 people from that center. And then there were many people who've reached out to help. Some of the teachers have had an exercise where the students write about their experience so they could express themselves. We're bringing in a lot of extra counseling to help some of the students. were asked to write about what was the most important thing that they lost in the fire, the physical object that they might have lost in the fire. And

009one high schooler mentioned his electric guitar and amplifiers had burned up. Somebody donated him an amplifier and guitar, and an elementary school girl's favorite instrument was the ukulele, and someone bought her a ukulele. So there's a lot of acts of kindness going on in the middle of this. I do want to thank the CDE team. We were there at the Office of Emergency Services from the very get-go that first Monday on. Juan Morelis, Leslie Taylor from the School Facilities and Transportation Division, along with Diane Waters and Sandeep Kaur helped with nutrition. We brought in 31,000 pounds of food to the evacuation centers and again offered all help we could. Early on, I was pleased that these agencies put the top priority to bring normalcy back to the lives of the students and families as best

010they could and to get students back in school. So we created a task force working with OES. We had an education team. Glenn led, co-chaired the effort along with Carolyn Thomas Jacobs from OES and coordinating efforts. Where's Glenn? Right there. night and day working with Cal Fire, with the National Guard, with Cal Toxics, Cal EPA, coordinating all these agencies and it was amazing working with the local OES offices and the statewide offices. So we provided information on the website very quickly and again, I just want to say thanks to all the people who helped. Our hearts go out to the victims and our prayers are with them as they recover. There's a long road to recovery, but we'll stay with them along the way. And teachers and administrators and classified were phenomenal in their helping

011students and families through this tragedy. With that I move to the CAASPP update. We have a key agenda item today, so I won't be long on this just to say it was an enormous undertaking. We had 3.2 million students tested on a single day in May 9th, 2017. Nearly 500,000 students took the online test, the largest single day of any such assessments ever anywhere. I want to thank Kerrick Ashley, Michelle Center, and all the CDE personnel who helped on this extraordinary effort. And more discussion ahead in the agenda. Suspensions and expulsion, some excellent news. I announced the numbers of students suspended and expelled declined for the fifth year in a row. Efforts to improve attendance by using more engaging instruction and more effective discipline and counseling are making significant progress. From 2011-12 to 2016-17, we've

012had a decline of suspensions by 46% and expulsions by 42%. And now we have detailed information due to the way we're collecting the data that includes suspension rates of student groups such as male, female, Latino, whites, African Americans, students with disabilities, foster youth, and students from low-income families. And locally, this data is being used to help provide action plans to further reduce absenteeism and help those students who would normally be suspended or out on the street without a caring, supportive environment. Last two items, Beyond Safe Havens, Back to School. We worked on an internal survey. We'll talk about that as the day goes along. CDE posted Beyond Safe Havens. It's part of our Back to School festival that we had. We had Attorney General Javier Becerra, Sacramento Mayor Daryl Steinberg. I want to thank Felisa.

013Ortiz-Laconne for being there and being a dynamic panelist advocating for our immigrant community. So thank you for your representation of the board and the work you do with the immigrant communities. We continue to hear reports about anxiety and fear. about the new federal immigration policies, bans on travel, and weakening of protections from discrimination. Some of the students or their relatives could suffer serious harm, will suffer serious harm, I believe, with the end of DACA. But we're not going to say that it's over. We're going to fight to get it back. Of utmost importance recognizing that a positive school climate and creating a welcoming inclusive environment is critical to the success of all students and this is backed up by a recent poll by Ed source in conjunction with the Institute of government studies at UC

014Berkeley the number one priority of parents and citizens surveyed 74% was safety and positive school climate and we'll talk more about the school climate issues later today At CDE, we're also doing an online department-wide climate survey to learn more about how we're doing and how our employees want to see what they want to see happen and how they want to help and how we can advance our cycle of continuous improvement. Finally, the system of support is coming together strongly. We're going to get more details on that, but I want to thank the State Board and your staff, the CESSA, working together with CCEE and CDE. We're really getting a coordinated, thoughtful, effective approach ready to go. We are finally... Continuing our resistance and persistence to make sure the federal budget cuts that have been posed

015do not happen. So at risk we know is Title II, $252 million for California in professional development, and Title IV, $135 million in after-school programs. But we're hoping and we're sensing there may be a bipartisan support for not implementing these horrible cuts that have been proposed. With that, we look forward to a productive meeting today. Thank you. Thank you. We're now ready for item one, and that is the selection and the recommendation of three finalists for submission to the governor and the action of the screening committee. And so Board Member Burr, who has spearheaded that effort and for many years for the board, will present this item. Thank you, President Kearse. Good morning, everybody. I don't know about you, but this is always my favorite meeting. It's the meeting at which we recognize all of

016the hard work of our students. I'm always reminded that we have such a bright future ahead because when we get to spend time with the students that you're going to hear from shortly, you realize how energetic they are, how engaged they are, how creative they are, how intellectual they are. And it's just a wonderful experience. So I'm pleased to be able to lead this effort. I want to say thank you to my board member colleagues who help every year, Board Member Holliday, Board Member Ortiz-Lincon, and this year, Board Member Gray, who I think helped mentor and was active in all of the presentations and the meetings that the students have been engaged in over the last couple of years. Just a brief summary of how we get to this point. As you remember at our

017last meeting, we asked for applications for the student board member slot. I'm pleased to report we got over 60 applications. They came from over 20 counties in the state. So there's a very high interest. Anybody who worries about civic engagement should take part in this process because it's very illuminating and very gratifying. Our job is the screening committee is to review all the applications and to recommend 12 candidates to the California Associated Student Councils and the Student Advisory Board on Education Conference. They then go through quite a rigorous process, and Jayden can share that with us if you'd like, to identify the top six candidates who then the screening committee interviews, and it's our job to recommend the top three finalists, and those finalists go to the governor and the governor will actually make the

018appointment in the spring. So I again just want to express my appreciation to my colleague members. I would like to at this point to recognize all of the candidates. I will start with the folks who were interviewed and narrow it down to the top three and at that point I will make a motion and ask that you approve our recommendations of the top three. So I'm going to start and we have certificates for everybody who participated. I'll call them up. one by one. And I'd ask you if you wouldn't mind standing up here after you get your presentations and certificates. And at the end, we'll have a group picture with President Kirst and Superintendent Torlakson. So the first semifinalist I would like to recognize is Juan Flores. Juan is a student at Valley High School

019in Elk Grove School District in Sacramento County. I also neglected to tell you that you do have an item at the end of them if you would like to follow along. The next semifinalist is Peyton Fuller. Peyton is a student at West Valley High School in Anderson Union High School District's Shasta County. Our next student is Connor Gussman. Connor is a student at C.K. McClatchy High School here in Sacramento City. Our next semifinalist is Camille Hermosillo. Camille is a student at Arrupa Valley High School in Arrupa Unified School District, Riverside County. Our next semifinalist is Brandon Plowman. Brandon is also a student at West Valley High School in Anderson Union High School District, Shasta County. And our last semifinalist is Denise Virgin. Denise is a student at Soledad High School in Soledad Unified in Monterey

020County. Our next three candidates were finalists. They were identified as candidates, and we had wonderful interviews with them. I would like to call up Elisa Escamilla. Elisa is a student at San Marino High School in Los Angeles. Next is Aaron Gelberg. Aaron is a student at the Academy of the Canyons in William S. Hart High School District, Los Angeles County. And our last candidate is Arvin Hariri. Arvin is a student at Berkeley High School in Alameda County. And now I'd like to recognize the finalists that we're recommending go forward to Governor Brown. First is Hema Cardenas. Hema is a student at the Life Academy of Health and Bioscience in Oakland Unified in Alameda County. Our next finalist is Lauren Kong. Lauren is a student at Palos Verdes Peninsula High School in Los Angeles County. And

021last but not least is Crystal Sung. Crystal is a student at Classical Academy High School, Escondido Union High School District, San Diego County. I should have said, and you can certainly see that the wonderful thing about these candidates is they reflect the fantastic diversity of our student population in California. And for that, we're very grateful. So I'd like to make a motion that we forward the top three candidates to Governor Brown for his consideration as appointment of student board member next year. Second. All right. Is there any board discussion on this? Do we need public comment? I don't think so. Okay, no public comment. All those in favor, raise your hands. Unanimous. Thank you. Congratulations all. And if President Kirst and Superintendent Tarleton have been... Thank you so much. We're going to have a quick

022break. President Kirst and Superintendent Tarleton. Hey, we're now ready for item two, which is another long-awaited item for us. And this is the report from the 2017 Student Advisory Board on Education. And our liaison, who does much, much hard work on this, Bruce Holliday, will introduce this item. Thank you. For those of you who may not be familiar with this annual agenda item, I'd like to take a moment to explain that for the past several days, the California Student Advisory Board on Education Conference, in conjunction with California Association of Student Councils, has been taking place here in Sacramento, attended by some of the best, brightest, and highest achieving students from all over the state. In teams, they work long hours each day at the conference to arrive at solutions to and suggestions for education issues

023that are important to them and to all Californians. This agenda item allows all of us to hear directly from the students, and it is always an enlightening and important opportunity. I had the pleasure of hearing a preview of the presentations yesterday, and I know you will be impressed with the students, their suggestions, and their ideas. So the students will come forward to present this. Thank you, Board Member Holliday and President Kirst. Good morning, distinguished members of the guest, distinguished board members and guests. My name is Maury Leveroni, and I'm currently a senior at Hamilton High School in the Hamilton Unified School District. I humbly stand before you today as the state president of the California Association of Student Councils. A couple of thoughts before I pass it over to Education Policy Director Ryan. On behalf

024of the California Association of Student Councils, I commend the State Board and staff thereof for their efforts in disseminating the applications for the student board member on the State Board of Education. Furthermore, a special thanks to board member Bruce Holliday for his continuous support of the Student Advisory Board on Education and... his efforts and being on our student panel last night. Without further ado, it is my pleasure to introduce Education Policy Director Ryan Chen. Good morning. Before I begin, I forgot to give this to him. Thank you, Maury, for that lovely introduction. That was very nice of you. Good morning and thank you Superintendent Torlakson, President Kirst, Vice President Strauss, and the STEAM Board for allowing us this amazing opportunity once again to speak before you on behalf of all the students in California. My

025name is Ryan Chen. I'm a senior at La Cunada High School in La Cunada Unified School District in LA, the Educational Policy Director for CASC, and the Director for the Student Advisory Board on Education Conference. 2017. For over 40 years students from all over California have kind of gathered and come together to evaluate the state of their public schools at this annual conference and work together to give feedback to at this meeting and a presentation. They've debated, they have argued, they've reassessed and they've come to consensus to reach the point that they are today. It's a rigorous, lengthy process. Some would say it's even jam-packed. But in the end, the finished product is well-developed and representative of the group, so it's very well worth it. Before we begin, one thing that I would like to

026point out about this conference that's maybe different in years past. This year, while in previous conferences, the topics have been pre-selected so that we have research team members who can do more in-depth research prior to the conference. This year, the decision was made to forego that and instead let the delegates choose directly. what their topics will be for this presentation. And that decision was made even though it added a lot of stress to the members of the research team. This decision was made so that their authentic voice could be conveyed to you and that their concerns could be conveyed directly to you. Today we will be, the five issues that we will be presenting are in front of you. They are in this order, restorative justice, student representation, CTE, counselor-student ratio, student-counselor ratio, sorry, and

027mental health. And then one final note before we begin. Over the past few days, I've learned that the delegates standing behind me are bright, they're talented, they're spunky, and they're always open to listen and learn. They're zealous for life, their passion for change, and their willingness to put in the work have really inspired me and they've inspired all the other staff members of the conference. And I know that as educators, administrators, and board members that you can appreciate the pride that I have in seeing their progress and their growth. After all, this is why we teach and this is why we educate. So before we begin, can we give a round of applause for the student delegates behind me? With that, I would like to introduce our first speaker, Mindy, who will be presenting a

028proposal on restorative justice. Good morning, honorable members of the State Board of Education. My name is Mindy Jun, and I represent Troy High School and the Fullerton Joint Union High School District. In the words of Tom Erner, the former president of the National Association of State Directors of Special Education, if a child doesn't know how to read, we teach. If a child doesn't know how to multiply, we teach. If a child doesn't know how to behave, we punish. How is this inconsistency in student treatment moral? This is neither fair nor the right way to approach the problem. Instead, it simply perpetuates the issue. I stand here today to shed light on the issues that cause the students of California to fall in a vicious cycle, bouncing between ineffective disciplinary actions as well as expectations of

029failure. Here at the Student Advisory Board on Education, the student delegation has come to a consensus that in California, standard disciplinary procedures in education, such as suspension or expulsion, do not serve their intended purposes. In order to effectively discipline and prevent further misbehavior, administrators must take action other than suspension and expulsion. This is because even where schools are not mandated to consider alternative means, suspensions and expulsions are so damaging that the administration needs to consider restorative justice in order to solve the issue. Restorative justice is formally defined as a system of criminal justice that focuses on the rehabilitation of offenders, the reconciliation with victims, and the community at large. The programs used in some districts include restorative discussions, dialogue circles, peer jury, and mediation and conferencing. According to the California School Boards Association, the disciplinary

030actions taken to punish poor behavior led to an increased risk of dropping out, repeating a grade, coming in contact with juvenile justice systems, second-time offenses, and lower test scores. While requiring schools to implement restorative justice is outside of the scope of the State Board of Education, there are still actions that can be taken by the board to take the first steps towards our goal. Evidently, harsher punishment like that of the current retributive actions needs to change in order to see any significant progress in correcting problematic student behavior. We firmly believe that restorative justice fulfills the duties that suspension and expulsion both fail to do. Restorative justice boasts a significant decrease in recidivism and dropout rates. Currently in California, the youth recidivism rate is at a 90%. However, youth who have completed restorative justice programs lower

031this rate to a range of 10% to 30%. In Merced, restorative justice reduced suspensions in its schools by 40%. Clearly, restorative justice works, and many members of the California State Legislature has noticed that, as shown by the Senate Bill of 607, which was introduced this year. The bill proposed to eliminate the authority to suspend or recommend expulsions for willful defiance of any pupils in K-12. It also encourages superintendents to provide alternatives to suspensions and expulsions. We suggest that the State Board of Education recommends the adoption of restorative justice programs as the foremost precursor to orthodox disciplinary procedures in all school districts for all misconducts. When we say any form of misconduct, we include the students who are exempt from the alternative measures in the current education code, which disproportionately punishes students for committing any acts

032listed. These acts range from profanity and vulgarity to their willful use of violence. This proposal allows each district to have the ability to choose from a variety of restorative justice programs to enforce them into their own schools, allowing them to customize their own restorative justice systems for their own students. We recognize that the State Board of Education does not have the ability to modify the current education code, nor is it able to mandate our proposal. So instead, we asked the State Board of Education to propagate our idea. With the State Board of Education's influence as leaders in the education system, voicing what we propose could take California one step closer to implementing restorative justice in many more districts across the state. And as a result, we would be able to reap the benefits that are

033already seen in districts that have implemented it. In one specific instance, a student in Oakland Unified had a 0.77 GPA and severe anger issues. After an incident in which he was charged with assault with a deadly weapon and making criminal threats, he was sentenced to probation. He went through restorative justice and since, he has not reoffended. Now, he has a 3.27 GPA and looks to attending the University of Florida. Hearing stories like these, the Student Advisory Board of Education recognizes the obligation to improve the lives of students across California. We need your help. Thank you. Oh, you have questions? Okay, questions? Member Rucker. Well, first of all, Ms. John, I want to thank you for the work that your panel did. Are any of the other folks who worked with you on this recommendation here

034in the room? Yes. Could I ask them to please stand? And while you are correct that the board doesn't have the authority to implement your recommendation as a mandate, we can participate in propagating the idea. I did have an opportunity since the previous board meeting to visit a high school here in Sacramento County that's implementing programs similar to what you are recommending. It's in the Sacramento City Unified School District at Luther Burbank High School. And you all are correct. The work has an influence not only on the students, but it changes the culture and climate. of the school. I had an opportunity to watch programs being implemented at that high school that integrated practices of restorative justice as part of direct instruction in every classroom I visited. And there were particular classrooms added to the

035master schedule as electives that targeted students such as the one that you identified and spoke to as an example from Oakland Unified. And your panel is correct. These practices do make a difference for students. And it also changes the relationship that the school as an authority figure and their educators have with all of the students in the school. And more particularly, it changes the relationship among the students themselves. So I really want to congratulate you all for taking on a very courageous conversation, not only about the impact it has on your peers, but the sustained effect it has on schooling in California. I think it's a wonderful example of innovative thinking, and thank you very much for doing that. Thank you. Other board comments? Board Member Sandoval. Thank you so much to all members of

036your team for this wonderful presentation and proposal. And I definitely support the propagation of this idea because what restorative practices do, and I'm actually a trained restorative justice facilitator, and I utilize restorative practices in my work as an educator, as a manager, as a leader. And what it does is it fosters a sense of mutual accountability. that we are all responsible to each other as human beings. And it's less about shame and blame and punishment and more about what can I do to make it right, because we're all going to have to live together after this incident. So thank you for your, again, courageous proposal. And I would say that I think it goes beyond programs. It's really we're looking at a culture change. climate change, a real shift in how we approach discipline and

037consequences. So thank you for that. But I agree that isolation and punishment is not the answer and that we have to really restore and repair our relationships with each other in order to survive. So thank you very much. Board member comments. If not, thank you. And we'll have the next presentation. Good morning Superintendent Torlakson, President Kirst, and Honorable Board Members. My name is Christine Kim and I'm a junior at Lee High School within the Campbell Union High School District in the Santa Clara County. Me and my My peers and I propose the creation of a youth advisory council to serve as a support system for the California State Student Board member. These students to be appointed to a formal position by the State Board would serve as an essential resource to the student board member

038to consider the diverse needs of the students in California. The student members position is unique compared to the other positions in the State Board of Education. In addition to having a valuable perspective on educational policy, the student board members main role is to be the voice for the 6.3 million students in California while simultaneously fulfilling their role as a full-time student. It is eminently clear that the student board member devotes a great amount of time to accurately represent these students in a very progressive state. However, one person cannot possibly represent all the different socioeconomic and cultural backgrounds that the state board must recognize. Each individual district has different needs and different perspectives that should be considered to provide students with the quality education that everyone really deserves. Because this representative serves as a liaison, between

039education on the state level and the local level, the representative should have access to a platform made up of a diverse group of students who can truly advise a student board member to be well informed to make decisions that truly embodies the various perspectives of the student. For example, in Maryland, the student board member on the state level has a youth advisory board that provides advice and different perspectives. Kyle Smith, the current state student board member in Maryland, describes this youth advisory board as efficient and effective. During my call with him, he stated, as I quote, In my role, it was really helpful because the advisory board kept me fresh. When you're in the boardroom, you tend to lose sight of why you ran for this position. That's why the advisory board makes it nice

040to go back directly to the source of student input. They provide a lot of new ideas that I would have never considered in the first place. They've given me inspiration in addition to support. Based on his experience, it is very clear that this that additional student input provides the student board member with a better understanding of the of his constituents and has a positive impact california is unique in that we are home to a student of a myriad of different backgrounds and cultures and is imperative that the student board member has access to these distinct perspectives based on these facts a youth advisory council panel would greatly augment the position of a student board member and help maximize their potential as a resource for the State Board of Education to properly make informed decisions that

041will truly benefit all these students in California. Having your support for this Youth Advisory Council would allow the student board member to receive a comprehensive understanding of the needs of the students. The State Board of Education has resources and contacts that surpass any other student organization and will allow students all over from California to receive the opportunity to truly voice their opinions. With the backing of the State Board of Education, the current and future student board members will have a sustainable yet constant resource for generations to come. The support of the State Board would allow for a fair application process that creates opportunities for involvement from students that need to be heard the most. We recognize that at the end of the day, the student board member is an independent vote and an independent thinker.

042But this advisor committee will act as a supplementary opinion that will help him or her contextualize the effects of the decisions. I urge you all as a state board of education for you to be an advocate of student voice and student representation. My peers and I would love to be of assistance to the state board and truly work to implement this plan in the near future. Thank you, and I yield my time to the floor. Thank you. Board member comments. Board member Gray. First, I'd like to thank Ms. Kim and I'd like to ask any other delegates who helped her with the proposal to please stand. First, I'd really like to thank you for the proposal. I think up here as a student board member, like you said, and like Kyle said, it's easy to

043lose sight of why you ran here, ran for the position in the first place. And moreover, it's hard to really contextualize, I guess, the impacts of the decisions I'm making. So I think having that advisory council could help a lot. I also think that... If the state board could back it I think it does two things that are extremely important I think one it allows it first it allows sustainability so that when I'm out of this position and the next student board member the next student board member is in my shoes that Council will be able to help them and continue to help them grow on their position and help them gain new perspective But also I think it's gonna allow reach and I love what you said Making sure that the voices that need

044to be heard the most are heard I come from a certain background and I can see things from my perspective, but it's really important that I'm able to understand how it affects everybody. And if the State Board can back this, I think that people whose voices oftentimes aren't heard will have the opportunity to voice their opinions. And I think that's so important. So thank you. Other board member comments? Board member Ortiz-Lacombe. Thank you for your presentation, and I think it's a really good idea that I think it would be worth exploring. I'm not sure what it would entail. I know that in my professional work, I work with youth, and we have a youth advisory council that's national, and it really helps for some of us who graduated high school. you know, 20 plus years

045ago, just to try to keep that relevant and important voice in our decision making. So I think it would be great for us to explore. I know that Jaden and our previous student board members have done an excellent job, but I know that sometimes, you know, we also need to consider other communities, other populations and other students who are very worthy of being part of this group. But, you know, we could only have one at this point. So, yeah, I want to support that. And I'm not sure what that. I wouldn't tell, but thank you. I think it's a very innovative idea and we should definitely look to pursue it. Thank you. No other comments? We're ready for the next presentation. Thank you again. Good morning, Superintendent Torlakson, President Kirst, and esteemed members of the

046board. My name is Teoman Tezjan and I'm from Berkeley High School in the Berkeley Unified School District. 47 million job openings are projected for 2018. These jobs are the professions that our education system needs to prepare its students for. And the California Department of Education recognizes this, and rightly so, as a state priority of college and career readiness. One avenue which the California Department of Education has identified to address the matter of college and career readiness is career technical education. Indeed, all CTE pathways are standardized to direct pupils to careers which will be viable for the next 10 years as detailed in the current CTE model curriculum standards. The 47 million job openings projected for 2018 will require real-world skills, skills that CTE addresses. Therefore, CTE is necessary to prepare students for the future. On

047behalf of the Student Advisory Board on Education, we propose that the State Board of Education uses the College and Career Indicator from the Dashboard to identify schools falling from median to very low levels, as represented by the yellow to red indicators. We recommend that the State Board of Education endorse career technical education to increase college and career readiness and identify best practices for schools which fall below the median of this indicator. These best practices should include a recommended increase of CTE courses at these schools as determined by the SBE. CTE programs support all students in preparing for their careers. For example, CTE is bound to have enormous benefits for those students who cannot possibly afford to attend college. According to the California Department of Education CTE fact sheet, 95% of non-college attending CTE students were

048employed within two years of high school graduation. In addition, since CTE courses not only emphasize career skills, but are also aligned with academic standards, they do not take away from the academic focus necessary to prepare students for further education in their field. For example, the CTE pathway in engineering at Beaumont High School and Beaumont Unified School District allows for students to find employment directly after high school via certifications that students may acquire, while also preparing students to pursue engineering at the collegiate level should they choose to. Thus, CTE courses help students meet both college and career readiness standards. Unfortunately, not all schools provide students with the same experiences as Beaumont. Many of my peers reported that their schools had either no career readiness programs or underdeveloped ones. For example, students mentioned their schools only had

049semester-long CTE courses, which lacked concrete connections to careers, and others explained that while their schools had a CTE pathway, one that connected to a specific career, they had but one pathway, so they couldn't serve the diversity of career interests held by students. Thus, career technical education is a necessary force in improving the career readiness of all students in the great state of California. We need to make students' education more relevant. Without career readiness truly incorporated into their instruction, students don't see connections between their education and their future. By incorporating career technical skills, students are invested in their own education. Seeing as career readiness is identified by the State Board of Education and the Student Advisory Board on Education as a priority for students, We recommend our proposal and effective expansion of career technical education as

050not only a supplement but a necessity Thank you for your time. I now yield the remainder to questions or comments. Board member Strauss has a question Actually, I want to really compliment this group in looking at our dashboard and taking a look at our recommendations and recommending the extension and enhancement of the college and career readiness. And I just want to support the students' comments about the future is changing, the careers are changing, and the point that career technical education actually helps all students and to identify exemplars for schools to take a look at and to look at some standards. It's an area that we are working on that the staff has identified as a need to continue. and I just wanted to support the students recommendations and particularly how the committee I know you're

051representing the whole committee so first let's see who the whole committee is who represented and helped develop this recommendation so I want to thank you all So I know having been part of the work, you all participated in the research to support this recommendation. So just to support you that you had good rationale, you supported it by looking at research and you gave us data. And so just a really well-prepared recommendation. And I just want to assure you that that's clearly an important priority for the board and the staff as we look to college and career readiness and preparing all of you and your siblings and friends who will come behind you that they'll be ready for the future. want to personally thank you for that work. Board Member Rocker. I also want to thank

052you for your recommendation. And I really appreciate knowing because of the change in the format that you were and you were able to select your own topic. This was a very authentic conversation in my own professional career. I have advocated steadily and continuously that there's balance in considering how high schools prepare students both for college and for career to enter either the workforce after high school graduation or to go on to college. What I'm most. appreciated about your presentation. Not only was it grounded in research, but it was connected directly to what this board, quite frankly, is still struggling with. While many people talk about a college and career indicator, the only conversations they collect data on or want to focus on are the college readiness information. But the fact is, you're quite right. There

053are a lot of organizations and post-secondary institutions, especially our community colleges, that have a lot of information. about high schools preparation for career readiness for their college high school graduates and I really appreciated how you talked about the transition from high school into career into the workforce and that the options remain option open for students to continue their education and that the focus is on readiness and continuing growth what and finally I want to close by saying it's really important to hear from students the things that adult are struggling to quantify and that policymakers continually look at but never give voice to. So I really appreciated in your report how you truly focused on drilling down to what is essential and important to the students in your school and the very specific example that you

054gave from Beaumont. That's an exemplar for a very good and effective practice. So thank you very much. Superintendent. Well, thank you and excellent excellent report and research and I agree with everything that members Strauss and Rucker just shared with you to let you know that our Department of Education has implemented about one and a half billion dollars of career technical education grants But it's been something new over the last five or six years and it's really taking a hold and Graduation rates where the state average is all-time high at 83% career technical education students who go through a series of courses 95 to 100 percent graduation rate. So everything you're saying is right on target. Some work is being done to analyze the current programs to see how effective the money was invested by different

055groups, which include business being part of the conversation with community colleges and high schools. And I urge you to connect to Senator Mike McGuire. He's a champion in the legislature. The current grant program is winding down. I think we're getting the last dollars out in actions we're taking over this next week. then the program sort of stops. And I think your timely commentary is really helpful. Stay focused and talk to Senator Mike McGuire. I think he'll be a good ally to your goals. Okay, if no further comments, we're ready for the next presentation. Good morning, Superintendent Torlakson, President Kearse, and esteemed members of the board. My name is Brittany Wong, and I am a junior at Lincoln High School in Stockton. Over this conference for the past few days, I've witnessed how my peers have

056expressed how they've been negatively impacted by the lack of attention given to them by their counselors. Since counselors are spread so thinly, they frequently make mistakes with students and their classes. Necessary credits get lost, beginning to affect students' grades and grade point averages, which directly correlate to the opportunities for a higher education, leading to a direct impact on the careers and the futures of students and their livelihood. I stand in front of you today to bring to attention the challenges that students across California face. Currently in California, we have a student-to-council ratio of 460 to 1, which puts us second highest in the nation for student-to-council ratios. We are currently down from the previous ratio of 945 to 1, which made our state average the highest in the nation before May of 2016. These numbers

057show progress, but now the duty falls on the State Board of Education. to advocate for students and ensure that schools are working closer to achieving an ideal ratio that equally shines alongside the national average of 477 to 1. Placing the responsibility of an overwhelming number of students and their individual needs under one person is irresponsible and an unrealistic ideal that leads to a myriad of issues. We, the Student Advisory Board on Education, feel that our schools High student-to-counsel ratio has critical negative impacts on the careers and futures of students. In turn, counselors hold a crucial role in setting up students for successful futures. It would be imperative that the students of California receive the adequate help they need academically from their high school counselors in our current education system. In our current system, counselors are

058unable to equally divide their time among students of all academic needs and specifications. We, the students of the California education system, feel that as the primary stakeholders that it serves, we deserve to have an adequate number of counselors per student that can advise us in maximizing our potentials. We strongly recommend that the State Board of Education ask districts to put the student-to-counselor ratio alongside... ratio of high schools among the top priorities when allocating funds under LCAP just as they do with the student to teacher ratio. We ask that the State Board of Education help us to advocate as we seek a limit to be implemented in districts statewide for the student to counselor ratio within high schools. These numbers can be estimated by the feedback from the school climate survey as they are being administrated

059in schools. With our current system, counselors are overwhelmed by the high ratios which limit the amount of attention one counselor can allot per student. Due to the high ratio, students feel that counselors only direct their attention towards certain demographics of students, which as a result leaves the remaining students without personal connections to their counselors, creating less of an impact towards a school's goal of a higher education rate. Because this ratio is not currently displayed on district websites and schools don't provide the information, the severity of this issue remains unknown. At Chaparral High School in Temecula, where they have a 791 to 1 student-to-counsel ratio, it took students up to a month to arrange physical meetings with counselors. All high school students demand equal attention from their counselors regardless of grade level. Freshmen need help from

060their counselors in adjusting to new academic demands of the rigorous coursework, and students and seniors demand equal attention as they begin to apply to colleges and prepare to enter the workforce. The serious needs of both of these groups show how counselors are equally important in all stages of high school. Therefore, it is imperative that there are enough counselors to ensure that all students are engaged in pursuing their academic aspirations and are making sure they stay on track throughout high school. We found that on our own campuses, the weak relationship between students and their counselors are having negative impacts on the futures and the careers of students. When you compare California, who is ranked second highest in the nation, to Vermont, who has an average ratio of 202 to 1, the benefits of having a low

061ratio become apparent. To complement their low ratio, Vermont is also top five in the nation for achievements in math and reading and has one of the highest high school graduation rates. Schools are able to promote the general welfare of their students by adjusting their ratios in order to create more personal relationships between counselors and the students they serve. Counselors serve as an integral part of student success and future planning, similar to any teacher. At Middle College High School in Stockton, students feel that counselors know them personally as people as they are able to attend to their issues personally because of the 240 to 1 student to counselor ratio. Teachers and counselors are equally important in their roles and services they provide to high school students. Therefore, it would only be fitting that funding be allocated

062under LCAP where schools are held responsible. As schools consider these issues and take steps into enhancing their counseling programs, they make direct investments into their students' success. By adding more counselors, schools are able to ensure their students are equipped with the correct necessities for their futures. We acknowledge this code is not within the jurisdiction of the State Board. Again, we stress that you use your position as the State Board of Education. to further spread awareness of this issue through a legislative platform by promoting this issue to districts. The student to counselor ratio is not well discussed in our education system and since counselors possess such a vital role in guiding students as they craft their futures, it is only appropriate that you take the action you can or give support in any way you can

063towards recognizing the severity of these issues. Thank you. I now open the floor. I know questions or remember Thank you so much. It's funny, my daughter actually graduated from Chaparral High School, so I understand the need. She came from a family where we could support her, and she didn't necessarily need the counseling services, but that kind of ratio is very concerning. Even though what you're talking about is a local decision, it's about bringing information forward to really encourage at the local level for the funding for more counselors. And one of the things that might be interesting is looking at the schools whose suspension rates went down. Did they increase their counseling? And so that kind of data helps bring forward and gives leverage to the sites when or to the districts when they're making decisions

064about counseling. But I applaud you and I think it's a very worthy and important cause that you're talking about. And I know at the principal level when we ask them would you rather have an AP or a counselor, in many situations they ask for a counselor. And as we look at the tiers of intervention and the multi-system of support, for tier two and tier three to be really successful. need counselors so thank you for what you're doing. I want to thank you and your committee so let's see all the students who helped you and I just want to talk about some of the main points of your direction so students in this group who talked about counselors I want to thank you all so you you supported your rationale with lots of specifics and on

065this Monday on the 6th of November I was in Santa Cruz County with their first ever counselor summit that they're hoping to do every year and talking about the need for counselors and the different work on that counselors do is very exciting and completely supported the idea of changing the role of counselors and the ways that students would be supported by counselors and They actually supported many of the details that you all live in your schools and so I think it's clearly an idea if we're gonna meet our goals of college and career readiness for all students the role the counselor makes such an important difference and And so I just wanted to support what the students said and to tell you that it is clearly on our radar as far as the need, not

066only the need of counselors, but what they do with their time. And we'll talk more about that in our next item. But I just wanted to thank you and support the work that you all presented today. Thank you. Thank you for an outstanding presentation. I really loved what you presented and you offered a very compelling. case. And I'm one of those students who didn't know her counselor. And I graduated high school, not prepared for college. By some miracle, I made it. I made it through. But I teach at UC Santa Barbara and upper division undergraduates. And every term that I teach, I have students who found themselves as first generation college students as I was. telling the same story over and over and over again, that they didn't have sufficient counseling support through high school,

067and they found their way into a UC by some other means. And it shouldn't be by miracles that we get there. There has to be a fundamental change in the system so that every student has access to those opportunities and that support. We all deserve to have that. So thank you for your outstanding work. Williams board member Williams. I want to thank you as well for first. Let me back up I love the fact that you chose your own topics. I think there's a richness and an authenticity to What you're saying and a passion that shows through that's very different sometimes in previous years that it's felt more like a academic challenge you know, here's a topic, how good, how well can you handle it? And this is much more moving to me to hear

068the topics and compelling to me. On this particular one, like Member Sandoval, I have personal experience that drives many of us in education is our personal experiences. When I was young, I also didn't get very good. academic counseling and I was the first in my family to go to college and it was a teacher that reached out to me that filled in for the lack of academic and career counseling that I should have gotten in the school and she helped me figure out what to go into and how to get into college and by doing so changed my life. changed my entire trajectory of my life. So I think it's incredibly important. In addition, now for the last five years, I've served as an education advocate and advisor for a low-income Latino family in South

069San Diego. And even though they're in a very, very good school, the guidance counselor to student ratio is too high. And not all the needs can get met of all the students, whether it's an emotional need or a kind of a nuanced academic need. They're generally doing pretty well, but they've got problems in one area, or they're going to be first in their family to go to college and their parents don't have a clue how to go about doing that. Whatever that is, there's not enough support even at that school. And that was a role that I've been able to kind of help fill in for these students and this family in the same way someone filled it in for me. when I was in high school and I have long felt like the ratio

070of academic counselors or guidance counselors which now fill more roles than just academic support that it's terribly important really important and as the as the as the world gets more complex and the career choices and job choices and the avenues to them get more complex, the need for more counselors per student grows more urgent. So I thank you for bringing this topic up and bringing it before our attention. Okay. Board member, it's on. Thank you. Excellent topic, excellent report. Thank you very much. And just echoing a lot of what my fellow board members have already said, I do think that this, the availability, training, access to counselors is the linchpin to a lot of our state goals. With regards to career... and college preparation, but also the social-emotional support and learning. We ask a lot

071of our counselors in school. So I think this is a topic that could be explored further. And I would just encourage that this topic maybe come up again for next year because I think there's so much more to this with regards to counselors and supporting the social-emotional aspects. And should we have counselors do both guidance and social-emotional? And what are some of the topics and training that we need to provide to our counselors? So I think there's a lot more here to explore as well. So thank you for raising this. Board Member Rocker. I also want to thank you for your recommendation. I read your... I have the opportunity and actually the good fortune to work in an elementary school district that hired counselors at all of their elementary schools. And so doing the work

072not only early on to provide support for the social-emotional learning of students in the early grades, but to make sure in particular as students transition from elementary school to middle school and high school that they have somebody who is able to guide them on making decisions about what their lives will be as they graduate from high school and beginning that process early on is important. So I want to thank you for paying attention to the school counseling need and the school counseling issue. Oftentimes when we talk about the threats to our workforce and the teacher shortage, we often forget to think about the shortage of other educators and in particular counselors. So again, I'm very glad that this year you all had a chance to pick your own topics and the fact that you chose

073to focus on counselors. lets me know we're not the only ones who are concerned about this and that there is something in the field going on that needs to be attended to. So thank you so much. Okay, I think that concludes that presentation, so we're ready for the next one. Good morning, esteemed board members. President Kirst. Superintendent Torlakson, and staff. I would like to thank you in advance for taking your time to listen to our proposal on mental health and public education. My name is Kimmel Batiste and I am a senior at the Athenian School in Danville, California. In both my community in Danville and my home community in Oakland, I see students thrive academically but struggle with various mental health issues. Listening to my peers who receive both public and charter educations talk about

074the constant stressors in their learning environment and the lack of mental health support through their education empowers me to stand before you today. I speak on behalf of the Student Advisory Board on Education and the California Association of Student Councils, which represents the 6.3 million forward-thinking scholars of California education, 2 million of whom suffer from depression. In crafting this proposal, my partners have shared some of their own observations regarding the lack of mental health curriculum on their school campus and the effects they have on the students and the culture of their learning environment. Matthew Huerta, an avid soccer player and a recent alum of Selmar High School, committed suicide roughly two weeks ago. His death has sparked huge conversation regarding mental health and the importance of equipping students and community members with the skills to

075become observers and upstanders for individuals battling with mental illness. His family shared that Matt, the youngest of six siblings, appeared to be a happy and normal kid, passionate about soccer, and excited to pursue his education at CSU Northridge in the near future. However, his relatives found a letter that expressed his desire for self-harm, and one day later, Matthew's body was found in the hills of Mount Olson. The Huerta family expressed that if preventative measures were in place, Matthew would have been exposed to different support systems, and teachers, students, and counselors could have detected early signs of self-harm. It is not enough for schools or districts to teach the importance of mental health when Sorry, it is not enough for schools or districts to teach the importance of mental health when a crisis occurs It is

076up to individuals like you and students like us to realize the inequalities within our state and make a positive and contributive change Throughout the state some schools have taken the initiative to implement mental health in their schools education Health classes have become a crucial vessel for this information The California recommended health standards include thorough guidelines for mental health curriculum. However, neither health nor health education is mandatory at this time. In meeting students from around the state over these past three days, some with health classes and some without, members of this council group found that students with health classes had a deeper understanding of mental health issues. In addition, there were some students who were required to take a full semester of health in order to graduate, and others had never taken a sex ed class

077in their life, let alone a mental health education. The discrepancies within the California education system regarding basic health classes is unjust. 28 states, from Washington to Arkansas to Indiana to Maryland, specifically require health courses as a graduation requirement for all public and charter schools. So I ask you, the board, why not California? The lack of accommodation or support for mental health when compared to the emphasis on academic success has a direct correlation with the fact that there is little curriculum for mental health and health education. This critical issue facing the state of California and the future of California education needs to be addressed. It is with that we present to you our proposal. The Student Advisory Board on Education recommends that the California State Board of Education urges the state legislature to mandate one semester

078or 0.5 units of health class curriculum in all public and charter schools. from middle school education to high school education. With your support, the Student Advisory Board on Education believes that individuals of the state legislature will feel inspired to change the ways of public education and implement mental health and health curriculum. Secondly, we hope that the California State Board of Education will recommend to all local districts to make this class mandatory in their schools. We fully understand that the State Board does not have jurisdiction over mandating statewide health curriculum to be implemented in all schools. But we believe that any input towards creating a healthier and safer environment for our students is putting us one step closer towards mental health awareness. We hope that you will join us in supporting Health as a state minimum

079course requirement and supporting health and wellness in California's education. Thank you Thank You Camille for your presentation and we do want to acknowledge those members of your group that were part of the research If they could please stand up You know, I can't speak to making this a requirement, but I could speak to the importance of it. We know that the number one youth group that suffers from suicide is LGBTQ. youth. We know that there's been more reports on trauma-induced factors that impact our youth and our students, whether it be fear of deportation or living in communities where there's high crime. You know, they've done plenty of studies that show that some of our youth exhibit the same symptoms as PTSD from, you know, veterans that are coming back from war. And so we know

080that there's a need for this. And I do want to say that the health framework will include some of these very dense but important topics. And we'll have more information on mental health. We'll have more information on trauma-induced factors so that they could be acknowledged and recognized. More strategies on how the trauma-sensitive schools can deal with that and address it in appropriate ways, age-appropriate ways as well. on gender identity, sexuality. So that is something that we are actively pursuing now and the framework is on its way and it's being developed. So just know that these topics, we understand the importance and they are being threaded into the framework. Thank you. Any other comments? I just wanted to comment that, oh sorry, Board Member Rucker. I also wanted to thank you for coming to and bringing

081this particular topic to the board. I think this is the first time since I've been on the board I've actually heard this topic brought up and a particular recommendation about providing instruction in the curriculum has been brought to the board. So I'm very excited to see this recommendation. Very excited to know that students have this concern as well. It's not just a concern that other policymakers who are outside the bubble of reality for you. at your schools and at the level that you operate with among your peer group that this is something you are paying attention to and these are very serious conversations for you all in your schools and in your community. So I was very excited to read this report and see this because as I said it's the first time we've seen

082this topic. It was a very unique discussion and a very unique presentation so I really want to praise you for doing that. I just wanted to point out that a lot of the issues that you have mentioned are issues before the legislature, as you indicated. So I think it would be good to think about how you can carry forward your work into the legislative arena as well as the state board area. So we want to thank everybody and all these students and all these great presentations today. So with that, I think the board will take a recess and we will come back at 10 o'clock. Okay, we're ready for item three. Good morning. I guess Superintendent Tarlekson must have gone outside. Oh, he's coming up. Good morning, Superintendent Tarlekson, President Kirsten, members of the board.

083I'm Cindy Kazanis, the Director of the Analysis, Measurement, and Accountability Reporting Division. And this is your standing item on accountability. It's a little hard to follow all of those wonderful kids. Students, rather, they did an impressive job. And I actually got my start through the California Association of Student Councils. So it's an association that helps breed leaders, for sure. Even decades later. So we have a lot to cover this morning. We're going to start with the technical stuff, so you're going to have to bear with me for several slides of numbers and charts and graphs. I don't tend to do this, at least with this body, but I think what you're deciding, it'll be helpful in the discussion. So the idea today is to walk through each of the attachments, pause after each of the

084attachments for clarifying questions from you all, and then at the conclusion of the attachments to open up for public comment and then for the board to deliberate. So sometimes we do it this way, sometimes we wait until the end, but they're distinct enough, and again, some of these are more technical than others, that that's how we'd like to handle the item today. So you're being asked today to take action on three items. The first is the academic indicator to approve new change cut scores, new high and medium status cut scores for mathematics, and a new color designation for the 5x5 grids. For the chronic absenteeism indicator, we're asking you to include information in the fall 2017 dashboard that will redirect users to the chronic absenteeism reports on DataQuest. We're also asking you to direct us

085to develop a recommendation for you to take up in March of 2018 on the proposed status cut scores that we will then use in March to update the fall 2017 dashboard. Finally, we are asking you within this indicator to also ask us to develop a recommendation to bring back to you next September for change, because we'll have two years of data. The third recommendation that we're asking you to act on is for local indicator. for LCFF priority seven, which is access to broad courses study. And we're asking you to adopt standards for the local indicator consistent with the standards that you've adopted for other local indicators. And Jeff will be presenting that piece. And the other presenter we'll have today actually will have a slew of presenters under the school conditions and climate work group.

086And that discussion will be led by Chief Deputy Glenn Price. So to get us started, I just wanted to ground us in really the tenets of the accountability system. And unlike the previous accountability system, what you've set forward for us in helping you to make these policy decisions is that you're going to continue to review the indicators and data. and the performance standards as it's appropriate. And that as new data becomes available or as we receive feedback from stakeholders over time, that it be considered for inclusion in the accountability tool. This is a graph that, for me, really tells the story. It tells the story how we're meaning ILCFF, the tenets of it, the law, in each of the priority areas, either using a state indicator or a local indicator. And so you can see

087we've broken up this chart by priority area. by state indicator and then by local indicator. The one that we're going to talk about later on today is under priority seven, which indicates in italics that it's a proposed new local indicator for inclusion for the fall 2018 dashboard. So we always like to talk to you about who we've talked to before we came to our recommendations. And for Attachment 3, the School Conditions and Climate Workgroup, they're going to discuss that input separately. But for the remainder of the attachments, we did engage with our LCFF stakeholder workgroup, which is made up of Education Coalition members, interested parties, the Equity Coalition. And the meeting that we had on October 11th was... really well attended. I happened to be sick that day, was on the phone, and you could

088not get a word in edgewise. It was very engaging, and there was a lot of discussion around the local indicator for priority seven. Melanie provided, or rather her staff provided, an update on the system of support. And then we did talk about the development of the chronic absenteeism indicator. There was a lot of interest in that, as well as some functionality updates and direct response to feedback we had received from this group. We then, after the agenda was posted, engaged in another conversation on Halloween. And we provided about, we had an hour set aside. We maybe spent 35 minutes walking through specifically attachments one and two. We also had some ad hoc meetings. There was a couple of groups that wanted to have an offline conversation, and so we did that as well. And then

089the other piece that we are very busy this fall getting out of this building, talking to the field and going to conferences. One group that always invites us that we appreciate is the Curriculum and Instruction Steering Committee from SESSA. We also had our annual North-South Assessment and Accountability Meetings that's hosted by the department. We got 1,500 people online for one of those sessions, and then the audience was full both at North and South. So a very impressive collaboration. That particular conference is led by my colleague, Michelle Center. We also talked to our colleagues in the state and federal program directors. They're obviously referred to as CASFEP. They were down in San Diego. We did a presentation there. And the Advisory Commission on Special Education continues to invite us back. So we've been at every commission

090meeting since last August. And then just to plug, next week we got invited to present at the Alternative Schools Conference. So we'll be talking about the Alternative Schools. Task Force and getting feedback from them and then we also will be at the annual SARA conference which is my peers the education research community. But without further ado let's get into the actual attachment one. the academic indicator. There were some of you that weren't on this board when we made this decision or these decisions over the last 12 months and so what I wanted to start off with is sort of framing how we got to where we are, walk you through what we're currently doing and then show you how our recommendations would affect current distributions and future distributions or current distributions rather. So we started

091this journey several years ago, but I'm only going to start with it in September of this past year. And at that time, the department was directed, or rather the board directed us to develop cut scores and performance categories for the academic indicator, obviously using grades three through eight. In September, we did not have... We had not yet conducted those data simulations, and so we were not able to provide that to you when we provided you the rest of the indicators for approval. So we promise to come back to you in November for that decision. If you remember in November, we ended up withdrawing our recommendation. There was an interest to move away from our methodology of at or above standards to using scale scores, which are vertically aligned. And this is something that is new

092with our Smarter Balanced assessments and is something that the board felt very strongly about moving towards. We committed during that period over the holidays to have all hands on deck to get you a recommendation by January. spent November and December putting forward a recommendation in January. It was provided as an addendum. We were working with a very compressed time frame. And similar to today's discussion and recommendation, we checked in with our colleagues internally. I work very closely, my staff works very closely with our assessment division, with our testing assessment contractor, educational testing services. We also work with our technical design group. And keep in mind, the technical design group is just that. It's a technical committee. It's not a policy committee. In fact, when we try to give them policy, they refuse to engage. It

093is simply meant to make sure that what we have done is accurate, is appropriate to move forward. And I have an example of how both of those groups provided really good feedback last January. So in January, there were four different simulations that we had conducted. We had asked for feedback from ETS. And in fact, the item provided the feedback by the group. So we said ETS, these were the pros and cons that they said for each of our simulations, as well as the technical design group. And in fact, both of those groups told us that two of the options we had considered or were considering were not technically sound. So we actually put that in the item and we fessed up to sometimes we don't get it right, but that's why we depend on our

094technical experts to help us get to a recommendation that we can fill is based, you know, that has appropriate technical merit. So back in January, you did approve. This was only, you know, 10 months ago, an academic indicator. based on student test scores that used, again, the scale score option, or rather the distance from level three. And you did that in knowing that it was a sound basis for comparisons across grades. It was consistent with the CAS messaging, that this is a level that students demonstrate mastery of standards. We also talked about the fact that parents have an understanding of the knowledge that level three stands for, and selecting... Fixed point other than level three because remember we looked at distance from level two we looked at we looked at several different options Would represent

095a communications challenge Stakeholder groups in January were very supportive of the decision to select. This is the methodology In January as well, I wanted to make sure that you, if you by chance looked back at that item, that we noted that the proposed cut scores for change, which we're taking up again today, to the 5x5 grid compared to the other state indicators. We're different. And so when making this information, or rather when we provided you this recommendation, we said that this was based on simulations and distributions for schools and districts and not for student groups. We do not make recommendations based on student group distributions. Finally, in September, just this last meeting, we asked that you delay the incorporation of the California Alternate Assessment in the academic indicator. We needed some extra time to figure

096out how to make these two different tests align into an accountability system, and that we would provide that information in the dashboard as a standalone information piece. So as you can see, over the course of... 12 months or 14 months, this indicator has been brought up probably more times than most of the other indicators. Just to give a basis for what it means to use the scale scores, this is a graphic that was included in the item. It's looking at the distance between a student's score on the Smarter Balanced assessments and the lowest score. for the standard met achievement level, so level three. So in this example, this student scored 25.05, 23 points below the lowest possible score. And we use that to calculate our distance from level three. Keep in mind that scale scores

097range by grade level and by content area. And I think, you know, as we look back at the last 12 months and we look back at how this has been communicated out, this is a very technical discussion. And it's been... We've we've we are trying to message this out to parents and so we've been asked the question How do you message out change to parents if you decide to change this? indicator and and and We keep going back to this is a measure for a school and a districts success This is not a student level accountability system a student receives a student report student Report from the assessments and that is you know, really the basis for for student looking at students and student success. This is a chart that helped us to better understand

098how different the scores were from this year to last year. So the two charts show you the distribution for math. the change cut scores and as we look at the field test which is on the left hand side of the screen the math for math spring 2017 field test which of course are the prior year scores versus the current year scores on the right hand side for the fall 2017 dashboard you can see the difference in these categories and how we're at 50% for spring was positive, it was on the positive side, it was skewed positive 6.4 versus 50% for the current existing scores at 0.1. So we are more symmetrical, more neatly aligned in the current distribution than we were last year. Last year, we were over 80% of LEAs had positive change, whereas

099this year we're about 50% of LEAs with positive change from the prior years. So these are the levels, and this is more in line with a lot of the other indicators. This is a chart that we always include in our technical manual, and that's where I pulled the one from the left on, and the one from the right we'll include when we release the dashboard later on this fall. So we considered a total of six options. First was status quo. do nothing. The second was option one, which we're recommending, which is, again, providing new change cut scores, new status cut scores for mathematics, and new status, new color designation for that 5x5 grid. Option two was to look at new color designation and current cut scores. And then we had options 3A and 3B, which

100are, you know, I feel like I'm echoing. Okay. Is this better? It's still echoing. All right. Try it again. Okay. Does that sound better? Does not sound better. You're still echoing. Oh, rock star. Can you talk louder into it? Talk louder into it? No. Okay, I'm going to try this again. Does it sound, you're still hearing an echo? There's still an echo. Not as bad. Is this better? Yeah, go ahead. Okay. So option three was to look at new cut scores and current color designation. And option 3B is new cut scores and alternative new color designation. And then option four, two-year average. gave you 29 pages worth of narrative, 46 tables, and we actually excluded some tables. We prepared some tables that were appropriate for charter schools because we've been asked before by the charter

101school community to make sure that the distributions didn't look different for charters versus non-charters. So we saved you from maybe an additional 10 tables. But we did... a lot of analysis and that analysis was done in conjunction with again our assessment division our tech our testing contractor, and our technical design group. In fact, we took this to the technical design group more than once. They told us to keep running different simulations. And when we finally came to them for our final round, they asked us to make additional changes. And so this has been a very fluid process in a very short time frame. It's always our intent to run these kinds of... of decisions by our stakeholders first. I started off the conversation with how we provided, asked for input from our stakeholders. I also

102wanted to share with you, we've been saying since you approved this indicator back in January that we thought we'd have to bring this back to you. We were very concerned about the volatility of scale scores, and so we've definitely, I feel like I'm echoing again. We have been indicating through our slides in our presentations this fall to our conferences and then to other interested parties that we would be making this back to you in November for a decision. So we have been very upfront that we have concerns with it and that of all the indicators, this is the one that we thought would probably need some adjustments. So ultimately, we are recommending action for option one, which is, again, new cut scores and new color designations. One of the reasons that we're looking at this

103is because we want to make sure that we're establishing goals based on actual picture of performance statewide that continue to be ambitious and sustainable. And I would highlight sustainable. We want to make sure this system, you know, it's not as volatile as we're seeing from year to year. Large swings in two performance categories or more is not... It's not conducted for the field to plan and so we will be showing you how this would affect what the current methodology looks like and what the recommended methodology looks like in the series of slides to come. So these are school results and what this shows you is on the left hand side the first two graphs are for English language arts. It's a comparison of the scores from 2016 to 2017. The last two graphs are mathematics

104for 2016 and 2017. So you can see how there has been an increase for English language arts using the current methodology. An increase of not only red. but extreme increase in the orange distribution. There's been a closing of the yellow performance categories and a closing of the green and blue for English language arts. For mathematics, you see similar trends with widening of the reds and the oranges as well as a closing of the yellows and blues and greens. The recommended methodology looks more like more even distribution. So we're again looking at applying it to 2016, 2017 current for English language arts in those first two columns. The last two columns are for mathematics for 2016 and 2017. So it's a bit tighter. The ends are, it looks like what you would see in a normal

105distribution. for 2017 at least, if we were to apply this new methodology. And again, this is for schools. This is a table that we've put together based on information that's already in the item. This table does not exist in the item. It was created specifically to have a discussion about how this would affect on the... technical assistance side. So what this shows you is the current distribution of red and oranges. We took in 2016 English language arts, 2016 mathematics. We look at 2017 using the current distribution, English language arts and mathematics. And then we apply the new methodology to the final two columns, option one. And what this shows you is that with the new methodology, you'd actually have an additional 36 districts identified for technical, in the reds and orange. They wouldn't necessarily be

106identified for technical assistance. We'll get into that a little bit later. And for mathematics, it would be 48 additional districts in red. The other piece of this item is we're asking you to look at a new color layout and new cut scores for mathematics. One thing I wanted to point out up front is that we made a small error on the level side. We noted at the bottom with an asterisk, medium status point was changed to zero and less than zero. Just a slight oversight. We'll make sure that we update that information and materials. So what this chart shows, and what it's designed to do and why we felt strongly about making this recommendation to you, is it's designed to bring stability to the indicator. In the current 5x5 grid, most of the rows have

107three colors. One row contains even four colors. This new recommended layout limits each row to two colors. And since each row only has two colors, LEAs and schools that maintain status will always be one of two colors, regardless of their change. So a school with a high status can only move from blue to green, even if it declines. And a school with a very low status can only move from red or orange, even if it increases. So a school with a red performance level must obtain a low status to move to the yellow performance category. So an example of this is a move from negative 85 to negative 70 distance from level 3. So what's changed here? And if you were to compare tables, it would be this is table 14 that I'm showing you

108compared to table 8. Table 8 is the old design. Table 8, the differences between that table and this table is the number of green cells do increase from 5 to 8. The number of blue cells stay the same. The number of yellow cells decrease from 7 to 5. then number of orange cells remain the same and number of red cells actually decreases to three. So one of our one of our colleagues on our technical design group provided us some feedback about Some of the questions have been, why don't we have grids that all look the same? And it would be an ideal world if we were to have grids that all look the same because what we're struggling with and why this is really important to deal with is we're looking at different indicators and

109different metrics with different distributions, different levels of measurement errors, and different patterns that change from year to year. So it's not feasible, and this person said, to force that kind of change into a 5x5 grid for the different indicators. So with that in mind, I'm going to close this out by just taking you through the technical assistance piece or differentiated assistance and intensive intervention. This is a diagram that was provided to you in an information memorandum in June, and I know that my colleague, Melanie Sheffy, will be talking to you a lot about this in the next item. This shows how districts will be identified this fall for technical assistance. This next slide shows you how it applies for this year. Keep in mind that we still have two state indicators that need to come

110fully on board and have performance categories or colors. That's the college career indicator and the chronic absenteeism. The local indicators also do not have two years of data, so we will not be using that this fall for technical assistance. What will be used are priority areas four, five, and six, which are academic indicator, English learner proficiency indicator, graduation rate, and suspension rate. To be identified for technical assistance, the same student group must meet two of the following criteria. They have to be in two different priority areas in order to be identified for technical assistance. So the academic indicator plays one component in priority four, but the English learner progress indicator also plays a component. And what I wanted to note, and why I like this chart is because it shows you for academic that we

111also look at oranges. No other indicator. According to the criteria that you approved looks at oranges. So that means something. And districts and schools need to look when they're thinking about their LCAP about dealing with not only on the academic indicator but all indicators for reds and orange. That's something that you've identified and you've told us to include in the dashboard. It's back. Okay. Well, that actually concludes my remarks on attachment one. What a great ending. I forgot. I apologize. I actually have one more thing to say. For the spring dashboard, we were not able to include data on homeless youth and foster youth. The fall dashboard is the first time you will see these student groups. So districts right now have a private preview. I'm going to talk a little bit about that private

112preview. But I think that will show it may make a difference when we're thinking about things not only like academic indicator, but other indicators as we're looking at technical assistance. We're going to pause for the presentation. I seem to be fine, right? No? Wow. So why don't we have our, and it doesn't help, do I come through better than she does in terms of? No. Yeah, but it's not louder or? Okay, so I think we ought to have the board member liaisons respond on this. First board member Burr. This is kind of like auto-tuning. We could do some karaoke. It might sound better. So a couple things first just to Cindy and especially Jenny and her staff Thank you so much for working through this and I just also want to say especially on behalf

113of Eileen and I that we've had many conversations with the department staff and one of the Discussions we had was should we put all these tables in? Because it does get to be a little bit mind-blowing and as somebody who worked for the legislative analysts for many years I love this stuff but I know many people don't and it's hard for them to kind of wade through. But we thought it was very important both to show the level of work, the level of thoughtfulness, the level of transparency, and I just have to say this, this is what continuous improvement looks like. And Cindy did a really nice job, I think, of laying out the... the underpinnings for this, what we talked about, that this is a learning curve for all of us. We have a

114new test. We have new standards. Everything is new. And really, I think sort of in my lay interpretation of this, what this reflects is that we made our best guess 10 months ago when it came to us about what the trend was going to be for student test scores. And we were off. The trend did not continue because one year a trend does not make. So now we're back at, well, okay, we've got to look at data, and we have to figure out, based on the evidence that we have, how do we make that work for purposes of... information to LEAs, to schools about what they need to know to work on behalf of their students. So that's really what you see here. I just also want to say that because probably of most of

115the people up here, I have the longest history with the technical design group, which actually goes back to 1992 when we did the original Public Schools Accountability Act. And way back then, all of us who are policymakers or experts in the Department of Ed said, boy, we have to have a lot of cycle. Geometricians statisticians who can figure this out and we have relied on them With great trustworthiness over all those years and and we should continue to do that in my view But they are just as Cindy suggested a technical design group. They're not a policymaking body They're giving us advice giving the department advice and that's what you see here in front of you The other thing I just want to emphasize, and I really appreciate this last slide you have up, Cindy,

116because although the academic indicator is only one indicator, and I just have to say this out loud, and for all the noise that's in the system, this is why we don't make single decisions based on high-stakes testing, because there's so much variability. The table that Cindy showed of the swings by grade level and subject matter. But we do acknowledge in our referral criteria for technical assistance that academic performance is important. And we give it greater weight by saying if you're in red or orange, you can be referred to technical assistance, which we don't do in other areas. So as you're trying to sort through those tables, and we can come back and do some more detail if you want, you'll see that when you combine red and orange, many, many more. LEAs and schools will

117fall into those categories than they would if we didn't make the change. And my final comment is the one thing that for me that came clear through the technical design group, the worst thing we could do is to do nothing. to say oh well we did the best job we could last year let's just keep at it because it will undermine the credibility and it will create additional volatility so for those reasons I think we're really going down the right path and should accept the department's recommendation so I just want to add to member Burr's comments so first I can't tell you as a board member how much I appreciate the staff's work on this. I mean, they revise, they listen to our concerns, they take it back to TDG, they bring it back to

118us. We get a draft late at night before a call in the morning, continuing to try to be responsive to our concerns and also what we're hearing from public input. And we want to make this work. And I think I just would support what Sue was saying is that We're working to get this right, but this is a really new system. It's not like we know exactly what it's going to look like, which I think is uncomfortable for many people who want to know just like, okay, tell me when it's going to come out. How am I going to get identified? What's my support going to look like? And so I think to leave, to not do anything, I don't want to do that for the credibility of the system, but for the work for

119school districts and staff that have to work on what's going to happen when they get their ratings. And the thing that I do realize is that things like graduation rate, that's stable data now. We're not tweaking that recommendation. This is pretty new. And as much as we've been talking for a couple of years about these assessments, these assessments are aligned to the new standards we've adopted as a state board and to the new frameworks and materials we've... supported. But what we know is they're not fully implemented yet in every classroom in California so that students are not getting fully aligned in all of our content areas. They're getting some but it's a capacity issue as we help teachers to learn. deeper learning about the standards and what it means for performance tasks and the kinds

120of deeper learning we're hoping for. And there was a wonderful quote from a researcher that says, our scores go up when we change the practices of what we do in classrooms every day, aligned to what we hope to be higher level instructional priorities. So I think the thing that is good as I talk to stakeholder groups is that we've moved to multiple measures and that we're not using a single measure to measure schools. And those of us who... live in schools know there's a lot of things that make a quality school including school climate and engagement and trust and counselors as we heard from students and opportunities to succeed at your highest level and interventions when you don't succeed so it's a complicated picture and I'm hearing full support for multiple measures just wanting to

121make sure we know how schools are progressing so I think this slide is really helpful as well as far as okay so like now we know what it is When are we going to get identified? And then what is our technical assistance and support going to look like, which is the rest of this item? The other thing that I realize is that this might not be the last time we revisit this. Many people want to know, like, when are we going to be there? But I just go back to the API days where we always had an 800. But the 800 meant different things, different years. Because remember, we didn't have science in ninth grade. And then we put science in ninth grade. And we had algebra in eighth grade. And then we had all

122these different things that tweaked it. Even though the number was the same, what made the combination really change? Although I don't know that everyone remembered all the changes over the years that those of us in schools and districts worked on. So the other thing that I would say. is that it is about capacity. And we've heard that on all of our work around instruction and assessment, that it's building capacity of teachers and leaders over time. And we know that it takes capacity to get capacity. And it takes a lot longer than our hopes would be. It just takes a while to not only train new teachers coming into the field, but all of us in the field in different stages of our careers. And I think that that is important. And also, I appreciate that

123we have a reliable team. technical design group that can give us advice. They don't make our policies for us. We make the final policy, but they are advising and working on all these numbers instead. I mean, I did more qualitative analysis, not the quantitative, because for me, the qualitative kind of met my style, although I used some quantitative. But the people who understand what it takes to do effective measurement, I think they provide a really important service to CDE and then advising us as a state board. So I just want to support that we're going to get there, I believe, like we got there on graduation rate. We're just not there yet. And I think that the tweak is important so that it's reliable and that we're not jumping. And I did have a question

124about the number of colors on each row and Cindy's rationale of why we're only going to put two colors on a row. Because I actually said you seem kind of stingy on the number of blue squares. And couldn't you put a few more? And the idea is that the rows only have two. colors and the idea that people wouldn't jump three colors in one measure that we are looking for change over time is important so just to support we've worked really hard to come to this recommendation as the best of the options we have and I just want to support the staff work and guiding and helping us to understand all the data I want to build on that we got permission to quote from a letter, email that Edward Hartle sent in response to

125some of the criticisms and some of the letters. And Edward Hartle is the Emeritus Professor at Stanford University's Graduate School of Education. He's on the technical advisory group for the California Department of Education, has been on it for many years. In 2016, Hartle got the American Psychological Association Award in terms of his work on test score analysis and other assessment issues. He has been president of the National Council on Measurement and Education. That is a major organization of researchers. He is the chair of the National Research Council's Board on testing and assessment that's run by the National Academy of Sciences in Washington DC And I could go on and on I want to read some of his comments because I think we have a dispute among analysts here, and I think the board needs to

126Consider the level of ability of some of these critics that we have and I want to read directly He has given us permission to read these comments First of all, he says test scores are typically unstable during the first few years after any new test is introduced. So part of what we're dealing with is instability. Remember, this is a computer adaptive test that has extended response. You have to defend your answer, for example. That is a performance exam where you work on a problem. Very different from an old fixed form CST test or, say, the SAT test. And what Hartle is saying is that it is – he is responding specifically to allegations by children now that the – and I'll quote directly from him. The children now letter, for example, endorsed by other organizations

127uses that grade equivalence, and it's trying to say that these are huge changes in grade equivalence using grade level standards. Hartle's comment is, grade levels are a type of norm-referenced score quite foreign to the IRT-based vertical scale used for smarter balanced assessments. and quite foreign to the standards-based philosophy and interpretive framework developed for California's current accountability system. The analogy as presented by children now is not only mathematically meaningless, but substantially inappropriate and misleading. So I think, you know, when we – and the media takes these things and accepts all these comments as equal, and I think we have to look behind at it, the forces here in that regard. He has other comments on this. He says that the technical group reluctantly concluded – this is in terms of changing the color grid, which is

128the second part of the issue here. He says, we reluctantly concluded that it was not feasible. to force a particular kind of consistency onto the system. The hard thing is the actual expert work of remedying the imperfection. So they worked with what they had and they... thought this was the best option. So that I think is typical of what we're looking at here. This is a technical problem. It's caused not by the board's interest in rejiggering all the reds and greens, but by the volatility that is to be expected in this. And any delay in moving on this would delay our ability to render technical assistance through our system of state and local support. I think we'd be criticized on the back end then as not getting our technical assistance off the ground. So that's

129the downside of delaying, and I'm concerned about that as well. I think if you could go back on your slides to back here. So if you look at the ELA option one compared to the current. Additional 36 LEAs that would be identified for technical assistance under option one. Right. For ELA and for mathematics, it was 48. And if you look at the reds and the oranges compared... It's not identified for TA. It's simply identified as red or orange. I apologize. Yes. But if you look at the chart, you have in... Look at the oranges. From the 216 math, if you look at them, they would grow under the option one, which you're recommending, right? Correct. The oranges grow from 217 to 600 under our recommended option. If that's trying to re-engineer the system, we're doing

130a really bad job of it. The reds go from 119 to 89. I think orange is not good. And so I think that this shows that we're not just trying to move people up in a huge way. So that would be my comments. And now I will desist and take board member comments. Ortiz-Lacombe. Thank you. So I'm not a quanti. And I'll just speak. I'm going to zoom out a little bit because I read the item when the rest of the public read the item. You know, we were projecting an upward trend, and we had a flat trend this year. So I need us to really consider the context of why some people are questioning the decision, including myself. When I first read it, I was like, what are we proposing? And the reason I

131have mixed feelings is I do think that something has to be done, and I do agree with a lot of the changes that are being proposed. It is a computer-adaptive assessment, but the same students that have succeeded continue to succeed. under this computer adaptive system assessment and the same students that have underperformed continue to underperform and so we need to put some context into our decisions because that is what is bringing us to this item right as i read it i had to consider that Now, again, I do think that there are some positive changes to this. I like the fact that we're narrowing the yellow bubble because that for me has always been problematic. Those schools and those LEAs that will be in that category that are not receiving assistance that could really linger

132in performance. And I think that, you know, we see the data here. There's still going to, we need support. We need capacity. We need to give it to our LEAs and schools. I think what's a little problematic to me is I feel rushed to make a decision that is pretty big. And I know that it makes me nervous, and I am type A, so I'll disclose that, to know that we may come back next year to the same conversation. When I thought we would come back to the same conversation, I thought it was to up the benchmarks and the targets because we're going to be doing so well, right? So I went back to... item that we had an addendum to gives information by student subgroups and so and it was viewed by CPAC they

133weighed in so for me this time around we you did list the stakeholder input I didn't know who actually had input in this particular item because those stakeholder input sessions could have been for the other items that we have in this same item the chronic absenteeism the climate workgroup and so That is where I feel there is a gap because I understand that where you said something about the student subgroups and I'm sorry I missed it. The echo was a little distracting. But going back to our item from January 2017, Addendum 1, it seemed to be a lot more detailed information that we were receiving. Subgroup information, the notion that LCFF is about equity and that it's focused on student subgroups. And so now I feel like we're missing that piece of it in this

134conversation. And it's hard for me to make a decision where I feel rushed and I feel that something has to be done, but the timeline to make these decisions is today. So that's where I sit. Thank you. Well, first of all, like Board Member Burr, I appreciated the 29-page report, but I have no balance in my life, so reading all of that was actually enjoyable to me. I have seen you at the airport at 5 in the morning. Yes, you've seen me at the airport at 5 in the morning. This is the type of stuff I do. But there were a couple of statements in this document that I would say were burying the lead. One of the statements that was important to me is... This information about what's reported in the academic indicator is

135not about percentages of students performing at particular levels. This really is about where the students, all of the students who took the test. were on the scale. And so one of the elements, one of the statements that was very important to me as I read through the rationale for making the change, not only on the change level cut score, but on the status cut score, is that the distance from the lowest score on level three is an average of all of the students' performance. on the scale. And so that was important for me to recognize and to understand because that is where the volatility is actually moving. The reason why I'm saying that is one of the things that's also stated in this report is that there was an overwhelming number of LEAs. that experienced

136a positive change. In the report it says between 2015 and 2016, 80% of LEAs experienced a positive change. And in 2017, the change distribution showed about 45% of LEAs having a positive change. Well, going from 80% to 45, even though there was still a positive change, that's one of the reasons why the scores were flat. One of the concerns I had when we originally adopted this particular scale. was I felt the cut scores were artificially high given what we knew about the performance gap among student subgroups. What is catching up with us now is that those performance gaps, while they have not remained constant, we are seeing a change in the distribution of other performance gap among those student subgroups. So I think that the recommendations that are being made that you've spelled out clearly,

137not only in the charts you've given us, but also in the explanation is very helpful. to me and it makes sense because it addresses the concern I originally had about not only that the cut scores disguise where the performance gaps were among the student subgroups. This actually gives a bit, because the distance from the level three is an average among all of those performances, it actually, those students in those lower performing subgroups actually skewed or pulled down that performance. So I think it's important to address that and that volatility was going to naturally come up because As I said, I think the cut scores originally were artificially high. The difference that I am concerned about or the difference that I want to understand is what is going, what will the, what is, what will be

138reported in the equity report look like? Because I actually think you were very generous on the number of green squares in this new distribution. And I'm trying to understand for any given LEA since the, since the average for those students is computed for each school in each LEA, what is the difference that would actually be shown or reported in what we see in the equity report? Will there be more? I'm not sure this next question is correct. Let me say this. Is there going to be a difference in how many or which student subgroups we see? who are showing up as red or orange. In particular, we know that statewide the trend right now shows that students with disabilities is the lowest performing student subgroup. But will this change that addresses the distribution differences and

139the gap in performance say between all students or some of the more traditionally better performing student subgroups over say students who are English learners will those students disappear in what's reported or disaggregated for them so we will see fewer English learners being identified as orange and they're more likely to be identified as yellow or green because this is going to be more favorably account for the progress or the gains that they are making. Sure, and perhaps I can also respond to Member Ortiz-Lacones' comparison with the January item. If we were to wait until January, I could give you that information. We have not done any student group analysis at this point. We, just like last year, received the data from our testing contractor in late August, early September. We need to make sure that we

140have certain processes in place before we actually do our data runs and our data simulations. And the student group is always the last piece to do, right? We want to make sure that the schools and the LEAs, which is how we... base our recommendation on for January. We provided the student group information simply because we had the time and we thought that you would find it of interest. But I don't know what it looks like. We have not done that analysis yet. That's the piece of missing information that for me informs this decision. But I do agree with the rationale that's presented here. And I really appreciate the honesty about what actually has been reported or what's been shown in the trend lines, both for LEAs and what the reason was why we ended up

141with these flat scores. I really appreciated that explanation. It helped me. And I agree with Dr. Hartle's explanation about what a... a lot of stakeholders are saying, when you get into this conversation and you start talking about scores, you know, people want to know if the scores went up. But the data behind or what the numbers mean, you know, when I was teaching students to do solving on word problems, it was really important to label the values that they came up with after they applied their formulas and came up with a solution. Because seven is either a lot when you're talking about kids or not a lot if you're talking about inches or centimeters. And so making a comparison about grade equivalents when this is not how this information is actually analyzed. is not helpful

142in this conversation because it disguises or even confuses and muddies the water about what the data actually means and what the labels in that data actually mean. So I really appreciate the time you took to write that out and explain it in the report. Thank you. So like Board Member Strau and Board Member Burr, I also remember the early days of the API. And if you remember, we actually started out with a different test. We started out with a national norm reference test. We didn't even have a California test at that time. So what went into the API did change over years. So this tweaking that we need to do based on the little bit of data we currently have, it does not necessarily alarm me. I do think we will need to look and

143pay close attention to it as we move forward, definitely. There is, however, I think another layer to this. When we are working and tweaking this system, And I completely trust our psychometricians and our technical design group and their recommendations. I think we do get questions from the public on why is this happening? What is going on with the scores? And we have received a couple letters that raise questions about the Smarter Balanced scores. And I'm hearing from the field that there's interim assessments that the... field's been giving in different ways and they're not seeing the same kind of correlation, some of those kinds of questions. So I'd like to ask staff to provide us with an update and if this has been looked into and what you have found. Eric Ashley, Deputy Superintendent, Performance Planning

144and Technology Branch. I appreciate the question. I actually was hoping to have an opportunity to come up and address the issue of our 2017 test scores. As many of you know, I sit on the Executive Committee for Smarter Balanced. One of the great advantages of being part of the Smarter Balanced Consortium is that the Consortium is us. It's the states. The Executive Committee is made up of state representation that is voted upon and voted into the Executive Committee. And I can tell you since the release of the scores, we've had a number of conversations with the executive committee about the scores themselves. Because there's absolutely nothing more important to the executive committee of Smarter Balanced. than our teachers and our schools having confidence in the assessment that they're getting and that the results that they're

145getting are accurate and can be used to improve teaching and learning. That's absolutely the number one concern. So the Executive Committee of Smarter Balanced has had a number of conversations in this area. And there's absolutely nothing wrong with raising questions when you see test scores and you see... You look at across states and you see some commonality and to raise some questions about it. Nothing wrong with raising questions. What isn't appropriate or responsible for that matter is speculating and coming up with all these theories about why the scores look like they do when all you're looking at are the scores that are off a website. You're not going to be able to pull... numbers off a website and make those kind of determinations. That's why we have technical advisory committees to do that kind of

146work for us to look into the depth of the data and find out exactly what the trends are that are happening there. Even worse than speculating is that then propose changes based on that speculation. So let me tell you what has been happening since the results were released. We asked Smarter Balanced first to see if they could get us a quick analysis of one of the issues or speculations that was raised, that there was somehow something wrong with the test item pool. That somehow the test items from 2017 were significantly different from the test items in 2016. And so American Institutes of Research did a preliminary analysis for us. It's not in formal. report form. It probably could be put into that form, but currently it's not. But they did provide that analysis back to

147Smarter Balanced, and Smarter Balanced turned around and gave that to the executive committee that said they looked at that test item pool. Now, you need to know that some of the speculation out there was that the questions were harder. Right and that's why scores were depressed, but there was also speculation that maybe the test items were easier and that because of the computer adaptive test it didn't allow because they were didn't allow students to show how much they know so we can speculation on all sides here so So a AR did take a look specifically at the test item pool and comparing 2016 to 2017 found absolutely no significant difference between those test item pools or in the way that they were delivered through the computer adaptive test. That report by AAR was also discussed

148at the Technical Advisory Committee. So, Martyr Balance has its own Technical Advisory Committee, and I can tell you from attending those meetings that they're some of the most impressive minds when you talk about psychometricians. and the knowledge about assessments and about statistics and data is unbelievable. So they discussed the AAR paper, and that technical advisory committee came to the same conclusion, that there was nothing that they could see right now that would lead us to believe that somehow the test item pool or the way that you delivered would depress the scores. But it didn't end there. The Technical Advisory Committee then gave advice to Smarter Balanced. As you probably know, Smarter Balanced does a technical report at the end of every administration of the test to look into issues such as this. And so the

149Technical Advisory Committee took time to also provide Smarter Balanced staff. with some specific suggestions on what they can do in that technical report, what to look at that might give them any indication that somehow the test item pool or the administration that there was something awry here. And so Smarter Balanced has taken that advice from the Technical Advisory Committee. We'll be looking at that in their technical report, and that technical report should be available to us and to you by about the end of February. So I want you to know that nothing is more important to the Executive Committee than making sure that teachers have confidence in this test. I will say that the speculation is a little disappointing to me and it's a little concerning. And what concerns me is that somehow the message

150will get out there that, gee, we don't need to improve our professional development or our instructional materials or our articulation across grade levels. It's just the scores we're depressed. That's the easy answer, right? And I think we're going to have a board item here later, item six, where Westa is going to tell us about... the implementation of Common Core and the California standards. And my guess is they're not going to say the job's done. My guess, they're going to say this is a work in progress and there still is work to be done. And quite frankly, we knew all along this test was going to be difficult because we knew the standards were going to be difficult. We raised the bar. So it shouldn't surprise anyone that the scores are level. And when I say

151level, I think you saw from Cindy's report, some districts did improve, others did not. So it's not like everyone in the state went in one direction. So some districts would be glad to accept the test as it is because their scores went up and others would say, gee, yeah. Why don't you look into that test issue? Maybe that's the reason our scores went down. But I want you to know, we take this seriously, and we'll be following up, and we'll make sure we'll get that technical report back to you. But before you go on, I should have mentioned that Tony Albert, the executive director of Smarter Balanced, is here, and he is also available to answer any specific questions that you might have. Okay, thank you. So first of all, gratitude to CDE staff and

152to Carrick as well and TDG in that it is a diverse group of psychometricians, and I think that's really important that they are researchers as well as practitioners. And I think... It's going to be really important because we're looking at trying to get stability in this indicator, like you mentioned, and that's really important at the local level. And so with that, there's going to have to be a narrative that explains all of this as we go forward. And I think hearing Carrick's story of really looking at what the executive committee considered and talking about what the rumors are to hopefully... build more confidence in the reason for the change. And one of the hallmarks, I think, that I really appreciated coming onto this board is that you made the commitment to always look at revisions

153as needed. And this is one of those examples of you found the information, you well researched it, and so now we need to move forward. And so with that, a question I had, or not a question, but a comment, I really appreciate that with the 5x5 grids that you've matched up. the change scores for both language arts and math. That's correct, right? Now they're going to be the same? And I think that that adds, that's again going to give more stability, if you will, as we go forward. Because at the local level... We're looking for stability, and we're looking for confidence. And so all of these demonstrated actions with good narratives that explain it will help us do that. And so I appreciate what you're doing. And ultimately, it's about the coordinated response, and I

154know we're going to be talking about that. And so we want to be able to respond to good data. So thank you very much for your work. State Superintendent. Great discussion and great work. I wanted to ask Kirk to come back up and just respond. Just the question's been raised around how the recommendation for the academic indicator was put together, what the process was. And you've shared that with me, and it boosted my confidence in what you're doing. And I saw it was very diligent, but if you could just describe that for us. Thank you. Karakashli Deputy Superintendent. So I thought Cindy Kazanis did a great job in some of her slides of talking about the stakeholder input and the board items and the information items and the in-depth analysis. And again, thank you for

155some of the comments from board members about their work. What Cindy and her staff does is absolutely phenomenal, especially with the time. compressions at times, the pressure of producing this data, what they do is really phenomenal. I don't think a week has gone by that Cindy and her staff haven't been in communication with one stakeholder group or another to talk about the academic indicator as well as the other indicators. So there has been a lot of input. This particular recommendation is compressed, right? This was because of the timing. This came quicker than we normally like to do. And I think you know that. And we have not... tried to take advantage of bringing things at the last minute when we don't need to. I think we've done a responsible job of getting the data to

156you just as quick as we can. So there has been input all along the way about this. We find ourselves in a situation where we do have a recommendation to make here. And if we want and we do want schools and districts to start looking at this data and using it and making changes, then it is important that we have a decision today. Let me also say. simply because I feel compelled to, that we spend a lot of time when we talk about these. the accountability system talking about the reds and the oranges. But just as a reminder, this is to be used by all schools and all districts because we have a basic belief that they can all improve. So this isn't just about reds and oranges. It's about everyone using this data because

157we believe everyone can get better. Let me make one comment about the technical design group, and I appreciate the comments. They do tremendous work, and they are volunteers. right they're not they're not being paid to be here um they they have worked with us for many years specifically to give recommendations to the superintendent and how it works is that these these are people as President Kirst was saying that are psychometricians and statisticians and local district people with a good deep knowledge of their local school district data to give recommendations to the state to to CDE so that we can then turn around and give some recommendations to you. It's not a pass-through recommendation. These are discussions that we have with technical people. And as you can see, this is a lot more technical than a

158lot of decisions that you have to make. But for these technical decisions, we need technical experts to make them. And so we bring them together. Often what we'll do is we'll bring two or three options to them and say, Give us the advantages and the disadvantages of these different options and tell us the technical soundness of these. And some they'll just say, yeah, good try, but that's not technically sound. You can throw that one away. Here are some other ones. Here's two more. This has advantages and disadvantages. Technically sound. This one has advantages. disadvantages technically sound, feel free to then take those, which we do. Then the CD takes them and puts them through a policy lens before we then come to you with a recommendation for you to choose. So again, it's not a

159pass-through recommendation, but we do need technical expertise because, as you can see, these are very technical. And I can't help myself but to make one comment about a letter or two that even made it into a news article talking about closed doors and secret meetings by the TDG. There is no secret about who's a member of the TDG. Anybody wants to know, just come ask me. I'll be glad to tell you. It's not a secret. We don't post their names on the website because we're trying to insulate them from those without technical expertise trying to overlay their policy recommendations onto what their recommendations should be to us. We want to keep them. as a group to look at the technical issues, not the policy issues. So we do try to protect them a little bit

160from stakeholder groups that have a policy recommendation and try to influence the decision that they might make. And what we're asking for is their technical expertise. So again, now meetings might be closed doors because they can get loud at times, but they're not secret. And I hope that helps. Board Member Burr. So I just wanted to come back, I appreciate the opportunity to say one more thing that I didn't get to emphasize, and this goes to Member Rucker's comment too about burying the lead a little bit. Cindy did mention this, but I want to reaffirm this, that we as a board, when they came to us initially with academic indicator cut scores, went, wait a minute, we are not going to recreate NCLB and have percent proficient. That's not what this system is about. We

161now have all these wonderful tools, courtesy of Smarter Balanced, that yield scale scores. So this is the first time, and you saw a table, which again I think is very helpful about the volatility, that we've handed this task to the department and said, make sense of these scale scores. And they vary by grade level, they vary by subject area. It's like doing three-dimensional chess, which is hard for all of us. at least we have the technical design group to help us. So I just wanted to, again, restate that, which is why the academic indicator looks different than all of the other indicators and why the grid is going to look different. And we just have to get comfortable with that if this is the system that we're going to have. I also just want to

162say to my friend Feliza that we will be here next year having this conversation. Because we are going to learn. We're going to learn based on the test scores of next year. And also because, and just to introduce another level of complexity, we've said, gee, we'd like to have a student growth model. Cindy smiles because we'll be back here in the spring talking about that. But this is a precursor to what that's going to look like. So get comfortable with the technical conversations. But I just want people to understand that that's what's kind of in front of us. The thing, though, I do really want to emphasize with respect to my friend Feliza and others is to... As you talk about this, and I think you will when you get to the dashboard indicator, I

163think there is confusion about why we're acting on this at an LEA and school level in the absence of student subgroup data when we know that student subgroup data is what will actually drive the technical assistance. So if you could clarify that so everybody feels a little bit more comfortable about the sequencing. So maybe something to help is to show the proposed grid. All the student groups will be assigned, will fall into one of these 25 boxes. And what we've done is provide, in fact, a different tool outside of the dashboard on the department's web page that shows where student groups. fall. It's there for the spring release, it'll be there for the fall release. But student groups will fall into one of these boxes. And by being these boxes tightening up and being more

164discreet, we'll be able to, again, I think identify schools and districts will be identified supports that are appropriate for those particular student groups. And I know that we've had conversations with with practitioners, with people at LEAs that have to actually explain this to the public. And I'm going to talk a little bit about the resources at the end of this presentation in attachment five. But just to preview, we do have a series of handouts that we're putting together. And I think, if anything, we need to make sure that we spend some time on the five by five grade and how important it is and how this is the basis for student group, where student group results will land, right? So they will also show in equity report, if they're in red or orange, you know,

165it's... displayed right there you can go into the student group report and and see it you know one click away how those particular groups are doing so does that does that respond to the question okay Board Member Ortiz-Lacombe. My only question, did you say that this information will come back to us in the form of a memo? You said there will be further information, and I just want to make sure, and I just want to say, I don't think this is a conspiracy theory. I want to put that out there. And a lot of my only hesitation was coming from the timing of it and making sure, like, it just feels that we have to make a really big decision right now. But I don't believe that we're trying to... fudge the data in a

166way that it's going to make us look better or it's going to take schools out of the red category. Do you have to say that it's probably best in conversation, it sounds a little bit more comprehensive than the memo. The memo did emphasize a lot on like how many schools and LEAs would fall in the red category. So maybe starting... And I understand that because this is your everyday. And for those that wrote the memo, maybe there's some things that you don't feel you need to explain because you understand each other. But for those of us that are reading the memo and don't necessarily have all the background information, I think it's better to start with, we have a flawed methodology. And it's creating A, B, and C, right? Versus getting into how many would

167end up in red. Because it did sound a little bit like we want to just mitigate that. So I do believe that we're doing this because we learned from the methodology that we adopted. But I also want, I am one of those people that need to see, like, how does this impact everything else, right? And so for me, it would be really helpful, regardless of... My vote today to have this information later on to see what does this mean for subgroups in particular in the form of a memo as soon as that data comes is available Just to build just to build a little bit on what remember to the cone said the communication from the department to parents Alternate organizations schools districts and is going to be critical to help Everyone understand just what

168we've all been talking about. We've been here what the last 30 40 minutes explaining this to ourselves right and talking about it and Somehow that has to be condensed into a paragraph or something for all the constituents out there and that's going to be tough So I encourage you to spend somebody to spend as much time as possible getting that right that will be very important and helpful okay uh now we're this is a composite item so should we move on to the next one yes and then take comment okay so we'll move on to the next okay So the good news is it's still me, but maybe the bad news is it's still me. But this is, I think, maybe a much shorter discussion because it's one number. So attachment two is, again, we're

169asking you to take action on this attachment. This is about the chronic absenteeism indicator. And we don't get to collect new data very often. And I have to tell you, this piece of data is really exciting. The field is... I've heard super excited just just over the map about the fact that we're actually collecting this data and the attachment that was in that was in the item gave you a preview or gave you an overview of how we went about the collection process because, again, we don't collect new data very often. So we made sure that our data stewards at the local level were prepared, that they had been properly trained. We also worked with our student attendance referral board. There's a state SARB. There's local SARBs as well. So Jenny. It reminded me, Jenny

170Singh of my staff, that she's presented to them several times on this particular piece of information. They are just very excited that we're collecting this data. The state SARB in particular did appreciate our recommendation to come out with sort of a second edition of the fall 2017 dashboard in March. So that is a piece of feedback. We did talk to the LCFF stakeholder group in early October about this, and they felt very strongly. We didn't have the data reports yet or this recommendation put forward. But as a foundation, they wanted to make sure that the data was of high quality. And as you can see, we affirmed that in the item, that the data is of high quality. We did have a majority of our LEAs submit data this past year. We are in the

171process of developing those data reports at the student group level, program participation, as well as at the school, LEA, and state levels. And similar to the reports that Superintendent Charligson pointed out this morning on suspension rates, these are going to have the same look and feel. And if you've had time to spend on data quests in the last week, we've just heard really great reviews, especially from particular equity groups, about the power of these reports. And in fact, Russ Brumberger from UCSB. tells us that he has a data request in, although we can't find it, on chronic absenteeism. So we know that they're lining up. We've been talking with attendance work. I had a call with Hedy Chang last week, and they are just very excited to be able to use this information. Similar to

172the suspension and expulsion, just to give you a background on DataQuest, which has run out of my division as well, we have lots of things about data reporting, not just the dashboard. But that web portal is where we keep... you know, all of our data information. The dashboard is the accountability tool. But for DataQuest, we've been doing a bit of an overhaul. And so once you get into that site, you will see changes if you're on the enrollment reports or, again, the suspension reports. We are planning or the recommendation is to provide a link to the DataQuest reports for this time period in which we need additional... time to actually do our work. So we need to run some data simulations and we want to come back to you in March to provide you that

173information on chronic absenteeism to put some cut scores together for status and then come back next fall with cut scores for change. And as a preview, I wanted to show you the reports that it doesn't show up so well because it's a screenshot, but this is a dynamic report. Just keep that in mind. You can change filters by charter versus non-charter, by ethnicity versus program subgroup. I tried to talk about this in terms of selecting shoes for Macy's, but this is maybe not so sexy. But it has the same kind of options, right? You can keep building on the filters. One of the things we did when we looked at the suspension reports, for example, is we looked at it by grade span. And one of the feedback points we've already gotten on the grade

174span for chronic absenteeism that we don't have in suspension is you need to look at K-8. And then they want us to look at kindergarten by itself. So those are some things we can build in as a filter option. What you will not be able to see is where it says program subgroups, that's where you would find foster youth, migrant. homeless, and then socioeconomically disadvantaged. So that information is in there. And then you can always see at the top there, you can filter it out by English learners and students with disabilities. Other piece that we're looking at is chronic absenteeism is 10% or more, but we actually collect information in other pockets. We're looking at less than 5%, 5% to 10%, 10% to 20%, and over 20%. So we've looked at the work of Attendance

175Works. We've looked at other states. We think that what we'll be able to put together using these filters will be... better than most states, if not even the best in the nation because of the programming capacity that we've put into DataQuest with these new reports. So with that, I'm happy to answer questions. I did want to also acknowledge that we've been working not only with our CALPADS office under my colleague Jerry Winkler's leadership, but also with Gordon Jackson in the Coordinated Student Support Division, whose office actually runs the state SARB and actually helps with model attendance policies. They have done tremendous work in helping us to get to where we are today. Comments on chronic absence? Okay, thank you. Yeah, we've spent a long time on this. Okay, yes, comment. Karen Valdez. Cindy, I just

176have one question. When you go onto the dashboard, there's going to be a link to get you to DataQuest, and it will take you right to that landing page on DataQuest, or you're still going to have to filter through? Our hope is that we would actually be able to take you to that particular school or a particular LEA, so it would be a one-click link to the school or the LEA reports. Okay. And then the difference with DataQuest now is you can actually, within that report, navigate to other reports without getting out of it. So it's a bit more trying to keep you all in one place as opposed to moving around a lot. Okay. Great. With that, I'm going to turn it over to Glenn Price to take up Attachment 3. Okay. This is

177broad course of study. This is climate? I had broad course of study. Yes. What conditions is climate? Okay. This is climate. Good afternoon, President Kirst, Superintendent Torlakson. I'm highly motivated Chief Deputy Glenn Price here to present on school conditions and climate. And I'm together with a great group of colleagues from the School Conditions and Climate Work Group. Why don't we just take a moment and introduce ourselves? Hi, I'm Hannah Melnick, Research and Policy Associate with the Learning Policy Institute. Good morning, everyone. Taryn Ishida, Executive Director, Californians for Justice. Good morning. Aaron Brengard, Principal of Catherine Smith Elementary School in San Jose, California. Good morning, Shannon Brown. I participated in the group as a teacher in San Juan Unified. Great. Thanks, everyone. We're just really excited to be with you today presenting the results of the

178work of this group, which has gone on for over a year. Our journey began with some... really exciting direction from the State Board to come together and look at school conditions and climate with a specific emphasis on Priority 6. And now we're before you with a framework which you've seen as part of the October memo, which you have as a handout on your desk, and there's some in the back as well. And the framework presents a really exciting and a phased pathway for looking at school conditions and climate. There's a timeline on page 41 where we're looking at moving forward over the next couple of years with some really exciting and inspirational improvements to the work going on in school conditions and climate in the state of California under Priority 6 in a relationship to

179the school conditions priorities as well. Just want to really appreciate as we get started the members of our work group. You see it there on the screen. We're just delighted to be able to bring together. a very diverse group of practitioners with principals and teachers like Erin and Shannon, to researchers and advocates like Hannah and Taryn, and I'm excited that they're going to be part of our presentation today. As you know from the scope that we presented to you back in January, we charged under board direction with... this working group with taking a look at the literature and there's an outstanding literature review and research part of the framework which you'll see included. We're not going to go over that in detail today but we worked hard to take a look at that research. We

180took a look at the approach of other states and how they were doing this and what we could learn from their experiences. And then last but not least, we worked really hard and you'll see in Appendix E detail of stakeholder engagement that we undertook in the state of California to bring forth the voices of administrators, the voices of teachers, the voices of students as we began to shape these exciting recommendations that we have to to present to you today and that included coming in front of the California Practitioners Advisory Group four times over the course of the last year to really get as much input as we possibly could on these on these recommendations. So we're going to go forward and we're going to briefly present these to you as we we summarize the path

181the path ahead this is an information item today but we are looking to come back to you in the future with some action items which will detail on the course of the presentation so let's start out Aaron with a this recap of the importance so what does it mean to have a positive school climate you know you see the words on the screen here you see them popping up safe welcoming respectful inclusive supportive These are the words that I don't think any of us would argue. These are the words that I think are grounding us in what school climate and conditions need to be. These are the words as a school principal that I need my families, my students, my staff saying about my school too. And I want you to think about these words

182as we're going through our work here, that what we're after and what all stakeholders do, come up with the same words, that we all recognize the importance of school climate and safety. All right. So what are our stakeholders saying, right? I think all of us here are in this room because we, yeah. Thank you for the help with the technology. We all are in this room right just to ground ourselves because we work for our students for our families and with them and so I think it's really important that we look at some of the recent data this poll from Berkeley and EdSource just came out recently and you can see among all of the different factors that we are measuring and the state is prioritizing by far number one more than anything else safe

183and positive school environment, right? And so this feedback, you know, didn't include students. It had 1,200 registered voters and an oversampling of parents. But as you all know, I'm here today because students have told me that this is also their number one. And not just any students, but the students that... have been left behind in our system, right? Students of color, low-income English learners, and foster youth. 2,000 of them actually told us through a survey at Californians for Justice that a positive environment that fosters relationships with students and staff is the number one thing they need to achieve and to achieve equity. So I think that powerful voice is why a lot of us have spent hundreds of hours now in this work group, why there's, again, I don't know, 60 students out in the

184lobby and why they've come back time and time again to talk to you all about why school climate matters so much. So given this importance that students, parents, and our voters have... elevated for all of us. I think the question for all of us in this room, the state board and the work group, was really how do we make this real? How do you really get all of our schools to follow the path to the desired result of a safe and positive school climate? And specifically, what are the policies we can put in place at the state level to guide them there? So that's the way we approached this work group. We took a very practical approach, right? We all are busy people. We know on the ground how busy our schools are, how busy

185our teachers are. And so we wanted to really narrow down on what are the essential recommendations that really, really matter. And when I say practical, I mean it in this way. Data meaning and use this is the right slide so we want to use in simple words right from one of our students Let's use the school conditions and climate data for continuous improvement It's good because schools are self evaluating and they can see what areas they are lacking on and they can make up ideas or actions to fix it right so in a simple simple term What do we mean by data? Data for us is not just collecting more data. We have a lot of that It's not a compliance process, right? It's really to make sure that we can derive meaning and new

186meaning from the data that's already available that will help drive improvement and use. So for data, that includes three key stakeholder groups. You know, we've been measuring, we've been serving students, but we feel like we also need to involve parents and families because we know how critical they are to the success of a school. And we are also missing the voice of teachers and administrators and classified staff. So the importance of really bringing in our staff. And we'll talk about the data later about why staff voice really matters. On meaning, meaning for us was about complementing that data with focus groups, with interviews, with classroom observations, really bringing more nuance. One of the administrators I met in the lobby today, she actually said that, interestingly, one of their kindergarten students talked about the lack of

187feeling safe in their school. But when they luckily followed up through the qualitative method, the student said, I don't feel safe because there could be monsters in my bathroom. Right? So the importance of making sure we look at the nuance behind the survey data is critical. And then lastly, more than anything else that I think matters is the use, actually leading to action. And this is where I just want to share that, you know, I know this is the part of the system that matters the most to all of us, right? We can have all this great data, we can have great policies, but if it doesn't lead to change, then the system won't be successful. And so this, I mean, we have to be real, I think this previous conversation about, you know, the

188capacity challenges that our schools have, all of the competing priorities. But, but, but, just to underscore, this priority six in particular, school climate and the integration of conditions and parent voice, parent engagement, this is really the heart of schools and what should be the heart of continuous improvement. And I'm saying this because you all know that I go around the state. I've been in thousands of hours of LCAP meetings locally. I come to these meetings. And there's a lot of conversation. I call my baby the LCFF baby because she's been with me to all these meetings. But the places that I've seen the most change firsthand has been when we take a lot of the data we have on the dashboard and complement it with this survey data, which speaks to people's hearts, right? So

189in my backyard, there's a high-performing high school. They are blue and green across the board, but when they disaggregated their graduation data, they could see that their Latino, Black, special ed, and English learner students were far behind. Students saw this in a dashboard meeting, in an LCAP dashboard meeting, and they went up to their principal and said, why is it that our Latino students are so far behind? And that actually spurred a lot of conversation, a deeper inquiry into what are the root causes of this? And it wasn't that they needed more science or math instruction. They actually found that it was a Latino and black student experience of feeling disconnected, undervalued, and that people didn't think they were smart. That's actually what was driving the low performance in their schools. Since then they've made

190a huge shift right they've used this data They've made meaning out of it and they've implemented 95% of their staff are now getting trained on bias on equity and how to really listen to students They've put in new practices of emergency protocol when there's been a big incident on campus called microaggressions Right just last week number of students reported being being targeted with the n-word You know, that is a real experience that holds students back from being able to learn. So I just wanted to ground that in what's really happening and why this data meaning and use is such a critical part of our recommendations. Thank you. So I want to just, I know there's an extensive literature review on page 11 of your package. I just want to underscore that research backs up what

191stakeholders are saying, that these learning. environments are shown by child development research by scientific literature to be fundamental for cognitive engagement in learning. This is actually about brain development. This is about how children learn. And a recent analysis of over 75 different studies of school climate showed that a positive school climate can mitigate the negative effects of poverty. So this really gets to the heart of, if you're looking at learning environments, you are starting to address issues of equity. There's also substantial research that shows that schools with positive school climates retain their teachers longer and they engage their families more. So there's reason to believe that if you address these issues of learning environments that you can actually start to have more productive students. But we also know that policy has driven us to not

192focus on what we know from research to be so important for so long. And that's why... Looking at survey data and looking at school climate data in particular is one way that we can start to change this equation. And surveys are ways to start asking introspective and important questions. Washoe School District is an example of a place that's starting to use these climate surveys annually over time to really start to look at what is behind the achievement gaps in their very large and diverse district. They're using it to triangulate test scores, absenteeism, and behavior data, things that we're trying to do on our dashboard as well. soliciting feedback from the community to what Taryn says, asking students and student data summits what these data mean to them, how they can inform the district's action. And

193they're seeing an uptick in their graduation rates. So for that, I'm going to pass it to Glenn, who's going to talk a little bit more about our stakeholders. I'm not Glenn. So what do stakeholders say? I mean, first and foremost, the first group to think about are students. Students want to be included in the conversation. I think you already heard that today. But any time that you're in a school, you know. And I just want to advocate for even young students want to be heard. We, of course, want our high schoolers, our middle schoolers involved. But in elementary school, the student voices are extremely important in guiding me in what I'm doing. Parents need their children to go to a safe, clean school, but they also want to have access to information. They want to

194know if this is a good school. They want to be involved. They want to be engaged. And we think it's making sure that we're collecting information and listening to the voices of our parents is critical as well. And of course practitioners as Glenn mentioned we met with the California practitioner advisory group four times December of 2016 February of 2017 June of 2017 and August of 2017 were times where we met to get feedback they did want to make sure that We were considering their workload balance and that support would be given on this item. A couple of other things of specific feedback from that group. They love the idea of highlighting a few areas to measure what's important as a state and also focus on conditions. They love the idea of surveying students, parents, and

195staff to make sure that we have a real sense of what's happening in our schools. They did ask us to consider the relationship between this indicator and other indicators as you heard Taryn talk about how they're all really interconnected. They wanted us to make sure that the timing of the survey was flexible so that it could be done annually or every other year. They also thought it was optimal to have vetted surveys and provide them free to LEAs. They wanted to us to make sure that this remains local and not prescriptive. And finally, they said they felt that the surveys are essential because they are vital for gaining information, but they are also limited. Again, as Taryn highlighted, just because something shows up as a data point, it means that we should go deeper into

196understanding what caused that answer so that we appropriately are able to address the need that comes up. So this leads us to our recommendations, which we divided into two parts. Those that are... really affecting this what the state needs to take action on and what the LEA is taking need to take action on for the state our first definition our first recommendation is that the board adopt a common definition of school conditions and climate this is a basic first step climates is a nebulous topic we just want to have clarity on what we're trying to address and measure and improve The second is to set criteria for setting valid and reliable, providing valid and reliable tools for all school districts to use. We know that measurement tools quality is incredibly important, which is why we're

197spending so much time talking about the measurement tools like the Smarter Balanced and investing so much money into those. We want to make sure that all school districts have valid and reliable survey tools as well. And we know that many school districts don't have the capacity to vet all these surveys on their own. I want to make note that 20 other states are making surveys available statewide through their ESSA plans. Many of them are using Title IV Part A funding for this. That said, we are not trying to say that all districts use the same surveys. We just want to make sure that the state sets some criteria for how local surveys can be vetted to be reliable and valid. A lot of us came into this work group thinking we need one single survey,

198and we have over time understood that's not our charge, and it's actually not what's in the best interest of schools, but they should be using valid tools. So third, we identified four different domains of school climate that both the research and stakeholders have indicated are really important for us to address. And the first two, safety and connectedness, they're not new to us, they're in statute. Conditions for teaching and learning, that is part of the charge of this work group is to look at school conditions. This could include things like connections to other... Priority areas such as access to a broad course of study and also includes conditions for learning like students mentions access Do they have access to counselors to nurses? We also identified empowerment as one domain of constructs. This might sound new, but

199again, it gets into things that are already state priorities, student engagement, parent involvement. And if you look at the description in your packets of what we mean by these domains and constructs, you're going to see language that's taken directly from the ad hoc parent or family engagement work group. And finally, we think it's a no-brainer that the statewide system of support needs to include supports for school climate. Both house districts are going to be using their survey data in a meaningful, impactful way, as well as tools for them to use to improve. On the LEA level, we're talking about phasing in an annual survey that would be for parents, staff, and students, and at least one of the three grade spans, at least one grade level each of the grade spans. I'd like to bring

200up, though, that concern around workload and just put a reminder on the fact that, well, the word priority in its origin meant one. It meant first, actually. It's from the 1400s. If I would if I would pick one priority when we have a list of eight priorities, priority six is touching all of these other priorities in some way. And we see that. And having an annual survey is highly critical, I think, in the work. I have students in the classes every year. We know that students want to hear what their voice is heard and it's important for us to listen to them. I Understand that it's workload and it definitely is workload. I'm not saying that I'm not Incredibly overworked in the position and there aren't a lot of things but but if I want

201to improve these priorities I've got to be paying attention really closely to the voices of our student staff and our parents to make decisions around What's best for my school every year? I'd argue more than once, but at one time is what our recommendation is. But we're backing that up with state-vetted surveys, so we don't have to all reinvent the wheel. We can have surveys that we say, here are ones that work, here are ones that give you that information. They meet the reliability and validity kind of threshold that we need to set. We're measuring those four dimensions that we heard Hannah bring up here around the climate and conditions and those ones that are reaching into all of our other state priorities. And then we're recommending a best practice. We've already heard it come

202up with Taryn talking about the kindergartner. We have to actually take survey data and then go do something. It's about usage around the data. I actually think that that's probably the most critical point is I first need to listen to the voices, but then I need to go deeper and see what's really going on to make improvement. And that's what this is about. It's about continuous improvement. That conversation keeps coming up over and over and over again. I hear you keep saying it, and I think that that's one of the critical pieces. And lastly, and definitely just as important as every other point, is that disaggregated data is how we're going to advocate for our underserved. The disaggregated data is a place that's going to bring us to a place that allows us to promote

203equity, helps us figure out where any school can find places where there are gaps that they need to address. Great so for all of us visual learners like myself this is a flowchart that kind of pieces together the recommendations that and the state and local level that Hannah and Aaron just brought up. So here we start with that's the state capitol building I believe. But with the California dashboard the story I illustrated we can see data we can see the colors but what the local climate surveys allow us to do is to kind of go deeper where the LEA would choose from either the local options, something they maybe are already using or something new they'd like to innovate and try, or the second option of a menu of surveys that the state has already

204vetted, making it a little bit more simple, maybe more straightforward for an LEA to pick up and go with. This then gets tied into the triangulation of feedback between students, parents, and staff, as we've talked about, and then the qualitative review here with this magnifying glass, right, going deeper as folks keep talking about the importance of that. But the part I want to highlight is really this last piece, this improvement process with that arrow going up, which is what we all want to see. We want to see upward trends. And then... really having that tie in coherently linked to the LCAP process, right, in an annual cycle. And so I want to say annual. I know, Mike, I've said annual to you a few times now. From a moral standpoint, of course, I deeply believe

205that because the voices of our students and our parents and our staff, their voices matter every year, not every other year. But from a practical standpoint. Annual is critical to the LCAP process. We do the LCAP process on an annual basis, right? So we set goals and we mark our progress on those on an annual basis. Secondly, continuous improvement. We keep talking about that. I know many of us have spent days now learning about improvement science and continuous improvement. And so those experts around continuous improvement will consistently tell you that you can't do something every other year. If you have measures and data that are only every other year, that actually kind of... contrast or contradicts the whole purpose of continuous improvement which is real-time results as quickly and rapidly as you can so that

206you can make meaning and that you can take action use them. So that's the full process that we're really approaching these recommendations from. Thank you. Thank you, Taryn. I think we're all visual learners at some level. We wanted to talk a little bit about, there's a pretty comprehensive set of recommendations that you have in front of you, and about the phased implementation of these recommendations. And looking at the recommendations that the... workgroup has developed responding to your direction, you can pretty much divide them into two categories. Ones that we believe the State Board of Education cannot act on within your purview and those that will be where we're going to need to go out and get some additional financial resources. So in the immediate short term, we're going to bring back to you, we'd like

207to bring back to you probably in March 2018, some of the important recommendations that we believe are under your authority to approve, and we're going to look at those in just a second. At the same time, we know that some of the recommendations from the School Conditions and Climate Work Group will require some additional resources. Implicit in providing state-vetted and state-supported surveys are really providing those to the field. free of charge as what the work group is recommending. And that helps us also work away from an inequity situation or work towards equity where some districts are now capable of procuring highly vetted, highly valid and reliable surveys. Others haven't been able to. We think the whole state should have access to those types of resources. But that will require talking to our colleagues across the

208street. and making those kinds of requests. So while we have been to the CPAC, as noted a couple of times, we want to go back to them now, now that we have this full set of recommendations. So we'll be presenting to the CPAC at their December meeting and perhaps following up with that at their February meeting as well. And we could provide you with an update and then come back in January and then come back in... in March with a set of recommendations for for action. So what those recommendations might include would be inclusive of these that are on your screen and in this in the slide deck. So the recommendation that we should include active work and support around school conditions and climate within the developing statewide system of support. The update for the

209self-reflection tool that's shown in Section 7, Recommendation 7.5 of your framework document that really allows LEAs to get into a deeper level of reflection around their data and results. And then, as we've heard around this issue of workload, to really look closely at minimizing the duplication of efforts as we move forward with local... tool consideration so an example of that might be given that we're recommending that parents be surveyed as part of priority six and some of the related school conditions priorities couldn't that also apply to the measurement work that we're doing under priority three our parental involvement and be able to take care of that simultaneously so that we're not adding to but in fact making more efficient and reducing workload at the at the local level so those would be some examples of

210recommendations that we'd like to bring back to you in in 2018 likely in March and I think with that we're excited about answering your questions hearing your comments on on this presentation And thank you. And I just want to, before we get into deeper discussion here from the public, just praise the work of the... working group is fantastic outstanding very helpful very comprehensive as We listened this morning to the youth ambassadors youth leaders of the future I think we heard over and over their interest and their deep concern about student voice being recognized and it reminded me of the months and months and hours and hours of hearings we had and hundreds of people who've been through these chambers to speak out I would Hazard a guess that the issue of student voice and school

211climate were at the top in terms of the you know the comment publicly Testified to importance as we develop the dashboard in our accountability system It's truly a priority and I think the emphasis on using the data to guide equity Goals implementation is really important I also believe that the recommendation for parent involvement and all staff, you know, teachers, classified, administrators, having survey data that gives their voice some feedback with the students. And I'll just observe, you know, this is consistent also with the survey that you showed from EdSource that 74% of the public wants to see positive school climate is implemented as a top priority. And I know we do spend, how much do we spend on our testing system? Millions and millions of dollars. And I think we need to invest more in

212these recommendations. So it would be a wise investment. Thanks. Yeah, I'd like to turn to our liaisons first. Well, so I would just say congratulations on your work. It's clear that you've spent lots of time and energy and research to come to these recommendations. There's a couple things I'll just say, too. I do think, and you've appropriately noted, that some of these are outside of our purview. I wish we could print money, but we can't. So this notion of needing additional resources would be something that you have to take to the legislature. You talked about in your presentation, but not in the recommendations, this notion of kind of a common definition. I think that's beyond our purview, candidly, and that would have to be something that the legislature would have to think about. I mean,

213we can talk about characteristics of school climate, but if we were to adopt a definition, I think we'd have to go through a regulatory process, and it would be a lot more complicated, because in the past, on different issues, the board has been heavily criticized for just randomly saying this is the way things should be, rather than going through an actual regulatory or statutory process. So just on the technical side of it. The only other thing, and I'll just speak to the annual versus more less frequently, the thing that I would appreciate going forward is thinking about this in terms of context. And I'll just use an example because, Taryn, your example was a great one. But we have a letter, for example, in our drop box file from a superintendent in Arcata. who is

214very articulate about she's the superintendent, she's the director of curriculum, she's the bus driver, you know, I'm making that up, but she's got lots of jobs, as I'm sure our principal knows well. And that is a very different context than running programs for 600,000 students in L.A. or what, you know, the 65,000 in Elk Grove. And I think as we make recommendations about this, Thinking about it in context, for example, is your student turnover greater than 5% every year? If so, I'm making up that number. I would rely on our researchers. Whatever is a reasonable turnover. Or does your teacher workforce turnover? Then you really, you have to be taking the temperature of your conditions. If you have a very volatile or unstable student population or teacher population or parent population, absolutely. But I'm guessing

215in Arcata, that maybe isn't the case. And so the beauty to me of LCFF is the acknowledgement that the local context matters. And I just want to make sure as we're approaching all of these priorities and data gathering that we respect that. And so going forward, I'd just like to see a little bit more on that. But again, I don't want to diminish any of the work that you've done because you've clearly worked really, really hard, and we appreciate you doing that on our behalf. Board Member Strauss. So I just want to follow up with what Member Burt said. And first, thank you for your commitment to the quality of the recommendations and the amount of work it took to come to this point. I would just say as a... long-term school principal with more

216than 20 years leading schools at elementary middle and high school the voices matter and the context is different and we need to get input and i think listening to students but also to parents and getting a sense of what happens in schools as well as being there every day and feeling and getting a sense of it it should inform practice and i think um President Kurtz and I met with administrators and superintendents and leaders who value this work tremendously. So I think we start from the point that the voice and the input matters. And we met at AXA last Thursday and then I was in Santa Cruz on Monday talking to counselors and superintendents and leaders, principals saying, so how often do you do surveys? Just to find out. And so there's a variance where...

217Everybody's doing some kind of surveys, but some do it electronically and they can tweak one or two questions. Some of them do a longer one every two years, but then do different grade levels different years. Some of them do different surveys for parents. So I think there's a, I believe sincerely there's a commitment to hear the voices and get the perspective. But at the last CPAC meeting, and I appreciate what Chief Deputy Price said, is that we went to CPAC and asked for input, and then it came back to you all. It hasn't gone back yet, but it will. And so I think listening to the voices of, we know what the peer research and advocate voice says, and those of us who live in the schools, but we have to think about what it

218takes to implement it. And I don't know that it's workload as well as it is using it for more than a bureaucratic checklist. And I think we heard that at CPAC, which is, I don't want to just say, okay. I gave the survey. Yep, I did this. Yep. I want to say you gave the survey And then you looked at the results, and then you made a decision to change some actions. And then you implemented that action, and then you gave it time to see if it made a difference, and then you surveyed again to see if you got the progress. So if you just make it a bureaucratic checklist, we won't get the results we want, which is to improve school quality and learning conditions for all kids. And so that's what I heard

219from the superintendent voice on CPAC, and these are people that are also very, very busy who spend volunteer time to give us. feedback and basically said if we want to survey every year we can or if we want to survey different groups different years but give us time to make this valuable and informed practice not just a bureaucratic checklist so i think that's the point that i think we want to make the other thing that i When I talked to the principals and school directors, district leaders in Santa Cruz just on Monday, they were saying they have very different formats. And so I appreciate the comment that you're recommending that we vet resources. But I actually am concerned. And I talked to one advocate who said, well, there's going to be a committee that's going

220to vet and approve people's surveys. And I don't think that's really our intent. Our intent is to say, look, you can use this survey or you can try this one, but here are some that we know are valid, reliable. So I think it's the idea of offering resources, which we've done instruction and all kinds of things, and then trusting local context. People are looking for that work and that help and guidance. But I think that's our responsibility at the state level to say these are ones that you can use this is longer this one takes this long this is good for high school kids it's not good for elementary those kind of things but i think we have to trust and respect the local context as far as the size i mean some schools are

221Well, there are at least six or eight, but some of them are 100 or 200. And as I met with Santa Barbara County SOUPS and also Santa Bernardino, so many of their districts, they're single school districts. The superintendent is often the principal and many other jobs as well. And that's a very different role from a unified district that could have 10 or 20,000 and lots of people to support the work. So I guess I want to go back to the idea that we want this to inform our practice and we want the best practices to help, but we don't want to make it. Just bureaucratic. Yes, we want some bureaucratic requirements, but we want to have them inform practice. That's what local control is about, so that we change practice for kids. Okay, Board Member

222Williams is next. I respect all the caveats and cautions that I'm hearing from that side and agree with them. That said, just kind of speaking from the heart, I heard the superintendent articulate that it felt like the work of your group. has accurately captured what we've been hearing over the last couple of years as we've tried to evolve and revise the accountability system from parents and from students about how important school climate is and about wanting to have a voice in that and wanting to be heard and appreciated. And I feel like it really captures that, and I think it's really important. And the team you've put together, if... If Glenn Price put this team together, it's an outstanding group and your leadership on this is really clear and your passion around it is really

223clear. I think I agree with the LPI presenter that it's, well, many of you said it's related to almost all other priority because... In some ways, it could be considered the emotional bedrock of all other priorities in terms of providing the kind of environment that everything else is more likely to support the success of everything else and of student learning and teachers' happiness and valuing their jobs. I want to call out, because this is going to come up in a few minutes. again, that it does relate to a broad course of study and that broad course of study, there's lots of research literature that indicates, especially for kids with adverse childhood experiences or low income, limited availability of resources at home, that having a broad course of study at the school where they have options

224and choices of things that they love to do and get to choose for themselves in addition to the mandatory academics is... can make all the difference in the world in terms of their motivation in the mandatory academics. So I do think we might wind up hearing speakers talk about maybe ways we can elevate even the importance of access to a broad course of study. I agree with the idea. And I agree with this all theoretically, okay, because I know resources are an issue, but I think the idea of having vetted surveys that are considered reliable and accurate and maybe each could take a slightly different approach as that as a resource for schools who don't already have their own or aren't happy with their own. I think that's an important thing to do. And of

225course doing something with the data. I can imagine, never having been a principal, but I used to run a nonprofit organization, so I can imagine that if you collected the data and you disaggregate, that you'd look at the data as a whole. and you also look at it by grade level, or you look at it desegregated by student subgroup, you may find that there's only an occasional problem that has to do with a few students or one student. There aren't pervasive problems. But you also may find that the students in a particular subgroup are all kind of reporting something very similar. And that's not a one-off going to ask the first grader what she's afraid of, and it turns out to be monsters. It's maybe a focus group or pulling. together and saying you know

226let's talk about this and let us hear more about that if i were a principal i I would love doing that. I would find this so important to do. And one of the things I would recommend that once we decide what all we're doing about this is that we make sure that there is information developed around why this is so important and why it's in the interest of the district and the school leaders to want to do it because it will help. It'll help their teacher retention. It'll help their student retention, their student academic success. It's not just another requirement by the state. It's something that really has a basis for that they will benefit from putting the time and the energy into it as well. Good work. Board Member Holliday. Thanks for all your

227work and providing if at the state level we can provide Potentially valid annual surveys and reliable surveys as a gift to Leas and districts that's that'd be wonderful. That'd be great. But in addition to that, I think we also have to provide back to the context idea, examples of the process that would be used, how you give a survey, when you give a survey, how you look at it and how you read it and how you understand it and tally it is every bit as important as what questions are on the survey. In your LEA level recommendations, number four is to deepen understanding of initial survey data. That's hard to do. That often is the place where we make... or at least I used to make really huge mistakes as I would hear voices of

228students and parents and think oh I know what the problem is it wasn't the problem at all what what responses come what voices and words are used often mask the truth of what the issue is in a school environment so once again A reliable survey is the starting point, but how you package this process and the ability to really understand and get at the heart of the problem is critical, I think. Okay, I had a comment. I'm glad to see the vetting for reliability and validity. I'm not sure how you can complete that all by March, and so I'm wondering what your schedule is on that. I've looked at these surveys. There are some of these surveys and there's really big differences between racial and ethnic group responses on this that troubled me and with

229some of them and so I think this is quite a Perhaps a lengthy task so the timing of all this has me a little baffled That's an excellent question, Dr. Kirsten. The recommendations that we would like to bring back to you in March 2018 do not include having the body of surveys already vetted, but would include these action items that you see on the screen. And if you look on page 42 of the framework, we've got a timeline there, which includes putting in place a technical design group akin to the one that we just had some discussion about. where we could set up the criteria and then begin the process of doing this such that we would really be looking to July 2019 as a start date for the vetted service. And, of course, that

230would also be dependent around successfully having secured the resources to do that. Okay, other questions? Aaron Valdez. Ninting, sorry. I think I'm hearing all the questions and concerns about surveys, and I totally get that. And I think in districts, there's survey fatigue because there's so many different surveys. So maybe value added as you're looking at this, and you kind of touched upon it, that the survey might also help get information for a broad course of study. It'll help get, you know, priority three for parent engagement. And so then that becomes a real asset, and it can be used for so many other decision-making, you know, in the in the process so that would be something i would really encourage you to look at thank you thank you um thank you very much great work on

231this and so i have a question first and then i was going to make some comments um can you describe what do you mean by state supported surveys so what are you envisioning as a state supported We'd be envisioning is making available to leas and this also speaks to the issue of Context or part part of the issue of context we know from our interaction with with leas that many superintendents particularly superintendents and principals in small Even to medium-sized leas have difficulty on their own putting together because they are so busy They have so many things to do that going home at night and developing a survey just becomes one more thing for them to do. And then so we're seeing a great diversity around the state of what kinds of surveys are being used

232from the very valid and reliable to ones that literally have been written on the back of a piece of notebook paper and then inserted into SurveyMonkey to exaggerate a little bit. But that kind of range. So what we'd like to see is the state provide, meaning the state would pay for the provision of surveys that have deemed to be valid and reliable. We're thinking a vetted menu of two to four of those, which actually harkens back to a day when the state was able to provide a California Healthy Kids Survey. Many of us might remember that. Every other year, but was provided free to LEAs across the trade and it was an incredibly valuable tool that many people use for a variety of different purposes. So we'd like to harken back if you will to

233that era, but do so with surveys that had been vetted by a technical design group and then provided not mandatorily, but if you would like to use this vetted survey that we're making available on pain for and also we would like to see provided as part of that. Some data analysis at the same time provided to the to the LA s yeah And I think that goes to member holidays point that it's not just about putting the questions together that really there's a whole Package that needs to be included if we were to really say it's a state supported survey so so in just my comments on the report I I had the sense of Duality, I guess when I was reading it not not so much conflict but duality one on one hand completely

234agreeing with the the report and the Recommendations and this is what's right for kids and this is what's right for school improvement We need to do all that and I appreciate the commitment passion with which This group put together these recommendations and reports my other side B was starting to Worry a little bit about the slippery slope. This is a local indicator, and that as we are looking at surveys as a local indicator, how far the state wants to go to prescribe to LEAs, and are we now veering from the intent of LCFF when we start? putting such a prescriptive making these these local indicators so prescriptive that It starts locking in and I'm not saying that right now the recommendations are necessarily doing doing that, but I'm afraid of the slippery slope And if

235this were very specific in two years from now, you're going to implement these surveys on an annual basis, then what about the other local indicators? As we start looking at them, are we now going to start being much more prescriptive? And then pretty soon they become statewide indicators and not local indicators. So I just want to... us be careful and look back at our framework and what the intent of this is and it's interesting that member burr brought up the superintendent from arcadia because i printed this out specifically because i absolutely empathize a lot with what it is to be a small school small district and then needing the resources but also understanding how much energy and time it's taking for right now under our new system, the data input into the state, CALPADS, TOMS,

236SmartAbout, all the stuff that we have to log into and all the different systems we have to log into to give data, and then also all the afterwards what to do with the data when it comes out. So there's a lot in the system as it is, and we really want our local, we want our People on the ground working specifically around students to improve student outcomes and not be running around doing all these technical Aspects so it's about building the relational not that the technical Aspects of the system so just being very careful and mindful of that so we move forward Okay Ortiz Lacone So first of all, just in the spirit of what we're trying to accomplish here, which is create a good school climate for parents and students and teachers, I also

237think that we need to be mindful of creating a good climate for those stakeholders in our board meeting. This would have been great for it to be the first item that we discussed, not first item, sub-item in item three, because I'm aware that we have... students and parents that are in the lobby that have been waiting for so many hours to give their input that have come from different parts of the state. And if I'm a high school student, I'm thinking, wow, these adults are taking a long time to hear me out, and I only have a minute. So I just want us to be mindful of our stakeholders and our public as well. So I'll start there. And I do want to thank the parents who took the day off from work to be

238here with us. So, muchas gracias a los padres que nos acompañan ahora. Second of all, I was looking at, thank you for all this work. You could tell that there's a lot of time invested in this, and this is an important conversation. And I understand where Board Member Sun is coming from. We need to have that balance, but I do think that we need some guardrails, right? Not too prescriptive, but not too loose. Because I think already we feel that some folks... feel that state indicators are the ones that matter and local indicators. Well, we have so much flexibility there, you know. And we need to make sure that people see both sets of indicators as equally important to us. And this particular indicator really influences the others, as we've heard. With the administration of

239the surveys, I agree with Board Member Holliday that we do need to provide some sense of timing. Most of us go to trainings. We get our survey about the training. the end of the training right so having the survey in September things are so new I really can't gauge if the school climate is working for me as a parent or as a student but if we wait too long then when can we implement the solutions to the problems that have been perceived so I like going deeper but I also understand that going deeper focus groups interviews observation requires time and so I want to support this notion that we need to do this frequently and constantly, but we also need to do it well. So we take our data. What does this data mean? Who

240didn't submit a survey? Because that's also a data point, right? Who's not submitting the surveys and why? And so all of that requires time to then reflect on it and then implement it. And I want to go back to implementation of this data, of our knowledge that we gather from the surveys. That is going to be critical. And that, I think, it should also be reflected here. At what point do we implement? recommendations do we go deeper and figure out what in the parents from this school submit data or why do they submit at such low numbers so just being mindful of that as we continue to work but thank you very much Okay, so I looked forward to this report and looked forward to this conversation because I did have an opportunity to talk

241with some of you before we came here today. And for me, what's important about this particular topic and this particular indicator is that for the first time, in the history of my personal involvement with California schools, either as a student, as a teacher, and then sitting on this board, we are asking schools to hold themselves accountable for how they treat the students and the parents in their school community. And we're asking them to hold themselves accountable to learn from those stakeholders by doing these surveys. And so I think in looking at these recommendations that you've made, and you've made a lot of recommendations, there are some differences between what are the actions that have a statewide context and what are the actions and recommendations that are more local and have some nuances. and how schools

242in Glen County would interpret and implement those recommendations, and how schools in larger urban districts would implement those recommendations. And so while it is important to understand that school climate and conditions reflect the fact that there's a diversity of opinion about how I feel, about how I fit. in school. There's not really here, the value added in these recommendations doesn't reflect the diversity of what schooling looks like in districts across the state. So I too read and responded and connected to the input that came from the superintendent. The thing that I continue to say as we have these accountability discussions, I understand we are all very busy. doing a lot of things. But in the context of the decisions that we are asking locals to make and the work that we're asking locals to do,

243are we sure that we're doing the right things? And so creating this, creating the work to decide to use these indicators more globally, annually, and in ways that guide districts to seek these frequent feedback episodes that may work for some districts. But in communities where the population is static, where the activities and the interaction and the engagement looks different and people take on different responsibilities, that extra work and that additional time without guidance on the other issues about doing a survey become important. So while I congratulate you on this work, I congratulate you on the depth of what it is you've done in making these recommendations. I think that the part... that is missing here, or at least the part that needs to be considered is the local context and the diversity of what that

244local context actually would mean for a number of these recommendations. So I'm glad to see these recommendations because personally, I've always felt it's important for school districts. as the direct connection that most parents will see between the state and what the state is doing for their child in providing a high-quality education. For California to ask school districts to hold themselves accountable for the attitudes and values that schools present to students and their parents, I think that's important. But the exercise of collecting data and doing the data analysis has to have meaning for the district, has to have meaning for the parents and the students, and has to have meaning for the actions that districts will identify and take. And doing the exercise for the sole purpose of checking off a box or making sure that

245we are annually practicing engagement, that's not engagement anymore. It stops being engagement and starts being something else. If we're truly looking at what the mechanisms and the rituals and the norms of engagement are, the mechanics of an annual survey alone simply aren't it. Board Member Williams. Just to clarify, mostly for myself, we're not, the board hasn't taken action to mandate a survey. Okay. So there is a local indicator on school climate, and a survey is one way of getting at that, measuring it, touching base with it, getting some information from it, but there are other ways as well. As of right now, with the local indicator that's been set up, we have asked districts to do a survey once every two years to do that in different grade spans and to post up an analysis

246of those results on the dashboard. So we have... The board has taken action to tell them to do that. That's already part of the dashboard. That's already part of our system. The difference here and what the work group is bringing forward is that we, and I think as opposed to wanting to mandate it as a state indicator, what the work group is bringing forward is that provide some additional support in doing that because we haven't provided any support until now in terms of. what we mean by that validity and reliability. We have asked for districts to execute a valid and reliable survey, but we haven't really talked about what that means or provided any actual surveys for them to use. So they're on their own. Okay. Well, thank you for that clarification. There's too many

247moving parts to what we've been doing the last couple years for me to remember that one exactly. And I think that's a good point. If you really look at where the statute starts with and where our recommendations landed, I mean, for me, I think we did the bare minimum, right, of recommendations. As Hannah mentioned, there were many other things we could have recommended, like a single survey for the whole state that could have a red, yellow, green. We didn't go that way. We valued the local control. So I just want to say when you really step back, step back, step back. there's 24 metrics that LCFF has required of districts to do annually, right? And I know that districts are under a lot of capacity and a number of strains. But I think the bigger

248question here to the board is, are we walking the talk? We keep saying that school climate matters. The polls show it. Our legislators say it. But our policies just need to match that. And I think right now there's just a disconnect from that. We aren't putting as much time and energy into these important indicators like school climate surveys and parent engagement as we are with many other indicators. So while I know there are challenges, I think the challenge to this board is really saying, are we going to walk the talk and try to close that gap of disconnect? Board members, thank you. I appreciate the discussion and just also the struggles that other board members have when they're thinking about things like frequency. How often do we administer? instrument but I know that there are

249other ways I work with in a Native American community and I know that surveys are not the way to go for us so we have to be really creative about how we elicit feedback and build that relationship with our constituents it's not just parents but staff teachers administrators grandparents siblings cousins I mean you can go on and on a lot of families make up the school environment so You know, I think you've done outstanding work, and I know that it's beyond maybe the scope of what we're able to do as far as adopting a definition, but I just want to acknowledge the good work that was done with the definition and the features and the lenses because I think it gives us a common language to work with because school climate can mean so many

250different things to different people, and I think this is very helpful in framing the conversation. And I think alignment with the statewide system of support is essential because we have to help people who work in schools build their capacity to do this important work of not only getting feedback, but how do you incorporate it and do something meaningful with it. Hey, I think we need to break for lunch if the board comments are done and so we're gonna get our lunch It's out here. I'd like us to get back in ten minutes or so if we can I know that might be tight, but we're running Really behind and then we'll have the next discussion from Jeff Beshears on the LCAP addendum so let's break and come back at 25 1.35, 12.35. Okay, the board

251will reconvene. We now have two more information items on item three, and next is Jeff Beshears. Hi, good afternoon. I'm Jeff Brashears, the director of the Local Agency System Support Office. So attachment four is talking about and recommending on the State Board to adopt a local indicator for priority seven, broad course of study, and along with it the standards that the Board adopted in September for the other local indicators. So here it shows again that in September of 16, the State Board approved the standards by which an LEA will look at their local indicators, and that is to look at and measure progress at the local level with locally available data, and then report those results to the local governing board at a regularly scheduled meeting. Those are the standards. And then they report those

252standards and how they've met those standards with a met. not met, not met for two or more years within the dashboard. And speaking of that, before we go on, just a little update about the local indicators for the fall 17 release. We have currently 419, I believe it's 19, 18 LEAs that have reported their local indicators. 19 county offices, 260 districts, and 139 charters. And also last Friday, Deputy Ashley sent out a letter to all the LEAs reminding them of the December 1 deadline for local indicators and also offering to extend the deadline for those LEAs that have been affected by the wildfires and ask them to reach out and work with them to extend that deadline as needed based on their needs and the impact of the wildfires. Here again are the local indicators

253that we currently have basics priority one priority two you can see And then also just to note priorities nine the coordination of services or expelled students in the coordination of services for foster youth are for County offices only so we have the draft that we have we've received feedback and put out and from We will seek feedback from the CPAC in the December meeting. We did present the local indicator, the draft indicator. We're not asking you to approve the self-reflection tool. That's just in draft form. We'd like your feedback based on that draft. But we presented that to the LCFF stakeholder group in October, got some feedback for the draft self-reflection tool. And then we'll also reach out to and continue to reach out to the external partnerships. So what we'll do is to again

254seek feedback from you from the self-reflection tool if you approve the local indicator and the standards consistent with the other local indicators. And then we anticipate taking feedback through February, bringing back a finalized self-reflection tool to you in March of 18 for the use of the fall 18 dashboard release. And that's that. So again, you have the draft self-reflection tool, and I'm happy to answer any questions at this point. OK. Board Member Ortiz-Lacone. So point of clarification, you're saying that we are asked to vote on the local priority for access to a broad course of study, but we're not voting on the standard. No, the standard is draft form the standard know that self-reflection tool is Stan is in draft I'm sorry does this over okay? The standard is is we're asking you to adopt

255that consistent with the other local indicators Which is then to collect data at the local level and then report that data collection Excuse me at the at the board meeting Then the reporting of the progress of that standard is the met not met or not met for two or more years within the dashboard itself Can I ask you about the standard? Here we're looking at the LCFF unduplicated pupil groups and individuals with exceptional needs service served. So I'm assuming this is our students with disabilities plus foster youth, low-income ELs, correct? If you're referring to the... The standard. The self-reflection tool or the standard? The standard. LEA annually measures the extent. Oh, yes. So those four student subgroups are mentioned, and my question is, why not just look at student subgroups? Because when we're looking at

256access to a broad course of study, a lot of that has to do with students from different ethnic and racial backgrounds that may not be as clearly identified who's having access and who's not if we are looking at the LCFF unduplicated people groups and students with disabilities. So the reference to the unduplicated pupil groups and the students with disabilities is part of the self-reflection tool, not the standard itself. The standard itself is, I can bring this back to, and that's good. I appreciate. There's the standard is measuring the LEA progress on a local performance indicator so the self-reflection tool we purposely when we we asked for some feedback to create a draft and we created a draft of the self-reflection tool in conjunction with SPE staff and And so what we had, we got very

257little feedback as far as the origination of the draft self-reflection tool. And so we put it there identifying, trying to walk that balance of not all students, but to look at the equity pieces as we're looking for the access to broad courses study. But it's a point well taken within the self-reflection tool to say all students and not, I mean, we intend all students, but a point well taken. Yes. Board member Valdez. A quick question on K-8 districts and when you're reporting broad course of study, are you going to give more direction or examples or exemplars for them to do when you would assume at an elementary school they're getting all subjects? So what would that mean? And with more clarity, I think that would be something I would like to see. I appreciate that.

258You're welcome. Board Member Williams. I'm one of the two NGSS liaisons, and so I read with interest the letter submitted by the California Science Teachers Association on this particular item, and they had some recommendations on strengthening the self-reflection tool for the local indicator. Basically, they were in the same spirit as the previous presentation we just heard. If we're going to ask people to do this, let's give them enough kind of specificity and resources so that we get meaningful data or they get themselves meaningful data. So I liked the recommendations that CSTA submitted for this, and I hope you'll consider them and take a look at them. I think they do provide the kinds of... specifics like, for example, these kinds of things and giving them more direction. And in addition, on the question of disaggregation,

259thinking about, again, NGSS, but thinking about STEM more broadly if we're talking about access to a broad curriculum, and especially thinking about computer science. Since we are now embarking on California on developing computer science standards, then we might want to call out gender. because there are gender equity issues for women in science and computer science. Sue Burr. Thank you. So, Jeff, this sort of dovetails, I think, on both what Feliza and Karen were talking about. I'm looking on page three of Attachment 4, and it says self-reflection tool, but then it's standard is under that. Is that the standard that you're asking us to adopt? Yes, the standard consistent with the other standards that you adopted for the other local indicators. Okay. So just two suggestions on that. We saw a lot of letters on this,

260and while I love to talk about Ed Code and memorize what the Ed Code says, I don't think people know what Ed Code 51210 really means. So I would really articulate that in lay language, what the subject area is. We got into this before about standards. There really is a definition of that, so I encourage you to put that in there. Don't remember maybe I need to go back and look at our other local standards that the Students are called out the way you have them called out here, and I think that was Feliz's question I mean I am an errant. I remember I apologize so on page three of four That is at the very beginning at the top of the page says the following is a draft of the self-reflection tool So it

261really calls out the standard more for This self-reflection tool in priority seven the standard is is more if you will on the on this the slide there that it is the standard is that you've adopted is to LEAs measure on a local performance indicator on locally available information. It is that. So it's a little chicken and egg thing going here. Yes. Right. You're calling it a standard in the self-reflection tool, which if I'm in a district and I read that, I go, oh, that's the standard. So I think we just need to be clear. We need to be clear what we mean by that, what we're asking them to reflect on, again, across the grade level. And I think it really will be important to give guidance to K-8. grade spans because as I recall

262if I go back and look at that ed code section it's calling out like science mathematics you know what would consider we would consider to be departmentalized classes that are offered generally at middle school and high school so what's a k8 are they're supposed to go we're enrolled in they're enrolled in third grade and you know these are the course of study I think there just needs to be some clarity around that for especially for k8 Point well taken. Thanks. Okay, so you'll make that clarification. Thank you. Okay, any other comments on this item, on this part of the item? Ortiz-Lacone. So I just want to make sure that we are going to name out that it's student subgroups that we want to make sure they have access to a broad course of study. So

263that won't fall off the self-reflection toolkit standard. Correct? So what I'm, I think that the confusion in the way this was written is, is the part on the very top part of page three is really the preamble to get to the self-reflection tool, kind of, if you will, minimal instructions to do so. So what I'm hearing is we need to better articulate exactly what the standard is, but then within the self-reflection tool, certainly to not let the call out of the unduplicated students and students with special needs to fall out in the actual response. which starts with about mid-page on page three. So they're not going to go away. We will make sure that we articulate exactly what the standard is as adopted by the State Board in September of 16, and then move through

264to better articulate the guidance that you're giving me here with the unduplicated students, and then breaking out with science and also with elementary grades as well. And just to be clear, you're going to come back with a recommendation for the self-reflection tool? Correct. Okay, thank you. Board Member Williams. Just quickly, it just occurred to me that when you mentioned disciplines, while we certainly expect a broad course of study to include standards that we have adopted by the board, you know, the problem under NCLB, one of the problems under NCLB for so long is that student, low-income students and those that were struggling frequently didn't have access to science because that was squeezed out for double sessions of math or English language arts. So I entered this thinking a broad course of study includes those things

265beyond the usual disciplines and then just realized, especially at the elementary level, it also includes things like history and things like science and student access to those and knowing whether or not that's equitable by the different groups. All right. I think we're ready for Cindy Kazanis to do some summary here. All right. We're going to close this out. Oh, I'm going to skip ahead. Alright, so this is our time timeline for the rollout of the dashboard assuming that you all take the recommendation from attachment one we will have if we skip ahead to close to the bottom of this slide We will have a public launch right after Thanksgiving districts have had access to a private preview of the dashboard since the 1st of November and we have done a webinar series which I'm going

266to talk about on the next slide so we have been sharing this this timeline out often frequently with within you know any discussions we have with with stakeholders and with our early a's Jeff mentioned in attachment for that the local indicator submission is fast approaching the deadline is is December 1st there may be some exceptions due to the wildfires we haven't I don't think we've heard back yet from from schools or districts and this is a district report of course And then the default rating after that would change to not met. So we have a six-part webinar series that we launched. We launched actually the last week of October. We've had 250 participants on each webinar. I suspect that the webinar tomorrow is going to be... pretty full. I would estimate well over 500 people

267would get on that webinar because it's all schools that will get a suspension rate indicator and of course those that have the subgroup of English learners, the thresholds would be interested in the webinar as well. You can see next week we're going to feature Melanie and her staff on systems of support. We're also going to highlight whatever changes are made today on academic and chronic absenteeism indicator and then we're going to close it out with now you have all this data. how do you use it in the local control and accountability plan process. The slides are being posted to the web page that is listed there. So we already have our first two webinar slides available. We have a communications toolkit in the works like we did for our field test. Janet Weeks of your

268staff has been extremely helpful along with other communications personnel on how to get this right and to message it frequently and in many different forms. So these are a couple of the things that we're planning. We also put in your basket over lunch three flyers that are close to done. They have not been posted yet. You will see these flyers. This is a new look and feel of our communications tools for these are sort of the most digestible things we have. So the ones that I gave you, and there are copies in the lobby, is on the college career indicator. We also have one on who's included in the English learner student group by indicator. And then an update on the dashboard release. So this is more on functionality. We've been asked, are we going

269to do these flyers for every indicator? And at this point, the answer is no. But we are doing modules. So we do have a PowerPoint that's narrated over. We do have actually... A county office has been helping us with all of our communication materials. So this is out of Sacramento County Office of Education. So to remember Holiday's point about messaging on the academic indicator, we know there are things that we're really good at and there's things that we need some help on. So this is something we also look for help on is how to communicate with others. These flyers will all be translated in the top five languages in the state. So we do have a translation. Service here, and that's how they'll be posted and accessible for for parents We are looking at updating

270our parent tab on our webpage as well to make sure that we have resources that are available And we've been talking to our other state stakeholder groups like the California collaborative on edge like education excellence on how we can collaborate to during this process And others to make sure that we we you know message out appropriately and try to get the word out on on the fact that we have a new system that is operational this fall so with that I'm happy to answer any last-minute questions or we can Open it up for public comment. We're never about this First of all, I absolutely love these flyers and the infographics and all of that. So thank you. And I know that you're saying that you probably can't get to others, but somehow or another, I

271would encourage you to do the academic indicator just so that that could be clearly explained. Okay. Thank you. Sure. Okay. All right. I think we're ready for public comment. Speakers 1 through 10, please line up for item number 3. Doug McRae, Elizabeth Martinez, Marcella Reyes, Rosa Maria Martinez-Ponce, Sonia Ordello, Lionel Velasquez, Toledo Mendoza Escobar, Guadalupe Celedon, and Floripas Flores. Hi. Good afternoon. I'm Doug McRae, a retired testing specialist from Monterey. On the academic indicators, none of the options in Adaption 1 should be approved, nor will they lead to a stable dashboard system over time. The focus has to be first on credible Smarter Balanced 2017 gain scores, including potential corrections, before adjustments to the dashboard cut scores or color schemes should be considered. Using statewide test scores for dashboard status and change measures requires an

272assessment program that yields comparable scores from year to year. Smarter Balanced 2017 scores do not meet this requirement. Details for how to address this situation are reflected in the one-page handout coming around, as well as the discussion, analysis, and consortium-wide data submitted to your electronic folder last week. Finally, to respond to the CDE staff discussion, a statement on a non-public AIR report. The reality of the 2017 contorsium ride gain scores and patterns that are documented in my September 27th document, although I'd be happy to look forward to reviewing the AIR report. Thank you. Go ahead. Come up. Yeah, one minute, and look at the warning light. It's the yellow warning light. Okay. We have a translator. Good afternoon everyone and to each of you here. My name is Guadalupe Celedon and I have four children.

273One of them is in special education in Santa Ana School District. I'm here today to ask you that the school climate be a clear priority under the system of LCFF. Ya que sin un ambiente escolar positivo, es imposible que nuestros estudiantes logren sus metas o cumplan sus sueños, puesto que el ambiente escolar debe o tiene que ser la antesala de preparación antes de entrar al aula listos y motivados para recibir la instrucción académica y aprender. Without school climate, positive school climate, it's impossible for our students to achieve their goals, to reach their dreams. School climate needs to be a prerequisite before entering the classroom ready and motivated to receive academic instruction and learn without any type of negative interference. play toes me a dos bullying etc lamentable meant a esta es la realidad que vive

274nuestros estudiantes y que nosotros como padres maestros y organizaciones queremos con la ayuda de ustedes cambiar Negative factors such as stress about homework, fights, fears around bullying, etc. Unfortunately, this is the reality that our students are living. And we hope that you working with us, parents and teachers, help us realize a better future and change for our students. Thank you. Muchas gracias. Buenas tardes a todos y cada uno de los presentes. Mi nombre es Marisela Reyes. Soy mamá de cuatro hijos en el sitio escolar de Santa Ana. Y para mí como madre de color, es muy importante que todos los estudiantes estén en un ambiente escolar felices, ya que esto favorecerá que nuestros hijos tengan un éxito total para llegar a la universidad. Good afternoon to everyone. My name is Maricela Reyes, and I'm a

275mother of four children in the school district of Santa Ana. As a mother of color, it's very important for me that all students are in a positive school environment. They're happy. This is going to support them to have success in everything and for them to be able to go to the university. As a mother, I want to see that all students are happy when they come to school and that school staff and administrators are ready to give them the resources they need according to their needs. For these reasons, I'm asking that school climate be a real priority in the system of LCFF and that we as adults and you feel and we decide for our students accordingly. Thank you. Good afternoon, State Board and all members. My name is Elizabeth Martinez and I am a

276mother of two children in the Santa Ana District. I am a leader in my community and in my school and I support the organization OCO, which is part of PICO California, for justice. As a mother, I worry about the well-being of students that have dreams. They want to go to the university, but they don't have the resources they need or the tools to reach those dreams. Es por eso que como persona adulta vengo a pedirles que podamos dialogar sobre el ambiente escolar. Si un joven tiene un clima escolar sano, es garantía de que se pueda graduar. Así que les pedimos que las encuestas sean una prioridad en el sistema de LCFF. Muchas gracias. For this reason, as an adult, I'm coming here to ask you for us to work together to move forward around school

277climate so that students can have a positive school climate and in this way they can graduate and go to the university. We want surveys to be a real priority under the LCFF system. Thank you very much. Good afternoon, State Board of Education. My name is Rosa Maria Martinez, and I am a mother of four children in the Santa Ana School District. I'm here with the organizations OCO, PICO, and Californians for Justice. As a mother and as a mother of color, I'm very concerned about school climate for our children. For me, school climate is at the heart of education. Sentirnos que pertenecemos a los padres. Sentirnos que pertenecemos padres y hijos porque aumenta la relación interpersonales. We as parents and students, we want to be part of, have a relationship around interpersonal relationships at school. Y

278la estimulación positiva de los estudiantes y la academia y lo académico. And positive school, I'm sorry, students and academics, positive environment for students and academic achievement. For that reason, I'm asking that surveys are a priority, surveys around school climate are a priority under the system of LCFF. También considero importante que nosotros como adultos podamos juntarnos con ustedes para tomar decisiones por nuestros niños. I also think it's important that we as adults get together so that we can make positive decisions for our children. Gracias. Thank you. Buenas tardes, BESA Directiva Estatal de Educación. Mi nombre es Teodoro Mendoza. Soy tutor de un niño en el distrito de Santa Ana. Good afternoon, State Board of Education. My name is Theodora Mendoza, and I am the tutor of a student in Santa Ana School District. Soy líder comunitario

279de la organización Oco Pico California por Justicia. I am a community leader for Oco Pico California and Californians for Justice. Para mí, el clima estudiantil es sumamente importante para los niños de color que al ir a la escuela no se sientan con miedo al hablar. For me, school climate is extremely important for students of color. To go to school, they can't feel fear to speak. I say this because students from our community live with a lot of fear. Like... like the deportation of their parents, the lack of money, delinquency, and stress. And that's why we're here asking you that surveys be a priority in the system of LCFF. And also to ask you that we sit together to talk about the future of our children. Thank you very much. California for justicia propia. Good afternoon

280everyone and to each of the members here. My name is Sonia Aluelo and I'm a mother of three children and a leader in the Santa Ana School District with the organization Ocopico and Californians for Justice. and attend school. That is why we want the survey to be a part of the LSC system. For this reason, for me, school climate is very important for my children that they feel safe and happy that they go to school. It's for these reasons that we want surveys to be a priority in the system of LCFF. We as parents want to work with you, you the State Board of Education, to make decisions that are going to support our students to have academic success as well as social emotional success. Thank you. Good afternoon, State Board of Education. My name

281is Leonel Velasquez. I'm a committee organizer in Santana with Ocopico in collaboration with California for Justice. With the LCFF, Santana is improving the education, the education, but for our students, parents, and parents of color and teachers, the school climate remains as a concern, a concern, very concern. And we believe that the school climate is the engine for education. That's why we want to ask you to make the school climate surveys a priority for the LCFF system. Also, as a family and as a representative of our education, we need to sit down together to decide for our students, our community. Thank you very much. God bless you. Good afternoon. My name is Floripe and I am a parent from Merced and I have two children in high school. Luckily, I am an active parent and participate

282in the LCAP process every year. Today, I want to stress the importance of improving the school climate and this should be the center of our students' success. I believe that the school climate has not been a top priority and we must change that. Students need to feel safe and have positive attitudes. for learning while parents feel welcome to engage. Feeling safe doesn't mean hiring more police officers or putting cameras or metal detector. What I mean is there's need to be increased like parents and students involving so we can create best practice with the school and hold each other accountable. Also community forums in multiple languages so we can freely express ourselves and feel validated. A forum that is inclusive too. We can all feel good about it. Therefore, please consider school climate as your top

283priority to this coming year. Thank you. Hi, I'm Angela DeRamos. I'm representing California Teachers Association, and this item is near and dear to my heart because I am a classroom teacher. So on behalf of the 325,000 colleagues in CTA and on behalf of our students, like these students, we need these climate metrics on the dashboard now. Last year, my school lost 11 teachers, 11 out of 33. Some teachers were asked to leave, some left on their own accord. The reasons differ according to who you ask, but in a system that relies on stability, collaboration, and relationship building, this was bad. What we have at my school, like many schools across the state, is a revolving door powered by dissatisfaction and lack of support. As a result of this exodus, I was the only tenured teacher

284left in the entire upper grade. Does this affect my ability to work with my team? Absolutely. My colleagues are treading water while sixth grade science camp planning and fundraiser are now all on me. $32,000, I'll accept donations afterwards. Does this in turn affect my students? Absolutely. Teachers who are on constant defense mode or survival mode cannot perform at their best. Teachers who are busy raising money for science camp rather than teaching science are not performing at their best either. Students need and deserve teachers at their best. Will this get in the way of what I need to do to serve my students? I try hard not to. But the honest truth is that my colleagues and I spend an inordinate amount of time trying to build a positive climate in places of deficit. And we

285really could use that help. This revolving door points to a deep need to address the conditions in which teachers and students work. School, climate, and culture matter. The misguided attempts by No Child Left Behind to tether accountability to a single test score was incredibly damaging. The joy of teaching and learning were systematically drained from the classroom, replaced by a Scantron. Scores went up, but at what cost? We've lived too long in the culture of testing rather than culture of learning. And sadly, despite the failings of No Child Left Behind, we are still operating under a testing culture. Where in this process or indeed in any school governance are the voices of the stakeholders? Where are the voices of the learners? The beginning and ending of all of everything we do should be what happens in

286the classroom. Yet in all of this, we, the teachers, the students, the parents, have very little voice. The climate survey would give this voice. Safety, communication, support. These are the conditions that foster collaboration. These are critical to a well-run school. I'd even argue that these metrics supersede the academic indicators because it's hard to learn or teach when you're scared, shut down, ignored, or coerced. Conversely, when school climate function well, the isolation of the classroom is replaced by a common goal, student learning. Let's not forget that student teaching and learning are at its heart social endeavors. The climate metrics are my working conditions and my students' learning conditions. The conditions in climate reporting should be given full consideration this year. Thank you. We have four groups representing the Californians for Justice. Would you please line up

287as well as Jaheim Jones? Hello, my name is Alina Cotton, and I'm a sophomore at Edison High School in Fresno and a leader with Californians for Justice. We are here today because this is an urgent moment to pause and reflect. Right now, students are experiencing a lot in and outside of our schools. Over 50% of students of color are experiencing high levels of stress because of the political moment. More than 200,000 of California students are homeless or have unstable housing. One-third of students report chronic sadness. One-third of students report being bullied or harassed at school. And one-third of students cannot report at least one caring adult. The Searching for Measures of College and Career Readiness report says students' relationship with the caring adult who makes them feel safe, believes in them, and shows a commitment

288to their success are key to their educational journeys. And we can only ensure that success when we make school climate a priority. Because when you make school climate a priority, you make homeless youth feel supported. You make sure foster youth don't fall through the cracks. You make students with disabilities feel valued. You make LGBTQ students feel safe. You make undocumented students feel that they belong. You stop black students from being punished and shamed. You prevent teen suicide. You end bullying. And you make sure that 2 million students that do not have a single caring adult will find one. And you ensure equity for our most vulnerable students in our schools. These are the responsibilities of the school board. In the third grade, I was in an honors math program and I was the only black

289student in an all-white class. I had a teacher who had low expectations of me and even said, not a lot of your people engage in activities like this. And expected me to drop out. I didn't understand at the time why he said these things to me, and I wasn't aware that I was being racially discriminated against. Throughout my year in his class, I would need extra help. When I would go up to him after class for one-on-one support, he would ignore me or pretend I wasn't there. It made me feel invisible, and it felt like I was the bad kid, even though I was, and still to this day, am a 4.0 student. Fortunately, I did not hold on to his perception of me, but what would have happened if I did believe him? What

290about those students who currently don't have an adult that believes in them? The suspension and chronic absenteeism indicator cannot capture this kind of data. But the California Healthy Kids survey question, there is a teacher or adult that believes that I will be a success, can. Yet the school climate survey indicator continues to be delayed, despite students really needing it. That's why my fellow students and I are here today. Because school climate is the heart of our education system. But right now, school climate is being singled out. It's not being treated like the same indicators. It's not annual like all the academic indicators. There is no funding to support it. There's also not the same attention or direction. You need to ask, what kind of message are you sending us as students? Today, we really want

291you to listen, reflect on our stories, and take action to make school climate a priority in the accountability system. Thank you. My name is Diamond Greer. I am currently a senior at Emiliano Zapata Street Academy in Oakland. I hold the position of LCAP Student Director for Oakland Unified. As a foster student at OUSD, there are many challenges that you face inside the school, such as having labels or stereotypes assumed about you. This makes the school really uncomfortable and unsafe space to be for foster students. It makes me feel like I am not seen as a priority, like I don't belong with the rest of the students. Knowing that I am a foster youth, there was no protocol to better support me. At my previous high school, many of the admin and staff would create relationships

292with the students based off stereotypes and assumptions. One of the assumptions that the administration held of me at my school was that my experience as a foster youth was the reason I got into altercations and fights. The principal would tell my guardian that I got into fights because I was troubled and I was delinquent, since I was not living with my biological parents. This made me feel terrible because that was not the reason I got into these situations. When students have negative experiences or issues happening at home and outside of school, they don't have an outlet to impact their experiences or have adults to support them. Getting into altercations and fights can be a consequence of students taking out those frustrations on each other. I felt that there was no one there to make sure

293that I was successful. Ultimately, I had to make the decision to leave the school and move on to a school where I could focus more on graduating and having positive relationships with my peers and staff. When I got to my new school, I felt the support from the beginning. My teachers and counselors didn't just feel like staff, they were mentors. I went through an interview process so that staff could get to know me better and know what my values are. This made me feel like I was a priority and that the staff actually cared about me. Also, when I got to my school, I had a whole team that was proactive in creating a plan to accommodate me with my 504 plan. Before and when I got there and every day I'm able to receive

294one-on-one support with any work that I have I've never gotten this type of support until now But at least now it feels that there is no way I can fail and that I can do so much more as a student However, what if I hadn't been as courageous enough to change schools? Currently there is no data that shows that foster youth have an adult who believes in their potential and support them Only from surveys like the California Healthy Kids Survey is there a question that asks if there is any adult that really cares about you. That's why I'm here to ask the State Board to take action because students like me can't wait. When you have a positive relationship with an adult, you don't just have a relationship. You have someone in your corner. Someone

295who is rooting for you and will work with you to make sure that you are successful. As students, we want to know that we are supported and motivated to do better and to do more than what people think we can be. Thank you. Hello, my name is Hannah. I'm a student at Long Beach Unified and a leader with Californians for Justice. I am here because I believe that school climate is the heart of fostering a positive learning environment to succeed. However, when I go to school, it's a different story. On any given Monday, I walk through the hallway and see police officers staring at my peers and I as if we're a threat. On Tuesday, I hear my teacher frustratedly confess that even though she wants to connect with her students, she's too swamped with

296work to do so. She admits that after class, she sits alone with no teachers or students to interact with. On Wednesday, yet another student of color is subjected to a bag check for simply wearing her hood up. She confusedly looks as a staff member zips open her backpack to search for something that isn't there. She leaves class with an embarrassed look plastered onto her face. On Thursday, a fight erupts during lunch and staff members rush behind students in golf carts, yelling at everyone to mind their own business and get to class before they get cited. Students walk into class alarmed and unsettled. By Friday, I am not surprised to see my peers with uneasy and disengaged faces, almost like they're robots. We simply follow the everyday motions with no means to reflect on everything that

297just happened. We remain shocked and silent. And this is what it's like every day for my peers and I, but it shouldn't be. More is possible if my school prioritized school climate and school climate surveys. If we had surveys, if we had safety, I would be able to walk into my hallways unafraid. I would be treated as a student willing to learn rather than a criminal case waiting to happen. If we had conditions for teaching and learning, every student would have at least five minutes to connect one-on-one with every student in their class and would have a place to collaborate with other staff members and share best practices. If we had relationships, I would have teachers who cared for me on my emotional well-being and not just see me for my grades. They would greet

298me with a smile, ask me how I'm doing, and I would walk out of class feeling energized. And finally, if we had empowerment, I would be able to have my voice valued and heard. We can have students who want to go to school every day, want to participate in class, want to excel in their classes, because we have teachers or counselors who are able to establish connections, even past high school, that are meaningful to us. If my school prioritized school climate, my peers and I would believe that we are meant to shine, that we could become future leaders, professionals, or even teachers. And you can help make this happen by taking action on the climate surveys indicator. Thank you. Hi, my name is Jaheim, and I am a freshman at Kennedy High School and a

299member of the RISE Center. 862 kids go to a school with the lowest test scores but the highest suspension rate but the lowest graduation rate in West Contra Costa Unified. We have 501 students ELD students but have two teachers and only one is fluent in both Spanish and English. My school, John F. Kennedy, has an abbreviation of JFK, but not for John F. Kennedy, but for Jails for Kids. My school has no windows, a rusted door, and some dull lighting. All I ask is just that we support. THE U.S. CLIMATE, SCHOOL CLIMATE BEING THE HEART OF EDUCATION WHICH WILL HELP THOSE WHO SUFFERS WITH DEPRESSION LIKE MYSELF AND THOSE WITH SERIOUS PROBLEMS ALSO. GET THE SUPPORT WE NEED. AGAIN, MY NAME IS JA'HEEM JONES AND ALL WE ASK IS TO SUPPORT THE SCHOOL CLIMATE INDICATORS

300WITH LCFF. THANK YOU. Hello, my name is Armand Matthews and I'm an 11th grader at Oakland High School and I lead it with Californians for Justice. As you've heard from the students before me, not doing anything on the climate survey indicator is a blow to students who cannot wait. We deserve to have schools where we feel like we belong, where we have the support and the conditions that we need to succeed. At school, I see students that are constantly struggling every day with things like depression, anxiety, stress, and trauma. experiencing things at home in our communities which make it difficult to come to class and do homework or focus in school. Even though we live in these realities every day, we are still expected to come to school and do all of our work. I

301have a friend who has PTSD because when he was younger his father was shot and killed in front of him. In the middle class, he was triggered by the sound of a gunshot and had a panic attack. No one knew he had PTSD or how to help him, so after his panic attack, they sent him home, and there was nothing done to help him find a solution to his problem. I wish someone was there to talk to him so he knows that he's not alone in what he's going through. And the reality is, there are so many more students who are going through issues like these on a day-to-day basis. How are students supposed to go to school and just focus on academics? The answer is that they can't. And adults can't ensure student success

302on academics alone. School climate is the heart of our education system. Climate surveys are our school's annual checkup. And when we don't have these checkups, we are neglecting our heart. When we neglect our heart, students don't have a caring adult in their classroom. When we neglect our heart, foster youth and homeless youth remain uncared for. When we neglect our heart, a parent doesn't feel welcomed on their child's campus. When we neglect our heart, students aren't prepared to go to college. And when we neglect our heart, students drop out of school. So as you can clearly see, when we neglect student climate, we neglect everything else in our education system. It is the lifeline of our schools. So if you are serious about closing the achievement gap and bringing equity to our most vulnerable students, don't

303continue to neglect school climate. Students have been putting in the time to improve our education system in our districts, schools, and here at the Capitol. We've been doing our part. It's been 23 months since we first asked you to prioritize school climate as an indicator. Students can't wait. We don't want to keep talking about how important school climate is. We want action. We want you to lead with us. So we are calling on the state board to show your leadership by uniting our teachers, administrators, and elected officials to take action for students and with students. Make this the year of school climate. Thank you. Okay, we're on now. Will speakers 17 through 28 please line up? Brian Lee, Alex Zuko, Fred James, Brian Ravis, Martha Alvarez, Michael McCormick, Lisa Morris, Christy Sinclair, Sarah Monte, Tracy

304Alamanto, Barbara Short, and Justin Conception. Thank you. Brian Lee on behalf of Fight Crime Invest in Kids, Ready Nation, and Mission Readiness. Strongly in support of the work group's recommendation for annual surveys. I feel like that there's a consensus here on a number of things. We all think that we need to let districts know that school climate is incredibly important. that it's linked to all the indicators, that it's critical to continuous improvement. The question is, how do we get there? Do we get there by saying, every other metric we care about every year, this one, only every other year? Do we get there by saying, oh, continuous improvement is an annual thing, but you don't have the benefit of climate survey data every year? I think the answer is pretty clear. Elevating this to an

305annual survey is so critical to send the message that this is important. And the resistance I find very frustrating. There's issues with small districts. It's true we can make accommodations like we did in the grids. People think this is something new. We've had the Healthy Kids Survey for 20 years. 70% of districts do it. Over 200 do it annually. It costs next to nothing. Time. Thank you very much. Good afternoon. I'm Alex Zuko from the California State PTA. We've submitted a letter that outlined everything, but wanted to come and give a vocal, strong support to the annual administrative school surveys to all the stakeholders. But specifically, we wanted to urge the board to adopt a definition. Because without a definition, you don't have the foundation. And we can appreciate the local aspect, but the state

306defines things all the time, and local agencies handle it. It will be extremely difficult to sustain the progress that the workgroup has made, the workgroup that everybody praised, if you don't actually set this foundation. That's all. I think I'll give something else for the rest of my time. Thank you. I'll take it. Good afternoon. Fred Jones on behalf of the California Council for the Social Studies. And we also submitted a letter that's public record in your packets, along with Ed Trust West and our science teachers, counterparts, and others. I'm here to talk about issue four of item three, which is priority seven, dealing with broad course of study. We think the recommendations thus far... are inadequate. We believe that each of these subject matter disciplines should be specifically called out and that the LEA should

307specifically provide either narrated responses or what have you about what they are doing for each of these subject matter. We spent a lot of time this morning talking about academic indicators. Well, that's English, language arts, and math. So if we go down that route and don't provide a robust opportunity to talk about the other subject matter disciplines, history, social science, science, career, technical education, and the arts, then we're going to be sending a wrong message and continuing to narrow curriculum, and we should be broadening it. Thank you. Eric Premack, Charter School Center. I wanted to share some concerns about the instability in the test scores, and I appreciate Board Member Burr's comment about not using. These sorts of results for high-stakes purposes, but for good or for ill the charter laws require that these be

308front and center as the primary Lever for making charter renewal decisions or at least that's the way some folks read the law And so the volatility in these scores is of extreme importance and interest to us We are very concerned about what strike what we believe is a lack of transparency with the private meetings To me meeting privately is meeting in secret particularly when you don't know the people on the Commission or the committees or the ones that I know it's only because of my personal contacts We think that there needs to be more transparency around this. We need a better grip on what these scores mean. I would hate to see us have a repeat. Many of us are old enough to remember the old testing system that collapsed 20-some years ago because of

309a lack of public confidence, and I think you're right on the verge of that right now. I'd urge you to get on that and appreciate the work that you do here. I have enormous respect for the work that CDE and other staff are doing on this. I thank the world of Mr. Ashley and his staff, but I do think we have a transparency issue here. Thank you. Good afternoon. Marta Alvarez on behalf of AXA, the Association of California School Administrators. First, on the academic indicator, we appreciate the CDE recommendation to make adjustments to the cup. points for the change aspect of the 5 by 5 grid in recognition of the latest SBAC results from 2017. At the same time, we echo comments by the board members about increasing communication to the field so that they

310can better inform the public why we had some leveled scores and results in this past assessment. Regarding the school climate condition, we echo many of the sentiments, probably all of the sentiments you expressed about making sure that while we want to elevate student voice and really empower students to feel welcome at school, that we also make this a local control, local indicator. We have many more comments that we provided in our letter and that we refer to you in terms of the implications for doing an annual survey and why that might be a challenge. Lastly, in the course access priority number seven, we look forward to providing more input regarding the implementation of that self-reflection tool. I'll quickly just summarize. We believe we need to integrate CCI in that conversation about the local analysis. also

311thinking about how can we ensure that the indicator itself is looking at other indicators. Thank you. Good afternoon. My name is Michael McCormick, Superintendent of the Val Verde Unified School District in Riverside County. We're a district of 20,000 students with an unduplicated count of 83.8%. I'm speaking today on behalf of AXA as a State Board Director and the past President of Region 19, which covers all of Riverside County. And finally, I'm a member of AXA's Accountability and Continuous Improvement Task Force. I'm here to report that we're very grateful for the improvements to the dashboard. Specifically, we thank CDE and the State Board of Education for pursuing functionality improvements to the dashboard and the equity report. It will be helpful to have schools listed on the dashboard website for parents to be able to see student

312performance on the multiple indicators. The webinar series, Rolling Out Individual Indicators, has been of great help to administrators and having the opportunity to see the data through the preview has enabled administrators to start analyzing their data. CDE staff has been very responsive to our dashboard questions and we are appreciative of the movement of the narrative statements from the bottom of the dashboard to the top of the dashboard. We also appreciate the parent resources including videos and the one-page fast start guide. Regarding the academic indicator, we thank you for the proposed adjustments to the change cut points in response to SBAC 2017 results. We concur it is important to look at the change over time to determine trends and patterns once stability in the data is achieved. We concur with board member Holliday and others that

313additional communication to the field is important to help explain why the 2017 SBAC scores were leveled in California and other states who are part of the Smarter Balanced Assessment Consortium so that we can really explain this to our local communities that we serve. With respect to the school climate surveys in Val Verde, we actually annually administer surveys to students, families, and school staff. The survey is much more comprehensive than just looking at school climate and school conditions. The survey is comprehensive and focused on various components of the LCAP. On the other hand, I'm hearing from AXA members that it is much preferred that districts be given the discretion on this local indicator of how often they administer the surveys, the types of questions asked, and whether to focus questions on students, families, or staff. The

314suggested annual requirement may not work for many school districts across the state as districts need time for programs, policies, and practices to be in place. And also realizing that there's lots of many local contexts. So lastly, we're... We're appreciative of all the changes that are being made and we encourage SBE and CDE to revisit opportunities to streamline and consolidate reporting requirements. We support the SBE and CDE commitment to a single coherent system initiative of reporting that keeps parents and communities that we serve as the readers of a single document. Thank you. Good afternoon, President Kirst, State Board members, and Superintendent Torlakson. I'm Barbara Short, the superintendent of the Arcata School District, and I thank you. From the bottom of my heart for your thoughtful consideration of the letter I sent you with this packet. One

315of the things I am appreciating about the State Board and the Department of Education is the collaboration at all levels between stakeholders, districts, parents, students. I'm very heartened by your thoughtful consideration of everyone's comments. I wanted to address many things, as you can tell by my letter, and it's going to be tough to keep it short. I'm here representing AXA Accountability Task Force, as well as small school district administrators throughout the state. My district has 520... public school students, and another 550 in four independent charters. I am a large district where I come from. There are about 100 of us up north, and I know there are another at least 75, close to 100, scattered around the south and eastern parts of the states. I serve as special ed director, superintendent, family resource center director,

316all of these things, as you noticed in the letter. I wanted to speak to the consolidation of data reporting because every request and accountability mandate that is generated by various offices and agencies all cross my desk. And what I hear over and over from administrators in my strata is that it's just constant and it requires so much energy to address. We really value data and data-driven. decision-making and evaluation and in fact our LCAPs are robust and data rich and I Invite any of you to please come visit me or even do a phone conference if you'd like a deep dive into my LCAP and our LCFF. It's comprehensive and meaningful because we have rich engagement with stakeholders. One way we do engage stakeholders is through our surveys. We do one annually. We are on the

317fifth year of this survey. Every year our stakeholder group of parents, staff, and community members reviews the items. We are comprehensive. Our survey addresses every possible, every priority the state wants plus some local priorities. And we have multiple items that address each construct such that we corral the information in a contextual way. We analyze our data, present it to the board, and present it in our LCAP. And then we use that data to make decisions regarding allocation, direction of the district, and allocation of our resources. Though that works for us, it may not work for all districts. Our small size has a lot of complications, as you heard in the letter, but it also allows us to be nimble, and our data is personalized. When we look at our test scores, we know all those

318data points. We know their families. We know their grandparents. And we do include grandparents and family members. Survey links are sent to everybody. So I want to thank you for your flexibility, your acknowledgement of all of the moving parts and pieces, that we do need time to work with this system as it is. It's just now becoming cemented, and I believe it is working. So thank you very much. Would the following people please line up? Marina Brown, Emily Athour. Susie Costa, Jill Grace, Julie White, Craig Givens, Samantha Tran, Martha Diaz, Fleur Faye Dibbs, Jesus Andrade, Terry Burns, Darryl Wu, and Mayim Deizi. President Kirst, I'm Brian Rivas speaking on behalf of the Education Trust West. With regard to the changes to the cut scores for the academic indicator, I wanted to share that our view

319is that the change combined with the way in which that you arrived at the decision, I think is going to leave you vulnerable to the criticism that you're changing the rules because you didn't like the outcome. One of the ways that you can lessen that, I think, maybe in the future is by being extra transparent or being overly transparent. I realize that maybe the technical design group needs to have its space, so to speak, but perhaps in the future you could have an extra stakeholder session that allows us to engage with the technical design group. We would like to have more transparency so that we can vet the conclusions because this is a pretty big change. I think that way you could avoid having a narrative. coming out of this meeting that critics say you're

320changing the rules because you didn't like the outcome. Secondly, on the access to a broad course of study, we think you could make it a hybrid indicator by combining statewide data. Thank you. Good afternoon. My name is Julie White, and I'm representing the core districts. I wanted to reiterate a point that Sue made earlier about the board's upcoming work on the student growth model. The student growth model, as you know, will substantially alleviate the volatility of test score change because it, by design, adjusts the predicted results for each student based on the actual results. We've offered to Eileen and to Karen for our course superintendents to come and to explain how the local grow how the core districts growth model is working at the local level and I just wanted to extend that invitation to

321all of you so that you Could consider it and our superintendents could come in a future meeting and talk about growth. Thanks Good afternoon, President Kirst and Superintendent Torgerson, members of the board. I am Jill Grace, the president of the California Science Teachers Association. And I thank the board members for their questions and direction to CDE staff on this issue, many of which are consistent with the recommendations we articulated in our letter. CSTA stands with our colleagues at the Education Trust West, California's Together, Children Now, California STEM Network, and the California Council for Social Studies. And we ask that you strengthen both state and local level. indicators for priority seven. As a state level indicator, the college and career indicator is a good start, especially if it evolves over time. CSTA recommends that CDE take

322full advantage of currently available course data for grades seven and eight and include that data in the state level indicator, strengthen the self-reflection tool to modify the language of the prompts as detailed in our letter, as well as provide sample tools for use by LEAs and schools. As Member Williams mentioned earlier today, access to a full and rich course of study for TK-12 students is critical for lifelong success for all students. Thank you. Good afternoon. Martha Zaragoza-Diaz representing the California Association for Bilingual Education and Californians Together. I would like to make two points with regards to item three. The first one is on the broader access, broader course access. This is an issue that has been problematic for English learners in terms of their access to a broader course of study. And this has been

323really a situation that has been identified to us by their parents. So we believe that the questions that are reflected in the self-reflection questionnaire or self-reflection tool will not provide meaningful information to the board with regards to the extent of access to a broad course of study, especially by other student groups and especially by English learners. We recommend that any reporting on this indicator must show access by individual subject area, by grade, and by subgroups to provide meaningful information to the board as to the extent of course access by these students. Thank you. Good afternoon. My name is Mayuri Desai, and I am a medical pathways teacher in the Garden Grove School District. I've traveled from Southern California to have my voice heard. As many others have said, I support prioritizing school climate issues, and

324when I first came to this country, I was bullied because of my culture. I'm very concerned about bullying in the classroom. Similar to other students here today, many Hindu American students face bullying in the classroom, and this is why I believe it's important to support school climate issues. Textbook content is often related to the bullying of my community students. Unfortunately, I won't be here tomorrow, so I urge you tomorrow to adopt only those textbooks that promote an active representation of all communities and urge you to reject the drafts from Hukton Mifflin Harcourt. Thank you for your hard work and for engaging with us for the past four years. Have a great week. Thank you. Good afternoon. Terry Burns on behalf of California School Boards Association. We appreciate the detailed and deeper explanation of option one

325today and are now comfortable with option one. We particularly like the narrowing of the yellow bands. We think that will reduce the broad disparity of types of districts that are in that band. And we concur with members Rucker and Ortiz-Lincon on the need to see how this impacts individual student groups, keeping that... focus on gaps and closing gaps will be significant. Since you're not acting on climate today, I will just thank those who address the capacity issues involved in that. And as we address broad course of study, we would similarly suggest that you might want to, for capacity reasons, not mandate annual collection of data. Course access is not likely to swing radically from year to year, so biannual would seem to be useful. We think that LEAs should be expected to provide data by

326school site and student group in question three of this self-evaluation tool. That was the purpose of this was to determine equity and who's getting it. We need to have that data. Just to clarify this item is on the state accountability system. It's not the history social science framework So comments on the history social science frameworks are not appropriate to this item that I'm going to be here tomorrow So for anyone's wishing to speak on that item that item will be heard tomorrow. Thank you Good afternoon. Lee Angela read on behalf of the small school districts Association want to think members Burr, Strauss, Reker, Ting for their comments relative to particularly the climate school climate conditions indicator and appreciate very much the understanding that districts are diverse and have different capacities and different needs. I want

327to thank my friend from Arcadia for so well representing small districts and I'm pleased to represent all of the districts on their behalf. There have been some comments made about what we should make adjustments for smalls Well, the fact is over 60% of our districts are smalls, right? And I think it's important to recognize that I want to thank the work group for for their efforts that they've put in I think they're a very well-intentioned very smart group and have done a lot of research But the fact is there's not a member on that working group who's responsible for an LCAP And it's important to have that balance and that perspective as you move forward with regard to the end Surveys just allow time for the input to have value, please Good afternoon, Sarah Lillis

328with Edvoice and on the academic indicator We do we know we're gonna be here again talking about reflecting on both the outcomes and the measures themselves But we hope that we can really think at this moment about what this process is going to look like because we are continue to be very uneasy with option number one both because of a lack of information about the technical side, but more importantly because of a Lack of opportunity to have a really robust policy conversation about what the implications of the shift are Not just the cut scores. We understand that there are technical reasons for that and we don't know what good practices on that how often you can shift it when it becomes sound statistical adjustment rather than Being honest about what the measures are but more

329on the shifting of the colors We really want to think about what the implications are for improving outcomes for students, right? That's what we're trying to do with this system and so We worry that this is setting a dangerous precedent I'm changing colors to identify fewer leas for support and assistance if the colors change over time Is there authentic and and if we're serious about change can we say green ever? Regardless of how you measure it is for significant decline. How can that be green if we really are serious about improvement and change? Good afternoon. Liz Guillen with Public Advocates. First of all, we concur with the eloquent comments and requests from parents and students that you heard earlier around the school climate survey. I want to urge you to use your personal clout to

330get the governor to put this in his budget in January, support for climate surveys. We recommend, and in our Equity Coalition letter, and I repeat now the suggestion to make the course access indicator a hybrid since there's plenty of state information around middle and high school courses. And you could use that as the state indicator. And for the K-8 or K-6 self-contained classroom courses, that would have to be done as a local indicator. I could see that. But we think you should make it a hybrid. We want to support Member Ortiz-Licón's suggestion to look closer at the comparability issues with the 2017 SBAC so that we're not rushing into changes in the academic indicator. Thank you. Good afternoon, Samantha Tran with Children Now. And I wanted to start by saying that we appreciate all of

331the work of the department, and in particular wanted to commend them on the suspension expulsion report. I think it will be really helpful. We are supportive of the working group's recommendation on the school climate annual survey, as well as the hybrid approach to the local indicator. On the technical design group conversation, I just want to comment that We don't think anything nefarious is going on, and we actually really appreciate and have the need for that type of body. From the public perspective, I think the issue that's being raised is we don't have a good sense of who the composition of that body is, what the agendas are, what the materials are, what the meeting minutes might look like. And because these technical issues impact policy and the timing of some of these decisions, we just

332think it needs to be... more publicly accessible and available. As was commented on earlier, Children Now does have concerns about not changing the color when you have a 15-point drop. And I know there were some issues raised about the analogy that we laid out in our letter. So let me try and come at it from a different perspective. When you have a district average student who's at the median and they drop... by 15 points, they're down to the 35th percentile. That's 150 districts that they're dropping. And this decision will mean that the color change, the color coding system doesn't change. Thank you very much. Okay, that concludes the public comment. And I think we're ready for a motion. Usually our liaisons make a motion. Yeah, maybe Cindy we have you have the motion up there

333There's three three parts to this one for the academic indicator one for the chronic absenteeism indicator and then finally for the LCFF Priority seven access to broad course of study local indicator So I'm prepared to make a motion on numbers one and two. So the academic indicator change that we spent a lot of time on and we did not game the system. I have to say that for the 10th time based on public comment. If we were going to game the system, we would not have increased the number of districts that need support. The chronic absenteeism recommendation is fine, but the third one on the broad course of study, I think there was some confusion about whether we were adopting a standard. There were lots of concerns raised about what does a broad course of

334study actually mean in English, math, actually articulating them. So I'm going to suggest that instead of using this language, that we actually say that for the broad course of study, Priority seven, approve creation of a local indicator and direct staff to come back to the board by March 2018 for approval of the standard in conjunction with approval of the self-reflection tool. So we'll see the standard and the tool at the same time, and we expect that the feedback you got here from us today would be incorporated into that. So that's my motion. Second. Second to that, further board discussion on this item. Board Member Ortiz-Lacombe. I just have a comment. I know that now we are anticipating perhaps revisiting the academic indicator, the change scores, the cut scores, perhaps the colors next year. And so

335we now know what the perception is. You know, I know that part of it is that we haven't built or restored trust with some groups. And I want to say that it was, from what I heard, it wasn't just equity advocates. It was a pretty wide range of stakeholders that expressed the same concerns about the process. Not always the recommendation, although there was something about the recommendation. But if we know that this is something that will likely come back next year, if we could start... sending out those memos, putting out that information, gathering stakeholders, because now we know what we're going to be encountering. And now we don't have to rush through it because we have the sense that this may be a reoccurring issue for a couple of years until the data and the

336methodology are able to stabilize. So let's restore some of the trust. Let's be transparent, or as transparent as we can be, because now we know. Please make sure that that communication goes out not only to board members, but obviously to to our the public as well. Thanks Any other comments? If not, we're all ready those in favor of the motion raise your hand Okay, it's unanimous Okay, we're now on to item four and it's only two o'clock so No break no stretch break feel free to stretch as I pull up my slides All right, I am so excited to be here for item four. My name is Melanie Sheppey. Good afternoon, President Kerr, Superintendent Torlakson, and members of the State Board. I'm the Director of the Improvement and Accountability Division at the California Department of

337Education, and I'm so excited to present the November update on California's system of support for local educational agencies and schools. Similar to the September update, we'll focus our conversation on the 2017-2018 school year. Normally when you have me up here, I make sure to connect it to my Greenleaf Scholars or something that's connecting us to what's going on in the field right now. But I think the student presenters for our last item did a really great job of that. And I don't think I could top any type of antidote or story. And so I just want to kind of have them be the ones who are rooting us so much in what's going on now. as we're at November in the 2017-18 school year. So for today's conversation, we'll begin with a review of support

338and assistance, and we'll touch on the possible examples of what support will look like for districts identified for additional support under the local control funding formula. Then we'll highlight specific examples of work that's already underway for local educational agencies and schools, similar to what we did in September. We'll close with a discussion on an approach to gathering feedback and how districts and county offices of education can access additional support when improvement efforts aren't meeting the needs of their students. Then we'll have time for questions and feedback. You've seen the next two slides before. This slide reviews the shifts and improvement under the local control funding formula. You'll notice the shift of support providers working alongside local educational agencies and their schools, identifying key challenges and opportunities. Also, the shift from packaged interventions and checklists for

339improvement to using a systematic approach that can be tailored to locally identified needs and strengths. And of course, one of the most important shifts is engagement with local educators and communities as part of the decision-making process. In addition to the shifts and improvement, there are elements of support. What's key here is that for some districts, the review of data and understanding underlying causes might be the exact support they need. For other districts, they will need access to additional expertise and potentially implementation support to make sure the changes they want to implement actually affect teaching and learning. When I was a principal, we held academic conferences every six weeks with each teacher to understand their data and the underlying causes. I highlight this because often the support needed, like I said, is that protected time with

340deep coaching to understand our students and their work through multiple lenses. And sometimes you need to do that and have additional support. As we worked with stakeholders, we learned that the shifts to improvement The elements of support which we just briefly went over were helpful But stakeholders wanted more information to help visualize what will support look like The item outlines a few scenarios of what support and assistance can look like for districts that are identified for differentiated assistance within different contexts There are several more that we could have described but we wanted to focus on the situation in scenarios that people were asking the most questions about This is not an attempt to create a checklist or the right way to go about supporting a district in their community, but instead a way to help

341visualize the future. Because people only know what they've previously experienced, and it's our job to help change expectations and try to further outline what support can look like under LCFF. So what does this mean? You'll notice in the examples in the item that the local educational agency is the primary driver of assistance. In each example, the data analysis and understanding underlining causes all should strengthen a district's local control accountability plan. There's no separate improvement plan. Instead, for each case, a district and their community accesses support to better understand their data and the underlining causes. They, with their community, determine what changes need to be made to current practice in order to get their students closer to their goals, which helps their local control and accountability plan evolve. As you all know, and as we've heard

342today, districts have different needs throughout California. And as a result, we need to leverage opportunities to make sure districts are connected with the right resources at the right time for their teachers and students. This is why today we'll continue to highlight support that is already underway. That support can be leveraged when we think about districts that will need additional support. These examples of support are available to all. And like I said, by highlighting them, we can further encourage that we have districts, when they're identified for additional support, can be connected to the right opportunities aligned to their needs and context. I want to recognize all the hard work of the presenters today. Each of them could honestly present on their work for hours. But for today in order no laughing about that. I'm just saying

343we're not going to Point of my next sentence So each of them could honestly present on their work for hours But for the purpose of today to continue to highlight work that's already done. They've they've created succinct slides and succinct presentations We'll begin with an overview of the coordinated approach of County offices of Education going a little bit more into detail into the work that they've done. Then we'll dive into California's multi-tiered system of support work and learn a little bit about the proposal for assistance, and we'll learn about a proposal for assistance to local educational agencies identified for support for the improvement of their students with disabilities. Then we'll learn about the Lake and Mendocino County Office of Education Professional Learning Network and the Sausalito Marin City Schools pilot with the California Collaborative for

344Educational Excellence. I'll come back up here to review our approach to gathering feedback, which I'm really excited about and looking forward to hearing your feedback on, and how districts and counties can access additional support if needed. So we will start. Good afternoon. I'm Christine Olmsted from Orange County Department of Education and I'm the incoming chair for the Curriculum and Instruction Steering Committee for CSASA. And so I was asked to do an overview of what we've been working on as our differentiated assistance approach to local education agencies with the release of the dashboard later this month. Just a brief history on that. We actually started working on this in 2014. A group of six county offices came together with CalEd partners to work on what we called the County Office Collaborative. And it was Sacramento County

345Office of Ed, Tulare, Ventura, Orange, Shasta, and Riverside. And we worked together. we were first looking at how do we provide supports for common core state standards. And instead, that all shifted because we felt that there was enough support for common core state standards with all the work CDE had already done, with our toolkits that were out, that we really started looking at, we knew the evaluation rubrics were coming out, which is now the dashboard, and said, how are we really going to support LCAP and LCFF and the evaluation rubrics? So since 2014, we've been working with this, and Michael Fullen actually worked with our team and helped us come up with kind of these key points that you see here today. So looking at building capacity through professional expertise and local decision making. fostering

346systemic collaboration, building a culture of co-learning and reflective inquiry, promoting a climate of candor, evidence, and urgency to take action, and results in improved student outcomes and leads leading to sustainable change. And our goal is really to work alongside our local districts, not come in and tell them what they need to do, but help them look to see where their areas of strength are and where their opportunities. opportunities are. So that's the first thing we'll be doing with them is looking at their dashboard analysis and how we can help them there. We'll also look at their local measures and their local indicators. We'll use system analysis tools. So as a county office education agency, the CIS committee has come up with an LEA self-assessment tool built on several different frameworks to be able to help

347LEAs look to see where they're at. We also have the Fidelity Integrity Assessment that comes out of the Swift Center out of the University of Kansas that helps schools look at how they are at building capacity. So we'll use that as system analysis for them. We'll help them with a root cause analysis as well. And so we have over 200 participants right now in improvement science looking at how do we help our local education agencies go through a root cause analysis. And then we'll do a synthesis of their findings and give them a written letter for them then to decide what they want to do based on our work with them and looking at their strengths and opportunities. So that's where we're at with county office level support for our local education agencies. And next,

348we will dive into California's work on multi-tiered systems of support. So I brought with me my team today on the other hat that we wear on scaling up multi-tiered system of support statewide. So with me today, I have Jamie Parsons, who is our administrator, Susan Huckanen, who is our Butte County Office of Education and Rural Cheerleader, and Edgar Montes, who's our director of our scaling up multi-tiered system of support statewide. So we're each going to talk to you about where we're at. in the process of this work that you have granted us to do for you. So Jamie's going to talk about our theory of action and our outcomes. Thank you, Christine, and good afternoon, board. The theory of action for the California MTSS initiative articulates our belief about how to most effectively improve instruction

349and student learning and support the whole child. It focuses on achievement gains for all students, whole system engagement, A strengths-based approach, which looks at what is working well already within districts. And implementing an MTSS framework with five evidence-based domains and ten features, which you will find in your folders that you've just received. And we call it the placemat. It is a two-sided document that says all means all. Our anticipated outcomes are to increase and improve services for all students with emphasis on student groups under LCFF. Currently, 69% of students in targeted student groups are students with disabilities. Of those 69%, 12% are socioeconomically disadvantaged, 15% are ELL, and 25% are foster youth. Our goal is to support student success in the least restrictive environment and foster greater inclusion because we know that sometimes students get

350sent to the land of somewhere to get fixed by someone and hope that someday they may come back. We will leverage and coordinate multiple school and community resources. To help schools not work in silos any longer We will implement a district wide and school wide system of academic behavioral and social and emotional support and We would like districts to include MTSS in their el caps for sustainability if you look on the flip side of the placemat You will see the alignment between the MTSS initiatives and the el cap priority areas my colleague Christine will enlighten you about our system of support. So one thing we really looked at as we developed this work was this concept of whole system engagement. And knowing that we needed to reach the over 1,000 districts and charter schools within

351our state, as well as 11,000 schools within our state, we knew we needed the support of the state, the counties, the regions, as well as the districts, the schools, ultimately because the students are the purpose of our work. And so what we've been working on for the past year is this concept of the district as the point of intervention and really focused on building district leadership capacity to best support their schools. So it isn't one more thing. We're not coming in and saying do MTSS now. MTSS is a framework of supports for students to meet their academic, social, and emotional and behavior needs. And when you don't have systems in place at the district level, to make sure all schools are supported equitably, it won't work. So we've been working with system change at the

352district level, and Edgar's going to talk to you a little bit more about what that is looking like for us. So this is our timeline of implementation. As Christine mentioned, our initiative seeks to engage all of our LEAs at every level. In addition to building capacity within our regions and county offices of education, our initial training also consisted of 92 school sites demonstrating promising practices of building their own MTSS framework. We refer to these as our knowledge development sites or a KDS. As part of our initiatives deliverables, our office is also awarding over 20 million in sub grants to help bring LEA teams to our trainings. In our first of three cohorts, we awarded 105 grants to LEAs throughout the state, impacting about 1,900 schools. This map here represents the 145 LEAs across our 11

353successor regions that we have started to engage in this work. On our website, ocde.us backslash sums, you can find an interactive map with all of our knowledge development sites and their promising practices. Last week, our deadline closed for our Cohort 2 application period, and as of this morning, we have an additional 170 LEA grant applications. Training for Cohort 2 begins in January of 2018, and our Cohort 3 application period will be released in February of 2018 with training in August of 2018. So this next slide is to highlight implementation science and the stages towards sustainability of one system of support for all students in every LEA and school across California. And what we know is that true sustainable change takes two to four and sometimes up to seven years depending on local context. The first

354stage of implementation is what we call laying the foundation or exploration. This is a critical stage and includes the identification of members that will convene as an implementation team. These members have oversight to identify resources such as strategies, coaching, facilitation, as well as examine measures of fidelity throughout the change process during each stage of implementation science. Knowledge development sites and cohort one have been in training to utilize the MTSS transformation in action tools that will ultimately help guide them through the process over the next four years in order to foster sustainable change. And you can find the transformation in action practices in that folder that you have. One of the rewards of this initiative, and true to my rural heart, is that it has provided equal funding across all LEAs, including those in rural and

355remote regions, which represents, as we've heard, up to 50% plus of the districts in California. As we gather data and roll out this initial foundational stage of our MTSS initiative, we've listened as LEAs and KDSs, or knowledge development sites, remind us that aligning initiatives under the umbrella of a system of support makes ultimate sense, yet it can be messy and at times disruptive, but nevertheless is transformational. And now we will be showing a video. that will highlight the impact of our initiative as it is reaching the hearts and minds of our educators across our state. In having this California SUMS that's going on statewide, we're beginning to speak the same language, which has not always happened in the state of California. You take the LCAP, you dance with MTSS, we're beginning to speak that same

356language. And while it means different things at different districts based on student need, we at least have some common vocabulary. Being part of SUMS has been extremely beneficial. I had an understanding of what I thought MTSS was. I think we all sort of had our own spin on it, but to be in the room with like-minded individuals, to have the guidance and the support of what is evidence-based, it has been invaluable to helping us really create a strategic plan. We have seen a huge decline in the number of SST referrals for this year. I would say the number of students referred for special education assessment and testing has also decreased dramatically on our campus this year. We're starting to see a lot more integration of our general and special education classes. We're increasing the number

357of co-teaching, and I just think that's because of the relationships that are being built staff-wide. For the last probably 15 years, we've been focused on intervention, but with MTSS, we are now going to layer that prevention in there. It's awesome. I love MTSS. It is just what are you doing to support students under this clear umbrella, a very concise, clear way of helping students as they move forward. So it really gives schools a focus and then the starting points and then where you can grow from there. And then sometimes you take some steps back, but then being able to assess where you're at along the way and evaluate your progress along the way, because that's a key component as well. This is groundbreaking stuff. You know, when I first got an education, I remember we

358were working on standards and the STAR test and you know that was already in play and I just didn't feel like that was where we needed to go to really make a difference for our students. So to be part of this initiative and knowing that we're changing education, public education is changing and we get to be a part of that. It's hard work but it is so gratifying. Good afternoon board members. Kristen Wright and Suzy Shin, the California Collaborative Educational Excellence. Can you hear us? No. Good afternoon. Better? Okay. We are pleased to be here, and it's great to follow this MTSS work because we all know that building special ed on a foundation of excellent general education is really the direction that we need to go as a state. So we're standing here before

359you on somewhat of a momentous occasion in that the level of collaboration and alignment that's occurring across not only state agencies and partners, but with our regional partners and our district partners is something that I've never seen before in my years of being here and doing this work. And we are really, as you know, with the Special Ed Task Force, we really were looking for one coherent system of education and This gets us as a state this dashboard the work that the board's done the work that everyone here has done and with our partners is bringing us closer to that realization and so as you know we have more than 750,000 students with disabilities in California and That represents about 11% of our student population. It's not a small number and so In order for

360us to really look at the challenges and make sure that students with disabilities are addressed, we're looking at our LCFF subgroups, and we see that within those subgroups, as was mentioned previously, our students with disabilities make up those subgroups too. So let's move on. Thank you. And this demonstrates, as you've seen before, the overlap that our previous presenters also mentioned. The important thing about this to mention is just that we see our students with disabilities in these subgroups that are slightly overrepresented. And that gives us an indication that what we like to see across the board is the same number of students with disabilities across our entire population. Again, that represents almost 70% of our students with disabilities are in one or more of the LCFF subgroups. And so really, this conversation that we're having

361today and that we're having ongoing is really about all our students that wear multiple challenges. And so we are here collectively and collaboratively to address those multiple challenges that all of our students come with. So just a reminder about... what our California Education Code states about the purpose of education in California is that each child is a unique person with unique needs. And the purpose of education is to enable each child to develop all of his or her own potential. And so we're really working towards realizing that greatest potential. And so as we're having these conversations, I just hope that we're not thinking about... taking something from someone to give someone else, but really helping to meet the needs of all of our students, which is really what the purpose of an MTSS framework is

362and really building a foundation of accessibility and universal design for learning. and all the great things that we're putting in place in our state. So I am really excited about this work. I'm really excited to be working here with CCEE, and we have a lot on our plate moving forward, both internally in the Special Ed Division to try to meet the needs and resource needs of all of our schools and districts, as well as our statewide needs and working collaboratively with our partners. So we'll talk a little bit more about the proposal that we have in front of you today. And the proposal that we are putting forth really focuses on creating a statewide approach to frame the way that all LEAs develop and then deliver support for every student within their system. And to

363ensure that's an integrated part of the larger statewide system that we are working on as a collaboration. Kristen laid out, we'll be talking here a little bit more about the why, as she just framed, as well as the how of this approach, but won't be focusing on the what's or be limiting it to, you know, limiting the kind of what we were going to be doing. That said, as the MTSS team that came up in front of us laid out for us, This work and our approach really does build on that framework that they've created for support, which is echoed in kind of our approach to ensuring that that statewide system of support is cohesive. So in front of that kind of pyramid in front of you for support, at that base we have universal

364supports and resources for all. And that means we want to remain focused on the fundamental philosophy of our approach, which is that this can't sit outside as a separate initiative or program or a set of practices and policies, but that we have to integrate this into our general education system and within that framework of the statewide system that we are currently building out across all LEAs. In that middle tier, you have targeted supports and resources for LEAs, and that works on coordinating, connecting, and developing existing and needed resources to support LEAs focused directly on addressing specific needs to implement both targeted and integrated approaches. And that means we'll be building on work that's already gone on, including the work of and the recommendations coming out of the Special Ed Task Force, our strands of work

365across our state-funded streams, as well as... work of our current professional learning networks we have through the CCEE. That top tier for direct intervention for identified LEAs, it speaks of the ways in which this work will need to be coordinated with direct intensive intervention provided to districts as a result of the statewide systems, again, that we're talking about. And as we think about this work, I can't emphasize enough this need to move forward with this in a way that is different from the way we've talked about it before in making sure that it is... part of an integrated system of delivery in the way that we support every student and every classroom. So as we think about the support framework, we're looking at three basic strands in the ways, again, that we're approaching this work.

366And this goes across all tiers of that pyramid that we laid out. That first strand. focuses on the identification the coordinating across existing programs and Ensuring that we're looking at what the county office is doing our self as recommendations from the SPED task force professional learning networks to ensure that we're not starting afresh or starting separately from good thinking that's already happened, making sure that we're integrating what's already happening and being used in our schools and systems, enters into the conversation for how we move forward. Looking at how we can support LEAs vet existing resources for quality, relevance, and usefulness, examining what resources currently exist and what now need to be developed. In the sort of innovation and research realm, we take a look at what kind of innovations are taking place in our LEAs.

367Where are some of these, you know, taking lessons learned from our bright spots and identifying best practices and thinking about what some new approaches and processing tools are and how they can be tested and thought about how they could be applied in new ways and how small successes can be scaled up. And then sort of taking a look at some of the analysis and identification of some of the bright spots and examining more deeply what is happening for our students in the field. Where can we take lessons learned from districts who are doing really great things? From our initiatives and our MTSS sums work to take a look at, again, what's happening in the field, how we can scale things up, for whom our strategy is working, and where do we need to come in

368and develop new resources and policies. We anticipate, you know, as we think about this work, we anticipate that a majority of our LEAs that are going to be identified for support in December are going to be identified as a result of the performance of the students with disabilities category. And I think that comes as, and, sorry. performance of their students with disabilities subgroup in the areas of their academic performance, their English learner progress indicator, as well as graduation suspension rates indicators. And we take a look at that and examining who our student body is, this really speaks to the fact that this is not a separate issue, but something that, again, we need to really think about and advocate for as a state system in terms of how we integrate these supports for every single

369student within our school system. So that said, this has been a very quick partnership to come together as we think about facilitating the approaches and the next steps that we need to take. We are facilitating. ongoing conversations and planning meetings again with our partners, the county offices of education, districts and charter schools, our SELPAs, research groups, to come together and get a grounding on again what is happening in the field and what we can learn from to build upon. We are planning for a launch of this in February. In addition, as part of a CCE next step in proposal, we are putting together a proposal for the development and implementation of two new professional learning networks that are going to be focused on the needs of LEA serving students with disabilities that will be integrated

370into part of the larger exchange of 58 existing professional learning networks across the state that are examining issues that are local to them integrated into this process of continuous improvement. And we will continue at the department to retool, realign, and re-prepare for more of an onslaught of individuals needing assistance. That doesn't say that we are not continuing to do our federal monitoring where we touch nearly every single LEA in California in some type of activity. So it's just about coordination, it's about alignment and cooperation with our partners so that we're all coordinated in that effort to assist. As one of our district superintendents just noted as we were talking about this issue around students with disabilities, that really the dashboards provide them the mechanism to bring special education and their students with disabilities and their

371families into the broader conversation. Some of the comments we've heard in the past that they felt a little bit of left out and that really it's a chance to dig into the data and determine what else they can do to address the needs of each and every one of their students. So we see this as an incredible opportunity. We feel at the table. And we look forward to coming to future meetings. I think next time we're coming as a special ed division to let you know sort of what the data says. So thank you. So we started big picture. We talked a little bit about the coordinated approach to county offices, talked a little bit about the multi-tiered system of support work. We got to hear about a proposal. And now we're going to go

372more specific, and we're going to hear about the work of the Lake and Mendocino County Offices of Education. Good afternoon, Cynthia Lenners. I'm the Assistant Superintendent for Lake County Office of Education. I'm here as a co-facilitator for last year's EAPLN with Aaron Carter. I would like to thank you for the opportunity to be here today. Okay, so I was talking to Josh when I came in and I said, you know, I think it was a year ago, maybe even this very week that I went to a meeting that they had to explain this thing called PLN. And we sat there and we talked about it and we were kind of like, whoa, that's kind of weird and maybe kind of interesting. Erin and I met and we decided that this was something that we wanted

373to do, but we didn't want to do it just as a single county. We wanted to do it across county offices and we wanted to do it with all of the districts that we could bring together across our county. So in our two counties, we decided that we needed to break down the rural isolation and we decided that we needed to bring districts and counties together. We developed an early adopter network with six districts across our two counties. It was so well received, and they wanted to maintain their relationships as a cohort that when we had the opportunity to apply again this year, we applied for a second PLN. We are now hosting two of them. 50% of the non-charter districts in Mendocino County are members of the PLN, and 100% of the districts in

374Lake County are members of one or the other PLN. You can see that our district size is quite varied. We have as small as 47 students to as large as one district with over 3,000 students. But we have a large, all of our districts have a large number of unduplicated students that we work with. You'll notice the collaboration is key, and this is really what our PLNs was built on. First off, the collaboration between the COEs and the CCEE. Secondly, the collaboration between our two county offices. Finally, the collaboration between our districts, our districts and our county offices. But more than that, the collaboration between districts across counties working together. We have several different types of meetings now for the 17-18 year that we're working on. The first one, once a month, we meet face-to-face

375trying to connect to these isolated districts to state initiatives. In doing that, we wanted to move away from the fact that we are the font of information and instead we wanted to have them have real conversations with people who could bring additional information. Over the course of the last year, we had Jeff Brashears present to us. Barbara Murchison was in the county. Nancy Brownell has been up a couple times. Tina Ochoa from CCEE joined us. And we have a number of other presenters who will be coming up this year. We engage in one problem of practice meeting each month. These problems are identified by the districts and they include people from district members from across both PLNs. We are using the information that we are learning ourselves working with CISC, CSESA, and the Improvement Science

376Initiative to address the things that are coming up in the districts. Finally, we have work group sub-meetings, which really allow us to deepen the PLN work and to make sure that we are constantly thinking about that continuous improvement cycle. We feel like our districts have felt heard, and we feel like our superintendents have felt a sense of unity. Hello, I'm Erin Carter and I'm the now currently I'm the principal at a elementary school in Lakeport I'm unified which is part of Lake County and what's really exciting is one to have the honor to come speak with you, but also to see the work that Cynthia and I were doing in the county office level and how that's impacting the work now that I'm doing at the school level so we can definitely see the correlation

377of the work that our PLNs are doing at the superintendent Superintendent level and how it's helping me do my work with my staff and students at the administrative level What was what we thought was really important is to show you the impact that our districts had Around just district leadership, but also the students that the PLNs had last year just from January through June and so our initial findings of our early adopter PLN gave us the Momentum to scale up to the two PLNs that we're doing now throughout the two counties. It was exciting to hear the superintendents asking that we're not in the early adopter. Initially everyone was like, oh I'm not sure if I want anyone to join this and then we had a few courageous members came up and said we want

378to be a part of this early adopter process but then at the end of it in June everybody wanted to participate because they saw the impact it was having and so we were really excited to be able to scale up this work. There was a greater understanding when we actually, as the county office was reading LCAPs, we saw a greater understanding of the process and the systematic implementation found within our participating districts when we read their LCAPs. We also saw a greater understanding of the metrics that were being used and how they were being applied within the long-range planning process of our districts. The knowledge and content of our guest speakers provided the targeted collaboration that was needed for us to grow in our understanding around this new system and the new template. We started

379viewing the LCAP template as a flexible tool that provided a collaborative roadmap to meet each one of our district's vision. We realized we shared similar challenges and we were able to safely collaborate together, sharing ideas and holding each other accountable on moving forward in our current LCAP work. A lot of our superintendents share many roles in their district and there wasn't a place where they could feel that they could bounce ideas off of and just be a leader that can let their guard down. And so this was a really good place for them to do that. The impact that it had on students is that our districts are more willing to think outside the box when providing programs and reflecting on the current programs to ensure that all means all. We integrated discussions around the

380CTE courses and pathways, our dual enrollment, NGSS hands-on instruction, and instructional best practices as one of the pillars as we met each time. And one of our big intentional focuses was around equity. And equity was our driving focus within each interaction that Cynthia and I... put in our agenda and through this equity lens we were able to reflect and create systems within the district and how we can truly engage parents and reflect on the practices we were currently doing and how can we scale those up so it really has a much deeper understanding for parents and students. So thank you and we really had a great time doing it and looking forward to more. And for our last highlight, we're going to have the Sausalito Marin City School District pilot come up and highlight their

381work. Chairman Kirst, Superintendent Torlakson, members of the board, ladies and gentlemen, thank you for the opportunity to present our partnership today. There is an African proverb that states, if you want to go fast, go alone. If you want to go far, go together. We are embracing this philosophy in Sausalito Marin City School District as we deeply engage in the important work on behalf of our students and as we move toward enduring positive change alongside our partners. With a 90% unduplicated count in our district, this work is more critical than ever. Hi, Tarina Morris, Deputy County Superintendent for the Marin County Office of Education. I'd like to introduce my colleague. Hi. Will McCoy, superintendent of the Sausalito Marin City School District. First of all, I'd like to say on behalf of our county superintendent, Mary Jane

382Burke, that we sincerely appreciate that Sausalito Marin City was chosen as a pilot district. We understand the political complexities that make it very difficult to step in and do this work, and we are emboldened by our partnership with the CCEE and the state. The slide that you see in front of you represents just a snapshot of what has happened over the last 12 months, 12 plus months. And you'll see there's been extensive complaints, reports, findings, investigations. And we are cognizant as a county office that Our role is now on the seventh intervention in over two decades in this district. And those interventions have largely entailed fiscal interventions through AB 1200. And so those of you that are aware of AB 1200, you know that there are some pretty specific and narrow parameters that outline what

383your intervention can be. And through LCFF and LCAP, it's enabled us to step and sort of broaden our intervention. So this intervention looks very different. And we have embraced it. I think you could say we are involved with the district on a daily basis, engaged in partnership with the district superintendent, and that partnership actively involves the CCEE. As a county office, we appreciate... and have deep respect for local context, local control. And we also recognize in a school district such as Sausalito-Marin City that has experienced extreme volatility, especially in administration. Mr. McCoy represents the fifth superintendent in six years. And so because of that, while we celebrate the positive and productive partnership that we currently have with the administration, we also recognize the volatility. And so we know as a county office that we can

384provide that continuity over time, both during the pilot with CCE and after. All of this work is very complex and multidimensional. It requires a team approach as we navigate the many challenges before us. We are deeply appreciative of our many partners most importantly the Marin County Office of Education and the CCEE Without these partnerships. We would otherwise be unable to move this very important work forward The complexity of the diagram shows all of the different levels that we are required to work with on a daily basis and MCOE and CCEE are Vital in helping us navigate those systems so When we partnered when we when we began our relationship with the CCE as a pilot and we really did this in tandem with the district We really looked at how best can we utilize the

385expertise the knowledge and the independence perspective that the CCE brings and To appreciate the complexities of our district. It's a small community serving a small number of students But with that smallness comes a lot of interest. And so we are the recipient of over 40 community-based organizations. And so just the complexities of trying to navigate and align their initiatives to the needs of our students can be daunting for a small school administration. And so the CCEE has provided and stepped into a very important role in the creation, I would say even the co-creation and the design of community advisory committee. that is currently working to design a community school model as outlined within one of the district's LCAP goals. And so the community school model is designed to provide or intended to provide wraparound services

386for our students and our families while engaging and partnering with community-based organizations. So through that, All parties, the district, the board, the community-based organizations, certainly the county office, appreciate and value the independent perspective that CCEE brings to the table. And they are regarded and looked to to provide evidence-based research strategies that we can incorporate and look to when we get down to the nitty-gritty designs of the community school. design. The complexities around our district are too numerous to name and what we know is that when we focus on one thing in a myopic sense and that is what is best for children, which of course is always debated and what we know is we all agree with all the partners around this community advisory committee, we all agree on what we want, but we don't

387necessarily agree on how we're going to get there. And the CCEE again has proven and provided that sort of neutrality that lends to a very healthy facilitation of those discussions. So as we move forward, we are sincerely honored to be part of the state's early multi-tiered system of support. We take this work very seriously. We are... genuinely excited to be a part of this work and we know that through the context or the respect for local context we think that that the students of the Sausalito Marin City actually have a fighting chance for the first time in decades because it is so locally driven this is the right work at the right time definitely with the right team we'd like to thank you for allowing us to speak today As I close, I want to

388cover two quick things and just let you know all our presenters will be here for questions. I want to close with a summary of our approach to gathering feedback and how to access additional support. This is the first time that there's been a collective, a collaborative, and a proactive approach to gathering feedback as improvement is happening. It will consist of a team who has conversations with districts, stakeholders, and support providers every six to eight weeks. The feedback from conversations will be shared among districts, providers, and the cross-agency group. The cross-agency meeting will reflect on the data and adjust course when necessary. We'll work with stakeholders to develop what these questions will look like. In early conversations, a group of teachers I was speaking with suggested, make sure whenever you ask questions about the district, explicitly

389ask about the stakeholders. How did data analysis... What did you learn from the data analysis as a district with your stakeholders? Just to explicitly pull that out. And so we know we need to work on developing those types of questions with stakeholders to get to the nuances that will get information that will actually help us know how we need to adjust course in the moment. Not two or three years later from a third party, which also could be great, but people who are owning the work and doing the work now asking how it's going regularly. The last piece is... We know our approach to assistance and support will come with challenges, which is why we wanted to build out a way for county offices of education and districts to access additional support and problem solving

390if the collaboration work isn't supporting the improvement of students and living up to the shifts that I've discussed earlier and we've discussed in previous board meetings. For county offices of education, they can consult and work within their regional teams and then SESSA and the cross-agency group to discuss concerns and develop solutions. For local educational agencies, they can directly contact representatives from the cross-agency group to discuss concerns and develop solutions. I want to thank you for your time. I want to thank my presenters so much. I think in 35 minutes, you got a little bit from me and you got five highlights of work that's already going on. So I just appreciate all their time and coming here and putting together their presentation as next steps will continue to gather feedback and of course, continue to

391coordinate and align the work of the different agencies. And so now any feedback from you or questions for myself or any of the presenters we had here today. Yeah, before that, I think we need to take a break. So the board will... You can rest up a little. So the board will take a 10-minute break until 5 after 3. State board will reconvene, and it's now time for member questions on item 4. So board member questions are welcome. Board member Sandoval. Hi. It's not so much a question, but just a comment. Thank you for a very well-prepared item. I really appreciate the examples. And it helped paint a picture of what support could look like and the fact that this is something that's available to all students, all districts, all schools. And so I think

392it really represents a positive shift with a focus on support for each student and then alignment across agencies, departments, offices to better serve all of our students. And I know that this has been something that just being a part of the Being the liaison with the Advisory Commission on special education this has been a long long time coming and so it's really exciting and The fact that we are seeing all of these different agencies supporting all of our students thinking collectively about all of our students as a whole group is phenomenal. And just the fact that we have special education represented in part of this broader conversation at all points is really essential. So I'm very happy to see that happening and believe that when we can help ensure that students are all being cared

393for and looked after, they're in a much better position to realize their potential, regardless of disability. So thank you. Any other board member comments on this item? It's an information item. Board Member Burr? Well, I just wanted to express my appreciation to Melanie for organizing this and for all of our presenters. I know sitting on the CCEE, I get an opportunity to see some of this more as the Superintendent Torlakson kind of up close and personal. So it's always great to hear the local stories. I really appreciate it, especially hearing about Lake and Mendocino County. I think that is a really interesting collaboration and a very good example of what a professional learning. network looks like. Because I think they even raised this, people were going, hmm, do we really want to get involved in

394this? What is this going to turn out to be? And it turns out to be a very rich learning experience. And I think also an exemplar of what we hope moving ahead, peer assistance will look like. like districts helping each other with their student challenges. So anyway, I really appreciate that. The one thing I just wanted to call people's attention to, because I know there's been a lot of conversation about this, and it's on Attachment 1, page 4. And we talked about this in item three quite a lot, so I won't repeat this. But this is the criteria to determine eligibility for differentiated assistance. And it talks about in there that county offices shall provide differentiated assistance to a school district under these various conditions. And there's been a lot of kind of chatter around

395that I've heard about, well, what if we don't want to work with our county offices? And I just want to make it clear that county offices are an integral linchpin in this whole system, as they have been for years and years and years. Tarina mentioned it around AB 1200, which is the fiscal accountability system. I get that people might have tense interpersonal relationships at a local level, but it is a linchpin of local support. You have to have those kind of relationships between the county offices and school districts. So I just want to make it clear that as a state, we've set the obligation that the first piece of work needs to be between the county office and the school district. If it doesn't work for whatever reason or the county says, I need additional

396help. I'm going to go to one of my regional partners or I'm going to call in the CCEE. I'm going to ask the Department of Ed to help. That's fine. That's what's happening with this cross-agency work. But I just, I didn't, I wanted to hit that head on because I think there was a lot of kind of undercurrent of, well, we're just going to tell our county to go take a hike. And I would just suggest, A, that's not appropriate for relationship building, but it's also not the way the state envisions a structure that can be accountable for improvement in student learning. So just wanted to call your attention to that. And we'll say finally that. I appreciated the presentation about what counties have done to build their capacity, both sort of generally around LCAP

397and LCFF support, but also more specifically about the multi-tiered system of support. So, again, thank you so much for being here. Board Member Ortiz-Lacone. Also want to thank you for this item and all the presenters who waited six hours to present to us. Thank you And and thank you for the work that you're already doing. I also Wanted to touch on something that board member Burr brought up with the obligation the legal obligation of the county office because when I was reading this it says that the legal obligation is to provide technical assistance by giving an assessment but that is the extent of that TA legal obligation. So I'm reading page four. To provide technical assistance will have been satisfied by simply making the needs assessment available to the district. But then there's a handoff,

398right? It goes somewhere else. And I think that was part of my confusion. I thought, okay, so let's say that the district and the county don't. Want for whatever reason which you know again. I think that we should be collaborative But they don't want to work with each other their legal obligation is to give an assessment To the district of the areas in need of improvement and then hand it off to someone else And my question was well then who does it go to is there gonna be a clear pathway for that? So they know who is now coordinating these services. You know I also read in the rubric that The LEA doesn't have to worry about who's coordinating the services, but it doesn't specify who then will be coordinating the services. The other question

399that I have, and only because we've discussed so many issues over the last year and a half, two years, and I'm not even really sure where we stand with... with some of the TA models. So I just want to clarify that under LCFF, LEAs are provided TA if any student subgroup meets the criteria for two or more LCFF priorities, that's state and local. Under ESSA, it is the schools that will be identified if they are in the bottom 5%. And that support is only geared towards state indicators. So I think the accountability division will come back with more proposals about the 5% in January. But from the proposals that have been put to date, they were based on the state indicators and the colors, also because those are the only ones at the school level.

400But I do know as the board discussion has unfolded, you all really want to make sure it's not, and we also do too, that it's not about now just supporting schools outside of districts. Right and I think that I'm just wondering when we're putting these tier systems of support if we're being mindful that at some point we'll also have the school data and the bottom 5% and trying to reconcile those two things because one will support the LEA state and local indicators and one will be geared towards state indicators alone at the school site level. And hopefully we'll be able to continue with this update item and as the five per as you all have more discussions on the five percent we can then talk about so how does support fit into what we've created

401under lcff so that it's one system and not two different pieces so i think that's a good point and hopefully this item can then tackle those discussions and get your feedback um you know in the spring and as as that makes sense thank you Board Member Strauss. Melanie, I just want to thank you and the entire team that presented. And I think we're continuing to try to build out a full system. But as I go out to districts, and I'm sure all of us in our interactions, people are like, okay, so when is this going to happen and what's it going to look like? And so the other question is there is a lot of discussion, as we've heard from our board members, on what's the role of the county. And it's my understanding that

402the county is the one that approves the LCAP. understands if a district is choosing to go for other support, the county is the point person that knows that's where they're getting their support. And I believe that the county officers are trying to also norm and coordinate their training among all of their teams since we have such a big state. So I appreciated looking, and I've appreciated the last meeting and this one, looking at what CCEE is doing with... the counties, especially different size counties like from Kern, which I thought last meeting was really effective to look at, and then the small districts, and then the uniqueness of the challenges of the context. Those of us who have been out in schools and school districts know that context makes a difference for everything, and change of

403leadership and instability and board changes and budget cuts and all those things. So again, I just think this is beginning to add some sort of more specifics for people to understand what... this whole system is going to look like because the dashboard will get publicly released and then we're going to identify the schools and then there's going to be some technical assistance. And I think as we all talk to different constituency groups, that's what they want to know. What's it going to look like? And I think the other issue that I'm understanding is if a county office identifies a couple of goals that they think a district should work on, but the district determines that for their context, it's different. The district does set the priority, just the county. monitors and focuses, they're managing sort

404of where is everybody in their process. Is that correct? And one, I think one, not analogy, but one example that I've heard folks use is that if a district is deeply involved in improvement efforts, I think the example in the item is CCEE pilot, deeply involved in improvement work with the special education division at CDE. Do they need another entity to help them organize their improvement work? They might not. But the county is still ultimately responsible for thinking about and supporting that district in their improvement. Journey for lack of a better word and so they need to be a part of those discussions and build if the relationships out there building that relationship So that they can say this is what's going on in this district I'm really excited about their pilot work and the

405work they're doing with CDE and This is how I'm also supporting that process and it might not be a huge lift It might be kind of helping them come back to center and reflect on all the work they're doing But that the counties are involved in that process, but that they don't need to all of a sudden make something go right as it's been going left with significant improvement efforts. Well, the other thing just that I appreciated, and then I'll end my comments and pass to another board member, is that I appreciated the special ed division and the discussion about MTSS and the work from Orange County. Because I'm seeing as I'm out in counties that different counties, like say Riverside County is collaborating with Santa Cruz County, and different counties are collaborating. So we

406saw a couple working together on a network. And I think that's really a strong resource as well as we help to build capacity from a larger district to a smaller district. And the MTSS work is really... impacting core instruction for all kids and the support for special ed students and for English learners as well and identification of issues for our subgroups that we're very concerned about so I appreciated the choice of the models but also the topics that were focused on as well so thank you Karen is next Okay. Well, first of all, I want to thank all of the participants who came or the presenters because you really breathe life into what we're talking about in terms of system of support. And these are going to be variations on a theme, and they're only

407going to get better over time. So I really appreciate that. My background, I spent years at the county office and working with all the districts in Riverside County. And so there's 23 of them, and really the work was about relationships. And so I know that that's something that's going to be really critical. It's December the 4th or whatever, and the dashboard's out, and we're a district. And I might not be a district that's all red, but I do want to have some support. So is it going to be the assumption we go straight to the county, or what's the process? Can we reach out to others, or are we expected first to go to the county? So specifically, if you're a district that has... student group that's not meeting the priority or not meeting the

408standard for two or more priorities then your county office is going to call you and that's going to be before the public release of the dashboard they have a very short time frame but As cross-agency group, we've been talking about how we need to make sure that districts have that personal phone call on that touch point before the dashboard goes public so that the county superintendent can talk with their district superintendents about what stakeholders they want to convene, how they want to begin conversations around their dashboard data, and also the other local data that they have that complements what will be on the dashboard. So for districts identified, they'll have that personal call with the county superintendent before the public. release of the dashboard. Now for districts that aren't eligible for differentiated assistance under the

409specific criteria, they are more than welcome. They can call their county office for support. They can also directly call. I know many districts have connections with specific staff at the California Department of Education. They can reach out for support. So there's no limitation or kind of ways to stay in your lane when asking for support because you see a need that you need support on in your dashboard. One last thing. I am part of a PLN right now, so I really understand the dynamics of it. I think it's wonderful, and I think it's perfectly timely right now for special ed. And so I really appreciate all of the work that's being done, but particularly looking at that student group. So thank you. Patricia Rockert. Well, thank you very much for doing this particular presentation this

410month. One of the concerns I've always had when we hear from county offices is getting an understanding about not only what it is they're doing, but hearing and understanding about what they believe their capacity is and making decisions of not only about going into and creating a relationship, but what their exit strategies are. And I appreciate also that you gave an explanation about... What it is districts should be using this pre-public preview period for, what kind of activities should be going on right now. I think it's really important to hear those things. It's not just they're sitting in a room and staring at the dashboard. There are some very critical conversations happening. I still am asking for some review, some evaluation, or in some way, some description of what their understanding is about their... In

411Riverside County, there's 23 districts. you know, what are they going to be doing or how are they going to differentiate the decisions about not only the level of collaboration they'll be doing, but at some point they have to start evaluating the outcomes of the work they're doing. What is their exit strategy? What is their strategy for balancing or countering with maximum capacity or maximum types of ways that they could do work with all of those counties, knowing that it could look very different. It could be 23 different types. of intervention activities going on. I'm still looking for and hoping to hear that and I realize Part of the reason why they can't really say a lot about that now is because things are just getting started. But even as things are just getting started, there

412still is a conversation or at least some thought that could be given to capacity. I'm seeing several of the staff from the collaborative who are here, and I've heard them give presentations and talk about not just capacity, but what it really means to differentiate what their conversations are. and what collaboration could really look like from the perspective of the collaboration. So I'm very confident he individually has that expertise and is actually thinking those things through. But I'm not really sure. I actually see real evidence that those are the conversations county offices are having as they're doing their planning and initiating those collaborative relationships with districts. I think that's a great point, and I know the success has begun to think about how they can evaluate the capacity and make sure that counties are networked with

413counties that will help build their capacity. And so in our next board item, we could be able to bring a summary of that work and what they're learning to the board. And then the other, in addition to evaluating their capacity, I'd like them to think about how are they evaluating. not the success of their interaction, but evaluating what they're doing. How do they know what they're doing is actually working? What is replicable from the work they're doing in one district that could have value someplace else? I don't want us to get to a moment where we have this one-size-fits-all type of activity going on in counties that have multiple districts and multiple intervention programs going on without really thinking through. what is of value to replicate, and is there somebody else who can do this

414differently or do this in a better way than what we've been doing? So while it's not, there isn't a requirement for them to report back to us their own self-evaluation or self-reflection. I'd like to hear about what they're doing to evaluate not only their capacity, but evaluate the value of what it is they're doing in their intervention. Who's next? Board Member Burr? So I want to build on what Member Rucker was talking about and make a suggestion going forward. And I think this also gets to Board Member Ortiz-Lincón. I want to get to the bottom line. It's the so what question. And we've heard what's the county's exit strategy. But to me, more importantly, is what's the district going to do if they've got two or more subgroups who are not meeting the priorities over

415what period of time? And I think we're shying away a little bit from that because I get, and I'm a big proponent of this, we wanted to move away from the shame and blame and we're going to take them over and we got a recipe and all that to this other end where it's the district's going to drive everything. They're going to tell us what their priorities are. which is fine, but the bottom line is they must be improving with their student groups across the priorities. And so I think what would be helpful, at least for me, being also a visual person, is what does that look like? What does that cycle of inquiry look like? Let's say we've identified a district that is struggling with their students with disabilities and is struggling with their

416English learners on the academic indicator. Pick whatever indicator you want. What happens at the end of the year when they've said in their LCAP, we're going to improve that, it's going to get better, and guess what? It doesn't. Then what happens? And that's what we try to get at this in a very process-oriented conversation about levels of support and differentiated assistance and who might help and who might not. But the bottom line is, what happens when the students still are struggling? what's going to happen and that that's what accountability is right it's accountability at the lea level it is ultimately accountability at the county level i agree with you member rucker that the county has got to be able to say are we are we helping them are we helping their students not are we

417having great conversations as adults but are we helping their students so i think Trying to be a little bit more specific about that and what those kind of those pathways are I think to Karen's conversation is hey I'm happy to work with my county to start with but what if we're both still scratching our heads at the end of a year or two and We're not seeing the kind of results that we want then what happens Yeah, thank you for reframing my comment because I'm trying to soft pedal that concern because I know we're still very early in the process, but that is my ultimate concern is on the other end of all this, did it work and did it make a difference? Board Member Holliday. Thank you for all the examples and I heard

418throughout many of the presentations references to charter schools But there wasn't really a clear example given of this process taking place with a particular charter school in le a and in that context the dynamics can be very different between County office and a particular charter school if the county office is an authorized Authorizer for that charter school or if the county office has denied renewal for a charter school and and School came up to the State Board and won on appeal that sort of situation the dynamic becomes quite different there so perhaps Well, first of all question is there a conversation going on about how? this process may be affected by the relationship between the charter school and county office of ed or the local district or whomever. So for this year on the fall

4192017 dashboard charters will not be identified for differentiated assistance But in the following year they could be identified for assistance from their authorizer And I think what you bring up of well for county off County of Office of Education is your authorizer. Who is your authorizer? What's your relationship? I think is an important issue that we should talk about but I also want to make sure it's clear that charter schools can access any of the different supports that we highlighted today. When I was looking at the list of the school or the local educational agencies in cohort one for the multi-tiered system of support work, several of them are charter schools, county offices of education, traditional districts. So I want to make sure as we're building our system of support that it's not exclusive because

420when you look at the statute for differentiated assistance for fall 2017, it focuses on traditional school districts. So I think we definitely can come back and talk more about how will this look for charter schools when they are identified and also make sure that we're being inclusive that the supports we're building especially the level one for all really are for all and I know a lot of county offices provide a lot of supports for their charter schools in their boundaries and that's just a part of how they do business so I think that's a good point okay any further board comments on this that's very helpful to the staff and we're ready for public comment Would the following individuals please line up to speak to item four? Martha Alvarez, Barbara Short, Julie White, Efren Marcado,

421Martha Diaz, and Andrea Ball. Good afternoon. Martha Alvarez with AXA. We stand ready to support the State Board and CDE when we roll out the system of support for school districts. We wanted to just comment on the communication tools. We look forward to being a support in that effort to ensure that we have the appropriate messaging to school districts. Particularly, we need two different communication tools, one for the administrators or the recipient of the system of support. And secondly, we also need differentiated messaging points for the communities who will be the ones trying to understand the system of support. In regards to... the state's approach for the system of support, we're really encouraged by the proposed proactive approach of doing the special ed, PLN, collective, and the test kitchen. We encourage CDE and CCE and

422other involved entities to also consider doing something similarly for EL students. As we know, we also have a need for that particular student subgroup. We see MTSS as a vehicle. for serving all students. And so we are encouraged by this work and what the state plans to do in the near term. Lastly, we see the networks as a support, as a way to really engage school districts. My last comment is because we see the networks as allowing districts to develop their own agency and internal responsibility and accountability. Thank you. Hello, Barbara Short. Thank you for allowing me to speak again. I did drive seven hours, and I'm trying to maximize my participation. Earlier today, you examined criteria that places a district in need of support, such as cut scores. I ask you to consider, is

423all data appropriately weighted for small districts? Is the context of populations considered? Will academic indicators be of primary import? Or as strategies are applied to different improvement areas, is it understood that we can't fix everything at once? And how will improvement be weighted against areas that aren't improving? Any school-based support should have a district component built into that system to build capacity. Again, I Thank you for considering local control and ask that local control choice and access is retained. Allow and promote organic systems of support such as the PLN or COE collaboration. The MTSS presentation earlier shows you the gold standard. This is really, truly where we all need to get to. And it requires a sea change on the ground in terms of attitudes, the way curriculum is structured, the way our school day

424is structured, everything. It's going to take some time. These test kitchens, these KDSs are very important. But looking at this as a long-term venture when we are trying to improve. everything all at once and many of our districts will be in that position. And then I would like to add that this needs to be low cost, no cost or added funding to support this happening. Just for example, teacher collaboration, that requires an extended day for teachers and doing MTSS or initiating many of these other changes on the ground in the classrooms is going to require extra time and effort on everybody's part. So I thank you so much for all the great work that you're doing here, and I'm amazed by the presenters. I am quite heartened having been here. This is my first time.

425I have a lot of positive messages to take back to the hinterlands. Yes. Thank you so much. Thank you. Good afternoon. I'm Julie White, and I'm here representing the core districts, and I drove eight miles. I want to call your attention to a report that is being released by PACE today that describes the core district's network and our work on continuous improvement. Patricia talked about seeking examples from the field about continuous improvement and the work that's being done to improve outcomes for kids. Our partnership with PACE is helping our districts better understand what it takes to be continuously improving and a continuously improving network. This Facts at a Glance document summarizes our continuous improvement work, including examples from our districts, providing recommendations for educators, and sharing major findings from PACE. In the report, PACE highlights

426the following four essential elements. Cross-functional school improvement teams. comprehensive systems analysis, a variety of data, and expert facilitation and learn by doing activities. To inform the systems of support in the state, the core districts will continue to share our learning with local educators, policymakers, and stakeholders. Good afternoon. Martha Zaragoza-Diaz representing CABE and Californians Together. We have three recommendations regarding this item. Given the fact that the State Board of Education approved the EL Roadmap, we recommend that inclusion of the appropriate provisions of the EL Roadmap be included in this new system of support for our troubled LEAs and schools. And we are very willing to work with CD staff and State Board staff. on this recommendation. Our second recommendation is that individuals providing technical assistance and support should be required to demonstrate experience and expertise in

427working with and getting improved results with the specific student groups with whom they will be working with. The third is the development of clear procedures and protocols by which LEAs and schools will be supported and working through their change process. We believe that inclusion of these or considerations of these recommendations will establish a more enhanced and appropriate system of support for ELs in the LEAs and schools that require support. Thank you. Good afternoon. Thank you. I'm Andrea Ball. I'm here today on behalf of two distinct groups that we work with. One is the Central Valley Education Coalition. It's a coalition of six counties in the Central Valley, their superintendents, the districts within them, and Pivot Learning Partners, a nonprofit technical assistance provider that works with districts around the state. To thank you, to thank the

428department and the CCEE and the OCDE for these presentations and the approach. And in this item two things that all of these groups have been calling out for help working with students with disabilities. So the collective and the approach outlined in this document, I think will be very helpful. And particularly calling out, identifying some bright spots of LEAs and then sharing that information with other LEAs because, and again, what's reflected in this proposal is really the districts, as my colleague earlier said, need to have some agency and some ability to understand and incorporate this work into their ongoing practices. So I want to thank you for this and urge you to continue this work. Thanks. Good afternoon. My name is Efren Mercado from Children Now. President Christ, I'd like to reference the other letter that

429we sent to the State Board regarding this item, how when it came out, we really appreciate a lot of the details that were laid out and really see the system moving forward really well. I want to highlight a couple key points. This idea that the statewide system of support is really about the level one supports we provide for every LEA to help each child. Level two will get the attention, but the level two is going to be the... the later on supports we also really appreciate the highlights that were given by christine holmsted regarding the county office work to be prepared for this initiative and look forward to seeing how that's going to play out and also very heartened by the idea the counties and the state agencies cce and cde working together on this.

430Moving forward, we think it's really important for the 2017-18 work to be evaluated to see where that work ends up and how we can build on it and also to discuss the funding for county offices education as they move forward with the work in the 2018-19 and in the future. Thank you. Good afternoon. Liz Guillen with Public Advocates. We agree with the calling out in the system of support that there be an assessment of root causes. But we want to state again that we think there should also be an assessment of the capacity for a major change process. We think that that lack of capacity or willingness to change may have contributed to prior efforts. that the state has made for school improvement. I also want to associate ourselves with the comments by Californians Together

431and CAVE. Thank you. Okay, that's the end of the public comment, and I think concludes the item since this is an information item, and now we're ready for item five. Well, that's not it. We'll get a slideshow here in a second. Hi, I'm Jeff Brashears. I'm the Director of Local Agency Assistance Support Office, and I'll be right with you. I'm technically challenged now. All right. Thank you very much. That's me. So item five is we're going to switch gears a little bit and this is an update for issues related to California's implementation of the Elementary and Secondary Education Act as amended by ESSA and also other federal programs. It's an information item. We're asking for you not to take any action, though we want some feedback on the LCAP federal addendum, which we'll talk about

432in a second. A couple other issues that are addressed in the item is talking about the LEA and state report card per pupil expenditure requirements. There's an update there about the regulations that have been withdrawn and some guidance that should be coming later this fall, as well as foretelling or foreshadowing that we're coming back, the special ed will be coming back in January and March to meet the federal requirements of the IDEA and the reporting requirements there. But what we would like to get some feedback from you on is the LEA plan. Every Student Succeeds Act requires the local education agencies to complete an LEA plan that addresses required provisions of ESSA. under which the LEA applies for federal education funds. LEAs that apply for these funds are required to complete an LCAP federal addendum

433or addendum along with the consolidated application. Those in conjunction with the LCAP, the addendum, and the consolidated application will serve as the ESSA LEA plan. Our endeavor is to meet the intent of California Education Code 52064, which allows the addendum and the LEA plan to work with the LCAP and allow LEAs to fill out a single plan to meet both requirements. So the CDE, in conjunction with the SBE staff and with the support of the California Comprehensive Center at WestEd, met with LEA representatives to design a new approach to the federal planning requirements and look at the LCAP process as well. Within that work, the team cross-referenced ESSA provisions. identified those provisions in LCFF priorities that potentially meet the ESSA provisions and to intertwine those. Those provisions are identified in the addendum template that is

434provided to you as an attachment and that was also in the October memorandum that we published for you. We received feedback from the California Practitioners Advisory Group originally. We'll go back to them in December to receive more feedback. We have a couple approaches that we would like to discuss with you today and get your thoughts and feedback as we come forward, again, probably in January for approval of the addendum to be utilized in the 18-19 school year. The current draft that you have was field tested by LEAs and especially with Title III representatives to look at just the functionality of the template or the addendum, pardon me, and receive some feedback with that. And so the current, this is, so the current. addendum in the current version looks at and approaches the ability for an

435LEA to address the ESSA provisions in one of three ways. And as you can see here, one way would be then to address within their LCAP those provisions and identify within the LCAP the goals and actions. It says page number. We have found out that the pagination as an LCAP is evolving is problematic. So that will change regardless, but it's still there in draft form. So that's one way to address the provisions. The second way would be then to address those provisions or show within another document that the LEA has how those provisions are addressed. And then to identify the document there as it's shown. also on the on the front page the cover page of the current draft also has the ability to link to these documents that are referenced with within the this

436addendum and then the third way is just right here within the document and to say these are this is how we approach and then we plan to approach these provisions specifically within on this document and not necessarily link it to the LCAP or within another document And coming back and also there you see at the bottom a not applicable or an A and that would only be for those provisions or LEAs that did not apply for a specific title program within ESSA. So another approach, and this is based on feedback from the October memo and talking to stakeholders and also feedback from those that were field testing the current draft of the addendum, and it's to recognize and to seek to leverage the LCAP process and really recognizing that an approvable LCAP will address many

437of the asset provisions. And so what we're looking at then is to, and we'll show you the provisions, but what we're arguing with this alternative or another approach is to argue that our LCFF priorities are big enough and grand enough, if you will, but they will encompass those ESSA provisions and don't need to be addressed again because they will be captured in an approvable LCAP. So if these are potential alignment working with legal and other offices within the building, the potential is then to reduce that 35-page addendum down by, I think it's 13. I counted and then I forgot to count, but I believe it's 13 that could reduce that. And even that could be further refined as we look at the Title I-D provisions, which has 12 provisions. I think some of those could

438be captured within the LCAP as well. So there within it would be these are provisions as an example of a provision that would be addressed within an LCAP and therefore again would not be further identified within the LCAP or listed out but just knowing that our priorities capture these provisions if you will. Further though, then there are ESSA provisions that are really not potentially aligned with LCAP and LCFF priorities and they're listed there. We have worked and I met with the SUCCESSAS Student Program Services Steering Committee last week and we actually looked at the Title I-D provisions and I think that those can be can be reduced and minimized as well. And so with that, here's again the same looking of the template, but in talking about it and looking at if these provisions are

439within the LCAP, we would recommend on this alternative or additional approach is to only have them address it within this document itself because we've already identified those provisions that would be covered with. with the LCFF priorities and for consistency of review throughout the state is to not link to another document but just limit the response to these provisions that are not captured by the LCAP or LCFF priorities in this document with a limitation of words but just to provide them there. That was fast. I could tell I was going fast. So that's what we have in the feedback on those two approaches. Again, we anticipate coming back with further feedback with CPAC, the LCFF stakeholder group, to further refine this, but we'd like your thoughts and feedback on these two approaches. Yeah, my first reaction

440is, how are you going to access the two? I mean, if some is in the LCAP, and I'm looking for the total picture. For federal aid. How easy will it be for me to access the LCAP if it's not in here? So the access would be those the LCAP is obviously approved either and school district LCAP is approved by the County Office of Education. What we're saying is that you would not necessarily access that to look at the provisions because we're saying that the LCFF priorities capture those provisions. And so by the mere fact of having an approvable, well-written LCAP, those provisions are met. The others then would come to the CDE as a reader's conference or somehow we're working through the details and work with the county offices to look at how we would

441then look at the other provisions that are not. addressed by the LCFF provisions. So those others then would come to the CDE to work on a way to have a consistent, approvable metric, if you will, for these other provisions that are not addressed within the LCAP. Yeah, I understand the other provisions. It's the first bunch of provisions. So the assumption is since the county approved the LCAP, there's a whole bunch of provisions for ESSA that are satisfied in the LCAP but the LCAP we don't get it at the state level and we don't want it so and there it you know researchers have found it difficult to access LCAPs and compare them so I'm a little lost on how one would see the whole picture of it. Right so I think again in is example

442that we used here was the stakeholder engagement provision. By going through the process of developing an LCAP plan and going through the process and writing to priority three, which is stakeholder engagement and parent involvement in programs and students, English learners and students with disabilities, that we believe that priority in the LCAP itself meets that provision. Not necessarily could be throughout the LCAP. It is not a specific goal, not necessarily specific action, but by default, if you will, that would be met. And then we would have, as we bring forth, because the state board approves the LEA plans, we would say that the date of the LCAP was approved along with these other provisions that have been written. Okay. Board member Rucker is next. All right, so Dr. Kirst, I share some of your confusion. I

443understand the goal is to simplify the development of the LEA plan as required under ESSA. I also understand that because there are overlapping requirements in the provisions in the ESSA statute and the provisions of the LCAP statute that there is a belief there is a way to consolidate the planning activities that the LEAs are doing. So I was following you there. So here's where I kind of got lost. I thought I heard you say... that when the county offices do their review of the LCAP In some way, when they approve those LCAPs, they're approving also the ESSA LEA plan. Is that what you meant to say? No. Okay. So then, I'm glad you said no, because then the next thing I heard was, in some way, then the LCAPs are going to end up coming

444to the department for the department to review for approval of the... ESSA part of the plan. That's where I'm lost. And I said that wrong on purpose because that's where I got lost. Okay. So with the alternative approach or the second approach where we're saying that the LCFF provisions meet... The LCFF priorities, pardon me, meet the asset provision. Got it. Right. So those, when that LCAP is approved, we're saying those 13 provisions have been met. They're not approving the LCAP. They're approving the, they're not approving the LEA plan. They're approving the LCAP. Who's approving the LEA plan? Ultimately, then, the you folks are going to the state board will approve the LEA plan, and it will come as a package, if you will, one saying that the LCAP has been approved by the county office

445on such and such date. So, therefore, those provisions have been met. Then the other provisions in this version would come to us. as in bulk for all LEAs, I mean all LEAs, all county offices, all charter schools would then come to the CDE. We would review these provisions that are not captured with the LCAP and then we would come to you as a package and saying through the CONAP through the LCAP approval process from the county office and through us reviewing these other provisions on this these separate sheets We're coming to you with a package then that the LEA plan is complete and approvable for you for each LEA. Okay. And then that takes me to the last question I have because you said this about four or five times that a well-written LCAP can

446actually meet the provisions in statute for the LCFF and in statute for ESSA. I'm going to start with the well-written LCAP. You know as well as I do. And I'm not going to finish that sentence. So what are you going to do with those? One, I'm going to get in trouble for saying well-written because I struck that from my talking points. But an LCAP, the LCFF, we're not, we're again arguing or proposing and looking at that the LCFF priorities envelop the asset provisions. And so if the LCAP is approvable, those provisions will be met. Okay. It would be better to say it that way. Yes. Thank you. Other comments on this I'm torn between two things One we want to integrate it with the L cap. We don't want to add even more pages to

447the L cap on the other hand my impression is that The last time we approved title one applications is between 203 and 206 2006 That's a long time ago under NCLB So my other objective is to force these various federal aid coordinators to rethink what they're doing with Title I. A lot of that is, in my view, sort of sunk costs that could be operating on sort of autopilot. So I'm not sure about that, but that's what I've heard in some cases. I don't know that this will force them to rethink what they've been doing since NCLB in 2003. It will now be 2018 when you come to us. So that's an awful long time to not have Title I perhaps rethought at the local level. Plus, we have a new law. So what's your

448thought on that? And, you know, this strikes like something will be easy to do, but it'll leave whatever we've been doing in place and not force any rethinking. On the other hand, I don't want to mound a paperwork on top of the LCAP. So you've got a hard problem, as in my view, a really hard problem. So I appreciate the question, and I failed to mention that on both drafts and both approaches, there's an overview and a narrative sheet, something similar to the LCAP. Story that the plan summary and it's on page three of attachment one is is to which I'm referring and it is it's it's a if you will it's a cue for leas to Rethink their approach to essa it says the implementation of s in California presents an opportunity to rethink

449the strategy and then it has two prompts for the leas to To respond one is the strategy explain the lea strategy for using federal funds to supplement and enhance the local priorities initiative used with federal and state funds and then the other is describe the efforts that the LEA will take to align the use of federal funds with activities funded by state and local funds and is appropriate across other federal programs so that prompt there and with the instructions of the language prior to the prompt is to encourage that rethinking dr. curse and not just do that which has been done for the last years under NCLB. It is a new start and it is the ability then to integrate it within the LCAP in such a way that is thoughtful, recognizing that the LCAP

450really is the vehicle for change within the LEA. And who would review those prompts? Those would come to the CDE. Okay. Well, another way of thinking about this is that, you know, you can... work through the application to shake up federal funds or you can look at the dashboard and then when you're not seeing results on that then you can work with federal aid through your intervention strategy, and you really have a more active role in challenging that spending if the dashboard shows not good outcomes. So I'm just trying to think this through, but those seem to be options. Most states have not been very aggressive. in following up on those since the end of NCI, when they got the waivers for NCLP and all that. So it's still a puzzle to me. And I

451would agree with that sir and I would say that the LEA plan is a narrative of how they're going to move forward with that so that doesn't change the implementation of the programs, it doesn't change the actual program requirements that they will do on an ongoing basis. You know it certainly would still require the instructions and the guidance from the from the title offices if you will to to make sure that the implementation of these programs move forward in accordance with and provide that support for the LEAs to to work forward to meeting the needs of the students. I'm going to, since Ting Sun and I are the liaisons for the federal, let me ask you first, and then I know there's some questions over here. What your thoughts are. We're the federal aid liaisons

452for the state board. So I appreciate. What the staff has done and the main idea, vision of what we would like this process to be integrating the federal and the state accountability systems and being able to display that. I think a lot of the concerns and some of the questions have already been expressed. I think my add to some of this is maybe figuring out a different way of organizing this, the grid, the... the form. It's very linear. You go through and we end up with another 50 pages on top of the 100 page LCAP that just LEA might have. So if there's some way of reformatting this and maybe talking to some designers or something to help so that you could see more of the information together rather than having to scroll through. I

453think some of that might be helpful and might help clarify some of the questions. Okay. Yeah, I well I agree with board member son I appreciate the attempt to try to Streamline and to and to give assistance to the LEA is for not having to go through this But I agree the structure is a little bit difficult I think to your point, Dr. Kirst, I see this as a great opportunity for them to rethink how they're using the money. And just to revisit my little broken record, this is a little teeny bit of money on top of what they get from the state. So if in the technical assistance realm, whether it's the counties that are working with them on the LCAPs or the department on the CONAP, to get that frame changed, right? First

454and foremost, you're doing what you say in the LCAP. And I think to Board Member Sun's point, in the LCAP and you've identified that you need, again, help with your students with disabilities, this is an objective. Well, what are you doing with your state money first? What are you doing with your federal money? And then whatever little slice of Title I or any of these other titles that you get, how do you integrate that in what you're trying to do? To get away from the categorical notion of, oh, this was this Title I money that I could only do ABC with. Yes. I agree. So how the form would do that, it doesn't, that latter sentence, I'm not sure how the current format you're using would do that. But it's a work in progress. Who's

455next? So a couple things. I think that this addendum would put the LCAP at the forefront. And this is a paradigm shift in terms of how we're looking at spending the dollars. I agree with you, Sue. And that they're using, I have heard the term, braiding of funds, still adhering to supplementing and supplanting, knowing all of those rules, but really being very deliberate, more intentional, and less siloed in how they're approaching using the dollars. So two questions. What kind of feedback have you gotten from the districts who have piloted this or from the counties who helped get feedback? That's one and then two does this change the FPM process at all or the way it's done? so the first the feedback is the Version that was in the in the addendum I mean it's the

456attachment here in the October is is We ask questions as far as the work is it not working and it feels pretty clunky to them And part of that question, part of that feedback is some of these should be met within the LCFF priorities. It seems redundant to address some of the provisions that would be within the LCAP. So that's part of the additional, you know, this second approach. The the FPM process would be is is being discussed now and that would be something that certainly That's one of the what-ifs and yes Yes, but kind of questions that are coming as we've been discussing is what does this look like for the FPM process? And that's an ongoing conversation and I don't have a definitive answers, you know from that at that point Yes, sorry.

457I've just kind of lost tracks getting late I was just gonna just when board member Sue Burr was talking about the form and what we're submitting it I the template I use is not the online template the template that we use at our school is is a is a private organization that kind of helps us put it together. And it does actually have a scroll down of where the funds come from, including federal funds. So it would be lovely to just have it in the LCAP, actually, because then I could say these are the different funds that are going towards us. Specific goals, so I don't know if we at some point if it's just too much to try to mesh it together But really that is the ideal what we're trying to get to

458is here's the goal and these are the different pockets of money that are going to be contributing to that So if I made couple things is the le a plan even the one that was back in three or four Well, it's a narrative. It's not necessarily the use of the you don't you do not identify the funds within the le a plan That's part of the L cap and moving forward Part of the reorganization on July 1 within CDE was the creation of the integrated planning support team, of which I have the privilege of captaining. And that team is looking at then how do we integrate these title funds within the LCAP in such a way that we can braid the funds, we can do it in such a way that meets the supplement, not

459supplant issues, and does further integrate. This is the plan. This is the application for the federal funds, but moving forward, the implementation is that's ongoing work now, looking how can we do that in such a way that that layers on top of the state and local funds. Okay. I think where I'm getting a little confused is where we are reporting. So we have the LCAP template, and I understand that we want that to be the driver. And we're reporting what we're doing, what our priorities are. And then with this little slither of money, though, that is intended to supplement, not supplant, in this addendum, At what point and where exactly by title are we asking our LEAs to report on, well, you're not. demonstrate that you're not supplanting, that you're actually, you know, we've already

460showed our allocations in the LCAP template, but where are we showing what we're doing with the different title funds in this addendum, beyond the narrative at the top? And that's not the intent of the addendum. The addendum is really the intent is to apply for federal funds. This is how we're going to move forward. What you're asking is is really moving forward kind of goes to the FPM question it goes to the implementation question which is not a former function of the addendum itself and And you said we're still waiting for guidance From the department correct and so we're piloting This and we're waiting for some guidance that is supposed to trickle down from the Department of Education We're waiting for guidance and this item is talking about the per pupil expenditures requirement. This is

461this is based on when and and assuming that our SS state plan is approved It's waiting for that approval To be able to then implement an LEA plan I'm trying to scroll back up where I said that we're waiting for some guidance on that's on the school site Budgeting that's sort of that's a whole nother ballgame That's that's like an extra provision hanging out there and and that somebody inserted in the law So that's a that's a new ballgame I'm still finally on my so the Federal aid is for schools. So ting. How would you handle that? I mean, well you are a school So we need to ask a big district. Yeah, if we're Sacramento City and I'm doing in Yes, Karen, okay if in Riverside if if we're a large organization and we

462have this tiny sliver of federal money and the l caps are district in that driven in its thinking and the the s of law is school-based thinking in its Orientation how would that that that doesn't mesh very easily does it or doesn't it so there's there's three parts to this um you know there's the l cap which we have and and we really want to have that be the driver and Then you have this Addendum which would be your lea plan and then there's the site single plan and it's the school plan the school level That's where they're going to show how they're expending the dollars and they're accountable and use a school site council and a whole system for that And that's the third piece I think that needs to have this alignment and

463if we're really looking and focusing all three documents on the lcap priorities That's going to move us forward. I think in timeline. There's two windows to con app so the consolidated app is an online process where districts go and and either report how they're spending or see what their allocations are going to be, if you will. And that's twice a year, and it kind of closes in June, which is the same time that the LCAP is due. So I see that alignment in time to be able to look at this. But a lot of the provisions are going to be on CONAP, if I remember this correctly, and then some will be inherent in the LCAP. which we'd have to identify. And then there are things that are not, and that's what you're talking about

464would come here. Right. Correct. Okay. Yeah, right. It's getting... So you wanted some feedback, so we gave it to you. That's what you came here for. You got what you came here for. Yeah, my mom used to say, careful what you wish for. Yeah, that's right. Board members are out. I mean, I think that as we talk to school leaders, they echo what we're saying is we want to get this aligned, and we want to not be doing lots of different documents when in technology-wise we could cross-populate things. And so when Mike and I met with... the administrators, the superintendents from the accountability task force, we told them, they asked, where is this in process? And we said, it's in process. So I think we need to keep working on it. I think we need

465to make this workable. I mean, having lots of different, I mean, people are reading the LCAP. I don't think in my district people read the LEA plan that I. helped work on even, I mean, we did it because it was a requirement to do it, but I think the LCAP has got a lot of input. People are looking at it, and people want alignment. They want to know where the money's going. They want to know what the actions are, and so I think this is an important thing to keep moving ahead on, and I know, Jeff, you said that you were going out for more stakeholder input, and so I'd like to see us keep this. What's your sense of the timeline? Is it spring when we'll have an alignment? If we're going to use

466this in... 18-19, I hope to bring it back to you in January so then the LEAs would have the ability to utilize it as part of the planning process with the LCAP. I would also say that to work with the single plan, that's also part of the work that we're working with the Title I office with our integrated planning support team to align those as well. And if I could say one other thing is the e-template is being developed. for the LCAP and these provisions are also being integrated within that. So part of that, these 35 pages look daunting and I would agree with that, but as you apply for, let's say you're not applying for Title I-D, you would mark that as far as the electronic template and then those 12 provisions wouldn't even

467show up for the LCAP. So then that makes it in and of itself a little less daunting to have those working. So we are working to integrate this. whatever direction that you give us is to have this as part of the e-template as well for that. And we are working to align the three plans as you suggested. Okay, Karen. And just one other piece is that the Title III plan is embedded in this. So that's another plan that kind of is consolidated. And then is this an annual plan or is this? Because CONEP is annual. Right. Well, the CONEP is annual, and that would be the guidance from the state board is to look at it. We're looking at it initially to have it aligned with the LCAP cycle. So for this time, it would

468be. for the last two years of this current three-year LCAP cycle, and then the frequency and under what considerations moving forward would be defer to your guidance. Okay, and then finally, districts are being told, is what I'm hearing, is that we cannot apply for CONAP and go through that process without doing this for the spring. So, I mean, there is a timeline that we've been told. Is that still correct? That's my understanding. Yeah. Board Member Rucker. And my question was related to what Board Member Valdez brought up concerning what notice or information have districts received concerning the consolidated application in September with the approval of the state plan. the ESSA state plan for the state to submit, one of the comments that was made was this was going to change what the state was going

469to do with and how they would treat the consolidated application process. So can you give us an overview of what kind of guidance districts have received to this point about what is going to happen with the CONEP? That's my first question. And then the second question would be, in preparation for this item, I went to the CDE website. and I went looking for the SPSA and it apparently was taken down in October the SPSA guide as well as the planning template so I'm assuming part of the reason why it was taken down was to do some work on creating some alignment in that document with the LCAP and the consolidated application process so I'd like to know if you could how that process is working because I'm thinking because we weren't really clear on what

470this template or this addendum would look like. You know, in my mind, thinking back to what the consolidated application online template looked like, there are parts and pieces of that that already work very well with what you're doing when districts develop and write their LCAP and similar to what's going on with the CIPSA. So I'm just kind of hoping it's not quite as complicated as it was explained. Was that gentle? That was gentle. Okay, yeah. Kerrick Ashley, Deputy Superintendent. Yes, so we'll come back in January with more information, including the single plan for student achievement and what kind of alignment that might have. And that's exactly right. It was taken down in order to align. And I can tell you, and we'll bring back more information about the consolidated application. I can tell you changes

471have already been made in that for provisions in law that were taken away, like the program improvement. requirements and that sort of thing so we'll bring it back okay are we ready for public comment i think will the following people please line up to speak for item five samantha tran sarah litis brian raivas and liz colin i wouldn't abandon you hi liz guian with public advocates um I think that this proposal for the addendum is definitely in the right direction. But we think that there should be a little bit more guidance for districts on it, especially because of the supplement not supplant role. And so in our letter from the Equity Coalition, we recommended that for each of the title funding sources that the LEA that there be some kind of a box that shows

472that allows them to explain how the use of funding meets the supplement not supplant requirement. In addition, we wanted to lift up the school School funding reporting requirement and we made a recommendation that a working group be created of stakeholders as well as FICMAD and SUCCESA CASBO AXA and so forth to help us Work on rolling this out. Thank you Samantha Tran with Children Now. We believe the state's moving in the right direction of bringing together the LCAP and the planning requirements at the federal level. Picking up on Board Member Ortiz-Lacone's comments about the supplement not supplant, in our third letter to the board, we included some recommendations on that front. We're also mindful of the conversation that you had about the site level expenditures, similar to what Liz just brought up, and think that

473absent guidance from the state districts, 1,000 districts across the state are going to have to spend time and resources to figure it out. And while the data won't be comparable, that communities will probably make comparisons. And so we do think that this recommendation of bringing a working group together to develop some guidelines would be helpful. Thank you. At the risk of being redundant, Sarah Lillis with EdVoice. And again, just want to urge the board to direct staff to actually convene a working group to look at the site fiscal requirements. We do think this is an opportunity, again, to lead to not wait for guidance from the federal government to actually figure out how this requirement could be a useful tool for the local planning that we as California have already set in place. So it's

474a requirement we have to do because of ESSA, but how can we make it about the California local planning process? meaningful and transparent and comparable tool as both the previous speakers mentioned. So thanks. President Kirst, Brian Revis on behalf of the Education Trust West. Really just in agreement with my colleagues about the need to proceed with establishing a work group and figuring out how the state's going to comply with the reporting of district and school site expenditures on a per pupil basis. I agree with my colleague, Ms. Tran, that We need to figure out how to do this consistently so that we can get comparable information across districts. Absent that, people will make those comparisons and assume the data are comparable. So I think we can avoid a lot of confusion and make the information

475more useful to all of us if we get to work on this and not wait for federal guidance that may or may not be coming. But we need to get to work on this. Thank you. Okay, that's the end of the public comment so we're now on to item six it's It's interesting, as I've observed before, we spend hours on accountability and very little time and hear very little public reaction to capacity building for our teachers to be able to teach this curriculum. To me, there's always been an imbalance in attention, and we welcome Wes, Ed, here to present something now on where we are on capacity for the board's deliberation. It's not just all accountability. Trying to find our slides here. President Kirst, Superintendent Torrex, and members of the board, it's great to be

476back. Can't hear us? How about this microphone? Is that better? Yeah, I think you're on. Let's see, we'll just put it in the middle. All right, yeah, let me step up. So we're back. We were here, Robert and I, March of 2016, talking about this issue of standards implementation. WestEd has been reporting on surveys and focus groups with California educators since 2015. around the implementation, the ongoing implementation of the California academic standards in English language arts and math. So my name is Rayno Mackinnon. I'm a researcher with WestEd in San Francisco. Robert Sheffield, director of California Initiatives for the Center for the Future of Teaching and Learning. And I'll give a brief overview of the presentation today. I think we're going to talk for about 20 minutes about what we're learning. First, we'll start with

477a background of the different initiatives that WestEd has underway around standards. Standards are a very multifaceted topic, obviously. It changes a lot. It has a lot of interrelated issues. We're going to talk about changes in teacher professional learning that we've seen from the data, as well as teacher involvement in some key school decisions, the needs that teachers report around implementing the academic standards. progress in each of those need areas. They're identified as high needs, but we're seeing progress and we have updated information about each of those need areas. We're going to focus mainly on materials and teacher collaboration particularly. Some information on the reported standards driven instructional shifts that teachers are reporting making because of the new standards for students. And then finally our work moving into next year. So that's the overview we're going

478to go. We're going to spend about one or two minutes on each topic. And first background, WestEd has been the California data partner for this Measure to Learn and Improve initiative, which is funded by the Bill and Melinda Gates Foundation since fall 2015. And the initiative involves 12 states examining the implementation of college and career-ready standards. So California is one of them. It also includes Florida, New York, Massachusetts, Tennessee, others. And we presented information from statewide surveys, part of a statewide survey effort primarily, led by each data partner. Now when we presented back to the board on March 2000, we had talked about fall 2015 surveys that we administered with the help of the board. And all of the state data partners were in a similar situation we were. We got low response rates among

479educators. In response to that, the initiative, the Gates folks and the state data partners decided to do the surveying through the Rand Corporation's American Teacher Panel Survey, which is an established survey of a set of teachers across the country and in each state. So we added some California questions to the core questions that are across. each of the states and we got a better response rates that way. So in October of 2016, asking teachers about the 15-16 school year and May 2017, asking teachers about 16-17, we got results from the American Teacher Panel. Now most of the data we're going to report today are from the May 2017 surveys of teachers. 798 teachers were in the panel for California, 479 responded, so that's about a 60% response rate. the results were weighted for non-response. So

480the margin of error is about 4.5% plus or minus. So when we show you figures, you can apply that margin of error to the numbers you see here. So we're hearing from just under 500 California teachers across geographic region, stratified for balance across geographic regions, grade span, subject area discipline. So we're getting some balance across demographics and areas. But we're not just going to talk about the results of the surveys. We have a lot of other initiatives, as you said, and Robert's going to list through some of them where else we pulled from for our presentation today. As we were putting together a report this year, we felt it would be absolutely critical as an agency to provide the best possible understanding that we could give on the variety of information that we've been finding

481across multiple initiatives within WestEd. So as we begin to think about what we're going to talk about today, you'll have the perspective of the MLI data that Rayno has just talked about, plus many qualitative data points that we want to share as well. As we begin to think about this, there's work that's been done through focus groups that has been led by members of my team, as well as district interviews that have been done with district leaders from a variety of different districts throughout the state. We also have gathered information from the California Comprehensive Center. In addition to work that's been done through our evaluation of the math and common networks and also work done with the leading and learning I3 grant that is taking place within Fresno as well as in Sac City. And

482then finally, information that comes from out of the Center for the Standards and Assessment Implementation. If anyone would like to find out more information about any of these projects, the web links are available either through the PDF documents or through the PowerPoints that are available to you all. With that context set, now we'll get into some of these. Yeah, and I think our slides are really based. We have a knowledge brief that Robert and I wrote. It's about 10 pages. It's a part of the agenda items. So that underlies everything we're going to say here. Nothing we say here is going to be beyond that brief. So it really underlies the work here. So the benefit of the survey data we have is that it's a panel survey. So it surveys the same panel of

483teachers over time. So you can see sort of within that group of teachers, you can see some changes within that group of California teachers. And the one thing we've seen. the data from comparing October 2016, which is the 15-16 school year, moving now to 2016-17, is that we're seeing shifts in the types of professional learning that teachers in California are reporting experiencing. Their higher proportions of the California teachers who answer the survey report this year, they were able to observe another teacher's classroom to get ideas for their own instruction to offer feedback. That went up to 54% of teachers said that they experienced that as opposed to 45%. 15, 16. And also teachers are meeting and working with their peers. That was smaller increases but just year to year bumps increases slightly to talk about

484standards and instruction. to develop materials or activities, to review student assessment data. And those types of activities, just think about what those experiences are. They're unique now and they're different than what we, Robert has a point about what you used to experience versus what you're now. I think we had a really interesting milestone when we reflect upon the movement towards local control that we're in the midst of right now and thinking about how do we scale in. local control initiatives, standards implementation, changes in various forms of policy into classrooms. And as a part of that movement, what we are seeing is that schools are reacting by making sure that teachers are having ownership of this particular work. And that's really significant thinking about where we were 15 to 20 years ago when we had a

485much more centralized rollout of that policy infrastructure that the state was building at that point, which also led to... centralized professional development, which looked like teachers being pulled out of classrooms, pulled out of schools, to central locations to sit through PD experiences that now we know are based upon the research to be ineffective PD practices. So we're now moving to a time and space where we're implementing the right types of professional learning structures to improve both the working and learning conditions of teachers, and we want to make sure that we're noting that as a part of this. So another point we got, another... benefit of the American teacher panel data is that it's across the state, the 12 states. So it does allow for some comparisons between a benchmarking with California versus the other large

486group of pooled 12 states. We had the individual teacher data from California, but not from the other states, but we did have the overall aggregate. proportions of responses from the other states for comparison purposes. And when we looked at those, we didn't see a lot of differences in the states, except in one key area. And that was consistent across a lot of questions, which we're going to go through now. And that was, you guys might have seen this in the EdSource piece last week, but teachers in California tend to report a higher level of involvement in key decisions at their school sites. For example, they report that they about six to seven percent more in California than other states report their comfortable voice and concerns in the school. They have a lot of informal opportunities

487to influence what happens in the school and they're involved in making important decisions in the school about six or seven percent higher in California than in the other pool of states. So that's it was an interesting and pretty consistent result that we saw. We do some of this state-to-state benchmarking in this presentation. We were hoping to do a bit more. Gates is funding a larger online tool called Bento, which is going to allow for... all users to compare states to do this kind of benchmarking online. That tool called Bento will be out in January. We were hoping we'd have it available now, but we're going to talk a little bit about that benchmarking, particularly in this area for this. But in a few months, that tool will be available for more of that work. And

488so I think a point here is, so what does this mean specifically? Well, teachers were asked... In particular, at your school, how much influence do teachers have over school policy in each of the following areas? Now, you can see them listed here. I know it's a little small, but before you look at the differences, just look from top to bottom. Teachers tend to report that teachers at their site have influence over materials, standards for student behavior, and determining how student progress is measured. Now, over a majority of teachers tend to say that they're less involved and less influence over... planning for budget, for example, down the bottom. But again, in this area, across all the questions, California teachers tended to report 4% to 5%, if not more, percent higher degree of involvement among teachers at

489their sites versus the overall sample of other MLI states. And I think that's really interesting because of how consistent it is. Now, I'd like to draw your attention to the second line, setting standards for student behavior. There was a separate research study last week that came out from Dr. Richard Ingersoll at the University of Pennsylvania that actually correlated the level of teacher leadership of the school positively with student achievement results. And in particular, level of teacher involvement at the site setting standards for student behavior had a high correlation with positive level of student achievement. That's a separate study that just came out last week. So it does seem to give the theory behind this for teacher involvement in key decisions at the site being an important aspect to improving learning and getting improved outcomes. So

490while we do want to celebrate the fact that California is outperforming other states based upon these indicators, when you begin to drill down and take a closer look at the indicators, what you'll notice is that these are the types of activities that we want teachers involved in. So while these numbers are positive, in many cases, we want to see 100%. of teachers reporting influence in these areas. So as we're thinking about this and we're continuing to try to build the right types of conditions for teachers to transition practice when necessary to the new types of standards or the practices that align to the standards, we need to be thinking about how can we help school level administrators, local leaders, as well as the teacher associations and teachers to become involved in conversations around how do

491we improve these indicators? How do we continue to accelerate the types of conditions within school buildings? that lead teachers to be able to say we really take ownership over this core work of improving our schools and implementing these new standards and instructional materials and assessments that have come along with this move that we're now in. I think another point just to make from the survey data is that in both the October 2016 and May 2017 surveys, 65% of teachers in California said that their school cultivates teacher leaders, that is teachers who influence structural practice of their site, to make progress in implementing the state standards. So that's pretty consistent. So that's about two-thirds of sites say they have a cadre of teacher leaders. That means a third of the sites don't. But that's a positive

492result. And then what do they do? Well, the teacher leaders, according to the surveys, they provide effective expertise and guidance during peer collaboration. So they provide guidance for collaboration. And they provide tools that help teachers work together. Now, 68% of teachers agreed with the first statement and 64% with the second. And these, again, were about 6% to 7% higher than the MLI states together. We have some materials out from our colleague Rebecca Perry, who leads the Math in Common Initiative, as well as Deb Segment at the Center for Standards and Assessment Implementation, looking at some districts' local practices in this area, teachers on special assignment or coaches, what they do, how they sort of lead this, what it really looks like. And they talk about sharing demonstration lessons to sort of do that, or to

493lead the school-based PD, lead the conversations at the sites. And then they meet together as their own learning network to talk about, well, how well are we, how are we building this culture? for learning. What are we doing? What are we learning from doing this? And so I think I direct you folks, it's in the bibliography at the back of our brief, to read those pieces as well because they really give a local flavor and show local practice in this area. And then finally, I think, not finally, but next point is I think this is a key one. So we're getting to the standards implementation area. Teachers were asked to select five things they need most advanced implementation of the California standards at their school. The most popular was higher quality textbooks curricular and instructional

494materials aligned with the standards 64% of teachers indicated that second was digital tools 52% teachers indicated that area more opportunities to collaborate with other teachers more time to observe other teachers teaching And finally, more effective strategies for teacher students with special needs. So these are the types of things that teachers are seeking and teachers are looking for. Now, we're going to talk a little bit more in the rest of the presentation about these areas and what we're learning about each of these areas, what we know from the various work that we have about what's happening in California in each of these need areas. So the first place that we'll take a deeper dive will be on instructional materials. And I'm sure that as any of you all have worked in schools, you know that instructional

495materials is kind of one of those hot button issues that we will always be having conversations about. So in order for us to continue to try to just kind of tease out some of these issues that teachers are talking with us about, I'm a social studies person by trade, so I'm really sorry I'm missing tomorrow. But in terms of what we've heard around pain points in instructional materials, we've primarily heard teachers as well as district leaders say, we need more help with our math instructional materials. So the information that we're going to share with you all now comes from a set of districts that have been outperforming, actually been growing rapidly in terms of the math performance. And we wanted to take a look at what are they doing in order to improve the use

496of their instructional materials. So the first point that we noticed was that they were making sure they included teachers within the core. of evaluating the resources, but more importantly gathering feedback on the degree to which the resources are actually meeting the expectations for which they were adopted. The next point that we're noticing is that districts are really beginning to organize themselves around best practice in terms of curriculum design and deployment, making sure that they're building units of study that include common assessments, include instructional materials and pacing guides that are around the framework. And we know that the framework serves as that basic anchoring document that guides all of us in terms of the implementation of the standards. The next piece that we're seeing is that districts are using the Smarter Balanced Interim Assessment Blocks, which

497is a resource that the state is invested in, a high-quality resource that in some cases has been underutilized, but we are starting to see districts that are using the platform in order to make sure that their curriculum and instruction materials and assessments are aligned in the best possible ways. And then finally, we are seeing some districts that are taking a different look at the way in which they're using their release days to make sure that teachers have the time to do the core work of examining assessment resources as well as developing curriculum to possibly fill gaps that they may be seeing within their adopted resources. Another view of this particular challenge of instructional materials just comes in the way that teachers are thinking about solving this challenge for themselves. And, you know, as a teacher

498myself, former teacher, I think all teachers supplement. So when we think, when we begin to talk about this idea of supplementing instructional materials, this is a common behavior of teachers. So it's not that... I don't have adopted resources. It's just that I know as a teacher my adopted resource is but one resource I use in my classroom amongst many. So what we're beginning to see is that with now the internet being kind of a common source of information for all of us, teacher behavior has shifted from going to my peer down the hall to ask about a resource to now going online to find as many resources as they possibly can. So while there's a new opportunity that's here, we also need to think about how do we help teachers to improve their sourcing of

499resources online. One of the other points that we're seeing is that teachers want to make sure that they have better materials for students that are possibly lower performing or have different types of needs that the materials may not meet or those students that are exceeding expectations that need to be stretched within their learning. So teachers are going online to find those types of resources. In terms of trying to get a handle on how teachers are thinking about the sourcing of resources, we were able to pick up a few trends this year, and these trends actually match some of the earlier trends that we picked up a couple of years ago through our focus groups. One of the first things that teachers, of course, look for when they're going online to find materials are, do the

500materials include the standards? Now, that is a type of alignment that is, first of all, good. We're going to make sure that the materials are on face value. aligned with the standards. But what we also know is that there has to be an appropriate depth of knowledge that's there. So many teachers know that as they're looking at tasks or assignments or reading materials online, that they have to just go beyond whether or not the standard is labeled on the materials to be sure that it's representative of the type of task or the complexity of text that aligns to the standards. We also know just plainly teachers want to just make sure that they're written well. And there are times in which instructional materials are not written well. So these teachers are looking to make sure

501that materials just have no errors. And finally, are they easy to use? So ease of use is always one of those core questions that teachers have in terms of any types of instructional materials that would be placed in front of them. How easy will this activity or this... this textbook or this instructional program be, how easy will I be able to use this within my classroom? So as we're thinking about how to continue to help teachers within this particular area, as we're thinking about it, we're beginning to wonder how can we either from the state, from the counties, districts, begin to help teachers with the right types of evaluation resources or professional development to help teachers to identify what a strong instructional task looks like or a text that may be more complex on grade

502level or better representative of grade level expectations than a different type of instructional text that they may pull from online. So we need to make sure that we're getting exemplar tasks and text in front of teachers and then helping them to go through the process of sourcing online. So next, I think we're going to shift away. I'd say there's a long story to tell about the materials. These are based not only on our work, but we have a couple briefs from other WestEd colleagues. Stacey Marple in our Oakland office. Dan Bugler have authored briefs around the selection criteria, the tacit criteria that teachers use in materials. Again, they're in the bibliography of the brief, and I'd encourage folks to read them. They go a little more detail about those types of decisions. Thinking about the

503collaboration piece. Again, teachers... wanted more opportunities to collaborate. Well, we also had questions about what types of professional supports and collaboration teachers received in 2016-17. And these numbers reflect the proportion who agreed with these statements. That there's about 80% agree that their school can be its grade level teams, PLCs, to support standards implementation. Teachers are working together, 73%, so they work together to develop or revise instructional units around the standards. 65% said their school leaders provide opportunities for them to talk about standards across grades and subjects. And I think down the bottom, when you see the school district leaders provide adequate time to support the school's implementation of California standards, 58% of teachers agreed with that. Last year, that was 50%. So they're getting a little more space. I think this shows that, again, change

504just in the last few years we're seeing. But again, if you compare it, like Robert said, with what we saw 15 years ago, previous standards adoptions, this is very different. These sort of the opportunities in the site to talk with one another and rely on each other to learn from one another around these. learning goals and expect expectations for students. The foundation is there to build on this. Again, these aren't perfect. We do want these to be a little bit higher, I think. This is saying about a third to 40% of teachers aren't getting these types of experiences. They didn't agree with these statements. I'd like to see them higher, but this doesn't indicate at least the majority are seeing positive sort of professional learning opportunities and more collaboration than they've seen in the past.

505But collaboration in and of itself isn't a panacea. So I think Robert likes to make this point. I think he makes it better than I do. As we've talked with instructional coaches, principals, as well as district administrators over the past year, almost all of them have said that they have some type of professional learning community or collaborative teams that are working within their school buildings. But many of these leaders are also noting to us that these structures are underutilized. In some cases, they're not meeting their potential. So as we go into the best practice literature, what we find is that in order to make sure that teachers are maximizing the use of this time, we really need to get them deeply involved into the work of analyzing the curricular choices that they're making, the types

506of activities that they're developing for students, and being sure that they're working on developing assessments together, and then beginning to think about how to implement practice in the classroom with some type of peer observation around that in order for them to be able to collect the data or information around their own practice and then be able to offer guidance towards each other. So we do have some points of light around that, and we want to make sure that we're communicating those points also. And Elizabeth Sturzich, who used to be at Stanford, now at Brown, has done a lot of work around that, around teachers when they first start talking saying, well, I'm teaching to the standards. But when they get sort of an inquiry protocol around really looking at student work, like, oh, I really

507need to do things differently in order to do this. So I think it's getting that collaboration more inquiry-based, a little more focused, a little more rigorous. And I think her work has really sort of brought that out. So we do want to make some points around just differentiated support because we know just the range of needs that exist among schools and districts throughout the state. So in our district interviews from last year, we chose to help take a heavier focus on what was happening in many of the rural communities. And then also the MLI data this year, we were able to disaggregate by geographic locale. So we have a better representation of the differences between some of our urban suburban areas and our rural areas. But what we're noting is that while instructional coaches were

508used as kind of the key vehicle towards building capacity for teachers within our urban and suburban areas. Our rural communities just don't have those resources. They don't have the same number of teachers within the pipeline. And when they pull a great teacher out of the classroom, it has a greater effect on their schools than it may have in one of our urban or suburban communities. So we don't see that same type of behavior occurring within many of those communities. But what we do see is a heavier reliance upon teachers that are... have greater expertise as well as greater experience. And these teachers are helping these districts to make the types of decisions that they need, in particular around their instructional materials. So we're seeing in those particular areas higher incidence of teachers reporting that they

509have some or great levels, great deal of influence on those types of decisions. We're also seeing within those communities a heavier reliance upon online professional. development resources. So somewhere about 11% difference is what we see. And we also see a higher rate of those teachers saying that the PD that they're receiving is higher quality than we're seeing within. urban and suburban areas. So kind of one of the key points is as we're thinking about differentiated support throughout the state, how can we make sure that we're delivering the types of PD that we see has uptake within certain geographic locales? So if we know that the rural communities have a greater appetite, greater desire, and in some cases, greater utilization of online professional development, how do we make sure that we include that within some type

510of differentiated support strategy? And as we begin to now transition out of supports and thinking more about instructional practices that we've observed, one of the key implementation approaches that we've seen throughout the state, and this represents about 66% of the districts that we've spoken to, is that they've made a really intense focus upon identifying the instructional strategies. that they feel would best meet the needs of their students and their teachers. And that's been the way in which they've chosen to work towards implementing the standards with some level of fidelity as well as proficiency for students. And Raina will share a little bit more about what that data looks like. Yeah, so teachers who on the surveys reported that they teach English language arts were asked to the... to the extent whether they're influenced on a

511list of instructional practices, 17 different instructional practices, had decreased, stayed the same, or increased since the implementation of the ELA standards in California. So these are the four instructional practices where a majority, over 50%, of ELA teachers indicated or reported increasing their emphasis since the implementation of the standards. I don't think you'd be surprised. They want their students, having their students explain their reasoning or thinking, solving problems, have them construct viable arguments. Analyze how two or more texts address similar themes. Majority of teachers in California, the ELA, are saying they're increasing the emphasis on doing this. There's less emphasis on test-taking strategies in other areas. But these are the areas where in ELA we're seeing increasing self-reported emphasis among teachers. And in math, again, I'm sorry this is a little bit smaller, but again, we

512had six... instructional practices that a majority of math teachers said they've increased their emphasis on since the math standards have been implemented. And again, explaining their reasoning, comparing contrast, different methods for solving a problem, and applying math in the real world where areas where the teachers, math teachers, are increasing their emphasis. Again, in math, we have benefits of standards for mathematical practice, and they align with this. So that's been really helpful, I think, in guiding a lot of the work among teachers. So I think we've really kind of raced through the slides today. I think if folks want to read the knowledge brief, we're happy to provide further information or connect you with colleagues as well. But we have other work in this area as we move into 2018 next year that Robert's going to

513touch on. First key area of focus for us will be on continuing to gather information on what's happening in the field. One of the key points that we're beginning, of course, we know and as well as we are part of is the development of networks throughout the state. And what we're identifying is one of the key challenges with so many networks blooming and blossoming is that there needs to be knowledge management. So as these networks continue to develop and construct new knowledge about how to solve their local problems, there will be... need to spread that knowledge and practice to different areas of the state and so what we know is that we want to be involved in that type of knowledge management as well as sharing of information. We'll look to do that through a

514few of the established networks that we already have in place. One of them being the California Standards Technical Assistance Network which is a group of nonprofits, higher educational institutions, as well as county offices of education that convene in order to work on being coherent with the state's infrastructure and being sure that they're sharing kind of the same message as they're doing the work within districts. We also continue to ramp up a pilot project that we started within Placer County earlier this year where we're looking to take many of the same types of items that generated the data that you're seeing within this presentation and help county offices of education as well as districts learn how to use those items. and construct their own local survey instruments in order to gather similar kinds of data. We're

515still involved in math in common. We're still involved in leading with learning. And we will be sharing the knowledge that we gain through a variety of different types of structures and presentations. I want to put one final pitch in for the millions of people who are watching the live stream. That our working group of California County Office leaders is still, we'd love, as the County Office folks who are watching, please reach out to us. We want to think about having a wide, diverse, geographically diverse group of County Office leaders thinking about survey items and what questions to ask of teachers in districts. Much like what we heard earlier about these surveys, as this is a burgeoning area, we want to get voices from all over the state, rural areas, urban areas, together to start talking

516about surveys. And the best way is to sort of learn from them and use them and then build a process for analyzing data and making... changes based on them so county office folks watching please reach out to robert and i we're going to get this working group up and running we're going to learn together okay thank you uh board member comments board member strauss i just want to thank you um both for first of all, for being here all day and then for waiting to present to us. But to me, the key points that you made is that we're doing it differently and there's some hopeful data here about teachers saying what really makes a difference, the value of collaborative time, the value of aligned materials and observing and learning from each other and that

517districts are implementing these standards in a different way than we did in the last round. And so I think that you have some important... findings that might help guide not only county offices as they support the districts in their area, but also district leaders as they look. And also because they've said that some people are being pulled out, but I know we have a substitute shortage now. So we're having to look at other ways to meet the goals and have those strategies where teachers can walk each other, they can collaborate around lesson planning, they can look at aligned materials. And I think what we're finding is some of the books that we've adopted early on, say in mathematics, weren't as aligned to the standards. And so districts are needing to do what you've said, which

518is supplemental materials that are aligned, that are well prepared, that they can use. And they need time. I've found districts doing that more in the summer or non-student days where they could get the whole department together. But I think the things that you focused on and the findings and what was important should be shared in a variety of networks, not just county, maybe AXA, instructional leaders, but... the county instructional leaders. But I think it's important to share these because I think we have still, as we know, a lot of work to do to touch all of our classrooms. Thanks. Board Member Ortiz-Lacombe. Thank you for your presentation. Thank you for the energy. And I seriously doubt that anyone is looking at this. No one. I mean, this is it. We're still here. And I don't

519blame them. I read your knowledge brief, and one of the things in page six that really stood out was that both the Center for the Future of Teaching, the interviews, and the RAND survey really identified the struggles around English learners that both principals and teachers have in identifying effective strategies to teach and reach these students. Can you tell me a little bit more about that? Because I was surprised that it was the one subgroup that was identified by principals and teachers. Yeah, what we found is it's a multifaceted issue, as we know, but in terms of kind of taking a slice of this from the standards implementation perspective, teachers as well as district leaders and principals are reporting that the instructional materials that they have are just not adequate. And what we found in particular

520in areas where there is, of course, a higher concentration of ELs, that data actually, that level of unsatisfaction is even higher. But there's also a different issue that shows up in some of our areas where there may be just very isolated pockets of ELs where they have no resources. So it goes from one end of the spectrum where we have resources that just don't meet our needs to I don't know what to do or what to find. Then we're also hearing that in terms of professional development, they still don't have the types of professional development resources necessary to move instruction in ways that. actually kind of aligned to the new vision of what we hope for English language development here in the state, where we're taking a perspective of saying that ELs are no longer

521a special population, but they are our population. So therefore, within our way of thinking about curriculum and instruction, English language development needs to be at the heart of it. So they're really still not seeing that type of professional development opportunities for themselves. And I imagine that as we do here in California, we will build it, but that's what they're calling for right now. Thank you. And that's very consistent with what I hear from some of the teachers and principals. So integrated designated still hasn't trickled down that expectation. So I think it's great that we are thinking of knowledge management and making sure that resources are distributed. But thank you for that. Board Member Rucker. Well, first of all, I want to thank you for coming and making this presentation. And you were a little modest

522in putting together the work that you've done on this, because I do know that you reached out to and talked to a number of stakeholders, not only in reviewing your data, but in reviewing your report and what the report says. So I really want to give you accommodation for doing that as well. I actually find the information in the report hopeful, because I started on this board along with a number of people. at a time when California. adopted the Common Core Standards to compete for a one-time funding in a grant. And there really was no plan for implementation or supporting the field to actually implement the standards. And so this state has done a lot of things after the fact of the adoption of the standards, not only to provide funding, but to provide the

523impetus in the field with a state plan and some state guidance and support on how to... how to do the implementation. And so while there were pockets of innovation and pockets of success that were really uneven across the field, what this report does say is over time, the field has gotten better at it. but there's still more work to do. So I think that the outcomes from this survey is very hopeful, not just because California teachers clearly feel because of the initiatives that have been implemented across multiple levels in the state to do this work, but also because the implementation returns them to the type of educational instructional practices that many of them believe were the types of teaching they want to do. I think that that's something that's important. also noted. Going a little

524bit further, we've been consistent with the expectation to implement the standards. It hasn't been, as has happened in some states with the change of administration, a change about the priority of implementing the standards has also shifted. This is also something that gives us great information to learn from with each successive adoption, with some of the information about what's helpful with materials and with doing the rollout and with providing the support to provide the additional curriculum. People have come and advised the board regularly on what the steps are, the natural order of implementing standards and adopting frameworks and doing all those things. And we know for English language arts and math, we did it out of order. And there was a reason why. that happened out of order that has nothing to do with the decision

525that was made initially to adopt the standards. But thank you for doing this. And I'm one of those people, you know, is the glass half full or half empty? You know, what's in the glass? I think that this is one of those times where the glass is definitely half full. Good. Board member. I know this is on ELA and math. And by the way, it was a really good presentation. And I learned a lot from it and appreciated how fast you talked. And it was both thoughtful and energizing at the same time, which is what we needed at this point in time. I'm one of the liaisons on Next Generation Science Standards and was very involved in... supporting the development of the California NGSS Early Implementation Initiative. And WestEd has a division somewhere in that

526large organization that's doing an evaluation on that initiative and asking some of the same, not exactly the same kind of way that you've done it, but getting at some of the same kind of information. What are they learning as they... Implement those standards so at some point in time I personally would love to see you get to you guys get together with that office and Kind of compare notes and see if there's a compare contrast kind of thing that's different or there's a place where there's Synergy going on. I know they're seeing some synergy between use of NGSS reading materials or scientific information and improved reading scores and interest with kids. So I think it could be a really interesting kind of pulling together what you've got because this is the other part of our

527standards. And it came second. So they may have actually been learning some stuff from the ELA and math implementation. NGSS has probably benefited the implementation of that, at least in this early implementation initiative, probably benefited from coming later and watching the implementation of ELA and math. So just quickly, we have started asking some questions within all of our investigations around NGSS. And while the early implementer districts are out running, many of the other districts are just still moving off of zero. And what they would say to us is that we have all of this other stuff that we're working on. Now, from a teacher's perspective, especially the elementary school teachers, we're thinking about now the third set of standards, third set of instructional materials. And oh, by the way, history, social science is rolling out

528too. So we're kind of getting to this place of just. the bandwidth is really limited. So we are going to need to just be very attentive to how teachers and districts are measuring themselves through this phase of standards implementation also. Yes, I understand that. And I know that the conceptual shifts were in some ways more larger in NGSS than they were in ELA and math. But it is also a fact that we're going to have an instructional materials review soon in 2018 and that we are piloting the NGSS assessment. And so lots of the supports as well as policy levers for implementation are coming on board. And this would be a good time to find out what has been learned by the early implementers that could support those that are just getting started. Erin Valdez.

529On one of the slides you talked about, the key resource for teachers was Google or I know a lot of teachers that go to Instagram or Pinterest and so forth. In some ways, it's a new dawn. It's a new way of thinking because they've connected themselves and they're in communities of learning. They've formed these themselves. But there's so much what Trish was talking about with NGSS and that whole idea of the new ELPAC test. Some of the sample questions really focus on NGSS kinds of tasks that they have to look at. And so the notion of rigor, I didn't see the talk of rigor in here when teachers are selecting materials. And then in addition, do you ever get into any kind of integration? Because that's really the ultimate. you know, application and that critical

530thinking and applying the disciplines across, not just the silo of content. Is that anything that comes into this or will in the future? We haven't done any investigations to measure for that yet. But in terms of your question around rigor, I guess the word rigor, we framed it differently. So we framed it as depth of knowledge. So and that kind of goes with the DOK structure that, you know, is a part of the standards as well as part of the assessment frame. To your point, that's where we kind of measure for some of that. They indicated that that was those are one of their their sort of tacit criteria But having more explicit criteria for quality assurance is a good thing and I think some of the districts that have these procedures are trying to

531do that they're trying to bring teachers together to use a rubric to sort of define what quality materials are supplemental materials are so they're trying to build that in there I think fortunately it's been very ad hoc and informal and I think the districts are trying to have a more of a QA process around that and bring to get teacher voice the teachers together to understand what they're doing and why they're doing it I think that's gonna be a beneficial If that can be mimicked across districts, we've got a potential. My question is to the department. How are we going to integrate all these findings into the system of support? We had a long talk about systems of support. So if teachers don't have... Let's just say good el materials in a certain place that

532are good or they're in rural areas What we need to get the system support and you are you all talking together? It seems to me and having having these work together because this is This is giving us a deeper picture, and a lot of this system of support stuff is up here in terms of what we hear. So how are we going to connect the top and the bottom of the reality at the bottom to the support structure at the top? I can say one thing before our colleagues. Yeah, they answered that question. But I'd say I've gotten phone calls this week since this was posted from county office folks who are in county communities of practice on math where their county offices are organizing amongst themselves to try and build rubrics for materials adoption

533to try and provide that. So we're hearing it from counties that are trying to do this work. So I think it is happening among the community of practice work and the county offices, certainly. Bruce holiday, I'm sorry you wait. No go ahead So what I've been hearing is just what we were talking about with member Williams comment and that is that we're getting to a threshold with NGS that's coming down and history and social science and Everything else that's going on and that there are principles and there are superintendents who are saying we can't move on to the next thing we have to stop right here and and we'll get to that next year or the year after if we can and yet there are others that seem to be able to somehow dance through

534all of these various changes more smoothly and integrate them in really positive ways. Those are the examples that if we can find them and if we can see them clearly and understand them clearly, how they do that so that nothing is being left behind arbitrarily, making a decision out of the blue that we're not going to... have any professional development on NGS or something like that so yeah it'd be great if we could see some of those and understand those but I am encouraged by you know what you've presented today it looks like we're we're moving down the down the road our data here is really teacher centric however if we think about it more from a systems type perspective to address your question A system needs to have some type of capacity or infrastructure

535in it in order to make sure that it can maneuver and change as it needs to through these various policy initiatives. So in many cases, what we're hearing from those districts that are saying, I just can't do this, it's really a systems challenge. They don't have any more capacity. As we're building these networks out, the networks have to build infrastructure within them for the policy messages and the practice changes to happen and then flow down to the school level. So I would say that, you know, as we're continuing to see these networks evolve, we need to be asking questions around infrastructure development, how they're thinking about measuring for capacity within systems, and also thinking about how does the network take that learning and then structure that down into a school building? Because in many of

536the designs that we're looking at right now, that part is missing. And so that may lead us to ask a number of questions three years from now about why the networks aren't working. So that's the way, you know, at least in terms of what we've gathered so far that I'd share with you. Hey, Board Member Williams. Just a quick question. Are the networks, are you guys developing the networks or supporting the networks or they're organically growing on their own and you're just aware of them? We are, I would say yes to all of those. And is this network in the context of improvement science? Within all of the networks that we are either managing or we're evaluating or we're just simply advising, there are some types of improvement methodologies within them. So those methodologies may

537be different, but yes, we are seeing some type of improvement science being utilized within the networks that we are a part of. Okay, thanks. Any other questions? If not, we'll have public comment. Well, the following individuals line up to speak to item six only. And just a reminder, this is for item six only. Reno McCohen and Krishnamaheshwari. Hi, my name is Krishnamaheshwari. I'm the founder and president of Hindupedia, the online encyclopedia of Hinduism, and an advocate of education. Math is really important historically for the Hindu religion and for us as a parent. I'm a resident of Cupertino. My three kids had better do well in math. But the reality is a relatively affluent neighborhood or district like Cupertino School District, my son in the second grade has half of his fellow students, including him, participating

538in supplemental math classes. I imagine it's much worse across the rest of the state. The West Ed folks talked a lot about teacher capacity. That's the only way kids do well in class. And the reality is that teachers do go online. And they go and try to find the best materials that they possibly can. Is that enough? I don't believe it is. Why can't you as the SBE and the IQC look at online resources and provide a set of recommended resources? Khan Academy is just one of many examples. These things can facilitate capacity building in a relatively cheap way. Thank you. Okay, that concludes the comment there. I guess we're ready to audit. Okay, I think we need another break. Okay, we're entering the home stretch. So let's come home. Let's come back at 5.15.

539Board will reconvene with item seven. Michelle Center is here to present. It is. So we'll begin. Go ahead. Good afternoon, President Kearse, board members. Michelle Center with the assessment division. Happy to be here. I hear from Member Rucker that she's still wide awake and ready to go. So I think they may have given you cookies. So they kept some really meaty items for the end. So forgive me, but we're going to have to go in the weeds a little bit on some of our assessments. So item seven before you today is specific to our California science. Test or cast as well as our California Spanish assessment or CSA and I will probably slip into using those acronyms We have several actions before you today Very similar across the two assessments. We are asking for your

540approval of cast related test development documents including the blueprint general achievement level descriptors and the score reporting structure We're asking for the same approval of those exact same documents for the California Spanish assessment Additionally, we are asking that you authorize the CDE to make technical edits as necessary to those documents. Specifically, I would like to acknowledge our intent to implement the two proposed technical edits submitted by the California Science Teachers Association. Those two edits relate to using the term science domain instead of science content domain, as well as adding a phrase on page four of the blueprint that specifies across the grade. banned. Those edits were in the letter to you from the California Science Teachers Association. Additionally, the CDE recommends the additional language on page three of attachment one in paragraph three, which would

541acknowledge that the California's environmental principles and concepts will also be used as the context for items as appropriate to the three science content domains. That was a recommendation. by the State Education and Environment Roundtable. So we would like to make those additional technical edits as well as any others that are necessary. Before I begin, I must thank my team. I must thank ETS and all of the experts and stakeholders that have helped us get here. These blueprints were not easy, particularly the Next Generation Science Assessment Blueprint was a very complex blueprint to develop. And I appreciate all the help. I appreciate the fact that, you know, I was able to say, can I drop in at the California Science Teachers Association Conference? Can you get me in and can you meet with me? Luckily, the

542fires took Superintendent Torlakson away, so I stole your spot and was able to get in for a meeting there. But it was so nice that so many experts really have been able to just drop what they're doing and help us as we develop these tests. So as you know, the high-level test design for the CAST, which is aligned to the California Next Generation Science Standards, was approved by the board in March 2016. It indicates that the cast will be administered in grades 5, 8, and 1 in high school. We administer the cast pilot test in spring of 2017. We will be administering the field test in spring of 2018, followed by the operational launch in 2019. The cast will be an online assessment which includes both discrete or standalone items as well as performance tasks.

543We anticipate the test will take approximately two hours. All of those facts are consistent with the high-level test design that you approved. The cast has three segments. Segment A is designed to measure a broad sample of the California NGSS performance expectations. includes discrete items. Segment B is designed to deeply measure the California Next Generation Science Standards and includes performance tasks. Segment C is designed to provide both a broad as well as a deep measure of the California NGSS performance expectations, and that will contain either a performance task or a set of discrete items. Segment C samples from the full range of the California NGSS performance expectations each year. During the development of the cast we have re-evaluated the high-level test design and proposed the following amendments. First, we propose that the grade 5 cast include

544performance expectations from grades 3 through 5. Second, segment C of the grade 5 assessment will include performance expectations from grades 3 through 5. Lastly, the cast will allow for performance tasks to cover either integrated or single science domains. The cast blueprint as shown in attachment one So later I'm going to show you images of that blueprint, and you're not going to be able to read them. So that's why I just said in attachment one, it's designed to capture the test integration of the three dimension of the California NGSS. The blueprint reflects the depth, breadth, and rigor of the standards. The blueprint allows for equal sampling of the three science domains of earth and space science, life science, and physical science. The blueprint indicates that utilizing the sampling in segment C will allow the cast to

545assess all the performance expectations each and every year and the blueprint ensures that segments A and B rotate through all the standards every three years. The cast blueprint consists of six tables. I feel like I should apologize when I say that, but it consists of six tables. I will highlight and explain only some of those tables that best illustrate the design of the test. So shown here is table two, located on page six, which displays the science domain coverage of segments A and B on the cast. Table two shows the number of possible items and points in segments A and B. The middle cells show the range for each disciplinary core idea at each grade level. For example, in the items by DCI grade 5 column, there will be between one and three items from

546PS1 matters and interactions. The right two columns show the distribution of questions and points for segments A and B. As you can see, there will be between 8 and 12 items for each science domain in Segment A. Two performance tasks for all science domains for Segment B, and a total amount of questions and points for Segment A and B can be found in the bottom right cells. Shown here is table 4, which I believe is on page 8 of your attachment. The table illustrates the integration of the three dimensions of the California NGSS and displays the distribution of the performance expectations, this example for the high school assessment. Tables 5 and 6 provide the distribution of performance expectations for grades 5 and 8. So the X's in the table represent performance expectations that integrate the

547three dimensions of the standards represented at each intersection. So for example, in the SUP1 row, the first X represents a performance expectation that combines science and engineering practices. which is about asking questions, cross-cutting concept 7, stability and change, and physical science, which is about waves, physical science 4. The margins on the right and bottom of this table indicate the total item ranges that exist per science and engineering practice, per disciplinary core idea, and per science domain. Shown here is table one, which is on page five, that displays the CAST claims. An assessment claim describes the outcomes achieved with mastery of the standards within that claim. For the CAST, the draft blueprint was developed to measure an overall summative claim and three specific summative claims. The overall claim states that students can demonstrate their knowledge and

548skills associated with the three dimensions of the California NGSS. The three science domain claims state that students can demonstrate their knowledge and skills associated with one of the three science domains. You are also asked to take action on the CAST reporting structure. As this draft reporting structure shows, the test will report an overall score that includes four achievement levels, and results will be reported for each of the three science domains. Over time, We will look at empirical evidence to determine if there are additional reporting categories that could be supported by our assessment. The last action before you related to the CAST is about our general achievement level descriptors. The CAST general achievement level descriptors provide descriptions of student performance. They'll be used to develop reporting level descriptors that describe levels of performance to educators, students,

549and parents. This table shows the four general achievement level descriptors, and they're also in your attachment two. So finally, we've included some of the comments from our national science experts quoted here who have reviewed the proposed blueprint achievement level descriptors and reporting structure we have brought to you today. Feedback from the likes of Dr. Jim Pellegrino and Dr. Kathleen Scalise, as well as the University of Stanford SNAP experts and California teachers, gives us the confidence that we're building a world-class science assessment that we should be proud of and that is aligned to the next generation science standards. Now is going to move right into the Spanish assessment if unless you prefer that I not I would like to interrupt to make an award of somebody who's here before you move is that okay? Give us

550a little bit breathing. Oh Okay, well we're trying to honor let me know and I need to I thought you were gonna break from from science to Spanish Maybe we should take questions on science. I mean, I can't do science in Spanish. At 510, it's a little bit too much. They're not that similar. Okay, here's Leilani. I want to, why don't you stand up and we'll recognize you. Our former staff member, Lalani Aguinaldo, and we want to acknowledge her outstanding service. I have a ward here and the SBA office, SBE office, where she was director of legislation. She came from the legislature and has quite a strong background. She has accepted a position with the School Services of California. And while at SBE, Leilani was responsible for working with the governor's office and managing the

551staff work on legislation. It's a huge undertaking with all those bills. She has played a key role in the board's work on accountability, working with CPAC and CCEE and at the California Teacher Commission. So we very much appreciate your service and wish you all the best in your future work, and we'll probably see you again. Thank you. Okay. Okay, now we're going to take some board reactions and public comment on science. I just can't combine those two. And Board Member Holliday. Thank you. I think it's an important decision in the... science test for grades five and eight to include a wider grade span of standards three through five and six through eight however will will the items in the in the test be more greatly aligned with The latter grade will there be more items

552associated with fifth grade as opposed to third grade will that will the items be different? Well, will the third grade? content be more less detailed broader Stroke kind of thing. It's you know, is there a plan to It they're not all going to be equal, you know, it's not going to be same you know size shape in each of those grade levels or or will it So when we have to go off grades, so for example going into grades three and four when we administer a test in grade five, we will ensure that that item is written to be grade level appropriate. to be given to a fifth grader. What we did is we looked at the performance expectations that were covered in grade five because originally we had intended to only be on grade

553level and there were large gaps in the science content if we did not go to grades three and four. So I'm not going to say that we're going to under emphasize them because instead of under emphasizing grade level, I think you'd under emphasize particular content. If that makes sense. We did this based on quite a bit of feedback from our technical advisors. We also went over this decision with Stanford, with Helen Quinn, and Jonathan Osborne, and feel very confident that this is the right thing to do in terms of going off grade. We also made sure that we did get some things in writing, since this will be something we'll have to submit through peer review and it isn't quite, it isn't normal to have to go off grade level. I think we have a

554very good argument for doing so for this particular test. I think you do too. And so I'm very happy to see this. I really appreciate all of the work you do to consult with the experts, which, you know, included lots of people, but the ones that are sticking out of my mind are the California Science Teachers Association, who just have a mastery of NGSS down like nobody, you know, probably even better than Achieve these days. So I appreciate that you seek them out, that you make time to talk with them. They are so committed to having all of these parts work together, the assessment and the professional learning and the student learning. And I also appreciate that you go to the higher ed faculty that not only know NGSS, but are experts in science performance

555assessment, have actually written many of the items for either NAEP or TIMSS or PISA. And between those two sources and then the others that you consult with, you know, I'm impressed and excited about this test and really appreciate your leadership on this. Also kudos to the work that has been gone into this development of this particular test and it is aspirational and I think we just foreshadowing a little bit ahead that this is coming ahead of science materials and some of those things but I do think that it sets up a structure and an expectation of where we would like our science curriculum to go. with this particular assessment. And so I think as we launch this and it goes through the pilot and field test phases that we would want to see information data

556as it comes back so that we could kind of see where we are as a state. And then I think particularly at the high school level, given that we are... allowing LEAs to decide what grade level they'd actually like to give that. I'd be very curious to see the results based on some of the data that comes back by grade level, by course taking level, all of that. So I'm hoping that we collect that information as we are going through the field and pilot. Any other board comments? Yeah. Go ahead. Did 26 states adopt in GSS? I think it was, I know there were 26 lead states in the development of it, so I'm not sure how many adopted, but okay. Have any of them expressed interest in our particular assessment? Can't hear. So that's.

557Could you repeat the answer? I couldn't hear how many. So I heard that there were 19 states. Plus the District of Columbia. Yeah, and Guam. And Guam. And I hear from Jill, who just represented that states are interested in our test. I would say I've gotten very positive feedback when we go and we present. And we try and present as often as we can the design of our test. And it is unique, but it's also something that probably not every state. could do because they don't have the size that we have. But I think it was just today, in fact, that we got another email from another state interested in the science alternate that we do. So it's not uncommon for us to get contacted by other states. OK, thank you. Okay, I think now

558you should proceed on Spanish and we'll take public comment on both. Okay, so we're going to go on to our Spanish assessment. As you know, you approved the high-level test design for the CSA in September of 2016. I remember that day because Member Rucker had stars in her eyes because Kenji Hakuta was here next to me. I wish he was here with me right now. You may remember. It's the only time that I've seen you. So giddy. So you will remember that the optional CSA will be administered in grades three through eight in high school, that they are aligned with a common core in Espanol. We administered the pilot test this fall. We're on track for administering a field test in fall of 2018 and we have an operational assessment scheduled for spring of 2019.

559The CSA is again, it's an online test. It will be machine scored and includes both discrete items as well as passage based items. The purpose of the CSA was to was included in the high-level test design that you all approved. The board approved the purpose to be a measure of students' competency in Spanish reading and language arts and provide student-level data in Spanish competency. The test may be used to evaluate the implementation of Spanish language arts programs at the local level, as well as provide a high school measure suitable, in part, for the state seal of biliteracy. This CSA is a voluntary test designed for students who are receiving instruction in Spanish in California or are seeking a measure that recognizes their Spanish specific reading, writing, and listening skills. The three CSA claims that we

560will be assessing given the blueprint before you today are reading, writing mechanics, and listening. You will note that we did change from the high-level test design from using the term writing to using the term writing mechanics because we believe it more accurately represents what is being measured on this assessment. This is a snapshot of the CSA test blueprint. In attachment 3, you can see the full blueprint for all grades and all grade bands. In the table, you can see the claim for the score reporting category, the content category, the total items by content or category, the total items by claim, as well as the percentage items by claim. The next action before you is approval of the CSA general achievement level descriptors. These provide, again, a general description of student performance across the three levels.

561They will be used to develop reporting achievement level descriptors that provide levels of performance to educators, students, parents, and the general public. The approval of these general achievement level descriptors will guide the development of domain and grade span specific achievement level descriptors. The final action before the board, because you know we have quite a few in this item, is to approve the CSA score reporting structure. At the top level of the overall scale score, overall score is, the top level is the overall scale score, and it has three achievement levels. The overall score is based on all test items from the three domains. Subscores for reading, writing mechanics, and listening will also be generated. finally we have included some of the comments from our national linguistic experts dr. Kenji Hakuta and dr. Guadalupe Valdez from

562Stanford University they have reviewed and approved of the proposed blueprints general achievement level descriptors and reporting structure that are before you today And this last slide includes the future actions, and gosh, there's a lot of bullets on here, for the CAST and the CSA. So you'll see that we have to continue our development and implementation of these three assessments, of these two assessments. We have our grade-specific achievement level descriptors coming back to you, revisions to the blueprint if necessary. We also are doing the same types of actions for CSA. So they're on a very similar track and will most likely be taken to the very same board meetings as long as we can meet that schedule given the actions are so similar. So are there any questions? Forward discussion on the questions and discussion on

563the Spanish assessment. Okay. Hearing none, then we're ready for the public comment on both of these assessments, science and Spanish. Speakers, for item number seven only, Doug McRae, Jill Grace, Martha Zargo-Ledias, please come to the podium, please. Good evening. My name is Martha Zaragoza-Diaz, and I'm representing CABE and Californians Together. First, I would like to thank CDE staff for all the work that they have done thus far on the California Spanish assessment. So my comments are specific to the to the CSA. We believe that the recommendation contained in this item by staff should also include the reference that's in the ESSA state plan regarding the fact that we are still working with CDE to look at the feasibility of the inclusion of the CSA in the state's accountability system. It is reflected in the ESSA

564state plan, and we believe that it should be reflected here in the recommendation. action specific to the CSA. We also have the question with regards to why is the CSA, why does it only have three levels of proficiency assessment? It is recommended that the board advise EDE to develop parallel reporting on proficiency levels for the CSA as with all of the other academic indicators. Thank you. Good evening. Jill Grace, president of the California Science Teachers Association, who now knows you're not supposed to give an answer from the back of the room. I extend my sincere thanks to Michelle and her team for their hard work and commitment to producing the best possible science assessment within the confines of our Ed Code. Her team has been generous with their time and have provided CSTA and other

565stakeholders with several opportunities to engage and provide input into the CAS development process. Our thanks to the department for adding our recommendations as well as those addressing the EPA. T&Cs as technical edits. So we are therefore pleased to support the department's recommendations for the blueprints with the technical edits, changes to fifth grade assessment, achievement level descriptors, and student reporting structure. I would also like to take this moment to express the need for additional assessment tools and resources such as interim and formative assessment tools in order to support teachers working with working to implement NGSS in all grades TK through 12. Implementing NGSS is a monumental lift and additional assessment supports developed and or vetted by the state and made available to all educators will help us realize the full vision of implementation of high quality

566science instruction. Thank you. Good afternoon. I'm Doug McRae, retired test specialist from Monterey. I wasn't planning to comment on this item, but the last slide got me. It promises something called post equated threshold scores, which is terminology I'm not familiar with. Does this terminology mean the threshold scores will be set using 2019 operational data with student score reporting delayed until fall of 2019? That's the right answer. Or does it mean the CDE intends to use the 2018 field test data, not suitable for standard setting, to generate preliminary scores? that would permit not yet valid student scores to be reported during the middle of testing. If the former, good. If the latter, that's not a good educational measurement practice. I sure hope my answer is the former. Thank you. Okay, we're now ready for a motion,

567I think. I will move CDE's recommendation. Do we do it in two parts, or do you want it together? Together, yeah. I would move CDE's recommendation. And I would second. Okay, any further board discussion? Board Member Ortiz-Lacombe. I do have a question as to why the three levels for the Spanish assessment. Let me address, when we... took the vote to say that we would explore how the Spanish assessment could be incorporated into the accountability system under ESSA. That vote is there. That's on record. And I understand that that conversation is still unfolding and it's still ongoing. And so... Not including it here doesn't mean that we're not still committed to that. And I want to make sure that that's understood because we did take a vote that it would be explored. Not that we're doing

568it, but that the possibility would be explored. But I vaguely remember that there was a reason why the three levels, but I can't remember why, to be honest right now. And I went back to my notes, and I can't find it. So if you could explain why three levels versus four, like the other assessments, that would be great. The number of levels is based on the number of test items that we have and the amount of precision that we think we can have given the blueprint before you given that we believe that three is the appropriate number of levels Smarter balanced is a is an adaptive test So it does have more levels than what you see before you in a recommendation for the Spanish So you don't have you don't have a lot of

569enough items to do four levels. You can only Yeah, it's a technical problem. Yeah, so we have a motion All those in favor raise your hands Okay, unanimous. Thank you. No Action item eight. I am still Michelle Center here before you for item eight So in this item, we ask the board to take three recommendations, hopefully one action, for you today. We ask that you approve preliminary summative ELPAC threshold scores for the overall scale score, the oral language composite score, the written language composite score. We ask that you approve the weighting to calculate the overall score. In other words, the percentage that we attribute to oral versus the written language composite scores. We also ask that you approve the LEA apportionment rates for the ELPAC. So the apportionment rates are included in the board item.

570The other two recommendations are included in attachment two. So our first request for your approval today is in regards to the preliminary scores, and you will note that I am using the word preliminary. So in order to prepare for the reporting of scores of the ELPAC annual testers that begin testing in February 1, 2018, we are asking for your approval of these preliminary threshold scores today. These scores are based on a standard setting that was conducted using field test data. We will be conducting a validation study of these results using the 2018 operational data. You may remember this diagram before you here. You saw this in the September 2018 or 2017 item where you approved the reporting hierarchy for the summative and the initial assessments. Based on that, there are four domains, the listening, speaking,

571reading, and writing. Those roll up to two composite scores you'll see here, oral and written, which also roll up to the overall scale score at the top tier. The standard setting plan includes methods used to recommend the threshold scores that are before you today, and we included information on that approach to standard setting in an information memorandum that you received in October. The next four slides provides the board with recommendations based upon that input from that standard setting activity as it was outlined in that information memorandum. Once approved, these preliminary threshold scores will be used in the reporting hierarchy and the student score reports that I showed you on that previous slide. The threshold scores you see today in this item have not been translated into a reporting scale. So once the board takes action

572on these preliminary threshold scores, we will place them on a reporting scale. In other words, these numbers that you see here today, you will never really see them again. They will be translated onto a reporting scale that is four digits long so that we don't have confusion between the SALT scale and the ELPAC scale. And we will also make sure that we try not to overlap the scores that are on Smarter Balanced so that there isn't confusion with those scale scores. So I just want to point that out because you're not going to see these numbers again. So to begin, we're asking the board to approve the composite weights. So I did take this in a little different order than what your item outlines because it seems to make a little more logical sense. So

573this first one is on the composite weights. In other words, what percentage oral versus the written language when you combine that into an overall score. You can see on this slide that we considered different approaches. Three different options were considered. And you'll see a commonality between how we approach the higher grades in the sense that they were all looked at as 50-50. But the difference is how we address kindergarten and grade one. whether or not we looked at a 90-10 split or a 70-30 split and so we looked at all of those and we actually presented information to our educators that were with us for standard setting and received their feedback. So the superintendent's recommendation is to use option two for the weighting of the overall score that is based on consensus feedback from the

574educators at the meeting. So what that option two is outlined here is kindergarten is 70% oral, 30% written, and then it moves to 50-50 starting in grade one. It's important to remember that this is the summative assessment. So this test is given later in the year than what CELTS is given now. And so it's an important thing to keep in mind when you look at that split and is one of the reasons why the educators specifically said they recommended that 70-30 versus the option one that you see before you there. So, the next tables that you will see use the option two, the recommended option, when conducting the calculations for the various slides. The CDE requests the approval of the recommended overall preliminary threshold scale scores for the summative ELPAC in this slide. As you

575can see, there are threshold scores for each of the seven grades or grade spans that are listed on this slide. Looking at the educator panel judgments for the proposed threshold scores, and those are in your attachment one, you'll note that the superintendent's recommendations closely align, but are not identical, to those recommendations of the panel. The superintendent's recommendation creates more consistency across the grades than the educator panel's recommendations. The table on this slide provides the recommendations for the oral language threshold scores and then the next one is the written and all of those numbers are in your board item. Attachment 4 contains the impact data that were considered when making the threshold score recommendations that you have. On pages 2 to 15 there are several bar graphs that display student performance on the CAS by ELPAC

576performance levels 1 through 4. Tables 1 through 7 represent the educator panel judgments, while 8 through 14 represent the superintendent's recommendations. These tables allow for comparisons of English-only student performance with that of level 4 on the ELPAC. The data show that students with higher ELPAC performance also have better ELA performance. The superintendent's recommendation made slight modifications to the panel judgments that resulted in the ELPAC level 4 performance being more closely aligned with how the English-only students performed. Now I'm going on to a different topic, the apportionment rate. So per education code, LEAs are allotted an apportionment annually for the administration of the ELPAC. We're recommending that the apportionment rate of $5 per pupil per assessment, just as we had for self-remain, are not shown here. So we're not yet done. Of course not. We have

577a lot of work still to do. So some of the activities to complete the development of the ELPAC are shown on this slide. So you'll see, if necessary, we will bring revised initial assessment blueprints to you in January. The summative assessment goes operational February 1, which is right around the corner. And also in February, we will be conducting standard setting for the initial assessment, with initial threshold scores being brought to you in March. Lastly, a summative threshold score validation study will be presented hopefully in summer of 2018. So the purpose of this study is to evaluate the degree to which the threshold scores and performance levels of the summative assessment accurately distinguish between the levels of students' English proficiency. The study will include educators with expertise in English language development who will evaluate the performance

578of their English learner students. These students will then take the ELPAC assessment. Statistical analysis will be completed to examine the relationship between the teacher's ratings of students' performance and the student's actual score on the ELPAC and the preliminary threshold scores that are before you today. Based on the results of that validation study, the department may bring forth additional recommendations to this board. So I think I did that pretty quick. That's the presentation. Thank you. Board member questions? Don't see any okay board member come a public comment that McRae and Martha Zaragoza Diaz Hi, good afternoon. I'm taking a great retired test specialist from Monterey the item addendum was posted last night at 6 p.m. Far too late for a reasonable vetting by local district personnel Sacramento advocates for the public Also, there's no compelling reason

579for the cut scores need to be approved today Only a false benefit of earlier release of not yet valid student results at the high risk that the invalid cut scores would come back to haunt interpretation of LCAP results for years to come. In a word, the vetting process had very limited appeal. No time to detail serious concerns with using field test data for standard setting verbally or via handout. In order to allow for reasonable time to vet this recommendation to ensure the validity of threshold scores, I'd ask the State Board table this item at least until the results of the validation study scheduled for early spring 2018 are available for review. The brief description of the validation study that can be done early spring doesn't have to wait for summer. Martha Zaragoza-Diaz, representing CABE and

580Californians Together. I would have to agree with Mr. McRae. Our members were very frustrated in light of the fact that they had very little time to thoroughly review the recommended threshold scores, the weighting, and the standard setting. I mean, this is a test that really has significant impact in our... English learners and it's a very important assessment and we have not been given an opportunity to thoroughly review this and we ask that this the board not decide or make any decisions on this and defer action until January or February of next year we do acknowledge and appreciate comparing the ELPAC levels to the CAASPP levels However, we have a recommendation with regards to the reclassification study. Staff is recommending that the reclass study only include field test data. We believe that that is not enough

581data to really ascertain how well the ELPAC is doing, and we recommend that in addition to the field test study, that also takes a look at the results from the first summative administration of the ELPAC. Thank you. Further board discussion Ortiz Lacombe Can you walk us a little bit through what stakeholders have been previewed to this information what reclassification study was alluded to only because I know that the item posted and You know, I was actually looking to see if we received any letters regarding this item. I don't remember seeing any letters, to be perfectly honest. And so I just want to get a sense of the process that has led us to this point. Thank you. Sure. I'm sorry. I'm not sure which study was. What was meant by the reclassification study, so I'm

582not familiar with a reclassification study in terms of the process we Had our panel. I think meet up through October 30th. I have that right October 30th and given that timeline we then need to take all the panel recommendations back we then met with our few of our technical advisors those that are more psychometricians from our technical advising group we did that on Halloween on October 31st and shared the data with them and received their their recommendations on what changes if any should be recommended that that may differ from the panel's recommendation I think that this process, well, of course, it would have been much better to have it up earlier and have it up at the same time as the item. It was not possible due to the timing of the panel meeting. I

583would also say that it's slightly different than maybe some of your other things that you see before you because it is already so influenced by educators, by the process that's used for the standard setting panel. So I would just point that. that out so board member's son So can you explain to me what were the implications of delaying a decision like this if we were to wait longer to have more feedback? What is the implication for the administration moving forward? So the administration is going to start on February 1st. That is when it will begin. A delay in action of these preliminary scores would most likely mean a delay in reporting of results. Okay, Board Member Rucker. Okay, so these recommendations from the panel didn't just come out of the blue. I know that you

584gave the date for when the panel actually met. Could you talk a little bit about the panel and the process that they went through to come? And that may be helpful to help us understand not only what the scores and the recommendation for the scores are based on, but who was actually in the room and involved in the conversation. Because I've sat in panel settings to look at standard setting activities, but they were for the academic test. not for the ELPAC. And so I think it would be helpful to understand exactly what the process was and who was involved. So I've asked Patty Barron to come up here. So Patty Barron is from ETS, and she's, I don't even know how many standard setting panels you have led for California, but numerous. And she also

585led the standard setting panel for ELPAC. So Patty, would you provide some details? Certainly, of course. Hello, I'm Patty Barron. I'm at the ETS office in Princeton, New Jersey, so I drove more than a few hours to get here today. And Michelle is correct. I've done more standard setting panels in California and across the world, actually, than I will want to count. This particular standard setting, Member Rucker, to answer your question directly, is one in which we followed protocol, which is you always want to have the educators who are working with the students, right? Let me ask you a favor. Sure. All right. So I'm from the South. And English is barely my second language. So if you could find a register of English that's a little bit plainer, that doesn't use big words like

586protocol and standard protocol to try to explain this process, that would really be helpful. Okay, point taken. Typically, we ask for educators who are familiar with the students or test takers. So in K-12, we talk about them as students. But in all standard setting you're looking for the the educators who know the test takers skills knowledge ability Expectations they know those people that are going to be taking the test So we recruited a particular type of educator to participate in this standard setting panel. That's correct So across the grades we selected across, California geographically and otherwise representative of California, the teachers who are working with the EL students. And they were from the grade spans and grades that would actually take the ELPAC assessment. That's correct. Keep going. Oh. Okay, well, could I ask you

587to tell me what question you want me to answer next? I want you to explain the process. After you did the recruiting and you brought all these educators who have this expertise about English learners who are teachers in the grades that would be tested on the ELPAC, you brought them into a room or you brought them to a location, and what did you actually ask them to do? Okay. So let me add to what I just said. Let me add to what I just said. We also, it was our desire and our expectation that the teachers in the room then, in the rooms, there were seven rooms, that those teachers were also familiar with the 2012 ELD standards to which the test is aligned. That's what I wanted to hear. So the recommendations for the

588standards, for the standard setting, is based on our state's ELD standards. That's correct. That's what I was hoping you would say. I'm glad you're happy so far. I'll keep going. So there were seven panels. We did three in one week, four in a different week, so that each grade or grade span was dealt with in its own room. And they went through, they took the test, which I put in quotes because they don't actually take the test. They actually saw the assessment items. So the teachers who participate, the educators who participate in this actually read and studied the assessment items. Yes, ma'am. And in studying the assessment items, what did you ask them to think about concerning the students who would take the test? That's right. So I can tell that you've seen standard settings.

589Yes, I have. Yes, I have. So we ask the teachers when they're taking the test. to think about what would be easy or hard for the students that they are there to talk about. And then we take them through the rest of the process now that they've sort of talked about the test and what might be hard or what might be easy. We take them through the method that is appropriate to the test. And in this case, there were two methods. So we had the reading and the listening multiple choice test, and we used a method that most of the members are probably familiar with called the bookmark method, where you take the items and you put them in order of difficulty empirically, and then the panelists are able to go through and just look

590at each item once. To say well, this is the beginning of level 2. This is the beginning of level 3 because based on our ELD standards based on the standards they are developing first after they Look through the test they look at the approved board general PLDs They look at the specific domain grade specific PLDs and they then develop a definition of that student who's just at the border And that's when they then are able to go into the bookmark process and place those stickers in the appropriate, in what they believe independently are the appropriate locations. Yeah, so what bookmarking basically does is based on our ELD standards, we're looking, the educators in the room actually studied an assessment item and identified. whether or not a student at this identified ELD level could answer the

591question correctly, could respond to the question correctly. And based on where they identified that answer would be no, that's where the cut point was recommended. So they stopped. Okay. So that's for reading and listening. Then for speaking and writing, we used a different methodology called the performance profile method because these are performance assessments. And we showed the panelists, the teachers in the room, a representation of a student at all of the score points, and in most cases, more than one student at the score points. It's a little bit harder to describe without pictures and, you know, little avatars. No, teachers who use rubrics or who give rubric models actually understand that. Okay, good. We're good. Keep going. Okay. So in the performance profile method, the job of the teacher or the panelist is simply to

592say, now that I've defined the beginning of level two, level three, level four, what score would I expect that borderline student to achieve? And so it's a very direct method for teachers. They're very familiar with thinking about, I have some students who are A, some students who are B, where's sort of the difference? And they're able to do that. And they did that by actually watching students who had been videotaped taking the speaking test and by actually looking at the writing performance of students who had taken the ELPAC writing domain. And then that was combined to create the oral and the written composite scores that Michelle has shown on the slide. And we then showed them what would be their tentative judgment on the overall. And we went through the whole process again because at

593this point, each one of them had an overall independent cut score that they then need to reconsider to finally say, I looked at all the domains and I'm now going to make a judgment about the overall. What might I want to change? And they look at impact data at that time to see what percentages would be falling into each level. Okay. And then, Michelle, could I ask you to go back to our chart, the chart for what it is you're actually recommending? So these scores then, these numbers, actually represent what? Based on the process, the two different processes that you described, these scores are a composite of... All four domains. Correct. Yes. Which were developed into oral and written composites and then, according to the weightings, developed these overall. Okay. And so the question, what

594is invisible here is for all four of those elements, how were they, how were these elements actually weighted? into the composite score. I think that that's what's being raised as a concern by the folk, particularly by Californians Together. They want to understand for each of the four elements or for each of the four domains that these composite scores represent. If they're weighted, they're saying they need to understand what the relative weights are and how is that actually reflected. in what these level two, three, and four scores mean. I think that's part of, and Martha, if I'm misinterpreting what you're asking, okay. I think that that's why they're asking for us to delay making a decision because they want to understand what these composite scores actually represent relative to the performance of the students in those

595four domains. So there is no weighting applied to the domains in order to calculate the composite. That weighting, if you want to call it, is natural based on the blueprint that was adopted by the board and the number of points that are associated with each of those domains. So this score is based on the composite score that we're seeing here is actually based on the blueprint that we have already previously adopted. Okay. Okay, Ortiz-Lacombe. I read the memo from October, and I just thought that the process had been very thorough. And I don't want to dismiss a process of educators and experts that came together to arrive at this point. How many educators, how many, I forget. I know that it was detailed. in the memo, but I also want to make sure that it's

596understood the comprehensive process that at least I read because I wasn't there. I think it would be a good idea for me to attend one of these things, but based on the way that it was described in the memo. It was a very comprehensive process, and I want to respect that. And I want to respect that it was practitioners and EL experts and people that have substantive knowledge. So do you know approximately how many folks came together to be part of this process? 70. About 70. 70. Okay. And as we look at the timeline and knowing that there are some concerns about input and stakeholder feedback, where can we include folks? And also maybe a conversation about reclassification study that was brought up that I don't know what that study is, but I think that

597it's worth us following up. But if you could go back to the timeline of next steps. Yes. I want us to be mindful that there are still some questions here that not everyone filled 100%. So if we could identify places where we could check in with folks, both practitioners, advocates, and other stakeholders. I looked over my shoulder to my colleague, Veronica, to see if she was able to recognize a study called the reclassification study. No, no. There is no such study. reclassification study that we have planned, so I'm not sure what that was referenced to, but we have spoken about the fact that once the operational assessment is in place, then we will have a validation study, and then we will look at how we will incorporate that into the reclassification criteria, because until we

598have the operational scores, we shouldn't be talking about the reclassification criteria. So, I mean, I think that the appropriate point in the timeline to continue to engage in the conversations is when we have that data from the validation study and determine whether or not it is recommending changes. And that is a good point in the timeline to have that conversation. Go ahead. I don't know how it happened. I'm sorry. Bill, the state board. I don't know what happened. Also, in this process, when do you anticipate looking or recommending the reclassification, the cut scores for reclassification? Because I think that is important for us. And I don't want to confound the two different topics here. If you look at the timeline that's in front of you. Actually, it's not. So after the assessment validation scores are

599done with the operational, that would come to the board. And we're thinking it would come to the board either September or November. And then after that, the reclassification criteria would come to the board. So either November or January of 2018. So then districts are going to have to rely on previous guidance that you all provided to reclassify students. And will you be sending out new notices? Because I know that there was something, a guidance that went out to the LEAs about using the CELT scores. So you're sending new notice to the LEAs, letting them know what is the protocol to reclassify until we have those cut scores. So the CELT score, the CELT is being given all the way up to January. And so that guidance still applies. I think part of why this item

600is here is so that they... the local education agencies would have some threshold scores, preliminary scores to use at the local level. There won't be a state reclassification criteria because it would be premature to do so. But at the local level, teachers know where their students are. And so having some threshold score that is being presented to you today for approval, it could be used as part of their criteria to reclassify. Now, this is what we have been doing as we transition. It's nothing different than what we did with CAASPP. when we transition from the old assessment to the new assessment. So this is the same thing with the English assessment. We're transitioning from one to the other, so we need to have that local flexibility until we have the new law also and the

601new criteria. Okay, I have sympathy with our speakers and our board members. I got this addendum at 6.08 Tuesday night, and so that's pretty late to get a reaction. So I hope this isn't happening again. My understanding is it's important to approve this item because... It will generate student score reports after the spring 2018 operational test. So this is another situation where we need to get on with it, even though the timing here is, you know, it's a very important and complex item to get at 6.08 on Tuesday night. And so I'm not happy about that and would like, you know, in the future to see if we can. You know have these items you know even the addendums earlier than than that time so Is there any further board discussion? I think we're ready

602for a motion and take board member son. Yes, so I would move forward the three Recommendations on this item too many words to say Three recommendations on this item from the department okay, if there's no further discussion and all those in favor raise your hands Okay. Now, we're on to the waivers. And so we have waivers 1 through 14 on consent. We're going to hear waiver 15. Before we move on, can we just confirm the vote? Were there any no votes or abstentions? It was unanimous? Thank you. So I would like a motion on consent waivers 1 through 14. Second. Any further board discussion? Any public comment on these waivers? Pardon me? Item 9 and item 12. This is W1 through 14. We might have speakers on waiver 15, which we're going to hear. Are

603there any speakers in the audience on waivers 1 through 14? No, we do not have any. Action of consolidated applications. Oh, it's an action item. It's not a waiver. Okay, but the motion is for waivers 1 through 14, all right? Okay, are there any speakers on that item? We have a speaker signed up for number 9, Doug McRae. No, no, W, I'm sorry. I'm looking at the wrong one. Sorry, new kid mistake. No, no, we don't. Okay, if there's no speakers, then we're ready to raise our hands and vote for this. Yes, okay. So I'm sorry, we're on to action item 15, which is the consolidated applications, the approval, and these the department will present. Regular consent. Okay, we need to do post-consent, sorry. Okay, those are items 9 through 14. That's on I move

604the consent items Are there any speakers on the proposed consent items 9 through 14? There are three okay? on item 12 Jill grace and on item 9 Doug McCrae Hello again, Jill Grace, President of CSTA. We are excited to support the list of reviewers and trainers recommended by the IQC and thank them for their thoughtful work. The critical role of the review and selection of high-quality instructional materials cannot be understated. It will have profound effects on successful implementation of the NGSS. With this in mind, CSTA provides the white paper, Priority Features of NGSS-Aligned Instructional Materials, with the intent that this be used as a resource for both IQC members and CDE staff who will be engaged in training appointed reviewers. This is intended to be a resource to aid in the selection of the highest

605possible quality instructional materials to support implementation. This should be used to enhance, not usurp, the guidance in our state framework. As you heard from WestEd earlier, they found that instructional materials from recent adoptions did not always meet teachers' needs. This parallels concerns we've heard from our members and colleagues, many of whom who've been involved in local adoption and screening processes with early NGSS materials. Understanding from this is what motivated us to collaborate with other West Coast NGSS states of Nevada, Oregon, and Washington to develop these recommendations. We thank you for your consideration of available tools that facilitate... review of materials for alignment and for listening to the priorities of educators to inform policies and programs that advance successful implementation of the NGSS. Thank you. Good evening. I'm Doug McCray, retired district guy from Monterey. This

606is on the ETS withholding of $170,000. And I have just really one question for it. I did any of the cited I got take the classes did any of the cited deficiencies in the agenda materials contribute to the delay in releasing statewide test results from August 29 to September 27 we'd have an answer to that I'd appreciate it okay we have ready for motion on proposed consent okay moved it sorry okay Okay, we need a vote. All right, all of them say vote. Thank you. Action item 15. Good evening, President Kerr, Superintendent Torlickson, and members of the board. My name is Jerry Winkler. I'm the Educational Data Management Division Director, and I'm here to present item 15 in the action item. The CDE is recommending regular approval for the CONEP for 14 LEAs on attachment

6071. conditional approval of two LEAs on attachment two. I also wanted to let you know that Superintendent Greenberg from Yosemite Unified School District is here in case there are questions. Yosemite is one of the conditional approvals on attachment two. That's our recommendation. And the conditions? Conditional approval? Basically that they have Yosemite has eight outstanding items on their federal program monitoring review and Stone Corral has, they're down to three items. So they're moving along quickly in getting these items resolved. So, and I think the items were listed in the item 15. Many of them have been resolved if you if you look item One and two for Yosemite were resolved in the last few days We've got three which is CEO seven eight Seventeen that are there that they're still working on right now and also

60826 and 27 and then EL 07 15 and 16 for Yosemite Stone Corral had EL07, 11, and 16 as well that they're still working on for their FPM. Any other board member questions? I understand that the motion then gives some time for them to bring things in order. I think the superintendents knew. I've visited Mariposa County, and they've had a lot of staff turnover, very difficult to fill all their various positions, but the new superintendent's working hard at it. Correct. And the new superintendent's been making progress significantly. Yeah, and Superintendent Greenberg is here in the room if there are questions. Okay. All right, is there any other board member questions on this? If not, I'm going to have a motion. Move the CDE recommendation. Any public comment? No public. All those in favor, raise your

609hand. Okay, action item 16, California education for a global economy. Did I say you saved the best for last? No, I'm just kidding. This is like a marathon, so all of you are marathon runners. Just from the second floor. Good afternoon, or actually evening now, President Chris, Superintendent Torrickson, and team members of the board. I'm Veronica Aguilar, Director of the English Learner Support Division. Here to provide you an update on the rulemaking process for the California Education for a Global Economy or California Edge Initiative, also known as Proposition 58. I do have a PowerPoint, but I'm trying to see if it's in here as item 16. I know where. No, can you find it? Or it's not? I have the flash drive. Okay. Here we go. So the proposed regulations are intended to provide clarity

610and direction to local education agencies for the purpose of complying with the provisions of the California Edge Initiative. The regulations are designed to guide local education agencies and school sites towards a process that incorporates parents, stakeholders into the process of establishing language acquisition programs. As you may recall, over... 70% of California voters approved Prop 58 on November 8th of 2016. The amended statutes became effective July 1st of 2017. During the July meeting, the State Board of Education approved the commencement of the rulemaking process and opened a 45-day public comment period. for members of the public to provide comments on the proposed regulations. The public comment period began on July 29 and ended September 11. During this time, we received 20 public comments. and 12 of which came in the form of a form letter. During

611the hearing, we received one public comment. After review and analysis of all public comments, the California Department of Education revised the proposed regulations and responded to the comments. In attachment four, you can see the summary of all those comments and the actions that were taken to incorporate all pertinent comments. Now, given the time, I'm not going to go over every single... amendment to the regulations, but only highlight a few. So on this slide, you can see that section 11300, 11301, 11309, 11310, and 11311 had some amendments, but they were relatively minor, and you could actually see them in bold and underlined in the regulations. One of the things I would like to highlight is the fact that multilingual was added to the title. To match the language of the proposition and to me that's significant

612also because we added a definition We also added two other definitions Or revised them to include the language of the proposition which includes transitional bilingual dual language programs and developmental programs as well as the structure English immersions as part of the language programs. We also did a strike through the structure English immersion as it was already defined in education code. So other significant areas would be the fact that we changed the wording of a change of to give guidance to the local education agencies to instead of immediately within 10 days there was comments that stated that what this immediately means. The other significant comment would be change 90 calendar days to 60 days of reaching the threshold to determine whether it's possible to implement the requested language acquisition program. There was also another comment about

613an appeal process. That was not added to the regulations, but I know you would have a question on it. And the reason it was not added was because the law would have to be amended for us to be able to include an appeal process. However, we were able to include that the district would need to provide in written form an explanation of the reasons the program cannot be provided. And that would give the parent the ability to do what we typically do as parents if we have some dissatisfaction with something in the process and go forward with the avenues that are present currently. The regulation timelines is on are presented on this slide upon approval of the commencement of the 15-day public comment period Members of the public may submit written comments to the address

614on the slide to the email address on the slide. This will allow the public to provide input on the amendments in the proposed regulations. The public may comment on the amendments which are identified in the regulation in bold and underlined or bold and strikethrough. These are the only sections open for public comment. All public comments are reviewed and taken into consideration during the revision of the proposed regulations. If there's substantive amendments are made to the proposed regulations, At the end of the 15-day comment period, we would come back to the board in January. If there are no substantive revisions, then the regulations would be deemed adopted by the board, and then the California Department of Education would submit to the Office of Administrative Law for approval, and the regulations would become effective July 1st of

6152018. I did want to address once the regulations do become effective, what next? And so we do have an implementation plan, and also we are aware that there's a need, as a courtesy to the public, to translate the regulations into at least the top language. Eighty-three percent of all English learners are Spanish-speaking. The second largest is Vietnamese at two percent. So currently we do have them translated on our website and the revised version will also be updated for the public comment period. As far as the implementation plan, we're planning to include the once the regulations are adopted and after July 1st of 2018, having some English Learner Roadmap trainings in the North, Central, and South, and incorporating the regulations training as part of those trainings. Our last English Learner Roadmap, the members of that committee

616actually volunteered to continue to assist. And so we're very fortunate, the CTA, AXA, CESA, CSBA, CCEE, we're all part of that group. were willing to support us in rolling out that information. We're also planning on, of course, using our regular networks, Bilingual Coordinators Network, Title III, other conferences, webinars, to disseminate the information. The motion upon you is the CDE requests the following action of the state board members to approve the commencement of the 15-day commencement period, designating a form of public comment period so members of the public may provide comments on the proposed amendments. And if there are no substantive amendments after the 15-day comment period, deem the proposed regulations adopted by the state board of education and direct the CDE staff to submit the regulations to the Office of Administrative Law. Before opening it

617up for questions, I would like to recognize the lead education program assistant who worked tirelessly on these regulations, and that would be Lori Kelling, and also the administrator, Elena Fajardo, who are sitting on the second row. And from the legal office, Patricia Alverson, Marsha Bedwell, Andrea Christensen, and the whole team there at the legal office. They are all to be commended for their work on these regulations. Thank you. Thank you. Board Member comments? Board Member Martiz-Lacombe. Thank you. Thank you, Veronica. I want to say, I want to thank those who submitted written comment because I think that they really, and the department for taking some of those ideas and that feedback and really strengthening our regulations here for EDGE. I know that we are looking to approve this to then post them in Spanish as

618well, and I think that's very fitting. given that this is about multilingualism. And so I understand why that was a concern initially expressed. For this item, the only thing that was still, for me, not resolved was something that I shared with you, and I know it's beyond our scope or our power, which is the appeal process. And I just want to emphasize that because I think that we did the best that we could by putting something in writing that parents can then take to their governance board. But I do think that, you know, we want to make sure. To acknowledge that, we also struggle with the fact that there was not something that gave us the power to put an appeal process into the regulations. So that, we heard it in some of the public

619comment, I'm sorry, the written comment that came in, is something that we also problematized. We just simply could not include it. Thank you. Make a quick comment. Sure. Just congratulate Veronica and the CDE team. The voters, I think, are going to be happy. The legislature, the governor, everybody should be happy that this moved so swiftly from a ballot box measure to reality. Thanks for that diligence. And it is phenomenal. Again, 73% of the voters in California embrace this measure, want to see bilingual education, want to see our students get a chance to be fully. Part of the global economy. Is there any public comment on this item? We do have one public speaker for item 16, Michael Flores Castaneda. We seem to have another public speaker that wants to. I'm over too. Martha Zaragoza-Diaz representing

620CABE and Californians Together. And I just want to thank CDE staff and state board for the opportunity to provide comments on these proposed regulations. I also want to thank everyone for the translation of the proposed regs and the revised regs in Spanish. I know that was done voluntarily. It's not required. So we really are appreciative, very appreciative of that. And we support the staff recommendation to go ahead and move forward with these revised regulations so they can go through the remaining administrative review process. Thank you. It's just faster than me this time. It won't happen again. Good evening. Michael Flores Castaneda with the California Teachers Association. And it was a year ago today that we celebrated the passage of Prop 58. I'm presenting with you on behalf of the CTA and our Prop 58 work

621group a letter that... addresses two areas of concern that weren't addressed in the CDE's latest revision. The first pertains to the amount of time schools must retain their written requests for language acquisition programs. The committee believes that the language should be changed from three to five years, keeping it in line with the existing timeframe for record retention for federally funded education programs such as Title IX. And the second area of concern is also regarding the parental requests, but more specifically the timeline for response. The current language reflects a change from 90 to 60 days. However, the CTA 58 work group believes that it should be further reduced to 45 days. 60 days is exceedingly long, especially when a school calendar year is only 180 days. And LEAs should be able to determine whether or not

622they could have such programs within the suggested 45 days and be able to respond in that timeline. And I'll just leave the letter here. Thank you. Okay, that concludes the public comment. We actually do have one other speaker who would like to address the board. Okay. After this item. What? I couldn't hear you. No more speakers. No more speakers. After this item. After this item. Okay. All right. We need a motion and go ahead. Motion to move the department's recommendation. Second. Seconded by Board Member Sun. Okay, no further board discussion. All those in favor, raise your hands. Okay, now for item 11, we have David Villarino-Gonzalez. Thank you very much. My name is David Villarino-Gonzalez. I'm the president, chief executive officer of the Farmworker Institute of Education and Leadership Development. It's a nonprofit organization founded

623by Cesar Chavez in 1978. And I've been the president and CEO since year 2000. On item number 11, the CTE grant for career pathway guidance, I just wanted to thank the Department of Education for all their hard work. This grant, $1.3 million, will go towards taking our ESL students. We've actually put through about 55,000 low-wage, low-skill farm workers in the state of California into literacy and ESL programs. We now have a high school charter school. We enroll about 400 adults from 18 to probably 65. And we are now bridging them to own their own business as worker-owned cooperatives. We have determined that this is the only way that we can really change some of the economic realities in rural communities, not just by training, not just by education, but giving in their own businesses to

624own. And this is a transition to do that. We wanted to thank you very, very much from the bottom of my heart. Okay, we're going to move State Board priorities until tomorrow. And in order to not have everybody seated, we're going to have a huge crowd, and then ask them to leave, and then we go into closed session. I'm going to read the following statement. I would like to announce that tomorrow morning at 8.30, the State Board will be going into closed session to discuss and or take action on the following litigation. Kapim versus Torlakson at all. The State Board of Education meeting will now be continued until 8.30 a.m. tomorrow morning. The first item will be the closed session discussion. So we are now to leave, but we're continuing tomorrow at 8.30.