Setting Description
The setting for this research is small project-based middle school in suburban San Diego. The school serves approximately 320 students in grades 6-8. Students are admitted via a zip-code lottery system that seeks to equitably represent the surrounding community. The student population therefore is ethnically and socioeconomically diverse. Approximately 54 percent of students are categorized as socioeconomically disadvantaged. Nineteen percent are English Language Learners and nine percent have disabilities (see Figure 1).
Figure 1
Student Population by Ethnicity
Figure 1: Percentage of the total population of the school categorized by ethnicity as reported in the schools internal records.
Students at the school are assessed in several different ways. In grades 6 and 8 students receive grades for each of their courses. The 7th grade team has decided to not use grades and instead uses a system of written feedback and reporting to students and families. In addition, all students participate in Student-led Conferences where they reflect on and share their learning with their families. Students also engage in Presentations of Learning during which they formally present their learning to teachers and peers, and in exhibitions for many of their projects. These exhibitions may be a part of the whole school event in March or they may be organized by individual teachers throughout the year.
Students also were previously assessed in accordance with California state’s standardized testing program, known as STAR. This test measures students knowledge and skills in alignment with the California State Standards in English Language Arts, Mathematics, Science and Social Studies. In the 2013-2014 school year, students participated in field testing for SmarterBalaced, the next generation assessment for the Common Core State Standards. In 2014-2015 students will take the Smarter Balanced assessment in grades 6-8.
Each year students participate in a student survey called Youth Truth. Youth Truth was designed by the Center for Effective Philanthropy (CEP), in collaboration with the Bill & Melinda Gates Foundation and is designed to gather feedback directly from students about their experiences in school. In addition several teachers also give their own personalized surveys to students to gather feedback throughout the year. These actions reflect the emphasis on personalized learning which is a key design principle at HTMCV. Teachers continuously and informally gather feedback and ideas from their students at the classroom level. In addition, HTMCV has a student senate which provides student leadership and input at the school level.
Methods
This project focused on the topic of including student voice in answering the question of how we can effectively measure schools. The literature review and a study of current models for accountability revealed that determining how best to measure school is a complex and multifaceted endeavor. It is clear, however, that for accountability systems to be most effective, they must be developed locally (Darling-Hammond et al., 2014). This project sought to build on that principle by including students among the local actors influencing the design of accountability systems. This project examined the question of what metrics students think should be included when measuring the effectiveness of a project-based middle school. The assumption that underlies my research question is that students can provide valuable insight into the complicated conversation about what elements of schools we should measure to determine their success.
Given the complexity of the question of how we should measure schools, the research for this project consisted of an iterative process of gathering data from students and co-analyzing it with 6th-8th grade student co-researchers. This approached both allowed me to unpack the data from students with other students to assist in the interpretation, and it also supported students in deepening their understanding of the topic. The table below outlines the overall progression of data collection and interpretation throughout the research cycle. The narrative that follows provides additional details about the process of data collection and analysis.
Table 1
Research Process
Research Step
Description
Focus Group #1
Initial Focus Group
One of my goals for this project was to authentically include student voice in each step of the data collection and analysis. I also recognized, however, that even though students could share important insights about what is most important to measure in schools, school accountability frameworks were not something with which students would be immediately familiar. The first step, therefore, was to determine how to frame the conversation to gather the most useful data. I began this process by meeting with a focus group of approximately eighteen students from the middle school. In this session, I explained the project to students and then asked them to share their ideas about what elements would be most important to making an amazing school.
This first focus group was instrumental in forming my next steps in several ways. First, it provided me with practice in framing the project to students. It also illuminated several themes of what was most important to students. These themes formed the foundation of my research. My experience also helped me see the importance of creating a culture of trust that would allow students to share their ideas freely. I did not take this step in my initial focus group and I think the conversation stalled due to this oversight. Based on my experience in this focus group, I decided to switch my approach from holding focus groups with different students to developing a cohort of co-researchers who could feel safe and supportive by one another and also develop a depth of understanding about this process. I also included a community-building introduction into each of my subsequent focus groups with students.
Interviews with Students
After my initial focus group, I decided to begin to develop a cohort of student co-researchers. The first step in this process was conducting individual and small group interviews with students. My goals for these interviews were to develop student interest in and understanding of this project while also gathering data about what students feel is important to measure in schools. I hoped also that interviewing students in small groups or individually would allow them to be more comfortable sharing their ideas. In addition, I decided to use this format both to gain a deeper understanding of students’ perspectives about what we criteria were most important to them in determining the quality of a school.
I interviewed a diverse group of students to gather a wide variety of perspectives. The students volunteered after I made an announcement during lunch. I wanted to open the process to all students and then narrow in on the students who were the most excited about and invested in this project. My hope was that the students I interviewed would become leaders of this project and demonstrate ownership of the research process moving forward. I interviewed four sixth grade students, one seventh grade student, and two eighth grade students in these initial interviews. Three of these students were female and four were male. The students also represented a wide range of ethnicities that roughly matched the demographics of the student body. In addition to middle school students, I also interviewed one high school senior to compare a late high school perspective with the middle school students’ ideas.
In the interviews, I first asked students what was most important to them about their school experience and what they thought we should measure to evaluate schools. In addition, to broaden their perspective, I shared with students a summary of the annual student experience survey data from our school. I had analyzed this data to help me understand, overall, how students experienced school and what was important to them. I shared this data with the students in my interviews and asked them to suggest, based on that the data, what was most important to students about their school experience and what we should use to measure schools.
After these interviews, I coded the responses from students about what they thought was most important to determining if a school was high quality into the following categories: relationships, students enjoying school, how teachers teach, fair rules, voice for students, doing work that is important, and college and career readiness. These categories emerged as themes through both my analysis of YouthTruth data, my initial focus group, and these student interviews.
Gathering and Interpreting Survey Data
Another theme that emerged from these interviews was the importance of including perspectives from many other students in this process. The students I interviewed were excited to gather more information from their peers. As a result of my interviews with students, I decided to develop a survey that students could use to gather data from other students. I used the categories I had used to code the interviews as the categories for the survey (see Appendix B) I created. I wanted to continue to include student voice, not just in the survey creation, but also in the interpretation of data. I set up a system where I met regularly with a small group of student co-researchers every Thursday during lunch to discuss the project. These meetings were voluntary and open to all students, so the group of students who attended varied, but a core group of the five most committed students joined me each week. The group of students was a diverse group in terms of gender, grade, and ethnicity. This was important to the project to ensure I included a wide range of perspectives.
In the second focus group, the first with the group of student co-researchers, I began by building background knowledge about school accountability and explaining the premise of the project. I then shared a summary of my interviews with students with the group as well as a draft of the survey. I asked students to critique the survey based on whether or not it was aligned to the elements that contribute to a high-quality school that students had described in their interviews. I also asked students to review the survey to ensure it would be accessible to their peers. After a few revisions based on student feedback, I finalized the survey and the students in our group distributed them to their peers to gather data.
In the next two sessions, we analyzed and interpreted the raw data from the student surveys. In each session, I shared the data with the students and asked them to look for themes and big ideas. After they had an opportunity to form their own analysis of the data, students shared out those big ideas and we had a conversation about the meaning of the data. For each of those sessions, I again coded the big ideas, grouping them around similar themes. I did the same process for the open-ended questions that students answered in the survey. I used the coding system to create a summary of survey data. This summary along with notes from the focus groups will be the foundation that my student co-researchers and I use to determine the primary message we share about how students think we should measure schools.
The setting for this research is small project-based middle school in suburban San Diego. The school serves approximately 320 students in grades 6-8. Students are admitted via a zip-code lottery system that seeks to equitably represent the surrounding community. The student population therefore is ethnically and socioeconomically diverse. Approximately 54 percent of students are categorized as socioeconomically disadvantaged. Nineteen percent are English Language Learners and nine percent have disabilities (see Figure 1).
Figure 1
Student Population by Ethnicity
Figure 1: Percentage of the total population of the school categorized by ethnicity as reported in the schools internal records.
Students at the school are assessed in several different ways. In grades 6 and 8 students receive grades for each of their courses. The 7th grade team has decided to not use grades and instead uses a system of written feedback and reporting to students and families. In addition, all students participate in Student-led Conferences where they reflect on and share their learning with their families. Students also engage in Presentations of Learning during which they formally present their learning to teachers and peers, and in exhibitions for many of their projects. These exhibitions may be a part of the whole school event in March or they may be organized by individual teachers throughout the year.
Students also were previously assessed in accordance with California state’s standardized testing program, known as STAR. This test measures students knowledge and skills in alignment with the California State Standards in English Language Arts, Mathematics, Science and Social Studies. In the 2013-2014 school year, students participated in field testing for SmarterBalaced, the next generation assessment for the Common Core State Standards. In 2014-2015 students will take the Smarter Balanced assessment in grades 6-8.
Each year students participate in a student survey called Youth Truth. Youth Truth was designed by the Center for Effective Philanthropy (CEP), in collaboration with the Bill & Melinda Gates Foundation and is designed to gather feedback directly from students about their experiences in school. In addition several teachers also give their own personalized surveys to students to gather feedback throughout the year. These actions reflect the emphasis on personalized learning which is a key design principle at HTMCV. Teachers continuously and informally gather feedback and ideas from their students at the classroom level. In addition, HTMCV has a student senate which provides student leadership and input at the school level.
Methods
This project focused on the topic of including student voice in answering the question of how we can effectively measure schools. The literature review and a study of current models for accountability revealed that determining how best to measure school is a complex and multifaceted endeavor. It is clear, however, that for accountability systems to be most effective, they must be developed locally (Darling-Hammond et al., 2014). This project sought to build on that principle by including students among the local actors influencing the design of accountability systems. This project examined the question of what metrics students think should be included when measuring the effectiveness of a project-based middle school. The assumption that underlies my research question is that students can provide valuable insight into the complicated conversation about what elements of schools we should measure to determine their success.
Given the complexity of the question of how we should measure schools, the research for this project consisted of an iterative process of gathering data from students and co-analyzing it with 6th-8th grade student co-researchers. This approached both allowed me to unpack the data from students with other students to assist in the interpretation, and it also supported students in deepening their understanding of the topic. The table below outlines the overall progression of data collection and interpretation throughout the research cycle. The narrative that follows provides additional details about the process of data collection and analysis.
Table 1
Research Process
Research Step
Description
Focus Group #1
- Piloted questions and framing of the topic
- Gathered initial data on what students think we should measure about schools
- Collected data to inform development of survey for students.
- Built student co-research understanding of school accountability process and ultimate goal of this project
- Reviewed summary of data collected from student interviews
- Critiqued survey developed based on student interviews and first focus group
- Distributed survey for students to collect data from their peers
- Reviewed and interpreted first round of raw data collected from student survey
- Began to discuss ideas to share with policymakers about school accountability
- Reviewed and interpreted second round of raw data collected from student survey
- Continued the discussion about ideas to share with policymakers about school accountability
- Reviewed summary of data from student survey
- Created prototype of podcast about what students think we should use to measure schools
Initial Focus Group
One of my goals for this project was to authentically include student voice in each step of the data collection and analysis. I also recognized, however, that even though students could share important insights about what is most important to measure in schools, school accountability frameworks were not something with which students would be immediately familiar. The first step, therefore, was to determine how to frame the conversation to gather the most useful data. I began this process by meeting with a focus group of approximately eighteen students from the middle school. In this session, I explained the project to students and then asked them to share their ideas about what elements would be most important to making an amazing school.
This first focus group was instrumental in forming my next steps in several ways. First, it provided me with practice in framing the project to students. It also illuminated several themes of what was most important to students. These themes formed the foundation of my research. My experience also helped me see the importance of creating a culture of trust that would allow students to share their ideas freely. I did not take this step in my initial focus group and I think the conversation stalled due to this oversight. Based on my experience in this focus group, I decided to switch my approach from holding focus groups with different students to developing a cohort of co-researchers who could feel safe and supportive by one another and also develop a depth of understanding about this process. I also included a community-building introduction into each of my subsequent focus groups with students.
Interviews with Students
After my initial focus group, I decided to begin to develop a cohort of student co-researchers. The first step in this process was conducting individual and small group interviews with students. My goals for these interviews were to develop student interest in and understanding of this project while also gathering data about what students feel is important to measure in schools. I hoped also that interviewing students in small groups or individually would allow them to be more comfortable sharing their ideas. In addition, I decided to use this format both to gain a deeper understanding of students’ perspectives about what we criteria were most important to them in determining the quality of a school.
I interviewed a diverse group of students to gather a wide variety of perspectives. The students volunteered after I made an announcement during lunch. I wanted to open the process to all students and then narrow in on the students who were the most excited about and invested in this project. My hope was that the students I interviewed would become leaders of this project and demonstrate ownership of the research process moving forward. I interviewed four sixth grade students, one seventh grade student, and two eighth grade students in these initial interviews. Three of these students were female and four were male. The students also represented a wide range of ethnicities that roughly matched the demographics of the student body. In addition to middle school students, I also interviewed one high school senior to compare a late high school perspective with the middle school students’ ideas.
In the interviews, I first asked students what was most important to them about their school experience and what they thought we should measure to evaluate schools. In addition, to broaden their perspective, I shared with students a summary of the annual student experience survey data from our school. I had analyzed this data to help me understand, overall, how students experienced school and what was important to them. I shared this data with the students in my interviews and asked them to suggest, based on that the data, what was most important to students about their school experience and what we should use to measure schools.
After these interviews, I coded the responses from students about what they thought was most important to determining if a school was high quality into the following categories: relationships, students enjoying school, how teachers teach, fair rules, voice for students, doing work that is important, and college and career readiness. These categories emerged as themes through both my analysis of YouthTruth data, my initial focus group, and these student interviews.
Gathering and Interpreting Survey Data
Another theme that emerged from these interviews was the importance of including perspectives from many other students in this process. The students I interviewed were excited to gather more information from their peers. As a result of my interviews with students, I decided to develop a survey that students could use to gather data from other students. I used the categories I had used to code the interviews as the categories for the survey (see Appendix B) I created. I wanted to continue to include student voice, not just in the survey creation, but also in the interpretation of data. I set up a system where I met regularly with a small group of student co-researchers every Thursday during lunch to discuss the project. These meetings were voluntary and open to all students, so the group of students who attended varied, but a core group of the five most committed students joined me each week. The group of students was a diverse group in terms of gender, grade, and ethnicity. This was important to the project to ensure I included a wide range of perspectives.
In the second focus group, the first with the group of student co-researchers, I began by building background knowledge about school accountability and explaining the premise of the project. I then shared a summary of my interviews with students with the group as well as a draft of the survey. I asked students to critique the survey based on whether or not it was aligned to the elements that contribute to a high-quality school that students had described in their interviews. I also asked students to review the survey to ensure it would be accessible to their peers. After a few revisions based on student feedback, I finalized the survey and the students in our group distributed them to their peers to gather data.
In the next two sessions, we analyzed and interpreted the raw data from the student surveys. In each session, I shared the data with the students and asked them to look for themes and big ideas. After they had an opportunity to form their own analysis of the data, students shared out those big ideas and we had a conversation about the meaning of the data. For each of those sessions, I again coded the big ideas, grouping them around similar themes. I did the same process for the open-ended questions that students answered in the survey. I used the coding system to create a summary of survey data. This summary along with notes from the focus groups will be the foundation that my student co-researchers and I use to determine the primary message we share about how students think we should measure schools.