Building the bridge between assessment and learning

Teachers are reflexive designers who continually manage multiple layers of accountability. We weave together ideas from the syllabus, school priorities, knowledge of our students, and our professional and personal strengths. These choices and decisions are woven together into summative assessment tasks. Often this work occurs late at night or on weekends, without the opportunity to discuss the ideas with colleagues.

As the designers, we should be ready to test and show that our decisions are trustworthy and valid.  This is what we mean by arguing for the validity of our assessment designs. At each juncture, we want to check the strength of the connection and make an argument for why we think our decisions have merit.

 We expect students to check their progress regularly with formative assessments, before they attempt something a little riskier like their summative assessment. We are proposing five progress checks, or inquiries into validity that teaching teams can take that will enhance the quality and equity of their assessment designs.

Tying it all together – Five inquiries into the validity of assessment designs

1. Think creatively about the purposes of assessment and how you can weave them together.
2. Develop a shared language of assessment so you can have the big conversations.
3. Know your student aspirations, their strengths and their struggles. Consider ways to build on what they know and remove barriers to their participation. Know the common risks to validity and having strategies to address these risks.
4. Make the often invisible assessment design work visible and open to scrutiny by creating an argument for the validity of the design
5. Design in stages of peer feedback to enhance intelligent accountability.

Why focus on local validity?

Validity is the overarching assessment principle that is strongly associated with the fairness of assessment. It is most commonly defined as the extent to which the inferences that are made about the outcomes from assessment tasks can be justified (Newton & Shaw, 2013). Validity is evident when assessment designs and decisions are trusted, and the assessment fits the purpose for which it was designed (Stobart, 2009).

As those who are able to observe most closely the impact of assessment outcomes, teachers need to be included as key agents in discussions about the validity of assessment (Sellar, Rutkowski, & Thompson 2017; Wiliam, 2014; Moss, 2016). Yet as validity is often written about in technical terms, teachers can feel like it is not something they have control over.

This proposed system for the validity of local assessment is designed to support a vision for assessment that promotes student equity and the agency of teachers and students.

Often validity is spoken about like it is an ideal or perfect state, and assessment designs seek to avoid risks to that validity. One of the problems is that when validity is described in deficit terms, teachers and systems will inevitably fall short of achieving that perfection. Perfect validity is not a realistic target as validity often relies on authentic and meaningful tasks that may not be easily replicated and reliability is achieved through replication. These two principles are often in tension, and like a see-saw are not able to be fulfilled to the same degree at the same time (Harlen, 2013). Also, our expectations about the validity of assessment ideally will change. As we become more innovative in assessment designs, and achieve more equitable outcomes with students, our ideas of good quality assessment processes and outcomes will also change and develop. As an ever-changing concept, validity is therefore a great topic for teacher inquiry.

This project explored validity as a dynamic process. The extent to which an assessment design is valid can be discussed when the inferences inherent in the design are made more visible. Responsive teaching means that adjustments to the formative assessment and pedagogic designs are made along the way to maintain fidelity to the overall assessment intent, and these actions can be considered and justified through peer reflection. Teachers are able to see how the lives of students are impacted by assessment inferences and decisions and take action in response to address issues of equity. This has particular significance for Queensland teachers who take responsibility for designing and grading most of the summative assessment tasks. Each design decision will shape the outcome for students and is worth talking about.

An argument for the validity of teacher assessment can be established through a practical process of inquiry into the coherence, completeness and assumptions that are inferred at each stage of the assessment process (Kane, 2016).  The following proposal for a system of local, intelligent accountability builds on the ideas of Wyatt-Smith & Klenowski (2014) who advocate for system level support for teachers to engage in local inquiries and Crooks, Kane & Cohen (1996) where validity is represented as testing the strength of the links in the chain of assessment design decisions.