Step 0: Remember to Use a Valid Measure of Student Achievement (Turn Your Teaching Into Publications #6)

Five years ago my school failed our regional accreditation visit for the gen ed criterion. “Noncompliant” was the official ruling. This didn’t come as much of a surprise, since we had slapped the institutional assessment together in about six months. (Our most recent assessment, by comparison, has spanned five years including 10 months of just planning.)

To give an idea of one of the mistakes we made the first time around, let me tell you about our assessment of the social sciences objective—an objective that asks that students analyze human behavior, experience, history, and so on. The key word here is “analyze.” Analysis is a skill, and like any skill, it can be shown in greater or lesser degrees. 

Painting is a skill. On the low end, it involved dunking a brush into paint and making marks on a wall or canvas. On the highest end, master painters create masterpieces or, for interior design, cover all walls, trim, and ceiling without streaks or splatter on the floor.

Bitmoji graphic

But our social sciences assessment included a whole bunch of items that had nothing to do with analysis. E.g., “Identify which of the following is not one of Erikson’s psychosocial stages of development.” Being able to recognize which words belong to which psychologist doesn’t require anything close to analysis. If anything, it requires a bit of memory. 100 correctly answered memory questions don't get us anywhere close to assessing a student’s analytic skills.

Analysis requires breaking something down into is smallest essential units. A historian can read a historical document and analyze, among other features, the dialect of language used, social power dynamics, ulterior motives, current events, beliefs and values, racism or classism, and so on. Each discipline has its own many ways of analyzing information.

Now, consider the following questions as assessment items for analysis, keeping in mind that analysis requires that you break something (a phrase, a behavior, an event, a document, etc.) down into its most basic essential units or parts and understand how they fit together. Decide which best exemplifies analysis:

1.     Define “behavioral analysis.”

2.     Give an example of “behavior analysis” from your own life.

3.     Thomas is a 2^nd grade student. He does not interact with his classmates or make eye-contact with the instructor. He hasn’t said more than his name during the entire school year. Based on this information, does Thomas satisfy the diagnostic criteria for Autism Spectrum Disorder (ASD)?

Now, I can imagine each of these questions being used to test student analytic skills, but the answer is "None of the above." None of the options I've given ask a student to demonstrate their analytic skills. 

Question 1 asks for a memorized definition. Question 2 asks for students to apply the concept “behavior analysis” to their own lives by giving an example. Question 3 sounds like analysis, but what it asks is for students to demonstrate their understanding of ASD diagnostic criteria by using them to interpret a novel case. All of these fall short of analysis, and therefore use of any of these questions would be meaningless in assessing student analytic skills.

But we can create an analysis question out of option 2 or 3. Instead of giving an example in question 2, a student can be asked to use behavior analysis to make a plan for behavioral change, such as to stop procrastinating. In this question, the student would have to understand the components of a habit and how behavioral change occurs (existing knowledge), and then use that information to analyze a real often problematic behavior (procrastination). That’s asking a whole lot, of course, but analysis is a demanding cognitive skill.

Next up is Step 1: Choosing a Learning Goal You'd Like to Support