Using ALPaCA grading for course assessment

Course assessment is a component of degree level assessment in our department and has typically been accomplished using student surveys and evaluation by peer instructors. While both of these tools have merit, we find them ultimately unsatisfactory since they do not probe the effectiveness of the instructor at moving the students toward proficiency in the course objectives. To address this shortcoming we have started to implement a modified version of competency-based grading or proficiency-based grading that we denote as \textbf{A}ssessment of \textbf{L}earning \textbf{P}roficiency \textbf{a}nd \textbf{C}ompetency \textbf{A}chievement (ALPaCA). In ALPaCA, we assign each problem on any graded activity to one or more specific learning objectives for the course. A student's grade in the course is then determined by the extent to which she or he has demonstrated proficiency or competency with each objective. The initial analysis of ALPaCA results obtained in general physics II has uncovered that while most students in this course are able to solve directed problems involving calculus, a smaller fraction of students can solve undirected problems involving calculus. This information points to students' inability to transfer skills from calculus courses into physics courses, and provides concrete targets for improvement in instruction.