When classroom interactions have to go online: the move to specifications grading in a project-based design course

Rebecca Quintana (School of Education, University of Michigan, Ann Arbor, Michigan, USA and Center for Academic Innovation, University of Michigan, Ann Arbor, Michigan, USA)
Chris Quintana (School of Education, University of Michigan, Ann Arbor, Michigan, USA)

Information and Learning Sciences

ISSN: 2398-5348

Article publication date: 16 July 2020

Issue publication date: 10 August 2020

Downloads
1971

Abstract

Purpose

The events surrounding the COVID-19 crisis had a profound effect on higher education, forcing students and instructors to face a sudden transition to wholly online learning contexts. This paper aims to examine how the design of a residential course was adapted to an online context and how this adaptation may prove beneficial to future iterations of the course.

Design/methodology/approach

This analysis centers on a master’s-level course in which students design software to support learning. One of the major changes to the course revolves around the transition from a traditional rubric-based grading scheme to a specifications grading system. This latter approach provides a series of binary (pass/fail) requirements (specifications) that students must meet to pass. Various forms of interactions were also altered during the transition; the authors investigate these in the paper.

Findings

This study found that the move to specifications grading helped students and the instructor to focus on the important work of meeting course learning goals. The approach also aligned well with authentic scenarios in which software projects are tested against certain specifications. Finally, this study concludes that thinking about specifications grading in the future can help us to develop more resilient pedagogical design approaches that respond to various forms of disruptions and changes.

Originality/value

The course design insights described in this paper illustrate alternative ways of instruction that can be especially useful during times of emergency, but which may also provide an added level of authenticity and learner motivation during times of stability.

Keywords

Citation

Quintana, R. and Quintana, C. (2020), "When classroom interactions have to go online: the move to specifications grading in a project-based design course", Information and Learning Sciences, Vol. 121 No. 7/8, pp. 525-532. https://doi.org/10.1108/ILS-04-2020-0119

Publisher

:

Emerald Publishing Limited

Copyright © 2020, Emerald Publishing Limited


1. Introduction

In Winter 2020, the COVID-19 public health crisis forced most colleges and universities to make an abrupt transition from face-to-face instruction to remote teaching. Instructors had to create supportive learning environments for students and bring the semester to a successful conclusion, even if that meant adjusting expectations for course completion. Additionally, they had to focus on their students’ mental and physical well-being. The goal of emergency remote teaching (ERT) (Bozkurt and Sharma, 2020) was to maintain continuity of instruction and to allow students to complete the semester, even if the final version of courses was not what was originally planned. In this article, we review the original design for a master’s-level education design course and contrast it with the modified online instantiation. We highlight a major outcome of this transition: the instructor’s decision to shift from traditional rubric grading to specifications grading – an approach that provides a series of binary (pass/fail) requirements (specifications) that students must meet to pass (Nilson, 2015). We offer these course design insights to illustrate alternative ways of instruction that can be especially useful during times of emergency, but which may also provide an added level of authenticity and learner motivation during times of stability.

2. Planned classroom interactions for a software design for learning course

Our analysis focuses on “Principles of Software Design for Learning,” a master’s-level course offered at a school of education and cross-listed at a school of information. Class meetings are 3-h class sessions, one day per week. The major course objective is for students to gain practical experience in designing learning technologies that address key issues in learning sciences, learning theory and human–computer interaction. The original course design integrates ideas from approaches for teaching design in higher education settings (Davis, 2017) and other work about project-based pedagogies in different fields (Krajcik and Blumenfeld, 2006). The course aims to create a knowledge community where learners actively construct knowledge and work within different communities of practice to find support, share and engage in disciplinary ideas and discourse with peers and experts (Slotta et al., 2018). In this case, those disciplinary ideas and discourse reflect authentic design as a complex set of practices that include a range of analytical and constructive activities (Lawson, 2006). The core course activity involves a semester-long group project (3–5 students) to design a learning technology, moving from project proposal to final conceptual prototype. The course includes an assigned set of weekly readings and class discussions on a range of course topics: design frameworks, designing for user experience versus learner experience, characteristics of learning technologies, how different learning theories (behaviorism, cognitive science and constructivism) inform the design of learning technologies and design techniques (storyboarding and prototyping).

Student–teacher and student–student interactions are key to scaffolding learning as students learn content and complete their design project. Weekly design reviews are pivotal and act as scaffolding mechanisms, as the instructor meets with groups to discuss progress, pose and answer questions and provide feedback. Long course periods allow groups to work on their projects in class, as the instructor rotates between groups. Additionally, the instructor occasionally pairs up groups for informal interim design reviews on the progress of each group’s design project, allowing for peer feedback. Table 1 summarizes the course’s original key design elements detailed by Quintana (2009). These course elements are designed around the classroom interactions model (Cohen et al., 2003), which emphasizes the various relationships that exist between and among instructors, students and content. For example, designed student–content interactions (as listed in Column 1) include: readings, review of lecture slides and assignments (as described in Column 2).

3. Challenges and concerns in the shift to emergency remote teaching

The ERT timing meant that the final four weeks of classes (12 h) would move online. This shift occurred as the lecture portion of the course was winding down while project work and design reviews were accelerating. It may seem that this would make for less demanding classes, but the final month is actually busy and sometimes stressful for the students as they finalize their software designs, presentations and final paper, which collectively comprise the bulk of the course grade. Design activity is complex as students use different tools, artifacts, research and storyboards.

Moving online forced the instructor to think about completing the remaining activities in ways that address course-learning goals while dealing with student and instructor anxiety. The instructor suddenly had little time (one week) to translate existing course plans, activities and resources to an online setting. Student anxiety was compounded by the abrupt shift of all courses to online – many of them having instructors with little experience in online instruction – along with sudden life changes: losing the support system of face-to-face social interactions and moving away from campus to go home or with friends (or a reduction in room-and-board services for those students who stayed on campus). Instructor plans had to take these life changes into account in addition to the instructional ramifications of modifying course activities.

4. Move to specifications grading

Given these stressors, the instructor approached the shift to online instruction through the lens of care theory (Robinson et al., 2020), which framed course modifications that aimed to alleviate student anxiety. These changes were also guided by the instructor’s replacement of analytic grading rubrics with a simplified specifications grading approach (Nilson, 2015) as a means of streamlining requirements for final high stakes assignments.

A specifications approach to assessment provides a series of requirements (specifications) for students to meet. All specifications are equally necessary, providing a passing grade if all are met and a failing grade otherwise. This approach to grading shares similarities with mastery learning, competency-based grading and contract learning (Elkins, 2016) and it carries certain advantages. For one, it can lead to more rigorous academic standards. Instead of students settling for lower quality work that meets only some of the instructor’s expectations, they are required to raise their efforts to the level of the specifications because there is no partial credit (Nilson, 2015).

Yet, paradoxically, this approach can lead students to experience reduced stress for two reasons (Nilson, 2015). First, they can see the instructor’s course expectations from the outset. There is less room for confusion or disagreement because students can consult a comprehensive specifications list (Nilson, 2015). Though rubrics can help make expectations more explicit for students (Jonsson and Svingby, 2007), the instructor found that the more simple, straightforward specifications list increased the clarity of expectations even more. Another factor is the introduction of “safety-net” mechanisms that create a more flexible, lower-stakes grading environment. With the opportunity to revise or drop work that fails to meet specifications, students can iterate without penalty, incorporate instructor feedback and hopefully end up with a higher quality product (Williams, 2018).

In this course, specifications grading represented a shift from analytic rubrics. Though the new specifications list kept the same categories used in the original rubric, it removed the added complexity of having four levels of proficiency for each category. Using a clear and detailed specifications list, the instructor hoped to alleviate student anxiety associated with their efforts to meet objectives at the highest levels of each rubric criterion. In this instantiation of specifications grading, students were assured of achieving an “A” on their final projects if they met the specifications requirements. Thus, the instructor was motivated to pursue a modified specifications grading approach. Though they were not able to incorporate the safety-net mechanisms that a full specifications approach provides, they had confidence that students would be successful in their final project work given the progress they had already made on design outlines and sketches.

5. Reconfigured classroom interactions for the software design for learning course

We now revisit the classroom interactions model (Cohen et al., 2003) to outline considerations for addressing instructional challenges emerging from this ERT context along with corresponding course modifications.

5.1 Teacher–content and student–content interactions

The main challenge involved modifying remaining course activities and assignments while considering student workload, anxiety and the pedagogical rationale for the modifications. The instructor also needed to consider the overall set of expectations and elements represented in the assignment rubrics and examine how to move to specifications grading while maintaining the integrity of the original instructional plan. The instructor eliminated the final paper requirement and asked students to incorporate the theoretical rationale originally slated for the paper into their project presentation. This allowed students to still consider the intersection of theory with their designs, but without the need for another large assignment.

The instructor presented students with a specifications list (Table 2) during the first synchronous online session and explained how each item is related to a course-learning goal. Each specification was followed by concrete details about what each specification category entailed and links to previous lecture slides that expanded on each requirement.

The instructor hoped that introducing the specifications list for remaining work would concentrate students’ focus on essential matters while also maintaining an explicit connection to key facets of the course. At the conclusion of the semester, students received confirmation through the course management system (CMS) that they had met all project requirements, along with detailed written instructor feedback with considerations for further development.

5.2 Teacher–student and student–student interactions

Additional challenges involved teacher–student and student–student interactions such as how group work and instructor design reviews would be impacted. Not only would students have to meet via videoconference but also group meetings would be complicated because many of the paper-based artifacts that are used during design activities (drawings, sticky notes with design research observations, etc.) could be difficult to replicate in a digital environment. The instructor now had to determine assignment modifications that focused on the group projects: the software prototypes and the final project class presentations (which now included aspects of the eliminated final paper assignment).

After moving online, the instructor continued to hold class at the regularly scheduled time through videoconferencing software. This decision was made after ensuring that students would be available during this time (i.e. within the same time zone) and had computers with videoconferencing capabilities. This decision allowed the instructor to use class time for design critiques as before, with some modifications. The instructor introduced whole-group critiques, with each group sharing progress and presenting their design outlines in one session and design sketches in a subsequent session. The instructor provided the class with a collaborative word processing document to ask questions of each group. In the first online session, the instructor asked student groups to attend to a subset of the entire specifications list in their presentations and questions (vision of learning context and rationale for pedagogical choices). The following week, the instructor asked students to focus on a different specifications subset (scenario design and scaffolding design framework). In this session, a second document was used for each group to list specific areas they wanted feedback on. These collaborative word processing documents provided a lasting record of insights and questions from the entire community, which could be used as a guide during group meetings that took place between class sessions. This approach allowed students to become familiar with all groups’ project designs, freeing up time in the final presentations to address theoretical considerations, which were originally part of a final paper. Following both of these presentation sessions, the instructor created “breakout rooms” that allowed students to resume their group work. Here, the instructor joined each group and focused discussion on another aspect of the learning goals category of the specifications list. Across this instructional sequence, the instructor and students were able to focus on all aspects of the specifications list over three class meetings. This ensured that project groups were well prepared to meet requirements of the specifications list in their final presentation.

6. Reflecting on the impact of specifications grading

Despite these remarkable instructional circumstances, students and their instructor adapted and persevered. The instructor was initially concerned that moving to specifications grading would lead students to do only the “bare minimum” and produce substandard project work. However, the instructor was delighted that students presented final projects of comparable quality to previous years. Anecdotally, student work was not adversely affected by the move to specifications grading. If anything, the simplified grading criteria seemed to free up students and the instructor to focus on the important work of meeting course learning goals, rather than worrying about the details of the various levels of achievement they could reach on the previously used analytic rubric. In breakout sessions with student project groups, the instructor witnessed productive conversations that centered around their software designs, rather than what the instructor was “looking for” in the final presentation. Crucially, the specifications grading approach helped illustrate to the students the design practices and learning theories that are a key component of the course’s learning objectives, while simultaneously demonstrating that students were successful in doing work that satisfied the specifications. All of this helped us see that students had reached the objective of the course.

Additionally, the online format inspired the instructor to ask students to invite respected professors and industry experts to join their final presentations. This added level of interaction between students and experts from outside the traditional class organization further supported the instructor’s goal of cultivating a knowledge community in ways that they had not been able to implement in previous course offerings. In this way, the rapid shift to the online environment provided an unexpected benefit. Another contrast to previous offerings of the course was that, paradoxically, even amidst the stressful circumstances surrounding COVID-19, students likely experienced lower levels of “grade anxiety” than they had in previous course offerings, as students may not always feel confident that they understand the full set of expectations for the final presentation. The specifications grading approach here made those expectations clearer to students, allowing for this lessening of pressure that allowed students to focus on input from guests and resulted in a celebratory atmosphere in the final class.

7. Emerging ideas for using specifications grading in the future

Future course iterations will benefit from this emergency transition to remote teaching. The changes we have described were guided by the idea of specifications grading to make near-term course modifications. But in doing so, it has also led to ideas about wider course modifications in the future, independent of format. Specifications grading was introduced in the latter part of the course and was applied to one major assignment. In future iterations, the instructor would introduce the approach at the beginning of the course and use it across all assignments, providing the pedagogical rationale for their choice and introducing it through smaller assignments. We can also envision building in the safety-net mechanisms from a full specifications approach, including opportunities for students to revise work that does not meet specifications standards. This would enable students to incorporate feedback and ultimately produce a higher quality product (Williams, 2018). The specifications approach seems well aligned with authentic scenarios where software projects are tested against certain specifications, which are either met or not. This approach is appealing to consider because it could be implemented in a course independent of modality, whether that is face-to-face, hybrid or online. Thinking about specifications grading in the future can help us develop more resilient pedagogical design approaches (Quintana, 2020) that respond to various forms of disruptions and changes.

Course interactions fostered by the elements of the original course

Interaction Course elements
Teacher–content interactions – Instructor development of assignments, rubrics, course readings, tools and CMS
  • Design critiques where individual students select and critique a learning technology

  • Semester-long group design project to design software that supports learning

  • Final group project presentation to summarize the project, learning goals, theoretical connections and design rationale

  • Final reflection paper where individual students reflect on their role on the group project and provide other feedback about their work

Student–content interactions – Student interactions with course materials
  • Course readings, using Perusall for social annotation to highlight, comment and respond to comments

  • Review of lecture slides for reference

  • Course assignments guided by rubrics

Teacher–student interactions
  • Lectures

  • In-class discussions

  • Design reviews for tailored, detailed feedback on projects

  • Reviewing and grading assignments to provide feedback and gauge student understanding

Student–student interactions
  • Regular student group design meetings in and out of class

  • Informal group presentations and peer feedback

  • Small group conversations prompted by instructor during lecture

  • Asynchronous commenting and discussion about readings in Perusall

Design project specifications list

Category Specifications
Vision of the learning context
  • Provides detailed description of key elements in learning environment such as domain concepts and activities, available resources (including peers and teacher) and available tools (software and hardware)

  • Provides nuanced description of planned learning activities and assessment strategies

Vision of the learner
  • Provides details about learners who will use tool, such as prerequisite knowledge and learner needs and characteristics

  • Uses various sources to create vision of learner, such as surveys, interviews or literature reviews

Clear vision of learning goals
  • Articulates clearly specified learning goals that allow learners to show evidence of their learning, including what learners should be able to know, do and feel

  • Learning goals are specific, observable and measurable

Scenario design
  • Paints a picture of how designed tool will be used by learners with a learning context

  • Characterizes the interactions that tool aims to support, such as learner to content, learner to learner

Rationale for pedagogical choices
  • Describes specific features of tool that supports learning (providing specific examples) and includes a rationale for how these features connect to learning theories, such as behaviorist- or constructivist-oriented pedagogies

Scaffolding design framework
  • Gives explanation of how tool instantiates ideas from the scaffolding design framework

Design outline and sketches
  • Includes design outlines for tool, which details information structure and usability flow

  • Includes design sketches or wireframes that detail aspects of user interface screens and interactivity

References

Bozkurt, A. and Sharma, R.C. (2020), “Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic”, Asian Journal of Distance Education, Vol. 15 No. 1.

Cohen, D.K., Raudenbush, S.W. and Ball, D.L. (2003), “Resources, instruction, and research”, Educational Evaluation and Policy Analysis, Vol. 25 No. 2, pp. 119-142.

Davis, M. (2017), Teaching Design, Allworth Press.

Elkins, D.M. (2016), “Grading to learn: an analysis of the importance and application of specifications grading in a communication course”, Kentucky Journal of Communication, Vol. 35 No. 2, pp. 26-48.

Jonsson, A. and Svingby, G. (2007), “The use of scoring rubrics: reliability, validity and educational consequences”, Educational Research Review, Vol. 2 No. 2, pp. 130-144, doi: 10.1016/j.edurev.2007.05.002.

Krajcik, J.S. and Blumenfeld, P.C. (2006), “Project-based learning”, in Sawyer, R.K. (Ed.), The Cambridge Handbook of the Learning Sciences, Cambridge University Press.

Lawson, B. (2006), How Designers Think: The Design Process Demystified, 4th ed. Architectural Press, London.

Nilson, L. (2015), Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time, Stylus Publishing, LLC.

Quintana, C. (2009), “A learning technology design course, deconstructed”, in DiGiano, C., Goldman, S. and Chorost, M. (Eds), Educating Learning Technology Designers: Guiding and Inspiring Creators of Innovative Educational Tools, Routledge, pp. 165-181.

Quintana, R.M. (2020), Resilient Teaching through Times of Crisis and Change [MOOC], Coursera.

Robinson, H., Al-Freih, M. and Kilgore, W. (2020), “Designing with care: towards a care-centered model for online learning design”, The International Journal of Information and Learning Technology, Vol. 37 No. 3, doi: 10.1108/IJILT-10-2019-0098.

Slotta, J.D., Quintana, R.M. and Moher, T. (2018), “Collective inquiry in communities of learners”, Fischer, F., Hmelo-Silver, C. E., Goldman, S. R. and Reimann, P. (Eds), International Handbook of the Learning Sciences, Routledge, New York, NY, pp. 308-317.

Williams, K. (2018), “Specifications-based grading in an introduction to proofs course”, Problems, Resources, and Issues in Mathematics Undergraduate Studies, Vol. 28 No. 2, pp. 128-142.

Acknowledgements

The authors would like to acknowledge Juan D. Pinto for his invaluable assistance in preparing this manuscript. Additionally, we would like to recognize the resilience and good humor of the students in this course.

This article is part of the special issue, “A Response to Emergency Transitions to Remote Online Education in K-12 and Higher Education,” which contains shorter, rapid-turnaround invited works, not subject to double blind peer review. The issue was called, managed and produced on short timeline in Summer 2020 toward pragmatic instructional application in the Fall 2020 semester.

Corresponding author

Rebecca Quintana can be contacted at: rebeccaq@umich.edu