Abstract
Purpose
The purpose of this paper is to share the author’s viewpoint on how to increase student response rate in course evaluation surveys.
Design/methodology/approach
The approach is to highlight measures which increased student response rate in online surveys of the author’s teaching evaluation at The University of the West Indies, Jamaica.
Findings
This viewpoint suggests that student response rate to course evaluation can be improved by the lecturer’s effective communication. The examples of effective communication are given in this paper.
Originality/value
This work will encourage the lecturers to initiate more student engagement to improve response rate of their teaching evaluation.
Keywords
Citation
Ahmad, T. (2018), "Teaching evaluation and student response rate", PSU Research Review, Vol. 2 No. 3, pp. 206-211. https://doi.org/10.1108/PRR-03-2018-0008
Publisher
:Emerald Publishing Limited
Copyright © 2018, Tashfeen Ahmad.
License
Published in PSU Research Review. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Importance of student evaluation of teaching
Student evaluation of teaching is important for a number of reasons. These evaluations ensure quality in university teaching, provide an independent method of gauging teacher’s effectiveness, guide in making decisions for major curriculum changes and professional development for faculty and help in establishing a framework to better quantify and reward good teaching outcomes.
Shift in student evaluation of teaching from paper-based to online surveys
Paper-based assessment has been the most common form of student evaluation of teaching worldwide. However, over the past decade, there has been a shift away from paper-based to online assessment. As internet is becoming more available and affordable, traditional paper-based data collection methods, seem expensive, time consuming and less efficient.
Positives of this shift to online evaluation
One of the most important positives is efficiency gains, in terms of turnaround time from students and significant cost savings. In addition, online evaluations allow students the time, ease and ability to refine, expand and reflect on responses without the constraint of an “in class” time bound environment to complete paper-based surveys. This increases student response to open-ended questions which provide qualitative data which is instrumental in improving teaching practices.
A review of the major literature works over the period 2000-2013 (as depicted in Table I) summarizes the main advantages of online method of evaluation.
Challenges of this shift to online evaluation
One of the biggest challenges is the low response and return rate of students to online evaluations when compared to paper-based evaluation responses (Benton et al., 2010; Goodman et al., 2015; Guder and Malliaris, 2010; Nowell et al., 2010).
A sample of the findings of response rates drawn from different research studies at various higher education institutions (as seen in Table II) over the years 1999-2013 indicates general lower return rates for online evaluations compared to paper-based evaluations in all except two cases (ranging as low as 17 per cent to a high of 83 per cent).
This low number of response rates, in online surveys, makes the data invalid. To mitigate this challenge, Nulty’s (2008) research provides a set of guidelines for required response rates to be considered valid and useful measure of accuracy for online evaluation. Since the larger is the class size, the lower response rate is required, Nulty recommends an ideal required response rate of 58 (size <20) and 35 per cent (>50) for accuracy of online survey results and to achieve validity.
Major reasons for the differences in response rates
The reasons for the differences in response rates range from gender and age factors (Hatfield and Coyle, 2013); privacy and anonymity (Khorsandi et al., 2012; Nevo et al., 2010); social pressure; distraction and location issues (Mau and Opengart, 2012); lack of engagement; incentives; communication; perceived inaction with feedback or general “survey fatigue” (Bennett and Nair, 2010); and demographic and economic variables peculiar to the institution of country (Morrison, 2011).
Solving the issue of low response rate
Bennett and Nair (2010) in their study at an Australian University were able to register an overall 83 per cent online response rate, but this was in response to the deliberate strategies and measures implemented to increase student involvement. Using effective engagement, communication and teacher–student participation techniques led to greater and more sustained response rates.
Measures to increase student online response rates
A vast amount of literature has been written about the problems and the strategies which can be used to encourage and increase the response rates of student online evaluation (Crews and Curtis, 2011; Morrison, 2011; Stowell et al., 2012).
The most comprehensive work done by Berk (2012) outlines a review of the problems and articulates an in-depth set of techniques and best practices which can be applied to increase online response rates. It should be noted however that he does not advocate a “one size fit all” solution but emphasizes that success in raising response rates will most likely be met by a combination of strategies and incentives over the long term.
In my opinion, the most important and fundamental ingredients for raising online response rates depends to large extent on the commitment, engagement and buy in of both students and teaching administrators to the process. For example, studies indicate that the biggest determinant for student participation in online evaluation is the level of engagement they obtain from teachers (Gaillard et al., 2011).
Those institutions which take the time to communicate and explain the process, how their responses will be used or incorporated to improve course delivery and outcomes experience increase in response rates (Wode and Keiser, 2011). On the other hand, students who do not feel a part of the process or think their feedback will not be taken seriously or valued or teachers who do not effect changes consequent on feedback experience lower response rates (Beran and Rokosh, 2009).
What can lecturers do to increase response rate?
The response rates are important as these evaluations are frequently used for consideration in tenure and promotion, hiring and pay increase decisions (Hammonds et al., 2017).
My viewpoint is that response rate can be increased if lecturers are informed about the timing of when the surveys are sent out, so they can also make a personal appeal (both in class and by email) to the students to complete their course evaluation surveys.
In this communication, lecturers should explain to the students how their comments would be taken seriously, and how it will be used to improve teaching (Heinert and Roberts, 2016).
The key is to inform students about the purpose of evaluations:
Let students know that you will use their feedback to make changes in the course.
Give students some examples of useful feedback you have received in the past, and how the course/pedagogy has benefited in response.
This best practice will show you improved results, and if you also want to score better in these evaluations, start giving chocolate cookies to your students (ESA, 2018).
Main advantages of using online course evaluation surveys
| Authors (Year) | Main advantages | Research focus areas |
|---|---|---|
| Hmieleski and Champagne (2000) | More written feedback Refine, reflect, expand on responses |
Student course evaluations |
| Kasiar et al. (2002) | More written feedback Refine, reflect, expand on responses |
Comparison of traditional and Web-based evaluation processes |
| Johnson (2002) | Richer and higher data collection | Online student ratings |
| Hardy (2003) | More written feedback Refine, reflect, expand on responses |
Online student ratings |
| Ballantyne (2003) | Richer and higher data collection | Online evaluations of teaching |
| Ballantyne (2004) | Richer and higher data collection | Online student survey and comments |
| Anderson et al. (2006) | Provide more feedback Richer and higher data collection |
Student course evaluations |
| Donovan et al. (2006) | Provide more comments about Lecturer |
Student feedback on online vs traditional course evaluations |
| Laubsch (2006) | More written feedback Refine, reflect, expand on responses |
Comparison of online and in person evaluations |
| Donovan et al. (2006) | Richer and higher data collection | Constructive student feedback on online and traditional evaluations |
| Emery et al. (2008) | Efficiency, cost savings, richer responses | Open source online evaluation experiences |
| Miller (2010) | Time and cost savings, richer responses | Online evaluations |
| Samuels (2013) | Richer responses, efficiency, quicker and cost savings | Academic departments use of online course evaluations |
Comparison of response rates (online versus paper-based evaluation)
| Authors (Year) | Institution | Response rates |
|---|---|---|
| Layne et al. (1999) | Southeastern University | 47% – online vs 60% – paper |
| Sax et al. (2003) | Several US institutions | 17% – online vs 24% – paper |
| Dommeyer et al. (2004) | California State University | 43% – online vs 75% – paper |
| Anderson et al. (2005) | University of Kentucky | 83% – online vs 80% – paper |
| Avery et al. (2006) | Cornell University | 47% – online vs 69% – paper |
| Laubsch (2006) | Fairleigh Dickinson University | 61% – online vs 82% – paper |
| Nair et al. (2008) | Monash University | 31% – online vs 56% – paper |
| Perrett (2013) | Large university in South US | 71% – online vs 68% – paper |
References
Anderson, H.M., Cain, J. and Bird, E. (2005), “Online student course evaluations: review of literature and a pilot study”, American Journal of Pharmaceutical Education, Vol. 69 No. 1, p. 5.
Anderson, J., Brown, G. and Spaeth, S. (2006), “Online student evaluations and response rates reconsidered”, Innovate: Journal of Online Education, Vol. 2 No. 6.
Avery, R.J., Bryant, W.K., Mathios, A., Kang, H. and Bell, D. (2006), “Electronic course evaluations: does an online delivery system influence student evaluations?”, The Journal of Economic Education, Vol. 37 No. 1, pp. 21-37.
Ballantyne, C. (2003), “Online evaluations of teaching: an examination of current practice and considerations for the future”, New Directions for Teaching and Learning, Vol. 2003 No. 96, pp. 103-112.
Ballantyne, C. (2004), “Online or on paper: an examination of the differences in response and respondents to a survey administered in two modes”, in International Conference of Australasian Evaluation Society, Adelaide, October.
Bennett, L. and Nair, C.S. (2010), “A recipe for effective participation rates for web‐based surveys”, Assessment & Evaluation in Higher Education, Vol. 35 No. 4, pp. 357-365.
Benton, S.L., Webster, R., Gross, A.B. and Pallett, W.H. (2010), An Analysis of IDEA Student Ratings of Instruction Using Paper versus Online Survey Methods 2002-2008 Data, The IDEA Technical Center.
Beran, T.N. and Rokosh, J.L. (2009), “Instructors’ perspectives on the utility of student ratings of instruction”, Instructional Science, Vol. 37 No. 2, pp. 171-184.
Berk, R.A. (2012), “Top 20 strategies to increase the online response rates of student rating scales”, International Journal of Technology in Teaching and Learning, Vol. 8 No. 2, pp. 98-107.
Crews, T.B. and Curtis, D.F. (2011), “Online course evaluations: faculty perspective and strategies for improved response rates”, Assessment & Evaluation in Higher Education, Vol. 36 No. 7, pp. 865-878.
Dommeyer, C.J., Baum, P., Hanna, R.W. and Chapman, K.S. (2004), “Gathering faculty teaching evaluations by in‐class and online surveys: their effects on response rates and evaluations”, Assessment & Evaluation in Higher Education, Vol. 29 No. 5, pp. 611-623.
Donovan, J., Mader, C.E. and Shinsky, J. (2006), “Constructive student feedback: online vs traditional course evaluations”, Journal of Interactive Online Learning, Vol. 5 No. 3, pp. 283-296.
Emery, L., Head, T., Zeckoski, A., Yu. and Borkowski, E. (2008), “Deploying an open source, online evaluation system: multiple experiences”, Presentation at Educause 2008, October 31, Orlando, FL.
ESA (European Society of Anaesthesiology) (2018), “Teachers who give cookie rewards score better in evaluations”, ScienceDaily, 17 June, available at: www.sciencedaily.com/releases/2018/06/180604182502.htm
Gaillard, F.D., Mitchell, S.P. and Kavota, V. (2011), “Students, faculty, and administrators perception of students evaluations of faculty in higher education business schools”, Journal of College Teaching & Learning (Tlc)), Vol. 3 No. 8.
Goodman, J., Anson, R. and Belcheir, M. (2015), “The effect of incentives and other faculty-driven strategies to increase online student evaluation response rates”, Assessment & Evaluation in Higher Education, Vol. 40 No. 7, pp. 958-970, doi: 10.1080/02602938.2014.960364.
Guder, F. and Malliaris, M. (2010), “Online and paper course evaluations”, American Journal of Business Education (AJBE), Vol. 3 No. 2, pp. 131-138.
Hammonds, F., Mariano, G.J., Ammons, G. and Chambers, S. (2017), “Student evaluations of teaching: improving teaching quality in higher education”, Perspectives: Policy & Practice in Higher Education, Vol. 21 No. 1, pp. 26-33, doi: 10.1080/13603108.2016.1227388.
Hardy, N. (2003), “Online ratings: fact and fiction”, New Directions for Teaching and Learning, Vol. 2003 No. 96, pp. 31-38.
Hatfield, C.L. and Coyle, E.A. (2013), “Factors that influence student completion of course and faculty evaluations”, American Journal of Pharmaceutical Education, Vol. 77 No. 2, p. 27.
Heinert, S. and Roberts, T.G. (2016), “Factors motivating students to respond to online course evaluations in the college of agricultural and life sciences at the university of Florida”, NACTA Journal, Vol. 60 No. 2, pp. 189-194, available at: http://eds.a.ebscohost.com.proxy1.ncu.edu
Hmieleski, K. and Champagne, M.V. (2000), “Plugging in to course evaluation”, The Technology Source.
Johnson, T. (2002), “Online student ratings: will students respond? Online student ratings of instruction”, New Directions for Teaching and Learning, Vol. 2003 No. 96, pp. 49-59.
Kasiar, J.B., Schroeder, S.L. and Holstad, S.G. (2002), “Comparison of traditional and web-based course evaluation processes in a required, team-taught pharmacotherapy course”, American Journal of Pharmaceutical Education, Vol. 66 No. 3, pp. 268-270.
Khorsandi, M., Kobra, A., Ghobadzadeh, M., Kalantari, M. and Seifei, M. (2012), “Online vs traditional teaching evaluation: a cross-sectional study”, Procedia – Social and Behavioral Sciences, Vol. 46 No. 2012, pp. 481-483.
Laubsch, P. (2006), “Online and in-person evaluations: a literature review and exploratory comparison”, Journal of Online Learning and Teaching, Vol. 2 No. 2, pp. 62-73.
Layne, B.H., DeCristoforo, J.R. and McGinty, D. (1999), “Electronic versus traditional student ratings of instruction”, Research in Higher Education, Vol. 40 No. 2, pp. 221-232.
Mau, R.R. and Opengart, R.A. (2012), “Comparing ratings: in-class (paper) vs out of class (online) student evaluations”, Higher Education Studies, Vol. 2 No. 3, p. 55.
Miller, M.H. (2010), “Online evaluations show same results, lower response rate”, The Chronicle of Higher Education.
Morrison, R. (2011), “A comparison of online versus traditional student end‐of‐course critiques in resident courses”, Assessment & Evaluation in Higher Education, Vol. 36 No. 6, pp. 627-641.
Nair, C.S., Adams, P. and Mertova, P. (2008), “Student engagement: the key to improving survey response rates”, Quality in Higher Education, Vol. 14 No. 3, pp. 225-232.
Nevo, D., McClean, R. and Nevo, S. (2010), “Harnessing information technology to improve the process of students’ evaluations of teaching: an exploration of students’ critical success factors of online evaluations”, Journal of Information Systems Education, Vol. 21 No. 1, p. 99.
Nowell, C., Gale, L.R. and Handley, B. (2010), “Assessing faculty performance using student evaluations of teaching in an uncontrolled setting”, Assessment & Evaluation in Higher Education, Vol. 35 No. 4, pp. 463-475.
Nulty, D.D. (2008), “The adequacy of response rates to online and paper surveys: what can be done?”, Assessment & Evaluation in Higher Education, Vol. 33 No. 3, pp. 301-314, doi: 10.1080/02602930701293231.
Perrett, J.J. (2013), “Exploring graduate and undergraduate course evaluations administered on paper and online: a case study”, Assessment & Evaluation in Higher Education, Vol. 38 No. 1, pp. 85-93.
Samuels, B. (2013), Increasing Number of Academic Departments Use Online Course Evaluations, CampusWest, 30 April.
Sax, L.J., Gilmartin, S.K. and Bryant, A.N. (2003), “Assessing response rates and nonresponse bias in web and paper surveys”, Research in Higher Education, Vol. 44 No. 4, pp. 409-432.
Stowell, J.R., Addison, W.E. and Smith, J.L. (2012), “Comparison of online and classroom-based student evaluations of instruction”, Assessment & Evaluation in Higher Education, Vol. 37 No. 4, pp. 465-473.
Wode, J. and Keiser, J. (2011), “Online course evaluation literature review and findings”. A Report from Academic Affairs, Columbia University, Chicago.
Corresponding author
About the author
Tashfeen Ahmad expanded his understanding of Psychology at Harvard University and joined The University of the West Indies, Mona, Jamaica, with 10 years of general management experience. He has taught courses in International Business, Production Management, Operations Management, Quality Management and Change Management. His research work focuses on the future of higher education and learning technologies.