The recognition of practice in online instruction is still subject to interpretation and different approaches as a result of the rapid changes in technology and its effect on society. The purpose of this paper is to address these differences through a synthesis that can be easily accessed and consulted by educators in the field of e-learning.
This paper reviews different examples of rubrics and instruments in higher education to propose a more comprehensive rubric that constitutes a synthesis of how some institutions in HE approach best practice in this field.
The proposed comprehensive rubric emanating from the synthesis of different approaches supports the development, remixing, sharing and integration of online modules and courses by providing a single reference point with as wide a range as possible of potential pedagogical tools, facilities and approaches to e-learning.
It is not within the scope of this paper to review quality assurance processes and administrative components, but to propose a rubric for course design and self-review of faculty and higher education institutions for a better alignment with what is regarded as current standard best practice.
Instructional designers in e-learning have a new comprehensive rubric that can consult at design stage.
Different approaches towards what is called “good practice” are brought together and analysed to provide a synthesis and a single source that can be consulted by practitioners in the field of e-learning.
Debattista, M. (2018), "A comprehensive rubric for instructional design in e-learning", International Journal of Information and Learning Technology, Vol. 35 No. 2, pp. 93-104. https://doi.org/10.1108/IJILT-09-2017-0092Download as .RIS
Emerald Publishing Limited
Copyright © 2018, Martin Debattista
Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Literature review: slightly different paths for e-learning
By the year 2002, the fusion between in-class instruction and online instruction was already being recognised as a major new trend; old practices were being transferred online, and different institutions had rather different understandings of the concept of e-learning (Frydenberg, 2002; Graham et al., 2013; Young, 2002). As late as 2011, Guri-Rosenblit and Gros (2011) concluded there were “noticeable gaps” in e-learning research and definition of terminology. One of the reasons for this lack of coherence is:
The technological environment within which modern education operates is becoming increasingly complex, offering new possibilities but also giving rise to challenges. We have seen a continual evolution of technologies and how they are used since the introduction of the Internet
(Conole and Alevizou, 2010, p. 9).
De Freitas and Conole (2010) observed the trend of more global, more networked and more mobile technology infrastructure and these are emerging in online pedagogy. Indeed, McLoughlin and Lee (2010) argued that this new landscape:
Calls for the active involvement of students in defining their learning goals and choosing both ICT tools and strategies for learning; it also requires recognition that user and learner generated content has a central place in a curriculum that fosters self-regulated learning
Learning in the digital age thus requires a re-thinking of teaching and learning and not just replicating existing practices with new technology (Beetham and Sharpe, 2013).
While in the early years of public internet the emphasis was on the technical foundations of e-learning, the pedagogical implications started shifting direction from managing the logistics of e-learning to managing the content (Govindasamy, 2002). An “interactive and constructive potential of e-learning” was being recognised, one which contrasts with the traditional sage on the stage approach of information transfer. However, it had to prove it was more than just a more convenient way to access content (Garrison and Anderson, 2011, p. 54).
The focus of course design and pedagogy has shifted from teaching to learning, with teachers also becoming learners in the process of professional development and engagement with their students. Rather than transferring the passive teacher-centred form of learning to the online domain, a constructivist approach would make the experience learner centred, where active learning and engagement takes place (Rovai, 2004). The emphasis in higher education has shifted from the delivery of instruction to the production of learning (Barr and Tagg, 1995).
Far from simply providing a more comfortable channel to access instructional content, Nagel and Kotzé (2010) found, “When students engage in online activities and take responsibility for the quality of interaction, they can have a superior learning experience” (p. 218). ICT can indeed be successfully applied to enhance the educational process, especially in making learners active participants (Tomte and Sutherland Olsen, 2014). On the other hand, the digital literacy of academic staff remains a challenge (Johnson et al., 2015), with many feeling the pressure of 24/7 online connectivity in the age of social media (Grove, 2017).
E-learning faces issues of suspicion (Casey, 2008) and quality (Jung and Latchem, 2012) when compared to traditional instruction. However, a meta-analysis by Means et al. (2013) on behalf of the US Department of Education reveals a significant increase in performance for blended learning but not for pure online learning. This confirmed earlier conclusions by Pankin et al. (2012).
In their literature review on the use of Web 2.0 technologies in HE, Conole and Alevizou (2010) observed a number of key challenges proposed when technology meets education, including the tensions around the nature of openness, and changes in the role of educators and students. HE in Europe seems to be responding positively to these rapid developments in e-learning, though the adoption of e-learning and MOOCs is gradual (Gaebel, 2015).
Empirical evidence is increasing in respect to online learning and new developments such as open educational resources (OER), mobile learning, software agents for online evaluation (Daradoumis et al., 2013), bring your own device, the flipped classroom, wearable technologies, adaptive learning technologies and the Internet of Things (Johnson et al., 2015). These have attracted the attention of researchers to an ever-higher degree in recent years. However, more research on OER and MOOCs is required (Bozkurt et al., 2015).
The NMC Horizon Report 2017 (HE Edition) refers to a medium-term trend of using digital tools to measure knowledge and skills acquisition in online learning environments, including collaboration and creativity (Adams Becker et al., 2017).
According to the Times Higher Education Teaching Survey 2017, half of academics and 68 per cent of administrators agree that students benefit from digitised content, but they evidence less enthusiasm for recording lectures and putting them online. The use of social media for instructor-student contact is still not widespread as many academics feel the pressure of constantly being in demand by students (Grove, 2017).
Crossbreeding between social network sites and e-learning is increasingly observable. Though Facebook is not a replacement for VLEs in education, the latter “could certainly learn something from Facebook and the nature of its user created groups and networks, instant communications, alerts and like/sharing features” (Hogg, 2013, p. 35). These VLEs, such as MOODLE and Blackboard, had to introduce features similar to the increasingly popular social network sites (Ronan, 2015), while some social network sites such as LinkedIn are assuming educational roles (Hoffman, 2015). This development and the proliferation of MOOCs are some of the most interesting but controversial trends in higher education right now (Johnson et al., 2014). Early research shows that MOOCs are indeed contributing to online learning, though more empirical research is needed (Gamage et al., 2015; Glance et al., 2013).
An overview of four different rubrics
Measurement through rubrics
The measure of the success or failure of whether technology-driven education delivers the promised success can be determined through rubrics based on traditional principles, updated to cover the introduction of new technologies. In this manner, different instructional strategies can be devised to serve the different learning domains, including intellectual and cognitive strategies, attitudes, etc. (Fenrich, 2008, p. 310).
Graham et al. (2001) refer to the Seven Principles for Good Practice in Undergraduate Education published in 1987 as a popular framework for the evaluation of traditional classroom-based education, based on 50 years of research. These principles have been adapted for online education:
Principle 1: good practice encourages student-faculty contact.
Principle 2: good practice encourages cooperation among students.
Principle 3: good practice encourages active learning.
Principle 4: good practice gives prompt feedback.
Principle 5: good practice emphasises time on task.
Principle 6: good practice communicates high expectations.
Principle 7: good practice respects diverse talents and ways of learning.
Established frameworks began to be adopted and adapted to e-learning. For example, the rubric provided by the Quality Online Course Initiative of the University of Illinois (2015) (www.ion.uillinois.edu/initiatives/qoci/index.asp) is based on six sections, each with specific criteria, that mirror the above-mentioned seven principles:
Instructional design – Criteria: 1.1 Structure, 1.2 Learning Goals/Objectives/Outcomes, 1.3 Course Information, 1.4 Instructional Strategies, 1.5 Academic Integrity, 1.6 Use of Multimedia.
Communication interaction and collaboration – Criteria: 2.1 Activities and Opportunities, 2.2 Organisation and Management, 2.3 Group Work.
Student evaluation and assessment – Criteria: 3.1 Goals and Objectives, 3.2 Strategies, 3.3 Grades, 3.4 Feedback, 3.5 Management.
Learner support and resources – Criteria: 4.1 Institutional/Programme Support and Resources, 4.2 Academic Support and Resources.
Web design – Criteria: 5.1 Layout Design 5.2 Use of Multimedia, 5.3 Use of Images, 5.4 Links/Navigation, 5.5 Accessibility.
Course evaluation – Criteria: 6.1 Layout/Design.
California State University provides a Quality Online Learning and Teaching instrument to measure the effectiveness and quality of online courses (Christie, 2014). The instrument can be used for both self-evaluation and peer evaluation. It consists of 58 objectives in ten sections:
Section 1: course overview and introduction (eight objectives).
Section 2: assessment of student learning (six objectives).
Section 3: instructional materials and resources (six objectives).
Section 4: students interaction and community (course design) (seven objectives).
Section 5: facilitation and instruction (course delivery) (eight objectives).
Section 6: technology for teaching and learning (five objectives).
Section 7: learner support and resources (four objectives).
Section 8: accessibility and universal design (seven objectives).
Section 9: course summary and wrap-up (three objectives).
Section 10: mobile design readiness (optional) (four objectives).
The University of Malta (2015) provides Minimum Standards for Study Units in the VLE with a shortlist of suggested elements that must be provided in face-to-face courses complemented by online study units (blended mode) to meet minimum standards. This “advisory” is provided by the IT Services at the University of Malta rather than a unit responsible for academic quality assurance or pedagogy. Indeed, quite significantly, these guidelines do not make any reference to pedagogy for e-learning.
The following are the elements listed in the advisory by the University of Malta (www.um.edu.mt/vle/staff/minimumstandards): study unit description, tutor profile on VLE, class announcements (provided by default in the VLE), general Q&A forum, communication statement (tutor-student communication protocol), course readings, other learning resources, and assessment outline (exemplars and use of anti-plagiarism software).
QualityMatters™ is a commercial product that provides course design rubrics for different levels in the education domain. One of them is specific to higher education and provides eight general standards that “work together to ensure students achieve desired learning outcomes” in online and blended learning (QualityMatters™, 2014, para 5). A score of 85 per cent qualifies a course to receive a QM certification for quality in course design.
The standards of QualityMatters™ (2014) are: course overview and introduction, learning objectives (competencies), assessment and measurement, instructional material, learner activities and learner interaction, course technology, learner support, accessibility and usability.
A synthesis of the rubrics to formulate the new comprehensive rubric
Synthesising a comprehensive rubric
The rubrics referenced in this document all have common criteria that cover the most basic elements that an online course should satisfy if it aspires to provide effective teaching and learning. These common standards are (in no particular order): instructional design, web design and technical access, communication between tutor/s and students, interactivity and community building, instructional resources with possible multimedia use, instructional support, assessment, and evaluation of the instruction with learner feedback (see Table I).
The four institutions under analysis all cover most – if not all – the criteria derived from the synthesis of their rubrics.
In terms of ratings, the California State University (2013) and QualityMatters™ (2014) provide their own rating scales. The former allows the adopter of the rubric to assign the same weighting range to all criteria (one to three points) according to the extent to which it is met or not, while the latter sets specific number of points (one, two or three) to be awarded to any individual standard when it is met (no points for partial or non-fulfilment). The QualityMatters™ rubric assigns the most points (three out of a maximum of three) to the statement of the learning objectives/competencies, assessment, the quality of instructional resources, tutor-learner interaction, learner support, and the ease of use of the technical platform where the virtual learning environment resides.
The comprehensive rubric
The synthesis of the four rubrics just referenced has produced the following comprehensive rubric that covers all the aspects mentioned by the four institutions. This comprehensive rubric is not a collation of the four rubrics but a synthesis of the separate approaches that – in the author’s view – reflects the context of e-learning as explained at the beginning of this paper. This rubric has not been tested in lab setting or a real-life scenario.
There are ten main standards, each containing specific standards.
Ten main standards and specific standards are as follows:
Instructional design – an analysis of the learning needs and the use of appropriate strategies and methods to meet them:
Structure of learning.
Learning aims and objectives – what the instructor needs to achieve with the learning process.
Learning outcomes – what learners need to achieve to have successfully completed the learning process.
Instructional strategies and methods.
Course opening – welcoming learners:
Accessibility – the instructor gives clear instructions on how to access all elements of the online learning environment.
Role – the instructor gives clear information about his professional role in the learning environment.
Description – a course description with pre-requisites (if any), clear learning outcomes and what is expected of the learners is provided.
Behaviour – the learners are made aware of regulations, policies and ethics than govern the course.
Integrity – the instructor is aware of the academic integrity needed to facilitate learning.
Technical competences – the learners are made aware of the technical competences needed to successfully reach the learning outcomes.
Ownership – the instructor gives learners the opportunity to share their own learning goals.
Assessment of learning – determining what the learner has learnt and subsequent accreditation:
Goals and objectives – the learners are aware of what is expected of them when they are assessed.
Strategies – clear, well-defined and measurable assessment of learning outcomes suited to the level of the learners.
Grading – grades are given in a fair and transparent manner through appropriate assessment instruments sanctioned by the institution.
Feedback – both instructor and learners are given the opportunity to provide feedback related to grading.
Management – learners have access to their grades and feedback at all times so that they can track their learning progress.
Interaction and community – the exchanges between instructor and learners that build a community that supports teaching and learning:
Fostering – the instructor welcomes learners and gives them the opportunity to communicate and create an online environment that fosters peer learning and engagement.
Management – community building is supported by clear instructions, rules and regulations. While the instructor facilitates engagement, learners are invested with the ownership of community building.
Peer learning – group work and other activities that foster peer learning are encouraged and structured not only to fulfil the learning outcomes, but also to present learners with an opportunity to learn skills and competences that go beyond such outcomes, e.g., digital literacy.
Instructional resources for teaching and learning:
Provision – learning materials are either provided by the instructor or the learners are given enough time to procure such resources. The difference between compulsory and optional resources is to be made clear.
Application – the instructor clearly explains how the resources are going to be applied and utilised.
Entitlement – the instructor makes sure that the resources indicated to fulfil the learning outcomes are open and accessible by all the learners without unwarranted technical, financial or administrative barriers. The use of OER should be encouraged.
Variety – learning resources are varied in terms of the multimedia content and multi-modal delivery channels to cater for the different learning preferences of learners.
Openness – the instructor should give learners the opportunity to suggest their own resources for adoption in the course.
Academic integrity – the instructor promotes best practice in the use of third party resources, including anti-plagiarism practices and sound academic research/writing practices. The use and/or adherence to the creative commons licensing framework is encouraged.
Learner support – learners enabled to achieve their maximum potential:
Instructional support – the instructor explains his/her role in the process.
Academic support – learners know how to obtain such services as mentoring, advice and other skills that support them in achieving the learning outcomes.
Technical support – learners know how to obtain technical support to overcome potential issues in accessing the learning area and achieving the learning outcomes.
Administrative support – learners know how to obtain administrative support to overcome potential issues in accessing the learning area and achieving the learning outcomes.
Technology design – technology is at the service of teaching and learning:
Support – all the utilised technologies and resources support the achievement of the aims and objectives of the instructor and the learning outcomes for learners.
Centricity – all technologies and resources used support a learner-centric rather than an instructor-centric educational approach. The learners must be in control and technology must assist them in achieving the learning outcomes.
Openness – the technical infrastructure used to deliver the teaching and learning is procured and implemented according to open standards and formats that maximise the value for money and the range of options to fulfil the learning outcomes and the academic needs of faculty and learners.
Authentication – authentication at different levels (device, software, virtual learning environment, specific course/learning area) should provide access to a safe and secure teaching and learning environment with the minimum number of steps possible to access the learning areas.
Access – the virtual learning environment/learning area is device/platform agnostic as much as possible, thus accessible over different software platforms, browsers and computing devices. The instructor provides alternative resources if any of these are not easily accessible for technical reasons related to special needs of learners.
Interface – the user interface and navigation in the learning area is simple enough to be conducive to teaching and learning without the need to possess advanced ICT skills and competences.
Investment – the technical requirements of the instructional resources and the virtual learning environment/learning space do not require learners to make any significant new investment in hardware, software and online services to be able to access and use these resources to fulfil the learning outcomes.
Management – learners are aware of the rules, regulations and policies at institutional and at learning community level that govern the use of the technological infrastructure supporting e-learning.
Course evaluation – feedback to improve teaching and learning:
Entitlement – instructors should give learners the opportunity to provide feedback on the whole learning experience. On the other hand, instructors should also be able to provide their feedback within their organisation.
Assessment – learners should have access to their grades and the course material after the closure of the course (depending on the institution’s access policies). The final grades should be provided within a reasonable timeframe after the closure of the course.
Resolution – all pending issues between the instructor and the learners are resolved.
Archiving – the instructor makes sure the course/learning area resources, texts, communication, etc., are backed-up or archived (in line with the institution’s access policies) in a safe and secure way.
Instructional design cycle:
Academic review – the instructor and the organisation review the course description, the experience gathered, and the evaluation given.
Technical review – the instructor, with the relevant technical unit in the organisation, reviews the performance of the technical infrastructure used to deliver teaching and learning.
Administrative review – the instructor, with the relevant administrative unit/s in the organisation, reviews the administrative processes supporting the delivery of teaching and learning.
This comprehensive rubric is being published under the following Creative Commons Licence: Attribution ShareAlike 4.0 International (CC BY-SA 4.0). This grant of the licence does not claim to cover the four rubrics individually.
Recommended actions in support of instructional design for e-learning in higher education
The following recommendations are compiled from the literature review and the development of the comprehensive rubric:
Teaching and learning should dictate the technological implementation of supporting tools and facilities, not the other way round.
An educational institution wishing to provide e-learning opportunities needs a clear vision and a strategy sustaining e-learning from an academic, technological and administrative point of view.
The use of e-learning pedagogies should be promoted as an official part of the professional work of academics, with tangible incentives and rewards that can take different forms, including professional recognition, financial and material rewards.
An educational institution wishing to implement e-learning needs to invest in academic training and support, in technical support and a sound technical infrastructure, and in administrative support.
An educational institution should take time in getting to know the views and needs of its faculty and students.
A VLE should not be utilised as a simple online repository of content or for document management.
E-learning should attempt to benefit from the affordances provided by technology as guided by the digital pedagogies and experience with e-learning, especially providing students with an element of control over the pace, time and path of study. Otherwise, it simply serves as an extension of traditional teaching practices.
Student-instructor and student-student interaction, collaboration and work through the VLE (community building).
A VLE alone does not provide a complete range of tools to facilitate online learning; therefore, complementary tools such as social media should be sought.
E-learning, with MOOCs for example, is an opportunity to explore micro-credentialing and accreditation of online learning.
The analysis of the rubrics has confirmed the myriad of possibilities provided by technology when applied to teaching and learning. Indeed, the resulting comprehensive rubric is rather long, and its components may indeed contribute to the welcome conclusion of the long-standing debate on whether e-learning is as rigorous and effective as traditional face-to-face environments (Casey, 2008; Jung and Latchem, 2012). Such a rubric will surely support faculty in the ever-increasing implementation of e-learning (Gaebel, 2015).
The active involvement of students in the learning process, rendered possible by technology, is well catered for in the comprehensive rubric. The early emphasis of technology in e-learning has given way to more credence in the pedagogical benefits (Garrison and Anderson, 2011; McLoughlin and Lee, 2010; Rovai, 2004).
The lack of common definitions of e-learning and its constituents is notable in the chosen rubrics, but there is nevertheless a common approach: empowering the educator to empower the student in an online environment that promotes learning.
Digital literacy of faculty and time pressures remains a challenge and even though technology is available, the application of the elements listed in the comprehensive rubric requires a level of digital competence from instructors (Conole and Alevizou, 2010; Grove, 2017; Johnson et al., 2015).
MOOCS, OER, BOYD, social elements, artificial intelligence, augmented and virtual reality are acquiring more space and attention in education, and the comprehensive rubric must take into consideration these new instructional approaches and updated digital pedagogies (De Freitas and Conole, 2010).
The tensions created by the implementation of technology in HE, especially the changing role of educators and students brought by more social, more ubiquitous and more open learning spaces, will surely bring to light any gaps between the planned and the actual implementation of e-learning. It is up to the educator, as a professional, to mind these gaps and bridge them.
Limits of scope
It is not within the scope of this paper to review quality assurance processes and administrative components, but to propose a rubric for course design and self-review of faculty and HE institutions for a better alignment with what is regarded as current standard best practice.
Suggestions for further research
The proposed comprehensive rubric does not provide a scale for assigning points when applied, thus giving a weighting to the elements of the rubric perceived as more important than others. There are other rubrics on e-learning by reputable HE institutions that could be included in the analysis. The litmus test for the rubric is its application in real-life situations. This is an excellent opportunity for follow-up research analysing the outcomes of its application.
The criteria in the rubrics by the three educational institutions and one commercial organisation under study
Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C. and Ananthanarayanan, V. (2017), “NMC horizon report: 2017 higher education edition”, New Media Consortium, Austin, TX.
Barr, R.B. and Tagg, J. (1995), “From teaching to learning: a new paradigm for undergraduate education”, Change, Vol. 27 No. 6, pp. 13-25.
Beetham, H. and Sharpe, R. (2013), Rethinking Pedagogy for a Digital Age: Designing for 21st century learning, Routledge, London.
Bozkurt, A., Akgun-Ozbek, E., Yilmazel, S., Erdogdu, E., Ucar, H., Guler, E. and Aydin, H. (2015), “Trends in distance education research: a content analysis of journals 2009-2013”, International Review of Research in Open and Distributed Learning, Vol. 16 No. 1, pp. 330-362.
California State University (2013), “CSU QOLT instruments: Non-awards versions”, available at: http://courseredesign.csuprojects.org/wp/qolt-nonawards-instruments/ (accessed 20 October 2016).
Casey, D.M. (2008), “A journey to legitimacy: the historical development of distance education through technology”, TechTrends: Linking Research & Practice to Improve Learning, Vol. 52 No. 2, pp. 45-51.
Christie, B. (2014), CSU Quality Online Learning and Teaching (QOLT) Evaluation Instrument Sections and Objectives, California State University, available at: http://qolt.sfsu.edu/sites/default/files/QOLT2-pgnumberslogoCore24CClicense111715.pdf (accessed 20 November 2016).
Conole, G. and Alevizou, P. (2010), “A literature review of the use of web 2.0 tools in higher education”, A report commissioned by the Higher Education Academy (HEA), Milton Keynes.
Daradoumis, T., Bassi, R., Xhafa, F. and Caballe, S. (2013), “A review on massive e-learning (MOOC) design, delivery and assessment”, Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, Compiegne, 23-38 October, doi: 10.1109/3PGCIC.2013.37.
De Freitas, S. and Conole, G. (2010), “Learners’ experiences: how pervasive and integrative tools influence expectations of study”, in Sharpe, R., Beetham, H. and De Freitas, S. (Eds), Rethinking Learning for the Digital Age: How Learners Shape their Own Experiences, Routledge, London, pp. 15-30.
Fenrich, P. (2008), “Instructional strategies”, in Hirtz, S. and Harper, D.M. (Eds), Education for a Digital World: Advice, Guidelines, and Effective Practice From Around the Globe, Commonwealth of Learning, Vancouver, pp. 309-320.
Frydenberg, J. (2002), “Quality standards in eLearning: a matrix of analysis”, The International Review of Research in Open and Distributed Learning, Vol. 3 No. 2.
Gaebel, M. (2015), “E-learning in European higher education institutions – results of two EUA studies in 2014 and 2015”, available at: http://bologna-yerevan2015.ehea.info/files/Michael%20Gaebel.pdf (accessed 20 December 2016).
Gamage, D., Fernando, S. and Perera, I. (2015), “Quality of MOOCs: a review of literature on effectiveness and quality aspects”, Ubi-Media Computing (UMEDIA), 8th International Conference, 24-26 August, Colombo, pp. 224-229.
Garrison, D.R. and Anderson, T. (2011), E-learning in the 21st Century: A Framework for Research and Practice, Taylor & Francis, New York, NY.
Glance, D.G., Forsey, M. and Riley, M. (2013), “The pedagogical foundations of massive open online courses”, First Monday, Vol. 18 No. 5, pp. 1-12.
Govindasamy, T. (2002), “Successful implementation of e-learning pedagogical considerations”, Internet and Higher Education, Vol. 4 No. 2, pp. 87-299.
Graham, C.R., Woodfield, W. and Harrison, J.B. (2013), “A framework for institutional adoption and implementation of blended learning in higher education”, Internet and Higher Education, Vol. 18 No. 3, pp. 4-14.
Graham, C.R., Cagiltay, K.C., Lim, B., Craner, J. and Duffy, T. (2001), “Seven principles of effective teaching: a practical lens for evaluating online courses”, available at: www.technologysource.org/article/274/?utm_content=buffere64be&utm_source=buffer&utm_medium=twitter&utm_campaign=Buffer (accessed 20 October 2016).
Grove, J. (2017), “The teaching survey 2017: results and analysis”, Times Higher Education (THE), available at: www.timeshighereducation.com/features/the-teaching-survey-2017-results-and-analysis (accessed 16 February 2017).
Guri-Rosenblit, S. and Gros, B. (2011), “E-learning: confusing terminology, research gaps and inherent challenges”, International Journal of E-Learning & Distance Education, Vol. 25 No. 1, pp. 1-17.
Hoffman, R. (2015), “Social networks will help education connect says Reid Hoffman”, available at: www.wired.co.uk/article/wired-world-2016-reid-hoffman (accessed 20 October 2016).
Hogg, S. (2013), “An informal use of Facebook to encourage student collaboration and motivation for off campus activities”, in Mallia, G. (Ed.), The Social Classroom: Integrating Social Network Use in Education, Information Science Reference, Hershey, PA, pp. 23-29.
Johnson, L., Adams Becker, S., Estrada, V. and Freeman, A. (2014), “NMC Horizon report: 2014 higher education edition”, The New Media Consortium, Austin, TX.
Johnson, L., Adams Becker, S., Estrada, V. and Freeman, A. (2015), “NMC horizon report: 2015 higher education edition”, The New Media Consortium, Austin, TX.
Jung, I. and Latchem, C. (2012), Quality Assurance and Accreditation in Distance Education: Models, Policies and Research, Routledge, New York, NY.
McLoughlin, C. and Lee, M.J. (2010), “Personalised and self-regulated learning in the web 2.0 era: international exemplars of innovative pedagogy using social software”, Educational Technology, Vol. 26 No. 1, pp. 28-43.
Means, B., Murphy, R. and Bakia, M. (2013), “The effectiveness of online and blended learning: a meta-analysis of the empirical literature”, Teachers College Record, Vol. 115 No. 3, pp. 1-47.
Nagel, L. and Kotzé, T.G. (2010), “Supersizing e-learning: what a CoI survey reveals about teaching presence in a large online class”, The Internet and Higher Education, Vol. 13 No. 1, pp. 45-51.
Pankin, J., Roberts, J. and Savio, M. (2012), “Blended learning at MIT”, available at: http://web.mit.edu/training/trainers/resources/blended_learning_at_mit.pdf (accessed 8 January 2017).
QualityMattersTM (2014), “Higher Ed course design rubric”, available at: www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric (accessed 20 November 2016).
Ronan, A. (2015), “Whatever happened to course management systems?”, available at: www.edudemic.com/whatever-happened-course-management-systems/ (accessed 20 December 2016).
Rovai, A.P. (2004), “A constructivist approach to online college learning”, Internet and Higher Education, Vol. 7 No. 2, pp. 79-93.
Tomte, C. and Sutherland Olsen, D. (2014), “Exploring quality in teaching and learning with ICT: a qualitative study”, Proceedings of the European Distance and E-Learning Network 2014 Research Workshop, Oxford, pp. 213-220.
University of Illinois (2015), “Quality online course initiative”, available at: www.ion.uillinois.edu/initiatives/qoci/rubric.asp (accessed 7 January 2017).
University of Malta (2015), “Minimum standards for study-units in the VLE”, available at: www.um.edu.mt/vle/staff/minimumstandards (accessed 10 January 2017).
Young, J.R. (2002), “‘Hybrid’ teaching seeks to end the divide between traditional and online instruction”, Chronicle of Higher Education, Vol. 48 No. 28, pp. A33-A34, available at: https://eric.ed.gov/?id=EJ645445 (accessed 10 January 2017).
California State University, Chico (2016), “Exemplary online instruction”, available at: www.csuchico.edu/eoi/index.shtml (accessed 5 January 2017).