Abstract
Purpose
In response to coronavirus disease 2019, California State University Long Beach (CSULB) announced mandatory online course conversions on March 12, 2020. The College of Business designed a Student Online Learning Experience Survey to explore learners' experience, needs, expectations and challenges in the online learning environment.
Design/methodology/approach
The time-sensitive survey questions were administered using Qualtrics with Institutional Review Board approval. The authors used 5-point Likert scales to rate students' experience and satisfaction and performed statistical analysis. They assessed students' written comments to further corroborate statistical findings.
Findings
The results identify students' satisfaction are highly correlated to content coverage and interaction of online learning technologies. A combination of BeachBoard, Zoom, e-mails and publisher's website is valued most by the learners. Project-based experiential design is ranked #1 by graduate students. Noticeably, the upward trend of satisfaction with online modality from sophomore to senior is probably attributable to learners' maturity and number of years studied at CSU system. Overall, students generally dislike proctoring devices due to concerns of privacy, inequalities, mental stress, etc.
Practical implications
The evidence-based results offer innovative pedagogical recommendations for business education in higher education.
Originality/value
While prior studies examine student perceptions and satisfaction within the online education system, the study aims to deeply investigate the students' experience after a large-scale two-week institutional emergency course conversion mandate. This study systematically reviews students' experience with four aspects of online learning: (1) the adequacy of instructional designs; (2) the effectiveness of technology; (3) the appropriateness of the online learning material and (4) the integrity of online assessment and testing tools.
Keywords
Citation
Chen, C., Landa, S., Padilla, A. and Yur-Austin, J. (2021), "Learners' experience and needs in online environments: adopting agility in teaching", Journal of Research in Innovative Teaching & Learning, Vol. 14 No. 1, pp. 18-31. https://doi.org/10.1108/JRIT-11-2020-0073
Publisher
:Emerald Publishing Limited
Copyright © 2021, Cindy Chen, Sabrina Landa, Aivanna Padilla and Jasmine Yur-Austin
License
Published in the Journal of Research in Innovative Teaching & Learning. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
1. Introduction
The evolvement of web-based technology, growing competition among higher education institutions, and learner acceptance of distance learning has fostered widespread support for the online teaching and learning paradigm (Dykman and Davis, 2008; Beaudoin, 2016). According to the US News and World Report (2019), over 100 business graduate programs have offered 100% online degrees. Despite this growth, a majority of US higher education institutions were ill-prepared to shift abruptly from traditional face-to-face to online modalities in March 2020 as a proven means to restore an otherwise halted education system (Dhawan, 2020).
Consequently, Spring 2020 was a tumultuous time for educators and learners alike. The risk and uncertainty caused by the coronavirus disease 2019 (COVID-19) outbreak forced higher education administrations to close campuses and sent faculty members scrambling to move their courses online. The support and availability of related online technologies enabled leadership at California State University Long Beach (CSULB) to forge ahead into “the great unknown” of emergency online alternative mode of instruction. Online course design typically requires time-consuming adjustments in instructional design (Grandzol and Grandzol, 2006), learning assessment (Dykman and Davis, 2008), communication (Gaytan and McEwen, 2007) and accessibility of tech support. Nonetheless, inexperienced faculty members urgently converted courses online within an unprecedented two-week timeframe.
To gauge the readiness of such online conversion, the College of Business (COB) Office of Accreditation collaborated with the Faculty Digital Learning Support Group in Spring 2020 to design a Student Online Learning Experience Survey. The student-centered survey focuses on “learners' voices” during the pandemic aiming to assess instructional modalities, detect advantages and disadvantages of teaching modalities, and identify factors that influence students' satisfaction with online education. Most importantly, the open-ended comment sections solicit learners' responses to explain analytical results and enable exploration of design effectiveness, impactful online learning environment characteristics and innovative pedagogies.
Our study contributes to extant online learning in higher education and adds to the current body of research in the following ways: (1) The student-centered survey assesses learners' experience, identifies needs and obstacles, and evaluates expectations, (2) The time-sensitive results enable us to evaluate the readiness of COB online course conversion, (3) The demographic data enable us to explore what influences students' online learning experience and (4) Higher education institutions can emulate our evidenced-based findings to promote and foster innovative pedagogical advancement and support students' lifelong learning mindset.
The remainder of the paper is structured in three sections. Section 2 describes our survey design and Section 3 discusses the results along with related prior studies evidence. Section 4 concludes and provides suggestions for higher education institutions.
2. Survey design
Following the March 12, 2020 emergency transition due to COVID-19, it was critical for the COB to assess its readiness to deliver quality online courses. Furthermore, the CSU system has announced the continuance of online instruction through at least Summer 2021. Therefore, the feedback we derive from our survey is expected to provide practical suggestions for administrators, faculty members and academic technology staff members to prepare for the academic year 2020–2021.
Consequently, we designed an Institutional Review Board approved survey to assess the COB's online course conversion readiness. The survey explores the learners' experience, needs, expectations and challenges in an online learning environment. The survey includes two parts – Demographic and Online Delivery Modality. The demographic section covers questions related to age, years in program, number of units taken, major options, ethnicity and employment status. A Likert scale question-set rates the learners' experience in four aspects of online learning: (1) the adequacy of instructional design (2) the effectiveness of technology; (3) the appropriateness of the online learning material and (4) the integrity of online assessment and testing tools.
The Qualtrics survey link was emailed to students on April 30, 2020 and response collection ended on May 28, 2020, marking the end of the Spring 2020 semester. 854 students participated in this survey. Among these respondents across the COB's eight options: Accountancy, Finance, Human Resource Management, Information Systems, International Business, Management, Marketing and Supply Chain Management, 16.63% (142) are graduate students and 83.37% (712) are undergraduate students. Most respondents in the undergraduate program are juniors (30.91%) and seniors (38.64%). 23.06% of our respondents took less than three courses (11.65%, 3 or 6 units; 11.41%, 9 units) in Spring 2020 and a super majority took four courses (42.24%, 12 units) and five or more course (34.71%, 15 or more units).
As a commuter school, the majority of our students commute to a job or other commitments. Our survey reports that 20.14%, 22.95%, 12.76% and 25.29% of students work full time, up to 20 h per week, more than 20 h per week or are not employed, respectively. Sadly, 18.85% of respondents lost their jobs due to COVID-19. Suggesting, prior to the pandemic, over 80% of our surveyed students were employed.
3. Survey results
3.1 Instructional modality
Our survey starts by assessing student satisfaction with hybrid/online/face-to-face learning. Students rate their opinions about instructional design options on a 5-point Likert scale ranging from 1 (strongly dislike) to 5 (strongly like). 645 students rate all three modalities: face-to-face, hybrid and online. As presented in Table 1, our results indicate that the average score ranges from 2.849 (dislike to neutral) for 100% online, 3.374 (neutral to like) for hybrid, to 4.088 (like to strong like) for face-to-face. The evidence indicates students strongly favor face-to-face instructional format.
We further classify the advantages of online learning design versus face-to-face. As exhibited in Table 1, the following factors receive a Likert score above 3: (1) Online submission processes are better in online courses (3.562), (2) Online quizzes and/or exams are better in online courses (3.364), (3) The recorded lectures, audios or videos are better in online courses (3.145) and (4) Online assignments and/or projects are better in online courses (3.018). However, students rate the following categories with an average Likert score below 3: (1) The interaction of various learning technologies is better in online courses (2.754) and (2) The content coverage is better in online courses (2.479). This evidence suggests learners are not satisfied with the technology usage and content coverage of online courses. We relate these factors to students' overall satisfaction in hybrid and online courses and attain the correlation coefficient findings in Table 1.
Noticeably, the listed factors are more correlated to students' satisfaction in online than in hybrid courses. In addition, content design and learning technologies are heavily weighted in students' satisfaction levels suggesting students have elevated expectations on content and technology in online courses. Our evidence is consistent with the study by Bao (2020) which posits high relevance between quality online course design and learning outcomes for students.
3.2 Technology delivery
Accordingly, we continue in our search for which aspects related to online course technologies are most important to students. There are many technologies available for online content delivery, including learning management systems (i.e. Canvas, Blackboard, Moodle, D2L), Zoom, Google's education suite, Microsoft Teams, WebEx, Skype, Adobe Connect and more (Dhawan, 2020; Basilaia and Kvavadze, 2020). However, such technology requirements may create challenges for students including, but not limited to, low Internet bandwidth, unaffordable systems requirements, shortage of laptop/tablet devices and audio/visual interruptions. (Dhawan, 2020; Filus et al., 2019). For instance, users are now all too familiar with the phrases, “can you see my screen?” and “can you hear me now?”.
Furthermore, Blau et al. (2016) document students in qualitative majors experience greater challenges with technology than their peers in quantitative majors. Similarly, Buabeng-Andoh (2018) reports that student attitudes about technology and its usefulness determine their willingness to adopt. Hence, assisting qualitative majors with technology is essential to help students excel in an online learning environment. We therefore expect undergraduate quantitative and qualitative majors to have differing opinions on course technologies. Extending the same rationale, we investigate whether undergraduate and graduate students exhibit different technology preferences. Meanwhile, we conjecture that freshman, sophomore and those with less CSU academic experience will have more difficulty adapting to online technology requirements. Our aim is akin to the study by Blau et al. (2016) of students' experience of online instructional course design and the adoption of technology. Our survey yields the following comments about the sufficiency of CSULB technologies used in online content delivery:
3.2.1 More the better
A combination of all resources should be utilized to provide students options for learning…
A combination of methods should be used for content delivery and communication… limiting the way [students]…receive their instruction and communication would enhance these struggles.
As long as the professor is clear on the expectations and provides a detailed outline, then the delivery of the content is easy.
3.2.2 Less is more
Having a combination of all three delivery technology is overwhelming!
It gets confusing having multiple classes use multiple third-party applications.
3.2.3 Pros and cons of technologies
Zoom is not as effective to me…plus it has a lot of privacy issues so I do not think we should be forced to use it.
Zoom was the best because the professor was able to give you the due dates for certain course assignments right there in front of you.
…When everyone uses their audio and camera [on Zoom] it is…a huge distraction…when it is not used the professors have a hard time recognizing when students are trying to ask a question. Not to mention that Zoom has technical difficulties.
BeachBoard is a good communication medium. Zoom is good for group meetings and short meetings with professors. Still there is nothing compared to face to face class meetings…
BeachBoard will give you all the information necessary but when it comes to asking questions that is where it fails…
Email is okay, but I do often tend to forget to check my email…
The book publisher's websites tend to have good educational tools, but…anyone can read a book…I want a professor…to tell me how a concept will connect with the real world.
BeachBoard, Email, and Zooms (depending on the material) were extremely effective. Just do not require students to show video during lectures it's distracting and uncomfortable…
Our learners' written responses suggest that effective use of technology requires instructors to be skilled in its usage toward synthesizing meaningful interactions, mindful participation and instructional explanation.
Relatedly, Table 2 assesses students' satisfaction of the combination of emails, BeachBoard, Zoom and publisher's website (if applicable). From 1 “Strongly Dissatisfied” to 5 “Strong Satisfied”, students rate their top choice.
Contrary to Blau et al. findings (2016), our survey results do not discern a major difference between opinions on learning technologies between quantitative majors (average Likert score: 3.90) and qualitative majors (average Likert score: 3.86). However, we examine differences in undergraduate and graduate students' experience using online learning technologies. Graduate students report a higher average Likert score of 4.27 which suggests having more work experience leads learners to embrace tech-based learning platforms. Similar results in the study by Agarwal and Kaushik (2020) focus on mature medical students who favor online technologies which allow the uninterrupted advancement of their studies. Our evidence also demonstrates an upward average satisfaction scores from 3.49, 3.83 to 3.94 for sophomore, junior and senior, respectively, implying students who have more experience in the CSU academic system adapt better to online learning technologies.
A distinct feature of online education is its convenience allowing for flexibility and accessibility for nontraditional learners (Christensen and Eyring, 2011; Dhawan, 2020; Song et al., 2004). This convenience aspect amplifies students' satisfaction or preferences in online learning systems (Cole et al., 2014; Kerby et al., 2014). Corroborating these prior studies, our full-time employed students report the highest 4.05 average Likert score, indicating higher satisfaction with online learning technologies. Our finding further solidifies “convenience” as a key advantage of online education.
Furthermore, students aged 28–40 years are more satisfied than their younger peers with respect to the application of online technologies. However, students older than 40 years express less satisfaction with technologies as indicated by the lowest average Likert score of 3.76. Most of the results displayed in Table 2 concur with our hypothesis that learner maturity (age, work experience, graduate vs undergraduate) affects the internalization of positive or negative perceptions about online learning technologies.
3.3 Content coverage
Effective course content is the most prominent factor influencing students' learning experience, satisfaction and the quality of online education (Sebastianelli et al., 2015). If we view the combination of technologies such as BeachBoard, Zoom and publisher websites as “bones” that support online learning, well-designed content is considered the “meat” that informs online learning pedagogies. Thus, our survey elicits learner opinions on eight online learning material options. Students rate each item from 1 (strongly dissatisfied) to 5 (strongly satisfied).
These results indicate students' satisfaction with online materials depends on the individual course. Furthermore, the effective delivery of learning content depends on effective online interactions. Relatedly, Fedock et al. (2019) postulates faculty members should be trained in the use of social media as a means of online communication. Faculty agility and adaptability are also skills needed in online content design (Buabeng-Andoh, 2018; Gilbert, 2019). For example, when delivering online lectures, instructors must be mindful of the instructional pace because they are no longer able to ascertain facial expressions as a means of determining learner comprehension (Bao, 2020). Thus, if together the course technologies (bones) and well-designed content (meat) make up the framework of an online course, instructors are considered the “blood-bearing arteries” that deliver life and purpose to the course connecting such pedagogical elements together. Furthermore, technology shall not overpower the “human element” in the online teaching environment (Serdiukov, 2001). Therefore, faculty members are challenged to facilitate engaging interactions and provide timely feedback. Relatedly, students share the following expectations:
[online course materials are appropriate] …when executed well…
A lot of the success of e-learning falls on the professor holding students accountable to participate.
The instructional methods used are only as good as the instructor's ability to connect concepts to the real world.
Table 3 identifies the top three online course materials ranked by undergraduate students are: (1) Videos of instructors using learning glass technology; (2) Animated PowerPoint with instructor narratives, and (3) Zoom synchronous class recordings. Surprisingly, YouTube videos are not popular among students. Finally, although students express that team projects requiring virtual peer meetings are challenging and require instructors to fairly assess contributions, project-based experiential course content design ranks #1 by graduate students while undergraduate students do not prefer this hands-on approach to learning.
3.4 Integrity of assessment
In addition to skillful selection of technology combinations and content delivery, seamless integration of proctoring tools increases confidence in the integrity and validity of online assessment (Andrade et al., 2020). While understanding the necessity of upholding standards to ensure academic integrity during assessments, our students express concerns regarding online testing technologies. Table 4 presents student opinions about the integrity of various testing technologies.
We are not surprised by the results illustrated in Table 4. From the available options, students favor option 1: Quiz + Open book/Open Notes. Prior studies (Bao, 2020; Dhawan, 2020) suggest that the necessity of monitoring students' performance may be higher due to increased distractions and lack of disciplined learners in online courses (Bao, 2020; Dhawan, 2020). However, stakeholders in the online education community warn against rushing to judgment about “cheating” which may lead to biased opinions against learners. Further review of student comments helps explain their reluctance toward online proctoring technologies.
3.4.1 Stress and anxiety
Classes that use lockdown browsers and monitor you is intimidating and makes the test taking process harder for students [adding] stress…
Having a Respondus monitor webcam is… excessive…taking exams at home is… very distracting and… implementing a webcam… creates unnecessary anxiety.
Hate the lockdown browser with webcam. It constantly says I should not move…making it more difficult for me to test.
… any type of monitoring activity and/or Lockdown browser adds stress and pressure [and can] even make students do worse.
3.4.2 Trust and security concerns
Didn't like installing additional software on my phone and computer.
Don't like Proctor U [specifically, giving] access to my personal information…and providing them access to my desktop.
3.4.3 Financial burden
… I do not have any finances to purchase [laptop] upgrades or a webcam as I am unable to find a job and behind on rent during this pandemic.
3.4.4 Doubt on proctor extensions to prevent cheating
Surveillance software can rather easily be circumvented by those intent on cheating…,
I cannot see any way to prevent people from cheating on tests when they take the test at home….
…I highly doubt BeachBoard quiz with lockdown browser and zoom monitoring prevents students from cheating…
Most commendably, students offer the following suggestions to ensure assessment integrity:
… an open book and note policy BUT focusing on concepts…the best way is by keeping the test open which allows greater room to present harder questions that require more thinking.
when in a real world business setting or professional environment will I be asked to work challenging problems from memory… I have found that any and all available resources when used in a testing environment helps facilitate true learning of difficult problems and concepts….
Exam based on textbooks is …old fashion. Voice narrated PPT presentations and case studies based on the topics of the textbook could be better option…[which] develop strategic thinking skills.
Using professors' own questions (instead of those pulled from a test bank), plus the allowance of notes and a book, are the best tests online.
…Beach Board Quiz with open book and open notes is motivating because taking a test is already stressful enough…
3.5 Instructor engagement and competency
Given the continued CSU mandate to “keep teaching” online through Summer 2021, instructor competency is another focal point of our survey which analyzes correlations between the five instructional activities and students' satisfaction of three online instruction modes: 100% Asynchronous, Weekly Synchronous Meetings and Hybrid.
The main findings in Table 5 include: (1) Top two instructor competencies “Instructors provide clear guidance on how to participate in online learning activities” and “Instructors keep the class engaged to achieve learning outcomes” are correlated across all three instructional design options. This finding suggests the most impactful aspects of the online learning experience are the students' ability to understand directives and stay engaged in the learning process. We also see from previous results that students prefer face-to-face instruction. Together, these findings emphasize the instructors' burden to stimulate participation, possibly drawing from proven aspects of face-to-face peer-to-peer and instructor–student interactions, to foster an engaging online learning environment. Effective, timely communication is also highly valued by students. Our results therefore agree with Adair's (2013) suggestions that online learners value: clearly defined assignments, instructor excellence and faculty responsiveness. Hence, our students' opinions of COB instructors' competence identify specific faculty professional development needs which suggests an impactful plan for future training. Additionally, students generally compliment instructor performance during the emergency transition period. Specifically, students comment instructors are “very accommodating”, “reassuring” and “helpful trying to make the best of the situation”. However, some students note differences in approaches as suggested by this comment below:
a few professors have adapted accordingly but a few others have changed the assignments to new assignments that we were not preparing for all semester and became more difficult… this may not entirely reflect the professor's ability/effort, but also their access to resources or inability to prepare in a timely manner.
Also shown in Table 5, students indicate that instructors, in some cases, make a considerable effort to keep students informed about due dates and timing for learning activities (average Likert score: 3.90). Further suggesting that many instructors readily adapt to online learning and make it a manageable experience by providing clear guidance on how to participate in online learning activities (average Likert score: 3.68) and feedback in a timely fashion (average Likert score: 3.71).
We understand that higher levels of engagement contribute to students' academic performance (Astin, 1993). Therefore, the challenge for instructors is to ensure learners are engaging in online learning activities. Relatedly, our survey solicits students' opinions on whether instructors can keep the class engaged. Some related excerpts are:
keeping students engaged in lecture has been the toughest obstacle for many professors in virtual classes,
I feel class presentation went on smoothly, but I do think class engagement decrease [online].
They can do so by teaching and not just following PowerPoints
Some professors record lectures and that makes it easier to go back and understand material .. Most professors however … really seems like they're talking to themselves and do not really engage us in the learning.
I think some of my professors have been very animated on and off the screen meaning the transition to online lecture was not that bad ….
Finally, our survey results demonstrate our commitment to probe deeply into our learners' experience in the online learning environment and advance aligned faculty training going forward.
4. Conclusions
Innovations in education have come a long way since the US Department of Education (2004) reported, “…there are innovations in instructional techniques or delivery systems, such as the use of new technology in the classroom…”. However, US higher education institutions proved inadequately prepared to adapt to the technological and pedagogical innovations needed to respond to the dire need to shift programs online due to the COVID-19 outbreak. Converting face-to-face classes to a virtual course design requires skillful pedagogical adjustments and proper technology support that inexperienced faculty members were unaware of at the start of the emergency shift online. Hence, we offer the following suggestions from the learners' perspective to facilitate integrating content and technology seamlessly in an online learning environment.
To understand the student experience with technology in distance learning, our evidence signifies the incremental changes of satisfaction scores across grade levels. These range from 3.49, 3.83 to 3.94 for sophomore, junior and senior, respectively, measuring the satisfaction with a combined use of BeachBoard, Zoom, emails and publisher's website in the online course design. This upward trend of satisfaction is attributable to learners' maturity and number years of study in the CSU system which is further supported by graduate learners' even higher Likert score of 4.2. Nonetheless, we find students older than 40 years likely experience more challenges navigating tech-based learning platforms as evidenced by a lower 3.76 Likert score. Finally, consistent with the emphasis on convenience in online education (Christensen and Eyring, 2011; Dhawan, 2020; Song et al., 2004; Cole et al., 2014; Kerby et al., 2014), our full-time employed students report a 4.05 average Likert score, indicating higher satisfaction with online learning technologies.
In addition to concurring with the imperative role of online course technology, our results show students rank the following more interactive learning materials as the most appropriate in online content design: (1) Instructor lectures using learning glass technology, (2) Animated PowerPoints with audio narratives and (3) Zoom class recordings. Also, despite challenges with virtual meetings and group freeloaders, graduate students value project-based experiential design the most whereas undergraduates seemed less enthused with such learning-by-doing assessments. Surprisingly, we find in virtual settings, YouTube videos are not a popular educational tool for our Generation Z learners (Global Research and Insights, 2018).
Despite many innovative technological and content-related features in online education, instructors and learners still have reservations about upholding academic integrity (Spalding, 2012). For instance, our students express enormous concern regarding online proctoring technologies such as Respondus LockDown Browser and monitor, webcams, Proctor U, Zoom monitoring and publisher testing platforms. The themes of these student responses include stress and anxiety, privacy and trust, effectiveness and financial burdens. Admirably, students do offer suggestions for reimagining online assessments including offering open note/text exams, alternative assessments, challenging and critical thinking style questions, and ultimately forgoing test bank questions.
To implement innovative changes in the online education system and achieve versatility and sustainability over time, higher education must address the critical aspects of online teaching and learning found in our study. Our survey offers a way to include “learners' voices” while shaping academic, administrative and legislative policies on distance learning programs. We offer practical guidance to instructors and technology support services regarding the increased need for training and support of first year and older students as evidenced by the differential satisfaction results of other mature and working students. We further provide guidance on developing appropriate and engaging online learning content to help faculty navigate away from static PowerPoint lectures and take advantage of innovative solutions. We glean from students' compliments of COB faculty members that exert considerable efforts during this challenging emergency online conversion period the importance of timely feedback and sound pedagogy. However, we acknowledge the pressing needs to support further faculty professional development and strike a balance between academic integrity and student well-being going forward. Ultimately, our findings illuminate ways to innovate online pedagogies toward building a quality learning experience. We hope these advances will foster the lifelong learner mindset we aim to see in our students as they opt to continue conveniently participating in our evolving online education system post-COVID-19.
Instructional modality
| Based on your overall education experiences, what is your opinion about the following instructional formats? | |
|---|---|
| 1: Strongly dislike to 5: Strongly like | Average |
| 1) Face-to-face is my preferred method of learning | 4.088 |
| 2) Online is my preferred method of learning | 2.849 |
| 3) Hybrid is my preferred method of learning | 3.374 |
| What is your opinion about the advantages of online learning compared to face-to-face learning? | |
|---|---|
| 1: Strongly disagreed to 5: Strongly agreed | Average |
| 1) The content coverage is better in online courses | 2.4792 |
| 2) The interaction of various learning technologies is better in online courses | 2.7545 |
| 3) The recorded lectures audios or videos are better in online courses | 3.1449 |
| 4) Online quizzes and/or exams are better in online courses | 3.3643 |
| 5) Online assignments and/or projects are better in online courses | 3.0180 |
| 6) Online submission processes are better in online courses | 3.5616 |
| We measure correlations between ratings of the above two survey questions | Online | Hybrid |
|---|---|---|
| 1) The content coverage is better in online courses | 0.7002 | 0.1369 |
| 2) The interaction of various learning technologies is better in online courses | 0.5891 | 0.1729 |
| 3) The recorded lectures audios or videos are better in online courses | 0.5556 | 0.1888 |
| 4) Online quizzes and/or exams are better in online courses | 0.5431 | 0.2202 |
| 5) Online assignments and/or projects are better in online courses | 0.5432 | 0.1471 |
| 6) Online submission processes are better in online courses | 0.5155 | 0.2205 |
Delivery technology
| What is your opinion about the effectiveness of the instructional delivery technologies such as BeachBoard, Zoom, emails and publisher's website? | ||
|---|---|---|
| 1: Strongly dissatisfied to 5: Strongly satisfied | Average | N |
| 1) Undergraduate (Quantitative majors) | ||
| a. Accountancy | 3.84 | 175 |
| b. Finance | 3.90 | 114 |
| c. Information systems | 3.90 | 67 |
| d. Supply chain management | 3.97 | 38 |
| Total average | 3.90 | |
| 2) Undergraduate (Qualitative majors) | ||
| e. Management | 3.75 | 121 |
| f. Human resource management | 4.16 | 58 |
| g. Marketing | 3.80 | 101 |
| h. International business | 3.74 | 42 |
| i. Double major | 3.88 | 11 |
| Total average | 3.86 | |
| 3) Graduate | ||
| a. EMBA | 4.10 | 64 |
| b. AMBA | 3.29 | 29 |
| c. SMBA | 4.67 | 4 |
| d. OMBA | 4.57 | 8 |
| e. MSA, MSIS, MSMA | 4.43 | 12 |
| Total average | 4.27 | |
| 4) Year in program as of Spring 2020 | ||
| a. Freshman | 4.23 | 14 |
| b. Sophomore | 3.49 | 60 |
| c. Junior | 3.83 | 264 |
| d. Senior | 3.94 | 330 |
| 5) Employment status | ||
| a. Full time | 4.05 | 172 |
| b. Part time ≤10 h | 3.79 | 52 |
| c. Part time 11–20 h | 3.74 | 144 |
| d. Part time 21–29 h | 3.73 | 71 |
| e. Part time 30–39 h | 3.87 | 38 |
| f. Not working | 3.79 | 215 |
| g. Just lost job due to COVID-19 | 3.97 | 161 |
| 6) Age | ||
| a. ≤22 | 3.84 | 356 |
| b. 23–27 | 3.82 | 271 |
| c. 28–32 | 4.02 | 95 |
| d. 32–36 | 4.08 | 44 |
| e. 36–40 | 3.93 | 38 |
| f. ≥40 | 3.76 | 48 |
Online material coverage
| What is your opinion about the appropriateness of the following online learning materials? |
|---|
| 1: Strongly dissatisfied to 5: Strongly satisfied |
| 1) PowerPoint – Static format is useful online learning material |
| 2) PowerPoint – Animated format including instructor's narrative audio or written comments is useful online learning material |
| 3) Video using the learning glass format to show an instructor illustrating lecture is useful online learning material |
| 4) External links from YouTube, Lynda or other sources is useful online learning material |
| 5) Recording from a Zoom synchronous class is useful online learning material |
| 6) Publisher complementary videos or other related material is useful online learning material |
| 7) BeachBoard Discussion Forum is useful online learning material |
| 8) Assignment and/or Project with instructions and due dates is useful online learning material |
| 1) Undergraduate | Average | |||||||
|---|---|---|---|---|---|---|---|---|
| (1) | (2) | (3) | (4) | (5) | (6) | (7) | (8) | |
| Quantitative majors | ||||||||
| a. Accountancy | 3.4583 | 3.9384 | 3.9023 | 3.5145 | 3.7552 | 3.4091 | 3.2385 | 3.6370 |
| b. Finance | 3.3452 | 3.7976 | 3.9359 | 3.5679 | 3.6786 | 3.4189 | 3.2000 | 3.6747 |
| c. Information systems | 3.5714 | 3.9821 | 4.0769 | 3.8846 | 3.9643 | 3.5435 | 3.2642 | 3.5091 |
| d. Supply chain management | 3.6667 | 4.0000 | 4.4444 | 4.0000 | 3.9677 | 3.6923 | 3.3448 | 3.9032 |
| Qualitative majors | ||||||||
| e. Management | 3.5842 | 3.9592 | 3.8842 | 3.4100 | 3.7700 | 3.3111 | 3.5417 | 3.8700 |
| f. Human resource management | 3.7600 | 4.1837 | 4.2083 | 3.8200 | 3.7800 | 3.5116 | 3.1429 | 3.6800 |
| g. Marketing | 3.2073 | 3.9512 | 3.9706 | 3.4321 | 3.8415 | 2.9867 | 2.8750 | 3.7317 |
| h. International business | 3.3784 | 3.7368 | 3.9394 | 3.5263 | 3.7105 | 3.3824 | 3.2432 | 3.3684 |
| i. Double major | 3.5556 | 4.2222 | 3.8000 | 3.2222 | 2.6667 | 3.6000 | 3.5556 | 3.1111 |
| 2) Graduate | ||||||||
| a. EMBA | 3.1429 | 3.8148 | 3.8478 | 3.7963 | 3.9038 | 3.4762 | 3.3208 | 3.9245 |
| b. AMBA | 2.7619 | 3.2381 | 3.9000 | 3.3810 | 3.6190 | 3.5263 | 3.0476 | 3.3500 |
| c. SMBA | 3.0000 | 3.0000 | 3.5000 | 3.0000 | 3.6667 | 3.6667 | 4.0000 | 4.0000 |
| d. OMBA | 3.5000 | 3.7500 | 4.6667 | 4.7500 | 4.7500 | 4.5000 | 4.5000 | 4.7500 |
| e. MSA, MSIS, MSMA | 3.5238 | 3.9444 | 4.3889 | 3.8571 | 4.4048 | 4.0000 | 3.7667 | 4.0556 |
| 3) Year in program as of Spring 2020 | ||||||||
| a. Freshman | 3.6667 | 3.8182 | 3.9000 | 4.0833 | 4.1667 | 3.9091 | 3.0833 | 4.5833 |
| b. Sophomore | 3.1923 | 3.7547 | 3.8478 | 3.3922 | 3.8077 | 3.0408 | 2.7600 | 3.1373 |
| c. Junior | 3.4338 | 4.0000 | 4.0493 | 3.5189 | 3.7070 | 3.3218 | 3.2341 | 3.6545 |
| d. Senior | 3.5038 | 3.9351 | 3.9915 | 3.6275 | 3.7947 | 3.3699 | 3.2800 | 3.7192 |
Assessment integrity
| What is your opinion about the integrity of the following online testing technologies? |
|---|
| 1: Strongly dissatisfied to 5: Strongly satisfied |
| 1) BeachBoard Quiz with open book and open notes is a valid testing technology ensuring academic integrity |
| 2) BeachBoard Quiz with LockDown Browser is a valid testing technology ensuring academic integrity |
| 3) BeachBoard Quiz with LockDown Browser + Respondus Monitor Webcam is a valid testing technology ensuring academic integrity |
| 4) BeachBoard Quiz with LockDown Browser + Zoom Monitoring is a valid testing technology ensuring academic integrity |
| 5) Proctor U is a valid testing technology ensuring academic integrity |
| 6) Publisher's online test technology is a valid testing technology ensuring academic integrity |
| (1) | (2) | (3) | (4) | (5) | (6) | |
|---|---|---|---|---|---|---|
| BeachBoard Quiz | x | x | x | x | ||
| Open book/Open Notes | x | |||||
| LockDown Browser | x | x | x | |||
| Respondus Webcam Monitor | x | |||||
| Zoom Monitor | x | |||||
| Proctor U | x | |||||
| Publisher Online Testing | x |
| Average | |||||||
|---|---|---|---|---|---|---|---|
| N | (1) | (2) | (3) | (4) | (5) | (6) | |
| Freshman | 14 | 4.4545 | 4.0909 | 2.4000 | 2.5000 | 2.1250 | 2.1429 |
| Sophomore | 58 | 4.0392 | 2.3878 | 1.8800 | 2.1429 | 1.9091 | 2.5750 |
| Junior | 262 | 4.3505 | 3.1667 | 2.1050 | 2.2475 | 2.3000 | 2.9651 |
| Senior | 326 | 4.1344 | 3.2561 | 2.5498 | 2.6239 | 2.7244 | 3.0405 |
| Graduate program status | 140 | 3.8505 | 3.3061 | 2.9583 | 3.0106 | 3.1111 | 3.2687 |
Instructor engagement
| 1: Strongly dissatisfied to 5: Strongly satisfied | Average |
|---|---|
| What is your opinion about the competency of your instructors in online teaching? | |
| 1) Instructors provide clear guidance on how to participate in online learning activities | 3.6770 |
| 2) Instructors keep the class informed about due dates/time of learning activities | 3.8996 |
| 3) Instructors provide feedback in a timely fashion | 3.7080 |
| 4) Instructors help familiarize the class on how to use the online learning technology, like Zoom, BeachBoard and the publisher's website | 3.5274 |
| 5) Instructors keep the class engaged to achieve learning outcomes | 3.4704 |
| What is your opinion about the adequacy of the following instructional design options? | |
| 1) 100% Asynchronous self-paced study combined with synchronous office hours is an adequate instructional design | 3.2614 |
| 2) Weekly synchronous class meeting is an adequate instructional design | 3.6285 |
| 3) Hybrid as a combination of self-paced online study and synchronous class meetings is an adequate instructional design | 3.6601 |
| We measure correlations between ratings of the above two survey questions | Asyn | Syn | Hybrid |
|---|---|---|---|
| 1) Instructors provide clear guidance on how to participate in online learning activities | 0.3162 | 0.4050 | 0.3278 |
| 2) Instructors keep the class informed about due dates/time of learning activities | 0.2402 | 0.3973 | 0.3253 |
| 3) Instructors provide feedback in a timely fashion | 0.2205 | 0.3508 | 0.3191 |
| 4) Instructors help familiarize the class with how to use the online learning technology like Zoom, BeachBoard and the publisher's website | 0.2799 | 0.3615 | 0.3227 |
| 5) Instructors keep the class engaged to achieve learning outcomes | 0.3468 | 0.3916 | 0.3586 |
References
Adair, D. (2013), “Quality in online learning: what does that mean for the online learner?”, available at: https://evolllution.com/revenue-streams/distance_online_learning/quality-online-learning-online-learner/ (accessed 30 January 2021).
Agarwal, S. and Kaushik, J.S. (2020), “Student's perception of online learning during COVID pandemic”, The Indian Journal of Pediatrics, Vol. 87 No. 7, p. 554.
Andrade, M.S., Miller, R.M., Kunz, M.B. and Ratliff, J.M. (2020), “Online learning in schools of business: the impact of quality assurance measures”, Journal of Education for Business, Vol. 95 No. 1, pp. 37-44.
Astin, A.W. (1993), What Matters in College: Four Critical Years Revisited, Jossey-Bass, San Francisco, CA.
Bao, W. (2020), “COVID-19 and online teaching in higher education: a case study of Peking University”, Human Behavior and Emerging Technologies, Vol. 2 No. 2, pp. 113-115.
Basilaia, G. and Kvavadze, D. (2020), “Transition to online education in schools during a SARS-CoV-2 coronavirus (COVID-19) pandemic in Georgia”, Pedagogical Research, Vol. 5 No. 4, em0060.
Beaudoin, M. (2016), “Issues in higher education – a primer for higher education decision makers”, in Barefoot, B.O. and Kinzie, J.L. (Eds), New Direction for Higher Education, Jossey-Base, Vol. 173, pp. 9-19, M.S. Andrade (Vol. Ed.), issues in distance education.
Blau, G., Pred, R., Drennan, R.B. and Kapanjie, D. (2016), “Correlates of perceived favorability of online courses for quantitative versus qualitative”, Undergraduate Business Majors, Vol. 91 No. 8, pp. 420-426.
Buabeng-Andoh (2018), “Predicting students’ intention to adopt mobile learning: a combination of theory of reasoned action and technology acceptance model”, Journal of Research in Innovative Teaching and Learning, Vol. 11 No. 2, pp. 178-191.
Christensen, C. and Eyring, H. (2011), The Innovative University: Changing the DNA of Higher Education from the inside Out, Jossey-Bass, San Francisco, CA.
Cole, M., Shelley, D. and Swartz, L. (2014), “Online instruction, e-learning, and student satisfaction: a three year study”, International Review of Research in Open and Distance Learning, Vol. 15 No. 6, pp. 111-131.
Dhawan, S. (2020), “Online learning: a panacea in the time of COVID-19 crisis”, Journal of Educational Technology, Vol. 49 No. 1, pp. 5-22.
Dykman, C.A. and Davis, C.K. (2008), “Online education forum part one- the shift toward online education”, Journal of Information Systems Education, Vol. 19, pp. 11-16.
Fedock, B.C., McCartney, M. and Neeley, D. (2019), “Online adjunct higher education teachers' perceptions of using social media sites as instructional approaches”, Journal of Research in Innovative Teaching and Learning, Vol. 12 No. 3, pp. 222-235.
Filus, R.M., Kleijin, R.A., Uijl, S.G., Prins, F.J., Rijen, H.V. and Grobbee, D.E. (2019), “Audio peer feedback to promote deep learning in online education”, Journal of Computer Assisted Learning, Vol. 35 No. 5, pp. 607-619.
Gaytan, J. and McEwen, B.C. (2007), “Effective online instructional and assessment strategies”, The American Journal of Distance Education, Vol. 21 No. 3, pp. 117-132.
Gilbert, M. (2019), “Student performance is linked to connecting effectively with teachers”, Journal of Research in Innovative Teaching and Learning, Vol. 12 No. 3, pp. 311-324.
Global Research and Insights (2018), Beyond Millennials: The Next Generation of Learners, Pearson, available at: https://www.pearson.com/content/dam/one-dot-com/one-dot-com/global/Files/news/news-annoucements/2018/The-Next-Generation-of-Learners_final.pdf (accessed 30 January 2021).
Grandzol, J. and Grandzol, C.J. (2006), “Best practice for online business education”, International Review of Research in Open and Distance Learning, Vol. 7 No. 1, pp. 1-18.
Kerby, M., Branham, K. and Mallinger, G. (2014), “Consumer-based higher education: the uncaring of learning”, Journal of Higher Education Theory and Practice, Vol. 14 No. 5, pp. 42-54.
Sebastianelli, R., Swift, C. and Tamimi, N. (2015), “Factors affecting perceived: earning, satisfaction, and quality in the online MBA: a structural equation modeling approach”, Journal of Education for Business, Vol. 90 No. 6, pp. 296-305.
Serdiukov, P. (2001), “Models of distance higher education: fully automated or partially human?”, Educational Technology Review, International Journal on Educational Technology Issues and Applications, Vol. 9 No. 1, pp. 15-25.
Song, L., Singleton, E., Hill, J.R. and Koh, M.H. (2004), “Improving online learning student perceptions of useful and challenging characteristics”, Internet and Higher Education, Vol. 7, pp. 59-70.
Spalding, E. (2012), “Claire fox: is consumerism bad for education? Manchester liberty league”, available at: https://libertyuom.wordpress.com/2012/04/30/claire-fox-is-consumerism-bad-for-education/ (accessed 28 November 2020).
U.S. Department of Education (2004), “What do we mean by ‘innovation’?”, available at: https://www2.ed.gov/about/offices/list/oii/about/definition.html#:∼:text=Innovation%20is%20based%20on%20curiosity,innovation%20comes%20in%20many%20forms (accessed 28 November 2020).
U.S. News and World Report (2019), “Best online MBA programs”, available at: https://www.usnews.com/education/online-education/mba/rankings (accessed 28 November 2020).