Development of mobile application through design-based research

Abstract

Purpose

The purpose of this paper is to illustrate the development and testing of an innovative mobile application using design-based research.

Design/methodology/approach

This paper reports on the process of transformation of existing printed course material into digitized content through design-based research where design, research and practice were concurrently applied through several iterations of the mobile application. For this transformation, one session each from BSc in Nursing, Bachelor of Pharmacy and Bachelor of Medical Laboratory Sciences was selected. In the first phase of the design-based research, the main research question was formulated. In the second phase, a mobile learning application (OUSL MLearn) was designed and developed to address the research question. In the third phase, this application was evaluated by five groups of stakeholders: content experts to validate the content; educational technologists to check the alignment of technical and pedagogical features; novice users to check the overall effectiveness of the application; developer to develop the application, to check the ease of usage; and researchers to identify the impact of this innovation. These stakeholders were closely involved throughout the whole process which lasted over a period of four months. At the end of this development phase, the results were reflected upon and used for further enrichment.

Findings

It was observed that the developed mobile application was accessible, appealing and pedagogically constructive for users. However, optimization, development time, technical and organizational issues, workload of academics and production costs were identified as major challenges.

Research limitations/implications

This study was based on the findings of a small sample of potential users.

Practical implications

The findings have implications for designing culturally adaptive interactive mobile applications.

Originality/value

This study will benefit practitioners to design culturally sensitive mobile learning courses and researchers to conduct design-based research.

Citation

Jayatilleke, B.G., Ranawaka, G.R., Wijesekera, C. and Kumarasinha, M.C.B. (2018), "Development of mobile application through design-based research", Asian Association of Open Universities Journal, Vol. 13 No. 2, pp. 145-168. https://doi.org/10.1108/AAOUJ-02-2018-0013

1. Introduction

Mobile technology is an exceptionally fast-growing field that is closely connected with our work and day-to-day lives. There are new developments added to its growth every day with emerging new patterns of usage, having both positive and negative implications.

In the twenty-first century, higher education institutions had to be reconstructed to adapt to changes with the increasing global competition, the growing need for higher education, the changing nature of information, rapid developments in Information and Communication Technologies (ICT) and the varying expectations and demographic features of learners (Kukulska-Hulme, 2005a). The changes in the dynamics of ICT, institutions and learners influence the academics working in higher education institutions to change their teaching approaches and strategies.

However, we have not seen a noteworthy adoption of these technologies in the education sector even though they are available everywhere (ubiquitous) and have tremendous potential in addressing needs of the individual learner through their unique capabilities. Furthermore, owing to the rapid changes of mobile technologies, including devices and communication technologies have opened up new research opportunities and even change the focus of research (Parsons, 2014). Krull and Duart (2017) reported that in higher education, mobile learning is a growing field of research as evidenced by reviewing journal publications between 2011 and 2015. The major results of their study were that the most researched theme was on m-learning applications and systems, used both quantitative and qualitative studies and were targeted at students. As both faculty and student adoption play a crucial part in the success of mobile learning initiatives, they recommend future studies to look into the implications for both faculty and students.

However, there is a scarcity of research articles related to mobile learning and methodological frameworks for designing sustainable mobile learning activities (Nouri et al., 2016). The purpose of this research study was to address this gap by applying design-based research in designing a mobile learning solution for the undergraduates of the Faculty of Health Sciences of the Open University of Sri Lanka (OUSL). It reports on the findings of the testing phase of the mobile solution by five groups of stakeholders: content experts, educational technologists, developer, novice users and researchers prior to the delivery of the first cycle.

The first section of the paper defines briefly the mobile learning, design-based research and employing design-based research in mobile applications, and stresses the importance of conducting design-based research for future technological innovations. The second section briefly describes the context. The third section examines the methodology adopted for the design-based research for new technological innovations in teaching learning using mobile applications. The fourth section is dedicated to the findings which were collected from all the stakeholders illustrating the potential for innovative teaching practices through mobile learning. The final section is a critical examination of the viewpoints expressed by all stakeholders and formulating guiding principles for both designing mobile learning solutions and on how to conduct design-based research in mobile applications.

2. Theoretical framework

2.1 Mobile learning

Mobile devices are portable, lightweight devices such as mobile phones (cellphones, or handphones), smartphones, palmtops and handheld computers (Personal Digital Assistants or PDAs), tablet PCs, laptop computers and personal media players. These devices can be carried around easily and used for communication and collaboration, and for teaching-learning activities that are different from what is possible with other media.

Traxler (2009) has pointed out that mobile devices together with mobile communication technologies have influenced all fields including education and currently undergoing a transformation. In fact, he called this transformation period as mobile era. He further stressed that most of these mobile devices are not designed specifically for education or training but designed for personal or individual usage which mainly used for one-to-one social interaction.

In the context of education, these mobile devices offer diverse learning opportunities such as portability, social interactivity, context sensitivity, connectivity, individuality and affordance to people in academic settings or non-academic settings (Crompton, 2013). Therefore, these mobile technologies are very useful for learners where they could engage in educational activities and learn by themselves without the constraints of having to come into the institution.

Since the introduction of the term mobile learning in 2005 (Crompton, 2013), many scholars and practitioners have attempted to define it and initial definitions were focused solely on devices and technologies or techno-centric (Crompton, 2013; Keskin and Kuzu, 2015). Most widely accepted mobile learning definition is “learning across multiple contexts, through social and content interactions, using personal electronic devices” (Crompton, 2013, p. 4). This definition encompasses four central constructs associated in mobile learning: pedagogy, technology, context and social interactions. Table I illustrates the categorization of the attributes of mobile learning into these central constructs.

With these attributes, it has much in common with other types of e-learning on desktop computers but allows more diverse and changing locations, more immediate (anytime) interaction, and connect through smaller, often wireless devices (Kukulska-Hulme, 2005a) enabling both advantages and drawbacks.

Learners can choose their own learning path to achieve their learning goal by using their own private mobile device. Hence, mobile learning can take place when the learner is not at a fixed, predetermined location or learning that happens when the learner takes advantage of the learning opportunities offered by mobile technologies (O’Malley et al., 2005, p. 7). As a result, they have spontaneous personal accesses to the large number of learning resources via the internet.

However, in order to create this kind of ambient technology, providers need to design “learning enhanced” buildings and public spaces, by providing devices or establishing systems to respond to on-the-spot interactions.

Several researchers have studied the development of the theoretical frameworks of mobile learning: mobile education (FRAME) based on activity theory (Koole, 2009), construction of knowledge through the exchange of knowledge via pervasive mobile devices (Sharples et al., 2007) based on the conversation theory, developed by Pask (1975), modification of transactional distance education theory for mobile learning (Park, 2011) and educational research on mobile communities (Frohberg, 2003).

Still there is a lack of transferable design frameworks on mobile learning (Cochrane, 2013) irrespective of those developed earlier. There is a greater need by the practitioners, instructional designers and trainers on design theories, so that they may in a better position to integrate mobile technologies into learning environments in an effective manner and to make these technologies more beneficial to users (Koszalka and Ntloedibe-Kuswani, 2010; Park, 2011; Rajasingham, 2011).

When designing instruction, generally learning content is regarded as the most important factor. Thomas et al. (2002) stressed that culture has to be considered as a “dimension” of instructional design capturing, three layers: purposeful intention, interaction with learners to involve them in the design process and for introspection to identify one’s own cultural values and biases. The importance of language, culture and context is also highlighted in teaching and learning (Gunawardena et al., 2009). Chen et al. (2006) emphasized the importance of paying attention to provide support (technical, learning and social) and resources (language, culture and context, and learning content), so that learners can make effective connections between resources and the support. Therefore, designing effective learning for global audiences requires not cultural neutrality but cultural “inclusivity” (Frechette et al., 2014; Henderson, 1996; Powel, 1997) as the online medium (internet) itself is a culturally derived phenomenon (Bowers, 2000) and functions as an incubator for a shared cultural experience (Selvin, 2000). Therefore, it is crucial to explore the “cultural understanding” of learners (Rogers et al., 2007). Furthermore, online tutors and mentors should be more sensitive to culture when facilitating knowledge construction to global audience via online where both learners and tutors bring their own cultural identities and they have to use face-saving and negotiation strategies to build trust to develop online communities (Gunawardena and Jayatilleke, 2014; Gunawardena et al., 2009).

The value of universal design concept has been argued as one of the best practices for learning to design standards-based lessons (Rao and Meo, 2016) and in creating culturally inclusive online course materials (Eberle and Childress, 2006; Rose and Meyer, 2000). It allows learners to select from different methods of instruction (reading learning resources), modes of expression (typing a response) and means of interaction (synchronous or asynchronous communication) (Frechette et al., 2014) to accommodate culturally different diverse learners. Emergence of a unique online visual culture is observed by mobile phone users by blending spoken and written languages or orality and literacy in communication. They produce various visual representations using both words and oral expressions interchangeably while on the move (Rha, 2014, p. 47). They develop various techniques combing literal properties of written characters to deliver their messages expressing emotions using:

• abbreviations (e.g. lol: “laugh out loud,” yolo: “you only live once”);

• hieroglyphics (e.g. xoxo: “hug kiss hug kiss,” T-T: “tears,”:-: “smiling eyes”), add a new bullet emotions; and

• special characters, which are unique visual representations and difficult to classify as wither text or image.

It also is necessary to get the views from learners on the designed product to shape the design itself (McLoughlin and Oliver, 2000) through evaluation studies.

2.2 Design-based research

The aim of the design-based research is to improve educational practices through systematic but flexible methodology through iterative analysis throughout the design, development and implementation of the product (Wang and Hannafin, 2005, pp. 6-7). These educational practices are based on the views gathered between researchers and practitioners in a normal setup and these practices would lead to formulate contextually sensitive design principles and theories (Wang and Hannafin, 2005). Thus, fulfilling the ultimate goal of design-based research by building stronger connection between educational research and real-world problems. In this scenario, researchers and practitioners are integral part of the research process and they are closely involved from the initial phase of design and development of the product to the final phase of implementation. However, research validity of design-based research is criticized by some due to the involvement of researchers where they felt that researchers may not be reliable and faithful in their judgments (Barab and Squire, 2004). Further, the intervention may not be replicated in other settings as design-based research is contextually dependent (Design-Based Research Collective, 2003).

Review conducted by Zheng (2015) showed that design-based research studies were conducted with diverse sample groups but mainly with students in higher education (29 percent), under various learning environments including distance learning (14 percent) and blended learning settings (12 percent). Natural science (38 percent) was selected as the most researched learning domain while medical science was the least selected learning domain (2 percent). The technological intervention was the major type of intervention used in the design-based research (53 percent) followed by the integrated teaching models (16 percent), other models (16 percent) and instructional methods (14 percent).

Research methods used in design-based research were mostly qualitative in nature (Zheng, 2015) but recently, there is a trend in using big data to refine designs and to build theory (Svihla, 2014). However, many researchers have pointed out the unavailability of established research process and guidance on how to conduct design-based research (Dede, 2004; Design-Based Research Collective, 2003; Engeström, 2011). While supporting the uncertainty about the research process carried out in design-based research, Easterday et al. (2014, p. 317) further identified three problems in design-based research:

1. uncertainty about how it differs from other forms of research;

2. uncertainty about how it differs from design, or why design is not research; and

3. uncertainty about what might make it effective (if it is).

In view of these uncertainties, they described the design process consisted of six iterative phases: focus the problem, understand the problem, define goals, conceive the outline of a solution, build the solution and test the solution (Easterday et al., 2014, p. 320) which are recursively nested within each other. These phases could be compared with the well-researched four-step framework of Reeves (2006), where he explains design-based research as a process that consists of four steps (Table II).

It is clearly evident from the table that the fourth step, that is documentation and reflection to produce design principles of Reeves’s framework, was not identified in Easteday et al.’s six-phase/step framework.

Ma and Harmon (2009) pointed out that the guidelines and the process presented by Reeves’s (2006) four-step framework provided valuable guidance on how to conduct design-based research from the long-term perspective; however, they felt the inadequacy of the framework to conduct design-based research at the individual level as it was not clear about the conduct of research activities in each step. They further enriched the Reeves’s framework by providing a more detailed and comprehensive development process incorporating research elements to the framework with specific guidance (Ma and Harmon, 2009, pp. 77-78). The main steps in the framework are connected linearly from steps 1 to 4 with connecting loops to each step. They mentioned that “by no means as clean and liner as it might appear” and “researchers may examine their own context to make appropriate modifications” (p. 80). These guidelines enable researchers who are new to design-based research to conduct design-based research systematically and logically.

Both design-based research and action research share many epistemological, ontological and methodological foundations thus sharing a common “meta-paradigm” pragmatism (Cole et al., 2005) and many find it difficult to distinguish the two (Easterday et al., 2014). Generally, design-based research is conducted by a research and design team whereas action research is carried out by a single teacher (practitioner), guided by a theory (Anderson and Shattuk, 2012), focused primarily on already designed product/process and its application into an everyday context (Ørngreen, 2015) and contributes toward theory building (Cole et al., 2005). However, collaboration between practitioners and researchers is not clearly described in the design-based research literature (Kolmos, 2015) and needs further investigation.

However, some researchers have combined design-based research with action research (Keskin and Kuzu, 2015). They feel that the research methodology most appropriate to the third step of the Reeves’s framework is through action research. It can make the product highly effective, efficient and useful by allowing repeated development of the product until all the identified errors of the product are resolved during the testing phase (Susman and Evered, 1978). In view of this notion, Keskin and Kuzu (2015) developed the model by combining the design-based research model put forward by Ma and Harmon (2009), and the action research cycle suggested by Susman and Evered (1978) regarding information systems. The model also has the same four steps proposed in the Reeve’s framework; however, all the steps in the Keskin and Kuzu’s model are interactive with each other. The design principles and the theory can be developed in the fourth step, based on the analyses of the data collected in each step (Ma and Harmon, 2009). By following iterative research process in the design-based research, it attempts to refine the innovation systematically while also proposing design principles unlike in evaluating innovative product or intervention at the end of the development phase.

2.3 Mobile applications employing design-based research

Many researchers have conducted reviews related to the application of design-based research as a research methodology in conducting various research studies (Anderson and Shattuk, 2012; Krull and Duart, 2017; Zheng, 2015). Major findings of these studies indicated that the majority used design-based research for technological interventions and applications. Anderson and Shattuk (2012) reported that the majority of interventions (68 percent) involved the use of online and mobile technologies. According to Zheng’s study reviewing research articles and publication from 2004 to 2013, technological interventions applications were the most researched area (53 percent) and were to test the effectiveness of the learning environment or a particular tool. However, the nature or the type of the tool was not specifically mentioned in his study. In reviewing journal publications between 2011 and 2015, Krull and Duart (2017) revealed that mobile learning applications and systems were the most researched area conducted using design-based research in higher education. Keskin and Kuzu (2015) combined design-based research and action research to conduct professional development program for academics using M-learning system.

3. Context

The OUSL is unique in its teaching methodology as it is the only national university in Sri Lanka which is dedicated to open and distance learning. Unlike in conventional universities, the OUSL mediates instructions mainly through print course materials. With the advent of various technologies, the OUSL has gone through generations of technology integrating audio-visual, multimedia and online learning into the core print course materials (Jayatilleke et al., 2009).

Having faced with many challenges with respect to distributing printed course material on time and to reduce production and delivery costs of the course materials, there have been many suggestions from time to time to use other technologies. However, print has remained as the core medium of instruction even though many such initiatives have been taken to promote offering courses entirely online.

Aligned with this notion and also considering the immense potential of using mobile technologies for learning, the OUSL has recently proposed an alternative option to address these challenges. Providing course materials in PDF format loaded on a tablet computer would be a viable option as tablet computers are becoming cheaper by the day, harnessing the potential of improving the learning experience and thereby effect institutional change.

Hence, Faculty of Health Sciences of the OUSL took the initiative to investigate the viability of transforming the existing print course material, and offer them through mobile learning for the undergraduates of the Faculty. This project was carried out from a research grant of the OUSL which enabled to experiment with novel mediating mobile technologies.

Three Bachelor’s degree programs are offered by the Faculty of Health Sciences; Nursing; Medical Laboratory Sciences and Pharmacy. One session each from a degree program was transformed retaining the already existing content and the original framework as these courses are still being offered by the OUSL.

4. Methodology

In this study, design-based research model put forward by Ma and Harmon (2009) was used as the framework as it provided the processes clearly (Figure 1). The “analysis of practical problems” is the first phase as in the Reeves’s model. In this phase, a practical problem is identified and the related literature about the practical problem is reviewed. The second phase is “development of solutions” for the practical problem identified in the first phase by conceptualizing a solution within theoretical framework, identifying research purpose and development method, developing a prototype that serves to address the research problem. The third phase is “evaluation and testing of solutions in practice.” The final phase is “documentation and reflection” where design principles are generated and documented in order to provide guidance for practitioners and researchers who are interested in conducting design-based research.

This study was also influenced by the design-based action research model put forward by Keskin and Kuzu (2015) where phase 3 is an iterative cycle rather than a linear process. In this phase, problems related to the prototype are recognized and action plans are developed. At the implementation, these plans are implemented and the consequences of the action are evaluated and reflected. This process continues until all problems are solved.

In this study, development and testing of the mobile application (phase 3) was carried out concurrently with the phase 2 through formative evaluation. Phase 2 and phase 3 were closely linked and phase 3 was incorporated in the phase 2 of the cycle (Figure 1). These two phases were not separated cycles as in Keskin and Kuzu’s model. Since phase 4 (documentation and reflection) was also closely connected with these two phases through reflection, the connection between phase 3 and 4 was illustrated in a two way arrow.

Since design-based research is a multi-phase study, the present study involved five groups of stakeholders. In this study, researchers took the initiative and were involved from the beginning of the design process together with the developer, content experts/practitioners and educational technologists. All these stakeholders were closely involved throughout the whole process which lasted over a period of four months. The formative evaluation was carried out with four content experts, four educational technologists, six novice users, four researchers and one developer.

4.1 First phase – analysis of a practical problem

4.1.1 Identifying a practical problem

The analysis of a practical problem by researchers and practitioners is the first phase of this design-based research. Researchers in this study are also practitioners: two are teaching zoology/health courses while other two are training academic staff on online learning/educational technology. They have experienced the practical problem faced by the OUSL for many years; that is the difficulty in producing timely printed course materials to OUSL students with increasing student numbers. This practical problem is equally important to both OUSL students and the institution (OUSL). Thus, researchers took the initiative to conduct this research study through design-based research, where they felt the problem was significant to the learning community of the OUSL. They also believe that the findings will provide evidence to inform decision makers on the viability of providing a tablet computer loaded with the content to students so that decision makers may be in a position to take data-driven decisions rather than taking ad hoc decisions.

4.1.2 Reviewing the literature to determine the significance of the problem

An earlier study carried out with the students of the British Open University showed that the majority preferred e-books as a complementary technology and still would like to receive print course materials (Kukulska-Hulme, 2005b, p. 130). Researcher further reported that learners faced difficulties in downloading e-books, getting satisfactory page and font size, navigation and cursor control, etc. With many technological advancements over the years, still students perceive printed texts are easier to read, understand and navigate, and have long-term access even though the digital texts are becoming cheaper (Baglione and Sullivan, 2016). Comparative studies have been conducted to investigate the effects of digital reading (e.g. reading a word or PDF file on screen) with print reading; however, not much research has been carried out into examining learners reading behaviors and the educational benefits of recent, more flexible visually presented texts (Rha, 2014, p. 51). Rogers-Estable (2018, p. 48) reported that many faculty members commented that if electronic texts (eTexts) are purely PDF files (or glorified PDFs) then there is no advantage in using them with students.

Based on the literature review, the decision was taken by the research team, not to provide a digitized text as a PDF (or as a glorified PDF) to learners (as an e-book) but to provide a mobile learning application with enhanced version of the already existing print material with additional pedagogical, technological (interactive), contextual and social interactive attributes associated with mobile learning with innovative strategies and tools. Social interactive attributes inherent to mobile learning was used; however, less priority was given to design peer/tutor interactions in this mobile application as it was designed as a stand-alone package to study offline considering the specific requirements of the target group, that is health professionals with demanding work pressure. However, learners have the opportunity to use the mobile application either online or offline. They can also use other channels such as e-mail and social media to discuss the content if they wish to collaborate socially.

4.1.3 Identifying the purpose and research questions for a development iteration

According to Ma and Harmon (2009), identifying the purpose and research questions for a development iteration was discussed as the third step in the second phase (p. 77) even though they have highlighted the importance of it before commencing the development (p. 80).

In this study, research questions were formulated in the first phase as it was felt necessary to identify the purpose and the research questions before starting the development as they will direct and guide the development process through research.

The following research questions were formulated after analyzing the practical problem:

RQ1.

How to design a mobile application using an existing print course material?

RQ1a.

What was the process carried out when transforming the existing print materials into mobile application?

RQ1b.

What types of interactivity features were added to the mobile application?

RQ1c.

What were the challenges faced by content experts, developer and educational technologists when designing mobile application?

4.2 Second phase – development of a solution

4.2.1 Conceptualizing a solution within a theoretical framework

In the second phase, a mobile learning application called “OUSL mobile learning” (OUSL MLearn) was designed and developed specifically for the Android mobile devices to address the principle research question within the theoretical framework.

The existing print course materials were originally designed based on teaching and learning theories such as Guided didactic conversation in distance education (Holmberg, 1983). However, transformation of existing printed course material into digitized content requires additional research related to mobile learning such as designing content with in-built interactive features for mobile devices. Furthermore, learning is situated and contextual. Thus, research and practice were concurrently applied through design-based research with several iterations of the mobile application.

4.2.2 Determining the role of research in developing the solution

Having conceptualized the solution, next step was to decide whether research should be conducted while developing the solution. Since the solution in this study was to develop a mobile application through several iterations, research was an integral part and relevant research studies with respect to the needs and the requirements of the stakeholders (teachers and students), learning preferences of students, cultural propensities were considered when designing user-interface, content development and system technical design.

4.2.3 Identifying development methods

This mobile application was based on the existing content of the printed material thus, restricting the design and development of the mobile application. Therefore, first step was to develop a prototype that was adequate for the purpose. Having reviewed the literature on various types of prototypes, an icon-based prototype was selected for the development of mobile application as the majority learners are visual learners (Rha, 2014).

4.2.4 Developing a prototype that serves the research purpose

In order to develop the mobile application, the first meeting was conducted with the content developers, educational technologists, researchers and the developer and discussed the overall objectives of this project. Three sessions (one session per a degree program) of the existing print materials were handed over to the developer highlighting the requirements, providing necessary information and devices (tablet computers). Developer was given the freedom to select appropriate technologies and tools to develop the mobile application to use with the specified tablet computer. This decision was based on the assumption that the OUSL will provide the standard tablet computers to all OUSL learners rather than requesting them to purchase or use their own tablet devices in order to minimize the technical problems. HTML was the main tool for the development and other tools were also used to enhance the capabilities of the mobile application.

First iteration was to discuss the prototype for the development of the session structure and subsequent iterations developed the prototypes for the course structure and the system architecture. Interactive features such as self-assessments questions and embedding videos were incorporated later.

The following section will describe in detail the design of the OUSL MLearn mobile application.

System architecture and implementation

The system architecture was designed for the entire university which serves as the mobile platform (OUSL MLearn) for the OUSL. In order to make the system user friendly, the unique icon-based system was designed (Figure 2(a) and (b)).

The navigational structure for this system was connected to the main pages forming a semantic network as illustrated in Figure 3. The icon-based system was connected to the home page of the system, then to the Programmes Page where students can select their own program, followed by the Course Page and the Session Page, respectively.

Course/session structure

Each course was designed in such a way to make the course as a stand-alone module which can be studied offline. This decision was made considering the baseline survey of the undergraduates of the Faculty of Health Sciences and their past experience of not accessing learning resources through internet (Jayatilleke, Wijesekara and Ranawaka, 2017).

The existing OUSL print materials were designed as self-instructional materials with intermittent activities to facilitate “guided didactic conversation” with text (Holmberg, 1983), incorporating advance organizers at the beginning and summaries at the end (Melton, 1997; Rowntree, 1990). These materials are designed using pedagogical features based on learning theories, research and practice.

These pedagogical features were retained in the mobile application. For instance, advance organizer at the beginning and summary at the end of each session were designed based on the pedagogical features of the original print course materials. Research shows that an advance organizer serves as a schema for the learner to associate new concepts with the already known concepts and to connect them meaningfully (Ausubel, 1960), whereas a summary (post organizer) provides a synopsis that helps the learner to get a holistic picture of the concepts learned in the session. In addition, advance organizers help diverse learners, in particular FD and FI learners, respectively. Research studies have revealed that FD learners are holistic in nature and need external guidance to solve problems while FI learners are serialistic and use their own cues to solve problems (Witkin et al., 1977). Since OUSL learners are diverse, course materials have to provide provision for these two groups to learn the content without the help of the teacher. However, content was re-designed as smaller chunks to suit the mobile screen to avoid overload of information based on Sweller’s (2011) cognitive load theory.

Additional features were also were incorporated to accommodate specific requirements necessary to learn using mobile devices. Each session was transformed considering the four pedagogical aspects of instructional design, namely information design, instruction design, interface design and interaction design. Findings of the earlier research studies on online learning were also considered in designing this mobile application (Table III).

Figure 4 illustrates the screen casts of the animated instructions proving study guidance on how to use mobile device.

Printed course materials use icons in front of the major pedagogical components such as learning outcomes, self-assessment activities (activities), online/video integration, etc. and use them as access devises. These icons were specially designed as authentic learning objects to maintain the OUSL identity across all OUSL materials. These icons were also used in this mobile application to maintain the OUSL identity. These features could be considered as contextual attributes of mobile learning. Certain new icons were added to represent functional specific requirements associated with the mobile application (e.g. Menu, Note, etc.). Typical icons generally represented in global community were used to represent images and settings. The layout of the program control, learner control and specific icons is illustrated in Figure 5.

This mobile application integrated a video, enriching the existing content using the affordances of mobile technologies and designed as an activity activity, based on the video. Generally, OUSL students do not watch videos, unless they are compulsory or integrated in the course materials. Thus, research and practice were considered in the design and development of this mobile application in line with the guidelines of the design-based research (Figure 6).

In addition to the instructional design features, adaptive technologies were also incorporated in the mobile application considering the needs of the heterogeneous nature of OUSL learners (Jayatilleke, 2016). Table III illustrates these features. Learner has the opportunity to adjust the size of the font (Figure 7) and images, taking notes, highlighting the text and copying and pasting facility were some of the adaptive technologies used in this application.

The next section provides the detail account of the evaluation and testing phase of the mobile application.

4.3 Third phase – evaluation and testing of the solution

In the third phase, this mobile application was regularly tested through formative evaluation which was an integral part of the design methodology. It helped to judge strengths and weakness of the innovation while still at its developing stage, for the purpose of revising the instruction. As mentioned earlier, second and third phases conducted concurrently and could not be separated during the research process. Certain features were added after getting the feedback from various stakeholders during the testing phase.

4.3.1 Identifying research methods

In this phase, appropriate research methods were identified, collected and analyzed data to answer the research questions. Qualitative methods were used in gathering data since design and development of the innovation need in-depth analysis of the innovation. A research diary, committee meeting records, observational sheets of the users while using the tablet interview schedule for users, and checklists for error identification were used as data collection tools.

Content developers of these three sessions (four females), educational technologists (two females and two males), researchers (three females and one male) and one developer (male) were the members of the research and development team from the inception of the research project.

Purposeful sample was used to select the subjects as novice users where they have not followed these courses before to test this innovative mobile application. All the novice users were graduates in different disciplines (BSc in Natural Science −3, BSc in Information Technology −2 and BA in Social Science −1) consisting of four females and two males representing age range of 25–35 years. All of them have smartphones and comfortable of using tablet computers.

4.3.2 Gathering and analyzing data to answer research questions

The second and third phases were carried out simultaneously with regular meetings with the developer and other stakeholders through testing of the mobile application. In these meetings, the application was evaluated by five groups of stakeholders: content experts to validate the content, educational technologists to check the alignment of technical and pedagogical features, novice users to check the overall effectiveness of the application for learning purposes, developer to develop the application, modify it with the feedback and check the ease of usage and researchers to identify the impact of this innovation.

Novice users were briefed about the purpose and asked them to go through the mobile application. The lead researcher observed and made notes using observational sheets while users were explored the mobile application. At the end of the product evaluation, researcher asked questions using a structured interview schedule about their perceptions of this mobile application, their likes, dislikes, challenges and suggestions for improvement. These viewpoints were categorized using content analysis to identify the major themes.

The entire development of the application was through eight iterations where feedback from different stakeholders at different stages was integrated to the OUSL MLearn system. First iteration was to develop the initial prototype for a session. Then three iterations were focused to develop the framework to design the entire system architecture for the entire university, considering faculty, departments and program requirements. Last four iterations were testing the developed prototype with additional requirements, adding pedagogical features along with the interactive features to the course structure and testing with novice users and content developers.

4.3.3 Drawing conclusions and determining research findings

At the end of the formative evaluation, conclusions were drawn based on the findings. All the stakeholders perceived benefits of the mobile application as an effective tool for learning. Many challenges were expressed by different stakeholders and will be discussed in the results and discussion section.

4.4 Fourth phase – documentation and reflection

This is very important phase in design-based research. Unless documentation and reflection, the generation of design principles and guidelines could not be constructed and the purpose of using design-based research is not fully achieved. Ma and Harmon (2009) recommended to provide two sets of principles based on the research study. One set of principles for the practitioners on the research findings specifically related to the instructional innovation/solution/product to improve their practices. The other set of principles for researchers who are interested in conducting design-based research on how to conduct design-based research based on the reflections on the research methodology.

In this current study, reflections were part of the whole process and not only restricted to the documentation phase. Researchers reflected the research processes in all phases from phases 2 to 4 and went back and forth while documenting the process in order to generate principles. At the end of each development phase, the results were re-examined, reflected upon and used for further enrichment, producing a continuous cycle of design-reflection-design. So, formative evaluation was integrated in the testing phase of the design-based research and the results were used to improve the system to make the instruction more effective and efficient. In this study, phases 2, 3 and 4 were all connected and could not be separated as distinct phases.

The current interface of mobile application and its functionalities are the result of revisions based on the suggestions/reflections during the formative evaluation of all three phases.

4.4.1 Synthesizing design principles for developing the proposed solution (mobile application)

Having gone through the reflections, the researchers felt the design-based research is very appropriate in designing and developing technology based innovations as user testing is part of the development process. Since both researchers and practitioners were involved from the beginning, their contributions were very useful in conceptualizing the solution within a theoretical framework. The following design principles were derived from the findings of the research study for mobile application.

Principle 1

Research team should have open discussions with all the stakeholders including the developer so that diverse strategies/solutions will emerge as a result. Team can discuss these strategies and identify best solutions in order to reduce the development time of the innovation.

Principle 2

Research team should consider the existing research and practices in the local context in order to develop cultural sensitive solutions as learning is contextual and situated.

Some of the research studies conducted in western world may not directly applicable to eastern cultures. This study was influenced by the research findings of earlier studies conducted at the OUSL with three groups of culturally diverse groups of learners (Sri Lankans, Pakistanis and Mauritians), where they interact via learning management system for seven weeks (Jayatilleke and Gunawardena, 2016; Jayatilleke, Kulasekara, Kumarasinha and Gunawardena, 2017).

Principle 3

Instructions should be integrated in the mobile application as animated learning objects considering the user needs; especially, if the application is designed for open and distance learners.

Principle 4

When developing mobile solutions, alternative technological strategies and adaptive technologies should be designed in order to accommodate diverse learners.

Principle 5

Adaptive technologies should also be integrated in the mobile application to accommodate differently abled learners to empower them while making them more inclusive in the mainstream education.

Principle 6

Institutional leadership for direction, guidance and providing mechanism for establishing support structures are crucial in order for the sustenance and adoption of innovative mobile solution. Otherwise, diffusion of innovations will be observed only at the individual level and gradually die down.

4.4.2 Synthesizing guidance for conducting design-based research

This study adapted the model proposed by Ma and Harmon (2009) and was also influenced by the research of Keskin and Kuzu (2015). The detailed development and research procedure in the Ma and Harmon’s (2009) model was very useful in designing the procedure to conduct design-based research. However, researchers of the current study had to modify the order of certain guidelines to suit the context and the user needs. Even Ma and Harmon (2009, p. 90) stated that researchers may examine their own context to make appropriate modifications to their model. Hence, following guiding principles are proposed for the researchers who are interested in conducting design-based research based on the reflections of this research and development team.

Principle 1

Identifying the purpose and research questions for a development iteration are very crucial in the design-based research as they provide the focus for the study. Thus, they should be included in the first phase of the research study – analysis of a practical problem (refer Section 4.1.3).

Principle 2

Identifying the importance of research at the beginning of the development of innovation of project upfront and decision should be taken to integrate research while developing solutions at the beginning prior to the development of the solution and give fullest attention to the research methodology along with the development phases of the solution.

Principle 3

Educational technologist should be included as a researcher in the design-based research team to provide guidance, direction and to facilitate theory-driven research process and thereby enabling theory-building outcomes of the innovation in an effective manner.

5. Results and discussion

The views expressed by the novel users indicated that the developed mobile application was generally efficient, simple to learn, easy to navigate, appealing and engaging. It was also pedagogically constructive as the content and the tools used in the application were useful from the perspective of both the content experts and the educational technologists. Thus, accomplishing the primary goal of this research study by providing effective instruction through mobile learning. It was also found that the developed mobile learning system was appropriate to the overall purpose of the university, could be served as a mobile learning system for the entire university and also could be used as an academic support system for the OUSL from the perspective of the developer.

Having gone through the reflection process and analyzing the qualitative data obtained by all the stakeholders using various tools, the challenges in implementing the MLearn for the entire university were identified using content analysis of the data. The categorized themes are illustrated in Table IV.

6. Conclusion and future direction

Having gone through this process, it was felt that the design-based research build on the principles of stakeholder centredness was effective in developing mobile learning application. This was due to the fact that the researchers and the practitioners were actively involved throughout the whole process and supported each other to produce an effective mobile application. The framework used in this study embeded the evaluation and testing of the solution phase (Phase 3) within the development of the solution phase (Phase 2) as these two phases are interconnected and run concurrently. Owing to the iterative cycles of the design-based research enabled the development of an effective mobile solution through several refinements based on existing research and practices.

Cowling and Birt (2018) also showed how the process of incremental reflection and refinement of the design-based research enabled the development of a mixed reality simulation to improve skills for students studying paramedic science at a distance.

The findings of the evaluation of the mobile application showed challenges with respect to development time, high production costs, technical and organizational issues, workload of academics and necessity of providing technical support both to remote students and faculty. Therefore, establishing adequate support structures for both teachers and students are essential for the sustenance of these innovative practices. This finding is in line with Montreux et al.’s (2015, p. 10) study where they also emphasized the importance of technical and pedagogical support to “stimulate teacher and student recognition of tablet devices’ potential in education.”

This application will be further evaluated through summative evaluation with actual students to assess the effectiveness of the mobile learning system to complete the design of the system fully.

The design and development of any instructional material depend on the target audience, the subject content and the organizational culture of the institution (context). As such, the findings of this study may not have a universal value; however, these findings may throw light on some of the pedagogical, technological, social interaction and contextual attributes including cultural dimensions that have to considered when designing mobile applications. It also provides guiding principles for designing both mobile solutions and on how to conduct design-based research in mobile learning.

Figures

Figure 1

Design-based research model of this study

Figure 2

Screen casts of the mobile application

Figure 3

Navigational structure

Figure 4

Animated Instructions proving study guidance on how to use the mobile device

Figure 5

Program control, learner control and specific icons

Figure 6

Video integration in the mobile application

Figure 7

Flexibility of using different font sizes

Table I

Categorization of the attributes of mobile learning into the central constructs

Attributes	Construct
Portable, ubiquitous (available everywhere and at any time of the day) and bite-sized	Technological
Personalized and situated learning experiences through applications, concepts, and often the ownership of devices modified for the user (Traxler, 2011)	Pedagogical
Formal or informal, self-directed or guided, planned educational program or unplanned/spontaneous learning experience, context aware and context neutral	Contextual
Pervasive (hardly noticed as it has closely linked with our day-to-day activities) and ambient – it has surrounded us entirely and perhaps natural as the air we breathe (Kukulska-Hulme, 2005a, p. 3)	Social interactions

Table II

Comparison of the processes of the design-based research of Reeves’s (2006) and Easterday et al.’s (2014) framework

Phases	Reeves’s (2006) framework	Easterday et al.’s (2014) framework
First step	Analysis of practical problems by researchers and practitioners	Focus the problem Understand the problem Define goals
Second step	Development of solutions within a theoretical framework	Conceive the outline of a solution Build the solution
Third step	Evaluation and testing of solutions in practice	Test the solution
Fourth step	Documentation and reflection to produce design principles	Not identified

Table III

Categories of instructional design and pedagogical/technological/contextual/social interaction attributes of the designed mobile application with supported research evidence

Categories of instructional design	Pedagogical (P)/Technological (T)/Contextual (C)/Social interaction (S) attributes		Research evidence
Information design	As smaller chunks to suit the mobile screen Static arrangements of text repositioned as visual objects	P and T	To avoid overload of information based on Sweller’s (2011) cognitive load theory In line with technology affordances of mobile learning
	Structured format	P	Preference for well-structured format by the Asian learners (Jayatilleke and Gunawardena, 2016; Jayatilleke, Kulasekara, Kumarasinha and Gunawardena, 2017).
Instruction design	Study guidance  Animated instructions on how to operate the tablet computer at the beginning with skipping facility (Figure 4)  Guidance when they have to rotate the tablet	P and T	Preference for clear instructions on navigation and e-activities (Jayatilleke, Kulasekara, Kumarasinha and Gunawardena, 2017) In line with technology affordances of mobile learning
	Learning outcomes at the end of a session Introduction as an advance organizer Summary Interactive glossary designed for the entire course Auto-generated report at the end of each session on learner performance for self-evaluation	P P P P and T T, S and C	Structure for ODL course materials based on Melton (1997) and Rowntree (1990)
Interface design	Simple navigational structure sequential arrangement of the content with activities	T and S P and C	Preference for linear and sequential arrangement of content with one activity at a time and had difficulty of engaging multi-tasks concurrently specially among Asian/Eastern learners (Jayatilleke and Gunawardena, 2016; Jayatilleke, Kulasekara, Kumarasinha and Gunawardena, 2017; Ku and Lohr, 2003)
	Flexibility of using both program and learner control options for navigation. System guides the program control option through buttons whereas learner control option was designed as a menu as an alternative strategy to support field independent learners where they can proceed the course in any sequence (Figure 5)	P, T, C and S	Evidence to show that field dependent (FD) learners prefer program control options while field independent learners (FI) favor learner control options (Yoon, 1993) Flexibility of using both program and learner control option for navigation were used to provide linear (monochromic) and multi structure with multiple tasks (polychromic/parallel) to accommodate culturally diverse learners in line with the concept of universal design by providing multi strategies to accommodate diverse learners (Rao and Meo, 2016)
	Specific authentic icons as “access devices” (learning outcomes, activity, video, etc.) and additional visual icons to the suit the mobile application (Menu, Notes, Images, etc.) to help in navigation (Figure 5)	P, C, S and T	To create online visual culture (Rha, 2014)
	In certain instances, labels were used to enhance the meaning of icons after the feedback of user testing with novice users (e.g. word “Menu” was added to the Menu icon and word “Note” was added to the Note icon)	P, C and S	In line with Paivio’s dual coding theory to present information in both text and images to facilitate the process of reading text and graphics at the same time (Clark and Paivio, 1991)
	Color scheme based on the university and faculty colors to distinguish study programs	T	In line with Keller’s ARCS theory of motivation to get and retain the interest (attention, relevance, confidence and satisfaction) – Keller (2009) In line with technology affordances of mobile learning
Interaction design	Diverse interactive activities for self-assessment (fill in the blanks, matching by dragging answers, tapping the correct answer, etc.) Providing teacher feedback for comparison Activities enabling several attempts to facilitate learning	P, C, S and T P P, C, S and T	Interactive learning assessments (ILAs) were useful especially with university students where they were able to see an expert answer and compare it to their own answers in an authentic learning activity (Svihla, 2014)
	Multimode activities (video) to retain the interest (Figure 6) Hypertext links to images for clarity 3D views to illustrate different profiles of the visual objects (e.g. lateral/frontal view of human skull) Animated images to explain processes for clarity (e.g. life cycle)	P, C, T and S P, T, S P, T, S, C P, T, S, C	In line with Keller’s ARCS theory of motivation to get and retain the interest (attention, relevance, confidence and Satisfaction) In line with technology affordances of mobile learning
Adaptive technologies	Selection options for font sizes (size 1, size 2, size 3) – (Figure 7) Images with zooming facility (display technology) Hypertext links to the glossary Notepad for making notes Option of highlighting the text while reading Auto generate reports on the notes and performance Option of copying and pasting the contents into the notepad or to any other document Option of sharing content with peers when connected to the internet option of printing Notes and Reports via e-mail	T and S P, T, C and S T and S T and S T, C and S T T and S	In line with technology affordances of mobile learning

Table IV

Factors identified through the reflections by all stakeholders in implementing the OUSL MLearn

Factor	Challenges
Time factor	Long development time to transform all sessions in the existing course materials into mobile learning Considerable time needed for carrying out usability testing and modifying errors
Cost factor	High development costs for developing and implementing mobile system for the entire university High initial costs for providing mobile devices for all learners unlike in permitting learners to use personal computers across diverse platforms. However, less recurrent costs by the institution for trouble shooting and customization of mobile devices or Providing an alternative solution to provide financial assistance for students to purchase/use personal mobile devices. However, high recurrent costs for customizing mobile devices across diverse platforms and providing technical assistance to large number of students
Technical factor	Needs optimization of the mobile application based on the performance of each mobile device to enhance the visual performance  Screen resolution  Design navigation  Sequence of the content and activities  Create user interactions through the interface  Develop interactive activities on the touch screen (e.g. drag and drop activities)  Use the device both vertically and horizontally
Teaching factor	Lack of staff time for academics for transformation of the content for mobile applications Lack of familiarity of the mobile devices by teachers to use in teaching Limited knowledge in designing interactive activities
Learner support factor	Needs induction training for students to use of mobile technologies Needs technical support throughout the learning process through a dedicated center to address technical issues on the spot
Organizational factor	Inadequate technological infrastructure to support the requirements of the entire university Limited availability of mobile devices to staff and students to experiment with innovative mobile practices scarcity of seed funding allocation for innovative educational practices Scarcity of support structures for the inventors to experiment novel ideas Lack of structures for sustenance of the technological interventions Needs effective leadership to promote and sustain innovations and creativity among academics