Mastering New Information through Facebook and a Discussion Forum : A Comparative Analysis

This paper investigated the effect of productive failure (PF) as an instructional strategy in computer-supported collaborative learning (CSCL) groups using Facebook and a discussion forum. PF is an instructional mode design that advocates the delaying of support for the learners during learning — the more they struggle, and even fail, while trying to master new information, the better they are likely to recall and apply that information later. PF has been used successfully in the classroom. However, it is not known whether the use of a PF instructional mode with adult learners in CSCL groups such as Facebook and discussion forums will produce such a positive effect. A discussion forum is an important platform used to deliver teaching and learning via the Web, while the use of social media, especially Facebook, for teaching and learning has gained prominence lately. This paper reports an initial study that compares a ‘productive failure’ instructional design in CSCL groups through Facebook and a discussion forum. Five Facebook and five discussion forum groups participated in the study. Both groups solved ill-structured complex problems in small groups without the provision of any support or scaffolding from their instructors. The findings suggest that the Facebook groups produced a variety of scope for discussion and deliberation for solving the problems and were more successful in sustaining the discussion compared to the discussion forum groups. Facebook groups also had a higher critical thinking ratio than the discussion forum groups. Based on these findings, the implications of a PF instructional design for adult learners are presented.


Introduction
Teaching and learning in open and distance education (ODE) is typically conducted via the blended pedagogy.In a typical scenario, the three components that make up blended learning are: face-to-face (F2F) tutorials, online learning and self-managed learning (see Figure 1).The blending of these three components appears to have worked rather well for the majority of adult learners.
Figure 1 Means to manifest the teaching-learning process at the Open University Malaysia (OUM): blended pedagogy (Zoraini, 2009) Online and face-to-face (F2F) learning are important components in blended pedagogy.Various instructional techniques have been employed to deliver F2F interaction, including tutorials, brainstorming, hands-on sessions, problem-solving and presentations.One of the instructional techniques for F2F interactions that has gained popularity is productive failure (PF).The PF instructional design advocates the delaying of support for the learners during the learning process (Kapur 2010); and it has been used successfully, particularly in secondary schools where there is regular contact with the instructors.
On the other hand, online learning refers to all forms of electronically supported learning and teaching.Typically, this is done by using learning management systems (LMS).One of the central tools in LMS is the threaded discussion board (also known as a 'discussion forum') which supports asynchronous communication among the participants.Nowadays, there is an increasing trend to use social media such as Facebook for online learning (Kabilan, Ahmad and Abidin, 2010).Discussion in the form of a collaborative learning approach or computer-supported collaborative learning (CSCL) is a popular form of 'activity' in both discussion board and Facebook groups when used for learning purposes.

Productive Failure
Socio-constructivist theories of learning highlight the importance of learner engagement in successful learning and emphasize complex problem-solving activities for meaningful learning.The complex nature of the problems demands that support, such as scaffolding and guidance or structures, is provided to enable learners to engage in solving them -without such support structures, learners may fail in their learning.This has led to a substantive amount of research examining students solving complex problems with the provision of various support structures and scaffolds (e.g.Cho & Jonassen, 2002;Ge & Land, 2003;Hmelo-Silver, 2004).
However, as explained in the productive failure (PF) instructional design, letting learners persist, struggle, and even fail in complex tasks that are beyond their skills and abilities may in fact enhance their learning later on (Kapur, 2010).The more they struggle while trying to master new information, the better they are likely to recall and apply that information later (ibid.).This approach is also supported by VanLehn et al.'s (2003) findings which suggested that it may well be more productive to delay support until a student reaches an impasse or a form of failure.His research shows that there is a relationship between structure and failure which should be capitalized on in the teaching and learning process by using the PF instructional strategy.

Purpose
The purpose of this study is to design a productive failure instructional cycle for adult-based interactions in CSCL discussion board groups and compare this with Facebook discussion groups.We wish to determine whether Facebook discussion groups are able to produce satisfactory learning outcomes on a par with discussion boards commonly used as academic media.

Participants
The participants were 25 second-year adult learners (age range 25 to 47) enrolled in the Bachelor of Information Technology programme at the Open University Malaysia (OUM).The students were from two programming classes taught by the same instructor.None of the students had experiences with the targeted concepts -object and classes -prior to the study.Object and class are important bodies of knowledge in programming subjects.

Research design
The instructor gave the students freedom to form their own small groups of two or three learners, resulting in five groups each using Facebook (FB) and discussion board (DB), with five dyads and five triads.Both the FB and DB groups worked for two weeks on the targeted concepts, and thus the amount of instructional time was held constant for the two conditions.A separate Facebook account was created to host the five FB groups who participated in this study (Figure 2); and the discussion board tool available in OUM's LMS, known as myVLE, was used to create the DB groups (Figure 3).Both groups took two weeks to solve an ill-structured problem using a collaborative learning approach.No extra support or scaffolding was provided during the group problem-solving in line with the PF philosophy.One ill-structured problem scenario was developed for the concepts of object and class.The problem in the task acts as a stimulus for learning to take place and represents a platform for the learners to engage in collaborative learning in their groups.The following guidelines for the preparation of a good 'ill-structured' question in the form of a task were applied to ensure effective collaborative learning among the learners (Johnson and Johnson, 1994): x The task is conceptual.
x The task requires a problem-solving approach.
x The task requires higher-level reasoning and critical thinking.
x The task emphasizes mastery.
x The quality of performance is needed.
The group discussion in DB and FB was conducted on the premise that students' learning is not so much a matter of building up correct responses or eliminating incorrect ones -the most important aspect was for students to have an opportunity in a group to test the adequacy of their ideas.The focus was on how the learners persisted in the problem-solving activity rather than on their actually being able to solve the problem successfully (Kapur, 2010).

Data Sources and Analysis
The problem-solving processes of the adult learners were analysed using both process and outcome measures with quantitative means at the group level.Because productive failure rests heavily on the nature of group dynamics, a multi-pronged group-level analysis was undertaken, using the: i. functional content of the discussions; ii.sequential patterns in the discussions; and iii. a critical thinking ratio of the group.
The first two measures can be seen as process measures and the third as a measure of group outcome.

Functional content of the discussions
The analysis of functional content provided information on 'what' the groups had discussed.Quantitative content analysis (QCA; Chi, 1997) was used to segment and code utterances.The unit of analysis was semantically defined as the function(s) that an intentional utterance served in the problem-solving process (Suthers, 2006).In this study, we adopted a functional category system (FCS) -an interaction coding scheme developed by Poole and Holmes (1995).In the FCS, every utterance was segmented into one or more interaction unit(s), and coded into categories as shown below: x Problem analysis (PA): Statements that define or state the causes behind a problem (e.g.'I think I must declare the variable here.') x Problem critique (PC): Statements that evaluate problem analysis statements (e.g.'How can you be sure that the variable must be declared here.') x Orientation (OO): Statements that attempt to orient or guide the group's process (e.g.'Let's take turns giving our ideas.') x Criteria development (CD): Statements that concern criteria for decision-making (e.g.'We need to plan the class program first.') x Solution development (SD): Suggestions of alternatives, ideas, proposals for solving the problem (e.g.'Use the second approach to solve the problem.') x Solution evaluation (SE): Statements that evaluate alternatives and give reasons, explicit or implicit, for the evaluations (e.g.'Yes, but how do we know that there should be three methods.') x Non-task (NT): Statements that do not have anything to do with the decision task.(e.g.'Why don't we continue tomorrow!').
The results of the functional content of the discussion for both groups and discussion board groups) are given in the table below.Here, the mean (M) and standard deviation (SD) scores for each of the FCS categories are presented for both of the groups.The results show that Facebook groups had interactional activity in OO, CD, SD and SE with a greater proportion of interactional activity in CD and SD, while the discussion board groups had a greater proportion of interactional activity in CD.This means that the Facebook groups had a more varied scope of discussion than the discussion board groups, but both groups were lacking in PA, PC, SE and OO.Both groups showed heavy involved in CD activity, with the Facebook group having a higher mean score in this area.The higher level of interaction for CD activity in the Facebook groups could paved the way for discussion on SD activity.
have excluded Non-task (NT) messages in the analysis, such as social-oriented postings as well as other discussion messages that do not convey clear meanings or directions.

Sequential analysis
The above analysis only provides an indication of 'what' the groups focused on, not the sequential patterns in their interactions.Sequential analysis -a technique used to detect such patterns -treats each interactional unit (defined earlier) as an observation, a coded sequence of these observations forming the problem-solving sequence of a group discussion (Erkens et al., 2003).This detects the various non-random aspects of sequences to reveal how certain types of interaction follow others more often than one would expect by chance 1992).It accomplishes this by identifying statistically significant transitions from one type of interactional activity to another (Bakeman and Gottman, 1997;Wampold, 1992).In order to perform the sequential analysis, we have used a lag-sequential analysis (LSA) tool known as multiple episode protocol analysis (MEPA) developed by Gijsbert Erkens (http://edugate.fss.uu.nl/mepa).The results of the LSA are given below in Figure 4.In Figure 4, a circled category means that groups in that condition were at least twice as likely to sustain that type of activity in a coherent cluster rather than its being spread throughout the discussion.It can be concluded from Figure 4 that the discussion board groups had focused discussion only on criteria development (CD), while the Facebook groups had focused discussion on CD and were at least twice as likely to be followed by solution development (SD) activity.Discussion board groups were likely to have a CD-CD-CD interactional sequence, while the Facebook groups likely to have CD-SD-SE interactional sequences.

Critical Thinking Ratio
One of the objectives of this paper is to quantify a critical thinking ratio (CTR) of the groups' discussion.In order to do so, we followed Newman, Webb and Cocrane's (1995) content analysis model.The theoretical concepts that support this instrument are group learning, deep learning and critical thinking.They argue that there is a clear link between critical thinking, social interaction and deep learning; and they developed a content analysis instrument based on Garrison's (1992) five stages of critical thinking.They identified ten categories: relevance, importance, novelty, outside knowledge, ambiguities, linking ideas, justification, critical assessment, practical utility, and width of understanding.For each category, a number of positive and negative indicators are formulated and most indicators are fairly obvious opposites.et al. adopt themes as the unit of analysis which can be used in domain-specific discussions such as programming.The units may be phrases, sentences, paragraphs or messages illustrating at least one of the indicators.They only mark and count the obvious examples, and ignore less clear indicators.The formula used to calculate the CTR is: where x+ is the count of statements contributing to critical thinking for the coding category and x-is the count of statements detracting from critical thinking for the category.
The minimum value of CTR is -1 (all uncritical thinking, all surface-level learning) and the maximum is +1 (all critical thinking, all deep-level learning) (Newman et al., 1995).Overall, the critical thinking ratio can be calculated by counting all the positive and negative postings in the forum and then applying the above formula.Example of how the discussion is tagged using this content analysis model is given below: As the analysis of CTR using the Newman model of content analysis is tedious and time-consuming, we analysed only three randomly chosen FB and DB groups.The critical thinking ratio calculated for these groups is given in Table 2.The findings show that the Facebook groups had a higher CTR than the discussion board groups.The Facebook groups managed to achieve a higher level of CTR as they had more positive statements in their postings.In addition, most of the learners in the FB groups had a higher CTR compared to the learners in the DB groups.

Discussion
study was designed to compare a productive failure instructional design of CSCL groups of FB and DB.We wanted to determine whether there is a hidden capacity to produce the desired result in delaying structure in the learning and performance space of adult students by having them in unscaffolded problem-solving of complex problems in these platforms.The conclusions from the study suggest that the FB groups outperformed their counterparts from the DB condition on the targeted concepts in the quality of discussion, as measured in the functional content areas, sequential analysis and the critical thinking ratio.The findings also suggest that FB groups produced a variety of scope of discussion and deliberation for solving the problems and were more successful in sustaining the discussion than the discussion forum groups.This is an important finding as it is often assumed that social media such as Facebook are not suitable for teaching and learning as they are merely a social tool.The productive failure instructional design in FB enabled the adult learners to generate and develop their own structure -such as concepts and method or approach (as demonstrated earlier in the quantitative study of the group discussion) -for solving complex problems (Kapur, 2010).The process of generating a diverse set of structures while exploring the problem and solution spaces as exhibited in the FB groups may have engendered sufficient knowledge differentiation.The 'community of inquiry' model (Garrison, Anderson and Archer, 2000) best explains the findings of this paper.
The Community of Inquiry theoretical framework represents a process of creating a deep and meaningful (collaborative-constructivist) learning experience in a text-based asynchronous learning platforms (Facebook and discussion boards are asynchronous platforms) through the development of three interdependent elements -social, cognitive and teaching presence.One of the important elements in a Community of Inquiry is social presence: 'the ability of participants to identify with the community (e.g.course of study), communicate purposefully in a trusting environment, and develop inter-personal relationships by way of projecting their individual (Garrison, 2009).Facebook, a social medium, is able to act effectively as a catalyst of social presence in order to boost academic discussion in learners/groups.

Implications of the study
Though it is difficult to identify implications from one study, with a small sample, we believe that the findings from this research can have some broader ramifications.Below, we have identified some implications that can be drawn from this study in the context of open and distance learning.
As implied greater learning takes place when students persevere and even fail in unstructured activities in the first instance, creating greater success in the next encounter with structured activity.Opportunities taken by students to attempt unstructured activities lead them to explore a wide range of actions, resulting in their being encouraged to be flexible and adaptive in the learning process.
However, perceptions of the use of Facebook and discussion forums may influence the type of responses that are elicited.Facebook is perceived as a social network site which allows communication in the form of short posts displayed asynchronously; and participation in such a site has resulted in seeing their contributions as free-flowing dialogue, and problem-solving and self-directed learning outside the classroom setting.The learning restrictions often found in a normal classroom environment are missing and this has a positive impact on the participation level, leading to meaningful learning.
The discussion forum allows students to work together on some activity in small groups or participate in ongoing discussions related to the coursework and to make some form of presentation to the others.It is often seen as an integral part of a course where participation in the discussion becomes a requirement for part of the evaluation of their performance in that course.A broad base of knowledge is acquired as collective experiences are shared on the assignment work given.
Advances in computer and communication technologies, the Internet and online education have become attractive and have revolutionized higher education, 'democratizing' it with access to a wider group of individuals.However teaching online is complex and requires understanding of how student participation can improve the level of interaction and the quality of the discussion, culminating in higher-order thinking skills and productive learning.Thus careful exploration of the technological tools available, and how they can be utilized to elicit the type of responses sought, will eventually make online education an education of choice.In this regard, Facebook seems to be a good choice.

Figure 2
Figure 2 A Facebook group

Table 1
Descriptive statistics for FCA category for the groups

Table 2
CTR for the groups and individual learners Note: The value shown in parentheses is the CTR of the individual learners in the group