Floundering among adult learners in classrooms : Fact or fallacy ?

Blended pedagogy is an important delivery mechanism for open and distance learning. Here, face-to-face meetings in the blended pedagogy model remain as an important platform for teaching and learning. While there are many instructional techniques employed in faceto-face meetings, there is an urgent need to determine how face-to-face interactions in the blended pedagogy can be elevated to boost students’ learning. This paper investigates whether productive failure (PF) as an instructional strategy boosts students’ understanding of the subject matter in a face-to-face tutorial. PF instructional design advocates the delaying of support for learners during learning. The more they struggle, and even fail, while trying to master new information, the better they are likely to recall and apply that information later. Current research on the impact of PF treatments has shown that effective learning is achieved when learners are presented with a cycle of low structure, high structure and low structure activities. PF instructional design has been used successfully, especially in secondary schools in which learners have regular contact with the instructor. It is unknown whether the use of PF instructional design among adult learners in face-to-face interaction will yield such a positive effect. Can PF instructional design be used in tutorials that cater for adult learners resulting in fruitful learning outcomes? This paper reports an initial study of a quasi-experiment that compares a ‘‘productive failure’’ instructional design with a traditional ‘‘lecture and practice’’ instructional design for a 2-hour tutorial session attended by adult learners. A total of 17 adult learners participated in the study. Learners experienced either a traditional lecture and practice teaching cycle or a PF cycle, where they solved complex problems in small groups without the provision of any support or scaffolds up until a consolidation lecture by their teacher during the last hour of the tutorial. Findings suggest that learners from the PF condition produced a variety of problem models and methods for solving the problems but were unsuccessful in their efforts, be it in groups or individually. They also reported low confidence in their solutions. Despite failing in their group and individual problem-solving efforts, learners from the PF condition performed better than their counterparts from the lecture and practice condition on both knowledge and higherorder application problems based on the post-test. Implications of PF instructional design for adult learners based on these findings are presented. Floundering among adult learners in classrooms: Fact or fallacy?


Introduction
Open and distance education (ODE) is fast becoming the way of providing education to the masses.ODE provides the opportunity for working adults to enrol in programmes that match their interests without leaving their job.Adult students are the main subscribers of ODE. Adult students are loosely identified to a larger group characterized as "non-traditional".While definitions vary, the National Center for Education Statistics (NCES) in United States (US) has come up with seven characteristics that typically define non-traditional students.According to the NCES, adult students often:  Among all the dimensions in Figure 1, the teaching and learning process is the most important component as it directly involves imparting knowledge to learners.This is also the place where academics interact with learners.Besides empowering students to adapt to current and future environments and to find solutions to challenges, learning is also the measure of the need that adults feel for connecting education with its application.As a consequence of this, colleges and universities continuously seek to impart critical thinking and problem-solving skills to their students through the curriculum.But acquiring these skills demands both experiential and problem-based methods.Teaching and learning in ODE is typically conducted via the blended pedagogy.In a typical scenario, the three components that make up blended learning are: face-to-face tutorials, online learning and self-managed learning (see Figure 2).The blend of these three components appears to have been the solution that has worked rather well for the majority of adult learners.
Figure 2 Means to manifest the teaching and learning process at Open University Malaysia (OUM): Blended pedagogy F2F meetings remain an important component as they enable learners to meet their instructors in direct communication.In a survey by Zoraini (2004), learners preferred F2F learning compared to the other components.Various instructional techniques have been employed to deliver F2F interactions.These include tutorials, brainstorming, hands-on sessions, problemsolving and presentation sessions.How can face-to-face interactions be improved further in blended pedagogy so that adult learners gain valuable knowledge?In this paper, we assess whether the productive failure (PF) instructional technique can boost students' learning.
PF instructional design has been used successfully, especially in secondary schools in which learners have regular contact with the instructor.It is unknown whether the use of PF instruction among adult learners in face-to-face interaction will yield such a positive effect.Can PF instructional design be used in tutorials such as in the blended pedagogy that caters for adult learners with fruitful learning outcomes?This will be the main focus of this paper.

Productive failure
Socio-constructivist theories of learning stress the importance of engaging learners to achieve successful learning.They stress on complex problem-solving activities for meaningful learning.The complex nature of the problems demands support such as scaffolding and guidance, or that support structures be provided to enable learners to engage in solving such problems.Without such support structures, learners may fail in their learning.This has led to a substantive amount of research examining students solving complex problems with the provision of various support structures and scaffolds (e.g., Cho & Jonassen, 2002;Ge & Land, 2003;Hmelo-Silver, 2004).
However, letting learners persist, struggle and even fail at tasks that are complex and beyond their skills and abilities, may in fact be a productive exercise that will enhance their learning process later on as is explained in the productive failure (PF) instructional design (Kapur, 2010).PF instructional design advocates the delaying of support for learners during the learning process (Kapur, 2010).The more they struggle and even fail while trying to master new information, the better they are likely to recall and apply that information later (Kapur, 2010).This is also supported by Van Lehn et al.'s (2003) findings, which suggest that it may well be more productive to delay that structure up until the student reaches an impasse or a form of failure.His research shows that there is a relationship between structure and failure which should be capitalised in the teaching and learning process by using the PF instructional strategy.

Purpose
The purpose of this study was to design a productive failure instructional cycle for adult-based interactions in a face-to-face tutorial and compare it with a conventional lecture class (i.e., strictly direct instruction).We wanted to determine if there is a hidden capacity to produce the desired outcome in the delaying structure in the learning and performance space of adult students.This is done by having them engage in un-scaffolded problem-solving of complex problems first, before the direct instruction.To achieve this, one classroom-based, quasiexperimental study with first-degree level adult students was carried out; each study targeting a 2-hour tutorial class.

Method
The research approach adopted for this study is discussed in the subsequent sections.

Participants
Participants (n = 17) were second-year students enrolled in the Bachelor of Information Technology programme (13 male, 4 female; 25 -47 years-old) at Open University Malaysia (OUM).Students were from two programming classes (8 and 9 respectively) taught by the same instructor.The average-ability students were assessed on their prior knowledge of programming through a pre-test.Students had limited or no experience with the targeted concepts  objects and classes  prior to the study.

Research design
The research design for this paper was adopted from Kapur (2010).A quasi-experimental design was used with one class (n = 8) assigned to the 'productive failure' (PF) condition and the other class (n = 9) assigned to the 'lecture and practice' (LP) condition.Both classes participated in the same number of tutorial hours for the targeted concepts, totalling two hours each.Thus, the amount of instructional time was held constant for the two conditions.Before the unit, all the students did a 20-minute, 7-item pre-test (Cronbach's alpha = .70)as a measure of their prior knowledge of the targeted concepts.There was no significant difference between the two conditions on the pre-test, p < 0.05.At the end of their two-hour tutorial class, all the students took a post-test (described later in the paper).

Productive failure (PF) class
The eight students in the PF class were assigned into groups.The instructor had given the students the freedom to form their groups resulting in three groups (two triads, one dyad).In the PF instructional design cycle, the student groups' took 45 minutes to work face-to-face on the ill-structured problem.Following this, the students took 15 minutes to solve one problem individually.The individual problems were designed as an extension of the ill-structured problem discussed earlier by the learners in a group.This extension problem required the students to write brief programming code.No extra support or scaffolds were provided during the group or individual problem-solving.The consolidation lecture was held towards the last one hour of the class where the instructor led a discussion of the targeted concepts.One illstructured problem scenario was developed for the concepts of object and class.The problem in the task acted as stimuli for the learning to take place and represented a platform for the learners to engage in collaborative learning in their group.The following guidelines for the preparation of a good "ill-structured" question in the form of a task were applied in order to ensure effective collaborative learning among the learners (Johnson & Johnson, 1994).They are: • The task is conceptual.
• The task requires a problem solving approach.
• The task requires higher-level reasoning and critical thinking.
• The task emphasizes mastery.
• Quality of performance is needed.
The group discussion in the PF class was also conducted on the premise that the learners' learning is not so much a matter of building up correct responses or eliminating incorrect responses.The most important thing is for students to have the opportunity in a group to test the adequacy of their ideas.It is the process of how the learners "persist" in the problem solving activity rather than actually being able to solve the problem successfully (Kapur, 2010).

Lecture and practice class (LP)
The nine students in the LP class were involved in instructor-led lectures guided by the same question discussed by the PF group but in a more structured manner.The teacher introduced the same concepts taught in the PF class, formulated a solution and encouraged students to ask questions.The teacher then discussed the solutions with the class using the guided handson approach.In short, the LP condition represented a design that was highly structured from the very beginning, with the teacher leading the students through a set of well-structured problems with feedback and practice sessions.
The PF condition represented a design that delayed structure (in the form of the consolidation lecture) up until the time when the students had completed the one ill-structured problem scenario and the corresponding individually solved problems without any instructional facilitation, support structures, or scaffolds.

Data sources and analysis
The data analysis procedures and the results are described in the following section.The problem-solving process followed by the adult learners was analysed using both process and outcome measures with qualitative and quantitative means.For qualitative process measures, we analysed the program code produced by the groups and also the group discussions.We audio taped the PF group's discussion.Group discussions were analysed to understand how solutions produced by the groups related to the various qualitative and quantitative methods  both domain-general and domain-specific  that groups used in their attempt to solve the problems.Quantitative process measures included analyses of group solutions, individual solutions to the extension problems and the corresponding confidence ratings.Outcome measures focused on the performance on the post-test.

Process analyses and results
A qualitative analysis of the group discussions revealed that groups, in general, generated several concepts and methods for solving the problems.These concepts and methods can be seen as structures generated and developed by the group to solve the problem (Anderson, 2000;Chi et al., 1981;Schwartz & Martin, 2004).The focus of analysing group discussions was limited to an analysis of the relationship between these student-generated structures, that is, how the solutions produced by the groups related to the various qualitative and quantitative methods  both domain general and domain-specific  that groups discussed and used in their attempt to solve the problem.Here is a solution submitted by one of the groups: We present an analysis of the discussion of the group whose solution is shown in Figure 3. Due to space constraints, only three excerpts from the group's discussion are presented in the order that they occurred.Note that the excerpts have undergone some minimal editing for language and grammar to make them more comprehensible.Excerpt 1 starts with S1 asking how to start.S2 advises the group members to read the question so that the group understands the problem better.After reading the question, S3 opines that they should write one program.S2 sounds confused.S1 counters that the group should write two programs.Obviously, in this excerpt, the group is still confused on how to solve the problem.In excerpt 2, S2 asks the group members to declare the "numbers".S1 and S3 agree that "numbers" need to be declared first.S1 volunteers to type the program into the computer.From excerpt 2, it is obvious that the group did not follow the "class-object" approach as instructed in the question.The group tried to write the code in an "ad-hoc" manner by creating disconnected code links.In excerpt 3, S3 asks the group members to type the class name first.Then, there is confusion whether to declare the "numbers" or to declare the "methods".Then, S1 comes in with another question on whether the program should be interactive.Excerpt 3 shows that the group ends up in more confusion as the code expands.
The analyses of the group solutions and the group discussions suggest that the group was ultimately unable to succeed in developing a solution to the problem although they brought in various constructs and ideas during the discussion.Some of these structures were rejected quickly through a guess-and-check method while others were ''abandoned'' when new ideas resulted in an understanding that the solution was not suitable for the problem.Classroom observations also suggested that groups were quite engaged and tended to persist in the problem-solving process although they were not able to solve the problem correctly.

Group solution
Analysis of the groups' solutions suggested that all groups were able to come up with an incomplete solution and it did not meet the problem's requirements.Based on the analysis done by the instructor, no group submitted an acceptable solution.

Confidence ratings
After solving the individual extension problems, students in the PF design condition rated the confidence they had in their solutions using a five-point Likert scale from 1 (not strongly confident) to 5 (strongly confident).The average confidence reported by students was low, M = 2.67, SD = 0.89.This is reflective of the poor group performance as discussed earlier.

Outcome analyses and results
Individual outcomes were measured using one post-test.The post-test targeted the content/ concepts (i.e., object and class) covered during the two-hour tutorial for both the PF and LP groups.

Post-test result
Students from both the PF and LP classes were given 20 minutes to complete a seven-item post-test (Cronbach's alpha = 0.71) comprising six well-structured problem items (in the form of multiple-choice questions) similar (not identical) to those on the pre-test as well as one item on a higher-order application-based problem.Students needed to write brief program code for this higher-order application-based question.
Controlling for the effect of prior knowledge (as measured by the pre-test), an ANCOVA revealed a statistically significant effect of condition (PF vs. LP) on post-test scores, F (1, 14) = 7.65, p = 0.0152, ES (effect size) = 0.35.The adjusted (i.e., after controlling for prior knowledge) mean performance of students in the PF class, M = 6.56,SD = 2.72, was better than that of students in the LP class, M = 3.67, SD = 3.423; an average difference of 41% points given that the maximum score possible on the post-test was 10.Levene's test for homogeneity of variance was not significant.
We also conducted further analysis by considering the well-structured and higher-order application items on the post-test separately.The findings are highlighted below: 1. On the well-structured items, students from the PF class scored higher, M = 4.13, SD = 1.46, than those from the LP class, M = 2.56, SD = 1.88.This effect was statistically significant, F (1, 14) = 5.22, p = 0.038, ES = 0.27.This difference amounted to 45% points (maximum score on these items was 6) with a low effect size.Notwithstanding this, it was remarkable that the PF students who were not able to come up with a proper solution during the group discussion still managed to outperform LP students on the well-structured items; the type of items that the LP students were introduced to during the lecture in their 2-hour session.
2. On the higher-order analysis and application item, students from the PF class scored higher, M = 2.56, SD = 1.45, than those from the LP class, M = 1.11,SD = 1.76; an average difference of 36% points (maximum score possible on this item was 4).This effect was statistically significant, F (1, 14) = 4.55, p = 0.043, ES = 0.26.Thus, students from the PF class outperformed those from the LP class on both the wellstructured items as well as the higher-order application-based item in the post-test even in a relatively-short, 2-hour design intervention.These results are also in agreement with the findings of Kapur (2010).

Discussion
This study was designed to compare a productive failure instructional design to a conventional lecture and practice instructional design.We wanted to determine if there is a hidden capacity to produce the desired result in delaying structure in the learning and performance space of adult students by having them engage in un-scaffolded problem-solving of complex problems first before direct instruction.The results of the study showed that despite seemingly failing in their collective and individual problem-solving efforts, students from the productive failure condition significantly outperformed their counterparts from the lecture and practice condition on the targeted concept in the post-test.This is an important finding because an often-made assumption in teaching and learning is that students need structure such as scaffolds to elevate their learning (Kapur, 2010).From the teacher point of view, structuring the lesson from the beginning is a good pedagogical approach.However, the findings of this study suggest that this assumption is not always valid.The productive failure instructional design has enabled the adult learners to generate and develop their own structure, such as concepts and methods or approaches (as demonstrated earlier in the qualitative study of the group discussion), for solving the complex problems (Kapur, 2010).The process of generating a diverse set of structures while exploring the problem and solution spaces may have engendered sufficient knowledge differentiation even though it did not result in a successful solution.Such knowledge differentiation was critical for learning because it prepared the students to better discern and understand those very concepts, representations, and methods when presented in a well-assembled, structured form during the consolidation lecture (Gibson & Gibson, 1955;Marton, 2007 as cited by Kapur, 2010).In the following section, the implications of the findings of this study are discussed.

Implications of the study
Although it is difficult to draw implications from one study; what more with a small sample, we believe that the findings from this study do lead to some broader implications.We have identified three broad implications that can be concluded from this study in the context of open and distance learning.Firstly, learners need to be actively engaged.Learning by doing, analogy and assimilation are important elements in pedagogy and, where possible, learning outcomes should relate to real-life experiences through simulation and application.Secondly, the learning environment should include problem-based as well as knowledge-based learning.
It has been acknowledged that if problem-based assignments are provided it would involve the higher-order thinking skills like analysis, synthesis and evaluation, while the lower levels of recall, comprehension and application are evidenced in knowledge-based learning.Finally, learning experiences support interaction and development of community and social interest.
Anything that relates to experiences is based on interaction and collaboration, which in turn lead to networking and support groups that enhance learning.Multiple interactions, group collaborations and cooperative learning provide an increased level of interaction.

Limitation of the study
It is definitely too early to attempt a broad generalisation of the claims based on a single study.These findings may only be attributed to the productive failure instructional design as a whole and not to its constituent design elements of collaboration, un-scaffolded solving of complex problem scenarios designed for persistence, and delay of structure (Brown, 1992).The main aim of this paper was to put greater emphasis on a comparison of designs as wholes vis-à-vis causal attribution of effects to design elements (Fishman et al., 2004;Tatar et al., 2008;Tharp & Gallimore, 1982).As such, this study, essentially, presents proof for a productive failure instructional design, and allows much space for future research.The research was done with a limited, small sample and to draw generalizations from it might seem premature.However, it is pertinent to realise that this research can be extended to more learners in the ODL mode in order to draw conclusions on the positive nature of the productive failure instructional design.
(a) have delayed enrolment into postsecondary education; (b) attend part-time; (c) are financially independent of parents; (d) work full-time while enrolled; (e) have dependents other than a spouse; (f ) are a single parent (in some cases); (g) do not possess the standard high school diploma.Serving adult learners and conventional learners are two different extremes.According to the Council for Adult and Experiential Learning (CAEL) based in the United States, there are eight principles of effectiveness for serving adult learners as given in the following figure.

Figure 1
Figure 1 Principles of effectiveness for serving adult learners

Figure 3
Figure 3 Solution submitted by one of the groups us read the question.S3: I think we need to write one program.S2: Then how? S1: I think we need to write two programs.
declare the numbers.S1: I think we should declare the numbers first.S3: Let us declare the numbers first.S1: Let me type in the computer.
the class and the name first.S3: We name the class "student".S1: Are you sure we need to declare the numbers first?S1: Because we are supposed to have the member method.S2: Why don't we look at the program that we wrote last week?S1: Is the program interactive?

Table 1
shows the PF and LP instructional cycle.

Table 1
PF versus LP