STUDENTS’ EVALUATION OF SUCCESS IN AN ONLINE MENTORING PROGRAM

This paper examines the contributors to a successful online educational program. In particular, it focuses on an online mentoring program called Tracking Canada’s Past (TCP) which was implemented in three high schools in British Columbia, Canada, in 2003. Tracking Canada’s Past investigated the application of online mentoring in the high school history curriculum through the use of Knowledge Forum® software—a web-based group workspace in which students could share and discuss their ongoing research with their online mentors and other students. The goal of TCP project was to help students understand the concept of history as a discipline through online mentoring and the use of “primary” sources, in addition to standard textbooks. There were 72 students and 16 online mentors involved in this study, approximately one mentor for each group of 5-11 students. Through a series of preand post-program surveys and interviews, data were collected on the students’ backgrounds, expectations for specific mentoring functions, affective responses to mentoring, and the mentoring functions they recognized receiving. Volunteer mentors were also asked about the mentoring functions and the type of advice they would offer to their students. Findings from this study indicated that students’ judgments of a successful online mentoring program were best predicted by the helpfulness of the questions mentors asked, the usefulness of the reading materials and/or web resources they recommended, the helpfulness of mentors in developing questions or ideas to investigate, the level of trust students placed in their mentors, and the helpfulness of the online workspace where students and mentors shared their ideas. These findings suggest that the most important determinants of a successful online mentoring program are those that online program designers have the ability to refine over time.

Online educational programs have been implemented with increasing frequency in elementary and secondary schools over the last two decades.In these educational programs, it is widely acknowledged that maintaining participants' interest and engagement is vital; but little empirical research has directly addressed the question of what rewards participants seek through their participation.Program designers have their own sense of what it means for their work to succeed, but what do the participants themselves think of as success?This paper examines the contributors to a successful online educational program.In particular, it focuses on an online mentoring program called Tracking Canada's Past (TCP) which was implemented in three high schools in British Columbia, Canada, in 2003. Tracking Canada's Past was designed to help high school students develop a deeper understanding of history.Using a communal database technology called Knowledge Forum software, students shared and discussed their ongoing research with their online mentors and other students.They learned history in a way that roughly parallels historical research.By the conclusion of the project, they wrote an original historical narrative based on their research.
Online mentoring or "telementoring" programs have been frequently implemented in the context of K-12 schools in North America as a way to link students' school work with communities of specialist adults (Harris et al., 1996;O'Neill, Wagner, et al., 1996;O'Neill & Gomez, 1998;Tsikalas & McMillan-Culp, 2000;O'Neill & Scardamalia, 2000;O'Neill et al., 2003).While face-to-face mentoring has a wide array of potential benefits, orchestrating mentoring in a purposeful way within a school or community always poses challenges.For example, geographic barriers, travel, and finding time for mentors to work with their students/mentees are some of the challenges to face-to-face mentoring.On the other hand, online mentoring or telementoring alleviates the constraints of time and space and enables the mentors and mentees to write to each other whenever they have time, and from wherever they are.In addition, telementoring provides an avenue for students to pursue their interest in subjects that they may not feel comfortable in discussing face-to-face (Cravens, 2002).Orchestrating mentoring relationships online increases teachers' pool of available mentors, as well.That is, they do not need to rely on only the businesses in their communities as sources of mentors (Foster, 1999;Hamilton & Scandura, 2003).
For these reasons, designing and implementing telementoring programs have been the focus of many researchers and educators in K-12.Based on some telementoring projects, an array of recommendations has been made about designing successful telementoring programs in schools.Researchers have tried to address the question of what it means for a mentoring program to succeed.Their recommendations and guidelines tend to focus on either the achievement of the intended outcomes of the program or the success of students and mentors relationships.Little empirical research has examined the students' judgments of success.This paper is intended to develop a greater understanding of what shapes students' judgments of success in telementoring relationships.Using the data from Tracking Canada's Past project, it will address how and to what degree the functions that mentors provided for their mentees, the mentees' expectations of the mentoring relationship, and students' demographic characteristics shaped high schoolers' judgments of success in the relationships.Knowing about students' evaluation of success can help the telementoring program designers have appropriate up-front training or orientation for the mentors and mentees.
In this study, there were 72 students and 16 online mentors.Through a series of pre-and post-program surveys and interviews, data were collected on the students' backgrounds, expectations for specific mentoring functions, affective responses to mentoring, and the mentoring functions they recognized receiving.Volunteer mentors were also asked about the mentoring functions and the type of advice they would offer to their students.Findings from this study indicated that students' judgments of a successful online mentoring program were best predicted by the helpfulness of the questions mentors asked, the usefulness of the reading materials and/or web resources they recommended, the helpfulness of mentors in developing questions or ideas to investigate, the level of trust students placed in their mentors, and the helpfulness of the online workspace where students and mentors shared

WORKING ONLINE
Based on students' expressed research interests in relate to Canadian Pacific Railway and mentors' expertise, the project staff grouped each student with peers across the participating schools, and assigned a volunteer mentor with the same or similar interest to that group.Students were then introduced to Knowledge Forum workspace where they could share and discuss their ongoing research.Each working group was given its own "view" (similar to a folder) in the Knowledge Forum, and a biographical description of the assigned mentor was posted in the view.In each view, students could discuss their ongoing research with one another and with their mentor and share their sources, interpretations, and accounts.However, students and mentors were able to access all the views, and read other students' and mentors' notes whenever they wished.Each mentor was asked to log on to the system and check his/her mentees' notes three times a week if possible.
To help structure students' work over the lengthy project, a set of "milestone" assignments was provided on the project website that were written with the aid of a Ph.D. student in history.Students went through the assignments with the help of their mentors and teachers.Mentors were provided with instructions on using the Knowledge Forum software through email, and an instructional website.Students received the same instructions from the administrator of the project, Dr. O'Neill, and his research assistants.The administrator also introduced the idea of online mentoring to the students in a face-to-face session.Many of the students were familiar with MSN Messenger and chatting online with friends; however, they were not familiar with the idea of working online with an adult on a research project.Two videos were also produced for participating classes: one explaining the goal of the project, who the participants were, etc.; and the other describing the roles of the mentors and students in a telementoring relationship.Each class received a copy of the videos.
In addition to pre-training for the volunteer mentors and students in TCP project, the participants were provided with some external support from a facilitator who was a Ph.D. student in History.Besides the aforementioned supports that she provided in preparing the list of topics of interest, the facilitator followed the online dialogues, helped mentors with issues that arose, and assisted some students when their assigned mentors were not able.

DATA SOURCES
The data analyzed for this study included:  Pen-and-paper surveys administered to students and adult mentors both prior to and after the curriculum unit.Pre surveys addressed students' backgrounds, expectations of mentoring, and conceptions of historical knowledge.Post surveys addressed students' satisfaction with their mentoring relationships, appreciation of the mentoring functions they received, and conceptions of historical knowledge  Videotaped small-group interviews with some students conducted after completing the unit about the challenges of the project, including work with mentors.This was a stratified sample, based on students' level of engagement with their mentors and the success they reported with their mentoring relationships. Face-to-face or telephone interviews conducted with each mentor after the completion of the unit about their satisfaction with the experience and challenges they faced in their work with students (30-45 minutes long). Automatically generated records of students' and mentors' activities with the Knowledge Forum throughout the curriculum unit (posting and reading).

DATA ANALYSIS
Quantitative analysis, supported by the qualitative data, was used to model the variables that predict students' judgments of success in their telementoring relationships.To begin, a correlation matrix was constructed using the entire array of collected quantitative data examined the strength of the associations between students' rated agreement with the statement "overall, the mentoring was a success for me" (a Likert-type item with a scale from 0 to 7) and the following variables:  Students' prior grades in school  Their ideas about learning in school  Their academic self-conception (their orientation to subjects such as English, Math or Fine Arts)  Their parents' educational attainment  Their plans after graduation  Their relationship with their mentors, including a) their mentors' friendliness, b) the degree to which they felt their mentors read their Knowledge Forum notes carefully, c) the degree of respect they felt their mentors showed for them, d) the trust they placed in their mentors, and e) the respect they held for their mentors. Their mentors' helpfulness (the functions that students reported their mentors provided for them during the project)  The number of messages they posted and read in the Knowledge Forum  The number of hours they put into writing the final report Analyses revealed that students' impressions of the overall success of mentoring least correlated with students' prior grades, their plans after graduation, and their academic self-conception.However, the variable "success" most correlated with the mentoring functions students reported receiving, students' relationship with their mentors, and students' online activities in the Knowledge Forum.Other variables did not correlate significantly with students' judgments of success in their telementoring relationships.
In the post survey, students were asked the degree to which they felt their mentors had provided each of a set of 13 mentoring functions-types of advice, guidance or help that mentors provided for them.Items were structured on a scale from 0 to 7, where 0 meant "not at all" and 7 meant "a lot".The functions listed were:  Help me come up with a project question/idea to investigate  Ask me questions to help me think about my project  Answer questions I have about specific people, events or ideas in history  Give me background information on my topic  Give me locations on the Internet where I can find resources to answer my questions  Help me to understand material I read about my topic  Suggest challenging things for me to do that could improve my project  Review my work as I go along and help me keep on track  Give me the names and addresses of other people to contact about my project  Help me to meet project deadlines  Suggest specific strategies that will help me get my work done  Suggest books or other sources that I should read  Help me understand what historians do each day Some of the responses to these items were highly inter-correlated, but others were relatively independent.Using all the variables that correlated significantly with students' judgments of success, stepwise multiple regressions both backwards and forwards was then implemented.Three mentoring functions as well as two other variables emerged as significant predictors to students' judgments of success:  My mentor asked me questions to help me think about my project  My mentor suggested books or other sources that I should read  My mentor helped me come up with a project question/idea to investigate  The trust that students place in their mentors, and  How helpful students consider Knowledge Forum to be Of the above factors, "My mentor asked me questions" made the strongest contribution to the model, predicting 44.6% of the variance in judged success.It is interesting to note that this variable as well as the other mentoring function that best predicted success, "My mentor helped me come up with project questions or ideas to investigate" were among the functions that students least expected to receive from their mentors before the start of the project.Based on the pre-survey, only 45-50% of the students were interested in receiving such advice before the project began.For the most part, students initially expected their mentors to provide them with background information, web links, and other reading materials about their topic.
One explanation for this reversal is that students simply did not have enough of the right kinds of experience to anticipate which types of advice would be beneficial to them in the kind of authentic historical inquiry demanded by Tracking Canada's Past.While many of them entered the program with little desire to be challenged or asked questions and a relatively strong desire to receive resources and information, over time they developed a quite different sense of what types of advice were valuable to meet the demands of the curriculum unit.
As O'Neill (2008) state mentees often started the conversation with their mentors in Knowledge Forum by asking them for pointers to learning resources (web sites or books) to get started.One student remarked, "…getting started is one of the hardest parts, because you are so sort of overwhelmed by, like, 'where do I go next?'"In their closing interviews, a number of mentees remarked on their desperate need for information resources.When asked what he had expected his mentor to do at the start of the project, one student's response was very direct: "to tell us information about our topic, how to get it and where to look, and tell us factual information on it."Other responses were more nuanced, but had a similar ring: Interviewer: Before starting the project, what did you think your mentor could do for you?Student: I thought that he would give me some sources like where I could go, like some websites or some books that I could go and like find some more information on the topic, like he could give me the name of the books and find out the information I required.Interviewer: And was it the same as you experienced in the project?Student: Not really.He was more vague, the information that he gave me.He wasn't as specific as I thought he would be.Like, he didn't give me the direct web link to the place I should go and search, instead he gave me the topics that were the things I should go and look for.
Therefore, while mentors indicated in the pre-survey that they were most interested in helping students understand more deeply what historians did as well as asking the students questions, and were less interested in providing web resources, students had the desire to receive learning resources such as web sites or books due to demands on their time.They wanted to complete their work as efficiently as possible.This shows that students/mentees may not appreciate the type of the advice that their mentor is providing for them (such as asking them questions or helping them shape their project question), however, these functions are the ones that contribute to students' judgments of success in their telementoring relationships.

DISCUSSION
The purpose of this analysis was to develop a deeper understanding of what high school students mean by "success" in their telementoring relationships, by examining the variables that best predicted their perceptions of success in these relationships.While the telementoring literature provides an array of recommendations about implementing "successful" telementoring programs for K-12, the application of these recommendations should ideally be informed by some detailed knowledge of the expectations that mentees carry into their telementoring relationships.To this end, I examined in detail 72 adolescents' expectations of their relationships with assigned telementors in a curriculum-based program, both prior to and after the project.I looked at their expectations with their mentors', and computed regression models to see which of the many variables in the dataset best predicted students' impressions of the overall success of their telementoring relationships.
Analyses revealed that students' judgments of success were best predicted by the helpfulness of the questions mentors asked, the usefulness of the reading materials and/or web resources they recommended, the helpfulness of mentors in developing questions or ideas to investigate, the level of trust students placed in their mentors, and the helpfulness of the online workspace where students and mentors shared their ideas.Variables such as students' average grades in school, their academic self-concept, and their plans for work or further schooling after graduation did not contribute any additional predictive power to the regression model, though they were significantly correlated with the "success" variable.This is an encouraging finding, since it suggests that even in a group of participants as diverse as TCP peroject's, the most important determinants of success are those that program designers have the ability to refine over time.Knowing about these factors will have important implications for the design of curriculum-based telementoring programs and their supporting materials.They help the telementoring program designers prepare better pre-training and orientations for their mentors and students/mentees.
It is interesting to note that among the variables that best predicted success, "My mentor asked me questions" and "My mentor helped me come up with project questions or ideas to investigate" were among the functions that students least expected to receive from their mentors before the start of the project.Only 45-50% of the students were interested in receiving such advice before the project began.For the most part, students initially expected their mentors to provide them with background information, web links, and other reading materials about their topic.This implies that students had little ability to predict which functions would actually be of value to them in the course of project.While many of them entered the program with little desire to be challenged or asked questions and a relatively strong desire to receive resources and information, over time they developed a quite different sense of what types of advice were valuable to meet the demands of the curriculum unit.
The implications of these findings, should they be reproduced by other researchers in different settings, are substantial.They suggest that mentees' initial expectations of their mentors in a program similar to TCP need to be respected, but not slavishly served.Mentee expectations can be both informed through up-front training, and transformed through the experience of being in relationship with a mentor.Furthermore, detailed knowledge of the expectations that mentees generally have for their mentors may allow program designers to provide more insightful guidance for mentors.They can be prepared to anticipate mentees' naïve expectations, and satisfy them enough to keep their charges engaged, but also plan to move them toward transforming these expectations into more mature ones.