Collaborative information searching as learning in academic group work

Dan Wu (School of Information Management, Wuhan University, Wuhan, China)
Shaobo Liang (School of Information Management, Wuhan University, Wuhan, China)
Wenting Yu (State Intellectual Property Office of the People’s Republic of China, Beijing, China)

Aslib Journal of Information Management

ISSN: 2050-3806

Publication date: 15 January 2018

Abstract

Purpose

The purpose of this paper is to explore users’ learning in the collaborative information search process when they conduct an academic task as a group.

Design/methodology/approach

This paper presents a longitudinal study for a three-month period on an actual task. The participants, who were undergraduate students, needed to write a research proposal in three months to apply for funding for a research project, including a three-hour experiment.

Findings

The results show that undergraduates’ learning in the collaborative search process for academic group work included knowledge reconstruction, tuning, and assimilation. Their understanding of the topic concepts improved through the process, and their attitudes became more optimistic. Besides, the learning in the collaborative information search process also enhanced participants’ skills in communication, research, information search, and collaboration. To improve learning outcomes, professional and appropriate academic resources are required, as well as effective division of labor, positive sharing behaviors, and use of collaborative systems.

Practical implications

The future development of collaborative information search systems should focus on the needs of academic research and support for elements such as instant communication and knowledge sharing.

Originality/value

This paper contributes to research into searching as learning by understanding undergraduates’ collaborative search behavior for writing a proposal.

Keywords

Citation

Wu, D., Liang, S. and Yu, W. (2018), "Collaborative information searching as learning in academic group work", Aslib Journal of Information Management, Vol. 70 No. 1, pp. 2-27. https://doi.org/10.1108/AJIM-03-2017-0063

Download as .RIS

Publisher

:

Emerald Publishing Limited

Copyright © 2018, Emerald Publishing Limited


Introduction

Learning can include the act of acquiring new knowledge and values, modifying and reinforcing existing behaviors and skills, and synthesizing information from different sources (Schacter et al., 2009, 2011, p. 264). It occurs in workplaces, daily life, and all other sectors of life. Past studies have shown that web-based learning activities can greatly help students to become active and self-directed learners, which often involves information searching tasks (Bilal, 2000; Hwang et al., 2008). Although searching used to be considered a process of interacting with information, such as information finding, selection, and use, it is, in fact, a natural learning behavior (Yin et al., 2013). Individuals search to acquire new knowledge or to restructure existing knowledge structures to meet their information needs, as well as to support their learning.

Studies about learning while searching have long been active topics in information literacy, which draws connections between information search and learning skills. Morris and Teevan (2010) defined collaborative search as “the subset of social search where several users share an information need, and actively work together to fulfill that need” (p. 2). It is common in learning activities. Users will collaborate when completing a task for which they do not have sufficient individual knowledge and will obtain help from others who are learning individually. Thus, users can also learn from each other and make progress together. Hence, learning in the collaborative information search process is not simply a set of search results, it is also a process of learning together and learning from each other. Within this context, we are curious about knowledge construction and its effect on people. Moreover, as technologies develop, collaborative search systems are needed to foster and enhance the learning experience during the search process, rather than for information searching only.

In this study, we aim to gain a further understanding of multi-member groups’ learning when they work as a group to complete an academic task that lasts for an extended period. We want to understand their behavior characteristics during learning in the collaborative information search process, the changes in their sentiments, and the role that the collaborative search system played in this process. Thus, we propose the following research questions:

RQ1.

How do undergraduates gain knowledge in the collaborative information search process?

RQ2.

What are the results and implications of undergraduates’ learning in the collaborative information search process?

We analyzed the time duration of different behavior in each group, the relations between group members’ queries, the use of learning resources in different stages, as well as the labor division to respond to the RQ1. Then we studied the results of academic task accomplishment, the members’ sentiment changes in in different stages, and their learning gains. Besides, we investigated the evaluation of and suggestions for the collaborative search system, to respond to the RQ2.

This study does not only focus on the personal search behavior, but also focus on behaviors in a more complex collaborative environment, which is based on the natural longitudinal study. We revealed the learning phenomena in the collaborative information seeking (CIS) process and enriched people’s understanding of CIS behavior.

The paper is organized as follows. First, we present a review of related works before describing our research design and methods. Next, we report the findings and analysis of our research results, and finally, our conclusions and suggestions for future works are given.

Related work

Computer-supported collaborative learning

Collaborative learning is a multi-disciplinary field that is relevant to areas such as education, psychology, sociology, computer science, and information science and technology. Dillenbourg (1999) argued that research on collaborative learning should focus on defining four aspects: situation, interactions, process, and effects. Stahl et al. (2006) considered that collaborative learning consists of individuals in groups as well as activities such as communication and sharing. Recent studies have tended to consider group task performance as a part of the evaluation of collaborative learning effects, rather than focusing only on individual task performance (Dillenbourg, 1999; Stahl et al., 2006). Reynolds (2016) focused on the relationship between process and learning results through the collaborative information search and knowledge-building practice of American middle-school students in school game design, and pointed out that student tasks, collaborative information search forms, and query results are related. Researchers also studied the benefits of collaborative systems or platforms for collaborative learning. Social interaction is recognized as a factor in collaborative knowledge gains; Aalst (2009) found that social interactions are one of the leading factors in group tasks.

From the view of information science, studies on collaborative learning cover aspects of collaborative search behavior and systems in collaborative learning settings. Some researchers have explained collaborative learning in the context of information seeking. For instance, Kim and Lee (2014) studied the complex dynamics of knowledge construction and information seeking in three different stages of a collaborative research project, and their results showed that students’ knowledge grew as the project progressed. The study also found that students who collaborated in research were confident when they formed a shared understanding of the topic at the beginning of the project. When conducting their information-seeking activities individually, students would feel more stressed. Sormunen et al. (2013) explored cooperative and collaborative learning models, together with information seeking and use in work tasks. By interviewing students following their group projects, they found that few student groups worked closely together. Some started in a similar way, but then divided the work into independent subtasks, working more loosely. This could waste much of the groups’ potential for collaborative learning.

Searching as learning

Various studies have been conducted, covering several dimensions. Some have explored the role of information searching in learning, and factors and concepts related to searching as learning (Hansen and Rieh, 2016; Vakkari, 2001; Wildemuth, 2004). Zhang and Soergel (2016) studied the conceptual changes in knowledge structures that take place in different forms, especially in the process of constructing significant meanings. Freund et al. (2016) believed that reading is the core component of the search, and studied the text on the understanding of the environment and the impact of learning.

The application of existing tools or models in searching as learning is another hot topic. For instance, individuals can integrate knowledge gains into the search process through concept maps and understand the relationship between human sense making, learning, and information seeking (Stange et al., 2014). As a visualization tool, concept maps can be used to evaluate changes in the user’s knowledge structure and also to evaluate the user’s search behavior (Egusa et al., 2014). Jansen et al. (2009) classified search tasks to verify whether there were some special characteristics in the learning process. They found that web searchers relied more on their own knowledge and information needs, while the searching was used mainly to check facts.

Other researchers have studied how the performance of searching as learning can be improved and have put forward new models, systems, and other methods. A new cognitive model named “CoLiDeS+Pic” was used to simulate the web-navigation process for semantic information from pictures (Oostendorp and Aggarwal, 2014). Karanam et al. (2016) found “CoLiDeS+” could predict more paths with the correct hyperlinks and has higher path-completion rates than “CoLiDeS,” providing accurate support during navigation and enhancing the performance of information searches. Bah and Carterette (2014) created a system to support continuous searching: this first created typical pseudo-documents and then sorted the information from the retrieval results according to how closely it matched the typical document. Saito and Miwa (2007) designed a learning environment that supported learners’ thinking activities while searching the information, and evaluated its educational effectiveness. This environment could visualize the learner’s search process and support two thinking activities, and could promote thinking by comparing the learner’s own processes to those of others. As an important aspect of information seeking, relationships between information needs and types of learning during the health information-seeking process have also been uncovered (Pian and Khoo, 2014).

From these studies, we can see that learning is embedded in the search process and in all aspects of individuals’ lives. However, existing studies have mainly focused on individual behaviors, and more studies are needed that focus on searching as learning under more complex collaborative contexts.

Collaborative search in academic scenarios

There are a number of terminologies that are closely related to collaborative search, such as CIS (Shah, 2014), collaborative information behavior (CIB) (Reddy and Jansen, 2008), and computer-supported cooperative work (CSCW) (Rodden, 1991). When individuals collaborate, they often require information search in their collaborative project (Shah, 2012). Other collaborative behaviors such as planning, sharing, editing, and synthesis are also involved when undertaking academic projects. Thus, the process of writing research proposals can be treated as an intersection of collaborative searching and collaborative learning.

Several studies have been carried out on collaborative searching and behaviors in academic scenarios (Foster, 2006), or in academic groups (Leeder and Shah, 2016). Renugadevi et al. (2014) found that collaborative searches could improve the relevancy of search results. Blake and Pratt (2006) conducted an observational study of scientists to understand their CIBs when resolving different experimental results. They put forward five suggestions for information systems to better support users’ synthesis activities. Hyldegård (2006) investigated whether Kuhlthau’s ISP model, which is used to describe an individual’s information-seeking process, could be applied to group members’ information behavior. In 2009, he conducted another experiment to gain further understandings of this issue. By analyzing library and information science (LIS) students’ activities and cognitive and sentiment experiences during the task process for a project assignment, he found that similarities and differences in behavior were between group and individual in the ISP model, so the model could not fully comply with collaborative problem-solving process and the information-seeking behavior involved (Hyldegård, 2009).

In summary, there have been many studies on computer-supported collaborative learning, searching as learning, and collaborative search in academic scenarios from different research perspectives. Researchers have mainly focused on individual behaviors in searching as learning, while we focused on more complex collaborative contexts. As for collaborative learning, search behavior and its influences on learning effects should be given more attention. Furthermore, studies on collaborative search in academic scenarios should focus on the promotion of searching on learning results.

Methodology

Work tasks

This study, conducted from January to March 2016, involved five groups of LIS undergraduates who were working on their research proposals to apply for funding for research projects. In this competition for research projects, a committee evaluates whether or not a group is successful based on the proposal. Participants in our study had the task of writing the research proposal, and we asked all groups to truthfully share with us their whole proposal-writing process, which lasted for three months. During this period, all the groups went through the stages of formulating a research topic, searching and reviewing relevant information, and finally, writing a research proposal. We did not impose any restriction on the research topics selected by the groups.

In contrast to previous studies, this study was carried out on a longitudinal basis in an actual academic setting, which enabled us to observe changes in participants’ sentiments, learning skills, collaboration skills, etc. In addition, it contributed to better observe and understand the behaviors of the same person(s) over time (Wildemuth, 2009). Furthermore, part of the learning results was based on the real results of the research project competition, which was more reasonable and reliable.

Task procedure

The work task for each group was divided into three stages. In the first stage (in January), group members could discuss and decide on the topic of their research proposals, and search for relevant information.

The second stage included an experiment based on a collaborative search system, Coagmento, with each group initially writing a proposal within the prescribed time. Because time was limited, it was difficult for the groups to finish the task, so we did not require the completion of proposals. Each group had 30 minutes’ training to become familiar with the system, two hours to search and write the proposal, and 30 minutes to answer a questionnaire. A group interview about participants’ collaboration in the procedure was conducted after the experiment. Since the writing is started at this stage, resulting in more data, we focused on the analysis of user behavior at this stage.

In the final stage, each group continued to revise and improve their proposals, and then submitted the final versions to committee. Following this, we conducted an interview with each group based on their performance during the whole task.

Data collection

To better uncover the collaborative behavioral patterns, we adopted various methods for collecting data.

At the beginning of the work task, structured diaries were used to collect participants’ background information, details of their habits when using learning resources, their assessment of the importance of different learning resources, as well as their sentiments regarding the task (see Appendix 1).

Following the search stage and the writing of the proposal in the three-hour experiment, we used questionnaires (see Appendix 2) to investigate participants’ sentiments, their cognition in relation to communication and information sharing during the collaborative information search process, and their evaluation of the collaborative search system’s functions.

During the three-hour experiment, all groups were asked to use Coagmento (Shah, 2010), a well-known collaborative information search system, to write the research proposal. Coagmento is a web browser plug-in based on user interaction rather than algorithmic mediation; users browse we pages and submit queries to public search engines. We chose it for our experiment because it is open-source and supports the Chinese-language environment. It can also provide functions to support information sharing, information rating, communication, collaborative reporting, and resource management. Specifically, it can be used to annotate, recommend, collect, and co-edit information, as well as to chat with others. However, the chat function does not support the posting of images, so instead we used QQ (popular instant messaging (IM) software similar to MSN Messenger that the students use frequently in daily life). During the experiment, group members worked on their own computers, and could only interact with each other via Coagmento or QQ. Participants wrote their research proposals collaboratively on EditPad in Coagmento, and their editing behaviors were recorded. There was no break during this experiment. Participants’ searching behaviors (recorded using screen-recording software), system logs, and chat contents were also collected. The participants mainly completed their draft of research proposals via Coagmento, so the data on Coagmento is more than other two stages.

After all groups had completed and handed in their final proposals, we examined participants’ labor division mode, the problems and successes, and suggestions on improving collaborative search systems, in order to understand their whole research processes by group interviews (see Appendix 3). The research is conducted in the actual settings, and there exists a committee to evaluate the proposals of groups. The evaluations were based on the certain appraisal mechanism, so we did not conduct pre-test or post-test, adopting the final judgment of the committee.

Participants

We recruited participants from groups involved in the undergraduate students’ research project competition, a university-level extracurricular academic activity held at Wuhan University. The organizing committee requires that each group must contain three to five people, and does not intervene in grouping, so the number of students in each group may not be identical. Five groups in this work were selected randomly from all the groups. In total there were 20 participants, consisting of one three-member group, three four-member groups and one five-member group. All of them were from the School of Information Management, Wuhan University, and from different majors, as shown in Table I.

The group members had known each other before our research, and they formed their groups spontaneously. Each group was paid ¥300 for their participation. We also investigated their habits in using learning resources (i.e. search engine, library and database, PDF, web) (see Appendix 1.2).

Data analysis method

We used quantitative and qualitative data analysis methods. We used the statistical methods, such as one-way ANOVA, to analyze the structured diaries and questionnaires through SPSS 20. The data follow the normal distribution, and we also carried out the homogeneity test of variance.

We applied descriptive statistics for screen recordings and system logs. The interview records were transcribed and analyzed thematically (Boyatzis, 1998) by ATLAS.ti 7, based on the grounded theory.

A qualitative analysis was used to encode the interview records. We used open coding and axial coding to represent the coding of the first level and second level. We recruited two teachers in the LIS domain to conduct this work, which helped to avoid results coding being too subjective. They first classified and described the original interview records in the short sequence of words, which were the open coding (the first level coding). Then they analyzed the concept of these words and merged them to a same concept, which was axial coding (the second level coding). We tested the validity of the coding results. The consistency test method was used to compare the coding results of two teachers to validate the consistency of coding. We used a formula, CA=(2×S)/(T1+T2), to test the consistency of coding results. The S in formula represents the same coding results from two teachers, T1 and T2 represents the total number of codes from two teachers, respectively. If the value of CA was close to 1, it indicated the higher consistency of coding results (Xu and Zhang, 2005).

Results analysis

Group members’ learning behavior

Time duration of different behaviors

In the experiment, this paper analyzes the periodical characteristics of collaborative academic search behavior from the longitudinal angle to understand members’ various behavior in learning, i.e., the connection between time and behavior.

Our study analyzed the temporal characteristics of participants’ collaborative behavior during academic tasks from the longitudinal perspective. This was based on the data on click-through from the screen-recording software. We used the timeline to show the process of collaborative academic searches, as shown in Figure 1. The ordinate represents the members in each group, while the abscissa represents the time duration. The different colors of the boxes in the figure represent the various types of behavior in the corresponding time period, and the length of the boxes represents the time duration of the behavior. Unfortunately, a 30-minute delay occurred for Group 4 owing to an unexpected system fault, but members of Group 4 still finished the tasks in this period.

As can be seen from Figure 1, the participants exhibited the following behaviors in the collaborative academic search process: searching on search engines, searching in databases, sorting out the literature, sorting out ideas, browsing the literature, discussion in groups, browsing websites, and writing. We grouped these academic behaviors into five types: discussion, searching, browsing, sorting out literature, and writing. Browsing took the most time, followed by searching and writing. In fact, these behaviors were not isolated, and often appeared alternately or simultaneously; for example, browsing websites was often accompanied by searching. In the search process, participants often reviewed the results to determine their usefulness. As for the writing process, participants would search for information if they encountered some problems. These behaviors also directly reflected the acquisition of new knowledge in the search process.

We chose 30 minutes as a time unit for counting the frequency of the five types of academic behavior mentioned in each time unit, as shown in Figure 2. The results showed that discussion appeared most often in the early stage of the experiment, while searching and browsing appeared most often in the middle stage. In the final stage, participants focused mainly on sorting out relevant literature and on writing. In other words, participants preferred to determine the direction first, and then to search for information. Finally, they summarized the existing knowledge and new knowledge to arrive at the final results.

Relations between group members’ queries

Every member of each group faced the same task, so the number of identical or similar queries reflects whether the group’s division of labor is reasonable and whether communication is timely to a certain degree. This could reflect how they searched for information in the learning process.

In line with the existing research, we can divide information search into two types – lookup search and exploratory search – according to whether the search target and need is clear. The target of lookup search is generally known, which means that the core search mechanism is to match the query and the document. In contrast, exploratory search is user oriented. There is no unique answer, and this reflects the correlation between information searching and learning.

We carried out a statistical analysis of query construction to understand the participants’ search behavior, using data from the Coagmento log. The total number of queries for the five groups was 57, 19, 29, 22, and 10, respectively. Apart from all members of Group 1 (23, 23, 11) and a member of Group 3 (19), all participants submitted nine queries or fewer. Besides, there existed a great difference in their distribution between Groups 1 and 3 and others, which reflected that they have weak impressions about their task.

Despite the small number of queries, there were some relationships between these queries. In order to reveal these relationships, we selected the identical queries and the similar queries submitted by the members in each group for statistical analysis; these were divided into three categories: identical queries, semantically identical queries, and related queries (Table II).

From the perspective of the search query, semantically identical queries included not only queries with different words, but also those with different languages, such as “film search” and another semantically identical queries in Chinese. Related queries were mainly those using different search strategies to expand or narrow the search range, e.g., “foreign creative space” and “American creative space.” Table II shows that there are more identical queries in Groups 1 and 4, which implies that members of these groups lacked communication about the search topic. If they have communicated fully, they would not submit the same queries, which means wasting time in the collaborative process. In contrast, the members of Group 5 would share important literature or content with one another. In addition, lookup search was mainly used to find the specific documents and webpages or the library’s home pages, e.g., Wiki and Zhihu (a knowledge forum). The number of exploratory searches was the highest, accounting for more than 50 percent.

Figure 3 shows the relationship between search queries and the topics of the tasks. Owing to the small number of queries from Group 5, and no repeats, we did not analyze their queries. The red oval boxes in the figure indicate the topics of participants’ proposals, while the node outside of the red oval boxes shows the queries submitted by these participants. Many queries were submitted sequentially, forming a tree-like structure. This reflects the fact that the groups’ knowledge building is a process of increasing the knowledge nodes, acquiring more concepts and relationships around the core topics.

We can understand that members of a group would reformulate the queries according to the task progress in order to obtain more relevant information. In addition, there exist connections between the queries around the search topics, involving various aspects of the task.

Use of learning resources

Use of the various resources searched was closely related to the success of search tasks and the learning results. Searching for academic information could involve more types of resource than searching for general information. We investigated the participants’ habits in relation to using learning resources through a structured diary in the first stage (see Appendix 1.2). The structured diary used a five-point Likert scale, using numbers from 1 (low) to 5 (high) that show the learning resources usage frequency. The results are shown in Table III.

From Table III, we observe differences between the groups’ usage habits in relation to learning resources. Overall, search engines, document-sharing platforms, and Q&A community are commonly used in each group, with these having the higher mean scores. Professional website and English-language database utilization rates are lower, with the mean value only around 2. This indicates that all groups are more inclined to use learning resources from the internet than professional academic resources provided by the library. However, there is significant difference between the groups in the usage of Chinese-language databases (Option 1, p=0.008). The difference between Group 5 and other groups was most significant, especially with Group 1. Members of Group 5 are all freshmen, and have not yet mastered the use of professional academic resources. Meanwhile, the members of Group 1 are juniors, and their understanding and use of these resources are better after three years of study and practice. In addition, Group 1’s mean scores on other learning resources except for Option 3 are higher, reflecting a stronger sense of information literacy.

We also investigated the participants’ assessments of the importance of different learning resources though structured diaries in the first stage (see Appendix 1.3). The results are shown in Table IV; a score of 1 means least important and 7 means most important.

All groups considered the use of academic search engines (Option 4), professional tools and websites (Option 3), Chinese-language database (Option 1), and search engine (Option 5) to be very important. From previous studies (Rowlands et al., 2008), 89 percent of college students start information retrieval from a search engine, while only 2 percent start from academic resources in the library. However, the information from the general network lacks the specialization and authority of the content for academic research. Thus, the participants in this study also considered that professional academic learning resources are more important.

In the second stage, we used Coagmento’s web statistics function to analyze the use of various types of resources on the web, as shown in Table V. The results showed that members used the search engines most, but there were few differences between search engines and library and database. The most popular academic resources were Chinese-language databases, such as CNKI and Wanfang Data. According to the screen records, most participants were still accustomed to visiting the database from the home page of the digital library.

Google Scholar was the second most popular search engine in the experiment. This indicated that members like to use specialized academic resources on the web while searching for academic information. This is because users can easily find more information on research topics through academic resources in order to understand those topics. In contrast, the contents of general search engine result pages were broader and less professional. Academic resources are characterized by authority and reliability, and can promote the development of research work effectively.

Although the rapid development of search engines has changed methods of finding information, users are increasingly dependent on search engines such as Google and Bing. However, these still cannot replace the position of the library, as the library has a large number and various types of well-integrated academic resources. Therefore, academic resources still play an important and irreplaceable role in research work. It is important to strengthen information resources and improve information services in order to meet the growing needs of users for academic information.

Labor division

Division of labor is an important issue in collaborative search behavior. The appropriate division of labor can have a significant effect on the completion of the task and help group member learn more effectively.

Depending on personal influence and ability, each member had a particular role in each group. Analysis of chat histories in three-hour experiment by ATLAS.ti 7 (see Table VI) indicates that there was a leading role in each group during the division of labor. The group leader was generally the member who understood the research topic deeply, or who understood it best within the group.

For example, from the chat history of QQ, we found that in most groups, the different aspects of the division of labor were listed by one member, who assigned particular tasks to other members. Accordingly, when the division of labor was mentioned in the interview (see Appendix 3.1), a member of Group 1 said: “We would choose one group member to understand the work from a macro point of view, and then decided the various aspects of the division of labor.”

In summary, when determining the research topics and coordinating the division of labor, there was always a group leader controlling the overall situation. There were several reasons for this. First, the academic work tasks were usually complex, with various dimensions, such as research topics and frameworks. Thus, it is necessary to have a member to sort out the ideas from all the group members. It could also help the group to find the direction of the task faster. Second, a number of individuals were involved in the academic task, and this would generate various opinions, ideas, and problems. A coordinator or leader was needed to conduct the division of labor and maintain the stability of the group. In addition, the task of writing a proposal for research projects generally takes a long time, from a few months to several years. In this process, some adjustments could occur in the different stages, and this also needs a leader to grasp the macro research process.

We also transcribed and analyzed the records of the interviews that followed the three-hour experiment (see Appendix 3.1) using ATLAS.ti 7 for qualitative study. The results are shown in Table VII.

The content of the research project application was very complicated and involved different aspects, so the division of labor (Group 1) by the contents is a reasonable approach, and one that could make up for any gaps in individual capabilities. Because the research project proposal had to follow a particular format, some groups divided labor according to this structure. Group 3 divided the labor entirely by the structure of the proposal. Group 2 began by dividing the labor by the structure of the proposal, and then divided the labor by subject subdivision and followed the labor division instructions that were written before the experiment. As a result, their experiment process went relatively smoothly. Initially, Group 4 also divided the labor by the structure of the proposal. However, a lack of preparation resulted in a chaotic division of labor, and this affected the progress of the experiment. After repeated discussions among members, Group 4 changed their approach and divided the labor by contents. Group 5 took a different approach to the division of labor. They initially searched together without a specific division of labor and shared the information among the group members. As the task progressed, members gradually formed their own focus according to the different direction of searching. According to the chat history from QQ, there were two stages to Group 5’s labor division, first by roles, and then by contents.

Learning results in the collaborative search process

In this section, we first used the final results of competition to evaluate each group’s learning results under the academic task. The sentiment changes, cognition on communication and information sharing, as well as the learning gains, were used to evaluate the members’ learning in collaboration.

Results of academic task accomplishment

Our work was based on an actual academic task relating to an application for research projects. Because it was difficult to finish and improve the proposals during the experiment, we did not set an evaluation index to evaluate the effects of participants’ collaboration. The competition has a committee to evaluate whether or not a group wins, based on their proposals. All five groups in this study handed over their proposals to the school, for consideration alongside many other proposals. The committee in the school, made up of five experts from different majors, evaluated whether or not a group would gain approval. The top 10 proposals would then be recommended to a university-level committee to decide whether or not a research project could be awarded national funding, as well as whether or not a review was necessary. The winner could gain national funding, or be awarded university-level funding.

This procedure guarantees the objectivity of the evaluation, so we also chose the final application results as the evaluation of each group’s work. In Table VIII, the numbers 1 (high) to 5 (low) indicate the evaluation rating of each research project. There were significant differences in the final application results.

Sentiment changes

As the work tasks lasted for a long period, the sentiments of group members could have changed. These changes could reflect the group members’ learning effects. Sentiment factors have an effect on the users’ search behavior and the final effects (Bilal, 2000; Wang et al., 2000; Chu et al., 2017), while there were fewer discussions focusing on the academic context.

The participants were asked to record their sentiments at the beginning of the work task. We divided sentiments into seven types (Rodden, 1991), namely three positive feelings (clarity, optimism, and relief), one neutral feeling (uncertainty), and three negative feelings (doubt, disappointment, and intension). Participants were asked to mark with a number from 1 (low) to 5 (high) to show the degree of these feelings.

We first investigated the sentiment states of three groups in the first stage of study using a structured diary (see Appendix 1.4), and the results are shown in Table IX. The questionnaire results for Groups 1 and 2 were invalid for the obvious contradictions in their structured diaries, so these are not shown.

Groups 3 and 4 experienced more positive feelings in the first stage, especially clarity and optimism, while the mean values for neutral and negative feelings are low. For Group 5, all mean values are relatively low, without significant differences, indicating that members of this group showed sentiment stability in the preparation and exploratory stage.

After the writing process in the three-hour experiment, we studied the sentiment states of the five groups (Appendix 2.1). All participants were asked to choose a number to truly represent their feelings, with 1 meaning totally disagree and 5 totally agree. The results for all the groups are shown in Table X.

This shows that at this stage, none of the groups had high levels for any of the types of feelings. However, the mean values for the positive feelings were higher than those for the negative feelings, and the participants tended to be optimistic after the experiment. In the follow-up interviews (see Appendix 3.2), four groups expressed positive feelings toward the task, with comments such as “OK,” “pretty good,” and “had fun during the process.” Only Group 4 thought their process “moved slowly and was a mess” and gave a negative summary of their work.

However, the groups did exhibit differences in sentiment cognition. Feelings such as uncertainty and disappointment showed highly significant differences, and feelings such as doubt showed significant differences. The values for positive feelings of Groups 3 and 5 were much higher than for their negative feelings. In contrast, the average values of all feelings showed little differences between Groups 1 and 2. This indicates that communication, preparation, and exploration in the early stage may have an impact on collaborative work and sentiment.

Communication and information sharing

Communication is an important aspect of the collaborative searching as learning process, which is also different from individual learning. The collision of ideas during communication provided the opportunity for members to learn from each other. There were five main methods of communication: e-mail, IM, call or SMS, video online, and face-to-face. We used a structured diary to investigate the importance of these five methods of communication in the first stage (see Appendix 1.5) using a five-point Likert scale ranging from “most important” (5) to “least important” (1). The results are shown in Table XI. IM (e.g. QQ) and face-to-face communication are both popular methods, as well as the most important methods, while video online is the least important method.

We also included seven options in the questionnaire to study the effect and influence of communication and information sharing in the process of collaborative search (see Appendix 2.2). The analytical results of the questionnaire are shown in Table XII.

As shown in Table XII, the mean scores for communication and information sharing for each group were higher than 4.00, and there were significant differences with regard to Option 1 (p=0.023 <0.05) and Option 3 (p=0.047 <0.05). Some group members communicated with each other often (Option 1), such as those in Groups 4 and 5, while others did so slightly less often, such as those in Groups 1 and 3.

The QQ chat history of Group 1 consisted mainly of discussion about the division of labor and the confirmation of research topics, with no communication about respective search results. The members focused on their own subtasks. As a result of Group 3’s thorough preparation before the experiment, its members’ focus was browsing information and writing following the division of labor.

The highest mean score among all the groups was for Option 3 (4.55), indicating that participants were willing to share information with others. The next highest mean scores were for Option 5 (4.40), Option 6 (4.45), and Option 7 (4.35), suggesting that all groups had a positive approach to sharing information in the collaborative search process. In particular, the mean score for Option 6 (4.45) indicated the positive improvement on knowledge from the information sharing. Group 5 had the highest mean scores for each individual option. From the system log analysis, we found that members of this group always shared useful information via QQ during the experiment.

Through the above analysis, we suggest that the frequency of communication could promote stronger awareness of information sharing. Stronger awareness of information sharing leads to higher evaluation of sharing. Thus, participants would be more willing to communicate with others and develop the habit of sharing, which is a virtuous circle.

Learning gains

Although the groups’ positive sentiments dominate in the collaborative learning process, some problems led to the emergence of negative sentiments. We studied the problems members faced in the experiment and the outcomes after the experiment to investigate their learning gains. Through the interview (see Appendix 3.3), we investigated these problems and the interview results were coded using ATLAS.ti 7 (see Table XIII).

We divide the problems in the participants’ learning into three categories: lack of research skills, problems in task planning, and problems in communication. As the participants are undergraduates, they could encounter difficulties in reading literature in English and in finding the information in the process of learning during collaborative information searches. In addition, their own research skills are limited, so some ideas about the proposal could not be fulfilled.

With regard to the task planning, the main problem was that the direction was not clear, and there were also problems with the selection of topics and the lack of preparation. The unclear direction included the formation of the pre-structure and the ideas for writing. Problems in communication occurred among group members, as well as between the group members and the teachers.

In the third stage, the activities of all five groups were few and simple. All groups spent most of the time on finishing the proposal, indicating that the final stage is a process for completion, synthesis, and reflection. Thus, we investigated what participants learned in the collaborative information search process, by transcribing and analyzing the interview records (see Appendix 3.4) following the experiment using ATLAS.ti 7, as show in Table XIV.

In the interview, the participants indicated that they gained various skills, such as research and teamwork skills. Members of Group 2 stated: “It was the first time I had conducted a research project. I didn’t know how it would progress and how to prepare it at first. But, through this experience, I gained better knowledge of the topic and was clear about how to conduct future studies.” Thus, participants developed research skills such as understanding the research process, data-collection methodologies, and writing. Participants also developed a deeper understanding of some research areas. Research skills were cultivated during the collaborative search, as were research interests. Nevertheless, in the process of collaboration, participants found it was important to find suitable group members. Members of Group 2 state: “It’s the most interesting part of the research to find like-minded friends and work together through thick and thin.” For members of Group 3, “during this collaboration, we formed collaborative work groups and this collaboration continued in daily studies.” Teamwork skills, including communication skills, were also enhanced. Members of Group 1 specifically mentioned that they gained more skills in information seeking, and that their information literacy was improved.

The learning gains in the collaborative search process were mainly research and teamwork skills. Compared with Table XIII, participants gain different skills after the task, and these skills could help them to address the difficulties they have had. For the academic task, many of the participants involved in the application for the research project gradually began to understand the process and learned the skills of data collection and writing. They also improved their research skills and, in the process, became interested in academic research. In particular, they recognized the importance of collaboration with suitable colleagues, something that could also strength their friendships.

Evaluation of and suggestions for the collaborative search system

Evaluation of the collaborative search system

Search systems play an important role in searching as learning (Chi et al., 2016). We therefore analyzed the performance of the collaborative search system to study its impact on learning. Coagmento has bookmark, recommend, annotate, snip, and editor functions to support collaborative work. However, in our study, only the bookmark, snip and editor functions were used by the five groups. There were 15 bookmarks and three snippets, contributed by seven members and two members, respectively.

We collected groups’ evaluation of the Coagmento functions using a five-point Likert scale, with 1 indicating “not at all useful” and 5 indicating “very useful” (see Appendix 2.3). The functions used during the experiment were highly regarded by participants, especially the editor function (see Table XV). Participants thought that the collaborative search system was helpful for collaborative search tasks.

However, the feedback in the interviews (see in Appendix 3.5) revealed a low rate of utilization of Coagmento: only Group 5 continued to adopt it in their collaboration to complete the proposal after the three-hour experiment. This situation was partly caused by users’ unfamiliarity with the collaborative tool and by their search habits. Nearly all groups mentioned in the interview that the collaborative search system did not run smoothly during the experiment, and this led them to abandon it in their later study. They further identified that IM software (QQ) was used as the replacement. Group 4 even stated: “The collaborative search system had nothing special for collaboration compared to the instant message software, and it could be totally replaced by QQ.”

Suggestions for the collaborative search system

In the interviews following the experiment (see in Appendix 3.5), we inquired about the problems encountered during the experiment and asked for suggestions for a collaborative search system, using ATLAS.ti 7 to code the records of the interviews (see Table XVI). We did not conduct association analysis owing to the dispersion of problems and suggestions from different groups.

It can be seen from these results that the main problem is that the collaborative search system often crashed and dropped the line, which could delay the process of groups tasks and affect the efficiency of learning. There is a very noteworthy phenomenon, namely the relationship between the collaborative search system and IM software. Of the participants, 12 stated that, “when I used instant messaging software, I forget to use the collaborative search systems” (or words to that effect). This relates to participants’ daily behavior. IM software is the most common communication tool used by users, and participants are more familiar with it than they are with the collaborative search system. In addition, the collaborative search system lacks support for cross-language usage. More importantly, IM software and the collaborative search system have some duplication in terms of their functions, e.g., sharing links or URLs. In this study, participants could use QQ to send links or screenshots and share documents in the discussion group. Hence, they thought there was nothing special about Coagmento, and that this system could be replaced by QQ.

The participants also gave some suggestions for improving the collaborative search system in academic group work. Group 1 preferred functions that could share information more efficiently about others’ work: “Shared information should be highlighted immediately. However, [in the current system] you need to click first if you want to check what others have recommended. It’s not convenient.” They added: “The system should record and highlight the important links that all members have clicked.” Group 5 said that they hoped the collaborative search system could provide some information related to their project.

Discussion

Learning behavior in the process of collaborative search

Prior studies have shown that users would continue to increase their level of knowledge in the collaborative information search process (Chi et al., 2016). Our research also confirmed that collaborative search had positive effects on learning. Collaboration in writing research proposals can be regarded as a way to stabilize knowledge structures. Analysis of our three-month results on search behaviors and sentiments shows that the participants’ stabilization of knowledge structures is similar to that described in the work of Vakkari (2016), including the three stages: restructuring, tuning, and assimilation. Individuals first search a wide range of information to clarify their information needs and to make sense of the research topic together, so the exploratory search occurred most frequently (Table II). New knowledge is acquired during searching.

Most participants’ sentiments were positive at the beginning of the task, although they did not feel relaxed (Table IX), as the work task seemed heavy. Based on a personal initial understanding of the topic, they shared the information they found and restructured their queries. This can be regarded as a way to restructure existing knowledge, and personal knowledge structures can also be influenced. Users often discuss and communicate with each other to reach agreement on a general concept of the topic together. The scope of the related concepts will be clearer in this tuning process. Preparation and exploration in the early stage will contribute to subsequent collaboration and will make it more likely that participants’ sentiments are positive. Finally, when writing a research proposal, participants integrated all the knowledge they have obtained. The final proposal was, in fact, produced by reflecting and assimilating the knowledge of all group members.

In summary, learning through collaborative information searching is a process of restructuring knowledge, including obtaining new knowledge and supplementing existing knowledge structures. As the task progresses, knowledge structures evolve through restructuring, tuning, and assimilation. Sentiment also changes from vague to clear, and shows a generally positive state (Table X).

The collaborative information search process could also enrich the gains of users. All the participants in this study had gained more skills by the end of the task (Table XIV). Our results indicate that communication is important in the collaborative search process, in contrast to an individual search (Table XI). Despite some communication problems and obstacles in the completion of work tasks, teamwork abilities will be strengthened and improved through communication (Table XII). Communication and teamwork abilities are vital benefits of collaboration. The experiences of collaborative search also bring other benefits for learning, especially research skills. Students can learn how to conduct research and, through collaboration, can compensate for any inadequacies in their individual abilities. Collaborative search asks students to explore and think for themselves, and will be an important supplement to traditional classroom education. Participants could directly develop more skills through collaborative information searching, which is part of information literacy (Table XIV). Such searching helps users to develop more relevant skills and achieve self-improvement. In our research, we studied learning in a collaborative search process within a real academic setting, and found that collaboration is essential in writing a research proposal. This does not simply mean the addition of personal search results only. It offers a great opportunity to participate actively, and is something from which everyone can benefit. This is not only because complex tasks can be completed more easily and effectively when collaborating. It is also because collaboration provides a chance to learn from each other and progress together.

Ways to support learning through collaborative search

As previously mentioned, there were differences between the results of the applications. We summarized the ways to improve the learning results, from the perspective of learning and collaboration to enhance the positive role of collaborative search in learning.

It is important to choose professional and appropriate resources to lay a solid foundation for learning. Our participants preferred the Chinese-language database (Table V), ensuring the authority and reliability of resources. Academic search engines such as Google Scholar also played an important role in enhancing the richness and ensuring the greater integration of resources. With regard to understanding simple fact-based information, public search engines such as Google might be more convenient. As well, exploration is central in learning and understanding (White, 2016), so the exploratory searches are essential in collaborative learning.

Collaboration effects also have significant impacts on the learning process and the results. At the beginning of the tasks, a reasonable division of labor was necessary. An effective division of labor at this stage could make up for any lack of personal knowledge through the progress of the whole group. Each group would choose a leader to determine the division of labor based on personal knowledge and skills (Table VI). A clear division of labor helps members not only to understand the information better, but also to avoid confusion and improve the efficiency of their work. In addition, positive communication and sharing are essential for saving time and supporting groups’ control of the progress of the work.

Implications

Through our study, collaborative search systems can provide positive support for learning in the collaborative search process, and collaboration is important for information retrieval system (Twidale et al., 1997).

During this experiment, the collaborative search system had a high acceptance rate but a low level of usage. According to the participants’ suggestions, the collaborative search system should provide instant communication to support the need for group discussion (Table XVI). The social aspects of information seeking should get more attention (Nichols and Twidale, 2011), and the importance of communication and information sharing has been shown, so communicating and sharing functions are necessary. This can promote knowledge tuning and assimilation. The system should also pay more attention to data mining and providing academic information. In order to improve learning efficiency, it is important for a collaborative search system to help group members grasp the task’s progress. Furthermore, concept maps should be applied to support group learning. Each group member can mark out the relevant knowledge node and its relation to other nodes on a common screen, forming a knowledge network diagram. Such a concept map can also make the knowledge structure clearer, and the different parts of the map can also provide a basis for the division of labor.

This study can also help the library to expand the knowledge of information literacy education. Information literacy is an important quality in the information age, and it is also an important aspect of the library carrying out educational work. In general, course of literature retrieval is main method of information literacy education in library, which emphasizes the skills of seeking information seeking and ability of evaluating information. The information-seeking process improves not only these two abilities, but also the comprehensive ability to solve the problem, which is accumulation of research ability and method. Therefore, the information literacy education of the library can be more diversified and pay more attention to the cultivation of comprehensive abilities.

Conclusions

This research explored undergraduates’ learning through a collaborative information search process for an academic group task. We conducted a three-month study using several methods to analyze participants’ learning in writing an actual research proposal. A three-hour experiment on a collaborative search system was designed to identify its role in supporting learning. In analyzing the data, we found that collaborative search could promote learning experiences and have a positive effect on collaborative learning. New knowledge and skills, such as communication and research skills, were developed through the collaborative search session process and updating of existing knowledge structures. In order to achieve a better learning effect, all the groups chose more professional academic information sources and also had effective collaboration as a support. We propose that collaborative search systems should be required to pay more attention to the characteristics and information needs of education, in order to provide an environment in which individuals can experience more effective learning.

The number of groups and participants in our research is relatively small and all participants are from one school. And there is a lack of variations in team sizes and specific nature of the task. Furthermore, besides the committee’s evaluation, we did not take proper instruments to measure learning. Thus, there is a need for future work to investigate more users in other knowledge domains using a larger sample. Learning metrics also need to be added in the future. In addition, the different factors in group learning results, such as learning experience, personal abilities, etc., should be addressed.

Figures

Time duration of different types of behavior in the experiment

Figure 1

Time duration of different types of behavior in the experiment

Time duration of five types of academic behavior in the experiment

Figure 2

Time duration of five types of academic behavior in the experiment

The relationship between search queries

Figure 3

The relationship between search queries

Background information of participants

Group no. Members Discipline Grade
1 3 Information management and information system Junior year
2 4 Library science Sophomore year
3 4 Information management and information system Sophomore year
4 4 Library science Sophomore year
5 5 Information management and information system Freshman year

Number and type of search queries

Group no. Total number Identical Semantically identical Related Lookup search Exploratory search
1 57 2 3 6 4 (7.02%) 53 (92.98%)
2 19 0 2 2 8 (42.11%) 11 (57.89%)
3 29 1 0 4 3 (10.34%) 26 (89.66%)
4 22 3 2 3 7 (31.82%) 15 (68.18%)
5 10 1 1 0 6 (60%) 4 (40%)

Usage habits in relation to learning resources at the beginning of the task

Mean score of each group
Option no. Group 1 Group 2 Group 3 Group 4 Group 5 Mean score SD SE ANOVA value of p
1 4.33 3.50 3.75 4.00 2.00 3.40 1.142 0.255 0.008**
2 3.67 2.25 1.75 2.00 2.00 2.25 1.070 0.239 0.145
3 1.33 2.25 1.50 2.00 2.40 1.95 1.146 0.256 0.683
4 4.33 3.00 3.75 2.75 2.80 3.25 1.070 0.239 0.202
5 5.00 5.00 5.00 5.00 4.20 4.80 0.696 0.156 0.306
6 4.67 4.50 5.00 4.75 4.40 4.65 0.587 0.131 0.653
7 4.33 4.50 4.75 4.75 3.80 4.40 0.995 0.222 0.626

Notes: Option 1: Chinese-language database; Option 2: English-language database; Option 3: Professional tools and websites; Option 4: Academic search engine; Option 5: Search engine, Option 6: Document-sharing platform; Option 7: Q&A community. **p⩽0.01

Participants’ assessments on the importance of different learning resources

Mean score of each group
Option no. Group 1 Group 2 Group 3 Group 4 Group 5 Mean score SD SE ANOVA value of p
1 4.67 4.25 2.50 5.00 4.40 4.15 2.059 0.460 0.514
2 4.67 4.25 3.00 2.50 2.60 3.30 1.593 0.356 0.219
3 5.00 5.75 3.50 2.75 4.20 4.20 1.704 0.381 0.090
4 3.67 4.75 5.00 4.25 4.60 4.50 1.906 0.426 0.930
5 3.33 3.00 5.50 5.75 3.20 4.15 2.368 0.530 0.274
6 3.67 3.25 3.75 4.75 4.60 4.05 1.791 0.400 0.755
7 3.00 2.75 4.75 3.00 4.40 3.65 2.519 0.563 0.745

Usage of different types of learning resources in the experiment

Web type Total
Search engine
Google 90 203
Baidu 44
Google Scholar 69
Library and database
Chinese-language database 151 189
English-language database 7
Library 31
PDF
Database 23 32
Webpage 9
Web
General website 127 176
Academic professional webpages 9
Document community 40

Coding results on the group roles from chat history

Group roles Vocabulary Word freq.
Leading roles Division of searching 2
Guidance and help in subtasks 2
Statement of labor division 1
Confirm the division structure 2
Remind others 1
Responsible for macro tasks 1
Other roles Recommend others to divide labor 1
Accept the subtasks 3

Coding results on the mode of labor division

Labor division mode Vocabulary Word freq. Correlation coefficient
By contents Division by contents 3 0.75
Division by structure of proposal 2 0.50
Subject subdivision 1 0.25
First macro, then refine 1 0.25
Division by reference 1 0.25
By roles No specific labor division 1 1.00
Work synchronously 1 1.00
Form the key aspects automatically 2 2.00
Search together 1 1.00

Final application results and rating of their applications

Group no. Final application results Ratings of tasks
1 Failed application 5
2 Successful application for university-level funding 3
3 Successful application for national funding 1
4 Successful application for university-level funding 3
5 Participated in the review of the university-level committee, while receiving only university-level funding 2

Sentiment states of three groups at the beginning of the work task

Feelings (mean)
Group no. Clarity Optimism Relief Uncertainty Doubt Disappointment Anxiety
3 4.25 4.56 3.64 1.86 1.75 1.39 1.64
4 4.00 4.25 2.75 2.00 1.50 1.25 1.75
5 2.50 3.25 3.00 2.25 3.00 2.75 3.00

Sentiment states for the five groups after the three-hour experiment

Mean of each group
Feelings Group 1 Group 2 Group 3 Group 4 Group 5 Mean SD ANOVA value of p
Clarity 3.00 3.50 3.50 2.75 3.60 3.30 0.923 0.665
Optimism 3.00 3.25 4.25 3.00 3.80 3.50 1.051 0.401
Relief 2.67 3.25 4.00 2.25 3.40 3.15 1.089 0.191
Uncertainty 4.00 3.50 2.50 3.00 1.80 2.85 0.988 0.003**
Doubt 3.33 2.75 1.75 2.75 2.00 2.45 0.826 0.041*
Disappointment 2.67 3.50 1.50 1.75 1.80 2.20 1.005 0.009**
Anxiety 2.00 3.00 1.25 1.75 1.60 1.90 0.968 0.088

Notes: *0.01<p<0.05; **p⩽0.01

Importance of different communication methods at the beginning of the task

Mean score of each group
Methods Group 1 Group 2 Group 3 Group 4 Group 5 Mean score SD SE ANOVA Value of p
E-mail 3.33 3.00 3.00 2.75 2.80 2.95 1.371 0.294 0.986
IM 3.33 3.00 3.50 4.75 4.00 3.75 1.020 0.228 0.117
Call/message 2.67 3.00 2.75 3.00 2.00 2.65 1.040 0.233 0.620
Video online 1.33 2.00 2.00 1.00 2.40 1.80 1.473 0.329 0.697
Face-to-face 4.33 4.00 3.75 3.50 3.80 3.85 1.226 0.274 0.943

Cognition of communication and information sharing

Mean score of each group
Option no. Group 1 Group 2 Group 3 Group 4 Group 5 Mean score SD SE ANOVA Value of p
1 3.33 4.50 3.75 4.75 4.80 4.30 0.801 0.179 0.023*
2 3.67 4.00 4.00 3.75 4.40 4.00 0.649 0.145 0.553
3 4.00 4.75 4.25 5.00 4.60 4.55 0.510 0.114 0.047*
4 3.67 4.50 3.50 4.75 4.60 4.25 0.786 0.176 0.058
5 3.67 4.50 4.25 4.50 4.80 4.40 0.598 0.134 0.107
6 4.00 4.50 4.25 4.50 4.80 4.45 0.510 0.114 0.258
7 4.00 4.50 4.00 4.25 4.80 4.35 0.671 0.150 0.373

Note: *0.01<p<0.05

Coding results on problems during the whole task

Problems Vocabulary Word freq. Correlation coefficient
Lack of research skills Lack of innovation ability 1 0.25
Difficulties in reading English 1 0.25
Difficulties in finding literature 4 1.00
Failure to achieve expected results 1 0.25
Problems in task planning Unclear direction 3 0.60
Selection of topics 1 0.20
Lack of preparation 1 0.20
Problems in communication Inconsistent opinions 1 0.50
Communication with teachers 1 0.50

Coding results on gains following the work task

Gains Vocabulary Word freq. Correlation coefficient
Research skills Understand the research area more 2 0.17
Understand the research process 3 0.50
Read English studies 1 0.17
Data collection and writing 1 0.17
Teamwork skills Find a suitable colleague 6 0.86
Focus on communication 1 0.14
Information literacy Improve information literacy 2 1.00
Others Topic is important 1 1.00
Instructor is important 1 1.00

Evaluation of Coagmento’s functions

Function Mean SD SE ANOVA value of p
Bookmark 3.80 1.322 0.296 0.030*
Recommend 3.35 1.089 0.244 0.548
Annotate 2.85 1.182 0.264 0.522
Snip 3.45 1.276 0.285 0.365
Editor 4.50 0.607 0.136 0.321
Chat 3.50 1.638 0.366 0.130
Notification 2.85 1.309 0.293 0.981
History 3.55 1.432 0.320 0.089
Notepad 3.40 1.273 0.285 0.944

Note: *0.01<p<0.05

Coding results about problems and suggestions on collaborative search system

Vocabulary Word freq.
Problems
System crashes, drop line 5
QQ can be replaced 4
Prevent the use of other software 2
Lack of reminder function 1
Some functions are repeated 3
Low rate of function utilization 2
Suggestions
Add a backup function 1
Mark the progress of personal tasks 1
Add a reminder function 2
Support copying and insertion of pictures 1
Add a voice function 1
Automatically provide information 1
Parent child window 1
Add a help function 1

Appendix 1. Structured diaries at the beginning of the work task (first stage)

Appendix 2. Questionnaires on the three-hour experiment (second stage)

Appendix 3. Interviews after the three-hour experiment (third stage)

3.1 Labor division mode

In the experiment process, what method of labor division did you choose?

3.2 Participants’ sentiment feelings after the three-hour experiment

How do you feel after this academic group work?

3.3 Participants’ problems during the whole task

What are the main problems that you faced during the whole task?

3.4 Participants’ gains after the whole task

What have you gained from this group academic task?

3.5 Participants’ evaluation of and suggestions for the collaborative search system

What are the main problems you faced when you used Coagmento?

Do you have any suggestions for a collaborative search system?

References

Aalst, J.V. (2009), “Distinguishing knowledge-sharing, knowledge-construction, and knowledge-creation discourses”, International Journal of Computer-Supported Collaborative Learning, Vol. 4 No. 3, pp. 259-287.

Bah, A. and Carterette, B. (2014), “Using ‘model’ Pseudo-Documents to improve searching-as-learning and search over sessions”, Searching as Learning Workshop IIiX 2014, Regensburg, August, available at: www.diigubc.ca/IIIXSAL/Papers/RabiouCarterette.pdf

Bilal, D. (2000), “Children’s use of the Yahooligans! Web search engine: I. cognitive, physical, and affective behaviors on fact-based search tasks”, Journal of the American Society for Information Science & Technology, Vol. 51 No. 7, pp. 646-665.

Blake, C. and Pratt, W. (2006), “Collaborative information synthesis II: recommendations for information systems to support synthesis activities”, Journal of the American Society for Information Science and Technology, Vol. 57 No. 14, pp. 1888-1895.

Boyatzis, R.E. (1998), Transforming Qualitative Information: Thematic Analysis and Code Development, Sage, Thousand Oaks, CA.

Chi, Y., Han, S., He, D. and Meng, R. (2016), “Exploring knowledge learning in collaborative information seeking process”, SIGIR 2016 Workshop: Searching As Learning, Pisa.

Chu, S., Sonnenwald, D.H., Wu, M. and Chang, S.J.L. (2017), “Theory Development within the information sciences”, Bulletin of the ASIS&T, Washington, DC, available at: www.asist.org/files/bulletin/feb-17/Chu_EtAl_2016Meeting.pdf

Dillenbourg, P. (1999), “What do you mean by ‘collaborative learning’?”, in Dillenbourg, P. (Ed.), Collaborative-Learning: Cognitive and Computational Approaches, Elsevier, Oxford, pp. 1-19.

Egusa, Y., Takaku, M. and Saito, H. (2014), “How to evaluate searching as learning”, Searching as Learning Workshop IIiX 2014, Regensburg, August, available at: www.diigubc.ca/IIIXSAL/Papers/Egusaetal.pdf

Foster, J. (2006), “Collaborative information seeking and retrieval”, Annual Review of Information Science and Technology, Vol. 40 No. 1, pp. 329-356.

Freund, L., Kopak, R. and O’Brien, H. (2016), “The effects of textual environment on reading comprehension: implications for searching as learning”, Journal of Information Science, Vol. 42 No. 1, pp. 79-93.

Hansen, P. and Rieh, S.Y. (2016), “Editorial: recent advances on searching as learning: an introduction to the special issue”, Journal of Information Science, Vol. 42 No. 1, pp. 3-6.

Hwang, G.J., Tsai, P.S., Tsai, C.C. and Tseng, J.C.R. (2008), “A novel approach for assisting teachers in analyzing student web-searching behaviors”, Computers & Education, Vol. 51 No. 2, pp. 926-938.

Hyldegård, J. (2006), “Collaborative information behaviour – exploring Kuhlthau’s information search process model in a group-based educational setting”, Information Processing & Management, Vol. 42 No. 1, pp. 276-298.

Hyldegård, J. (2009), “Beyond the search process – exploring group members’ information behavior in context”, Information Processing & Management, Vol. 45 No. 1, pp. 142-158.

Jansen, B.J., Booth, D. and Smith, B. (2009), “Using the taxonomy of cognitive learning to model online searching”, Information Processing & Management, Vol. 45 No. 6, pp. 643-663.

Karanam, S., van Oostendorp, H. and Fu, W.T. (2016), “Performance of computational cognitive models of web-navigation on real websites”, Journal of Information Science, Vol. 42 No. 1, pp. 94-113.

Kim, J. and Lee, J. (2014), “Knowledge construction and information seeking in collaborative learning”, Canadian Journal of Information and Library Science, Vol. 38 No. 1, pp. 1-21.

Leeder, C. and Shah, C. (2016), “Collaborative information seeking in student group projects”, Aslib Journal of Information Management, Vol. 68 No. 5, pp. 526-544.

Morris, M.R. and Teevan, J. (2010), “Collaborative web search: Who, what, where, when, and why”, in Marchionini, G. (Ed.), Synthesis Lectures on Information Concepts, Retrieval, and Services, Morgan & Claypool Publishers, San Rafael, pp. 1-85.

Nichols, D.M. and Twidale, M.B. (2011), “Recommendation, collaboration and social search”, in Ruthven, I. and Kelly, D. (Eds), Interactive Information Retrieval, Facet Publishing, London, pp. 205-220, available at: https://researchcommons.waikato.ac.nz/bitstream/handle/10289/5659/recommendation-collaboration-social-search.pdf?sequence=1&isAllowed=y

Oostendorp, H.V. and Aggarwal, S. (2014), “Improving information seeking performance”, Searching as Learning Workshop IIiX 2014, Regensburg, August, available at: www.diigubc.ca/IIIXSAL/Papers/OostendorpAggarwal.pdf

Pian, W. and Khoo, C.S.G. (2014), “Health information learning within the context of health information seeking on a discussion forum”, Searching as Learning Workshop IIiX 2014, Regensburg, August, available at: www.diigubc.ca/IIIXSAL/Papers/PianKhooAbst.pdf

Reddy, M.C. and Jansen, B.J. (2008), “A model for understanding collaborative information behavior in context: a study of two healthcare groups”, Information Processing & Management, Vol. 41 No. 1, pp. 256-273.

Renugadevi, S., Geetha, T.V., Gayathiri, R.L., Prathyusha, S. and Kaviya, T. (2014), “Collaborative search using an implicitly formed academic network”, Aslib Journal of Information Management, Vol. 66 No. 5, pp. 537-552.

Reynolds, R.B. (2016), “Relationships among tasks, collaborative inquiry processes, inquiry resolutions, and knowledge outcomes in adolescents during guided discovery-based game design in school”, Journal of Information Science, Vol. 42 No. 1, pp. 35-58.

Rodden, T. (1991), “A survey of CSCW systems”, Interacting with Computers, Vol. 3 No. 3, pp. 319-353.

Rowlands, I., Nicholas, D., Williams, P., Huntington, P., Fieldhouse, M., Gunter, B., Withey, R., Jamali, H.R., Dobrowolski, T. and Tenopir, C. (2008), “The Google generation: the information behaviour of the researcher of the future”, Aslib Proceedings, Vol. 60 No. 4, pp. 290-310.

Saito, H. and Miwa, K. (2007), “Construction of a learning environment supporting learners’ reflection: a case of information seeking on the Web”, Computers & Education, Vol. 49 No. 2, pp. 214-229.

Schacter, D.L., Gilbert, D.T. and Wegner, D.M. (2009, 2011), Psychology, 2nd ed., Worth Publishers, New York, NY.

Shah, C. (2010), “Coagmento – a collaborative information seeking, synthesis and sense-making framework”, Proceeding of the 2010 ACM Conference on Computer Supported Cooperative Work, pp. 527-527.

Shah, C. (2012), Collaborative Information Seeking: The Art and Science of Making the Whole Greater Than the Sum of All, Springer, Berlin, pp. 3-8.

Shah, C. (2014), “Collaborative information seeking”, Journal of the Association for Information Science and Technology, Vol. 65 No. 2, pp. 215-236.

Sormunen, E., Tanni, M. and Heinström, J. (2013), “Students’ engagement in collaborative knowledge construction in group assignments for information literacy”, Information Research: An International Electronic Journal, Vol. 18 No. 3, pp. 1-16.

Stahl, G., Koschmann, T. and Suthers, D. (2006), “Computer-supported collaborative learning: an historical perspective”, in Sawyer, R.K. (Ed.), Cambridge Handbook of the Learning Sciences, Cambridge University Press, Cambridge, pp. 409-426.

Stange, D., Heyn, H. and Nürnberger, A. (2014), “Capturing information gain in information seeking with concept Maps”, Searching as Learning Workshop IIiX 2014, Regensburg, August, available at: www.diigubc.ca/IIIXSAL/Papers/StangetalAbst.pdf

Twidale, M.B., Nichols, D.M. and Paice, C.D. (1997), “Browsing is a collaborative process”, Information Processing & Management, Vol. 33 No. 6, pp. 761-783.

Vakkari, P. (2001), “A theory of the task-based information retrieval process: a summary and generalisation of a longitudinal study”, Journal of Documentation, Vol. 57 No. 1, pp. 44-60.

Vakkari, P. (2016), “Searching as learning: a systematization based on literature”, Journal of Information Science, Vol. 42 No. 1, pp. 7-18.

Wang, P., Hawk, W.B. and Tenopir, C. (2000), “Users’ interaction with World Wide Web resources: an exploratory study using a holistic approach”, Information Processing & Management, Vol. 36 No. 2, pp. 229-251.

White, R.W. (2016), Interactions With Search Systems, Cambridge University Press, Cambridge, pp. 201-230.

Wildemuth, B.M. (2004), “The effects of domain knowledge on search tactic formulation”, Journal of the American Society for Information Science and Technology, Vol. 55 No. 3, pp. 246-258.

Wildemuth, B.M. (2009), Applications of Social Research Methods to Questions in Information and Library Science, Library Unlimited, Westport, CT, pp. 73-82.

Xu, J.P. and Zhang, H.C. (2005), “Testing intercoder reliability by multi-approaches in qualitative research”, Psychological Science, Vol. 28 No. 6, pp. 1430-1432.

Yin, C., Sung, H.-Y., Hwang, G.-J., Hirokawa, S., Chu, H.-C., Flanagan, B. and Tabata, Y. (2013), “Learning by searching: a learning environment that provides searching and analysis facilities for supporting trend analysis activities”, Educational Technology & Society, Vol. 16 No. 3, pp. 286-300.

Zhang, P. and Soergel, D. (2016), “Process patterns and conceptual changes in knowledge representations during information seeking and sensemaking: a qualitative user study”, Journal of Information Science, Vol. 42 No. 1, pp. 59-78.

Supplementary materials

AJIM_70_1.pdf (10.6 MB)

Acknowledgements

This work is supported by National Natural Science Foundation of China (No. 71673204), and also an outcome of the Wuhan University independent research project (Humanities and Social Sciences) “Human-Computer Interaction and Collaboration Team” (Whu2016020) supported by “The Fundamental Research Funds for the Central Universities.”

Corresponding author

Dan Wu can be contacted at: woodan@whu.edu.cn