Search results

This study aims to investigate quality of feedback as offered by supervisor-assessors with varying levels of assessor expertise following assessment of performance in residency training in a health care setting. It furthermore investigates if and how different levels of assessor expertise influence feedback characteristics.

Experienced (n=18) and non-experienced (n=16) supervisor-assessors with different levels of assessor expertise in general practice (GP) watched two videotapes, each presenting a trainee in a “real-life” patient encounter. After watching each videotape, participants documented performance ratings, wrote down narrative feedback comments and verbalized their feedback. Deductive content analysis of feedback protocols was used to explore quality of feedback. Between-group differences were assessed using qualitative-based quantitative analysis of feedback data.

Overall, specificity and usefulness of both written and verbal feedback was limited. Differences in assessor expertise did not seem to affect feedback quality.

Results of the study are limited to a specific setting (GP) and assessment context. Further study in other settings and larger sample sizes may contribute to better understanding of the relation between assessor characteristics and feedback quality.

Findings suggest that even with supervisor-assessors with varying levels of assessor expertise who are trained in performance assessment and the provision of performance feedback, high-quality feedback is not self-evident; coaching “on the job” of feedback providers and continuous evaluation of feedback processes in performance management systems is crucial. Instruments should facilitate provision of meaningful feedback in writing.

The paper investigates quality of feedback immediately following assessment of performance, and links feedback quality to assessor expertise. Findings can contribute to improvement of performance management systems and assessments for developmental purposes.

With behavioral experiments and protocol analysis, researchers can capture cognition in action. Using behavioral experiments, they can study realized behavior, not perception or self-reports. And they can do that in a controlled laboratory environment to establish causality, curbing spurious relationships. With protocol analysis, a method to elicit decision-makers’ thoughts, researchers can tap into cognitive processes. In combination, the two methods offer a novel approach to grasp mental processes alongside behavior, to reach causality and replicate findings. We describe the methods, demonstrate how researchers can apply them, and share practices from the design of experimental instruments to the replication of findings.

Traditional methods of studying and evaluating the use of online public access catalogs (OPACs) are discussed and compared to the protocol method. Verbal protocols are spoken records of people describing their work; they uncover detailed data about what people are thinking as they attempt to solve problems. The results of the Carnegie‐Mellon University protocol study of OP AC users are discussed.

This paper contributes to leadership categorization theory by advocating a new method to surface people's implicit leadership theories. The purpose of this new approach is to simultaneously capture individual difference in how they conceptualize leadership but within a common framework to allow for comparison of within- and between-person effects.

The authors conduct a narrative review of the implicit leadership theory, leadership categorization theory, cognitive mapping and verbal protocol literature with the purpose of surfacing a research method that will overcome the problems of over-simplification and over-individualization in existing methods.

The authors argue that using a combination of cognitive mapping and verbal protocols can capture the idiosyncrasies of individual lay theories of leadership while retaining the ability to compare people's responses through a common framework. The authors provide an example of how this method can be used to elicit people's perceptions of one aspect of implicit leadership theories, intelligence.

This new method will provide a methodology to test the subset propositions advocated by leadership categorization theory. These include the idea that subordinate level implicit leadership theories contain a subset of attributes found in the basic-level implicit leadership theories, that there is attribute integrity in superordinate implicit leadership theories through the levels, and the idea that people define leadership differently depending on the context they are observing.

Whereas previous approaches to surfacing people's implicit leadership theories either heavily constrain their responses with a predetermined generic suite of attributes or are totally open-ended and idiosyncratic, the authors advocate an approach that combines the best of both.

Behavioural studies of valuers have suggested that valuers rely on a number of cognitive strategies involving reasoning and intuition when undertaking a valuation task. However, there are few studies of the actual reasoning mechanisms in valuation. In other fields, much attention has been paid to forward and backward reasoning, as this shows the choices and decisions that are made in undertaking a complex task. This paper studied this during a valuation task. The purpose of this paper is twofold: first, to develop a methodological approach for empirical research on valuers’ reasoning, and, second, to report expert-novice differences on valuers’ use of forward and backward reasoning during a valuation problem solving.

The study utilised a verbal protocol analysis (VPA) to elicit think-aloud data from a purposive sample of a group of valuers of different levels of expertise undertaking a commercial-valuation task. Through a content analysis interpretive strategy, the transcripts were analysed into different cognitive segments identifying the forward and backward reasoning strategies.

The findings showed that valuers accomplished the valuation task by dividing the overall problem into sub-problems. These sub-problems are thereafter solved by integrating available data with existing knowledge by relying more on forward reasoning than backward reasoning. However, there were effects associated with the level of expertise in the way the processes of forward and backward reasoning are used, with the expert and intermediate valuers being more thorough and comprehensive in their reasoning process than the novices.

This study explores the possibility that forward and backward reasoning play an important role in commercial valuation problem solving using a limited sample of valuers. Given this, data cannot be generalised to all valuation practice settings but may motivate future research that examines the effectiveness of forward and backward reasoning in diverse valuation practice settings and develops a holistic model of valuation reasoning.

The findings of this study are applicable to valuation practice. Future training efforts need to evaluate the usefulness of teaching problem solving and explicitly recognise forward and backward reasoning, along with other problem-solving strategies uncovered in this study, as standard training strategies for influencing the quality of valuation decisions.

By adopting VPA, this study employs an insightful and rich dataset which allows an interpretation of thoughts of valuers into cognitive reasoning strategies that provide a deeper level of understanding of how valuers solve valuation problem; this has not been possible in previous related valuation studies.

This paper aims to report the development of an Islamic service quality scale that is derived from the literature, verbal protocol method interviews, and survey. Design/methodology/approach

Verbal protocol interviews were conducted with 24 men and 12 women from Indonesia. A pilot testing of the questionnaire was conducted with four Indonesian students. The items were further refined and pilot tested with six Indonesian students. Exploratory factor analysis and reliability analysis (n

The results indicate a robust measure of Islamic service quality: general Islamic values, Halal/Haram, attention to Islamic religious activities, honesty, modesty, and humaneness and trustworthiness. Future researchers can apply the Islamic service quality measure to Muslim consumers in other countries. Successful service providers need to be cognizant of the intrinsic roles played by Islamic values and practices among the Muslim consumers.

Major limitations include the recall ability of consumers during the verbal protocol method of interviewing, potential blending of Islamic values and Javanese culture, and the extent of separation of state and religion. Future researchers can apply the Islamic service quality measure to Muslim consumers in other countries.

Successful service providers need to be cognizant of the intrinsic roles played by Islamic values and practices among the Muslim consumers.

This paper contributes to the extant literature on Islamic marketing by developing a unique measure of service quality that is pertinent to Muslim consumers.

Nutrition labels are seen as offering the potential to educate consumers about healthy eating and to encourage and enable them to make healthy food choices. This study used verbal protocol analysis, a method new to food choice research, to examine which parts of the nutrition label are currently used by consumers when shopping “normally” and for “healthy” foods. The implications of the findings for nutrition educators are discussed.

Aims to improve understanding of an important and widely used management tool for motivating the salesforce – sales contests. Begins by introducing the question: what factors are associated with salespeople′s feelings towards a sales contest? Several potentially relevant characteristics that are expected to be associated with various feelings towards contests are discussed. To test the hypotheses, data were collected through verbal protocols and surveys from salespeople belonging to a division of a Fortune 100 firm. Results indicate that salespeople′s self‐esteem, commitment level, and career stage play a role in influencing feelings towards the sales contest.

This paper investigates how respondents to conjoint experiments process information and choose among product profiles, and how this varies with their knowledge about the product. Models for estimating conjoint attribute weights are almost exclusively based on principles of compensatory decision making. The paper aims to explore to what extent and in what way these basic principles of conjoint modelling are violated.

Data were obtained from a verbal protocol study where 18 undergraduate students each performed a total of 28 stated choice tasks while “thinking aloud”.

Results show that cognitive operations consistent with compensatory decision rules constitute a majority of the total number of operations performed across tasks and respondents. However, few respondents exhibited a consistent use of compensatory‐type processes throughout their choice sets. Results suggest that individual preferences interact with characteristics of the choice sets to instigate changes in information processing. It also appears that complete strategies are seldom used. Finally, respondents' knowledge about the product influences the cognitive operations that respondents use in solving conjoint tasks.

Results are based on responses from 18 undergraduate students, which makes generalizations hard.

One implication of this work is that one should apply a more flexible model framework to allow detecting the existence of non‐compensatory strategies.

This paper is one of few which aim to implement findings in behavvioral decision research within the context of conjoint analysis.