Test of a brief scale designed to measure high-level managers ’ indirect leadership

Purpose – The Indirect Leadership Questionnaire (ILQ) is constructed to measure important parts of a theoretical model of indirect leadership to be used in leadership courses for high-level managers. The ILQ consists of 15 model-derived items plus extra space that provides for free-text comments. This study aims to evaluate the psychometric properties of the ILQ, its predictive power and practical usefulness in indirect leadership coursesandto evaluatetheindirectleadership modelin the lightof new data. Design/methodology/approach – The study sample consisted of 225 higher managers and 6 – 10 of their subordinates(total: 1,703). The latter were working at least two hierarchical levels below their higher manager. The managersmade self-ratings, and the subordinatesrated theirrespectivehighermanagerusing the ILQ. Findings – Results showed that the ILQ has acceptable psychometric properties in terms of dimensionality (two model-compatible factors were obtained), reliability, discriminability and predictive power. Previous problems with many “ Don ’ t know ” responses from subordinates working two or more hierarchical levels below their higher manager were considerably reduced. A qualitative analysis of the free-text responses showed that indirect leadership behaviors that can be seen as inspirational in ﬂ uence and communication skills should be more emphasized in future indirect leadership courses. The leaders ’ personality, task-related competenceandperformance-orientationwere alsohighlighted more than in theindirectleadership model. Research limitations/implications – It was concluded that the ILQ is a useful tool in indirect leadership coursesbutthat itis toolimited to be used as a basis fortheorydevelopmentof indirectleadership. Originality/value – The ILQ has acceptable psychometric properties and provides an easy-to-use tool to gain practical,usable knowledgeof the “ how ’ s ” of indirect leadership.


Introduction
This study focused on the development of a brief questionnaire on indirect leadership, to be used in leadership training of high-level managers.Much research has been devoted to various aspects of direct or face-to-face leadership.In contrast, there are fewer studies on indirect leadership, or the influence of a leader on employees not reporting directly to him or her (Gordon and Yukl, 2004).Most of the existing research appears to focus on functional requirements of indirect leadership related to organizational levels (Johnsen, 2002;Luthans, 2005;Mintzberg, 1980;Watson, 2001;Yukl, 2002).Typically, these writings identify a number of "what's" of strategic or executive leadership, such as planning, organizing, problem-solving, informing, delegating and being visionary.Another research tradition has focused on desirable psychological characteristics of high-level leaders as individuals (Jacobs and Jaques, 1991;Glaser andStrauss, 1967, Jaques, 1976;Keegan, 1982;Klenke, 2005).Common themes here include being authentic, resilient, energetic and having conceptual skills.
Little attention has been paid to the "how's" of indirect leadership, here defined as follows: "What leaders do to influence individuals two or more hierarchical levels below themselves."An early attempt to study indirect leadership was done by Yammarino (1994), who analyzed two principal forms of indirect leadership.One is the cascade model, where leadership is filtered through intermediaries.The other is the by-pass model, where higher-level managers interact directly with employees two or more levels below themselves, without involving those in between.This research has been followed up by Avolio et al. (2004), Li et al. (2018) and Liu and Liao (2013), who all point to the challenges following from structural distance between leaders and followers rather than to the "how's" of indirect leadership.
Despite the existing valuable structuring of indirect leadership, and Yammarino's (1994) framework in particular, it has been argued that knowledge of the underlying processes, the "how's" of indirect leadership rests on a weak empirical ground (Gordon and Yukl, 2004).Following from this, a qualitative case study was performed to enhance the understanding of the issue (Larsson et al., 2005).This study, in turn, was followed by a quantitative investigation where the generalizability of the qualitatively developed model was tested (Larsson et al., 2007).The model is presented in Figure 1.
The model is limited to influence processes from a top-down perspective.As shown in Figure 1, indirect leadership may be understood as a process beginning with leaders at higher organizational levels forming ideas and mental models with regard to what should be done (visions and goals) as well as how to get it done (implementation).The following influence process takes place simultaneously along two tracks.The first is more actionoriented and is referred to as link.This is usually made up of a single individual or a small group of directly subordinate leaders.The link transmits the messages to the lower organizational levels.This form of influence includes whom you select as a link and how you interact directly with this/these individual(s).The second track is more image-oriented and is referred to as good example.Leaders higher up have an influence on lower organizational levels by being good, or less good, examples.The image-oriented influence process consists of four different components: (1) the higher leaders' basic attitude, which is comprised of his or her value base, individualized consideration and responsibility; (2) the leaders' driving force in terms of positive energy and conscious efforts; (3) the leaders' inspiration influence including appreciation of subordinates' efforts and encouragement of participation; and (4) the leaders' communication including information handling and clarity.

MRR
Both presented forms of influence are exposed to a filter in-between each hierarchical level.This means that information is sorted or distorted.In favorable cases, employees will trust the link and the upper leadership.These represent necessary conditions in terms of active involvement and participation.In unfavorable cases, this trust is lacking.This creates a breeding ground for redefinitions of messages and may result in penalties and rewards becoming necessary to ensure obedience (Larsson et al., 2005(Larsson et al., , 2018)).
When the inductively developed model of indirect leadership was operationalized in the aforementioned quantitative study (Larsson et al., 2007), three separate questionnaires were constructed, designed to cover all aspects of the model: one for the high-level manager whose indirect leadership should be evaluated (130 items), one for the high-level managers' directly subordinate managers (173 items) and one for employees further down in the hierarchy (143 items).When the model was later transformed into an indirect leadership course for high-level managers (henceforth labeled "IL-course"), it was obvious that the abovementioned questionnaires were too extensive.Several attempts to develop short, userfriendly versions, based on a selection or combination of items from the abovementioned longer questionnaires, failed.The main problem was that employees at lower hierarchical levels responded "Don't know" to a high number of items.The questions were expressed as behaviors and people further down in the organization had simply observed too little of the high-level managers' behaviors to be able to rate them reliably.In particular, this applied to

Model of indirect leadership
Test of a brief scale different aspects of the high-level managers' interaction with their subordinate managers reporting directly to him or her.
The IL courses grew in popularity and the need for a user-friendly questionnaire increased.Therefore, a new attempt was made with a new opening formulation of every item, a richer description of the content of the question and the possibility for the respondent to add free-text responses to each question.The new opening to each question read: "In my view the leader [. ..]".With an empirical base of 225 high-level managers' self-ratings and ratings made by more than 1,700 of their subordinates (see Method section below), we decided to make an evaluation of the psychometric properties, the predictive power and the practical usefulness of the new Indirect Leadership Questionnaire (ILQ) in indirect leadership courses.
Psychometric properties were restricted to include the dimensionality of the questionnaire, the reliability of dimension indices and the scale's capacity to discriminate between different subgroups; in this case the higher managers and their subordinates, female and male subordinates and subordinates from different kinds of work environment.The predictive power was restricted to an analysis of how the indirect leadership dimensions contributed to the subordinates' trust in their higher manager (see Figure 1).Although the empirical evidence of the relationship between higher managers' indirect leadership behaviors and the subordinates' trust in their higher manager was restricted to the aforementioned quantitative model-testing study (Larsson et al., 2007), the model suggests that a positive association exists.This led to the following hypothesis: H1.Favorable subordinate experiences of higher managers' indirect leadership behaviors are positively associated with their trust in higher managers and vice versa.
The practical usefulness of the questionnaire was restricted to an evaluation of the proportion of the subordinates' "Don't know" responses to each item.These aspects constituted the first aim of the present study.
When examining the obtained free-text responses (there was much written), it became obvious that these texts could be used to assess the content of the more than 15 years old qualitatively generated model of indirect leadership (Figure 1).This led to the second aim of this study: to evaluate the existing, theoretical model in the light of new data.

Participants
The study sample consisted of 225 higher managers and 1,703 of their subordinates.The sample of managers constituted all higher managers who had taken part in five-day courses in indirect leadership conducted by the Swedish Defence University at the time the study was carried out.The subordinates were working at least two hierarchical levels below their higher manager.Data were collected as part of the preparation for the five-day courses.About half of the higher managers had ten subordinates making ratings, the other half had somewhat fewer raters (the subordinates were selected by their higher manager, see below).Most participants (94% of the higher managers and 91% of the subordinates) were found in health care (48% of the higher managers and 41% of the subordinates), the educational sector (30% of the higher managers and 31% of the subordinates) or in private service organizations, mainly from telecom and transportation companies (16% of the higher managers and 19% of the subordinates).Approximately 60% of the managers, as well as their subordinates, were women, most were 30-50 years old and most worked in organizations having more than 500 employees.

MRR
The response rate was 100% among the higher managers.This was self-evident as responding to the questionnaire (before the leadership course) was a prerequisite for attending the course.Before each course, each manager also provided the course leader with a maximum of ten e-mail addresses of subordinates (who worked two or more hierarchical levels below him or her).All responses were handled by the Swedish Defence University, which provided the course leader with a report for each higher manager before the course.Each subordinate was connected to his or her higher manager with a key.During the leadership course, the manager could see his or her own ratings and the means of the ratings given by his or her subordinates.They could also see all free-text responses related to him or her, but they could not see who had written the text.
We have no data to substantiate whether the difference between the 2,250 theoretically maximum number of subordinates and the 1,703 received responses was caused by some managers not providing the questionnaire to ten subordinates and/or some subordinates refusing or forgetting to respond.No higher manager had fewer than six responding subordinates.

Questionnaire
Data were collected using a digital version of the ILQ.It consisted of 15 questions designed to map the theoretical indirect leadership model (Figure 1) plus one "Other comments" question.Each item had a nine-point response scale ranging from 1 (never or almost never) to 9 (always or almost always).The participants could also write a free-text response in connection to each question.The questionnaire is shown in the Results section below.

Analysis
The 1,703 subordinate participants were not independent in a statistical sense because each of the 225 higher managers was evaluated by the responding subordinates who rated him or her.Therefore, one subordinate was randomly selected for each higher manager.This was made by choosing the first received subordinate rating for each higher manager (the Web-based data collection system consecutively ordered the incoming responses from the subordinates to each higher manager when they were submitted to the Swedish Defence University).All presented results are based on this selection.To control the representativeness of the selected group, a new random selection of one subordinate per higher manager was done among the remaining subordinates (the third received subordinate response for each higher manager).Comparisons between the two subsamples of subordinates will be commented on in the text.
A factor analysis (principal axis factoring with oblique rotation) was performed.Oblique rotation was chosen because we expected interrelationships between emergent latent factors.Items with a factor loading of 0.40 or higher in a given factor, and having factor loadings of 0.29 or lower in the other factors, were accepted in the factor.Four items were excluded from the factor analysis because 20% or more of the subordinates had endorsed the "Don't know" response alternative.One item designed to measure trust in the leader was also omitted from the analysis.This item (presented in the Results section) was regarded as an outcome of the higher managers' indirect leadership.Thus, the factor analysis was based on the remaining ten items.Factor scores were computed by adding the raw scores of the items belonging to the factor and dividing the sum by the number of items.Thus, factor scores could range from 1 (lowest frequency) to 9 (highest frequency).The factor analysis based on the higher managers' responses yielded the clearest result, and it is this analysis that is shown in the Results section.
Cronbach's alpha coefficients were computed for each of the two obtained factors.Comparisons of the means of higher managers and their subordinates, and between female Test of a brief scale and male subordinates, were performed using t-tests.One-way analysis of variance, followed by Scheff e tests, were used to compare the means of the subordinates from different kinds of work environment.The two factors were regarded as predictors in a multiple regression analysis.Here, the aforementioned item designed to measure trust in the manager was used as the dependent variable.Statistical significance was assumed at p < 0.05.The free-text responses to all 16 questions from all 1,703 subordinates were analyzed qualitatively.To make the richness of data manageable, the free-text responses of 100 randomly selected subordinates were used for each of the 16 questions.Initially, a high number of codes were identified within each of the questions (responses regarded as having similar meanings although differently worded).An overview of the generated codes showed that there was a considerable overlap between the different codes stemming from the 16 questions.Therefore, the second step of the analysis, to bring together codes with resembling meaning to categories, was done using all generated codes, irrespective of which question they originated in.The outcome of the second step was eight categories.
The qualitative analysis performed could be likened to so-called open coding in the grounded theory method (Glaser and Strauss, 1967).The two steps in the analysis were not strictly sequential.Rather, we moved back and forth between the written text responses, the codes and the categories.Doubtful classifications were discussed among the authors until agreement was reached.

Ethics
The project was approved by the Leadership Advisory Board of the Swedish Defence University.All participants provided written informed consent.

Quantitative part
Proportion of "don't know" responses, dimensionality, reliability and discriminability.The proportion of "Don't know" responses from the subordinates ranged between 12% and 19% on the ten accepted questions.
The factor loadings shown in Table 1 indicate two comparatively clear factors that, taken together, explained 40.5% of the variance.The Kaiser-Meyer-Olkin (KMO) measure verified the sampling adequacy for the analysis: KMO = 0.87 (values between 0.80 and 0.90 are described as "great" by Field, 2013, p. 877).Reliability estimates (Cronbach's alpha) showed 0.80 for Factor 1 (competent, committed, exemplary model, communicative) and 0.70 for factor 2 (vision, goals, delegates, supports subordinate managers).Two items in Factor 2 were accepted despite having factor loadings lower than 0.40.
Table 1 further shows that the subordinates consistently gave their higher managers more favorable ratings than the managers did themselves.The mean differences are statistically significant on nine of ten questions, as well as on the two factors (not shown in Table 1).No statistically significant differences were found between female and male subordinates (t-tests, not shown in Table 1).The means of the subordinates from different kinds of work environment resulted in significant differences on nine of ten items and the two factors.The respondents from the private service sector consistently rated the higher managers less favorably than did the subordinates in the health-care and educational sectors (not shown in Table 1).In all the abovementioned aspects, fairly similar results were noted in the second set of randomly selected subordinates.
Regression analysis.A multiple regression analysis was performed using the subordinates' ratings.The two factor scores constituted the independent variables and were Test of a brief scale entered simultaneously.The dependent variable was the question designed to measure trust in the leader.The item read: "I have confidence in/trust in the leader I evaluate."This item was regarded as an outcome of the higher managers' indirect leadership (see Figure 1).The mean score among the subordinates was 7.47 (standard deviation [SD] = 1.48).
Table 2 shows that a comparatively high adjusted R 2 (0.71) was obtained.The two factors made significant contributions to the amount of explained variance.Almost identical results were obtained in the second set of randomly selected subordinates.However, the adjusted R 2 was lower: 0.34.The results confirm H1.

Qualitative part
Table 3 summarizes the result of the qualitative coding of the free-text responses given by all 1,703 subordinate participants to the 16 questions.Eight categories emerged from the analysis, each with a number of underpinning codes and actual free-text responses.The table notes that the categories "Inspiration" and "Communication," respectively, were found in the free-text responses to all 16 questions.The categories "Personality" and "Performanceorientation" were also found in more than half of the questions.The presentation of illustrating free-text responses shows that the borders between the categories are not razorsharp.The local context [the sentence(s) immediately preceding or following a selected illustration] contributed to the interpretation of the meaning of a given response.

Discussion
The first aim of this study was to evaluate the psychometric properties, the predictive power and the practical usefulness of the newly developed, theory-driven ILQ in indirect leadership courses.We begin by discussing the obtained raw values of the 15 items.A first reflection is that the problem with many "Don't know" responses, noted in previous pilot versions, was considerably reduced.Only 4 of 15 questions had a higher proportion than 20% of "Don't know" responses.However, additional efforts are still desirable to further improve the ILQ in this respect.On the other hand, it seems reasonable that employees two or more hierarchical levels below a high-level manager do not have knowledge of all of his or her activities.Thus, we argue that theoretical validity should carry a stronger weight than an ambition to simply reduce the number of "Don't know" responses when additional items are written.
A second reflection related to the raw values is the high absolute level of the scores obtained.This indicates that the high-level managers themselves, and even more so their subordinates, are satisfied with the indirect leadership.Possibly due to the high mean scores, the ILQ showed a partly acceptable and partly weak discriminability (see below).
The longer, descriptive wordings of the items have been appreciated by indirect leadership course leaders (oral communication).Combined with free-text responses, they  The manager is very good at summarizing and explaining complex questions in an uncomplicated way.Finds the core of question quickly, which makes you understand what he wants.To present vision, goals and messages can be complex to convey, but it is a strength that this manager is able to explain these kind of contexts Personality (11) Self-confidence, self-knowledge, prestigeless, patient, humble, calm, adaptable, flexible, courageous Anna is balanced.Is calm in meetings with others.I have never seen her flare up or lose her temper because of stress.She seems to be able to put the different parts of life together Performanceorientation (9) Business-like, hard-driven, solutionoriented, can read situations, acts in line with the best of the activity area, gives clear instructions Becomes too committed, so it almost drains energy (negative example) Exemplary model (8) Good role model, lives as he/she learns, visible, everyone is worth the same Bertil's tempo is high and gets a lot done.This can cause subordinates to put equal demands on themselves on keeping the same high tempo.Not everybody manages this (negative example) Individualized consideration (8) Cares about his/her coworkers, is responsive, welcoming, understands others, admits own mistakes, shares own thoughts and feelings She stays a moment at the coffee table and just sits down and listens and make small talks.Important things tend to come up there Task-related competence (6) Knowledgeable, full of energy, competent, experienced Gives well-structured presentations.Always leads and is well-prepared.Keeps order and makes sure everyone can have their say in turn order Structureorientation (5) Provides conditions, supervises the work atmosphere, requires leadership training, develops the staff, organizes common activities Much depends on that everything works at all levels.Our team leader lacks a clear mission from our top manager.Therefore, he has no possibility to work long-term strategically (negative example) Note: The figure below each category shows how many of the 16 possible questions the category could be found in Source: Created by authors Test of a brief scale covers many aspects of the model aspect "image-oriented influence" (see Figure 1).The factor "Vision, goals, delegates, supports subordinate managers" maps the top box of the indirect leadership model and the action-oriented form of influence described in the model (Figure 1).The reliability of the factor "Vision, goals, delegate, support subordinate managers" is acceptable but rather low (0.70).This is probably related to the limited number of items (four) and the factor needs to be strengthened.The discriminability of the ILQ was tested in three ways.First, the high-level managers' self-ratings were compared with their subordinates' ratings.A clear result emergedthe subordinates consistently gave higher (more favorable) ratings.A possible explanation of this is that high-level managers about to enter a leadership course where their ratings will be exposed may have a tendency to underestimate themselves in order not to look too presumptuous.Another possible explanation in line with Li et al. (2018) and Liu and Liao (2013) is that the subordinates perceived the structural distance to their managers as low.From the perspective of the organizations involved, the high ratings from both high-level managers and their subordinates are obviously positive.
A second test of discriminability was based on a comparison of the ratings provided by female and male subordinates.No significant differences emerged.This could indicate an instrument weakness.An alternative explanation is the overall high mean scores given by both men and women, making possible gender differences difficult to detect.The third test of discriminability involved a comparison of the mean scores given by subordinates in different kinds of work environment.No differences were found between subordinates in the health-care and educational sectors, whereas the subordinates working in the private service sector scored significantly lower (less favorable) on both factors and all individual items, but one.A possible explanation is that managers in the private sector experience demands to act "harder" than managers in public human relations-oriented work environments.In summary, the results gave a mixed picture regarding the discriminability of the ILQ.
The evaluation of the predictive power of the ILQ was based on the outcome question on confidence/trust in the high-level manager.A multiple regression analysis showed that the associations were in the expected direction and H1 was confirmed.
To conclude, the psychometric properties of the ILQ are acceptable for use in indirect leadership courses.It is easy to use digitally and uncomplicated for course leaders to summarize before or during a course.Additional and valid questions, possible for employees two or more levels below the high-level leader to evaluate, are desirable.However, we regard the rating scales of this short questionnaire as insufficient when it comes to scientifically based studies of the phenomenon "indirect leadership" in itself.This is mainly because too many model aspects are insufficiently covered with the ten items finally accepted in this study.
The second aim of the study was to evaluate the existing, theoretical model (Figure 1) in the light of new data.Two dominating results emerged from the analysis of the free-text responses.First, there appears to be a high degree of overlap between the different parts of the indirect leadership model when the qualitatively generated codes and categories are scrutinized.This means that the same categories can be found at different parts of the model.In particular, the categories "Inspiration" and "Communication" could be found in the free-text responses to all questions of the ILQ.The corresponding content can be found in the model, but it does not play the same dominant role there.This is notable and should be addressed in future research as well as during IL courses.
The second notable result is the relatively high presence of the categories "Personality" and "Performance-orientation."Based on the generated codes, the latter can be looked upon as a combination of the categories "Personality" and "Task-related competence."These MRR aspects concern the high-level manager as a person and cannot be found in the model of indirect leadership.However, they are included in the developmental leadership model (Larsson et al., 2003(Larsson et al., , 2018)), a Scandinavian adaptation of the transformational leadership model (Bass, 1998).The result indicates that subordinates ascribe high importance to this kind of person-factors, and it should receive additional attention in future research as well as in indirect leadership courses.
The model of indirect leadership and the ILQ focus on a number of how-aspects of this kind of leadership.While most existing research emphasize what-aspects of indirect leadership (Avolio et al., 2004;Johnsen, 2002;Li et al., 2018;Liu and Liao, 2013;Luthans, 2005;Mintzberg, 1980;Watson, 2001;Yukl, 2002), we argue that a how-framework yields an additional aspect of indirect leadership.Future research may consider theoretically integrating the indirect leadership model with other related leadership models such as transformational leadership (Bass, 1998) or its Scandinavian adaptation developmental leadership (Larsson et al., 2003(Larsson et al., , 2018)).In the latter, the similarities with the image-based influence components basic attitude, inspiration and communication have already been noted, while the driving force aspect is only found in the indirect leadership model (Larsson et al., 2018).Gordon and Yukl (2004) conclude that indirect leadership forms of leadership must be compatible with direct leadership behaviors used by managers at all levels of the organization.The suggested theory integration could contribute to this and similar kinds of arguments in favor of leadership theory integration have been presented by Walumbwa et al. (2008).
Methodologically, we feel the study has five strengths.First, the ILQ items were theoretically driven.Second, the items mainly describe behaviors rather than leader traits or behavioral intentions.This should be advantageous from a reliability perspective.Third, a validity aspect of an instrument is how it is associated with a criterion-related measure.The outcome of the regression analysis using confidence/trust in the high-level leader as dependent variable, supports what could be predicted by the theoretical model.Fourth, the qualitatively generated categories based on free-text responses add insights into indirect leadership, not shown in the indirect leadership model.Finally, the large sample size is a study strength.
A self-evident limitation of the study is that the used model of indirect leadership, like most models, does not include all relevant or important constructs.The qualitative analysis of free-text responses in this case indicates possible model modifications needing further studies.Another limitation is that we have no control of the higher managers' selection of subordinates.They were simply instructed to ask a maximum of ten subordinates at least two hierarchical levels below themselves if they were willing to participate.
Study weaknesses include lack of data on the organizational contexts the study participants came from.The observed differences between different kinds of work environment highlight this need.The limited method of measuring the results of indirect leadership (only ratings of confidence/trust in the high-level leader) is another weak spot.Knowledge of how the studied aspects develop over time is also lacking, which calls for longitudinal study designs.It should also be noted that the codes and categories qualitatively derived from the data are concepts of a sensitizing rather than definitive character as described by Blumer (1954).

Conclusions and practical implications
The following conclusions can be drawn from this study: The ILQ has acceptable psychometric properties when used as a supportive tool in indirect leadership courses.The support is strengthened when the ratings are seen Test of a brief scale together with the free-text responses.It is desirable to add valid items where the proportion of "Don't know" responses is low.The rating scales in the ILQ are too limited to be used as a basis for theorydeveloping research on indirect leadership because too many potentially relevant aspects are not covered by the ten-item instrument.Indirect leadership behaviors which reflect inspiration and communication skills should be further studied and more emphasized in indirect leadership courses.
During indirect leadership courses, as well as in future research, special attention should be paid to the high-level manager's personality, task-related competence and performance-orientation.
Combining theories of indirect leadership and transformational leadership, or its Scandinavian adaptation developmental leadership, could provide a valuable educational and leadership course development.
Figure 1.Model of indirect leadership

Table 2 .
The dimensionality of the ILQ was tested with an explorative factor analysis, and two factors emerged.The factor "Competent, committed, exemplary model, communicative"