Effectiveness of educational leadership through directors' performance in online higher education. The biggest online university in Spain

José Manuel Palomino Fernández (International University of La Rioja, Logroño, Spain)
María Pilar Cáceres Reche (Department of Didactics and School Organisation, University of Granada, Granada, Spain) (University of Granada, Granada, Spain)
Fernando Lara Lara (Department of Didactics and School Organisation, University of Granada, Granada, Spain) (University of Granada, Granada, Spain)
Blanca Berral Ortiz (Department of Didactics and School Organisation, University of Granada, Granada, Spain) (University of Granada, Granada, Spain)

Leadership & Organization Development Journal

ISSN: 0143-7739

Article publication date: 17 November 2023

Issue publication date: 26 March 2024

930

Abstract

Purpose

The study aims to analyze pedagogical e-leadership in online higher education in Spain through the application of VAL-ED at the International University of La Rioja.

Design/methodology/approach

The methodology used to achieve the objectives has been a descriptive and quantitative methodology through a cross-sectional study based on the implementation of the questionnaire: Adaptation of the VAL-ED to the university context, developed by Palomino et al. (2022a).

Findings

Indeed, the data obtained allow the authors to affirm how both directors and supervisors, as well as teachers, have been able to evaluate the leadership behavior of directors, compared to the competence standards of VAL-ED, having obtained very positive results that show how their leadership is, without a doubt, oriented to students. Given the fact that in the second specific objective the study sets out to determine to what extent the three groups of respondents (faculty, supervisors and directors) coincide in the effective performance of the pedagogical leadership of the directors, it can be highlighted that no great differences have been found in the responses of effectiveness obtained from directors, supervisors and teachers, since the results of the resulting scores among these three groups of respondents were reasonably similar.

Originality/value

The research is original as the sample was collected personally by the authors of this article.

Keywords

Citation

Palomino Fernández, J.M., Cáceres Reche, M.P., Lara Lara, F. and Berral Ortiz, B. (2024), "Effectiveness of educational leadership through directors' performance in online higher education. The biggest online university in Spain", Leadership & Organization Development Journal, Vol. 45 No. 2, pp. 169-189. https://doi.org/10.1108/LODJ-02-2023-0055

Publisher

:

Emerald Publishing Limited

Copyright © 2023, José Manuel Palomino Fernández, María Pilar Cáceres Reche, Fernando Lara Lara and Blanca Berral Ortiz

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


Educational Leadership is becoming a significant factor in improving education and a political priority since it can positively influence the results of both institutions and students. In virtual environments, leadership is called e-leadership or virtual leadership. Indeed, virtual learning has entered the mainstream of higher education as an agent of strategic change, and this transformation requires e-learning leaders. But e-leadership research in the university context has not yet been sufficiently developed. This research explores to what extent the influence of learning management in higher education and specifically in online training contexts, such as the International University of La Rioja, is aimed at increasing and improving the performance of its students, as well as in improving the quality of institutions. There has been an analysis of the pedagogical e-leadership in online higher education in Spain through the application of the Vanderbilt Assessment of Leadership in Education at the International University of La Rioja, specifically in the Faculty of Education, identifying the effectiveness of behavior in the director's leadership, taking the Vanderbilt Assessment of Leadership in Education competency standards as a reference and verifying which areas of leadership behavior represent the greatest strengths, limitations and main needs for improvement.

Introduction

The use of information and communication technologies (ICT) nowadays is standardized in teaching-learning processes. This reality implies that educational organizations must be transformed to become flexible and innovative structures, open to new teaching methods and didactic resources, putting at the center the individual capable of reflecting and analyzing their environment, highlighting the constructivist methodology in which the student builds his own knowledge through autonomous work processes and the use of different technological tools (Palomino and Ramos, 2021).

In this context, virtual education is a reality characterized by being a teaching-learning process based on the principles of an active pedagogy, in which the student is responsible for participating frequently and effectively in different virtual environments (Aurangzeb and Mazhar, 2020). Unsurprisingly Chwen-Li et al. (2022) state that we can consider virtual learning as a new paradigm in the learning process, as it does not require face-to-face interaction and can be done without relying on a traditional classroom environment. However, as Cordie and Lin (2018) state, the prevalence of virtual learning leads to unique challenges, including dealing with various stakeholders, providing an effective product to students and establishing new paradigms on how to provide educational components.

Likewise, a new element stands out as an important part of this process: the reality of leadership. Indeed, leadership has emerged not only as a fundamental element for institutional success in achieving this transformation but also as a key factor in improving student performance. Not surprisingly, leadership in education is becoming a major factor in the improvement of education and a political priority. Indeed, in Marichal-Guevara et al. (2018), we find how various studies conclude that the exercise of leadership in education determines, in an important way, the quality of education.

When it comes to improving the quality and results of students in higher education institutions, different leadership models have been proposed; however, in of all of them, it is worth highlighting the role of pedagogical leadership in improving both the performance of students and the quality of universities in an effective way.

That is why in higher education institutions that teach their studies in the online modality, we will refer to pedagogical e-leadership to refer to that model of pedagogical leadership that is developed in virtual environments, since this reality will also influence leadership in educational institutions, facilitating the student both to reflect and analyze their environment from a constructivist approach, in which he himself builds his knowledge through autonomous work processes (Ibarra et al., 2023).

Indeed, in virtual educational environments, the context of traditional leadership changes, since they allow us to develop and carry out a multitude of jobs, projects and tasks at any time and in any place, both in real and virtual spaces. In fact, some authors define e-leadership not only as an extension of traditional leadership but as a fundamental change in the way in which leaders and people relate to each other within organizations, as well as the way in which different organizations relate to each other (Tintoré and Gairín, 2022).

We cannot ignore the fact that e-leadership takes shape in a context where collaboration and interaction, which seek to create and distribute an organizational vision in which goals, organizations and individuals are united, will be mediated by ICT (Antonopoulou et al., 2021).

Along the same lines, Avolio and Kahai (2003) state that, although e-leadership is a concept that emerged relatively early and with continued conceptual ambiguity, there will be significant differences between leading a traditional organization and leading those that have virtual environments mediated by ICTs, since these environments seem to require leaders who can deal with the paradoxes and dilemmas, as well as the complexity of the behaviors associated with the use of ICTs. It is essential to consider this nuance since educational e-leadership is based on the whole theory of leadership, but it does so assuming the characteristics of its nature and content.

Miller and Ives (2020) state how e-learning has entered the mainstream of higher education as a strategic change agent, and this transformation requires e-learning leaders to develop the skills to successfully innovate at a time of increased competition and rapid technological change.

On the other hand, when we transfer the concept of leadership to the field of education, the concept of pedagogical leadership stands out, understood as that which is oriented toward improving student performance and results and is widely recognized as an important and effective element in improving educational quality. If, on the one hand, the first factor that has a significant impact on student learning is the teaching staff, the second element that can greatly influence the performance of a faculty, by facilitating a series of conditions and environments to improve the context in which it can better perform its work and students improve their learning, will be leadership (Anderson, 2010).

Not surprisingly, as Harris et al. (2022) assert, leadership in educational contexts can have a particularly positive influence on both institutional and student outcomes when it is distributed. Indeed, management and quality are related in many ways, and a strong leadership position enables distance education programs to achieve their goals successfully and efficiently while remaining responsive to the needs of their beneficiaries and users (Chang et al., 2022).

However, in a study conducted by Palomino et al. (2022a) in which they analyze the incidence of pedagogical e-leadership in higher education institutions from the most current scientific literature, we find how the scarce amount of work in this line indicates that research is still insufficient, thus evidencing the need to further deepen this area of research, which will revert in the efficiency and effectiveness of organizational development, as well as in the improvement of all dimensions of educational institutions of higher education. So, when it comes to understanding this reality, it is convenient to start from the fact that “Higher Education institutions are complex organizations in which their management is a challenge for leaders” (Smith, 2020, p. 39).

That is why the need arises to continue the research work observing the necessary scientific standards that ensure the quality of knowledge, to understand to what extent the influence of learning management in higher education, and specifically in online training contexts such as The International University of La Rioja (from now on UNIR), is aimed at increasing and improving the performance of its students, as well as in improving the quality of institutions. This, then, is the research problem from where we start this study.

UNIR was created with the intention of allowing access to higher education to all people who, for different reasons, cannot study in person and who need to combine their learning with the different tasks they do every day. To achieve this goal, they have developed a unique methodology, with a strategic plan that seeks to ensure quality education. The features that define this methodology are the following: effective teaching focused individually on each student, adapting the pace of their training to their specific circumstances – UNIR provides a tutor to each student, to make it easy to provide individual tuition, finally, UNIR ensures the full commitment of faculty and management staff to the success of each student (UNIR, 2022).

Among the values that UNIR marks as a university, the following stand out: an offer of degrees and programs that meet the needs of society; careful attention to the quality of teaching and resources; commitment to student success; personalization of teaching tasks with the accompaniment of the student throughout their learning process; ability to respond to the expectations of stakeholders; and constant attention to innovation in teaching and learning methodology (UNIR, 2022).

UNIR has 7 faculties and schools, in which 154 degrees and official masters and 95 own degrees are taught. The Faculty of Education currently offers 15 degrees, 23 official master's degrees, 7 own master's degrees and 17 continuing education degrees.

The UNIR team is made up of teaching and research staff (PDI) and management and administration staff (PGA). Both work in the same direction so that everything works effectively in each sector of the university.

Objectives

General

The study aims to analyze pedagogical e-leadership in online higher education in Spain through the application of VAL-ED at the International University of La Rioja. The Vanderbilt Leadership in Education Assessment (VAL-ED) is a multiple assessment tool, with an evidence-based rating scale, that assesses the behaviors of directors known to directly influence faculty performance and, through them, student learning (Vanderbilt University, 2011a, c).

Specific

  1. Identify the effectiveness of behavior in the leadership of the director taking as reference the competence standards of VAL-ED.

  2. Determine to what extent the three groups of respondents (faculty, supervisors and directors) agree on the effective performance of the pedagogical leadership of directors.

  3. Verify which areas of leadership behavior represent the greatest strengths, limitations and top needs for improvement

Methodology

The methodology used to achieve our objectives has been a descriptive and quantitative methodology through a cross-sectional study based on the implementation of the questionnaire: Adaptation of the VAL-ED to the university context, developed by Palomino et al. (2023).

These authors highlight how, for decades, educational research has focused on finding ways to improve leadership effectiveness, valuing managerial evaluations as an important tool that can detect those improvements. However, developing these assessment tools has always been challenging, because they vary greatly from each other in both content and focus, as well as different perspectives on what leadership behaviors should be assessed. In response to this need, in 2005, a team of researchers from Vanderbilt University and the University of Pennsylvania, both in the United States, developed a valid, reliable and theoretically grounded assessment model of managerial educational leadership that has recently been adapted to the context of higher education. Notably, the VAL-ED assumes that effective student-centered leadership lies at the intersection of two dimensions: the core components built through key processes.

As we find in Palomino et a. (2022b), it is important to note that the VAL-ED has been designed to evaluate critical leadership behaviors to perform diagnostic analysis, performance feedback, progress tracking, professional development planning and summative evaluation. These authors highlight how there have been different studies in which the validity and reliability of the VAL-ED have been tested:

Polikoff et al. (2009) evaluated the differential operation of VAL-ED elements. Porter et al. (2011) studied the validity and reliability of VAL-ED through a nationwide study in the United States. Minor et al. (2014) also wanted to check the validity of VAL-ED in primary and secondary schools in the United States. They carried out a study in which they analyzed the accuracy with which the VAL-ED scores can identify the belonging to the two groups previously selected by the superintendents (directors whose performance of their functions was in the top 20% and the bottom 20%). Goldring et al. (2015) investigated the psychometric characteristics of the Vanderbilt Assessment of Leadership in Education, analyzing its convergent and divergent validity. They hypothesized that VAL-ED is highly correlated with other measures of instructional leadership. Likewise, in their study, Goff et al. (2015) highlight how numerous researchers have conducted several studies to validate VAL-ED, proving to be a reliable instrument that can be used in multiple contexts. Minor et al. (2017) evaluated the test-retest validity of the VAL-ED. They found that the principal and teacher grades of moment 1 and moment 2 have significant, positive and meaningful correlations. With these results, the Vanderbilt Evaluation of Leadership in Education (VAL-ED) instrument is, therefore, a reliable and valid instrument to measure the effectiveness of educational leadership.

On the other hand, we know that when assessment instruments are used as intended with a sufficiently significant sample for each of the groups that have interacted with a director, these instruments (in our case the adaptation of the VAL-ED in the context of higher education) will produce reliable and valid results on key leadership behaviors focused on learning, facilitating interpretations that will allow directors to discover areas for improvement that affect leadership focused on more effective learning (Palomino et al., 2022b).

Data collection tool

The questionnaire used to evaluate the pedagogical leadership of directors has been the translation and adaptation of the VAL-ED to the Spanish university context developed by Palomino et al. (2023). These authors considered theoretical and empirical antecedents, as well as the judgment of several experts in the subject of the universities of Granada and the International University of La Rioja to validate the adequacy of the questions in the university context. It was also validated by a graduate in translation and interpreting, a graduate in English philology and a graduate in Hispanic philology to guarantee the correct translation.

Since leadership focused on effective learning is at the intersection of two dimensions, the main components created through key processes, the VAL–ED has been designed to evaluate both the main components and the key processes. The main components refer to the characteristics of educational institutions that support student learning and improve the ability of the teaching staff to teach their students. On the other hand, key processes refer to how leaders create and manage those major components.

Main components

As we have already highlighted and as we find in Porter et al. (2008), the VAL-ED framework includes six main components that represent the constructs of learning-centered leadership:

  1. High standards for student learning: These are defined as the degree to which leadership ensures that individuals, teams and academic goals are aligned to achieve rigorous learning academically and socially.

  2. Rigorous curriculum: It is understood as the content of the instruction and which must be ambitious.

  3. Quality instruction: As these authors point out, a rigorous curriculum by itself is insufficient to ensure optimal student learning, since quality instruction is also required, that is, an effective instructional practice that maximizes the academic performance and social learning of students.

  4. Culture of learning and professional behavior: Another major component of the assessment framework is leadership that ensures that the institution is organized, rather than from a bureaucratic point of view, as a learning community, in which student development from both an academic and social point of view is at the center.

  5. Connections to external community: Leading an institution with high expectations and academic achievement on the part of all students also requires strong connections with the community.

  6. Performance accountability: There is an individual and collective responsibility among the leader, faculty, students and community to achieve rigorous academic and social learning goals.

Key processes

Similarly, the conceptual framework presents six key processes which, although they are interconnected and are recursive and reactive between them, for the purposes of descriptive evaluation and analysis, are reviewed each individually.

  1. Planning is understood as the articulation of a shared direction, as well as the implementation of coherent policies, practices, and procedures, which helps to focus resources, tasks, and people.

  2. Implementation consists of putting into practice the necessary activities to achieve high standards of performance by students.

  3. Supporting, In that leaders must create enabling condition and; ensure and use the financial, political, technological and human resources necessary to promote academic and social learning. Supporting is a key process that secures the resources needed to make core components available and well utilized.

  4. Advocating is understood as addressing the diverse needs of students, ensuring that policies in the educational institution do not pose or create barriers for certain students, as well as that students with special educational needs receive content-rich instruction.

  5. Communicating consists of the development, use and maintenance of information exchange systems between members of educational institutions and with their external communities. This communicating should inform, promote and link institutions being key in supporting students' academic and social learning.

  6. Monitoring involves the collection and analysis of data in a systematic way to make judgments that guide decisions and actions for continuous improvement (see Table 1).

According to empirical research (Hallinger and Heck, 1996; Leithwood et al., 2004), this assessment model does not foresee the direct effects of leadership behaviors on student success, but rather those leadership behaviors that will lead to changes in teacher performance that, in turn, will lead to student success (Murphy and Meyers, 2008).

Finally, the fact of being a 360-degree evaluation tool implies that the different key people around the director (that is, faculty, the director himself and the supervisors of the latter) will be the ones who answer the questionnaire to evaluate leadership. In order to verify the reliability of the questionnaires, Cronbach's alpha value was calculated for each of them, obtaining the following values: 0.896 for the Supervisors questionnaire, 0.948 for the directors/coordinators questionnaire and 0.938 for the teachers' questionnaire. Since all the values obtained are very close to 1, we can conclude that the three questionnaires obtained are reliable.

Sample

The sample has been obtained from the Faculty of Education of the International University of La Rioja. Of the 38 official degrees offered at the Faculty of Education of the International University of La Rioja, 11 participated in the study, obtaining the following sample of participants: 11 directors, being 3 degree directors and 8 directors of official master's degrees; 5 supervisors of the directors of the 11 participating degrees, with 2 supervisors of directors of bachelor's degrees and official master's degrees, and 3 supervisors of directors of official master's degrees; finally, a total of 89 professors from the 11 degrees participated in the study.

In this way, a participant sample formed by 105 people, representing 100% of the directors of the degrees; 100% of the supervisors of these, and more than 75% of the teachers who teach in each degree was obtained. We should take into account that neither the directors of the master's dissertation subject, nor the teachers of the practices subject, have participated. The sample represents in a heterogeneous way the different degrees of the Faculty of Education and the interested parts.

Procedure

To evaluate the leadership evidence of the three agents involved (faculty, directors/coordinators and supervisors), data collection was carried out between May 2021 and February 2022. In all, 105 questionnaires were collected. The instrument was applied in digital format, and the participants in the research were previously informed about the objective of the study. Their participation was voluntary, and responses confidential and anonymous.

To achieve the potential of VAL-ED for its intended purposes and to be managed as designed, as suggested by Vanderbilt University (2011a, b), the following aspects were considered in both the key usage summary and the implementation guide:

  1. The teachers invited to complete the VAL-ED had to result in a representative and reasonably large sample. In this sense, we obtained 89 responses that, although they do not represent the 100% percentage of faculty members who should have responded to the surveys, we can consider as a sufficient number of responses to carry out the study. On the other hand, it should be noted that, since both the appropriate supervisors were included, together with the directors and the faculty members of the degrees, the result is a 360-degree evaluation.

  2. Similarly, the evaluation was coordinated by a neutral and objective person who did not complete the VAL-ED.

  3. It is recommended, on the other hand, that the evaluation should only be carried out, at the earliest, during the end of the second month of the academic year, as this increases the likelihood that respondents have had a reasonable opportunity to interact with the director they are evaluating. In this sense, it should be noted that the questionnaires were collected at the end of the second semester of the 2020–21 academic year and at the end of the first semester of the 2021–22 academic year.

  4. Teachers were guaranteed that their answers were anonymous, and they were allowed adequate time to read, reflect on the evidence, and rate the director's behavior.

Analysis and interpretation of results

As we found in Vanderbilt University (2011a, c), the VAL-ED behavior inventory provides information on a total score, six subscales for core components, six subscales for key processes separately for each group of respondents and an overall average among the groups of respondents.

The most important score that results from the VAL-ED is the overall total effectiveness score of the director. This score is based on the average ratings of all respondents, where each group of respondents has the same weighting and is reported on the 5-point effectiveness metric used to rate each of the 72 elements of the instrument. Therefore, the director's overall total effectiveness score and core component and key process subscale scores are reported on a continuous scale from a minimum of 1.0 (non-effective) to a maximum of 5.0 (very effective). The various effectiveness subscale ratings for core component and key process behaviors fall within this same operating range.

Comparison of proficiency standards with cut-off scores

Vanderbilt University (2011a, c) highlights that to have an interpretative framework that allows us to identify the level of competence of the director, it is necessary that we define each of them. For this, a series of cut-off scores will be established, which will establish each of the levels.

  1. Lower than basic: A leader at the sub-core level displays leadership behaviors in key components and processes at levels of effectiveness that, over time, are unlikely to influence faculty and student performances.

  2. Basic: A leader at the basic level of competency displays leadership behaviors in key components and processes at levels of effectiveness that, over time, are likely to influence and result in acceptable added value for performance for some, but not all, groups of students.

  3. Competent: A competent leader exhibits leadership behaviors in key components and processes at levels of effectiveness that, over time, are likely to influence the faculty and result in acceptable added value for the performance of all students.

  4. Distinguished: A distinguished leader exhibits leadership behaviors in key components and processes at levels of effectiveness that, over time, are sure to influence the faculty and result in strong added value for the performance of all students.

The three cut-off scores used to differentiate the four levels of leadership proficiency are as follows: 3.29 will determine the boundary between the basic and lower levels than basic; 3.60 between basic and competent; and 4.00 between competent and distinguished.

The result of these cut-off scores implies that directors who score in the range of 1.0–3.28 will be rated with leadership behavior at the “lower than basic” level. Directors who score average in the range of 3.29–3.59 will be described as leadership behavior at the “basic” level. Directors who score an average response in the range of 3.60–3.99 will be described as leadership behavior at the “competent” level. Finally, directors who score average in the range of 4.00–5.00 will be described as leadership behavior at the “distinguished” level.

Results

As discussed before, the VAL-ED behavior inventory provides information on a total score, six subscales for core components, six subscales for key processes separately for each group of respondents and an overall average among the groups of respondents. The core components and key processes are based on the same information, so while their information is redundant, the two separate profiles offer diagnostic information on how a director's behavior can be improved to achieve a more effective institution and, in turn, improve student performance.

The data obtained allow us to analyze the pedagogical e-leadership in the Faculty of Education of the UNIR, as well as identify the effectiveness of behavior in the leadership of the directors taking as reference the competence standards of VAL-ED, as well as verify which areas of leadership behavior represent the greatest strengths, limitations and main needs for improvement.

As can be seen from Tables 2 and 3, scores obtained at the intersection of the “high standards for student learning” core component with each of the key processes result in distinguished performance by directors in each of them. The scores obtained at the intersection of the “rigorous curriculum” core component with each of the key processes result in a distinguished performance in all these, except for the key component “planning”, in which the performance would be competent.

Regarding the scores obtained at the intersection of the main component “quality instruction” with each of the key processes, it is worth noting how we find a distinguished performance in all of these, apart from the key component “Monitoring”, in which the performance would be competent. In addition, scores obtained at the intersection of the main component “culture of learning and professional behavior” with each of the key processes result in distinguished performance by directors in all of them.

It is noteworthy, however, how the scores obtained at the intersection of the main component “Connections to external community” with each of the key processes result in competent performance in each of them, except for the key component “Supporting”, in which the performance would be basic. Finally, the scores obtained at the intersection of the main component “performance accountability” with each of the key processes again result in distinguished performance by directors in all of these.

When we compare the performance results obtained by each of the groups of respondents – directors, supervisors and faculty (Tables 4–6) – to determine to what extent the three groups of respondents (faculty, supervisors and directors) agree on the effective performance of the pedagogical e-leadership of the directors, as we proposed in the second specific objective, we observe how the results of the interaction of the main components “standards for learning of high students” and “rigorous curriculum”, coincide with all of them obtaining a distinguished assessment in the performance of the directors.

In the case of the main component “quality instruction”, the level of performance obtained in four of the key processes coincides, finding differences in the key processes of planning and monitoring in the case of the group of respondents of teachers; and advocating and monitoring in the group of respondents of supervisors, who rate them with a performance as competent, compared to the distinguished with which directors are evaluated in all of them.

When we analyze the results of the main component “culture of learning and professional behavior," the level of performance obtained in the six key processes coincides when we compare the responses of the groups of directors and teachers. Both groups qualified it with the level of distinction. However, when comparing the scores of the group of directors with that of supervisors, we find differences in the key process of monitoring: while directors rate performance with the level of distinguished, the group of supervisors qualifies it as competent.

It is in the results of the main component “connections to external community” that we find greater differences in the performance scores of each of the key processes. The first thing that stands out is that it is the component in which the lowest scores have been obtained in all groups of respondents. On the other hand, it highlights how it is the supervisors of the directors and the directors group of respondents that give the lowest scores to the performance of the key processes of this component, since the highest level of performance obtained is that of competent in the communicating process, having being valued with the lower than basic level. The teachers value a competent performance in four of the processes evaluated: implementing, supporting, communicating and monitoring, compared to the three that we find in the group of directors: implementing, supporting and advocating.

It should be noted that in the main component “responsibility for performance,” it is the group of teachers that give the lowest score to the planning process, qualifying it with a competent performance, compared to distinguished that the groups of directors and supervisors grant. In the rest of the processes of this component, all groups of respondents agree in their evaluation, evaluating the performance of directors with a distinguished level.

As we have discussed before, while it is true that the core components and key processes are based on the same information, studying the two separate profiles offers diagnostic information on how a director's behavior can be improved to achieve a more effective institution and, in turn, improve student performance. From the results obtained (Table 7), it can be seen that the level of performance of directors in the main components is distinguished, except for the component “connections to external community” that obtains a level of performance of competent.

On the other hand, when analyzing the performance level of the key processes (Table 8), we observe how the level of competence obtained is distinguished in all of them.

Conclusions

The evolution of virtual learning environments underlines the importance of the role of leaders in improving both the development of institutions and the achievement of proposed goals and objectives. Indeed, the success of these institutions is closely linked to meeting the needs of students and the demands of a society in permanent change. In this regard, it is essential that leaders understand these needs, so that they can provide opportunities to develop effective programs focused on both the student and the mission and academic culture of the institution.

In this research, through the application of the adaptation and translation of the VAL-ED for the university context, we have been able to advance a better understanding of the effectiveness of e-leadership behaviors focused on the learning of the managers of the Faculty of Education of the International University of La Rioja. Likewise, we have also been able to identify those areas of e-leadership behavior that represent areas of relative strength, and which represent areas of possible improvement on which further work is needed.

The main objective was to analyze the pedagogical e-leadership in online higher education in Spain through the application of the VAL-ED at the International University of La Rioja, specifically in the Faculty of Education. To do this, first we have sought, as we set out in the first objective, to identify the effectiveness of behavior in the director's leadership, taking the VAL-ED competency standards as a reference. Likewise, it has been possible to verify which areas of leadership behavior represent the greatest strengths, limitations and main needs for improvement.

Indeed, the data obtained allow us to affirm how both directors and supervisors, as well as faculty, have been able to evaluate the e-leadership behaviors exercised in a virtual teaching environment by managers in comparison with the VAL-ED competency standards, having obtained very positive results that show us how their leadership is oriented to students. Given the fact that in the second specific objective we set out to determine to what extent the three groups of respondents (faculty, supervisors and directors) coincide in the effective performance of the pedagogical leadership of the directors, it can be highlighted that no great differences have been found in the responses of effectiveness obtained from directors, supervisors and teachers; since the results of the resulting scores among these three groups of respondents were reasonably similar, this allows us to affirm that the data obtained are valid and in line with the reality of the e-leadership of directors.

Among the limitations that we find in this study, the first one that we should highlight is the sample size of the group of teachers, as although a response rate of 75% or more is considered high and therefore desirable by increasing the likelihood that the resulting assessment data will be representative of the interacting respondents with the director, the ideal would have been to have a 100% response rate, as it has happened with the rest of the groups (directors and supervisors).

On the other hand, it should also be noted that, when determining the level of effectiveness and performance of directors, we have worked with average scores. These, like any test score, are observed scores and are likely to have some associated error, although it should be added that the possible error in the VAL-ED is very low.

Likewise, we know that one of the characteristics that the VAL-ED has when applied in contexts of compulsory education (primary and secondary) is that it evaluates to what extent the leadership behavior of the director is effective compared to a national sample of principals. In this sense, as this study is the first in which the questionnaire is applied in a context of higher education, added to the fact that we do not have a sufficiently significant sample of results at the national level, we have not been able to make this comparison.

Lastly, and as we find in Palomino et al. (2022a), given the small number of works in this line, and in view of the results obtained, it is worth highlighting the need to go deeper into this area of research, that will lead to the efficiency and effectiveness of organizational development, as well as in the improvement of all dimensions of educational institutions of higher education. Therefore, the research work must continue observing the necessary scientific standards that ensure the quality of knowledge, in order to understand to what extent the influence of learning management in Higher Education is aimed in order to increase and improve the performance of its students, as well as the improvement of the quality of the institutions.

Among the main future lines of research that could be inspired by these results, on the one hand, is the investigation of the leadership models that are being carried out in educational institutions. On the other hand, it would also be necessary to analyze the role of the leader both in the promotion of student performance and in the increase of quality in higher education institutions.

VAL-ED 360 assessment

VAL-ED 360 assessment
PlanningImplementingSupportingAdvocatingCommunicatingMonitoring
Standards for student learning raisedQuestion 1Question 1Question 1Question 1Question 1Question 1
Question 2Question 2Question 2Question 2Question 2Question 2
EvidenceEvidenceEvidenceEvidenceEvidenceEvidence
Rigorous curriculumQuestion 1Question 1Question 1Question 1Question 1Question 1
Question 2Question 2Question 2Question 2Question 2Question 2
EvidenceEvidenceEvidenceEvidenceEvidenceEvidence
Quality instructionQuestion 1Question 1Question 1Question 1Question 1Question 1
Question 2Question 2Question 2Question 2Question 2Question 2
EvidenceEvidenceEvidenceEvidenceEvidenceEvidence
Culture of learning and professional behaviorQuestion 1Question 1Question 1Question 1Question 1Question 1
Question 2Question 2Question 2Question 2Question 2Question 2
EvidenceEvidenceEvidenceEvidenceEvidenceEvidence
Connections to external communityQuestion 1Question 1Question 1Question 1Question 1Question 1
Question 2Question 2Question 2Question 2Question 2Question 2
EvidenceEvidenceEvidenceEvidenceEvidenceEvidence
Performance accountabilityQuestion 1Question 1Question 1Question 1Question 1Question 1
Question 2Question 2Question 2Question 2Question 2Question 2
EvidenceEvidenceEvidenceEvidenceEvidenceEvidence

Source(s): Authors’ work

VAL-ED 360 results by groups and totals of the intersection of each main component and key processes

Cp*PC**D, S and P scoresTotal mean D, S and PPerformance
Standards for student learning raisedPlanning1. Directors4.454.38Distinguished
2. Supervisors4.45
3. Teachers4.26
Implementing1. Directors4.504.34Distinguished
2. Supervisors4.45
3. Teachers4.08
Supporting1. Directors4.544.43Distinguished
2. Supervisors4.13
3. Teachers4.62
Advocating1. Directors4.184.19Distinguished
2. Supervisors4.36
3. Teachers4.04
Communicating1. Directors4.864.43Distinguished
2. Supervisors4.40
3. Teachers4.18
Monitoring1. Directors4.134.62Distinguished
2. Supervisors4.72
3. Teachers4.28
Rigorous curriculumPlanning1. Directors3.593.92Competent
2. Supervisors3.72
3. Teachers3.90
Implementing1. Directors4.454.03Distinguished
2. Supervisors4.40
3. Teachers4.10
Supporting1. Directors4.774.28Distinguished
2. Supervisors4.27
3. Teachers4.14
Advocating1. Directors4.404.41Distinguished
2. Supervisors4.45
3. Teachers4.02
Communicating1. Directors4.684.35Distinguished
2. Supervisors4.31
3. Teachers4.34
Monitoring1. Directors4.364.24Distinguished
2. Supervisors4.22
3. Teachers4.16
Quality instructionPlanning1. Directors4.404.17Distinguished
2. Supervisors3.95
3. Teachers4.22
Implementing1. Directors4.724.39Distinguished
2. Supervisors4.68
3. Teachers4.08
Supporting1. Directors4.684.59Distinguished
2. Supervisors4.72
3. Teachers4.32
Advocating1. Directors4.594.33Distinguished
2. Supervisors4.36
3. Teachers3.96
Communicating1. Directors4.274.32Distinguished
2. Supervisors4.13
3. Teachers4.24
Monitoring1. Directors4.683.94Competent
2. Supervisors3.81
3. Teachers3.80
Culture of learning and professional behaviorPlanning1. Directors4.864.41Distinguished
2. Supervisors4.50
3. Teachers4.06
Implementing1. Directors4.684.43Distinguished
2. Supervisors4.40
3. Teachers4.02
Supporting1. Directors4.544.59Distinguished
2. Supervisors4.77
3. Teachers4.34
Advocating1. Directors4.454.32Distinguished
2. Supervisors4.31
3. Teachers4.12
Communicating1. Directors4.454.46Distinguished
2. Supervisors4.45
3. Teachers4.48
Monitoring1. Directors4.134.09Distinguished
2. Supervisors3.72
3. Teachers4.10
Connections to external communityPlanning1. Directors4.273.80Competent
2. Supervisors3.27
3. Teachers4.00
Implementing1. Directors3.903.72Competent
2. Supervisors3.27
3. Teachers3.62
Supporting1. Directors3.953.52Basic
2. Supervisors2.81
3. Teachers3.84
Advocating1. Directors3.953.61Competent
2. Supervisors3.04
3. Teachers4.08
Communicating1. Directors4.453.89Competent
2. Supervisors3.63
3. Teachers3.82
Monitoring1. Directors4.503.80Competent
2. Supervisors3.13
3. Teachers3.94
Performance accountabilityPlanning1. Directors3.904.20Distinguished
2. Supervisors4.18
3. Teachers4.08
Implementing1. Directors4.544.25Distinguished
2. Supervisors4.13
3. Teachers4.28
Supporting1. Directors4.634.36Distinguished
2. Supervisors4.18
3. Teachers4.14
Advocating1. Directors4.504.22Distinguished
2. Supervisors4.04
3. Teachers4.10
Communicating1. Directors4.904.58Distinguished
2. Supervisors4.45
3. Teachers4.38
Monitoring1. Directors4.774.30Distinguished
2. Supervisors4.09
3. Teachers4.06
Total average score4.21Distinguished

Source(s): Authors’ work

Results total VAL-ED 360 scores on the intersection matrix of each major component and key processes

PlanningImplementingSupportingAdvocatingCommunicatingMonitoring
Standards for student learning raisedDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.384.344.434.194.434.62
Rigorous curriculumCompetentDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
3.924.034.284.414.354.24
Quality instructionDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedCompetent
4.174.394.594.334.323.94
Culture of learning and professional behaviorDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.414.434.594.324.464.09
Connections to external communityCompetentCompetentBasicCompetentCompetentCompetent
3.803.723.523.613.893.80
Performance accountabilityDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.204.254.364.224.584.30

Source(s): Authors’ work

Results VAL-ED 360 scores of directors in the intersection matrix of each main component and key processes

PlanningImplementingSupportingAdvocatingCommunicatingMonitoring
Standards for student learning raisedDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.454.504.544.184.834.13
Rigorous curriculumCompetentDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
3.594.454.774.404.684.36
Quality instructionDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.404.724.684.594.274.68
Culture of learning and professional behaviorDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.864.684.544.454.454.13
Connections to external communityDistinguishedCompetentCompetentCompetentDistinguishedDistinguished
4.273.903.953.954.454.50
Performance accountabilityCompetentDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
3.904.544.634.504.904.77

Source(s): Authors’ work

Results VAL-ED 360 scores of supervisors in the intersection matrix of each main component and key processes

PlanningImplementingSupportingAdvocatingCommunicatingMonitoring
Standards for student learning raisedDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.454.454.134.364.404.72
Rigorous curriculumCompetentDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
3.724.404.274.404.314.22
Quality instructionCompetentDistinguishedDistinguishedDistinguishedDistinguishedCompetent
3.954.604.724.364.133.81
Culture of learning and professional behaviorDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedCompetent
4.504.404.774.314.453.72
Connections to external communityLower than basicLower than basicLower than basicLower than basicCompetentLower than basic
3.273.272.813.043.633.13
Performance accountabilityDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.184.134.184.044.454.09

Source(s): Authors’ work

Results scores VAL-ED 360 of the teachers in the intersection matrix of each main component and the key processes

PlanningImplementingSupportingAdvocatingCommunicatingMonitoring
Standards for student learning raisedDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.264.084.624.044.184.28
Rigorous curriculumCompetentDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
3.904.104.144.024.344.16
Quality instructionDistinguishedDistinguishedDistinguishedCompetentDistinguishedCompetent
4.224.084.323.964.243.80
Culture of learning and professional behaviorDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.064.024.344.124.484.10
Connections to external communityDistinguishedCompetentCompetentDistinguishedCompetentCompetent
4.003.623.844.083.823.94
Performance accountabilityDistinguishedDistinguishedDistinguishedDistinguishedDistinguishedDistinguished
4.084.284.144.104.384.06

Source(s): Authors’ work

Results VAL-ED 360 scores in the performance of the principal components

Main componentsD, S and P scoresTotal mean D, S and PPerformance
Standards for student learning raisedDirectors4.444.39Distinguished
Supervisors4.41
Teachers4.24
Rigorous curriculumDirectors4.374.22Distinguished
Supervisors4.22
Teachers4.11
Quality instructionDirectors4.554.29Distinguished
Supervisors4.27
Teachers4.10
Culture of learning and professional behaviorDirectors4.514.38Distinguished
Supervisors4.35
Teachers4.18
Connections to external communityDirectors4.173.72Competent
Supervisors3.19
Teachers3.88
Performance accountabilityDirectors4.544.31Distinguished
Supervisors4.17
Teachers4.26

Source(s): Authors’ work

Results scores VAL-ED 360 in the performance of key processes

Key processesD, S and P scoresTotal mean D, S and PPerformance
PlanningDirectors4.244.11Distinguished
Supervisors4.01
Teachers4.08
ImplementingDirectors4.464.23Distinguished
Supervisors4.22
Teachers4.03
SupportingDirectors4.514.29Distinguished
Supervisors4.14
Teachers4.23
AdvocatingDirectors4.344.16Distinguished
Supervisors4.09
Teachers4.05
CommunicatingDirectors4.604.34Distinguished
Supervisors4.20
Teachers4.24
MonitoringDirectors4.424.13Distinguished
Supervisors3.94
Teachers4.05

Source(s): Authors’ work

The authors report there are no competing interests to declare.

References

Anderson, S. (2010), “Liderazgo Directivo: Claves para una mejor escuela”, Psicoperspectivas. Individuo y Sociedad, Vol. 9 No. 2, pp. 34-52.

Antonopoulou, H., Halkiopoulos, C., Barlou, O. and Beligiannis, G. (2021), “Digital Leader and transforational leadership in higher Education”, Proceedings of INTED2021 Conference, pp. 9616-9624, doi: 10.21125/inted.2021.2005.

Aurangzeb, W. and Mazhar, U. (2020), “Analysis of E-leadership practices in ameliorating learning environment of higher education institutions”, Pakistan Journal of Distance and Online Learning, Vol. 5 No. 2, pp. 1-16.

Avolio, B.J. and Kahai, S. (2003), “Adding the ‘e’ to e-leadership: how it may impact your leadership”, Organizational Dynamics, Vol. 31 No. 4, pp. 325-338, doi: 10.1016/S0090-2616(02)00133-X.

Chang, C.L., Arisanti, I., Octoyuda, E. and Insan, I. (2022), “E-leadership analysis during pandemic outbreak to enhanced learning in higher education”, TEM Journal, Vol. 11 No. 2, pp. 932-938.

Cordie, L. and Lin, X. (2018), “The e‐revolution in higher education: e‐learning and e‐leaders”, Journal of Leadership Studies, Vol. 12 No. 3, pp. 76-78.

Goff, P., Salisbury, J. and Blitz, M. (2015), “Comparing CALL and VAL-ED: an illustrative application of a decision matrix for selecting among leadership feedback instruments”, WCER Working Paper No. 2015-5, Wisconsin Center for Education Research.

Goldring, E., Cravens, X., Porter, A., Murphy, J. and Elliott, S. (2015), “The convergent and divergent validity of the vanderbilt assessment of leadership in education (VAL-ED): instructional leadership and emotional intelligence”, Journal of Educational Administration, Vol. 53 No. 2, pp. 177-196, doi: 10.1108/JEA-06-2013-0067.

Hallinger, P. and Heck, R.H. (1996), “Reassessing the principal's role in school effectiveness: a review of empirical research, 1980-1995”, Educational Administration Quarterly, Vol. 32 No. 1, pp. 5-44.

Harris, A., Jones, M. and Ismail, N. (2022), “Distributed leadership: taking a retrospective and contemporary view of the evidence base”, School Leadership and Management, pp. 1-19, doi: 10.1080/13632434.2022.2109620.

Ibarra, M.S., Rodríguez, G., Lukas-Mujika, J.F. and Santos-Berrondo, A. (2023), “Medios e instrumentos para evaluar los resultados de aprendizaje en másteres universitarios. Análisis de la percepción del profesorado sobre su práctica evaluativa”, Educación XXI, Vol. 26 No. 1, pp. 21-45, doi: 10.5944/educxx1.33443.

Leithwood, K., Seashore-Louis, K., Anderson, S. and Wahlstrom, K. (2004), How Leadership Influences Student Learning, Minneapolis: Center for Applied Research and Educational Improvement, Toronto, Ontario Institute for Studies in Education, available at: http://www.wallacefoundation.org/SiteCollectionDocuments/WF/Knowledge%20Center/Attachments/PDF/ReviewofResearch-LearningFromLeadership.pdf

Marichal-Guevara, O.C., Rodríguez-García, A.M. and Cáceres-Reche, M.P. (2018), “Impact of the focus group on managerial leadership and its inclusive ‘praxis’ in the Cuban educational context”, Ma, J. and León, Y.T. (Eds), Book of Proceedings of the XV International Congress and XXXV Conference on Universities and Inclusive Education, Granada, pp. 177-186.

Miller, G.E. and Ives, K.S. (2020), Leading the eLearning Transformation of Higher Education: Leadership Strategies for the Next Generation, Stylus, New York.

Minor, E.C., Porter, A.C., Murphy, J., Goldring, E.B., Cravens, X., Stephen, N. and Elloitt, S.N. (2014), “A known group analysis validity study of the Vanderbilt Assessment of Leadership in Education in US elementary and secondary schools”, Educational Assessment, Evaluation and Accountability, Vol. 26 No. 1, pp. 29-48, doi: 10.1007/s11092-013-9180-z.

Minor, E.C., Porter, A.C., Murphy, J., Goldring, E.B. and Elloitt, S.N. (2017), “A test-retest analysis of the Vanderbilt assessment for leadership in education in the USA”, Educational Assessment, Evaluation and Accountability, Vol. 29, pp. 211-224, doi: 10.1007/s11092-016-9254-9.

Murphy, J. and Meyers, C. (2008), Turning Around Failing Schools: Leadership Lessons from the Organizational Sciences, Corwin Press, National Staff Development Council and American Association of School Administrators, CA.

Palomino, J.M., Cáceres, Ma. P. and Ramos, M. (2021), “E-leadership and distance learning in Higher Education. Main keys”, in Marin-Marin, J.A., De la Cruz Campos, J., Pozo-Sanchez, S. and Gómez-García, G. (Eds), Educational Research and Innovation in the Face of the Challenges of Sustainable Development, Dykinson, pp. 67-77.

Palomino, J.M., Cáceres, M.P., Pérez, L. and Ramos, M. (2022a), “Incidence of pedagogical leadership in the students' performance in higher education”, The International Journal of Educational Organization and Leadership, Vol. 29 No. 2, pp. 103-116, doi: 10.18848/2329-1656/CGP/v29i02/103-116.

Palomino, J.M., Cáceres, Ma. P., Hinojo, F.J. and Aznar, I. (2022b), “Towards a pedagogical leadership in Higher Education”, in Hinojo, F.J., Marichal, C.A.A., Cáceres, Ma. P. and Barrientos, O. (Eds), Research Contributions Derived from the Ibero-American Network for Research in Leadership and Educational Practices, Dykinson, pp. 65-78.

Palomino, J.M., Cáceres, M.P. and Lara, F. (2023), “Evaluation of pedagogical leadership through the Vanderbilt assessment of leadership in education (VAL-ED). Adaptation to the context of higher education in Spain”, Cogent Social Sciences (OASS), Vol. 9 No. 2, pp. 1-17, doi: 10.1080/23311886.2023.2243720.

Polikoff, M., May, H., Porter, A., Elliot, S., Goldring, E. and Murphy, J. (2009), “An examination of differential item functioning on the Vanderbilt assessment of leadership in education”, Journal of School Leadership, Vol. 19 No. 6, pp. 661-669, doi: 10.1177/105268460901900602.

Porter, A.C., Murphy, J., Goldring, E., Elliot, S.N., Polikoff, M.S. and May, H. (2011), Vanderbilt Assessment of Leadership in Education: Technical Manual, Discovery Education Assessment.

Porter, A.C., Murphy, J., Goldring, E.B., Elliott, S.N., Polikoff, M.S. and May, H. (2008), Vanderbilt Assessment of Leadership in Education (VAL-ED), [Database record], APA PsycTests, Nashville.

Smith, M.L. (2020), “Transformational leadership in higher education in Panama”, Latitude, Vol. 2 No. 13, pp. 38-75, doi: 10.55946/LATITUDE.V2I13.96.

Tintoré, M. and Gairín, J. (2022), “Tres décadas de investigación sobre liderazgo educativo en España. Un mapeo sistemático”, REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, Vol. 20 No. 3, pp. 5-24, doi: 10.15366/reice2022.20.3.001.

UNIR (2022), “Misión y Visión”, available at: https://www.unir.net/universidad-online/

Vanderbilt University (2011a), “VAL-ED handbook”, Vanderbilt Assessment of Leadership in Education, Discovery Education Assessment.

Vanderbilt University (2011b), “VAL-ED framework”, Vanderbilt Assessment of Leadership in Education, Discovery Education Assessment.

Vanderbilt University (2011c), “VAL-ED performance rubric”, Vanderbilt Assessment of Leadership in Education, Discovery Education Assessment.

Acknowledgements

This study comes from a broader research, derived from a doctoral thesis work titled: “Analysis of pedagogical e-Leadership in Distance University Education. Implications for educational improvement”. This research was sponsored by the project “eLeadership in Training Centres and implementation of a management model for quality and efficiency improvement”. Reference contract/agreement nº: 4683. Publicly funded through competitive bidding. He also coordinates the Master's degree in Teaching English as a Foreign Language as well as in the area of bilingualism at the International University of La Rioja.

Corresponding author

Blanca Berral Ortiz is the corresponding author and can be contacted at: blancaberral@ugr.es

About the authors

José Manuel Palomino Fernández is PhD student at the University of Granada. Faculty of Education. Line of research includes curriculum, organization and training for equity in the knowledge society. Thesis is “Analysis of pedagogical e-leadership in Distance University Education.”

María Pilar Cáceres Reche is a professor in the Department of Didactics and School Organisation at the University of Granada (Spain) and director of the Research Group “Leadership, Development and Educational Research” (LEADER Group, SEJ-604). Her research interests focus on teaching innovation and ICT, leadership and organizational development.

Fernando Lara Lara is a professor in the Department of Didactics and School Organisation at the University of Granada (Spain), and is a research group member of “Analysis of the Reality of Education” (HUM672).

Blanca Berral Ortiz is a child school teacher. She has a predoctoral contract, linked to the Department of Didactics and School Organization (DOE) of the University of Granada (UGR). She is a doctoral student in educational sciences, and her research area is focused on migrant students, literacy skills and technological resources.

Related articles