Search results

This article presents an investigation of the suitability of using quantitative or qualitative data for individual competencies assessment. Specifically, the primary purpose of this article is to identify if the results provided by quantitative and qualitative instruments focused on individual competencies are convergent.

In order to do the investigation proposed, a survey on individual competencies comprising a group of employees of the administrative area of a steel company was carried out. A total of 268 evaluations were collected and analyzed.

The analysis of the employee's performance appraisals provided by ratings and narrative comments indicates a low correlation between these assessments. The reasons for such low correlation include the qualitative assessments variability, the restricted list of competencies used in the quantitative assessments and the analytical format of quantitative assessments.

The study indicates that quantitative and qualitative assessments should be carried out jointly so that they can generate more comprehensive results. When the combined use is not possible, the quantitative approach is better suited for assessing performance, while the qualitative approach provides more valuable insights for boosting people development processes.

A case study methodology was applied as a major component of a mixed-methods approach to the evaluation of a mobile dementia education and support service in the Bega Valley Shire, New South Wales, Australia. In-depth interviews with people with dementia (PWD), their carers, programme staff, family members and service providers and document analysis including analysis of client case notes and client database were used.

The strengths of the case study approach included: (i) simultaneous evaluation of programme process and worth, (ii) eliciting the theory of change and addressing the problem of attribution, (iii) demonstrating the impact of the programme on earlier steps identified along the causal pathway (iv) understanding the complexity of confounding factors, (v) eliciting the critical role of the social, cultural and political context, (vi) understanding the importance of influences contributing to differences in programme impact for different participants and (vii) providing insight into how programme participants experience the value of the programme including unintended benefits.

The broader case of the collective experience of dementia and as part of this experience, the impact of a mobile programme of support and education, in a predominately rural area grew from the investigation of the programme experience of ‘individual cases’ of carers and PWD. Investigation of living conditions, relationships, service interactions through observation and increased depth of interviews with service providers and family members would have provided valuable perspectives and thicker description of the case for increased understanding of the case and strength of the evaluation.

If the use of information technology (IT) supporting clinical trial projects offers opportunities to optimize the underlying information management process, the intricacy of the identification and evaluation of relevant IT options is generally seen as a complex task in healthcare. Hence, the purpose of this paper is to examine the problem of ex ante information system evaluation, and assess the impact of IT on the information management process underlying clinical trials.

Combining Unified Modeling Language (UML) and system dynamics modeling, a simulation model for evaluating IT was developed. This modeling effort relies on a case study conducted in a clinical research organization, which, at that time, faced an IT investment dilemma.

Some illustrative results of sensitivity analyzes conducted on error rates in clinical data transmission are presented. These simulation results allow for quantifying the impact of different IT options on human resources' efforts, time delays and costs of clinical trials projects. Notably, the results show that although the technology has no real influence on the duration of a clinical trial project, it impacts the number of projects that can be carried out simultaneously.

The research provides insights into the development of an innovative approach appropriate to the evaluation of IT supporting clinical trials, through the use of a mixed‐method based on qualitative and quantitative modeling. The results illustrate two critical issues addressed in the IS literature: the necessity to extend IT evaluation beyond the quantitative‐qualitative dichotomy; and the role of evaluation in organizational learning, and in learning about business dimensions.

The purpose of this paper is to conduct an in‐depth quantitative and qualitative evaluation of a family‐based weight loss and healthy life style programme for clinically obese children in England.

The mixed method case study evaluation used included obtaining pre and post measurements of anthropometry and a range of attitude and behavioural attributes. The qualitative phase of the study involved in‐depth interviews and focus groups.

The programme is demanding and resource intensive and designed as an intervention for children needing most help with their weight. Participants included the families of five referred children (n=18 individuals) and the intervention team (n=7). All but one child had reduced BMI centiles at the end of the programme. There were also improvements to a number of self‐report aspects of healthy eating and levels of activity and quality of life, self‐esteem and levels of depression indicators. The qualitative evaluation generated a number of insightful data themes.

Limitations include the case study design and small sample numbers. Also weight loss is an important indicator of any weight management programme's success however the short length of time programme was run is a barrier to seeing any substantive changes in any of the participating children's weights.

The evaluation conducted gives insights into the positive aspects of the programme and can inform development of similar programmes.

There are few examples of in‐depth and comprehensive quantitative and qualitative approaches used to evaluate this type of intervention.

As cyber-attacks continue to grow, organisations adopting the internet-of-things (IoT) have continued to react to security concerns that threaten their businesses within the current highly competitive environment. Many recorded industrial cyber-attacks have successfully beaten technical security solutions by exploiting human-factor vulnerabilities related to security knowledge and skills and manipulating human elements into inadvertently conveying access to critical industrial assets. Knowledge and skill capabilities contribute to human analytical proficiencies for enhanced cybersecurity readiness. Thus, a human-factored security endeavour is required to investigate the capabilities of the human constituents (workforce) to appropriately recognise and respond to cyber intrusion events within the industrial control system (ICS) environment.

A quantitative approach (statistical analysis) is adopted to provide an approach to quantify the potential cybersecurity capability aptitudes of industrial human actors, identify the least security-capable workforce in the operational domain with the greatest susceptibility likelihood to cyber-attacks (i.e. weakest link) and guide the enhancement of security assurance. To support these objectives, a Human-factored Cyber Security Capability Evaluation approach is presented using conceptual analysis techniques.

Using a test scenario, the approach demonstrates the capacity to proffer an efficient evaluation of workforce security knowledge and skills capabilities and the identification of weakest link in the workforce.

The approach can enable organisations to gain better workforce security perspectives like security-consciousness, alertness and response aptitudes, thus guiding organisations into adopting strategic means of appropriating security remediation outlines, scopes and resources without undue wastes or redundancies.

This paper demonstrates originality by providing a framework and computational approach for characterising and quantify human-factor security capabilities based on security knowledge and security skills. It also supports the identification of potential security weakest links amongst an evaluated industrial workforce (human agents), some key security susceptibility areas and relevant control interventions. The model and validation results demonstrate the application of action research. This paper demonstrates originality by illustrating how action research can be applied within socio-technical dimensions to solve recurrent and dynamic problems related to industrial environment cyber security improvement. It provides value by demonstrating how theoretical security knowledge (awareness) and practical security skills can help resolve cyber security response and control uncertainties within industrial organisations.

By using an integrated theoretical analysis model and a practice analysis model, this study constructed an evaluation indicator system for public participation in post-disaster housing reconstruction.

This study investigated and summarised the public participation of earthquake victims in house building in Dujiangyan. A total of 16 analysis elements were proposed, including the subject and object for evaluation. A mixed qualitative and quantitative evaluation model was established.

The implementation results indicated that this evaluation index system was objective oriented and had public policy attributes. The index can effectively reflect the diverse needs of disaster victims, address multiple social aspects and evaluate and rate public participation.

This paper proposed a mixed research method that integrates analysis processes, such as theories and practical investigation, which are the pillars of public participation evaluation. The evaluation indicators had quantitative characteristics.

This paper aims to propose a new qualitative indicator for the evaluation of the productions of researchers in any discipline.

Based on the study of existing quantitative indicators, the authors’ approach consisted of the hybridization of two indicators. This hybridization is based on the individual H_index (Hi_index) and H_index contemporary (Hc_index) weighted by qualitative factors. The initial sources of the data are online bibliographic databases, such as Google Scholar and Publish or Perish.

A new scientometric indicator was used to compare the scientific production quality of researchers and their classification (as part of a research community) as the classification of national and international research institutions. The authors have applied a new indicator to compare and classify the members of their laboratory, RIADI, according to their quality of scientific production.

The indicator is an improvement of the H_index. It is a measure that can have an impact on society (influencing research attitudes, affecting quality of research). By this contribution, the authors measure more than one aspect by involving all the external factors that can affect the quality of research.

This paper fulfils a gap in the literature concerning the absence of a qualitative indicator among the set of existing quantitative measures. Additionally, this paper addresses the limitations of the existing qualitative practices, such as peer review and citation analysis. In the new qualitative indicator, the authors involve all of these qualitative aspects: the influence of the age of the paper, the number of co-authors, the order of the co-authors, the impact factor of journals and the conference rankings.

The purpose of this article is to describe a proposition for the evaluation of citations of scientific papers, which could serve as a supplement to the existing Rule Book of the Ministry of the Republic of Serbia, which is used in the procedure of electing candidates for particular academic and research titles. The evaluation and quantitative presentation of the results and evaluation of citations were carried out on data taken from the database of the Current Research Information System of the University of Novi Sad (CRIS UNS), which is harmonized with the Rule Book of the Ministry with respect to the evaluation of published scientific results of researchers.

There are different criteria to evaluate the quality of scientific papers based on their citations. The pertinent parameters can be the total number of citations, the number of citations in a defined time period and by assigning the appropriate weighting values to the citations. This work proposes a procedure of assigning the citation weighting values based on the evaluation of the scientific results in which the citation appeared according to the Rule Book in the Republic of Serbia. Based on this, the authors introduced the impact factor of researchers as the ratio of the number of points of the evaluated citations and the number of points of the evaluated papers of the researcher.

Results showed that the research information system CRIS UNS can be extended to the evaluation of citations for a single researcher, groups of researchers and institutions.

The proposed solution enables the evaluation of citations in the process of election and promotion of academic staff. In this way, there is a means for measuring the scientific influence of a researcher in the relevant scientific area.

The evaluation of citations may be included in the national strategies of scientific development, funding and evaluation of research projects; for promotions of academic staff at the universities and other academic institutions; and ranking of researchers and research organizations.

The main idea presented in the paper is the definition of a rule book (or several rule books) for the evaluation of citations. Based on the evaluation of citations, the authors proposed the term “the impact factor of researcher”.

This paper aims to explore the possibilities in using such national, statutory data sets for evaluating change and the challenges of understanding service patterns and outcomes in complex cases when only a limited view can be gained using existing data. The discussion also explores how methodologies can adapt to an evaluation in these circumstances.

This paper examines the use of data routinely collected by local authorities (LAs) as part of the evaluation of innovation. Issues entailed are discussed and illustrated through two case studies of evaluations conducted by the research team within the context of children’s social care in England.

The quantitative analysis of LA data can play an important role in evaluating innovation but researchers will need to address challenges related to: selection of a suitable methodology; identifying appropriate comparator data; accessing data and assessing its quality; and sustaining and increasing the value of analytic work beyond the end of the research. Examples are provided of how the two case studies experienced and addressed these challenges.

• Quasi-experimental methods can be beneficial tools for understanding the impact of innovation in children’s services, but researchers should also consider the complexity of children’s social care and the use of mixed and appropriate methods. • Those funding innovative practice should consider the additional burden on those working with data and the related data infrastructure if wishing to document and analyse innovation in a robust way. • Data, which may be assumed to be uniform may in fact not be when considered at a multi-area or national level, and further study of the data recording practice of social care professionals is required.

The paper discusses some common issues experienced in quasi-experimental approaches to the quantitative evaluation of children’s services, which have, until recently, been rarely used in the sector. There are important considerations, which are of relevance to researchers, service leads in children’s social care, data and performance leads and funders of innovation.

The purpose of this paper is to investigate how Big Data can be used in project evaluations.

The study is based on literature research and interviews with 15 professionals in IT, project and asset management and government agencies. The authors discuss and illustrate what data that can be used for project evaluations and discuss potential obstacles.

New data is creating new opportunities to analyse a phenomenon based on different types of data. Interesting data categories include: internet traffic, movement-related data, physical environment data and data in organisational internal systems. The authors show how these data categories can be applied in project evaluations.

Big Data gives an opportunity to add quantitative data in ex post evaluations. Use of Big Data can serve as a step towards a stronger technology focus in evaluations of projects.

There are major advantages in using Big Data, increasing the opportunities to find indicators that are relevant when a project is evaluated.

Possible problematic issues related to use of Big Data that are addressed in the study include: availability, applicability, relevance, privacy policy, ownership, cost and competence. The study indicates that none of the challenges need to hinder use of Big Data when evaluating projects, provided that the issues are properly managed.

The study illustrates how Big Data can be applied in project management research.