Search results

1 – 10 of over 1000
Click here to view access options
Article
Publication date: 1 February 2016

Avraham Levi

– The purpose of this paper is to explain why ROC analysis is an inappropriate replacement for probative analysis in lineup research.

Abstract

Purpose

The purpose of this paper is to explain why ROC analysis is an inappropriate replacement for probative analysis in lineup research.

Design/methodology/approach

Taking as the medical example comparing two methods to detect the presence of a malignant tumor (Mickes et al., 2012), and operationally defining ROC analysis: radiologists are shown the results from two methods. Their confidence judgments create a graph of correct identifications by mistaken ones. The author can compare the methods on radiologists’ ability to differentiate sick from healthy. Lineup researchers create two distinct lineups. In target-present lineups, witnesses differentiate between the target and the foils, not the target and the innocent suspect. In target-absent lineups, witnesses cannot even differentiate between innocent suspects and foils, having seen none.

Findings

Eyewitness ROC curves are similar to probative analysis, but provide less useful information.

Research limitations/implications

Researchers ware warned against using ROC when conducting lineup research.

Originality/value

Preventing inappropriate use of ROC analysis.

Details

Journal of Criminal Psychology, vol. 6 no. 1
Type: Research Article
ISSN: 2009-3829

Keywords

Click here to view access options
Article
Publication date: 1 July 2003

Ana Cristina Braga and Pedro Oliveira

Receiver operating characteristic (ROC) analysis is a powerful tool to measure and specify the problems with diagnostic performance in medicine. Describes this analysis

Downloads
1818

Abstract

Receiver operating characteristic (ROC) analysis is a powerful tool to measure and specify the problems with diagnostic performance in medicine. Describes this analysis and the importance of the index area under ROC curve, using some examples to demonstrate its application. The study was conducted on two sets of new‐borns with very low birth weight, coming from neonatal intensive care units from Portuguese hospitals. The first application uses correlated samples, and aims to define which of the five indices of clinical seriousness can be considered the best to evaluate the risk of death for babies with very low birth weight. In the second application, regarding independent samples, compares four Portuguese hospitals, the aim being to identify the neonatal intensive care unit which presents the best performance in terms of care to the new‐borns, i.e. evaluated through the comparison of the clinical severity indices.

Details

International Journal of Health Care Quality Assurance, vol. 16 no. 4
Type: Research Article
ISSN: 0952-6862

Keywords

Click here to view access options
Article
Publication date: 13 November 2007

Stylianos Z. Xanthopoulos and Christos T. Nakas

The purpose of this article is to introduce Receiver Operating Characteristic (ROC) surfaces and hyper‐surfaces within a banking context as natural generalizations of the…

Downloads
1182

Abstract

Purpose

The purpose of this article is to introduce Receiver Operating Characteristic (ROC) surfaces and hyper‐surfaces within a banking context as natural generalizations of the ROC curve.

Design/methodology/approach

Nonparametric ROC analysis using U‐statistics theory was used.

Findings

Application of the proposed methodology on data from a small size Greek bank illustrates the usefulness of ROC analysis for scoring systems assessment. The area under the ROC curve and the volume under the ROC surface and hyper‐surface are useful diagnostic indices for the assessment of credit rating systems and scorecards. The notion of statistical significance is not adequate for the evaluation of the loan granting strategy of a financial institution.

Originality/value

This article will be of value to financial institutions during the process of evaluation/validation of rating models.

Details

The Journal of Risk Finance, vol. 8 no. 5
Type: Research Article
ISSN: 1526-5943

Keywords

Open Access
Article
Publication date: 1 June 2017

Maher M. Alarfaj, Charles Secolsky and Fahad S. Alshaya

This study sheds light on the prediction of success using cutoff scores for student grades adopted for a required Physics pathway course for study in a health professions…

Abstract

This study sheds light on the prediction of success using cutoff scores for student grades adopted for a required Physics pathway course for study in a health professions program at King Saud University in Saudi Arabia. Data on course grade and GPA for approximately 10,000 students enrolled in this course between 2008–2014, were analyzed. Receiver Operating Characteristic (ROC) curve analysis was used to determine cutoffs for course grades using ranges of GPA. This procedure has promise as a new method for quantitatively arriving at cutoff scores using an external criterion requiring less human judgment than most existing standard setting methods. The cutoff scores produced show that GPAs of students who complete the Physics course yield successive performance tiers that are lower than expected. In addition, the correlation between GPA and course grade for Physics is only 0.63 and therefore only 39% of the variation in GPA explains course grade. As a result of the findings of the study, the decision was made to maintain the existing standards thereby requiring higher grades in the Physics course for students seeking to enter a health professions course of study.

ﻧﺗﻟا ﺎﮭﺗردﻗو ﺔﯾدﺣﻟا تﺎﺟردﻟا ﻰﻠﻋ ءوﺿﻟا ﺔﯾﻟﺎﺣﻟا ﺔﺳاردﻟا طﻠﺳﺗ ﻲﻓ ﺔﺑﻠطﻟا حﺎﺟﻧ ﻰﻠﻋ ﺔﯾؤﺑﻲﻟوﻻا ءﺎﯾزﯾﻔﻟا ررﻘﻣ زﯾﻓ)145( ، دﻌﯾ يذﻟاوﻠﻋ ﺎﯾﺳﺎﺳا ﺎﺑﻠطﺗﻣكﻠﻣﻟا ﺔﻌﻣﺎﺟ ﻲﻓ ﺔﯾﺣﺻﻟا تﺎﺻﺻﺧﺗﻟا ﺔﺑﻠط ﻰ ﺔﻘﻠﻌﺗﻣﻟا تﺎﻧﺎﯾﺑﻟا ﻊﻣﺟ مﺗ دﻘﻓ ﺔﯾﻠﻋو ،ﺔﯾدوﻌﺳﻟا ﺔﯾﺑرﻌﻟا ﺔﻛﻠﻣﻣﻟﺎﺑ دوﻌﺳ نﻣ برﺎﻘﯾ ﺎﻣﻟ ررﻘﻣﻟا اذھ تﺎﺟردﺑ10000 ماوﻋﻻا نﯾﺑ ررﻘﻣﻟا اذﮭﺑ اوﻘﺣﺗﻟا نﯾذﻠﻟا ﺔﺑﻠطﻟا نﻣ2008 - 2014 .ﺔﯾﻣﻛارﺗﻟا مﮭﺗﻻدﻌﻣو ،م ﺗﻟو مادﺧﺗﺳا مﺗ دﻘﻓ ،تﺎﻧﺎﯾﺑﻟا هذھ لﯾﻠﺣ تﺎﯾﻠﻣﻌﻟا لﯾﻐﺷﺗ ﺔﯾﺻﺎﺧ ﻰﻧﺣﻧﻣReceiver Operating Characteristic (ROC) تﺎﺟردﻟا دﯾدﺣﺗﻟ نﻣ دﺣﻟاو ﺔﯾدﺣﻟا تﺎﺟردﻟا ﻰﻟا لوﺻوﻠﻟ ﺔﺛﯾدﺣﻟا ﺔﯾﻣﻛﻟا قرطﻟا نﻣ ﺔﻘﯾرطﻟا هذھ دﻌﺗ ثﯾﺣ ،ﺔﯾﻣﻛارﺗﻟا تﻻدﻌﻣﻟا نﻣ ﺔﻔﻠﺗﺧﻣ تﺎﻗﺎطﻧﻟ ﺔﯾدﺣﻟارﯾﺛﺄﺗﻟا .يرﺷﺑﻟا نﻣﺿ نﺎﻛ ررﻘﻣﻟا اذھ زﺎﺗﺟا نﻣﻟ ﺔﯾﻣﻛارﺗﻟا تﻻدﻌﻣﻟا نا ﻰﻟا ترﺎﺷا دﻗ ﺎﮭﯾﻠﻋ لوﺻﺣﻟا مﺗ ﻲﺗﻟا ﮫﯾدﺣﻟا تﺎﺟردﻟا نﺎﻓ ﺔﯾﻠﻋو تﻐﻠﺑ ﺔﺑﻠطﻟا تﺎﺟردو ﺔﯾﻣﻛارﺗﻟا تﻻدﻌﻣﻟا نﯾﺑ ﺔﯾطﺎﺑﺗرﻻا ﺔﻗﻼﻌﻟا نا ﺎﻣﻛ ،ﻊﻗوﺗﻣﻟا نﻣ لﻗا تﺎﻗﺎطﻧ0.63 ﻲﻧﻌﯾ ﺎﻣﻣ ، نا 39% نﻣتﺎﻧﯾﺎﺑﺗﻟا ﺔﯾﻣھا نﯾﺑﺗﯾ ﺞﺋﺎﺗﻧ نﻣ ﺔﯾﻠﻋ لوﺻﺣﻟا مﺗ ﺎﻣﻟ ﺎﻘﻓوو .رﻘﻣﻟا كﻟذ ﻲﻓ مﮭﺗﺎﺟرد رﯾﺳﻔﺗ ﻲﻓ مﮭﺳﺗ نا نﻛﻣﯾ ﺔﺑﻠطﻠﻟ ﺔﯾﻣﻛارﺗﻟا تﻻدﻌﻣﻟا ﻲﻓﺔظﻓﺎﺣﻣﻟا ﻰﻠﻋ ررﻘﻣﻟا رﯾﯾﺎﻌﻣﺔﯾﻟﺎﺣﻟا ﻊﻣدﯾﻛﺄﺗ لوﺻﺣﺔﺑﻠطﻟا تﺎﺻﺻﺧﺗﻟﺎﺑ قﺎﺣﺗﻟﻼﻟ نﯾﺑﻏارﻟاﺔﯾﺣﺻﻟا تﺎﺟرد ﻰﻠﻋﺔﻌﻔﺗرﻣ .ءﺎﯾزﯾﻔﻟا ررﻘﻣ ﻲﻓ

Details

Learning and Teaching in Higher Education: Gulf Perspectives, vol. 14 no. 1
Type: Research Article
ISSN: 2077-5504

Click here to view access options
Article
Publication date: 5 June 2017

Stan De Spiegelaere, Monique Ramioul and Guy Van Gyes

The purpose of this paper is to identify different job types in the Belgian electricity sector and their relations with employee outcomes such as work engagement and…

Abstract

Purpose

The purpose of this paper is to identify different job types in the Belgian electricity sector and their relations with employee outcomes such as work engagement and innovative work behaviour (IWB).

Design/methodology/approach

This paper uses a combination of latent profile analysis and relative operating characteristics (ROC) analysis.

Findings

Depending on the job resources and demands, five different job types are identified corresponding largely to the Karasek and Theorell (1990) job types. Their relation with the outcomes is not parallel with low-strain jobs performing best for work engagement, and active jobs for IWB.

Research limitations/implications

The combination of methods used in this study increases significantly the ease of communication of the findings, yet an external benchmark for the ROC analysis would be preferable.

Practical implications

To foster engagement and IWB with employees one should focus on the job content and only increase demands if they are combined with sufficient resources.

Originality/value

This research is the first in its kind that relates latent job types with different employee outcomes using a combination of latent profile and ROC analysis.

Details

Employee Relations, vol. 39 no. 4
Type: Research Article
ISSN: 0142-5455

Keywords

Click here to view access options
Article
Publication date: 1 February 1989

Majid Jaraiedi and Wafik H. Iskander

Signal Detection Theory (SDT) has recently been used to evaluate the performance of imperfect inspectors. SDT model is based on a priori probabilities and perceived…

Abstract

Signal Detection Theory (SDT) has recently been used to evaluate the performance of imperfect inspectors. SDT model is based on a priori probabilities and perceived payoffs and penalties to study inspectors′ behaviour. In this article, Bayes′ theorem is used to compute posterior probabilities of the two types of inspection error. These posterior probabilities give rise to the definition of Receiver Analysis Curves (RAC), which depict the “after the facts” consequences of inspection error. A cost model is also developed that reflects the true benefits and costs of inspection accuracy to the organisation.

Details

International Journal of Quality & Reliability Management, vol. 6 no. 2
Type: Research Article
ISSN: 0265-671X

Keywords

Click here to view access options
Article
Publication date: 1 February 2013

Mehmet Tolga Taner, Bulent Sezen and Kamal Atwat

This paper aims to compare two diagnostic performance measures, i.e. signal‐to‐noise ratio (S/N ratio) and partial area under receiver operating characteristic curves…

Abstract

Purpose

This paper aims to compare two diagnostic performance measures, i.e. signal‐to‐noise ratio (S/N ratio) and partial area under receiver operating characteristic curves (pAUC). It proposes the use of S/N ratio rather than pAUC for establishing optimal cut‐off point for diagnostic biomarkers.

Design/methodology/approach

This paper discusses the properties, uses, advantages and shortcomings of the two performance measures, namely the partial area under receiver operating characteristic curve (pAUC) and Taguchi's signal‐to‐noise (S/N) ratio. The benefits of S/N ratio have been illustrated in a sample of four biomarkers, each having five cut‐off points. The S/N ratio is compared to the pAUC index. The SAS software is employed to calculate pAUC and AUC.

Findings

This paper shows that S/N ratio can be used as a measure of diagnostic accuracy. The cut‐off point with the highest S/N ratio is the optimal cut‐off point for the biomarker. The proposed method has the advantages of being easier, more practical and less costly than that of pAUC.

Practical implications

This paper includes implications for the development of a more practical, equally powerful and less costly means of measuring clinical accuracy thereby reducing the costs and risks resulting from wrong selection of cut‐off point can be decreased.

Originality/value

This paper supports suggestions in the recent literature to replace pAUC with a new, more meaningful index.

Details

International Journal of Health Care Quality Assurance, vol. 26 no. 2
Type: Research Article
ISSN: 0952-6862

Keywords

Click here to view access options
Article
Publication date: 9 May 2008

Geng Cui, Man Leung Wong, Guichang Zhang and Lin Li

The purpose of this paper is to assess the performance of competing methods and model selection, which are non‐trivial issues given the financial implications. Researchers…

Downloads
2583

Abstract

Purpose

The purpose of this paper is to assess the performance of competing methods and model selection, which are non‐trivial issues given the financial implications. Researchers have adopted various methods including statistical models and machine learning methods such as neural networks to assist decision making in direct marketing. However, due to the different performance criteria and validation techniques currently in practice, comparing different methods is often not straightforward.

Design/methodology/approach

This study compares the performance of neural networks with that of classification and regression tree, latent class models and logistic regression using three criteria – simple error rate, area under the receiver operating characteristic curve (AUROC), and cumulative lift – and two validation methods, i.e. bootstrap and stratified k‐fold cross‐validation. Systematic experiments are conducted to compare their performance.

Findings

The results suggest that these methods vary in performance across different criteria and validation methods. Overall, neural networks outperform the others in AUROC value and cumulative lifts, and the stratified ten‐fold cross‐validation produces more accurate results than bootstrap validation.

Practical implications

To select predictive models to support direct marketing decisions, researchers need to adopt appropriate performance criteria and validation procedures.

Originality/value

The study addresses the key issues in model selection, i.e. performance criteria and validation methods, and conducts systematic analyses to generate the findings and practical implications.

Details

Marketing Intelligence & Planning, vol. 26 no. 3
Type: Research Article
ISSN: 0263-4503

Keywords

Click here to view access options
Article
Publication date: 13 September 2019

Guru Prasad Bhandari, Ratneshwer Gupta and Satyanshu Kumar Upadhyay

Software fault prediction is an important concept that can be applied at an early stage of the software life cycle. Effective prediction of faults may improve the…

Abstract

Purpose

Software fault prediction is an important concept that can be applied at an early stage of the software life cycle. Effective prediction of faults may improve the reliability and testability of software systems. As service-oriented architecture (SOA)-based systems become more and more complex, the interaction between participating services increases frequently. The component services may generate enormous reports and fault information. Although considerable research has stressed on developing fault-proneness prediction models in service-oriented systems (SOS) using machine learning (ML) techniques, there has been little work on assessing how effective the source code metrics are for fault prediction. The paper aims to discuss this issue.

Design/methodology/approach

In this paper, the authors have proposed a fault prediction framework to investigate fault prediction in SOS using metrics of web services. The effectiveness of the model has been explored by applying six ML techniques, namely, Naïve Bayes, Artificial Networks (ANN), Adaptive Boosting (AdaBoost), decision tree, Random Forests and Support Vector Machine (SVM), along with five feature selection techniques to extract the essential metrics. The authors have explored accuracy, precision, recall, f-measure and receiver operating characteristic curves of the area under curve values as performance measures.

Findings

The experimental results show that the proposed system can classify the fault-proneness of web services, whether the service is faulty or non-faulty, as a binary-valued output automatically and effectively.

Research limitations/implications

One possible threat to internal validity in the study is the unknown effects of undiscovered faults. Specifically, the authors have injected possible faults into the classes using Java C3.0 tool and only fixed faults are injected into the classes. However, considering the Java C3.0 community of development, testing and use, the authors can generalize that the undiscovered faults should be few and have less impact on the results presented in this study, and that the results may be limited to the investigated complexity metrics and the used ML techniques.

Originality/value

In the literature, only few studies have been observed to directly concentrate on metrics-based fault-proneness prediction of SOS using ML techniques. However, most of the contributions are regarding the fault prediction of the general systems rather than SOS. A majority of them have considered reliability, changeability, maintainability using a logging/history-based approach and mathematical modeling rather than fault prediction in SOS using metrics. Thus, the authors have extended the above contributions further by applying supervised ML techniques over web services metrics and measured their capability by employing fault injection methods.

Details

Data Technologies and Applications, vol. 53 no. 4
Type: Research Article
ISSN: 2514-9288

Keywords

Click here to view access options
Article
Publication date: 4 April 2016

Tsui-Hua Huang, Yungho Leu and Wen-Tsao Pan

In order to avoid enterprise crisis and cause the domino effect, which influences the investment return of investors, the national economy, and financial crisis…

Abstract

Purpose

In order to avoid enterprise crisis and cause the domino effect, which influences the investment return of investors, the national economy, and financial crisis, establishing a complete set of feasible financial early warning model can help to prevent the possibility of enterprise crisis in advance, and thus, reduce the influence on society and the economy. The purpose of this paper is to develop an efficient financial crisis warning model.

Design/methodology/approach

First, the fruit fly optimization algorithm (FOA) is used to adjust the coefficients of the parameters in the ZSCORE model (we call it the FOA_ZSCORE model), and the difference between the forecasted value and the real target value is calculated. Afterward, the generalized regressive neural network (GRNN model), with optimized spread by FOA (we call it FOA_GRNN model), is used to forecast the difference to promote the forecasting accuracy. Various models, including ZSCORE, FOA_ZSCORE, FOA_ZSCORE+GRNN, and FOA_ZSCORE+FOA_GRNN, are trained and tested. Finally, different models are compared based on their prediction accuracies and ROC curves. Furthermore, more appropriate parameters, which are different from the parameters in the original ZSCORE model, are selected by using the multivariate adaptive regression splines (MARS) method.

Findings

The hybrid model of the FOA_ZSCORE together with the FOA_GRNN offers the highest prediction accuracy, compared to other models; the MARS can be used to select more appropriate parameters to further improve the performance of the prediction models.

Originality/value

This paper proposes a hybrid model, FOA_ZSCORE+FOA_GRNN which offers better performance than the original ZSCORE model.

1 – 10 of over 1000