Abstract
Purpose
This research aims to improve the performance of rail fastener defect inspection method for multi railways, to effectively ensure the safety of railway operation.
Design/methodology/approach
Firstly, a fastener region location method based on online learning strategy was proposed, which can locate fastener regions according to the prior knowledge of track image and template matching method. Online learning strategy is used to update the template library dynamically, so that the method not only can locate fastener regions in the track images of multi railways, but also can automatically collect and annotate fastener samples. Secondly, a fastener defect recognition method based on deep convolutional neural network was proposed. The structure of recognition network was designed according to the smaller size and the relatively single content of the fastener region. The data augmentation method based on the sample random sorting strategy is adopted to reduce the impact of the imbalance of sample size on recognition performance.
Findings
Test verification of the proposed method is conducted based on the rail fastener datasets of multi railways. Specifically, fastener location module has achieved an average detection rate of 99.36%, and fastener defect recognition module has achieved an average precision of 96.82%.
Originality/value
The proposed method can accurately locate fastener regions and identify fastener defect in the track images of different railways, which has high reliability and strong adaptability to multi railways.
Keywords
Citation
Liu, J., Huang, Y., Wang, S., Zhao, X., Zou, Q. and Zhang, X. (2022), "Rail fastener defect inspection method for multi railways based on machine vision", Railway Sciences, Vol. 1 No. 2, pp. 210-223. https://doi.org/10.1108/RS-04-2022-0012
Publisher
:Emerald Publishing Limited
Copyright © 2022, Junbo Liu, Yaping Huang, Shengchun Wang, Xinxin Zhao, Qi Zou and Xingyuan Zhang
License
Published in Railway Sciences. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
1. Introduction
With the rapid expansion of the railway network in China, how to effectively ensure the safety of railway operation has become a key issue in railway maintenance. As the important track components connecting the rail and the ballast bed, the fasteners are prone to damages caused by rail vibration, environmental temperature difference and so forth, which may result in rail displacement and even serious safety accidents such as train derailment. In recent years, the object inspection method based on machine vision technology has received extensive attention from academia and industry due to its high efficiency, reliability and low cost. Domestic and foreign scholars have proposed many inspection methods based on machine vision, which are used for rail detect inspection (Ren et al., 2011; Li & Ren, 2012a, b; Li, Zhang, Ren, Dai, & Li, 2016; Sun, Liu, Qin, & Zhang, 2018), railway bridge bolt inspection (Zhao, Qian, & Liu, 2018), OCS defect inspection (Zhou et al., 2015), rail plug inspection (Du et al., 2017) and rail fastener defect inspection (Dai et al., 2018).
The machine-vision-based rail fastener defect inspection systems obtain the track images through the high-speed line-scan camera, which are installed at the bottom of the inspection train. Then, the fastener regions are accurately located by the fastener region location module. Finally, the fastener defect recognition module is used to identify the fastener state. For the fastener region location, the existing methods are to detect the geometric features, such as the edges and corners of the rails, sleepers and fasteners (Li, Trinh, Haas, Otto, & Pankanti, 2013), and then use the distribution of track components to locate the fastener region. However, these primary image features have poor anti-interference ability. Moreover, most of the existing methods are designed for specific ballast bed types, which cannot be applied to the rail fastener inspection for multi railways, resulting in poor generalization.
For the fastener defect recognition, the existing methods can be divided into supervised learning and unsupervised learning. The supervised fastener recognition methods use wavelet transform (WT) (Mazzeo, Nitti, Stella, & Distante, 2004), independent component analysis (ICA) (Mazzeo, Ancona, Stella, & Distante, 2003) and other hand-designed features to represent the fastener region, and then use multilayer perception (MLP) (Marino, Distante, Mazzeo, & Stella, 2007), latent Dirichlet allocation (LDA) (Feng et al., 2013), AdaBoosting (Xia, Xie, & Jiang, 2010), Viola-Jones (Rubinsztejn, 2011), Bayesian compression sensing (BCS) (Liu, Xiong, Li, & Li, 2016) and other classification methods to identify fastener defects. The unsupervised fastener recognition methods use the histogram of oriented gradient (HOG) (Dou, Huang, Li, & Luo, 2014) or local binary pattern (LBP) (Fan, Cosman, Hou, & Li, 2018) to represent the fastener region, and then use the template matching method to identify fastener defects. However, in multi railways inspection, the supervised fastener recognition methods require to recollect and annotate a large number of fastener samples on each railway for train model, and the training process is very time-consuming and laborious. The unsupervised fastener recognition methods require no training, but the performance is slightly low.
In this paper, a fastener region location method based on online learning strategy is proposed, which not only can locate fastener regions, but also automatically collect and annotate a large number of fastener samples. Then, the annotated fastener samples are used to pre-train the fastener defect recognition network to learn the depth features of the fastener image. Finally, the pre-training model is applied to multi railways by using transfer learning. The experiment results show that the proposed method can meet the requirements of rail fastener inspection task in multi railways, which has great theoretical significance and practical value.
2. Fastener region location method based on online learning strategy
Fastener region location method refers to locate and extract the fastener region from the original track image, accurately. Most of the existing fastener region location methods are based on edge features and mutual information correlation. These methods first detect the edge features of rails and sleepers, and use their position relationship to determine fastener regions. However, in multi railways, the type of ballast beds and fasteners are not fixed, which will result in poor location performance. Figure 1 shows the type of ballast beds and fasteners in different railways.
Aiming at the problem that the existing fastener location methods are not applicable to multi railways inspection, a fastener region location method based on online learning strategy is proposed.
2.1 Fastener region location
The original track image contains 5 pieces of prior information: each track image contains only 1 rail; the rail is perpendicular to x-axis of the image; the rail width is a fixed value; the width of fastener region is fixed and distributed on both sides of the rail; each track image contains at least 6 fastener regions.
Based on the above prior information, the rail boundary location method (Huang, Luo, & Wang, 2012) based on line segment detector (LSD) is used to quickly locate the rail, which can reduce the scope of the candidate fastener areas. Then, the sub-windows are extracted in the candidate fastener areas using the sliding window method, and the similarity of HOG features between the fastener templates and the sub-windows are calculated. Finally, the sub-window with the highest similarity is selected as the optimal fastener region, and other fastener regions are inferred according to the distribution of track components and the symmetry of fastener region.
The track image is obtained by the inspection system (Xu, Shi, Ren, Han, & Wang, 2013). The size of each image is (800 × 1,230) pixels, the size of each fastener region is (90 × 100) pixels, the rail width is 60 pixels, the lateral spacing of the fastener regions is 55–65 pixels and the longitudinal spacing is 275–315 pixels. The schematic diagram of the fastener region location method is shown in Figure 2, where the red dashed line indicates the located rail boundary, the green dashed line area is the candidate fastener area, the green rectangle box indicates the optimal fastener region selected and the yellow rectangle box surrounded by the blue dashed line indicates the inferred fastener regions.
2.2 Online learning strategy
The online learning strategy is used to dynamically update the fastener template library. Specifically, the fastener template library is divided into online library and off-line library. The off-line library contains manually reinspected fastener templates, which will not be updated in the inspection process. During the inspection process, the online library will use the K-nearest neighbor (K-NN) method to calculate the similarity of the located fastener regions, and then dynamically update according to the update rules.
The online learning strategy is based on a priori knowledge: the track images of adjacent frames will not be abrupt changes in light conditions, trackside environment and fastener types. Therefore, the update rules for the online template library are as follows: add the fastener regions with the highest similarity on both sides of the rail to the tail of the normal fastener region queue in the online template library; randomly extract 2 background regions in the nonfastener regions on both sides of the rail and add them to the tail of the background region queue in the online template library; if the length of a queue in the online template library is greater than the preset threshold N, delete the template at the head of the queue.
According to the update rules, the templates which at the queue head are derived from far-spaced track images, and the templates newly inserted at the tail of the queue are derived from track images acquired over a recent time period. Therefore, the light conditions, trackside environment and fastener type of the queue head template may differ significantly from the current track image. When the template queue length exceeds the preset threshold N, the template at the head of the queue is deleted first. This strategy can ensure that the online template library can adapt to the lighting conditions, trackside environment and fastener type of the current track image, and improve the accuracy of the fastener region location method.
2.3 Automatic sample annotation
Currently, the supervised fastener defect recognition method requires a large number of training samples during the training process. However, it is very time-consuming and laborious to manually collect and annotate the fastener samples in actual applications. Therefore, the automatic sample annotation method is proposed, which can pre-classify the located fastener regions by online template library, and established a large fastener training data set to pretrain the fastener defect recognition network.
The detailed process of automatic sample annotation is shown in Figure 3. In the fastener region location stage, in order to improve the location efficiency, only the normal template and background template in the template library are used, but in the automatic sample annotation stage, the normal, damaged and missing fastener templates are used to calculate the classification score of the fastener regions, and the fastener region with the classification score higher than the threshold λ is used as the training sample.
The common classification basis for the K-NN method is the voting method. However, in the rail fastener defect inspection task, the number of defective fasteners is very small, which will result in the imbalance between the number of normal fastener templates and the number of defective fastener templates. Moreover, there are also wrongly annotated noise templates, which will reduce the classification performance of the K-NN method. Therefore, a weight function is designed:
The weight function calculates the similarity between the input fastener region x and the average Bhattacharyya coefficient of each type of fastener template, which can not only deal with the imbalance of the number of different types of fastener templates, but also reduce the sensitivity of the K-NN method to fastener templates and improve the classification accuracy.
Compared with the previous supervised fastener defect recognition method, the automatic sample annotation method avoids manual operation, which improves the automation degree of the fastener defect inspection system. Note that the automatically annotated fastener samples may have a small amount of false annotation.
3. Fastener defects recognition method based on deep convolutional neural network
Deep convolutional neural networks have been widely used in image recognition, target tracking and other tasks, which have made great breakthroughs. Compared with the traditional image classification method, the deep convolutional neural network is inspired by the perspective of bionics, which uses multiple layers of convolution to simulate the hierarchical structure in the human cerebral nervous system. After multiple iterative learning, it can learn and extract more discriminatory features to effectively improve the performance of image recognition.
When using deep convolutional neural networks for image classification tasks, a large number of training samples need to be input. However, in the fastener inspection for multi railways, manual annotation of a large number of fastener samples for each railway is very labor- and material-consuming. The proposed method can automatically annotate a large number of fastener samples for pretraining deep convolutional neural networks. Then, for each railway to be inspected, the pretraining model can be fine-tuned with a small number of fastener samples (of the railway to be inspected) to achieve a good classification performance. In the actual inspection task, the fastener samples can also be automatically collected to regularly fine-tune the fastener defect recognition model, thus resisting the degradation of model classification performance caused by environmental differences (e.g. rainy and snowy weather) due to weather and season changes. Consequently, the fastener defect recognition model can achieve better classification performance on this railway.
3.1 Network structure design
Based on the AlexNet (Krizhevsky, Sutskever, & Hinton, 2012) model, the network structure of the fastener defect recognition method is designed according to the characteristics of the fastener region images. Due to the small size and simple content of the fastener region images, the first 2 convolution layers of the AlexNet model are deleted, and the number of neurons in the fully connected layer is reduced to improve the calculation speed of the network. At the same time, in order to keep the size of the receptive field of the image features unchanged, the size of the convolution kernel in the convolution layer is increased. The network structure of the fastener defect recognition method is shown in Figure 4. In the figure, the convolution kernel of each convolution layer is (7 × 7) pixels with a step size of 1, and the number of convolution kernels of 3 convolution layers is 128, 256 and 512, respectively; the maximum value is selected by the pooling operation, the window size is (2 × 2) pixels and the step size is 2; the 1st and 2nd fully connected layers have 1,024 nerve nodes for expressing the image features extracted from the convolution layers in high-dimensional space; the last fully connected layer has 3 nerve nodes representing the categories of fastener region image classification, that is, normal, damaged and missing.
In addition, each convolution layer and fully connected layer are followed by a rectified linear unit (ReLU) as an activation function, which helps to accelerate the convergence of the network. Dropout strategies are used after the 1st and 2nd fully connected layers to effectively prevent overfitting of the network. The local response normalization (LRN) layer is not used in the network structure of the designed fastener defect recognition method, because the LRN layer requires additional memory overhead and calculation cost but has little improvement in network performance (Simonyan and Zisserman, 2015).
3.2 Data augmentation method
Deep convolutional neural networks require a large number of training samples, and the number of each type of training sample shall be roughly balanced. However, in the fastener defect inspection task, the number of defective fasteners is far lower than that of normal fasteners, which is not conducive to the training of deep convolutional neural network. Therefore, a random sample sorting strategy is proposed to reduce the impact of sample quantity imbalance on recognition performance. The random sample sorting strategy is divided into 5 steps as follows.
Step 1 Sort the training data set according to the sample label;
Step 2 Find the category with the largest number of samples and record the number of samples as M;
Step 3 Create a list of numbers with length of M for each category and conduct random sorting;
Step 4 Read the number m in the number list of each category through traversal, and calculate the residue with the total number of this category to get an index value. Find the corresponding image according to this index value and add it to the image list L.
Step 5 Combine the image list L of all categories, randomly sort it and use the list as the input to the network.
In order to prevent a large number of duplicate samples from overfitting the network model, Gaussian noise is added to each defective fastener sample at the data input layer of the network, and Gaussian parameters are randomly selected to reduce the repetition of training data. This method has low computation overhead, and the image after increasing noise does not need to be stored in hard disk. In the experiment, the value range of the mean value is [−2, 2], the value range of the standard deviation is [0, 1] and the value range of the Gaussian coefficient is [16, 64].
4. Experiment and evaluation
The fastener defect inspection system is developed with C++, which uses the tool libraries such as Opencv3.0, CUDA8.0 and Caffe (Jia et al., 2014). The computing server includes two Intel E5-2630v4 CPUs and one NVIDIA Tesla k40c GPU computer card.
4.1 Multi railways fastener data set
Since there is no published data set of rail fastener in the railway inspection task, all the rail fastener data are collected from the real railway line and annotated manually. The data set consists of 2 parts: fastener region location test data and fastener defect recognition test data. This part of the test data is only annotated with the location of the fastener region, which is used to test the performance of the fastener region locating method and automatically collect the fastener samples. This part of the fastener defect recognition test data is annotated with the classification label of fasteners to fine-tune and test the performance of fastener defect recognition method. The rail fastener data set contains a variety of common fastener types, which can be divided into hook type, nut type and α type according to the shape of fasteners. In addition, in the fastener data of ballasted railway lines, the fastener region mostly covered by ballast is marked as damaged fastener. Details of rail fastener test data in each railway line are given in Tables 1 and 2.
4.2 Fastener region location
The detection rate is used as the evaluation index in the fastener region location experiment, and the threshold of intersection over union (IoU) is set to be 0.9 to ensure the effectiveness of fastener region location.
Two schemes are adopted for the experiment: Scheme 1 adopts the off-line part of the template library; Scheme 2 adopts the method proposed in this paper, that is, adding the online template library. The maximum queue length of the online template library is set to 300, and the off-line template library of the two schemes is the same, that is, it contains 20 fastener template images and 20 nonfastener template images, and the number K of adjacent templates is set to 19.
The experimental results of each line are shown in Table 3. The average detection rate of Scheme 1 is lower than that of Scheme 2; the detection rate of each line varies greatly in Scheme 1, while the detection rate of each line in Scheme 2 is basically above 99%, indicating that the fastener region location method in this paper has high reliability and good adaptability to multi railways.
In order to test the influence of the maximum queue length of the online template library on the locating results of the fastener region, the number K of adjacent templates is fixed to 19, and the detection rate and detection speed (unit: number of frames per second) are used as evaluation indicators. The data of railway line 1# are adopted for the experiment, and the results are shown in Table 4.
From Table 4, it can be seen that with the increase of the maximum queue length of the online template library, the detection rate also increases in a small extent, but the detection speed decreases continuously, affecting the operation efficiency of the system. In general, for the method proposed in this paper, the increase of maximum queue length of the online template library improves the detection rate, but also reduces the detection speed, thus affecting the system efficiency. In actual application, the method can be flexibly adjusted according to requirements.
In order to test the influence of the number of adjacent templates on the locating results of the fastener region, the maximum queue length of the online template library is set to 300, and then different K values are taken for experiment. The data of railway line 1# are adopted for the experiment, and the results are shown in Table 5.
In Table 5, a larger K value will increase the detection rate of the fastener region location method to a certain extent, but it does not greatly improve the rate and will lead to the loss of system efficiency.
4.3 Fastener defect recognition
In the fastener defect recognition experiment, the reliability of the method is evaluated by 3 indicators: accuracy rate, recall rate and F1 score.
First, the training samples are automatically collected on the data of lines 1#, 2#, 3# and 4# by using the automatic sample annotation method, and finally 55,686 normal fasteners, 1,596 damaged fasteners and 1,080 missing fasteners are collected. The collected fastener sample set is then data-enhanced in conjunction with the random sample sorting strategy proposed in this paper for pretraining the fastener defect recognition method. Finally, the pretraining network model is fine-tuned with the fastener data of lines 5#, 6# and 7#, respectively. For the fairness of the experiment, the same number of samples are used for fine-tuning of each railway line, that is, 2,000 images of normal, damaged and lost fastener samples, and the remaining fastener sample images were used to test method performance.
The network model is trained by the stochastic gradient descent (SGD) method. The input sample size is 128, the momentum is set as 0.9, the weight attenuation value is set as 0.0005 and the Dropout ratio is set as 0.5. The training includes 20 epochs, and the initial learning rate is set as 10−4, which will decrease by 10 times after every 5 epochs of training. The experiment results are shown in Table 6.
From Table 6, it can be seen that the experiment results of line 7# are slightly better than those of lines 5# and 6#, because there is only 1 type of fasteners on line 7# and 2 types of fasteners on lines 5# and 6#; the fastener defect recognition method has achieved relatively reliable performance with only a small number of training samples, indicating that the fastener defect recognition method proposed in this paper has good adaptability to multi railways.
To demonstrate the advantages of fastener defect recognition method, 5 experiment schemes are proposed for a comparative experiment, in which the data of line 7# are used as the test data. Scheme 1 is to recognize the fastener defects by using the off-line part of the template library and K-NN method. Scheme 2 is to recognize the fastener defects with an additional online template library and K-NN method. Scheme 3 is the automatic sample annotation method proposed in this paper, that is, classification weight function is added on the basis of Scheme 2. The advantage of these 3 schemes is that they require no training on classification models. Scheme 4 is to directly use the pretrained network model for fastener defect recognition, in which the data of line 7# are not used for fine-tuning. Scheme 5 is to fine-tune the pretrained network model with the data of line 7#. The results are shown in Table 7.
In Table 7, the comprehensive performance of Scheme 4 and Scheme 5 is far higher than those of other schemes, which significantly improves the reliability of the fastener defect inspection system. Moreover, fine-tuning of fastener samples on the line to be detected can effectively improve the recognition performance of the fastener defect recognition method on this line. In addition, the automatic sample annotation method proposed in this paper is superior to the method using only the off-line template library in terms of recognition performance. Scheme 2 does not use the weight function proposed in this paper. The number of normal fastener templates in the online template library is much larger than the number of defective fastener templates, resulting in poor recognition performance, which is in line with the expectations in this paper. The automatic sample annotation method proposed in this paper, although not suitable as the final classifier, has a high accuracy of recognition and is not required to be trained. It is very suitable for automatic annotation of fastener samples.
4.4 Comparison of image features
For image recognition, it is very important to extract discriminating image features. The HOG and LBP features are commonly used for comparison with the image features extracted. The results are shown in Table 8.
In Table 8, although the traditional HOG and LBP methods can be used to describe rail fasteners, the extracted image features are rough and contain a large number of background elements. However, the image features extracted by the fastener defect recognition method based on the deep convolutional neural network focus on the rail fasteners, ignoring the characteristics of the background elements. Obviously, the image features extracted by the fastener defect recognition method based on the deep convolutional neural network are more discriminating and insensitive to the changes in the lighting conditions of the images.
5. Conclusion
In this paper, a rail fastener defect inspection method for multi railways based on machine vision is proposed, which can be used for inspection tasks on multi railways. The fastener region location method based on the online learning strategy not only enhances the reliability and multi-railway adaptability of the fastener region location module, but also automatically collects and annotates the fastener samples. The fastener defect recognition method based on the deep convolutional neural network improves the reliability of the fastener defect recognition module and improves the multi-railway adaptability of the network model by using the fine-tuning technology. The experimental results show that the proposed method has high reliability and good multi-railway adaptability, and the performance is superior to the previous fastener defect inspection method, which has strong practical value and theoretical significance for the rail fastener defect inspection of multi railways.
Figures
Details of fastener region location test data
| Line no | Number of track images/Nr | Number of fasteners/Nr | Type |
|---|---|---|---|
| 1# | 2,931 | 17,380 | Hook/Bolt |
| 2# | 3,112 | 18,486 | Bolt |
| 3# | 3,072 | 17,903 | Hook |
| 4# | 3,171 | 19,289 | α type |
Details of fastener defect recognition test data
| Line no | Number of fasteners of different types/Nr | Type | ||
|---|---|---|---|---|
| Normal | Damage | Lost | ||
| 5# | 24,670 | 11,105 | 11,505 | Hook/Bolt type |
| 6# | 13,946 | 6,383 | 7,567 | Hook/α type |
| 7# | 18,456 | 8,476 | 8,505 | Hook type |
Detection rate of fastener region location by different schemes/%
| Line no | Scheme 1 | Scheme 2 |
|---|---|---|
| 1# | 97.00 | 99.28 |
| 2# | 96.84 | 98.95 |
| 3# | 95.16 | 99.65 |
| 4# | 87.23 | 99.57 |
| Average | 94.06 | 99.36 |
The results under different queue lengths
| Maximum queue length/Nr | Detection rate/% | Detection speed/(frame s−1) |
|---|---|---|
| 150 | 99.16 | 60 |
| 200 | 99.22 | 54 |
| 300 | 99.28 | 42 |
| 400 | 99.31 | 31 |
| 600 | 99.34 | 23 |
| 1,000 | 99.35 | 5 |
The experiment results with different K values
| K/Nr | Detection rate/% | Detection speed/(frame s−1) |
|---|---|---|
| 11 | 99.21 | 43 |
| 19 | 99.28 | 42 |
| 37 | 99.26 | 37 |
| 59 | 99.31 | 34 |
| 113 | 99.32 | 27 |
| 211 | 99.31 | 19 |
The results of fastener defect recognition for multi railways
| Line no | Precision/% | Recall/% | F1 score |
|---|---|---|---|
| 5# | 98.86 | 79.24 | 0.8797 |
| 6# | 94.31 | 90.01 | 0.9211 |
| 7# | 97.29 | 93.28 | 0.9524 |
| Average | 96.82 | 87.51 | 0.9177 |
Comparative results of fastener defect recognition
| Scheme | Precision/% | Recall/% | F1 score |
|---|---|---|---|
| Scheme 1 | 96.34 | 46.97 | 0.6315 |
| Scheme 2 | 60.92 | 37.53 | 0.4645 |
| Scheme 3 | 98.81 | 65.06 | 0.7850 |
| Scheme 4 | 90.02 | 81.50 | 0.8980 |
| Scheme 5 | 97.29 | 93.28 | 0.9524 |
Comparison of image features extracted by different methods
 |
References
Dai, P., Wang, S., Du, X., Han, Q., Wang, H., & Ren, S (2018). Image recognition method for the fastener defect of ballastless track based on semi-supervised deep learning. China Railway Science, 39(4), 43–49.
Dou, Y., Huang, Y., Li, Q., & Luo, S (2014). A fast template matching-based algorithm for railway bolts detection. International Journal of Machine Learning and Cybernetics, 5(6), 835–844.
Du, X., Dai, P., Li, Y., Cheng, Y., Wang, S., Han, Q., & Wang, H (2017). Automatic detection algorithm for rail plug based on deep learning. China Railway Science, 38(3), 89–96.
Fan, H., Cosman, P., Hou, Y., & Li, B (2018). High speed railway fastener detection based on line local binary pattern. IEEE Signal Processing Letters, 25(6), 788–792.
Feng, H., Jiang, Z., Xie, F., Yang, P., Shi, J., & Chen, L (2013). Automatic fastener classification and defect detection in vision-based railway inspection systems. IEEE Transactions on Instrumentation and Measurement, 63(4), 877–888.
Huang, Y., Luo, S., & Wang, S (2012). Combining boundary and region information with bolt prior for rail surface detection. IEICE Transactions on Information and Systems, 95(2), 690–693.
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., & Girshick, R (2014). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia, 675-678. New York: ACM.
Krizhevsky, A., Sutskever, I., & Hinton, G (2012). ImageNet Classification with Deep Convolutional Neural Networks, 1097-1105. New York: Advances in Neural Information Processing Systems.
Li, Q., & Ren, S (2012a). A visual detection system for rail surface defects. IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews, 42(6), 1531–1542.
Li, Q., & Ren, S (2012b). A real-time visual inspection system for discrete surface defects of rail heads. IEEE Transactions on Instrumentation and Measurement, 61(8), 2189–2199.
Li, Y., Trinh, H., Haas, N., Otto, C., & Pankanti, S (2013). Rail component detection, optimization, and assessment for automatic rail track inspection. IEEE Transactions on Intelligent Transportation Systems, 15(2), 760–770.
Li, Q., Zhang, H., Ren, S., Dai, P., & Li, W (2016). Detection method for rail corrugation based on rail image feature in frequency domain. China Railway Science, 37(1), 24–30.
Liu, J., Xiong, Y., Li, B., & Li, L (2016). Research on automatic inspection algorithm for railway fastener defects based on computer vision. Journal of the China Railway Society, 38(8), 73–80.
Marino, F., Distante, A., Mazzeo, P., & Stella, E (2007). A real-time visual inspection system for railway maintenance: automatic hexagonal-headed bolts detection. IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews, 37(3), 418–428.
Mazzeo, P., Ancona, N., Stella, E. and Distante, A. (2003). Visual recognition of hexagonal headed bolts by comparing ICA to wavelets. Proceedings of the 2003 IEEE International Symposium on Intelligent Control, 636-641.
Mazzeo, P., Nitti, M., Stella, E., & Distante, A (2004). Visual recognition of fastening bolts for railroad maintenance. Pattern Recognition Letters, 25(6), 669–677.
Ren, S., Li, Q., Xu, G., Han, Q., Luo, S., & Feng, Q (2011). Research on robust fast algorithm of rail surface defect detection. China Railway Science, 32(1), 25–29.
Rubinsztejn, Y (2011). Automatic Detection of Objects of Interest from Rail Track Images. Dissertation. The University of Manchester.
Simonyan, K. and Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd IEEE International Conference on Learning Representations, San Diego.
Sun, C., Liu, J., Qin, Y., & Zhang, Y (2018). Intelligent detection method for rail flaw based on deep learning. China Railway Science, 39(5), 51–57.
Xia, Y., Xie, F. and Jiang, Z. (2010). Broken railway fastener detection based on adaboost algorithm. Proceedings of IEEE International Conference on Optoelectronics and Image Processing, 314-317. Haikou.
Xu, G., Shi, T., Ren, S., Han, Q., & Wang, D (2013). Development of the on-board track inspection system based on computer vision. China Railway Science, 34(1), 139–144.
Zhao, X., Qian, S., & Liu, X (2018). Image identification method for high-strength bolt missing on railway bridge based on convolution neural network. China Railway Science, 39(4), 56–62.
Zhou, W., Sun, Z., Ren, S., Zhang, W., Wang, H., Dai, P., & Wang, Y (2015). Measurement method for geometric parameters of overhead contact line based on multi-view stereovision. China Railway Science, 36(5), 104–109.
Acknowledgements
Funding: Project funded by the Key Research and Development Project of China Academy of Railway Sciences Corporation Limited (2021YJ310).