A systematic strategy of pallet identification and picking based on deep learning techniques
ISSN: 0143-991x
Article publication date: 11 January 2023
Issue publication date: 17 February 2023
Abstract
Purpose
This paper presents a comprehensive pallet-picking approach for forklift robots, comprising a pallet identification and localization algorithm (PILA) to detect and locate the pallet and a vehicle alignment algorithm (VAA) to align the vehicle fork arms with the targeted pallet.
Design/methodology/approach
Opposing vision-based methods or point cloud data strategies, we utilize a low-cost RGB-D camera, and thus PILA exploits both RGB and depth data to quickly and precisely recognize and localize the pallet. The developed method guarantees a high identification rate from RGB images and more precise 3D localization information than a depth camera. Additionally, a deep neural network (DNN) method is applied to detect and locate the pallet in the RGB images. Specifically, the point cloud data is correlated with the labeled region of interest (RoI) in the RGB images, and the pallet's front-face plane is extracted from the point cloud. Furthermore, PILA introduces a universal geometrical rule to identify the pallet's center as a “T-shape” without depending on specific pallet types. Finally, VAA is proposed to implement the vehicle approaching and pallet picking operations as a “proof-of-concept” to test PILA’s performance.
Findings
Experimentally, the orientation angle and centric location of the two kinds of pallets are investigated without any artificial marking. The results show that the pallet could be located with a three-dimensional localization accuracy of 1 cm and an angle resolution of 0.4 degrees at a distance of 3 m with the vehicle control algorithm.
Research limitations/implications
PILA’s performance is limited by the current depth camera’s range (< = 3 m), and this is expected to be improved by using a better depth measurement device in the future.
Originality/value
The results demonstrate that the pallets can be located with an accuracy of 1cm along the x, y, and z directions and affording an angular resolution of 0.4 degrees at a distance of 3m in 700ms.
Keywords
Acknowledgements
Funding: This work was financially supported by the National Natural Science Foundation of China (#61975228) and Dalian Technology Bureau (Covid-19 Emergency Fund) and Jilin Science and Technology Development Plan Project (Grant # 20220203053SF).
Conflicts of interest/competing interests: The authors declare no conflict of interest.
Code or data availability: TDS software demo using PILA can be found at the following link or contact: qsong@soleilwares.com.
Source code: www.github.com/unlogical0327/TDS_V1/tree/master/src.
Pallet Data set: www.github.com/unlogical0327/Pallet_database.
Author contributions: Conceptualization, Gunayu Ding and Qi Song; data curation, Yongao Li; formal analysis, Yongyao Li, Qi Song and Guanyu Ding; investigation, Chao Li, Qinglei Zhao and Sen Wang; methodology, Qi Song, Guanyu Ding, Chao Li and Yongyao Li; software, Qi Song, Guanyu Ding and Chao Li; supervision, Qi Song, validation, Yongyao Li and Qi Song; writing – original draft, Yongyao Li and Qi Song; writing – review and editing, Yongyao Li, Qinglei Zhao and Qi Song; All authors have read and agreed to the published version of the manuscript.
Ethics approval: Not applicable.
Consent to participate: Not applicable.
Consent for publication: Agree.
Citation
Li, Y., Ding, G., Li, C., Wang, S., Zhao, Q. and Song, Q. (2023), "A systematic strategy of pallet identification and picking based on deep learning techniques", Industrial Robot, Vol. 50 No. 2, pp. 353-365. https://doi.org/10.1108/IR-05-2022-0123
Publisher
:Emerald Publishing Limited
Copyright © 2022, Emerald Publishing Limited