To read this content please select one of the options below:

A systematic strategy of pallet identification and picking based on deep learning techniques

Yongyao Li (MetaX Technology Co.,Ltd, Ningbo, Zhejiang, China)
Guanyu Ding (MetaX Technology Co.,Ltd, Ningbo, Zhejiang, China)
Chao Li (MetaX Technology Co.,Ltd, Ningbo, Zhejiang, China)
Sen Wang (MetaX Technology Co.,Ltd, Ningbo, Zhejiang, China)
Qinglei Zhao (Changchun Institute of Optics Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, China)
Qi Song (Suzhou Institute of Biomedical Engineering and Technology, Suzhou, China)

Industrial Robot

ISSN: 0143-991x

Article publication date: 11 January 2023

Issue publication date: 17 February 2023

246

Abstract

Purpose

This paper presents a comprehensive pallet-picking approach for forklift robots, comprising a pallet identification and localization algorithm (PILA) to detect and locate the pallet and a vehicle alignment algorithm (VAA) to align the vehicle fork arms with the targeted pallet.

Design/methodology/approach

Opposing vision-based methods or point cloud data strategies, we utilize a low-cost RGB-D camera, and thus PILA exploits both RGB and depth data to quickly and precisely recognize and localize the pallet. The developed method guarantees a high identification rate from RGB images and more precise 3D localization information than a depth camera. Additionally, a deep neural network (DNN) method is applied to detect and locate the pallet in the RGB images. Specifically, the point cloud data is correlated with the labeled region of interest (RoI) in the RGB images, and the pallet's front-face plane is extracted from the point cloud. Furthermore, PILA introduces a universal geometrical rule to identify the pallet's center as a “T-shape” without depending on specific pallet types. Finally, VAA is proposed to implement the vehicle approaching and pallet picking operations as a “proof-of-concept” to test PILA’s performance.

Findings

Experimentally, the orientation angle and centric location of the two kinds of pallets are investigated without any artificial marking. The results show that the pallet could be located with a three-dimensional localization accuracy of 1 cm and an angle resolution of 0.4 degrees at a distance of 3 m with the vehicle control algorithm.

Research limitations/implications

PILA’s performance is limited by the current depth camera’s range (< = 3 m), and this is expected to be improved by using a better depth measurement device in the future.

Originality/value

The results demonstrate that the pallets can be located with an accuracy of 1cm along the x, y, and z directions and affording an angular resolution of 0.4 degrees at a distance of 3m in 700ms.

Keywords

Acknowledgements

Funding: This work was financially supported by the National Natural Science Foundation of China (#61975228) and Dalian Technology Bureau (Covid-19 Emergency Fund) and Jilin Science and Technology Development Plan Project (Grant # 20220203053SF).

Conflicts of interest/competing interests: The authors declare no conflict of interest.

Code or data availability: TDS software demo using PILA can be found at the following link or contact: qsong@soleilwares.com.

Source code: www.github.com/unlogical0327/TDS_V1/tree/master/src.

www.github.com/ChinaLyy/VAA.

Pallet Data set: www.github.com/unlogical0327/Pallet_database.

Author contributions: Conceptualization, Gunayu Ding and Qi Song; data curation, Yongao Li; formal analysis, Yongyao Li, Qi Song and Guanyu Ding; investigation, Chao Li, Qinglei Zhao and Sen Wang; methodology, Qi Song, Guanyu Ding, Chao Li and Yongyao Li; software, Qi Song, Guanyu Ding and Chao Li; supervision, Qi Song, validation, Yongyao Li and Qi Song; writing – original draft, Yongyao Li and Qi Song; writing – review and editing, Yongyao Li, Qinglei Zhao and Qi Song; All authors have read and agreed to the published version of the manuscript.

Ethics approval: Not applicable.

Consent to participate: Not applicable.

Consent for publication: Agree.

Citation

Li, Y., Ding, G., Li, C., Wang, S., Zhao, Q. and Song, Q. (2023), "A systematic strategy of pallet identification and picking based on deep learning techniques", Industrial Robot, Vol. 50 No. 2, pp. 353-365. https://doi.org/10.1108/IR-05-2022-0123

Publisher

:

Emerald Publishing Limited

Copyright © 2022, Emerald Publishing Limited

Related articles