Parallel channel and position attention-guided feature pyramid for pig face posture detection

Zhiwei Hu, Hongwen Yan, Tiantian Lou

Abstract


The area of the pig's face contains rich biological information, such as eyes, nose, and ear. The high-precision detection of pig face postures is crucial to the identification of pigs, and it can also provide fundamental archival information for the study of abnormal behavioral characteristics and regularities. In this study, a series of attention blocks were embedded in Feature Pyramid Network (FPN) for automatic detection of the pig face posture in group-breeding environments. Firstly, the Channel Attention Block (CAB) and Position Attention Block (PAB) were proposed to capture the channel dependencies and the pixel-level long-range relationships, respectively. Secondly, a variety of attention modules are proposed to effectively combine the two kinds of attention information, specifically including Parallel Channel Position (PCP), Cascade Position Channel (CPC), and Cascade Channel Position (CCP), which fuse the channel and position attention information in both parallel or cascade ways. Finally, the verification experiments on three task networks with two backbone networks were conducted for different attention blocks or modules. A total of 45 pigs in 8 pigpens were used as the research objects. Experimental results show that attention-based models perform better. Especially, with Faster Region Convolutional Neural Network (Faster R-CNN) as the task network and ResNet101 as the backbone network, after the introduction of the PCP module, the Average Precision (AP) indicators of the face poses of Downward with head-on face (D-O), Downward with lateral face (D-L), Level with head-on face (L-O), Level with lateral face (L-L), Upward with head-on face (U-O), and Upward with lateral face (U-L) achieve 91.55%, 90.36%, 90.10%, 90.05%, 85.96%, and 87.92%, respectively. Ablation experiments show that the PAB attention block is not as effective as the CAB attention block, and the parallel combination method is better than the cascade manner. Taking Faster R-CNN as the task network and ResNet101 as the backbone network, the heatmap visualization of different layers of FPN before and after adding PCP shows that, compared with the non-PCP module, the PCP module can more easily aggregate denser and richer contextual information, this, in turn, enhances long-range dependencies to improve feature representation. At the same time, the model based on PCP attention can effectively detect the pig face posture of different ages, different scenes, and different light intensities, which can help lay the foundation for subsequent individual identification and behavior analysis of pigs.
Keywords: objection detection, attention mechanism, feature pyramid network, face posture detection, pig
DOI: 10.25165/j.ijabe.20221506.7329

Citation: Hu Z W, Yan H W, Lou T T. Parallel channel and position attention-guided feature pyramid for pig face posture detection. Int J Agric & Biol Eng, 2022; 15(6): 222–234.

Keywords


objection detection, attention mechanism, feature pyramid network, face posture detection, pig

Full Text:

PDF

References


Hansen M F, Smith M L, Smith L N, Salter M G, Baxte E M, Farish M, et al. Towards on-farm pig face recognition using convolutional neural networks. Computers in Industry, 2018; 98: 145–152.

Marsot M, Mei J Q, Shan X C, Ye L Y, Feng P, Yan X J, et al. An adaptive pig face recognition approach using Convolutional Neural Networks. Computers and Electronics in Agriculture,2020; 173: 105386. doi: 10.1016/j.compag.2020.105386.

Zhang K F, Li D, Huang J Y, Chen Y F. Automated video behavior recognition of pigs using two-stream convolutional networks. Sensors, 2020; 20(4): 1085. doi: 10.3390/s20041085.

Condotta, I C, Brown-Brandl T M, Silva-Miranda K O, Stinn J P. Evaluation of a depth sensor for mass estimation of growing and finishing pigs. Biosystems Engineering, 2018; 173: 11–18.

Valletta J J, Torney C, Kings M, Thornton A, Madde J. Applications of machine learning in animal behavior studies. Animal Behavior, 2017; 124: 203–220.

Nasirahmadi A, Sturm B, Olsson A C, Jeppsson K H, Müller S, Edwards S, et al. Automatic scoring of lateral and sternal lying posture in grouped pigs using image processing and Support Vector Machine. Computers and electronics in agriculture, 2019; 156: 475–481.

Shi C, Zhang J L, Teng G H. Mobile measuring system based on LabVIEW for pig body components estimation in a large-scale farm. Computers and electronics in agriculture, 2019; 156: 399–405.

Chen C, Zhu W X, Liu D, Steibel J, Siegford J, Wurtz K, et al. Detection of aggressive behaviors in pigs using a RealSence depth sensor. Computers and Electronics in Agriculture, 2019; 166, 105003. doi: 10.1016/j.compag.2019.105003.

da Fonseca F N, Abe J M, de Alencar Nääs I, da Silva Cordeiro A F, do Amaral F V, Ungaro H C. Automatic prediction of stress in piglets (Sus Scrofa) using infrared skin temperature. Computers and Electronics in Agriculture, 2020; 168: 105148. doi: 10.1016/j.compag.2019.105148.

Sun L Q, Chen S H, Liu T, Liu C H, Liu Y. Pig target tracking algorithm based on multi-channel color feature fusion. Int J Agric & Biol Eng, 2020; 13(3): 180–185.

Wu X W, Sahoo D, Hoi S C H. Recent advances in deep learning for object detection. Neurocomputing, 2020; 396: 39–64.

Zhang Y Q, Chu J, Leng L, Miao J. Mask-refined R-CNN: A network for refining object details in instance segmentation. Sensors, 2020; 20(4), 1010. doi: 10.3390/s20041010.

Kamilaris A, Prenafeta-Boldú F X. Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 2018; 147: 70–90.

Tassinari P, Bovo M, Benni S, Franzoni S, Poggi M, Mammi L M E, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn. Computers and Electronics in Agriculture, 2021; 182: 106030. doi: 10.1016/j.compag.2021.106030.

Jiang M, Rao Y, Zhang J Y, Shen Y M. Automatic behavior recognition of group-housed goats using deep learning. Computers and Electronics in Agriculture, 2020; 177: 105706. doi: 10.1016/j.compag.2020.105706.

Su D, Qiao Y, Kong H, Sukkarieh S. Real time detection of inter-row ryegrass in wheat farms using deep learning. Biosystems Engineering, 2021; 204: 198–211.

Chen C, Zhu W, Steibel J, Siegford J, Wurtz K, Han J, et al. Recognition of aggressive episodes of pigs based on convolutional neural network and long short-term memory. Computers and Electronics in Agriculture, 2020; 169: 105166. doi: 10.1016/j.compag.2019.105166.

Chen C, Zhu W X, Steibel J P, Siegford J M, Han J, Norton T J. Classification of drinking and drinker-playing in pigs by a video-based deep learning method. Biosystems Engineering, 2020; 196: 1–14.

Yang A, Huang H, Yang X, Li S, Chen C, Gan H, et al. Automated video analysis of sow nursing behavior based on fully convolutional network and oriented optical flow. Computers and Electronics in Agriculture, 2019; 167: 105048. doi: 10.1016/j.compag.2019.105048.

Hu Z W, Yang H, Lou T T, Hu G, Xie Q Q, Huang J M. Extraction of pig contour based on fully convolutional networks. Journal of South China Agricultural University 2018; 39(6): 111–119. (in Chinese)

Hu Z W, Yang H, Lou T T. Dual attention-guided feature pyramid network for instance segmentation of group pigs. Computers and Electronics in Agriculture, 2021; 186: 106140. doi: 10.1016/ j.compag.2021.106140.

Tian M X, Guo H, Chen H, Wang Q, Long C J, Ma Y H. Automated pig counting using deep learning. Computers and Electronics in Agriculture, 2019; 163 104840. doi: 10.1016/j.compag.2019.05.049.

Nasirahmadi A, Sturm B, Edwards S, Jeppsson K H, Olsson A C, Müller S, et al. Deep learning and machine vision approaches for posture detection of individual pigs. Sensors, 2019; 19(17): 3738. doi: 10.3390/s19173738.

Zheng C, Zhu X, Yang X, Wang L, Tu S, Xue Y. Automatic recognition of lactating sow postures from depth images by deep learning detector. Computers and Electronics in Agriculture, 2018; 147: 51–63.

Tong W, Chen W T, Han W, Li X J, Wang L Z. Channel-attention-based DenseNet network for remote sensing image scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020; 13: 4121–4132.

Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017; pp.4700–4708.

Chen C F, Gong D H, Wang H, Li Z F, Wong K Y K. Learning spatial attention for face super-resolution. IEEE Transactions on Image Processing, 2020; 30: 1219–1231.

Woo S, Park J, Lee J Y, Kweon I S. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), IEEE, 2018; pp.3–19.

Park J, Woo S, Lee J Y, Kweon I S. Bam: Bottleneck attention module. arXiv, 2018; arXiv preprint arXiv:1807.06514.

Roy A G, Navab N, Wachinger C. Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018; pp.421–429.

Fu J, Liu J, Tian H J, Li Y, Bao Y J, Fang Z W, et al. Dual attention network for scene segmentation. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach: IEEE, 2019; pp.3146–3154. doi: 10.1109/CVPR.2019.00326.

Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2014; pp.580–587.

Girshick R. Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, 2015; pp.1440–1448.

Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016; 39(6): 1137–1149.

Lin T, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2019; pp.936–944.

Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S. Generalized intersection over union: A metric and a loss for bounding box regression. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2019; pp.658–666.

Cai Z, Vasconcelos N. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018; pp.6154–6162.

Pang J M, Chen K, Shi J P, Feng H J, Ouyang W L, Lin D H. Libra R-CNN: Towards balanced learning for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2019; pp.821–830.

He K M, Zhang X Y, Ren S Q, Sun J. Deep Residual Learning for Image Recognition. computer vision and pattern recognition. In: 2016 IEEE Conference on Computer Vision and Patter Recognition (CVPR), IEEE, 2016; pp.770–778.

Wu T Y, Tang S, Zhang R, Cao J, Zhang Y D. Cgnet: A light-weight context guided network for semantic segmentation. IEEE Transactions on Image Processing, 2020; 30: 1169–1179.

Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018; 7132–7141.

open-mmlab/mmdetection. Available: https://github.com/open-mmlab/ mmdetection. Accessed on [2022-01-15].

Visual Object Classes Challenge 2012 (VOC 2012). Available: http://host.robots.ox.ac.uk/pascal/VOC/voc2012. Accessed on [2022-01-15].




Copyright (c) 2022 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office