Browsing by Author "Ye, Zhongfu"

Now showing 1 - 5 of 5
  • Thumbnail Image
    Item
    Deep Predictive Neural Network: Unsupervised Learning for Hand Pose Estimation
    (2019-08-15) Banzi, Jamal; Bulugu, Isack; Ye, Zhongfu
    The discriminative approaches for hand pose estimation from depth images usually require dense annotated data to train a supervised network. Additionally, generative methods depend on temporal information in generating candidate poses which can be trapped due to local minima during the optimization process. Different from these methods, we propose a hybrid two-stage deep predictive neural network approach that performs predictive coding of image sequences of hand poses in order to capture latent features underlying a given image. Firstly, we train a deep convolutional neural network (CNN) for direct regression of hand joints position. Secondly, we add an unsupervised error term as a part of the recurrent architecture connected with predictive coding portion. An error regression term (ERT) ensures minimal residual errors of the estimated values while the predictive coding portion allows training of the network without the supervision of image sequences, so no dense annotation of data is required. We conduct a complete experiment using two challenging public datasets, ICVL and NYU. Using the ICVL datasets, our approach improved accuracy over the current state of the art methods with an average error joint of 7.5mm. We also achieve 12.2mm average error joint on NYU dataset which is the smallest error to be achieved on all state-of-art approaches.
  • Thumbnail Image
    Item
    Higher-Order Local Autocorrelation Feature Extraction Methodology for Hand Gestures Recognition
    (IEEE, 2017-12-25) Bulugu, Isack; Ye, Zhongfu; Banzi, Jamal; Bulugu, Isack
    A novel feature extraction method for hand gesture recognition from sequences of image frames is described and tested. The proposed method employs higher order local autocorrelation (HLAC) features for feature extraction. The features are extracted using different masks from Grey-scale images for characterising hands image texture with respect to the possible position, and the product of the pixels marked in white. Then features with the most useful information are selected based on mutual information quotient (MIQ). Multiple linear discriminant analysis (LDA) classifier is adopted to classify different hand gestures. Experiments on the NUS dataset illustrate that the HLAC is efficient for hand gesture recognition compared with other feature extraction methods.
  • Thumbnail Image
    Item
    Learning a deep predictive coding network for a semi-supervised 3D-hand pose estimation
    (IEEE, 2020-03-27) Bulugu, Isack; Banzi, Jamal; Huang, Shiliang; Ye, Zhongfu
    In this paper we present a CNN based approach for a real time 3D-hand pose estimation from the depth sequence. Prior discriminative approaches have achieved remarkable success but are facing two main challenges: Firstly, the methods are fully supervised hence require large numbers of annotated training data to extract the dynamic information from a hand representation. Secondly, unreliable hand detectors based on strong assumptions or a weak detector which often fail in several situations like complex environment and multiple hands. In contrast to these methods, this paper presents an approach that can be considered as semi-supervised by performing predictive coding of image sequences of hand poses in order to capture latent features underlying a given image without supervision. The hand is modelled using a novel latent tree dependency model ( LDTM ) which transforms internal joint location to an explicit representation. Then the modeled hand topology is integrated with the pose estimator using data dependent method to jointly learn latent variables of the posterior pose appearance and the pose configuration respectively. Finally, an unsupervised error term which is a part of the recurrent architecture ensures smooth estimations of the final pose. Experiments on three challenging public datasets, ICVL, MSRA, and NYU demonstrate the significant performance of the proposed method which is comparable or better than state-of-the-art approaches.
  • Thumbnail Image
    Item
    A novel Hand Pose Estimation using Dicriminative Deep Model and Transductive Learning Approach for Occlusion Handling and Reduced Descrepancy
    (IEEE, 2017-05-11) Banzi, Jamal; Bulugu, Isack; Ye, Zhongfu
    Discriminative based model have demonstrated an epic distinction in hand pose estimation. However there are key challenges to be solved on how to intergrate the self-similar parts of fingers which often occlude each other and how to reduce descrepancy among synthetic and realistic data for an accurate estimation. To handle occlusion which lead to inaccurate estimation, this paper presents a probabilistic model for finger position detection framework. In this framework the visibility correlation among fingers aid in predicting the occluded part between fingers thereby estimating hand pose accurately. Unlike convectional occlusion handling approach which assumes occluded parts of fingers as independent detection target, this paper presents a discriminative deep model which learns the visibility relationship among the occluded parts of fingers at multiple layers. In addition, we propose the semi-supervised Transductive Regression(STR) forest for classification and regression to minimise discrepancy among realistic and synthetic pose data. Experimental results demonstrate promising performance with respect to occlusion handling, and discrepancy reduction with higher degree of accuracy over state-of-the-art approaches.
  • Thumbnail Image
    Item
    Scale Invariant Static Hand-Postures Detection using Extended Higher-order Local Autocorrelation Features
    (Foundation of Computer Science (FCS), NY, USA, 2016-02-17) Bulugu, Isack; Ye, Zhongfu
    This paper presents scale invariant static hand postures detection methods using extended HLAC features extractedfrom Log-Polar images. Scale changes of a handposture in an image are represented as shift in Log-Polar image. Robustness of the method is achieved through extracting spectral features from theeach row of the Log-Polar image. Linear Discriminant Analysis was used to combine features with simple classification methods in order to realize scale invariant hand postures detection and classification.The method was successful tested by performing experiment using NSU hand posture dataset images which consists 10 classes of postures, 24 samples of images per class, which are captured by the position and size of the hand within the image frame. The results showed that the detection rate using Extended-HLAC can averaged reach 94.63% higher than using HLAC features on a Intel Core i5-4590 CPU running at 3.3 GHz.