Knowing where one is heading during
navigation brings assured happiness. Electrical Engineering is just the
splendid world where my ambition lies, where I will dedicate myself, and where
my life will scintillate.
Research interests
- Facial expression recognition
|
|
Bayes All |
Bayes FS |
AdaBoost |
Bayes GP (Our Approach) |
NL-SVM |
GP (Our Approach) |
|
Feature Selection |
Hand |
Hand |
Hand |
Automatic |
Hand |
Automatic |
|
Accuracy |
63.3% |
71.0% |
71.9% |
71% |
91.9% |
76.19% |
|
# Features |
612 |
60 |
80 |
25 |
612 |
35 |
Unlike current research on facial expression recognition that generally selects visually meaningful feature by hands, our learning method can discover the features automatically in a genetic programming-based approach that uses Gabor wavelet representation for primitive features and linear/nonlinear operators to synthesize new features. These new features are used to train a Bayes classifier1 and support vector machine classifier2 that is used for recognizing the facial expressions. The learned operator and classifier are used on unseen testing images. To make use of random nature of a genetic program, we design a multi-agent scheme to boost the performance
Compared to the previous work, our experimental results show that GP can find good composite operators. Our GP-based algorithm is effective in extracting feature vectors for classification. In our approach, we do not need to perform any pre-processing of the raw image and we do not need to find any reference points on the face.
Associated Publications:
- Bir Bhanu, Jiangang Yu, Xuejun Tan, Yingqiang Lin: Feature Synthesis Using Genetic Programming for Face Expression Recognition. GECCO (2) 2004: 896-907
- Jiangang Yu, Bir Bhanu: Evolutionary feature synthesis for facial expression recognition. Pattern Recognition Letters 27(11): 1289-1298 (2006)
·
Super-resolution of Facial images on video
( a ) ( b ) ( c ) ( d )
( a ) LR Image ( b ) Bicubic interpolated image
(a) (b) (c)
(a) LR images
Reconstruction-based super-resolution has been widely treated in computer vision. However, super-resolution of facial images has received very little attention. Super-resolution from facial images may suffer from subtle facial expression variation, non-rigid complex motion model, visibility and occlusion, and illumination and reflectance variations. Due to these reasons, most of existing super-resolution algorithms are not applicable to facial video sequences.
We propose a new method for enhancing the resolution of low-resolution facial image by handling the facial image non-uniformly. We segment facial image into different regions corresponding to different motion models and estimate the motions non-uniformly of tracked regions in the consecutive frames. The experimental results provide a proof of the concept for our method and show that our method gives better results than handling the face uniformly.
Associated Publications:
1. Jiangang Yu, Bir Bhanu: Super-resolution Restoration of Facial Images in Video. ICPR (4) 2006: 342-345
2. J. Yu, B. Bhanu, A. K. Roy-Chowdhury, Y. Xu: Super-resolved 3D Facial Texture in Video and Its Use for Face Recognition. Submitted to IEEE Tran. On PAMI.
· Super-resolution of facial images in video integrating illumination changes and 3D tracking
We incrementally super-resolve facial video with illumination changes. We first track video sequence taken under different illuminations and then super-resolve a video of human face into high resolution video. We propose a framework where pose and illumination invariant tracking and super-resolution take place in a closed-loop. First, given two frames of the video sequence, we recover the illumination, 3D motion and shape parameters. This information is used to obtain the super-resolved 3D texture using the Iterative Back-Projection (IBP) algorithm. The super-resolved texture, in turn, is used to improve the estimation of illumination and motion in combination with the incoming frames. This process continues for the entire video, obtaining a super-resolved facial texture for the 3D model as new frames come in. Note that we incrementally super-resolve facial texture through the whole video.
The following video shows the original video (left), video by bicubic interpolation (middle), and video generated by our approach (right).
The following video shows the comparison of super-resolution results between traditional open-loop approach and our closed-loop approach.
Closed-loop
Open-loop
Original
Some of the super-resolved images:
The first row shows the original input LR images. The second row shows the super-resolved images with current pose and illumination. The third row shows the pose and illumination normalized images.
The first row shows the original input LR images. The second row shows bicubic interpolated images with current pose and illumination. The third row shows the results of our approach with current pose and illumination. The third row shows the pose and illumination normalized images with respect to the middle one.
Original LR images are shown in the first row, the second row shows our super-resolved images with pose and illumination variations. Pose and illumination normalized super-resolved images are shown in the third row.
Associated Publications:
1. J. Yu, B. Bhanu, Y. Xu and A. Chowdhury: Super-resolve Facial Texture
under Changing Pose and Illumination. ICIP’07 (Accepted)
Miscellous
