ALBERT

Beschreibung: We present a calibration technique for an acoustic imaging microphone array, combined with a digital camera. Computer vision and acoustic time of arrival data are used to obtain microphone coordinates in the camera reference frame. Our new method allows acoustic maps to be plotted onto the camera images without the need for additional camera alignment or calibration. Microphones and cameras may be placed in an ad-hoc arrangement and, after calibration, the coordinates of the microphones are known in the reference frame of a camera in the array. No prior knowledge of microphone positions, inter-microphone spacings, or air temperature is required. This technique is applied to a spherical microphone array and a mean difference of 3 mm was obtained between the coordinates obtained with this calibration technique and those measured using a precision mechanical method.

Beschreibung: In this paper, we present a transform scheme where a secondary transform is applied after the conventional DCT for intra as well as inter prediction residues. Our approach is applicable to any block-based video codec that employs transforms along the horizontal and vertical direction separably. The secondary transform is applied to the lower K ( ${rm K}={4}$ or 8) frequency coefficients of the output of conventional DCT at block with dimensions 8 and larger. The proposed transform scheme has low complexity as it is applied only to the top-left portion of the DCT output, especially in the context of large blocks such as 32 $,times,$ 32 where an alternate non-DCT 32 $,times,$ 32 transform would have a prohibitive implementation hardware cost. The proposed technique is single-pass, and the choice of whether to use the secondary transform is solely based on the prediction direction for intra residue, and on transform unit location in the prediction unit for the inter residue. The scheme requires no additional signaling information or R-D search. Our simulation results show that the proposed transform scheme provides significant BD-rate improvement over the conventional DCT-based coding scheme. Finally, we also show how to implement the proposed secondary transforms with low latency in hardware.

Beschreibung: In this paper, we describe a novel algorithm for unsupervised segmentation of images with low depth of field (DOF). First of all, a multi-scale reblurring model is used to detect the object of interest (OOI) in saliency space. Then, to determine the boundary of OOI, an active contour model based on hybrid energy function is proposed. In this model, a global energy item related with the saliency map is adopted to find the global minimum, and a local energy term regarding the low DOF image is used to improve the segmentation precision. In addition, an adaptive parameter is attached to this model to balance the weight of global and local energy. Furthermore, an unsupervised curve initialization method is designed to reduce the number of evolution iterations. Finally, we conduct experiments on various low DOF images, and the results demonstrate the high robustness and precision of the proposed approach.

Beschreibung: In this paper, we propose a novel method that treats pose estimation as a problem with the constraints of human segmentation consistency from single images. Different from the previous paper, we integrate pose estimation and object segmentation into a joint optimization. With the support of segmentation consistency, we can obtain more reliable pose results. Through analyzing the energy function of pose estimation and human segmentation, we convert the pose estimation into a binary optimization problem that has the same formation as segmentation. The top-down pose shape cues, bottom-up visual cues, and the consistency constraints that penalize the mismatching of pose and human foreground are incorporated into our final objective function. Qualitative and quantitative experimental results demonstrate the merits of our method in pose estimation on Ramanan benchmark and Buffy data sets.

Beschreibung: A real-time and accurate object detection framework, ${rm C}^{4}$ , is proposed in this paper. ${rm C}^{4}$ achieves 20 fps speed and the state-of-the-art detection accuracy, using only one processing thread without resorting to special hardware such as GPU. The real-time accurate object detection is made possible by two contributions. First, we conjecture (with supporting experiments) that contour is what we should capture and signs of comparisons among neighboring pixels are the key information to capture contour cues. Second, we show that the CENTRIST visual descriptor is suitable for contour based object detection, because it encodes the sign information and can implicitly represent the global contour. When CENTRIST and linear classifier are used, we propose a computational method that does not need to explicitly generate feature vectors. It involves no image preprocessing or feature vector normalization, and only requires $O(1)$ steps to test an image patch. ${rm C}^{4}$ is also friendly to further hardware acceleration. It has been applied to detect objects such as pedestrians, faces, and cars on benchmark data sets. It has comparable detection accuracy with state-of-the-art methods, and has a clear advantage in detection speed.

Beschreibung: Previous image clutter metrics were proposed on the thought that clutter was just a perceptual effect, while we identify clutter as both perceptual and cognitive effects. Under this identification, we give a new definition of image clutter metric by analyzing the research results in the fields of visual psychology and psychophysics. According to the definition, we further put forward a DisSIMilarity (DSIM) based image clutter metric, which can also be taken as a kind of HVS-based signal-to-clutter ratio. The earlier image clutter metrics produced limited success in predicting targeting performance mainly since they did not consider brain cognitive characteristics. We develop a brain cognitive dissimilarity measure (BCDM) as a quantitative estimate of the selection weights which are allocated by brain attentional mechanism to affect visual selection processes. A human vision perceptual dissimilarity measure (VPDM), fully embodying vision perceptual properties, is first established between the target and clutter images, and then we utilize the BCDM between the two images as selection weights to pool the VPDM to be a clutter metric, which can be called DSIM metric. The metric is tested in Search_2 dataset provided by TNO Human Factors Research Institute of Netherlands. Error analysis and correlation tests demonstrate that the DSIM metric makes a more significant improvement than previously proposed metrics in predicting 62 observers' targeting performances including detection probability, false alarm probability and search time.

Beschreibung: Despite continuous improvements in optical flow in the last three decades, the ability for optical flow algorithms to handle illumination variation is still an unsolved challenge. To improve the ability to interpret apparent object motion in video containing illumination variation, an illumination-robust optical flow method is designed. This method decouples brightness into reflectance and illumination components using a stochastic technique; reflectance is given higher weight to ensure robustness against illumination, which is suppressed. Illumination experiments using the Middlebury and University of Oulu databases demonstrate the decoupled method's improvement when compared with state-of-the-art. In addition, a novel technique is implemented to visualize optical flow output, which is especially useful to compare different optical flow methods in the absence of the ground truth.

Beschreibung: Accurate grain segmentation on 3D superalloy images is very important in materials science and engineering. From grain segmentation, we can derive the underlying superalloy grains' micro-structures, based on how many important physical, mechanical, and chemical properties of the superalloy samples can be evaluated. Grain segmentation is, however, usually a very challenging problem because: 1) even a small 3D superalloy sample may contain hundreds of grains; 2) carbides and noises may degrade the imaging quality; and 3) the intensity within a grain may not be homogeneous. In addition, the same grain may present different appearances, e.g., different intensities, under different microscope settings. In practice, a 3D superalloy image may contain multichannel information where each channel corresponds to a specific microscope setting. In this paper, we develop a multichannel edge-weighted centroidal Voronoi tessellation (MCEWCVT) algorithm to effectively and robustly segment the superalloy grains from 3D multichannel superalloy images. MCEWCVT performs segmentation by minimizing an energy function, which encodes both the multichannel voxel-intensity similarity within each cluster in the intensity domain and the smoothness of segmentation boundaries in the 3D image domain. In the experiment, we first quantitatively evaluate the proposed MCEWCVT algorithm on a four-channel Ni-based 3D superalloy data set (IN100) against the manually annotated ground-truth segmentation. We further evaluate the MCEWCVT algorithm on two synthesized four-channel superalloy data sets. The qualitative and quantitative comparisons of 18 existing image segmentation algorithms demonstrate the effectiveness and robustness of the proposed MCEWCVT algorithm.

Beschreibung: Local binary pattern (LBP) is sensitive to noise. Local ternary pattern (LTP) partially solves this problem. Both LBP and LTP, however, treat the corrupted image patterns as they are. In view of this, we propose a noise-resistant LBP (NRLBP) to preserve the image local structures in presence of noise. The small pixel difference is vulnerable to noise. Thus, we encode it as an uncertain state first, and then determine its value based on the other bits of the LBP code. It is widely accepted that most of the image local structures are represented by uniform codes and noise patterns most likely fall into the non-uniform codes. Therefore, we assign the value of an uncertain bit hence as to form possible uniform codes. Thus, we develop an error-correction mechanism to recover the distorted image patterns. In addition, we find that some image patterns such as lines are not captured in uniform codes. Those line patterns may appear less frequently than uniform codes, but they represent a set of important local primitives for pattern recognition. Thus, we propose an extended noise-resistant LBP (ENRLBP) to capture line patterns. The proposed NRLBP and ENRLBP are more resistant to noise compared with LBP, LTP, and many other variants. On various applications, the proposed NRLBP and ENRLBP demonstrate superior performance to LBP/LTP variants.

Beschreibung: To overcome the difficulties in applying gradient-based operators to color images, Di Zenzo introduced the color tensor, an operator that provides a gradient field for multichannel images. An elegant application for this operator was developed in the domain of multichannel image visualization: Socolinsky and Wolff proposed to reintegrate Di Zenzo's gradient by solving a Poisson equation, yielding a greyscale representation of the multispectral contrast of the input image. Di Zenzo's gradients are, however, generally not integrable and some approximation must be introduced. Thus, the resulting image can suffer from artifacts such as the smearing of edges. In this paper, we focus on the integrability of Di Zenzo's gradients. We show that the integrability of the obtained field can be improved dramatically through a simple desaturation of the color image (as in the HSV color space). This result can be readily extended to multispectral images by defining an analogue to saturation. We present several results explaining what happens to color tensors as the saturation changes. Significantly we show that small changes of the saturation in the linear image space can result in large improvements in the integrability of tensor gradients calculated in logarithmic color space. This result is important for two reasons. 1) Log-differences are more perceptually meaningful. 2) In log-space we can operate with retinex algorithms, which are well known techniques for contrast enhancement. We propose that they can be used to “put back” any contrast that might be lost in the desaturation step and, more importantly, they can enhance contrast at the same time as reintegrating the gradient field because of their relation to partial differential equations. Finally, we evaluate our method psychophysically. Compared with other commonly used image fusion methods, experiments show that our data fusion using the Di Zenzo color tensor after desaturating the image and where a si- ple contrast boost is applied is strongly preferred.

Beschreibung: Water reflection, a typical imperfect reflection symmetry problem, plays an important role in image content analysis. Existing techniques of symmetry recognition, however, cannot recognize water reflection images correctly because of the complex and various distortions caused by the water wave. Hence, we propose a novel water reflection recognition technique to solve the problem. First, we construct a novel feature space composed of motion blur invariant moments in low-frequency curvelet space and of curvelet coefficients in high-frequency curvelet space. Second, we propose an efficient algorithm including two sub-algorithms: low-frequency reflection cost minimization and high-frequency curvelet coefficients discrimination to classify water reflection images and to determine the reflection axis. Through experimenting on authentic images in a series of tasks, the proposed techniques prove effective and reliable in classifying water reflection images and detecting the reflection axis, as well as in retrieving images with water reflection.

Beschreibung: Poisson inverse problems arise in many modern imaging applications, including biomedical and astronomical ones. The main challenge is to obtain an estimate of the underlying image from a set of measurements degraded by a linear operator and further corrupted by Poisson noise. In this paper, we propose an efficient framework for Poisson image reconstruction, under a regularization approach, which depends on matrix-valued regularization operators. In particular, the employed regularizers involve the Hessian as the regularization operator and Schatten matrix norms as the potential functions. For the solution of the problem, we propose two optimization algorithms that are specifically tailored to the Poisson nature of the noise. These algorithms are based on an augmented-Lagrangian formulation of the problem and correspond to two variants of the alternating direction method of multipliers. Further, we derive a link that relates the proximal map of an $ell_{p}$ norm with the proximal map of a Schatten matrix norm of order $p$ . This link plays a key role in the development of one of the proposed algorithms. Finally, we provide experimental results on natural and biological images for the task of Poisson image deblurring and demonstrate the practical relevance and effectiveness of the proposed framework.

Beschreibung: In this paper, we propose an object-aware holistic superpixel selection (HPS) method to automatically select the discriminative superpixels of an image for image classification purpose. Through only considering the selected superpixels, the interference of cluttered background on the object can be alleviated effectively and thus the classification performance is significantly enhanced. In particular, for an image, HPS first selects the discriminative superpixels for the characteristics of certain class, which can together match the object template of this class well. In addition, these superpixels compose a class-specific matching region. Through performing such superpixel selection for several most probable classes, respectively, HPS generates multiple class-specific matching regions for a single image. Then, HPS merges these matching regions into an integral object region through exploiting their pixel-level intersection information. Finally, such object region instead of the original image is used for image classification. An appealing advantage of HPS is the ability to alleviate the interference of cluttered background yet not require the object to be segmented out accurately. We evaluate the proposed HPS on four challenging image classification benchmark datasets: Oxford-IIIT PET 37, Caltech-UCSD Birds 200, Caltech 101, and PASCAL VOC 2011. The experimental results consistently show that the proposed HPS can remarkably improve the classification performance.

Beschreibung: This paper proposes a novel image-based framework to manipulate the illumination of human face through adaptive layer decomposition. According to our framework, only a single reference image, without any knowledge of the 3D geometry or material information of the input face, is needed. To transfer the illumination effects of a reference face image to a normal lighting face, we first decompose the lightness layers of the reference and the input images into large-scale and detail layers through weighted least squares (WLS) filter with adaptive smoothing parameters according to the gradient values of the face images. The large-scale layer of the reference image is filtered with the guidance of the input image by guided filter with adaptive smoothing parameters according to the face structures. The relit result is obtained by replacing the largescale layer of the input image with that of the reference image. To normalize the illumination effects of a non-normal lighting face (i.e., face delighting), we introduce similar reflectance prior to the layer decomposition stage by WLS filter, which make the normalized result less affected by the high contrast light and shadow effects of the input face. Through these two procedures, we can change the illumination effects of a non-normal lighting face by first normalizing the illumination and then transferring the illumination of another reference face to it. We acquire convincing relit results of both face relighting and delighting on numerous input and reference face images with various illumination effects and genders. Comparisons with previous papers show that our framework is less affected by geometry differences and can preserve better the identification structure and skin color of the input face.

Beschreibung: In this paper a contrast-guided image interpolation method is proposed that incorporates contrast information into the image interpolation process. Given the image under interpolation, four binary contrast-guided decision maps (CDMs) are generated and used to guide the interpolation filtering through two sequential stages: 1) the 45 $^{circ}$ and 135 $^{circ}$ CDMs for interpolating the diagonal pixels and 2) the 0 $^{circ}$ and 90 $^{circ}$ CDMs for interpolating the row and column pixels. After applying edge detection to the input image, the generation of a CDM lies in evaluating those nearby non-edge pixels of each detected edge for re-classifying them possibly as edge pixels. This decision is realized by solving two generalized diffusion equations over the computed directional variation (DV) fields using a derived numerical approach to diffuse or spread the contrast boundaries or edges, respectively. The amount of diffusion or spreading is proportional to the amount of local contrast measured at each detected edge. The diffused DV fields are then thresholded for yielding the binary CDMs, respectively. Therefore, the decision bands with variable widths will be created on each CDM. The two CDMs generated in each stage will be exploited as the guidance maps to conduct the interpolation process: for each declared edge pixel on the CDM, a 1-D directional filtering will be applied to estimate its associated to-be-interpolated pixel along the direction as indicated by the respective CDM; otherwise, a 2-D directionless or isotropic filtering will be used instead to estimate the associated missing pixels for each declared non-edge pixel. Extensive simulation results have clearly shown that the proposed contra- t-guided image interpolation is superior to other state-of-the-art edge-guided image interpolation methods. In addition, the computational complexity is relatively low when compared with existing methods; hence, it is fairly attractive for real-time image applications.

Beschreibung: We address the following subspace learning problem: supposing we are given a set of labeled, corrupted training data points, how to learn the underlying subspace, which contains three components: an intrinsic subspace that captures certain desired properties of a data set, a penalty subspace that fits the undesired properties of the data, and an error container that models the gross corruptions possibly existing in the data. Given a set of data points, these three components can be learned by solving a nuclear norm regularized optimization problem, which is convex and can be efficiently solved in polynomial time. Using the method as a tool, we propose a new discriminant analysis (i.e., supervised subspace learning) algorithm called Corruptions Tolerant Discriminant Analysis (CTDA), in which the intrinsic subspace is used to capture the features with high within-class similarity, the penalty subspace takes the role of modeling the undesired features with high between-class similarity, and the error container takes charge of fitting the possible corruptions in the data. We show that CTDA can well handle the gross corruptions possibly existing in the training data, whereas previous linear discriminant analysis algorithms arguably fail in such a setting. Extensive experiments conducted on two benchmark human face data sets and one object recognition data set show that CTDA outperforms the related algorithms.

Beschreibung: Image quality metrics (IQMs), such as the mean squared error (MSE) and the structural similarity index (SSIM), are quantitative measures to approximate perceived visual quality. In this paper, through analyzing the relationship between the MSE and the SSIM under an additive noise distortion model, we propose a perceptually relevant MSE-based IQM, MSE-SSIM, which is expressed in terms of the variance of the source image and the MSE between the source and distorted images. Evaluations on three publicly available databases (LIVE, CSIQ, and TID2008) show that the proposed metric, despite requiring less computation, compares favourably in performance to several existing IQMs. In addition, due to its simplicity, MSE-SSIM is amenable for the use in a wide range of image and video tasks that involve solving an optimization problem. As an example, MSE-SSIM is used as the objective function in designing a Wiener filter that aims at optimizing the perceptual visual quality of the output. Experimental results show that the images filtered with a MSE-SSIM-optimal Wiener filter have better visual quality than those filtered with a MSE-optimal Wiener filter.

Beschreibung: In this paper, we propose a unified energy minimization model for segmentation of non-smooth image structures, e.g., textures, based on Mumford–Shah functional and linear patch model. We consider that image patches of a non-smooth image structure can be modeled by a patch subspace, and image patches of different non-smooth image structures belong to different patch subspaces, which leads to a computational framework for segmentation of non-smooth image structures. Motivated by the Mumford–Shah model, we show that this segmentation framework is equivalent to minimizing a piecewise linear patch reconstruction energy. We also prove that the error of segmentation is bounded by the error of the linear patch reconstruction, meaning that improving the linear patch reconstruction for each region leads to reduction of the segmentation error. In addition, we derive an algorithm for the linear patch reconstruction with proven global optimality and linear rate of convergence. The segmentation in our method is achieved by minimizing a single energy functional without requiring predefined features. Hence, compared with the previous methods that require predefined texture features, our method can be more suitable for handling general textures in unsupervised segmentation. As a by-product, our method also produces a dictionary of optimized orthonormal descriptors for each segmented region. We mainly evaluate our method on the Brodatz textures. The experiments validate our theoretical claims and show the clear superior performance of our methods over other related methods for segmentation of the textures.

Beschreibung: Look-up table (LUT) halftoning is an efficient way to construct halftone images and approximately simulate the dot distribution of the learned halftone image set. In this paper, a general mechanism named multiple look-up table (MLUT) halftoning is proposed to generate the halftones of direct binary search (DBS), whereas the high efficient characteristic of the LUT is still preserved. In the MLUT, the standard deviation is adopted as an important feature to classify various tables. In addition, the proposed quick standard deviation evaluation is employed to yield an extremely low computational complexity in calculating the standard deviation. In the parameter optimization, the autocorrelation is adopted because it can fully characterize the periodicity of dot distribution. Experimental results demonstrate that the dot distribution generated by the proposed method approximates to that of the DBS, which enables the proposed scheme as a very competitive candidate in the copying and printing industry.

Beschreibung: Due to several sources of multipath in through-wall radar sensing, such as walls, floors, and ceilings, there could exist multipath ghosts associated with a few genuine targets in the synthetic aperture beamformed image. The multipath ghosts are false positives and therefore confusable with genuine targets. Here, we develop a multipath exploitation technique using point spread functions, which associate and map back the multipath ghosts to their genuine targets, thereby increasing the effective signal-to-clutter ratio (SCR) at the genuine target locations. To do so, we first develop a multipath model advocating the Householder transformation, which permits modeling multiple reflections at multiple walls, and also allows for unconventional room/building geometries. Second, closed-form solutions of the multipath ghost locations assuming free space propagation are derived. Third, a nonlinear least squares optimization is formulated and initialized with these free space solutions to localize the multipath ghosts in through-wall radar sensing. The exploitation approach is general and does not require a priori assumptions on the number of targets. The free space multipath ghost locations and exploitation technique derived here may be used as is for multipath exploitation in urban canyons via synthetic aperture radar. Analytical expressions quantifying the SCR gain after multipath exploitation are derived. The analysis is validated with experimental EM results using finite-difference time-domain simulations.

Beschreibung: Image recovery from undersampled data has always been challenging due to its implicit ill-posed nature but becomes fascinating with the emerging compressed sensing (CS) theory. This paper proposes a novel gradient based dictionary learning method for image recovery, which effectively integrates the popular total variation (TV) and dictionary learning technique into the same framework. Specifically, we first train dictionaries from the horizontal and vertical gradients of the image and then reconstruct the desired image using the sparse representations of both derivatives. The proposed method enables local features in the gradient images to be captured effectively, and can be viewed as an adaptive extension of the TV regularization. The results of various experiments on MR images consistently demonstrate that the proposed algorithm efficiently recovers images and presents advantages over the current leading CS reconstruction approaches.

Beschreibung: Video based human body pose estimation seeks to estimate the human body pose from an image or a video sequence, which captures a person exhibiting some activities. To handle noise and occlusion, a pose prior model is often constructed and is subsequently combined with the pose estimated from the image data to achieve a more robust body pose tracking. Various body prior models have been proposed. Most of them are data-driven, typically learned from 3D motion capture data. In addition to being expensive and time-consuming to collect, these data-based prior models cannot generalize well to activities and subjects not present in the motion capture data. To alleviate this problem, we propose to learn the prior model from anatomic, biomechanics, and physical constraints, rather than from the motion capture data. For this, we propose methods that can effectively capture different types of constraints and systematically encode them into the prior model. Experiments on benchmark data sets show the proposed prior model, compared with data-based prior models, achieves comparable performance for body motions that are present in the training data. It, however, significantly outperforms the data-based prior models in generalization to different body motions and to different subjects.

Beschreibung: This paper presents a new automatic image annotation algorithm. First, we introduce a new similarity measure between images: compactness. This uses low level visual descriptors for determining the similarity between two images. Compactness shows how close test image features lie to training image feature cluster centers. The measure provides the core for a k-nearest neighbor type image annotation method. Afterward, a formalism for defining different transfer techniques is devised and several label transfer techniques are provided. The method as whole is evaluated on four image annotation benchmarks. The results on these sets validate the accuracy of the approach, which outperforms many state-of-the-art annotation methods. The method presented here requires a simple training process, efficiently combines different feature types and performs better than complex learning algorithms, even in this incipient form. The main contributions of this paper are the usage of compactness as a similarity measure that enables efficient low level feature comparison and an annotation algorithm based on label transfer.

Beschreibung: This paper introduces a novel method of image reconstruction from a finite number of projections by processing the image along parallel rays. The geometry from the image plane is transferred to the Cartesian lattice by means of using the original image's line-integrals to calculate the line-sums of the discrete image to be reconstructed. Such a transformation of geometry allows for the 2D discrete paired transform, whose complete set of functions is defined by directions, to be effectively used in the exact reconstruction of the original image. The model of image reconstruction is described, and both examples and experimental results of implementation of the proposed method are provided for reconstruction on the Cartesian lattice of size $2^{r}times 2^{r}$ , where $rgeq 2$ .

Beschreibung: Compressed sensing (CS) is an important theory for sub-Nyquist sampling and recovery of compressible data. Recently, it has been extended to cope with the case where corruption to the CS data is modeled as impulsive noise. The new formulation, termed as robust CS, combines robust statistics and CS into a single framework to suppress outliers in the CS recovery. To solve the newly formulated robust CS problem, a scheme that iteratively solves a number of CS problems—the solutions from which provably converge to the true robust CS solution—is suggested. This scheme is, however, rather inefficient as it has to use existing CS solvers as a proxy. To overcome limitations with the original robust CS algorithm, we propose in this paper more computationally efficient algorithms by following latest advances in large-scale convex optimization for nonsmooth regularization. Furthermore, we also extend the robust CS formulation to various settings, including additional affine constraints, $ell_{1}$ -norm loss function, mix-norm regularization, and multitasking, so as to further improve robust CS and derive simple but effective algorithms to solve these extensions. We demonstrate that the new algorithms provide much better computational advantage over the original robust CS method on the original robust CS formulation, and effectively solve more sophisticated extensions where the original methods simply cannot. We demonstrate the usefulness of the extensions on several imaging tasks.

Beschreibung: Recently, the median filtering (MF) detector as a forensic tool for the recovery of images' processing history has attracted wide interest. This paper presents a novel method for the blind detection of MF in digital images. Following some strongly indicative analyses in the difference domain of images, we introduce two new feature sets that allow us to distinguish a median-filtered image from an untouched image or average-filtered one. The effectiveness of the proposed features is verified with evidence from exhaustive experiments on a large composite image database. Compared with prior arts, the proposed method achieves significant performance improvement in the case of low resolution and strong JPEG post-compression. In addition, it is demonstrated that our method is more robust against additive noise than other existing MF detectors. With analyses and extensive experimental researches presented in this paper, we hope that the proposed method will add a new tool to the arsenal of forensic analysts.

Beschreibung: Most tracking-by-detection algorithms train discriminative classifiers to separate target objects from their surrounding background. In this setting, noisy samples are likely to be included when they are not properly sampled, thereby causing visual drift. The multiple instance learning (MIL) paradigm has been recently applied to alleviate this problem. However, important prior information of instance labels and the most correct positive instance (i.e., the tracking result in the current frame) can be exploited using a novel formulation much simpler than an MIL approach. In this paper, we show that integrating such prior information into a supervised learning algorithm can handle visual drift more effectively and efficiently than the existing MIL tracker. We present an online discriminative feature selection algorithm that optimizes the objective function in the steepest ascent direction with respect to the positive samples while in the steepest descent direction with respect to the negative ones. Therefore, the trained classifier directly couples its score with the importance of samples, leading to a more robust and efficient tracker. Numerous experimental evaluations with state-of-the-art algorithms on challenging sequences demonstrate the merits of the proposed algorithm.

Beschreibung: Multilevel thresholding amounts to segmenting a gray-level image into several distinct regions. This paper presents a 2D histogram based multilevel thresholding approach to improve the separation between objects. Recent studies indicate that the results obtained with 2D histogram oriented approaches are superior to those obtained with 1D histogram based techniques in the context of bi-level thresholding. Here, a method to incorporate 2D histogram related information for generalized multilevel thresholding is proposed using the maximum Tsallis entropy. Differential evolution (DE), a simple yet efficient evolutionary algorithm of current interest, is employed to improve the computational efficiency of the proposed method. The performance of DE is investigated extensively through comparison with other well-known nature inspired global optimization techniques such as genetic algorithm, particle swarm optimization, artificial bee colony, and simulated annealing. In addition, the outcome of the proposed method is evaluated using a well known benchmark—the Berkley segmentation data set (BSDS300) with 300 distinct images.

Beschreibung: This paper initially develops the discrete-point sampling operator's concept, model, and parameters that we have previously proposed, and makes its belt-shaped regions in a discrete-point sampling map more salient and appropriate for centerline extraction. The cross-sectional features of these belt-shaped regions are then analyzed and seven types of feature points are defined to facilitate descriptions of such features. Based on these feature points, a three-level detection system is proposed, including feature points, line segments, and centerlines, to extract centerlines from the belt-shaped regions. Eight basic types of centerlines and five types of relationships among the centerlines are defined by computational geometry algorithms, and Gestalt laws are used to cluster them into groupings. If some prior information about a desired shape is available, retrieval grouping may be carried out by a discrete-point sampling map, the purpose of which is to find centerlines by best matching with prior information. Discrete-point sampling effectually overcomes the influences of interference from noise, textures, and uneven illumination, and greatly reduces the difficulty of centerline extraction. Centerline clustered groupings and retrieval grouping can offer a strong anti-interference ability with nonlinear deformations such as articulation and occlusion. This method can extract large-scale complex shapes combined of lines and planes from complex images. The wheel location results of noise test and other shape extraction experiments show that our method has a strong capability to persist with nonlinear deformations.

Beschreibung: In this paper, we propose a novel feature adaptive co-segmentation method that can learn adaptive features of different image groups for accurate common objects segmentation. We also propose image complexity awareness for adaptive feature learning. In the proposed method, the original images are first ranked according to the image complexities that are measured by superpixel changing cue and object detection cue. Then, the unsupervised segments of the simple images are used to learn the adaptive features, which are achieved using an expectation-minimization algorithm combining $l$ 1-regularized least squares optimization with the consideration of the confidence of the simple image segmentation accuracies and the fitness of the learned model. The error rate of the final co-segmentation is tested by the experiments on different image groups and verified to be lower than the existing state-of-the-art co-segmentation methods.

Beschreibung: This paper proposes a new super-resolution (SR) scheme for landmark images by retrieving correlated web images. Using correlated web images significantly improves the exemplar-based SR. Given a low-resolution (LR) image, we extract local descriptors from its up-sampled version and bundle the descriptors according to their spatial relationship to retrieve correlated high-resolution (HR) images from the web. Though similar in content, the retrieved images are usually taken with different illumination, focal lengths, and shot perspectives, resulting in uncertainty for the HR detail approximation. To solve this problem, we first propose aligning these images to the up-sampled LR image through a global registration, which identifies the corresponding regions in these images and reduces the mismatching. Second, we propose a structure-aware matching criterion and adaptive block sizes to improve the mapping accuracy between LR and HR patches. Finally, these matched HR patches are blended together by solving an energy minimization problem to recover the desired HR image. Experimental results demonstrate that our SR scheme achieves significant improvement compared with four state-of-the-art schemes in terms of both subjective and objective qualities.

Beschreibung: Multidimensional linear phase perfect reconstruction filter bank (MDLPPRFB) can be designed and implemented via lattice structure. The lattice structure for the MDLPPRFB with filter support ${cal N}({rm M}Xi)$ has been published by Muramatsu , where M is the decimation matrix, ${Xi}$ is a positive integer diagonal matrix, and ${cal N}({rm N})$ denotes the set of integer vectors in the fundamental parallelepiped of the matrix N. Obviously, if ${Xi}$ is chosen to be other positive diagonal matrices instead of only positive integer ones, the corresponding lattice structure would provide more choices of filter banks, offering better trade-off between filter support and filter performance. We call such resulted filter bank as generalized-support MDLPPRFB (GSMDLPPRFB). The lattice structure for GSMDLPPRFB, however, cannot be designed by simply generalizing the process that Muramatsu employed. Furthermore, the related theories to assist the design also become different from those used by Muramatsu . Such issues will be addressed in this paper. To guide the design of GSMDLPPRFB, the necessary and sufficient conditions are established for a generalized-support multidimensional filter bank to be linear-phase. To determine the cases we can find a GSMDLPPRFB, the necessary conditions about the existence of it are proposed to be related with filter support and symmetry polarity (i.e., the number of symmetric filters $n_{s}$ and antisymmetric filters $n_{a}$ ). Based on a process (different from the one Muramatsu used) that combines several polyphase - atrices to construct the starting block, one of the core building blocks of lattice structure, the lattice structure for GSMDLPPRFB is developed and shown to be minimal. Additionally, the result in this paper includes Muramatsu's as a special case.

Beschreibung: We present a new upsampling method to enhance the spatial resolution of depth images. Given a low-resolution depth image from an active depth sensor and a potentially high-resolution color image from a passive RGB camera, we formulate it as an adaptive cost aggregation problem and solve it using the bilateral filter. The formulation synergistically combines the median and bilateral filters thus it better preserves the depth edges and is more robust to noise. Numerical and visual evaluations on a total of 37 Middlebury data sets demonstrate the effectiveness of our method. A real-time high-resolution depth capturing system is also developed using commercial active depth sensor based on the proposed upsampling method.

Beschreibung: Multiresolution strategies are commonly used in the nonrigid registration to avoid local minima in the optimization space. Generally, a step-by-step hierarchical approach is adopted, in which the registration starts on a level with reduced complexity (downsampled images, global transformations), then continuing to levels with increased complexity, until the finest level is reached. In this paper, we propose two alternative multiresolution strategies for both the data and transformation models, in which different resolution levels are considered simultaneously instead of subsequently. Through combining the different strategies for data and transformation, we systematically define 3 $,times,$ 3 multiresolution schemes, including both existing and novel methods. Experiments on 10 pairs of computed tomography lung data sets showed that the best performing strategy resulted in a reduction of the upper quartile of the mean target registration error from 2 to 1.5 mm, compared with the conventionally hierarchical multiresolution method, while achieving smoother deformations. Experiments with intersubject registration of 18 3D T1-weighted MRI brain scans confirmed that simultaneous multiresolution strategies produce more accurate registration results (median of mean overlap increased from 0.55 to 0.57) and smoother deformation fields than the traditionally hierarchical method. Evaluation of robustness indicated that the largest differences in accuracy between methods are observed for structures with a relatively large initial misalignment.

Beschreibung: Estimating the amount of blur in a given image is important for computer vision applications. More specifically, the spatially varying defocus point-spread-functions (PSFs) over an image reveal geometric information of the scene, and their estimate can also be used to recover an all-in-focus image. A PSF for a defocus blur can be specified by a single parameter indicating its scale. Most existing algorithms can only select an optimal blur from a finite set of candidate PSFs for each pixel. Some of those methods require a coded aperture filter inserted in the camera. In this paper, we present an algorithm estimating a defocus scale map from a single image, which is applicable to conventional cameras. This method is capable of measuring the probability of local defocus scale in the continuous domain. It also takes smoothness and color edge information into consideration to generate a coherent blur map indicating the amount of blur at each pixel. Simulated and real data experiments illustrate excellent performance and its successful applications in foreground/background segmentation.

Beschreibung: A model of visual masking, which reveals the visibility of stimuli in the human visual system (HVS), is useful in perceptual based image/video processing. The existing visual masking function mainly considers luminance contrast, which always overestimates the visibility threshold of the edge region and underestimates that of the texture region. Recent research on visual perception indicates that the HVS is sensitive to orderly regions that possess regular structures and insensitive to disorderly regions that possess uncertain structures. Therefore, structural uncertainty is another determining factor on visual masking. In this paper, we introduce a novel pattern masking function based on both luminance contrast and structural uncertainty. Through mimicking the internal generative mechanism of the HVS, a prediction model is firstly employed to separate out the unpredictable uncertainty from an input image. In addition, an improved local binary pattern is introduced to compute the structural uncertainty. Finally, combining luminance contrast with structural uncertainty, the pattern masking function is deduced. Experimental result demonstrates that the proposed pattern masking function outperforms the existing visual masking function. Furthermore, we extend the pattern masking function to just noticeable difference (JND) estimation and introduce a novel pixel domain JND model. Subjective viewing test confirms that the proposed JND model is more consistent with the HVS than the existing JND models.

Beschreibung: A new method for video watermarking is presented in this paper. In the proposed method, data are embedded in the LL subband of wavelet coefficients, and decoding is performed based on the comparison among the elements of the first principal component resulting from empirical principal component analysis (PCA). The locations for data embedding are selected such that they offer the most robust PCA-based decoding. Data are inserted in the LL subband in an adaptive manner based on the energy of high frequency subbands and visual saliency. Extensive testing was performed under various types of attacks, such as spatial attacks (uniform and Gaussian noise and median filtering), compression attacks (MPEG-2, H. 263, and H. 264), and temporal attacks (frame repetition, frame averaging, frame swapping, and frame rate conversion). The results show that the proposed method offers improved performance compared with several methods from the literature, especially under additive noise and compression attacks.

Beschreibung: This paper addresses the problem of combining multiple radar images of the same scene to produce a more informative composite image. The proposed approach for probabilistic fuzzy logic-based image fusion automatically forms fuzzy membership functions using the Gaussian-Rayleigh mixture distribution. It fuses the input pixel values directly without requiring fuzzification and defuzzification, thereby removing the subjective nature of the existing fuzzy logic methods. In this paper, the proposed approach is applied to through-the-wall radar imaging in urban sensing and evaluated on real multi-view and polarimetric data. Experimental results show that the proposed approach yields improved image contrast and enhances target detection.

Beschreibung: The sensitivity of the human visual system decreases dramatically with increasing distance from the fixation location in a video frame. Accurate prediction of a viewer's gaze location has the potential to improve bit allocation, rate control, error resilience, and quality evaluation in video compression. Commercially, delivery of football video content is of great interest because of the very high number of consumers. In this paper, we propose a gaze location prediction system for high definition broadcast football video. The proposed system uses knowledge about the context, extracted through analysis of a gaze tracking study that we performed, to build a suitable prior map. We further classify the complex context into different categories through shot classification thus allowing our model to prelearn the task pertinence of each object category and build the prior map automatically. We thus avoid the limitation of assigning the viewers a specific task, allowing our gaze prediction system to work under free-viewing conditions. Bayesian integration of bottom-up features and top-down priors is finally applied to predict the gaze locations. Results show that the prediction performance of the proposed model is better than that of other top-down models that we adapted to this context.

Beschreibung: Reconstruction of unknown objects by microwave illumination requires efficient inversion for measured electromagnetic scattering data. In the integral equation approach for reconstructing dielectric objects based on the Born iterative method or its variations, the volume integral equations are involved because the imaging domain is fully inhomogeneous. When solving the forward scattering integral equation, the Nyström method is used because the traditional method of moments may be inconvenient due to the inhomogeneity of the imaging domain. The benefits of the Nyström method include the simple implementation without using any basis and testing functions and low requirement on geometrical discretization. When solving the inverse scattering integral equation, the Gauss–Newton minimization approach with a line search method (LSM) and multiplicative regularization method (MRM) is employed. The LSM can optimize the search of step size in each iteration, whereas the MRM may reduce the number of numerical experiments for choosing the regularization parameter. Numerical examples for reconstructing typical dielectric objects under limited observation angles are presented to illustrate the inversion approach.

Beschreibung: Segmentation based tracing algorithms detect the extent and borders of an object in a given frame $I_{Z}$ by propagating results from frames $I_{1leq z〈Z}$ . Although application specific tracers have been forthcoming, techniques that automatically adapt across applications have been less explored. We approach this problem by learning a prior model on topological dynamics that encourages segmentation transitions across frames that are most likely for a given application. Further, we augment a generic tracing technique with a locality sensitive prior derived from dense optic flow fields for deformation guidance. The proposed approach comprises two stages where the generic tracer initially yields multiple segmentation transitions when its parameters are perturbed, and the learnt topology prior subsequently propagates high scoring segmentations. Because the learnt topology model wraps around a generic tracer and adapts it by setting its free parameters, the need for careful parameter tuning is completely obviated. Through extensive experimental validation in surveillance, biological and medical image datasets, we verify the applicability of the proposed model while demonstrating good tracing performance under severe clutter.

Beschreibung: In prediction-error expansion (PEE) based reversible data hiding, better exploiting image redundancy usually leads to a superior performance. However, the correlations among prediction-errors are not considered and utilized in current PEE based methods. Specifically, in PEE, the prediction-errors are modified individually in data embedding. In this paper, to better exploit these correlations, instead of utilizing prediction-errors individually, we propose to consider every two adjacent prediction-errors jointly to generate a sequence consisting of prediction-error pairs. Then, based on the sequence and the resulting 2D prediction-error histogram, a more efficient embedding strategy, namely, pairwise PEE, can be designed to achieve an improved performance. The superiority of our method is verified through extensive experiments.

Beschreibung: In this paper, we propose an approach for object detection via structural feature selection and part-based shape model. It automatically learns a shape model from cluttered training images without need to explicitly use bounding boxes on objects. Our approach first builds a class-specific codebook of local contour features, and then generates structural feature descriptors by combining context shape information. These descriptors are robust to both within-class variations and scale changes. Through exploring pairwise image matching using fast earth mover's distance, feature weights can be iteratively updated. Those discriminative foreground features are assigned high weights and then selected to build a part-based shape model. Finally, object detection is performed by matching each testing image with this model. Experiments show that the proposed method is very effective. It has achieved comparable performance to the state-of-the-art shape-based detection methods, but requires much less training information.

Beschreibung: One of the most fundamental problems in image processing and computer vision is the inherent ambiguity that exists between texture edges and object boundaries in real-world images and video. Despite this ambiguity, many applications in computer vision and image processing often use image edge strength with the assumption that these edges approximate object depth boundaries. However, this assumption is often invalidated by real world data, and this discrepancy is a significant limitation in many of today's image processing methods. We address this issue by introducing a simple, low-level, and patch-consistency assumption that leverages the extra information present in video data to resolve this ambiguity. Through analyzing how well patches can be modeled by simple transformations over time, we can obtain an indication of which image edges correspond to texture edges versus object boundaries. Our approach is simple to implement and has the potential to improve a wide range of image and video-based applications by suppressing the detrimental effects of strong texture edges on regularization terms. We validate our approach by presenting results on a variety of scene types and directly incorporating our augmented edge map into existing image segmentation and optical flow applications, showing results that better correspond to object boundaries.

Beschreibung: The robust detection of small targets is one of the key techniques in infrared search and tracking applications. A novel small target detection method in a single infrared image is proposed in this paper. Initially, the traditional infrared image model is generalized to a new infrared patch-image model using local patch construction. Then, because of the non-local self-correlation property of the infrared background image, based on the new model small target detection is formulated as an optimization problem of recovering low-rank and sparse matrices, which is effectively solved using stable principle component pursuit. Finally, a simple adaptive segmentation method is used to segment the target image and the segmentation result can be refined by post-processing. Extensive synthetic and real data experiments show that under different clutter backgrounds the proposed method not only works more stably for different target sizes and signal-to-clutter ratio values, but also has better detection performance compared with conventional baseline methods.

Beschreibung: Recently, a new biometrics identifier, namely finger knuckle print (FKP), has been proposed for personal authentication with very interesting results. One of the advantages of FKP verification lies in its user friendliness in data collection. However, the user flexibility in positioning fingers also leads to a certain degree of pose variations in the collected query FKP images. The widely used Gabor filtering based competitive coding scheme is sensitive to such variations, resulting in many false rejections. We propose to alleviate this problem by reconstructing the query sample with a dictionary learned from the template samples in the gallery set. The reconstructed FKP image can reduce much the enlarged matching distance caused by finger pose variations; however, both the intra-class and inter-class distances will be reduced. We then propose a score level adaptive binary fusion rule to adaptively fuse the matching distances before and after reconstruction, aiming to reduce the false rejections without increasing much the false acceptances. Experimental results on the benchmark PolyU FKP database show that the proposed method significantly improves the FKP verification accuracy.

Beschreibung: Factorization-based techniques explain arrays of observations using a relatively small number of factors and provide an essential arsenal for multi-dimensional data analysis. Most factorization models are, however, developed on general arrays of scalar values. For a class of practical data arising from observing spatial signals including images, it is desirable for a model to consider general observations, e.g., handling a vector field and non-exchangeable factors, e.g., handling spatial connections between the columns and the rows of the data. In this paper, a probabilistic model for factorization is proposed. We adopt Bayesian hierarchical modeling and treat the factors as latent random variables. A Markov structure is imposed on the distribution of factors to account for the spatial connections. The model is designed to represent vector arrays sampled from fields of continuous domains. Therefore, a tailored observation model is developed to represent the link between the factor product and the data. The proposed technique has been shown effective in analyzing optical flow fields computed on both synthetic images and real-life videoclips.

Beschreibung: In a previous paper, Chang presented a method for iterative decoding of FEC-based multiple description codes in image transmission. In this correspondence, we clarify that an outer interleaver used in the above research was previously proposed for the iterative decoding and optimization of product codes in image transmission.

Beschreibung: To use interdependence between the primary components of the deformation field for smooth and non-smooth registration problems, the channel-by-channel total variation- or standard vectorial total variation (SVTV)-based regularization has been extended to a more flexible and efficient technique, allowing high quality regularization procedures. Based on this method, this paper proposes a fast nonlinear multigrid (NMG) method for solving the underlying Euler–Lagrange system of two coupled second-order nonlinear partial differential equations. Numerical experiments using both synthetic and realistic images not only confirm that the recommended VTV-based regularization yields better registration qualities for a wide range of applications than those of the SVTV-based regularization, but also that the proposed NMG method is fast, accurate, and reliable in delivering visually-pleasing registration results.

Beschreibung: The sparsity of images in a transform domain or dictionary has been exploited in many applications in image processing. For example, analytical sparsifying transforms, such as wavelets and discrete cosine transform (DCT), have been extensively used in compression standards. Recently, synthesis sparsifying dictionaries that are directly adapted to the data have become popular especially in applications such as image denoising. Following up on our recent research, where we introduced the idea of learning square sparsifying transforms, we propose here novel problem formulations for learning doubly sparse transforms for signals or image patches. These transforms are a product of a fixed, fast analytic transform such as the DCT, and an adaptive matrix constrained to be sparse. Such transforms can be learnt, stored, and implemented efficiently. We show the superior promise of our learnt transforms as compared with analytical sparsifying transforms such as the DCT for image representation. We also show promising performance in image denoising that compares favorably with approaches involving learnt synthesis dictionaries such as the K-SVD algorithm. The proposed approach is also much faster than K-SVD denoising.

Beschreibung: Block transform coded images usually suffer from annoying artifacts at low bit rates, caused by the coarse quantization of transform coefficients. In this paper, we propose a new method to reduce compression artifacts by the overlapped-block transform coefficient estimation from non-local blocks. In the proposed method, the discrete cosine transform coefficients of each block are estimated by adaptively fusing two prediction values based on their reliabilities. One prediction is the quantized values of coefficients decoded from the compressed bitstream, whose reliability is determined by quantization steps. The other prediction is the weighted average of the coefficients in nonlocal blocks, whose reliability depends on the variance of the coefficients in these blocks. The weights are used to distinguish the effectiveness of the coefficients in nonlocal blocks to predict original coefficients and are determined by block similarity in transform domain. To solve the optimization problem, the overlapped blocks are divided into several subsets. Each subset contains nonoverlapped blocks covering the whole image and is optimized independently. Therefore, the overall optimization is reduced to a set of sub-optimization problems, which can be easily solved. Finally, we provide a strategy for parameter selection based on the compression levels. Experimental results show that the proposed method can remarkably reduce compression artifacts and significantly improve both the subjective and objective qualities of block transform coded images.

Beschreibung: Switching the liquid crystal display (LCD) backlight of a portable multimedia device to a low power level saves energy but results in poor image quality especially for the low-luminance image areas. In this paper, we propose an image enhancement algorithm that overcomes such effects of dim LCD backlight by taking the human visual property into consideration. It boosts the luminance of image areas below the perceptual threshold while preserving the contrast of the other image areas. We apply the just noticeable difference theory and decompose an image into an HVS response layer and a background luminance layer. The boosting and compression processes, which enhance the visibility of the low-luminance image areas, are carried out in the background luminance layer to avoid luminance gradient rever-sal and over-compensation. The contrast of the processed image is further enhanced by exploiting the Craik-O'Brein-Cornsweet visual illusion. Experimental results are provided to show the performance of the proposed algorithm.

Beschreibung: Exploiting the sparsity within representation models for images is critical for image denoising. The best currently available denoising methods take advantage of the sparsity from image self-similarity, pre-learned, and fixed representations. Most of these methods, however, still have difficulties in tackling high noise levels or noise models other than Gaussian. In this paper, the multiresolution structure and sparsity of wavelets are employed by nonlocal dictionary learning in each decomposition level of the wavelets. Experimental results show that our proposed method outperforms two state-of-the-art image denoising algorithms on higher noise levels. Furthermore, our approach is more adaptive to the less extensively researched uniform noise.

Beschreibung: Two novel unsupervised dimensionality reduction techniques, termed sparse distance preserving embedding (SDPE) and sparse proximity preserving embedding (SPPE), are proposed for feature extraction and classification. SDPE and SPPE perform in the clean data space recovered by sparse representation and enhanced Euclidean distances over noise removed data are employed to measure pairwise similarities of points. In extracting informative features, SDPE and SPPE aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics. This paper calculates the sparsest representation of all vectors jointly by a convex optimization. The sparsest codes enable certain local information of data to be preserved, and can endow SDPE and SPPE a natural discriminating power, adaptive neighborhood and robust characteristic against noise and errors in delivering low-dimensional embeddings. We also mathematically show SDPE and SPPE can be effectively extended for discriminant learning in a supervised manner. The validity of SDPE and SPPE is examined by extensive simulations. Comparison with other related state-of-the-art unsupervised algorithms show that promising results are delivered by our techniques.

Beschreibung: Modern lossy image coding systems generate a quality progressive codestream that, truncated at increasing rates, produces an image with decreasing distortion. Quality progressivity is commonly provided by an embedded quantizer that employs uniform scalar deadzone quantization (USDQ) together with a bitplane coding strategy. This paper introduces a 2-step scalar deadzone quantization (2SDQ) scheme that achieves same coding performance as that of USDQ while reducing the coding passes and the emitted symbols of the bitplane coding engine. This serves to reduce the computational costs of the codec and/or to code high dynamic range images. The main insights behind 2SDQ are the use of two quantization step sizes that approximate wavelet coefficients with more or less precision depending on their density, and a rate-distortion optimization technique that adjusts the distortion decreases produced when coding 2SDQ indexes. The integration of 2SDQ in current codecs is straightforward. The applicability and efficiency of 2SDQ are demonstrated within the framework of JPEG2000.

Beschreibung: In pedestrian detection, as sophisticated feature descriptors are used for improving detection accuracy, its processing speed becomes a critical issue. In this paper, we propose a novel speed-up scheme based on multiple-instance pruning (MIP), one of the soft cascade methods, to enhance the processing speed of support vector machine (SVM) classifiers. Our scheme mainly consists of three steps. First, we regularly split an SVM classifier into multiple parts and build a cascade structure using them. Next, we rearrange the cascade structure for enhancing the rejection rate, and then train the rejection threshold of each stage composing the cascade structure using the MIP. To verify the validity of our scheme, we apply it to a pedestrian classifier using co-occurrence histograms of oriented gradients trained by an SVM, and experimental results show that the processing time for classification of the proposed scheme is as low as one-hundredth of the original classifier without sacrificing detection accuracy.

Beschreibung: A novel component-level dictionary learning framework that exploits image/video group characteristics based on sparse representation is introduced in this paper. Unlike the previous methods that select the dictionaries to best reconstruct the data, we present an energy minimization formulation that jointly optimizes the learning of both sparse dictionary and component-level importance within one unified framework to provide a discriminative and sparse representation for image/video groups. The importance measures how well each feature component represents the group property with the dictionary. Then, the dictionary is updated iteratively to reduce the influence of unimportant components, thus refining the sparse representation for each group. In the end, by keeping the top K important components, a compact representation is obtained for the sparse coding dictionary. Experimental results on several public image and video data sets are shown to demonstrate the superior performance of the proposed algorithm compared with the-state-of-the-art methods.

Beschreibung: In this paper, we present a syntactic string matching approach to solve the frontal face recognition problem. String matching is a powerful partial matching technique, but is not suitable for frontal face recognition due to its requirement of globally sequential representation and the complex nature of human faces, containing discontinuous and non-sequential features. Here, we build a compact syntactic Stringface representation, which is an ensemble of strings. A novel ensemble string matching approach that can perform non-sequential string matching between two Stringfaces is proposed. It is invariant to the sequential order of strings and the direction of each string. The embedded partial matching mechanism enables our method to automatically use every piece of non-occluded region, regardless of shape, in the recognition process. The encouraging results demonstrate the feasibility and effectiveness of using syntactic methods for face recognition from a single exemplar image per person, breaking the barrier that prevents string matching techniques from being used for addressing complex image recognition problems. The proposed method not only achieved significantly better performance in recognizing partially occluded faces, but also showed its ability to perform direct matching between sketch faces and photo faces.

Beschreibung: Fingerprints and palmprints are the most common authentic biometrics for personal identification, especially for forensic security. Previous research have been proposed to speed up the searching process in fingerprint and palmprint identification systems, such as those based on classification or indexing, in which the deterioration of identification accuracy is hard to avert. In this paper, a novel hierarchical minutiae matching algorithm for fingerprint and palmprint identification systems is proposed. This method decomposes the matching step into several stages and rejects many false fingerprints or palmprints on different stages, thus it can save much time while preserving a high identification rate. Experimental results show that the proposed algorithm can save almost 50% searching time compared with traditional methods and illustrate its effectiveness.

Beschreibung: Multiscale representations of images have become a standard tool in image analysis. Such representations offer a number of advantages over fixed-scale methods, including the potential for improved performance in denoising, compression, and the ability to represent distinct but complementary information that exists at various scales. A variety of multiresolution transforms exist, including both orthogonal decompositions such as wavelets as well as nonorthogonal, overcomplete representations. Recently, techniques for finding adaptive, sparse representations have yielded state-of-the-art results when applied to traditional image processing problems. Attempts at developing multiscale versions of these so-called dictionary learning models have yielded modest but encouraging results. However, none of these techniques has sought to combine a rigorous statistical formulation of the multiscale dictionary learning problem and the ability to share atoms across scales. We present a model for multiscale dictionary learning that overcomes some of the drawbacks of previous approaches by first decomposing an input into a pyramid of distinct frequency bands using a recursive filtering scheme, after which we perform dictionary learning and sparse coding on the individual levels of the resulting pyramid. The associated image model allows us to use a single set of adapted dictionary atoms that is shared—and learned—across all scales in the model. The underlying statistical model of our proposed method is fully Bayesian and allows for efficient inference of parameters, including the level of additive noise for denoising applications. We apply the proposed model to several common image processing problems including non-Gaussian and nonstationary denoising of real-world color images.

Beschreibung: In this paper, the registration problem is formulated as a point to model distance minimization. Unlike most of the existing works, which are based on minimizing a point-wise correspondence term, this formulation avoids the correspondence search that is time-consuming. In the first stage, the target set is described through an implicit function by employing a linear least squares fitting. This function can be either an implicit polynomial or an implicit B-spline from a coarse to fine representation. In the second stage, we show how the obtained implicit representation is used as an interface to convert point-to-point registration into point-to-implicit problem. Furthermore, we show that this registration distance is smooth and can be minimized through the Levengberg–Marquardt algorithm. All the formulations presented for both stages are compact and easy to implement. In addition, we show that our registration method can be handled using any implicit representation though some are coarse and others provide finer representations; hence, a tradeoff between speed and accuracy can be set by employing the right implicit function. Experimental results and comparisons in 2D and 3D show the robustness and the speed of convergence of the proposed approach.

Beschreibung: This paper aims toward improving background suppression from video frames by incorporating multiresolution features in Gaussian mixture model (GMM). GMM has proven its place for background modeling due to its better applicability and robustness compared with other popular methods in literature. However, GMM fails in a number of situations such as noisy and non-stationary background, slow foregrounds, and illumination variation. Extensions to GMM have also been proposed to increase accuracy in expense of increased complexity, decrease in execution speed, and reduced applicability. In view of the above, this paper aims to provide a methodology to assimilate useful multiresolution features with GMM that considerably improves the performance. The contributions of this paper are: 1) a novel framework to incorporate wavelet subbands in GMM to improve its performance; 2) an approach to incorporate variable number of clusters in the aforesaid framework; and 3) a generic platform to use any multiresolution decomposition based GMM for background suppression. Extensive experimentations on several video sequences are performed to verify the improvement in accuracy compared with conventional GMM as well as a number of state-of-the-arts approaches. Along with qualitative and quantitative analysis, justification on the use of multiresolution is provided for clarification.

Beschreibung: Histogram shifting (HS) is a useful technique of reversible data hiding (RDH). With HS-based RDH, high capacity and low distortion can be achieved efficiently. In this paper, we revisit the HS technique and present a general framework to construct HS-based RDH. By the proposed framework, one can get a RDH algorithm by simply designing the so-called shifting and embedding functions. Moreover, by taking specific shifting and embedding functions, we show that several RDH algorithms reported in the literature are special cases of this general construction. In addition, two novel and efficient RDH algorithms are also introduced to further demonstrate the universality and applicability of our framework. It is expected that more efficient RDH algorithms can be devised according to the proposed framework by carefully designing the shifting and embedding functions.

Beschreibung: Ellipse fitting is widely applied in the fields of computer vision and automatic industry control, in which the procedure of ellipse fitting often follows the preprocessing step of edge detection in the original image. Therefore, the ellipse fitting method also depends on the accuracy of edge detection besides their own performance, especially due to the introduced outliers and edge point errors from edge detection which will cause severe performance degradation. In this paper, we develop a robust ellipse fitting method to alleviate the influence of outliers. The proposed algorithm solves ellipse parameters by linearly combining a subset of (“more accurate”) data points (formed from edge points) rather than all data points (which contain possible outliers). In addition, considering that squaring the fitting residuals can magnify the contributions of these extreme data points, our algorithm replaces it with the absolute residuals to reduce this influence. Moreover, the norm of data point errors is bounded, and the worst case performance optimization is formed to be robust against data point errors. The resulting mixed $l1hbox{--}l2$ optimization problem is further derived as a second-order cone programming one and solved by the computationally efficient interior-point methods. Note that the fitting approach developed in this paper specifically deals with the overdetermined system, whereas the current sparse representation theory is only applied to underdetermined systems. Therefore, the proposed algorithm can be looked upon as an extended application and development of the sparse representation theory. Some simulated and experimental examples are presented to illustrate the effectiveness of the proposed ellipse fitting approach.

Beschreibung: Using shape information has gained increasing concerns in the task of image labeling. In this paper, we present a dynamic hybrid Markov random field (DHMRF), which explicitly captures middle-level object shape and low-level visual appearance (e.g., texture and color) for image labeling. Each node in DHMRF is described by either a deformable template or an appearance model as visual prototype. On the other hand, the edges encode two types of intersections: co-occurrence and spatial layered context, with respect to the labels and prototypes of connected nodes. To learn the DHMRF model, an iterative algorithm is designed to automatically select the most informative features and estimate model parameters. The algorithm achieves high computational efficiency since a branch-and-bound schema is introduced to estimate model parameters. Compared with previous methods, which usually employ implicit shape cues, our DHMRF model seamlessly integrates color, texture, and shape cues to inference labeling output, and thus produces more accurate and reliable results. Extensive experiments validate its superiority over other state-of-the-art methods in terms of recognition accuracy and implementation efficiency on: 1) the MSRC 21-class dataset, and 2) the lotus hill institute 15-class dataset.

Beschreibung: In this paper, we present an approach that simultaneously clusters images and learns dictionaries from the clusters. The method learns dictionaries and clusters images in the radon transform domain. The main feature of the proposed approach is that it provides both in-plane rotation and scale invariant clustering, which is useful in numerous applications, including content-based image retrieval (CBIR). We demonstrate the effectiveness of our rotation and scale invariant clustering method on a series of CBIR experiments. Experiments are performed on the Smithsonian isolated leaf, Kimia shape, and Brodatz texture datasets. Our method provides both good retrieval performance and greater robustness compared to standard Gabor-based and three state-of-the-art shape-based methods that have similar objectives.

Beschreibung: In this paper, we develop a decomposition model to restore blurred images with missing pixels. Our assumption is that the underlying image is the superposition of cartoon and texture components. We use the total variation norm and its dual norm to regularize the cartoon and texture, respectively. We recommend an efficient numerical algorithm based on the splitting versions of augmented Lagrangian method to solve the problem. Theoretically, the existence of a minimizer to the energy function and the convergence of the algorithm are guaranteed. In contrast to recently developed methods for deblurring images, the proposed algorithm not only gives the restored image, but also gives a decomposition of cartoon and texture parts. These two parts can be further used in segmentation and inpainting problems. Numerical comparisons between this algorithm and some state-of-the-art methods are also reported.

Beschreibung: Complexities of dynamic volumetric imaging challenge the available computer vision techniques on a number of different fronts. This paper examines the relationship between the estimation accuracy and required amount of smoothness for a general solution from a robust statistics perspective. We show that a (surprisingly) small amount of local smoothing is required to satisfy both the necessary and sufficient conditions for accurate optic flow estimation. This notion is called “just enough” smoothing, and its proper implementation has a profound effect on the preservation of local information in processing 3D dynamic scans. To demonstrate the effect of “just enough” smoothing, a robust 3D optic flow method with quantized local smoothing is presented, and the effect of local smoothing on the accuracy of motion estimation in dynamic lung CT images is examined using both synthetic and real image sequences with ground truth.

Beschreibung: Exploiting a priori known structural information lies at the core of many image reconstruction methods that can be stated as inverse problems. The synthesis model, which assumes that images can be decomposed into a linear combination of very few atoms of some dictionary, is now a well established tool for the design of image reconstruction algorithms. An interesting alternative is the analysis model, where the signal is multiplied by an analysis operator and the outcome is assumed to be sparse. This approach has only recently gained increasing interest. The quality of reconstruction methods based on an analysis model severely depends on the right choice of the suitable operator. In this paper, we present an algorithm for learning an analysis operator from training images. Our method is based on $ell_{p}$ -norm minimization on the set of full rank matrices with normalized columns. We carefully introduce the employed conjugate gradient method on manifolds, and explain the underlying geometry of the constraints. Moreover, we compare our approach to state-of-the-art methods for image denoising, inpainting, and single image super-resolution. Our numerical results show competitive performance of our general approach in all presented applications compared to the specialized state-of-the-art techniques.

Beschreibung: In this paper, the properties of a new class of causal Markov random fields, named symmetric Markov mesh random field, are initially discussed. It is shown that the symmetric Markov mesh random fields from the upper corners are equivalent to the symmetric Markov mesh random fields from the lower corners. Based on this new random field, a symmetric, corner-independent, and isotropic image model is then derived which incorporates the dependency of a pixel on all its neighbors. The introduced image model comprises the product of several local 1D density and 2D joint density functions of pixels in an image thus making it computationally tractable and practically feasible by allowing the use of histogram and joint histogram approximations to estimate the model parameters. An image restoration application is also presented to confirm the effectiveness of the model developed. The experimental results demonstrate that this new model provides an improved tool for image modeling purposes compared to the conventional Markov random field models.

Beschreibung: This paper presents, for the first time, a unified blind method for multi-image super-resolution (MISR or SR), single-image blur deconvolution (SIBD), and multi-image blur deconvolution (MIBD) of low-resolution (LR) images degraded by linear space-invariant (LSI) blur, aliasing, and additive white Gaussian noise (AWGN). The proposed approach is based on alternating minimization (AM) of a new cost function with respect to the unknown high-resolution (HR) image and blurs. The regularization term for the HR image is based upon the Huber-Markov random field (HMRF) model, which is a type of variational integral that exploits the piecewise smooth nature of the HR image. The blur estimation process is supported by an edge-emphasizing smoothing operation, which improves the quality of blur estimates by enhancing strong soft edges toward step edges, while filtering out weak structures. The parameters are updated gradually so that the number of salient edges used for blur estimation increases at each iteration. For better performance, the blur estimation is done in the filter domain rather than the pixel domain, i.e., using the gradients of the LR and HR images. The regularization term for the blur is Gaussian (L2 norm), which allows for fast noniterative optimization in the frequency domain. We accelerate the processing time of SR reconstruction by separating the upsampling and registration processes from the optimization procedure. Simulation results on both synthetic and real-life images (from a novel computational imager) confirm the robustness and effectiveness of the proposed method.

Beschreibung: We study state-based video communication where a client simultaneously informs the server about the presence status of various packets in its buffer. In sender-driven transmission, the client periodically sends to the server a single acknowledgement packet that provides information about all packets that have arrived at the client by the time the acknowledgment is sent. In receiver-driven streaming, the client periodically sends to the server a single request packet that comprises a transmission schedule for sending missing data to the client over a horizon of time. We develop a comprehensive optimization framework that enables computing packet transmission decisions that maximize the end-to-end video quality for the given bandwidth resources, in both prospective scenarios. The core step of the optimization comprises computing the probability that a single packet will be communicated in error as a function of the expected transmission redundancy (or cost) used to communicate the packet. Through comprehensive simulation experiments, we carefully examine the performance advances that our framework enables relative to state-of-the-art scheduling systems that employ regular acknowledgement or request packets. Consistent gains in video quality of up to 2B are demonstrated across a variety of content types. We show that there is a direct analogy between the error-cost efficiency of streaming a single packet and the overall rate-distortion performance of streaming the whole content. In the case of sender-driven transmission, we develop an effective modeling approach that accurately characterizes the end-to-end performance as a function of the packet loss rate on the backward channel and the source encoding characteristics.

Beschreibung: Many computational models of visual attention performing well in predicting salient areas of 2D images have been proposed in the literature. The emerging applications of stereoscopic 3D display bring an additional depth of information affecting the human viewing behavior, and require extensions of the efforts made in 2D visual modeling. In this paper, we propose a new computational model of visual attention for stereoscopic 3D still images. Apart from detecting salient areas based on 2D visual features, the proposed model takes depth as an additional visual dimension. The measure of depth saliency is derived from the eye movement data obtained from an eye-tracking experiment using synthetic stimuli. Two different ways of integrating depth information in the modeling of 3D visual attention are then proposed and examined. For the performance evaluation of 3D visual attention models, we have created an eye-tracking database, which contains stereoscopic images of natural content and is publicly available, along with this paper. The proposed model gives a good performance, compared to that of state-of-the-art 2D models on 2D images. The results also suggest that a better performance is obtained when depth information is taken into account through the creation of a depth saliency map, rather than when it is integrated by a weighting method.

Beschreibung: This paper investigates priority encoding transmission (PET) protection for streaming scalably compressed video streams over erasure channels, for the scenarios where a small number of retransmissions are allowed. In principle, the optimal protection depends not only on the importance of each stream element, but also on the expected channel behavior. By formulating a collection of hypotheses concerning its own behavior in future transmissions, limited-retransmission PET (LR-PET) effectively constructs channel codes spanning multiple transmission slots and thus offers better protection efficiency than the original PET. As the number of transmission opportunities increases, the optimization for LR-PET becomes very challenging because the number of hypothetical retransmission paths increases exponentially. As a key contribution, this paper develops a method to derive the effective recovery-probability versus redundancy-rate characteristic for the LR-PET procedure with any number of transmission opportunities. This significantly accelerates the protection assignment procedure in the original LR-PET with only two transmissions, and also makes a quick and optimal protection assignment feasible for scenarios where more transmissions are possible. This paper also gives a concrete proof to the redundancy embedding property of the channel codes formed by LR-PET, which allows for a decoupled optimization for sequentially dependent source elements with convex utility-length characteristic. This essentially justifies the source-independent construction of the protection convex hull for LR-PET.

Beschreibung: It is very important to extract high quality texture features from images. This is, however, often laborious, because the randomness in the color distribution patterns for texture elements makes texture measurement very difficult, despite these elements having a very similar visual appearance. In this paper, we propose the use of multi-scale color histograms to measure the effect of color distribution patterns efficiently and without having to compute the actual patterns, which saves considerable effort. Meanwhile, the hue-saturation-intensity color model is mainly adopted to take the advantage of human visual experiences in texture recognition. We discuss and validate the effectiveness and efficiency of our method by applying to various benchmarks. The results show that we can extract quality dominant textures automatically in real time, and faster by several orders of magnitude than existing methods.

Beschreibung: In this paper, we propose a hybrid method that combines Gaussian process learning, a particle filter, and annealing to track the 3D pose of a human subject in video sequences. Our approach, which we refer to as annealed Gaussian process guided particle filter, comprises two steps. In the training step, we use a supervised learning method to train a Gaussian process regressor that takes the silhouette descriptor as an input and produces multiple output poses modeled by a mixture of Gaussian distributions. In the tracking step, the output pose distributions from the Gaussian process regression are combined with the annealed particle filter to track the 3D pose in each frame of the video sequence. Our experiments show that the proposed method does not require initialization and does not lose tracking of the pose. We compare our approach with a standard annealed particle filter using the HumanEva-I dataset and with other state of the art approaches using the HumanEva-II dataset. The evaluation results show that our approach can successfully track the 3D human pose over long video sequences and give more accurate pose tracking results than the annealed particle filter.

Beschreibung: Textures in images can often be well modeled using self-similar processes while they may simultaneously display anisotropy. The present contribution thus aims at studying jointly selfsimilarity and anisotropy by focusing on a specific classical class of Gaussian anisotropic selfsimilar processes. It will be first shown that accurate joint estimates of the anisotropy and selfsimilarity parameters are performed by replacing the standard 2D-discrete wavelet transform with the hyperbolic wavelet transform, which permits the use of different dilation factors along the horizontal and vertical axes. Defining anisotropy requires a reference direction that needs not a priori match the horizontal and vertical axes according to which the images are digitized; this discrepancy defines a rotation angle. Second, we show that this rotation angle can be jointly estimated. Third, a nonparametric bootstrap based procedure is described, which provides confidence intervals in addition to the estimates themselves and enables us to construct an isotropy test procedure, which can be applied to a single texture image. Fourth, the robustness and versatility of the proposed analysis are illustrated by being applied to a large variety of different isotropic and anisotropic self-similar fields. As an illustration, we show that a true anisotropy built-in self-similarity can be disentangled from an isotropic self-similarity to which an anisotropic trend has been superimposed.

Beschreibung: Establishing a dense vertex-to-vertex anthropometric correspondence between 3D faces is an important and fundamental problem in 3D face research, which can contribute to most applications of 3D faces. This paper proposes a sparse facial deformable model to automatically achieve this task. For an input 3D face, the basic idea is to generate a new 3D face that has the same mesh topology as a reference face and the highly similar shape to the input face, and whose vertices correspond to those of the reference face in an anthropometric sense. Two constraints: 1) the shape constraint and 2) correspondence constraint are modeled in our method to satisfy the three requirements. The shape constraint is solved by a novel face deformation approach in which a normal-ray scheme is integrated to the closest-vertex scheme to keep high-curvature shapes in deformation. The correspondence constraint is based on an assumption that if the vertices on 3D faces are corresponded, their shape signals lie on a manifold and each face signal can be represented sparsely by a few typical items in a dictionary. The dictionary can be well learnt and contains the distribution information of the corresponded vertices. The correspondence information can be conveyed to the sparse representation of the generated 3D face. Thus, a patch-based sparse representation is proposed as the correspondence constraint. By solving the correspondence constraint iteratively, the vertices of the generated face can be adjusted to correspondence positions gradually. At the early iteration steps, smaller sparsity thresholds are set that yield larger representation errors but better globally corresponded vertices. At the later steps, relatively larger sparsity thresholds are used to encode local shapes. By this method, the vertices in the new face approach the right positions progressively until the final global correspondence is reached. Our method is automatic, and the manual work is needed only in training procedure- The experimental results on a large-scale publicly available 3D face data set, BU-3DFE, demonstrate that our method achieves better performance than existing methods.

Beschreibung: Generating ensembles from multiple individual classifiers is a popular approach to raise the accuracy of the decision. As a rule for decision making, majority voting is a usually applied model. In this paper, we generalize classical majority voting by incorporating probability terms $p_{n,k}$ to constrain the basic framework. These terms control whether a correct or false decision is made if $k$ correct votes are present among the total number of $n$ . This generalization is motivated by object detection problems, where the members of the ensemble are image processing algorithms giving their votes as pixels in the image domain. In this scenario, the terms $p_{n,k}$ can be specialized by a geometric constraint. Namely, the votes should fall inside a region matching the size and shape of the object to vote together. We give several theoretical results in this new model for both dependent and independent classifiers, whose individual accuracies may also differ. As a real world example, we present our ensemble-based system developed for the detection of the optic disc in retinal images. For this problem, experimental results are shown to demonstrate the characterization capability of this system. We also investigate how the generalized model can help us to improve an ensemble with extending it by adding a new algorithm.

Beschreibung: Advances in machine learning technology have made efficient image denoising possible. In this paper, we propose a new ridgelet support vector machine (RSVM) for image noise reduction. Multiscale ridgelet support vector filter (MRSVF) is first deduced from RSVM, to produce a multiscale, multidirection, undecimated, dyadic, aliasing, and shift-invariant geometric multiscale ridgelet support vector transform (GMRSVT). Then, multiscale dictionaries are learned from examples to reduce noises existed in GMRSVT coefficients. Compared with the available approaches, the proposed method has the following characteristics. The proposed MRSVF can extract the salient features associated with the linear singularities of images. Consequently, GMRSVT can well approximate edges, contours and textures in images, and avoid ringing effects suffered from sampling in the multiscale decomposition of images. Sparse coding is explored for noise reduction via the learned multiscale and overcomplete dictionaries. Some experiments are taken on natural images, and the results show the efficiency of the proposed method.

Beschreibung: An image processing observational technique for the stereoscopic reconstruction of the waveform of oceanic sea states is developed. The technique incorporates the enforcement of any given statistical wave law modeling the quasi-Gaussianity of oceanic waves observed in nature. The problem is posed in a variational optimization framework, where the desired waveform is obtained as the minimizer of a cost functional that combines image observations, smoothness priors and a weak statistical constraint. The minimizer is obtained by combining gradient descent and multigrid methods on the necessary optimality equations of the cost functional. Robust photometric error criteria and a spatial intensity compensation model are also developed to improve the performance of the presented image matching strategy. The weak statistical constraint is thoroughly evaluated in combination with other elements presented to reconstruct and enforce constraints on experimental stereo data, demonstrating the improvement in the estimation of the observed ocean surface.

Beschreibung: This paper introduces an efficient method for lossless compression of depth map images, using the representation of a depth image in terms of three entities: 1) the crack-edges; 2) the constant depth regions enclosed by them; and 3) the depth value over each region. The starting representation is identical with that used in a very efficient coder for palette images, the piecewise-constant image model coding, but the techniques used for coding the elements of the representation are more advanced and especially suitable for the type of redundancy present in depth images. Initially, the vertical and horizontal crack-edges separating the constant depth regions are transmitted by 2D context coding using optimally pruned context trees. Both the encoder and decoder can reconstruct the regions of constant depth from the transmitted crack-edge image. The depth value in a given region is encoded using the depth values of the neighboring regions already encoded, exploiting the natural smoothness of the depth variation, and the mutual exclusiveness of the values in neighboring regions. The encoding method is suitable for lossless compression of depth images, obtaining compression of about 10–65 times, and additionally can be used as the entropy coding stage for lossy depth compression.

Beschreibung: Color sampling based matting methods find the best known samples for foreground and background colors of unknown pixels. Such methods do not perform well if there is an overlap in the color distribution of foreground and background regions because color cannot distinguish between these regions and hence, the selected samples cannot reliably estimate the matte. Furthermore, current sampling based matting methods choose samples that are located around the boundaries of foreground and background regions. In this paper, we overcome these two problems. First, we propose texture as a feature that can complement color to improve matting by discriminating between known regions with similar colors. The contribution of texture and color is automatically estimated by analyzing the content of the image. Second, we combine local sampling with a global sampling scheme that prevents true foreground or background samples to be missed during the sample collection stage. An objective function containing color and texture components is optimized to choose the best foreground and background pair among a set of candidate pairs. Experiments are carried out on a benchmark data set and an independent evaluation of the results shows that the proposed method is ranked first among all other image matting methods.

Beschreibung: We derive shortest-path constraints from graph models of structure adjacency relations and introduce them in a joint centroidal Voronoi image clustering and Graph Cut multiobject semiautomatic segmentation framework. The vicinity prior model thus defined is a piecewise-constant model incurring multiple levels of penalization capturing the spatial configuration of structures in multiobject segmentation. Qualitative and quantitative analyses and comparison with a Potts prior-based approach and our previous contribution on synthetic, simulated, and real medical images show that the vicinity prior allows for the correct segmentation of distinct structures having identical intensity profiles and improves the precision of segmentation boundary placement while being fairly robust to clustering resolution. The clustering approach we take to simplify images prior to segmentation strikes a good balance between boundary adaptivity and cluster compactness criteria furthermore allowing to control the trade-off. Compared with a direct application of segmentation on voxels, the clustering step improves the overall runtime and memory footprint of the segmentation process up to an order of magnitude without compromising the quality of the result.

Beschreibung: This paper devises an augmented active surface model for the recovery of small structures in a low resolution and high noise setting, where the role of regularization is especially important. The emphasis here is on evaluating performance using real clinical computed tomography (CT) data with comparisons made to an objective ground truth acquired using micro-CT. In this paper, we show that the application of conventional active contour methods to small objects leads to non-optimal results because of the inherent properties of the energy terms and their interactions with one another. We show that the blind use of a gradient magnitude based energy performs poorly at these object scales and that the point spread function (PSF) is a critical factor that needs to be accounted for. We propose a new model that augments the external energy with prior knowledge by incorporating the PSF and the assumption of reasonably constant underlying CT numbers.

Beschreibung: Scene refocusing beyond extended depth of field for users to observe objects effectively is aimed by researchers in computational photography, microscopic imaging, and so on. Ordinary all-in-focus image reconstruction from a sequence of multi-focus images achieves extended depth of field, where reconstructed images would be captured through a pinhole in the center on the lens. In this paper, we propose a novel method for reconstructing all-in-focus images through shifted pinholes on the lens based on 3D frequency analysis of multi-focus images. Such shifted pinhole images are obtained by a linear combination of multi-focus images with scene-independent 2D filters in the frequency domain. The proposed method enables us to efficiently synthesize dense 4D light field on the lens plane for image-based rendering, especially, robust scene refocusing with arbitrary bokeh. Our novel method using simple linear filters achieves not only reconstruction of all-in-focus images even for shifted pinholes more robustly than the conventional methods depending on scene/focus estimation, but also scene refocusing without suffering from limitation of resolution in comparison with recent approaches using special devices such as lens arrays in computational photography.

Beschreibung: Normalized cut is a powerful method for image segmentation as well as data clustering. However, it does not perform well in challenging segmentation problems, such as segmenting objects in a complex background. Researchers have attempted to incorporate priors or constraints to handle such cases. Available priors in image segmentation problems may be hard or soft, unary or pairwise, but only hard must-link constraints and two-class settings are well studied. The main difficulties may lie in the following aspects: 1) the nontransitive nature of cannot-link constraints makes it hard to use such constraints in multi-class settings and 2) in multi-class or pairwise settings, the output labels have inconsistent representations with given priors, making soft priors difficult to use. In this paper, we propose novel algorithms, which can handle both hard and soft, both unary and pairwise priors in multi-class settings and provide closed form and efficient solutions. We also apply the proposed algorithms to the problem of object segmentation, producing good results by further introducing a spatial regularity term. Experiments show that the proposed algorithms outperform the state-of-the-art algorithms significantly in clustering accuracy. Other merits of the proposed algorithms are also demonstrated.

Beschreibung: Contextual information has been widely used as a rich source of information to segment multiple objects in an image. A contextual model uses the relationships between the objects in a scene to facilitate object detection and segmentation. Using contextual information from different objects in an effective way for object segmentation, however, remains a difficult problem. In this paper, we introduce a novel framework, called multiclass multiscale (MCMS) series contextual model, which uses contextual information from multiple objects and at different scales for learning discriminative models in a supervised setting. The MCMS model incorporates cross-object and inter-object information into one probabilistic framework and thus is able to capture geometrical relationships and dependencies among multiple objects in addition to local information from each single object present in an image. We demonstrate that our MCMS model improves object segmentation performance in electron microscopy images and provides a coherent segmentation of multiple objects. Through speeding up the segmentation process, the proposed method will allow neurobiologists to move beyond individual specimens and analyze populations paving the way for understanding neurodegenerative diseases at the microscopic level.

Beschreibung: In many biological or medical applications, images that contain sequences of shapes are common. The existence of high inter-individual variability makes their interpretation complex. In this paper, we address the computer-assisted interpretation of such images and we investigate how we can remove or reduce these image variabilities. The proposed approach relies on the development of an efficient image registration technique. We first show the inadequacy of state-of-the-art intensity-based and feature-based registration techniques for the considered image datasets. Then, we propose a robust variational method which benefits from the geometrical information present in this type of images. In the proposed non-rigid geodesics-based registration, the successive shapes are represented by a level-set representation, which we rely on to carry out the registration. The successive level sets are regarded as elements in a shape space and the corresponding matching is that of the optimal geodesic path. The proposed registration scheme is tested on synthetic and real images. The comparison against results of state-of-the-art methods proves the relevance of the proposed method for this type of images.

Beschreibung: Dynamic contrast-enhanced magnetic resonance (DCE-MR) of the breast is especially robust for the diagnosis of cancer in high-risk women due to its high sensitivity. Its specificity may be, however, compromised since several benign masses take up contrast agent as malignant lesions do. In this paper, we propose a novel method of 3D multifractal analysis to characterize the spatial complexity (spatial arrangement of texture) of breast tumors at multiple scales. Self-similar properties are extracted from the estimation of the multifractal scaling exponent for each clinical case, using lacunarity as the multifractal measure. These properties include several descriptors of the multifractal spectra reflecting the morphology and internal spatial structure of the enhanced lesions relatively to normal tissue. The results suggest that the combined multifractal characteristics can be effective to distinguish benign and malignant findings, judged by the performance of the support vector machine classification method evaluated by receiver operating characteristics with an area under the curve of 0.96. In addition, this paper confirms the presence of multifractality in DCE-MR volumes of the breast, whereby multiple degrees of self-similarity prevail at multiple scales. The proposed feature extraction and classification method have the potential to complement the interpretation of the radiologists and supply a computer-aided diagnosis system.

Beschreibung: We present a two stage framework for automatic video text removal to detect and remove embedded video texts and fill-in their remaining regions by appropriate data. In the video text detection stage, text locations in each frame are found via an unsupervised clustering performed on the connected components produced by the stroke width transform (SWT). Since SWT needs an accurate edge map, we develop a novel edge detector which benefits from the geometric features revealed by the bandlet transform. Next, the motion patterns of the text objects of each frame are analyzed to localize video texts. The detected video text regions are removed, then the video is restored by an inpainting scheme. The proposed video inpainting approach applies spatio-temporal geometric flows extracted by bandlets to reconstruct the missing data. A 3D volume regularization algorithm, which takes advantage of bandlet bases in exploiting the anisotropic regularities, is introduced to carry out the inpainting task. The method does not need extra processes to satisfy visual consistency. The experimental results demonstrate the effectiveness of both our proposed video text detection approach and the video completion technique, and consequently the entire automatic video text removal and restoration process.

Beschreibung: A novel motion-compensated frame interpolation (MCFI) algorithm to increase video temporal resolutions based on multihypothesis motion estimation and texture optimization is proposed in this paper. Initially, we form multiple motion hypotheses for each pixel by employing different motion estimation parameters, i.e., different block sizes and directions. Then, we determine the best motion hypothesis for each pixel by solving a labeling problem and optimizing the parameters. In the labeling problem, the cost function is composed of color, shape, and smoothness terms. Finally, we refine the motion hypothesis field based on the texture optimization technique and blend multiple source pixels to interpolate each pixel in the intermediate frame. Simulation results demonstrate that the proposed algorithm provides significantly better MCFI performance than conventional algorithms.

Beschreibung: High angle annular dark field (HAADF)-scanning transmission electron microscope (STEM) data is increasingly being used in the physical sciences to research materials in 3D because it reduces the effects of Bragg diffraction seen in bright field TEM data. Typically, tomographic reconstructions are performed by directly applying either filtered back projection (FBP) or the simultaneous iterative reconstruction technique (SIRT) to the data. Since HAADF-STEM tomography is a limited angle tomography modality with low signal to noise ratio, these methods can result in significant artifacts in the reconstructed volume. In this paper, we develop a model based iterative reconstruction algorithm for HAADF-STEM tomography. We combine a model for image formation in HAADF-STEM tomography along with a prior model to formulate the tomographic reconstruction as a maximum a posteriori probability (MAP) estimation problem. Our formulation also accounts for certain missing measurements by treating them as nuisance parameters in the MAP estimation framework. We adapt the iterative coordinate descent algorithm to develop an efficient method to minimize the corresponding MAP cost function. Reconstructions of simulated as well as experimental data sets show results that are superior to FBP and SIRT reconstructions, significantly suppressing artifacts and enhancing contrast.

Beschreibung: Shape from defocus (SFD) is one of the most popular techniques in monocular 3D vision. While most SFD approaches require two or more images of the same scene captured at a fixed view point, this paper presents an efficient approach to estimate absolute depth from a single defocused image. Instead of directly measuring defocus level of each pixel, we propose to design a sequence of aperture-shape filters to segment a defocused image by defocus level. A boundary-weighted belief propagation algorithm is employed to obtain a smooth depth map. We also give an estimation of depth error. Extensive experiments show that our approach outperforms the state-of-the-art single-image SFD approaches both in precision of the estimated absolute depth and running time.

Beschreibung: This paper presents a novel example-based single-image superresolution procedure that upscales to high-resolution (HR) a given low-resolution (LR) input image without relying on an external dictionary of image examples. The dictionary instead is built from the LR input image itself, by generating a double pyramid of recursively scaled, and subsequently interpolated, images, from which self-examples are extracted. The upscaling procedure is multipass, i.e., the output image is constructed by means of gradual increases, and consists in learning special linear mapping functions on this double pyramid, as many as the number of patches in the current image to upscale. More precisely, for each LR patch, similar self-examples are found, and, because of them, a linear function is learned to directly map it into its HR version. Iterative back projection is also employed to ensure consistency at each pass of the procedure. Extensive experiments and comparisons with other state-of-the-art methods, based both on external and internal dictionaries, show that our algorithm can produce visually pleasant upscalings, with sharp edges and well reconstructed details. Moreover, when considering objective metrics, such as Peak signal-to-noise ratio and Structural similarity, our method turns out to give the best performance.

Beschreibung: Bag-of-features (BoFs) representation has been extensively applied to deal with various computer vision applications. To extract discriminative and descriptive BoF, one important step is to learn a good dictionary to minimize the quantization loss between local features and codewords. While most existing visual dictionary learning approaches are engaged with unsupervised feature quantization, the latest trend has turned to supervised learning by harnessing the semantic labels of images or regions. However, such labels are typically too expensive to acquire, which restricts the scalability of supervised dictionary learning approaches. In this paper, we propose to leverage image attributes to weakly supervise the dictionary learning procedure without requiring any actual labels. As a key contribution, our approach establishes a generative hidden Markov random field (HMRF), which models the quantized codewords as the observed states and the image attributes as the hidden states, respectively. Dictionary learning is then performed by supervised grouping the observed states, where the supervised information is stemmed from the hidden states of the HMRF. In such a way, the proposed dictionary learning approach incorporates the image attributes to learn a semantic-preserving BoF representation without any genuine supervision. Experiments in large-scale image retrieval and classification tasks corroborate that our approach significantly outperforms the state-of-the-art unsupervised dictionary learning approaches.

Beschreibung: In image classification, recognition or retrieval systems, image contents are commonly described by global features. However, the global features generally contain noise from the background, occlusion, or irrelevant objects in the images. Thus, only part of the global feature elements is informative for describing the objects of interest and useful for the image analysis tasks. In this paper, we propose algorithms to automatically discover the subgroups of highly correlated feature elements within predefined global features. To this end, we first propose a novel mixture sparse regression (MSR) method, which groups the elements of a single vector according to the membership conveyed by their sparse regression coefficients. Based on MSR, we proceed to develop the autogrouped sparse representation (ASR), which groups correlated feature elements together through fusing their individual sparse representations over multiple samples. We apply ASR/MSR in two practical visual analysis tasks: 1) multilabel image classification and 2) motion segmentation. Comprehensive experimental evaluations show that our proposed methods are able to achieve superior performance compared with the state-of-the-art classification on these two tasks.

Beschreibung: In compressive sensing, wavelet space is widely used to generate sparse signal (image signal in particular) representations. In this paper, we propose a novel approach of statistical context modeling to increase the level of sparsity of wavelet image representations. It is shown, contrary to a widely held assumption, that high-frequency wavelet coefficients have nonzero mean distributions if conditioned on local image structures. Removing this bias can make wavelet image representations sparser, i.e., having a greater number of zero and close-to-zero coefficients. The resulting unbiased probability models can significantly improve the performance of existing wavelet-based compressive image reconstruction methods in both PSNR and visual quality. An efficient algorithm is presented to solve the compressive image recovery (CIR) problem using the refined models. Experimental results on both simulated compressive sensing (CS) image data and real CS image data show that the new CIR method significantly outperforms existing CIR methods in both PSNR and visual quality.

Beschreibung: In recent years, image priors based on nonlocal self-similarity and low-rank approximation have been proven as powerful tools for image restoration. Many restoration methods group similar patches as a matrix and recover the underlying low-rank structure from the corrupted matrix via rank minimization. However, both the nonlocally redundant and low-rank properties are highly content dependent, and whether they can faithfully characterize a wide range of natural images still remains unclear. In this paper, we analyze these two properties and provide quantifications of them in a data-driven and parametric way, respectively, obtaining the new measures of regional redundancy and nonlocal patch rank. Leveraging these prior leads to an adaptive image restoration method with content-awareness. In particular, our method iteratively removes outliers and recovers latent fine details. To handle outliers, we propose an adaptive low-rank and sparse matrix approximation algorithm to encourage the estimated nonlocal rank in the patch matrix. The guidance of regional redundancy further gives rise to the “denoise” quality. In the detail recovery step, we propose an adaptive joint kernel regression algorithm using the redundancy measure to determine the confidence of each regression group. It also bridges the gap between our online and offline dictionary learning schemes. Experiments on synthetic and real-world images show the efficacy of our method in image deblurring and super-resolution tasks, especially when subject to practical outliers such as rain drops.

Beschreibung: This paper provides an extension of the 1D Hilbert Huang transform for the analysis of images using recent optimization techniques. The proposed method consists of: 1) adaptively decomposing an image into oscillating parts called intrinsic mode functions (IMFs) using a mode decomposition procedure and 2) providing a local spectral analysis of the obtained IMFs in order to get the local amplitudes, frequencies, and orientations. For the decomposition step, we propose two robust 2D mode decompositions based on nonsmooth convex optimization: 1) a genuine 2D approach, which constrains the local extrema of the IMFs and 2) a pseudo-2D approach, which separately constrains the extrema of lines, columns, and diagonals. The spectral analysis step is an optimization strategy based on Prony annihilation property and applied on small square patches of the IMFs. The resulting 2D Prony–Huang transform is validated on simulated and real data.