ALBERT

Beschreibung: Postdisaster search and rescue is an important application of ultrawideband (UWB) radar systems, which mainly detect trapped victims by their respiratory-motion response. The development of a respiratory-motion detection (RMD) algorithm that can eliminate nonstationary clutter and noise is a challenging task for the application. A new algorithm is proposed to deal with the task in this letter. It uses the multichannel singular spectrum analysis (MSSA) technique to reconstruct the respiratory-motion response detected by a UWB radar. During the reconstruction, the periodicity and range interrelation characteristics of the response are exploited to adaptively identify signal subspaces. The performance of the algorithm is verified both by simulated and real data. The results show its improved performance over the reference algorithms, e.g., a singular-value-decomposition-based algorithm. The adaptive-MSSA-based RMD algorithm has great promise not only in practical use but also for future research of UWB-radar-based human being remote sensing.

Beschreibung: Land–sea contamination observed in Soil Moisture and Ocean Salinity (SMOS) brightness temperature images is found to have two main contributions: the floor error inherent of image reconstruction and a multiplicative error either in the antenna temperature or in the visibility samples measured by the correlator. The origin of this last one is traced down to SMOS calibration parameters to yield a simple correction scheme, which is validated against several geophysical scenarios. Autoconsistency rules in interferometric synthesis together with redundant and complementary calibration procedures provide a robust SMOS calibration scheme.

Beschreibung: This letter proposes a novel algorithm, which is based on the generalized method of moments (GMM), for the estimation and correction of phase errors induced in synthetic aperture radar (SAR) imagery. The GMM algorithm is used to replace the original phase-estimation kernel in the basic structure of the phase-gradient-autofocus algorithm. Since this novel algorithm does not require the observed signal to be a certain distribution model, it is able to estimate arbitrary phase errors. The GMM algorithm has the ability of estimating range-dependent phase errors, which makes it an efficient estimator. As a result, higher accuracy of the estimated phase errors and a better focused image can be achieved. Excellent results have been obtained in autofocusing and imaging experiments on real SAR data.

Beschreibung: Strong clutter reflections of terrain and marine surfaces obscure the contrast between the target-of-interest and clutter (terrain and marine surface reflections) in synthetic aperture radar (SAR) images and consequently hinder the efficiency of image interpretation and analysis. To overcome this problem, this letter proposes an efficient clutter suppression method in SAR images, which is named shedding irrelevant patterns (SIP). The essence is to construct a regression function that can suppress clutter and preserve the target patterns concurrently. We assume that the clutter is irrelevant to the target-of-interest and distinguishable in patterns in terms of image-pixel distribution and intensity (spatial information). Experimental results show the efficiency of the proposed method in both clutter suppression and target pattern preservation.

Beschreibung: A concern in hyperspectral image classification is the high number of required training samples. When traditional classifiers are applied, feature reduction (FR) techniques are the most common approaches to deal with this problem. Subspace-based classifiers, which are developed based on high-dimensional space characteristics, are another way to handle the high dimension of hyperspectral images. In this letter, a novel subspace-based classification approach is proposed and compared with basic and improved subspace-based classifiers. The proposed classifier is also compared with traditional classifiers that are accompanied by an FR technique and the well-known support vector machine classifier. Experimental results prove the efficiency of the proposed method, especially when a limited number of training samples are available. Furthermore, the proposed method has a very high level of automation and simplicity, as it has no parameters to be set.

Beschreibung: To improve the spatial density of measurement points of persistent-scatterer interferometry, distributed scatterer (DS) should be considered and processed. An important procedure in DS interferometry is the phase triangulation (PT). This letter introduces two modified PT algorithms (i.e., equal-weighted PT and coherence-weighted PT) and analyzes the mathematical relations between different published PT methods (i.e., the maximum-likelihood phase estimator, least squares estimator, and eigendecomposition-based phase estimators). The analysis shows that the above five PT methods share very similar mathematical forms with different weight values in the estimation procedure.

Beschreibung: Sparse representation-based classifier and its variants have been widely adopted for hyperspectral image (HSI) classification recently. However, sparse representation is unstable so that similar features might obtain significantly different sparse codes. Despite the instability, we find that the sparse codes follow a class-dependent distribution under the structured dictionary consisting of training samples from all classes. Based on this observation, a novel discriminative feature, sparse code histogram (SCH), is developed for HSI classification. By counting the SCH of each sample from the sparse codes of its spatial neighbors, we can statistically obtain the distribution pattern of sparse codes of the class to which the sample belongs, and then treat the SCH as a new feature for classification. To reduce the possible outliers among the neighbors, a shape-adaptive neighborhood extractor is also employed to enhance the stability of the histogram feature. Experimental results demonstrate that SCH enjoys a strong discriminative power, which can achieve notably better performance than several state-of-the-art methods for HSI classification with limited training samples.

Beschreibung: Several detection statistics have been proposed for detecting fine ground disturbances between two synthetic aperture radar (SAR) images, such as vehicle tracks. The standard method involves estimating a local correlation coefficient between images. Other methods have been proposed using various statistical hypothesis tests. One of these alternative methods is a generalized likelihood ratio test (GLRT), which compares a full-correlation image model to a no-correlation image model. In this letter, we expand the GLRT to polarimetric SAR data and derive the appropriate GLRT detection statistics. Additionally, we explore relaxing the equal variance/equal polarimetric covariance assumptions used in previous results and find improved performance on macroscopic scene changes.

Beschreibung: For downward-looking linear array 3-D synthetic aperture radar, the resolution in cross-track direction is much lower than the ones in range and azimuth. Hence, superresolution reconstruction algorithms are desired. Since the cross-track signal to be reconstructed is sparse in the object domain, compressive sensing algorithm has been used. However, the imaging processing on the 3-D scene brings large computational loads, which renders challenges in both data acquisition and processing. To cover this shortage, truncated singular value decomposition is utilized to reconstruct a reduced-redundancy spatial measurement matrix. The proposed algorithm provides advantages in terms of computational time while maintaining the quality of the scene reconstructions. Moreover, our results on uniform linear array are generally applicable to sparse nonuniform linear array. Superresolution properties and reconstruction accuracies are demonstrated using simulations under the noise and clutter scenarios.

Beschreibung: This letter proposes a signal processing method of passive bistatic radar (PBR) exploiting an uncooperative radar as an illuminator. Compared with other opportunity illuminators, the transmitting signal of a radar usually has a better ambiguity function, which leads to a higher range resolution. Two channels are needed in PBR system. The reference channel is used to estimate radar signal parameters and reconstruct directly propagated signal. The surveillance channel is used to receive scattered wave. An array antenna and a simultaneous multibeam algorithm are necessary in the surveillance channel due to the flexible beam scanning of the uncooperative radar. The procedure of the proposed method is explained in detail, which is then followed by a field experiment. Preliminary results from the field experiment show that the proposed method can be applied to target angle and bistatic range measurement successfully.

Beschreibung: In order to achieve 3-D imaging with an airborne down-looking linear-array synthetic aperture radar (LASAR), a uniform virtual antenna array may be obtained by aperture synthesis of the cross-track sparse multiple-input–multiple-output array. However, the actual 3-D imaging quality is unavoidably degraded by errors in the virtual element position. In this letter, we investigate the effects of these errors on the forms and the degrees of image quality degradation by decomposing the error-related stochastic processes via an orthogonal transform based on discrete Legendre polynomials. It should be noted that these analyses are helpful for designing a LASAR system and providing a reference for specifying the requisite precision of measurement devices and calibration methods. Finally, we briefly consider the use of calibration methods to eliminate the effects of errors.

Beschreibung: In problems where labeled data are scarce, semisupervised learning (SSL) techniques are an attractive framework that can exploit both labeled and unlabeled data. These approaches typically rely on a smoothness assumption such that examples that are similar in input space should also be similar in label space. In many domains, such as remotely sensed hyperspectral image (HSI) classification, the data violate this assumption. In response, we propose a general method by which a neighborhood graph used in SSL is learned using binary classifiers that are trained to predict whether a pair of pixels shares the same label. Working within the framework of semisupervised neural networks (SSNNs), we show that our approach improves on the performance of the SSNN on two HSI data sets.

Beschreibung: In this letter, new models for the spatial correlation of sea clutter texture and intensity are proposed as improved versions of current power law models or exponential decay model. The models for texture have three unknown parameters, and thus can be called triparametric models. The structure of the models is a weighted sum of two components, which can describe the decaying process of the correlation coefficient with spatial lags, as well as the periodic behavior due to the existence of transient coherent structures in sea clutter. Unknown parameters are optimized by the nonlinear least square fit method. Models for sea clutter intensity can be obtained through a linear transform for uncorrelated speckle based on the compound-Gaussian representation of sea clutter. The proposed models are validated and compared with current models using S- and C-band measured sea clutter data. Analysis results indicate the effectiveness of the proposed models in that they can describe the behavior of spatial correlation coefficients with higher accuracy.

Beschreibung: Terrestrial laser scanning (TLS) has become a popular tool for acquiring source data points which can be used to construct digital elevation models (DEMs) for a wide number of applications. A TLS point cloud often has a very fine spatial resolution, which can represent well the spatial variation of a terrain surface. However, the uncertainty in DEMs created from this relatively new type of source data is not well understood, which forms the focus of this letter. TLS survey data representing four terrain surfaces of different characteristics were used to explore the effects of surface complexity and typical TLS data density (in terms of data point spacing) on DEM accuracy. The spatial variation in TLS data can be decomposed into parts corresponding to the signal of spatial variation (of terrain surfaces) and noise due to measurement error. We found a linear relation between the DEM error and the typical TLS data spacings considered (30–100 mm) which arises as a function of the interpolation error, and a constant contribution from the propagated data noise. This letter quantifies these components for each of the four surfaces considered and shows that, for the interpolation method considered here, higher density sampling would not be beneficial.

Beschreibung: A novel way to estimate the live fuel moisture content (LFMC) was explored from the ratio of canopy water content (CWC) and foliage dry biomass (FDB). The CWC was estimated using the PROSAIL (PROSPECT + SAIL) radiative transfer model from the Landsat 8 product. A weak constraint 4-D variational data assimilation method was employed to assimilate the temporally estimated leaf area index into a soil-water-atmosphere-plant (SWAP) model for optimizing the model control variables. Then, the SWAP model was reinitialized with this optimum set of control variables, and better prediction of FDB was obtained. Results showed that a high accuracy level was achieved for the estimated CWC ( $R^{2}=0.91$ , $mbox{RMSE}=84.74 mbox{g/m}^2$ ) and FDB ( $R^2=0.88$ , $mbox{RMSE}=48.54 mbox{g/m} ^2$ ) when compared with in situ measured values. However, the accuracy level of estimated LFMC was poor ( $R^2=0.59$ , $mbox{RMSE} =30.85%$ ) . Further analyses find that the estimated LFMC is reliable for low LFMC but challenged for high LFMC, which indicates that the presented method still makes sense to the assessment of wildfire risk since the wildfire generally occurs when the vegetation is in low LFMC condition.

Beschreibung: In this letter, we present the use of experimental human micro-Doppler signature data gathered by a multistatic radar system to discriminate between unarmed and potentially armed personnel walking along different trajectories. Different ways of extracting suitable features from the spectrograms of the micro-Doppler signatures are discussed, particularly empirical features such as Doppler bandwidth, periodicity, and others, and features extracted from singular value decomposition (SVD) vectors. High classification accuracy of armed versus unarmed personnel (between 90% and 97% depending on the walking trajectory of the people) can be achieved with a single SVD-based feature, in comparison with using four empirical features. The impact on classification performance of different aspect angles and the benefit of combining multistatic information is also evaluated in this letter.

Beschreibung: Automatic urban area detection in remote sensing images is an important application in the field of earth observation. Most of the existing methods employ feature classifiers and thereby contain a data training process. Moreover, some methods cannot detect urban areas in complex scenes accurately. This letter proposes an automatic urban area detection method that uses multiple features that have different resolutions. First, a downsampled low-resolution image is used to segment the candidate area. After the corner points of the urban area are extracted, a weighted Gaussian voting matrix technique is employed to integrate the corner points into the candidate area. Then, the edge features and homogeneous region are extracted by using the original high-resolution image. Using these results as the input, the processes of guided filtering and contrast enhancement can finally detect accurately the urban areas. This method combines multiple features, such as corner, edge, and regional characteristics, to detect the urban areas. The experimental results show that the proposed method has better detection accuracy for urban areas than the existing algorithms.

Beschreibung: In marine sciences, time series are often nonlinear and nonstationary. Adequate and specific methods are needed to analyze such series. In this letter, an application of the empirical mode decomposition method (EMD) associated to the Hilbert spectral analysis (HSA) is presented. Furthermore, EMD-based time-dependent intrinsic correlation (TDIC) analysis is applied to consider the correlation between two nonstationary time series. Four temperature time series obtained from automatic measurements in nearshore waters of the Réunion island are considered, recorded every 10 min from July 2011 to January 2012. The application of the EMD on these series and the estimation of their power spectra using the HSA are illustrated. The authors identify low-frequency tidal waves and display the pattern of correlations at different scales and different locations. By TDIC analysis, it was concluded that the high-frequency modes have small correlation, whereas the trends are perfectly correlated.

Beschreibung: Detecting vehicles in aerial images provides important information for traffic management and urban planning. Detecting the cars in the images is challenging due to the relatively small size of the target objects and the complex background in man-made areas. It is particularly challenging if the goal is near-real-time detection, i.e., within few seconds, on large images without any additional information, e.g., road database and accurate target size. We present a method that can detect the vehicles on a 21-MPixel original frame image without accurate scale information within seconds on a laptop single threaded. In addition to the bounding box of the vehicles, we extract also orientation and type (car/truck) information. First, we apply a fast binary detector using integral channel features in a soft-cascade structure. In the next step, we apply a multiclass classifier on the output of the binary detector, which gives the orientation and type of the vehicles. We evaluate our method on a challenging data set of original aerial images over Munich and a data set captured from an unmanned aerial vehicle (UAV).

Beschreibung: We compare five slope correction methods developed by Walter et al. , Montes et al. , Schleppi et al. , España et al. , and Gonsamo et al. (referred to as WAL, MON, SCH, ESP, and GON, respectively) using artificial fisheye pictures simulated by graphics software and a lookup table (LUT) retrieval method. The LUT is built by simulating the directional gap fraction as a function of leaf area index (LAI) and average leaf inclination angle (ALIA) using the Poisson law. LAI and ALIA estimates correspond to the case of the LUT that provides the lowest root-mean-square error between the observed gap fractions after slope correction and the simulated ones. Three LAI values (1.5, 3.5, and 5.5), four ALIA values (26.8°, 45°, 57.5°, and 63.2°), and three slope angles (0°, 20°, and 50°) constituted 36 samples of random scenes. ESP is recommended because its results are accurate and independent on the leaf angle distribution (LAD), while GON only performs well for spherical LAD. The three other methods present less good performances with underestimation or overestimation of LAI and/or ALIA depending on the LAD, and the recommended order for them is MON, SCH, and WAL.

Beschreibung: In this letter, an improved phase correlation (PC) method based on 2-D plane fitting and the maximum kernel density estimator (MKDE) is proposed, which combines the idea of Stone's method and robust estimator MKDE. The proposed PC method first utilizes a vector filter to minimize the noise errors of the phase angle matrix and then unwraps the filtered phase angle matrix by the use of the minimum cost network flow unwrapping algorithm. Afterward, the unwrapped phase angle matrix is robustly fitted via MKDE, and the slope coefficients of the 2-D plane indicate the subpixel shifts between images. The experiments revealed that the improved method can effectively avoid the impact of outliers on the phase angle matrix during the plane fitting and is robust to aliasing and noise. The matching accuracy can reach 1/50th of a pixel using simulated data. The real image sequence tracking experiment was also undertaken to demonstrate the effectiveness of the proposed PC method with a registration accuracy of root-mean-square error better than 0.1 pixels.

Beschreibung: Accurately mapping forest carbon density by combining sample plots and remotely sensed images has become popular because this method provides spatially explicit estimates. However, mixed pixels often impede the improvement of the estimation. In this letter, regression modeling and spectral unmixing analysis were integrated to improve the estimation of forest carbon density for the You County of Hunan, China, using Landsat Thematic Mapper images. Linear spectral unmixing with and without a constraint (LSUWC and LSUWOC) and nonlinear spectral unmixing (NSU) were compared to derive the fractions of five endmembers, particularly forests. Stepwise regression, logistic regression, and polynomial regression (PR) with and without the forest fraction used as an independent variable and the product of the forest fraction image and the map from the best model without the forest fraction were compared. The models were developed using 56 sample plots, and their results were validated using 26 test plots. The decomposition of mixed pixels was assessed using higher spatial resolution SPOT images and a corresponding land cover map. The results showed that 1) LSUWC more accurately estimated the endmember fractions than LSUWOC and NSU, 2) PR had the greatest estimation accuracy of forest carbon, and 3) combining regression modeling and spectral unmixing increased the estimation accuracy by 31%–39%, and introducing the forest fraction into the regressions performed better than the product of forest fraction image and the results from PR without the fraction. This implied that the integrations provided great potential in reducing the impacts of mixed pixels in mapping forest carbon.

Beschreibung: This letter proposes a multiresolution technique to address the high computational cost in remote sensing image registration. The scale-invariant feature transform is applied to detect keypoints and descriptors, and then, global information combined with descriptors is utilized to establish keypoint mappings. Keypoints are first classified according to their octaves. Then, in the lowest resolution, the keypoints of the largest octave are mapped with descriptors and the global information, giving an initial affine transformation $T_0$ . In the next octave, the keypoints of the second largest octave are mapped by employing $T_0$ to narrow the space of matching keypoints. By this means, the process of establishing keypoint correspondences is conducted from one resolution (octave) to the next as the obtained transformation gets finer until we get to the highest resolution. Due to the high computational expense of computing global information, the proposed technique is important for aligning large-size remote sensing imagery. Experimental results show that the proposed method can achieve a comparable registration accuracy but with a less computational cost than directly building keypoint mappings on images of large size.

Beschreibung: In this letter, we present an efficient parallel implementation of composite kernels in support vector machines (SVMs) for hyperspectral image (HSI) classification. Our implementation makes effective use of commodity graphics processing units (GPUs). Specifically, we port the calculation of composite kernels to GPUs, perform intensive computations based on NVidia's compute unified device architecture, and execute the rest of the operations related with control and small data calculations in the CPU. Our experimental results, conducted using real hyperspectral data sets and NVidia GPU platforms, indicate significant improvements in terms of computational effectiveness, achieving near-real-time performance of spatial–spectral HSI classification for the first time in the literature.

Beschreibung: Compared with airborne laser scanning, terrestrial laser scanning (TLS) offers ground-based point cloud data of trees and provides greater potential to accurately estimate tree and stand parameters. However, there is a lack of effective methods to accurately identify locations of individual trees from TLS point cloud data. It is also unknown whether the estimation accuracy of the parameters, including tree height (H), diameter at breast height (DBH), and so on, using TLS can meet the requirement of forest management and planning. In this letter, a novel method to effectively process point cloud data and further determine the locations of individual trees in a stand based on the central coordinates of point cloud data on a defined grid according to the largest DBH was developed. Moreover, a point-cloud-data-based convex hull algorithm and the cylinder method were, respectively, used to estimate DBH and H of individual trees. This study was conducted in a pure Chinese fir plantation of 45 trees located in Huang-Feng-Qiao forest farm, You County of Hunan, China. The comparison of the estimated and observed values showed that the obtained tree locations had errors of less than 20 cm, and the relative root mean square errors for the estimates of both DBH and H were less than 5%. This implies that TLS is very promising for the retrieval of tree and stand parameters in forest stands. For the applications of these methods to mixed forests with a structure of multilayer canopies, further examination is needed.

Beschreibung: In this letter, a novel algorithm for attitude measurement based on a 3-D electromagnetic model (3-D em-model) is proposed. The 3-D em-model is established offline based on the geometric structure of the target, and it can be used to predict the scattering features at different target attitudes. In order to measure the attitude of the air target, we design a bistatic step frequency radar system. The directions of the two radars' lines of sight (LOSs) relative to the target are acquired by matching the high-resolution range profiles (HRRPs) from the target echoes to the HRRPs generated from the 3-D em-model. Since the directions of two radars' LOSs relative to the Earth are already known, the absolute attitude of the target can be acquired. The innovative contributions of this letter are as follows: 1) A comprehensive theoretical analysis of air target attitude measurement based on its own 3-D em-model is proposed; 2) the method can be applied to different kinds of air targets such as aircraft, satellite, missile, etc.; 3) the proposed attitude measurement method does not require target motion model in advance; and 4) the proposed algorithm can be applied to any kind of step frequency waveforms. Experiments using both data predicted by a high-frequency electromagnetic code and data measured in the chamber verify the validity of the method.

Beschreibung: The Soil Moisture and Ocean Salinity (SMOS) and Soil Moisture Active Passive (SMAP) missions provide Level-1 brightness temperature (Tb) observations that are used for global soil moisture estimation. However, the nature of these Tb data differs: the SMOS Tb observations contain atmospheric and select reflected extraterrestrial (“Sky”) radiation, whereas the SMAP Tb data are corrected for these contributions, using auxiliary near-surface information. Furthermore, the SMOS Tb observations are multiangular, whereas the SMAP Tb is measured at 40° incidence angle only. This letter discusses how SMOS Tb, SMAP Tb, and radiative transfer modeling components can be aligned in order to enable a seamless exchange of SMOS and SMAP Tb data in soil moisture retrieval and assimilation systems. The aggregated contribution of the atmospheric and reflected Sky radiation is, on average, about 1 K for horizontally polarized Tb and 0.5 K for vertically polarized Tb at 40° incidence angle, but local and short-term values regularly exceed 5 K.

Beschreibung: Histopathological grading of cancer not only offers an insight to the patients’ prognosis but also helps in making individual treatment plans. Mitosis counts in histopathological slides play a crucial role for invasive breast cancer grading using the Nottingham grading system. Pathologists perform this grading by manual examinations of a few thousand images for each patient. Hence, finding the mitotic figures from these images is a tedious job and also prone to observer variability due to variations in the appearances of the mitotic cells. We propose a fast and accurate approach for automatic mitosis detection from histopathological images. We employ area morphological scale space for cell segmentation. The scale space is constructed in a novel manner by restricting the scales with the maximization of relative-entropy between the cells and the background. This results in precise cell segmentation. The segmented cells are classified in mitotic and non-mitotic category using the random forest classifier. Experiments show at least 12% improvement in $F_{1}$ score on more than 450 histopathological images at $40times $ magnification.

Beschreibung: This paper proposes a fast multi-band image fusion algorithm, which combines a high-spatial low-spectral resolution image and a low-spatial high-spectral resolution image. The well admitted forward model is explored to form the likelihoods of the observations. Maximizing the likelihoods leads to solving a Sylvester equation. By exploiting the properties of the circulant and downsampling matrices associated with the fusion problem, a closed-form solution for the corresponding Sylvester equation is obtained explicitly, getting rid of any iterative update step. Coupled with the alternating direction method of multipliers and the block coordinate descent method, the proposed algorithm can be easily generalized to incorporate prior information for the fusion problem, allowing a Bayesian estimator. Simulation results show that the proposed algorithm achieves the same performance as the existing algorithms with the advantage of significantly decreasing the computational complexity of these algorithms.

Beschreibung: In recent years, baggage screening at airports has included the use of dual-energy X-ray computed tomography (DECT), an advanced technology for nondestructive evaluation. The main challenge remains to reliably find and identify threat objects in the bag from DECT data. This task is particularly hard due to the wide variety of objects, the high clutter, and the presence of metal, which causes streaks and shading in the scanner images. Image noise and artifacts are generally much more severe than in medical CT and can lead to splitting of objects and inaccurate object labeling. The conventional approach performs object segmentation and material identification in two decoupled processes. Dual-energy information is typically not used for the segmentation, and object localization is not explicitly used to stabilize the material parameter estimates. We propose a novel learning-based framework for joint segmentation and identification of objects directly from volumetric DECT images, which is robust to streaks, noise and variability due to clutter. We focus on segmenting and identifying a small set of objects of interest with characteristics that are learned from training images, and consider everything else as background. We include data weighting to mitigate metal artifacts and incorporate an object boundary field to reduce object splitting. The overall formulation is posed as a multilabel discrete optimization problem and solved using an efficient graph-cut algorithm. We test the method on real data and show its potential for producing accurate labels of the objects of interest without splits in the presence of metal and clutter.

Beschreibung: Feature point matching is a fundamental and challenging problem in many computer vision applications. In this paper, a robust feature point matching algorithm named spatial order constraints bilateral-neighbor vote (SOCBV) is proposed to remove outliers for a set of matches (including outliers) between two images. A directed ${k}$ nearest neighbor ( knn ) graph of match sets is generated, and the problem of feature point matching is formulated as a binary discrimination problem. In the discrimination process, the class labeled matrix is built via the spatial order constraints defined on the edges that connect a point to its knn . Then, the posterior inlier class probability of each match is estimated with the knn density estimation and spatial order constraints. The vote of each match is determined by averaging all posterior class probabilities that originate from its associative inliers set and is used for removing outliers. The algorithm iteratively removes outliers from the directed graph and recomputes the votes until the stopping condition is satisfied. Compared with other popular algorithms, such as RANSAC, RSOC, GTM, SOC and WGTM, experiments under various testing data sets demonstrate strong robustness for the proposed algorithm.

Beschreibung: This paper presents a novel low-complexity motion estimation and mode decision algorithm for encoding multiple quality layers following the H.264/scalable video coding standard, considering both coarse grain scalability (CGS) and medium grain scalability (MGS). The proposed algorithm conducts motion estimation and mode decision only at the base layer (BL) and enforces the higher layers to inherit the motion and mode decisions of the BL. In order for the decision made at the BL to be nearly optimal for all layers, we use the highest layer reconstructed frame as the reference frame for motion estimation and set the Lagrangian multipliers according to the quantization parameter of the current and higher layers. We also propose a simple early skip/direct decision to further boost the encoding speed. Mode decision and motion estimation is conducted at a higher layer only if the layer below it uses the skip/direct mode for a block. Significant complexity reduction can be achieved because the mode and motion estimation is performed at most once for each macroblock. Because the mode and motion information only needs to be transmitted once, we also achieve a slightly better rate-distortion (R–D) performance for typical videos. Experiments have shown more than $2times $ (up to $5times $ ) speedup for a three-layer encoder against the conventional R–D optimized reference software JSVM on both CIF and HD sequences, and for both CGS and MGS, with the tradeoff of the coding efficiency measured by the Bjontegaard delta rate.

Beschreibung: In this paper, we propose a novel unifying framework using a Markov network to learn the relationships among multiple classifiers. In face recognition, we assume that we have several complementary classifiers available, and assign observation nodes to the features of a query image and hidden nodes to those of gallery images. Under the Markov assumption, we connect each hidden node to its corresponding observation node and the hidden nodes of neighboring classifiers. For each observation-hidden node pair, we collect the set of gallery candidates most similar to the observation instance, and capture the relationship between the hidden nodes in terms of a similarity matrix among the retrieved gallery images. Posterior probabilities in the hidden nodes are computed using the belief propagation algorithm, and we use marginal probability as the new similarity value of the classifier. The novelty of our proposed framework lies in the method that considers classifier dependence using the results of each neighboring classifier. We present the extensive evaluation results for two different protocols, known and unknown image variation tests, using four publicly available databases: 1) the Face Recognition Grand Challenge ver. 2.0; 2) XM2VTS; 3) BANCA; and 4) Multi-PIE. The result shows that our framework consistently yields improved recognition rates in various situations.

Beschreibung: Ellipse fitting is widely applied in the fields of computer vision and automatic manufacture. However, the introduced edge point errors (especially outliers) from image edge detection will cause severe performance degradation of the subsequent ellipse fitting procedure. To alleviate the influence of outliers, we develop a robust ellipse fitting method in this paper. The main contributions of this paper are as follows. First, to be robust against the outliers, we introduce the maximum correntropy criterion into the constrained least-square (CLS) ellipse fitting method, and apply the half-quadratic optimization algorithm to solve the nonlinear and nonconvex problem in an alternate manner. Second, to ensure that the obtained solution is related to an ellipse, we introduce a special quadratic equality constraint into the aforementioned CLS model, which results in the nonconvex quadratically constrained quadratic programming problem. Finally, we derive the semidefinite relaxation version of the aforementioned problem in terms of the trace operator and thus determine the ellipse parameters using semidefinite programming. Some simulated and experimental examples are presented to illustrate the effectiveness of the proposed ellipse fitting approach.

Beschreibung: State-of-the-art web image search frameworks are often based on the bag-of-visual-words (BoVWs) model and the inverted index structure. Despite the simplicity, efficiency, and scalability, they often suffer from low precision and/or recall, due to the limited stability of local features and the considerable information loss on the quantization stage. To refine the quality of retrieved images, various postprocessing methods have been adopted after the initial search process. In this paper, we investigate the online querying process from a graph-based perspective. We introduce a heterogeneous graph model containing both image and feature nodes explicitly, and propose an efficient reranking approach consisting of two successive modules, i.e., incremental query expansion and image-feature voting, to improve the recall and precision, respectively. Compared with the conventional reranking algorithms, our method does not require using geometric information of visual words, therefore enjoys low consumptions of both time and memory. Moreover, our method is independent of the initial search process, and could cooperate with many BoVW-based image search pipelines, or adopted after other postprocessing algorithms. We evaluate our approach on large-scale image search tasks and verify its competitive search performance.

Beschreibung: The study of fluid flow through solid matter by computed tomography (CT) imaging has many applications, ranging from petroleum and aquifer engineering to biomedical, manufacturing, and environmental research. To avoid motion artifacts, current experiments are often limited to slow fluid flow dynamics. This severely limits the applicability of the technique. In this paper, a new iterative CT reconstruction algorithm for improved a temporal/spatial resolution in the imaging of fluid flow through solid matter is introduced. The proposed algorithm exploits prior knowledge in two ways. First, the time-varying object is assumed to consist of stationary (the solid matter) and dynamic regions (the fluid flow). Second, the attenuation curve of a particular voxel in the dynamic region is modeled by a piecewise constant function over time, which is in accordance with the actual advancing fluid/air boundary. Quantitative and qualitative results on different simulation experiments and a real neutron tomography data set show that, in comparison with the state-of-the-art algorithms, the proposed algorithm allows reconstruction from substantially fewer projections per rotation without image quality loss. Therefore, the temporal resolution can be substantially increased, and thus fluid flow experiments with faster dynamics can be performed.

Beschreibung: Most existing approaches for RGB-D indoor scene labeling employ hand-crafted features for each modality independently and combine them in a heuristic manner. There has been some attempt on directly learning features from raw RGB-D data, but the performance is not satisfactory. In this paper, we propose an unsupervised joint feature learning and encoding (JFLE) framework for RGB-D scene labeling. The main novelty of our learning framework lies in the joint optimization of feature learning and feature encoding in a coherent way, which significantly boosts the performance. By stacking basic learning structure, higher level features are derived and combined with lower level features for better representing RGB-D data. Moreover, to explore the nonlinear intrinsic characteristic of data, we further propose a more general joint deep feature learning and encoding (JDFLE) framework that introduces the nonlinear mapping into JFLE. The experimental results on the benchmark NYU depth dataset show that our approaches achieve competitive performance, compared with the state-of-the-art methods, while our methods do not need complex feature handcrafting and feature combination and can be easily applied to other data sets.

Beschreibung: Out-of-focus blur occurs frequently in multispectral imaging systems when the camera is well focused at a specific (reference) imaging channel. As the effective focal lengths of the lens are wavelength dependent, the blurriness levels of the images at individual channels are different. This paper proposes a multispectral image deblurring framework to restore out-of-focus spectral images based on the characteristic of interchannel correlation (ICC). The ICC is investigated based on the fact that a high-dimensional color spectrum can be linearly approximated using rather a few number of intrinsic spectra. In the method, the spectral images are classified into an out-of-focus set and a well-focused set via blurriness computation. For each out-of-focus image, a guiding image is derived from the well-focused spectral images and is used as the image prior in the deblurring framework. The out-of-focus blur is modeled as a Gaussian point spread function, which is further employed as the blur kernel prior. The regularization parameters in the image deblurring framework are determined using generalized cross validation, and thus the proposed method does not need any parameter tuning. The experimental results validate that the method performs well on multispectral image deblurring and outperforms the state of the arts.

Beschreibung: How to produce the difference data of the two temporal images is a crucial factor in image change detection. In this letter, we propose multicontextual mutual information data (MMID) based on the bivariate Gaussian distribution (BGD) for synthetic aperture radar (SAR) image change detection and illustrate their superiorities over the classical difference data. MMID, which are an improved form of image spatial mutual information, are constructed based on the quadrilateral Markov random field (QMRF) and can be factored into the linear combination of the entropies. Then to adapt MMID to the change detection, we construct the 2-D entropies based on the BGD. In this way, MMID are able to capture the intertemporal statistical dependence of the two temporal images and thus can be taken as the feature-level difference data rather than the pixel-level data. The maximum-likelihood method, the automatic threshold method, and the Markov random field method are performed on the MMID of the real two temporal SAR images for the change detection. Experimental results demonstrate the superiorities of MMID over the traditional difference data.

Beschreibung: Spectral unmixing has been a popular technique for analyzing remotely sensed hyperspectral images. The goal of unmixing is to find a collection of pure spectral constituents (called endmembers ) that can explain each (possibly mixed) pixel of the scene as a combination of endmembers, weighted by their coverage fractions in the pixel or abundances . Over the last years, many algorithms have been presented to address the three main parts of the spectral unmixing chain: 1) estimation of the number of endmembers; 2) identification of the endmember signatures; and 3) estimation of the per-pixel fractional abundances. However, to date, there is no standardized tool that integrates these algorithms in a unified framework. In this letter, we present HyperMix, an open-source tool for spectral unmixing that integrates different approaches for spectral unmixing and allows building unmixing chains in graphical fashion, so that the end-user can define one or several spectral unmixing chains in fully configurable mode. HyperMix provides efficient implementations of most of the algorithms used for spectral unmixing, so that the tool automatically recognizes if the computer has a graphics processing unit (GPU) available and optimizes the execution of these algorithms in the GPU. This allows for the execution of spectral unmixing chains on large hyperspectral scenes in computationally efficient fashion. The tool is available online from http://hypercomphypermix.blogspot.com.es and has been validated with real hyperspectral scenes, providing state-of-the-art unmixing results.

Beschreibung: Seismic signals are nonlinear, and the seismic state-space model can be described as a nonlinear system. The particle filter (PF) method, as an effective method for estimating the state of a nonlinear system, can be applied to deal with seismic random noise attenuation. However, PF suffers from sample impoverishment caused by resampling, which results in serious loss of valid seismic information and leads to inaccurate representation of the reflected signal. To address the impoverishment issue and to further improve the particle quality, we propose a novel method to suppress seismic random noise—the adaptive fission particle filter (AFPF). In AFPF, all the particles undergo a fission process and produce “offspring” particles to maintain particle diversity. To implement the adaptation and to monitor the degree of fission, we apply a fission factor, which takes into account weights that indicate the quality of the particles. This leads to significant improvements in the particle quality, i.e., the proportion of highly weighted particles is increased. The effective seismic information provided by the resulting particles reproduces the true signal more reliably, reducing the bias of PF. In addition, we establish a dynamic state-space model suitable for seismic signals. Experimental results on synthetic records and field data illustrate the superior performance of AFPF in noise attenuation and reflected signal preservation compared with the PF.

Beschreibung: We present a novel spatiotemporal saliency detection method to estimate salient regions in videos based on the gradient flow field and energy optimization. The proposed gradient flow field incorporates two distinctive features: 1) intra-frame boundary information and 2) inter-frame motion information together for indicating the salient regions. Based on the effective utilization of both intra-frame and inter-frame information in the gradient flow field, our algorithm is robust enough to estimate the object and background in complex scenes with various motion patterns and appearances. Then, we introduce local as well as global contrast saliency measures using the foreground and background information estimated from the gradient flow field. These enhanced contrast saliency cues uniformly highlight an entire object. We further propose a new energy function to encourage the spatiotemporal consistency of the output saliency maps, which is seldom explored in previous video saliency methods. The experimental results show that the proposed algorithm outperforms state-of-the-art video saliency detection methods.

Beschreibung: Hyperspectral unmixing is one of the crucial steps for many hyperspectral applications. The problem of hyperspectral unmixing has proved to be a difficult task in unsupervised work settings where the endmembers and abundances are both unknown. In addition, this task becomes more challenging in the case that the spectral bands are degraded by noise. This paper presents a robust model for unsupervised hyperspectral unmixing. Specifically, our model is developed with the correntropy-based metric where the nonnegative constraints on both endmembers and abundances are imposed to keep physical significance. Besides, a sparsity prior is explicitly formulated to constrain the distribution of the abundances of each endmember. To solve our model, a half-quadratic optimization technique is developed to convert the original complex optimization problem into an iteratively reweighted nonnegative matrix factorization with sparsity constraints. As a result, the optimization of our model can adaptively assign small weights to noisy bands and put more emphasis on noise-free bands. In addition, with sparsity constraints, our model can naturally generate sparse abundances. Experiments on synthetic and real data demonstrate the effectiveness of our model in comparison to the related state-of-the-art unmixing models.

Beschreibung: The primary objective of the TanDEM-X mission is the generation of a global high-precision digital elevation model (DEM) by using synthetic aperture radar interferometry. This letter presents the developed strategy for estimating the relative height error of the TanDEM-X DEM on a global scale. The mosaicking process of the final DEM combines all acquisitions at full resolution and is expected to be finished by late 2016. On the other hand, global mosaics can be generated starting from quicklook images already available for each single input data take. These downsized mosaics are operationally used to analyze the performance improvement that can be achieved by combining multiple acquisitions over the same ground areas and are a powerful mean for optimizing further acquisition planning. This letter reports the expected global performance of the final TanDEM-X product in advance of the full-resolution DEM. Knowledge of the global status of the TanDEM-X DEM relative height error is fundamental for optimizing the acquisition strategy and, therefore, the final performance and represents a valuable input for the scientific community as well as for selecting suitable areas for further interferometric experiments on a global scale.

Beschreibung: Support vector machines (SVMs) have been applied to land cover classification, and a number of studies have demonstrated their ability to increase classification accuracy. The high correlation between the data set and SVM training model parameters indicates the high performance of the classification model. To improve the correlation, research has focused on the integration of SVMs and other algorithms for data set selection and SVM training model parameter estimation. This letter proposes a novel method, based on a particle filter (PF), of estimating SVM training model parameters according to an observation system. By treating the SVM training function as the observation system of the PF, the new method automatically updates the SVM training model parameters to values that are more appropriate for the data set and can provide a better classification model than can the original model, wherein the parameters are set by trial and error. Various experiments were conducted using Radarsat-2 synthetic aperture radar data from the 2011 Thailand flood. The proposed method provides superior performance and a more accurate analysis compared with the standard SVM.

Beschreibung: Maize is a widely planted crop in China and in other areas of the world and plays an important role in grain production. Monitoring the growth status of maize using remote sensing technology is an important component of precision agriculture and height, as a crucial growth indicator for maize, can be retrieved from light detection and ranging (LIDAR) data. However, height extraction for crops, such as maize using airborne laser scanning point clouds results in a great number of uncertainties and challenges. Here, airborne full-waveform LIDAR data were used to extract maize height. In the first step, a workflow was designed based on the Gold deconvolution algorithm combined with a basic data process technique. The method was then tested and was determined to be effective for capturing the portion of the waveform interacting with the tops of vegetation, characterized by lower amplitude stemming from the ground. Therefore, the number of second returns from point clouds was dramatically increased. During the experiment, the number of point clouds increased nearly 50% for three of the four maize plots, as compared with the original point clouds. Compared with the commonly used Gaussian fitting algorithm, the deconvolution algorithm had the advantage of extracting an accurate position for overlapping weak signals. The height percentiles indicated that the original and Gaussian decomposition derived point clouds data underestimated and deconvolution algorithm can accurately reflect the true height of maize, particularly for the 75% and 95% height percentiles.

Beschreibung: The Moderate Resolution Imaging Spectroradiometer (MODIS) is an important data set in global burned-area mapping. The MODIS global burned-area product has a coarse spatial resolution at approximately 500 m, which often introduces errors to the mapped burned areas. In this letter, a novel subpixel mapping (SPM) approach was proposed to produce burned-area maps at the fine spatial resolution similar to Landsat imagery, by exploring the spectral and spatial information provided by the second and fifth bands of MODIS. The proposed SPM approach aims to refine the estimate of burned areas, which have been detected by the MODIS global burned-area product. The performance of the proposed SPM approach was assessed with an experiment area containing six burned areas, by comparing with the MODIS burned-area product MCD45. The result shows that the average omission error decreased from 52.26% for MCD45 to 16.74% for SPM, and the average commission error decreased from 21.76% for MCD45 to 12.54% for SPM. The kappa value increased from 0.5583 for MCD45 to 0.8756 for SPM, indicating that the proposed SPM approach is effective in reducing the influence of the coarse spatial resolution of MODIS imagery in mapping a burned area and refining existing global burned-area products.

Beschreibung: Radiometric distortions caused by rugged terrain make the classification of forest types from satellite imagery a challenge. Various band-specific topographic normalization models are expected to eliminate or reduce these effects. The quality of these models also depends on the approach to estimate empirical parameters. Generally, a global estimation of these parameters from a whole satellite image is simple, but it may tend to overcorrection, particularly for larger areas. A land-cover-specific method usually performs better, but it requires obtaining a priori land classification, which presents another challenge in many cases. Empirical parameters can be directly estimated from local pixels in a given window. In this letter, we propose and evaluate a central-pixel-based parameter estimation method for topographic normalization using local window pixels. We tested the method with Landsat 8 imagery and the Advanced Spaceborne Thermal Emission and Reflection Radiometer Global Digital Elevation Model (ASTER GDEM) in very rough terrain with diverse forest types. Visual comparison and statistical analyses showed that the proposed method performed better at a range of window sizes compared with an uncorrected image or with a global parameter estimation approach. The intraclass spectral variability of each forest type has been reduced significantly, and it can yield higher accuracy of forest type classification. The proposed method does not require the a priori knowledge of land covers. Its simplicity and robustness suggest that this method has the potential to be a standard preprocessing approach for optical satellite imagery, particularly for rough terrain.

Beschreibung: We present a hierarchical grid-based, globally optimal tracking-by-detection approach to track an unknown number of targets in complex and dense scenarios, particularly addressing the challenges of complex interaction and mutual occlusion. Frame-by-frame detection is performed by hierarchical likelihood grids, matching shape templates through a fast oriented distance transform. To allow recovery from misdetections, common heuristics such as nonmaxima suppression within observations is eschewed. Within a discretized state-space, the data association problem is formulated as a grid-based network flow model, resulting in a convex problem casted into an integer linear programming form, giving a global optimal solution. In addition, we show how a behavior cue (body orientation) can be integrated into our association affinity model, providing valuable hints for resolving ambiguities between crossing trajectories. Unlike traditional motion-based approaches, we estimate body orientation by a hybrid methodology, which combines the merits of motion-based and 3D appearance-based orientation estimation, thus being capable of dealing also with still-standing or slowly moving targets. The performance of our method is demonstrated through experiments on a large variety of benchmark video sequences, including both indoor and outdoor scenarios.

Beschreibung: Many impulse noise (IN) reduction methods suffer from two obstacles, the improper noise detectors and imperfect filters they used. To address such issue, in this paper, a weighted couple sparse representation model is presented to remove IN. In the proposed model, the complicated relationships between the reconstructed and the noisy images are exploited to make the coding coefficients more appropriate to recover the noise-free image. Moreover, the image pixels are classified into clear, slightly corrupted, and heavily corrupted ones. Different data-fidelity regularizations are then accordingly applied to different pixels to further improve the denoising performance. In our proposed method, the dictionary is directly trained on the noisy raw data by addressing a weighted rank-one minimization problem, which can capture more features of the original data. Experimental results demonstrate that the proposed method is superior to several state-of-the-art denoising methods.

Beschreibung: In this paper, a hierarchical multi-task structural learning algorithm is developed to support large-scale plant species identification, where a visual tree is constructed for organizing large numbers of plant species in a coarse-to-fine fashion and determining the inter-related learning tasks automatically. For a given parent node on the visual tree, it contains a set of sibling coarse-grained categories of plant species or sibling fine-grained plant species, and a multi-task structural learning algorithm is developed to train their inter-related classifiers jointly for enhancing their discrimination power. The inter-level relationship constraint, e.g., a plant image must first be assigned to a parent node (high-level non-leaf node) correctly if it can further be assigned to the most relevant child node (low-level non-leaf node or leaf node) on the visual tree, is formally defined and leveraged to learn more discriminative tree classifiers over the visual tree. Our experimental results have demonstrated the effectiveness of our hierarchical multi-task structural learning algorithm on training more discriminative tree classifiers for large-scale plant species identification.

Beschreibung: Many methods have been developed to detect damaged buildings due to earthquake. However, little attention has been paid to analyze slightly affected buildings. In this letter, an unsupervised method is presented to detect earthquake-triggered “ roof-holes ” on rural houses from unmanned aerial vehicle (UAV) images. First, both orthomosaic and gradient images are generated from a set of UAV images. Then, a modified Chinese restaurant franchise model is used to learn an unsupervised model of the geo-object classes in the area by fusing both oversegmented orthomosaic and gradient images. Finally, “roof-holes” on rural houses are detected using the learned model. The performance of the proposed method is evaluated in terms of both qualitative and quantitative indexes.

Beschreibung: Automatic target generation process (ATGP) has been used in a wide range of applications in hyperspectral image analysis. It performs a sequence of orthogonal subspace projections to extract potential targets of interest. This letter presents a recursive version of the ATGP, which is referred to as the recursive ATGP (RATGP) and has three advantages over the ATGP as follows: 1) there is no need of inverting a matrix as the ATGP does for finding each new target; 2) there is a significant reduction in the computational complexity in the hardware design due to its recursive structure; and 3) there is an automatic stopping rule that can be derived by the Neyman–Pearson detection theory to terminate the algorithm.

Beschreibung: In this letter, we propose a novel automatic algorithm for road extraction from remote sensing images. The algorithm includes low- and high-level processing. In the low-level processing, we determine a normalized second derivative map of road profiles of a generalized bar shape, which is width invariant and contrast proportional, and accordingly obtain initial road center pixels. In the high-level processing, using the map and initial center pixels, we initially determine road segments. The segments are then locally refined using their orientation randomness and length-to-width ratio and further refined via global graph-cut optimization. A final road network is thereby extracted in a robust manner. Experimental results demonstrate that the proposed algorithm provides noticeably more robust and higher road extraction performance in various images compared with the existing algorithms.

Beschreibung: Remote sensing images often need to be coded and/or transmitted with constrained computational resources. Among other features, such images commonly have high spatial, spectral, and bit-depth resolution, which may render difficult their handling. This letter introduces an embedded quantization scheme based on two-step scalar deadzone quantization (2SDQ) that enhances the quality of transmitted images when coded with a constrained number of bits. The proposed scheme is devised for use in JPEG2000. It is named cell-based 2SDQ since it uses cells, i.e., small sets of wavelet coefficients within the codeblocks defined by JPEG2000. Cells permit a finer discrimination of coefficients in which to apply the proposed quantizer. Experimental results indicate that the proposed scheme is especially beneficial for high bit-depth hyperspectral images.

Beschreibung: High suspended solid (SS) concentrations in coastal waters are aesthetically undesirable, and adversely affect fisheries and coastal ecosystems. Environmental agencies usually require frequent measurements of SS over coastal regions at a spatially detailed level for water quality assessment and control. To develop a method for SS estimation in the complex coastal waters of Hong Kong, an archive of 57 Landsat Thematic Mapper (TM), Enhanced Thematic Mapper Plus (ETM+), and HJ-1 A/B Charged Couple Device (CCD) images over a 13-year period from January 2000 to December 2012 was used. Atmospherically corrected Landsat TM/ETM+ and HJ-1 A/B CCD bands 1–4 along with 240 in situ field samples of SS concentration collected within 2 h of image acquisition, were used to develop and validate regression models over a wide range of SS concentrations from 0.5–56.0 mg/L. The best representation of actual SS concentrations was given by the log-transformed combination of Band 2 (Green, 0.52–0.60 $mumbox{m}$ ) and Band 3 (Red, 0.63–0.69 $mumbox{m} $ ), with correlation coefficient (R) of 0.85, root-mean-square error of 2.60 mg/L and mean absolute error of 2.04 mg/L. This is attributed to the sensitivity of SS to green and red wavelengths specific to the characteristic refractive index and grain size of SS found in Hong Kong waters. This letter is considered more robust than previous studies, due to the much larger number of images and in situ samples used for model development and validation, as well as the different times of year and wide range of SS concentrations investigated.

Beschreibung: The quantitative estimation of the fractional cover of carbonate rock (CR) is critical for natural resource management and ecological conservation in karst areas. Based on the analysis of spectral properties of CR together with other land cover types, we proposed two CR indices (CRIs) and established the model that represents the relationships between the CRIs and the fractional cover of CR. Then, the fractional cover of CR was estimated by using the developed model. Experimental results on Landsat-8 Operational Land Imager images acquired at Southwestern China demonstrated the effectiveness of the developed model. Compared with other indices, the proposed CRIs show the highest correlations with the fractional cover of CR.

Beschreibung: This paper proposes a two-stage texture synthesis algorithm. At the first stage, a structure tensor map carrying information about the local orientation is synthesized from the exemplar’s data and used at the second stage to constrain the synthesis of the texture. Keeping in mind that the algorithm should be able to reproduce as faithfully as possible the visual aspect, statistics, and morphology of the input sample, the method is tested on various textures and compared objectively with existing methods, highlighting its strength in successfully synthesizing the output texture in many situations where traditional algorithms fail to reproduce the exemplar’s patterns. The promising results pave the way towards the synthesis of accurately large and multi-scale patterns as it is the case for carbon material samples showing laminar structures, for example.

Beschreibung: An image search reranking (ISR) technique aims at refining text-based search results by mining images’ visual content. Feature extraction and ranking function design are two key steps in ISR. Inspired by the idea of hypersphere in one-class classification, this paper proposes a feature extraction algorithm named hypersphere-based relevance preserving projection (HRPP) and a ranking function called hypersphere-based rank (H-Rank). Specifically, an HRPP is a spectral embedding algorithm to transform an original high-dimensional feature space into an intrinsically low-dimensional hypersphere space by preserving the manifold structure and a relevance relationship among the images. An H-Rank is a simple but effective ranking algorithm to sort the images by their distances to the hypersphere center. Moreover, to capture the user’s intent with minimum human interaction, a reversed $k$ -nearest neighbor (KNN) algorithm is proposed, which harvests enough pseudorelevant images by requiring that the user gives only one click on the initially searched images. The HRPP method with reversed KNN is named one-click-based HRPP (OC-HRPP). Finally, an OC-HRPP algorithm and the H-Rank algorithm form a new ISR method, H-reranking. Extensive experimental results on three large real-world data sets show that the proposed algorithms are effective. Moreover, the fact that only one relevant image is required to be labeled makes it has a strong practical significance.

Beschreibung: In this paper, we propose a novel method for image fusion with a high-resolution panchromatic image and a low-resolution multispectral (Ms) image at the same geographical location. The fusion is formulated as a convex optimization problem which minimizes a linear combination of a least-squares fitting term and a dynamic gradient sparsity regularizer. The former is to preserve accurate spectral information of the Ms image, while the latter is to keep sharp edges of the high-resolution panchromatic image. We further propose to simultaneously register the two images during the fusing process, which is naturally achieved by virtue of the dynamic gradient sparsity property. An efficient algorithm is then devised to solve the optimization problem, accomplishing a linear computational complexity in the size of the output image in each iteration. We compare our method against six state-of-the-art image fusion methods on Ms image data sets from four satellites. Extensive experimental results demonstrate that the proposed method substantially outperforms the others in terms of both spatial and spectral qualities. We also show that our method can provide high-quality products from coarsely registered real-world IKONOS data sets. Finally, a MATLAB implementation is provided to facilitate future research.

Beschreibung: Automatic fluorescent particle tracking is an essential task to study the dynamics of a large number of biological structures at a sub-cellular level. We have developed a probabilistic particle tracking approach based on multi-scale detection and two-step multi-frame association. The multi-scale detection scheme allows coping with particles in close proximity. For finding associations, we have developed a two-step multi-frame algorithm, which is based on a temporally semiglobal formulation as well as spatially local and global optimization. In the first step, reliable associations are determined for each particle individually in local neighborhoods. In the second step, the global spatial information over multiple frames is exploited jointly to determine optimal associations. The multi-scale detection scheme and the multi-frame association finding algorithm have been combined with a probabilistic tracking approach based on the Kalman filter. We have successfully applied our probabilistic tracking approach to synthetic as well as real microscopy image sequences of virus particles and quantified the performance. We found that the proposed approach outperforms previous approaches.

Beschreibung: In this paper, we propose a novel model, a discriminatively learned iterative shrinkage (DLIS) model, for color image denoising. The DLIS is a generalization of wavelet shrinkage by iteratively performing shrinkage over patch groups and whole image aggregation. We discriminatively learn the shrinkage functions and basis from the training pairs of noisy/noise-free images, which can adaptively handle different noise characteristics in luminance/chrominance channels, and the unknown structured noise in real-captured color images. Furthermore, to remove the splotchy real color noises, we design a Laplacian pyramid-based denoising framework to progressively recover the clean image from the coarsest scale to the finest scale by the DLIS model learned from the real color noises. Experiments show that our proposed approach can achieve the state-of-the-art denoising results on both synthetic denoising benchmark and real-captured color images.

Beschreibung: In cross-view action recognition, what you saw in one view is different from what you recognize in another view, since the data distribution even the feature space can change from one view to another. In this paper, we address the problem of transferring action models learned in one view (source view) to another different view (target view), where action instances from these two views are represented by heterogeneous features. A novel learning method, called heterogeneous transfer discriminant-analysis of canonical correlations (HTDCC), is proposed to discover a discriminative common feature space for linking source view and target view to transfer knowledge between them. Two projection matrices are learned to, respectively, map data from the source view and the target view into a common feature space via simultaneously minimizing the canonical correlations of interclass training data, maximizing the canonical correlations of intraclass training data, and reducing the data distribution mismatch between the source and target views in the common feature space. In our method, the source view and the target view neither share any common features nor have any corresponding action instances. Moreover, our HTDCC method is capable of handling only a few or even no labeled samples available in the target view, and can also be easily extended to the situation of multiple source views. We additionally propose a weighting learning framework for multiple source views adaptation to effectively leverage action knowledge learned from multiple source views for the recognition task in the target view. Under this framework, different source views are assigned different weights according to their different relevances to the target view. Each weight represents how contributive the corresponding source view is to the target view. Extensive experiments on the IXMAS data set demonstrate the effectiveness of HTDCC on learning the common feature space for heterogeneous cross-view action rec- gnition. In addition, the weighting learning framework can achieve promising results on automatically adapting multiple transferred source-view knowledge to the target view.

Beschreibung: A complete encoding solution for efficient intra-based depth map compression is proposed in this paper. The algorithm, denominated predictive depth coding (PDC), was specifically developed to efficiently represent the characteristics of depth maps, mostly composed by smooth areas delimited by sharp edges. At its core, PDC involves a directional intra prediction framework and a straightforward residue coding method, combined with an optimized flexible block partitioning scheme. In order to improve the algorithm in the presence of depth edges that cannot be efficiently predicted by the directional modes, a constrained depth modeling mode, based on explicit edge representation, was developed. For residue coding, a simple and low complexity approach was investigated, using constant and linear residue modeling, depending on the prediction mode. The performance of the proposed intra depth map coding approach was evaluated based on the quality of the synthesized views using the encoded depth maps and original texture views. The experimental tests based on all intra configuration demonstrated the superior rate-distortion performance of PDC, with average bitrate savings of 6%, when compared with the current state-of-the-art intra depth map coding solution present in the 3D extension of a high-efficiency video coding (3D-HEVC) standard. By using view synthesis optimization in both PDC and 3D-HEVC encoders, the average bitrate savings increase to 14.3%. This suggests that the proposed method, without using transform-based residue coding, is an efficient alternative to the current 3D-HEVC algorithm for intra depth map coding.

Beschreibung: Person re-identification aims to match people across non-overlapping camera views, which is an important but challenging task in video surveillance. In order to obtain a robust metric for matching, metric learning has been introduced recently. Most existing works focus on seeking a Mahalanobis distance by employing sparse pairwise constraints, which utilize image pairs with the same person identity as positive samples, and select a small portion of those with different identities as negative samples. However, this training strategy has abandoned a large amount of discriminative information, and ignored the relative similarities. In this paper, we propose a novel relevance metric learning method with listwise constraints (RMLLCs) by adopting listwise similarities, which consist of the similarity list of each image with respect to all remaining images. By virtue of listwise similarities, RMLLC could capture all pairwise similarities, and consequently learn a more discriminative metric by enforcing the metric to conserve predefined similarity lists in a low-dimensional projection subspace. Despite the performance enhancement, RMLLC using predefined similarity lists fails to capture the relative relevance information, which is often unavailable in practice. To address this problem, we further introduce a rectification term to automatically exploit the relative similarities, and develop an efficient alternating iterative algorithm to jointly learn the optimal metric and the rectification term. Extensive experiments on four publicly available benchmarking data sets are carried out and demonstrate that the proposed method is significantly superior to the state-of-the-art approaches. The results also show that the introduction of the rectification term could further boost the performance of RMLLC.

Beschreibung: Tomographic iterative reconstruction methods need a very thorough modeling of data. This point becomes critical when the number of available projections is limited. At the core of this issue is the projector design, i.e., the numerical model relating the representation of the object of interest to the projections on the detector. Voxel driven and ray driven projection models are widely used for their short execution time in spite of their coarse approximations. Distance driven model has an improved accuracy but makes strong approximations to project voxel basis functions. Cubic voxel basis functions are anisotropic, accurately modeling their projection is, therefore, computationally expensive. Both smoother and more isotropic basis functions better represent the continuous functions and provide simpler projectors. These considerations have led to the development of spherically symmetric volume elements, called blobs. Set apart their isotropy, blobs are often considered too computationally expensive in practice. In this paper, we consider using separable B-splines as basis functions to represent the object, and we propose to approximate the projection of these basis functions by a 2D separable model. When the degree of the B-splines increases, their isotropy improves and projections can be computed regardless of their orientation. The degree and the sampling of the B-splines can be chosen according to a tradeoff between approximation quality and computational complexity. We quantitatively measure the good accuracy of our model and compare it with other projectors, such as the distance-driven and the model proposed by Long et al. From the numerical experiments, we demonstrate that our projector with an improved accuracy better preserves the quality of the reconstruction as the number of projections decreases. Our projector with cubic B-splines requires about twice as many operations as a model based on voxel basis functions. Higher accuracy projectors can be used to - mprove the resolution of the existing systems, or to reduce the number of projections required to reach a given resolution, potentially reducing the dose absorbed by the patient.

Beschreibung: Despite important recent advances, the vulnerability of biometric systems to spoofing attacks is still an open problem. Spoof attacks occur when impostor users present synthetic biometric samples of a valid user to the biometric system seeking to deceive it. Considering the case of face biometrics, a spoofing attack consists in presenting a fake sample (e.g., photograph, digital video, or even a 3D mask) to the acquisition sensor with the facial information of a valid user. In this paper, we introduce a low cost and software-based method for detecting spoofing attempts in face recognition systems. Our hypothesis is that during acquisition, there will be inevitable artifacts left behind in the recaptured biometric samples allowing us to create a discriminative signature of the video generated by the biometric sensor. To characterize these artifacts, we extract time-spectral feature descriptors from the video, which can be understood as a low-level feature descriptor that gathers temporal and spectral information across the biometric sample and use the visual codebook concept to find mid-level feature descriptors computed from the low-level ones. Such descriptors are more robust for detecting several kinds of attacks than the low-level ones. The experimental results show the effectiveness of the proposed method for detecting different types of attacks in a variety of scenarios and data sets, including photos, videos, and 3D masks.

Beschreibung: Target representation is a necessary component for a robust tracker. However, during tracking, many complicated factors may make the accumulated errors in the representation significantly large, leading to tracking drift. This paper aims to improve the robustness of target representation to avoid the influence of the accumulated errors, such that the tracker only acquires the information that facilitates tracking and ignores the distractions. We observe that the locally mutual relations between the feature observations of temporally obtained targets are beneficial to the subspace representation in visual tracking. Thus, we propose a novel subspace learning algorithm for visual tracking, which imposes joint row-wise sparsity structure on the target subspace to adaptively exclude distractive information. The sparsity is induced by exploiting the locally mutual relations between the feature observations during learning. To this end, we formulate tracking as a subspace sparsity inducing problem. A large number of experiments on various challenging video sequences demonstrate that our tracker outperforms many other state-of-the-art trackers.

Beschreibung: Color-to-gray (C2G) image conversion is the process of transforming a color image into a grayscale one. Despite its wide usage in real-world applications, little work has been dedicated to compare the performance of C2G conversion algorithms. Subjective evaluation is reliable but is also inconvenient and time consuming. Here, we make one of the first attempts to develop an objective quality model that automatically predicts the perceived quality of C2G converted images. Inspired by the philosophy of the structural similarity index, we propose a C2G structural similarity (C2G-SSIM) index, which evaluates the luminance, contrast, and structure similarities between the reference color image and the C2G converted image. The three components are then combined depending on image type to yield an overall quality measure. Experimental results show that the proposed C2G-SSIM index has close agreement with subjective rankings and significantly outperforms existing objective quality metrics for C2G conversion. To explore the potentials of C2G-SSIM, we further demonstrate its use in two applications: 1) automatic parameter tuning for C2G conversion algorithms and 2) adaptive fusion of C2G converted images.

Beschreibung: In this paper, we propose a skin classification method exploiting faces and bodies automatically detected in the image, to adaptively initialize individual ad hoc skin classifiers. Each classifier is initialized by a face and body couple or by a single face, if no reliable body is detected. Thus, the proposed method builds an ad hoc skin classifier for each person in the image, resulting in a classifier less dependent from changes in skin color due to tan levels, races, genders, and illumination conditions. The experimental results on a heterogeneous data set of labeled images show that our proposal outperforms the state-of-the-art methods, and that this improvement is statistically significant.

Beschreibung: Local binary descriptors are attracting increasingly attention due to their great advantages in computational speed, which are able to achieve real-time performance in numerous image/vision applications. Various methods have been proposed to learn data-dependent binary descriptors. However, most existing binary descriptors aim overly at computational simplicity at the expense of significant information loss which causes ambiguity in similarity measure using Hamming distance. In this paper, by considering multiple features might share complementary information, we present a novel local binary descriptor, referred as ring-based multi-grouped descriptor (RMGD), to successfully bridge the performance gap between current binary and floated-point descriptors. Our contributions are twofold. First, we introduce a new pooling configuration based on spatial ring-region sampling, allowing for involving binary tests on the full set of pairwise regions with different shapes, scales, and distances. This leads to a more meaningful description than the existing methods which normally apply a limited set of pooling configurations. Then, an extended Adaboost is proposed for an efficient bit selection by emphasizing high variance and low correlation, achieving a highly compact representation. Second, the RMGD is computed from multiple image properties where binary strings are extracted. We cast multi-grouped features integration as rankSVM or sparse support vector machine learning problem, so that different features can compensate strongly for each other, which is the key to discriminativeness and robustness. The performance of the RMGD was evaluated on a number of publicly available benchmarks, where the RMGD outperforms the state-of-the-art binary descriptors significantly.

Beschreibung: Palmprint recognition (PR) is an effective technology for personal recognition. A main problem, which deteriorates the performance of PR, is the deformations of palmprint images. This problem becomes more severe on contactless occasions, in which images are acquired without any guiding mechanisms, and hence critically limits the applications of PR. To solve the deformation problems, in this paper, a model for non-linearly deformed palmprint matching is derived by approximating non-linear deformed palmprint images with piecewise-linear deformed stable regions. Based on this model, a novel approach for deformed palmprint matching, named key point-based block growing (KPBG), is proposed. In KPBG, an iterative M-estimator sample consensus algorithm based on scale invariant feature transform features is devised to compute piecewise-linear transformations to approximate the non-linear deformations of palmprints, and then, the stable regions complying with the linear transformations are decided using a block growing algorithm. Palmprint feature extraction and matching are performed over these stable regions to compute matching scores for decision. Experiments on several public palmprint databases show that the proposed models and the KPBG approach can effectively solve the deformation problem in palmprint verification and outperform the state-of-the-art methods.

Beschreibung: Nonnegative Tucker decomposition (NTD) is a powerful tool for the extraction of nonnegative parts-based and physically meaningful latent components from high-dimensional tensor data while preserving the natural multilinear structure of data. However, as the data tensor often has multiple modes and is large scale, the existing NTD algorithms suffer from a very high computational complexity in terms of both storage and computation time, which has been one major obstacle for practical applications of NTD. To overcome these disadvantages, we show how low (multilinear) rank approximation (LRA) of tensors is able to significantly simplify the computation of the gradients of the cost function, upon which a family of efficient first-order NTD algorithms are developed. Besides dramatically reducing the storage complexity and running time, the new algorithms are quite flexible and robust to noise, because any well-established LRA approaches can be applied. We also show how nonnegativity incorporating sparsity substantially improves the uniqueness property and partially alleviates the curse of dimensionality of the Tucker decompositions. Simulation results on synthetic and real-world data justify the validity and high efficiency of the proposed NTD algorithms.

Beschreibung: The problem of estimating the parameters of a Rayleigh-Rice mixture density is often encountered in image analysis (e.g., remote sensing and medical image processing). In this paper, we address this general problem in the framework of change detection (CD) in multitemporal and multispectral images. One widely used approach to CD in multispectral images is based on the change vector analysis. Here, the distribution of the magnitude of the difference image can be theoretically modeled by a Rayleigh-Rice mixture density. However, given the complexity of this model, in applications, a Gaussian-mixture approximation is often considered, which may affect the CD results. In this paper, we present a novel technique for parameter estimation of the Rayleigh-Rice density that is based on a specific definition of the expectation-maximization algorithm. The proposed technique, which is characterized by good theoretical properties, iteratively updates the parameters and does not depend on specific optimization routines. Several numerical experiments on synthetic data demonstrate the effectiveness of the method, which is general and can be applied to any image processing problem involving the Rayleigh-Rice mixture density. In the CD context, the Rayleigh-Rice model (which is theoretically derived) outperforms other empirical models. Experiments on real multitemporal and multispectral remote sensing images confirm the validity of the model by returning significantly higher CD accuracies than those obtained by using the state-of-the-art approaches.

Beschreibung: In this paper, we propose a very simple deep learning network for image classification that is based on very basic data processing components: 1) cascaded principal component analysis (PCA); 2) binary hashing; and 3) blockwise histograms. In the proposed architecture, the PCA is employed to learn multistage filter banks. This is followed by simple binary hashing and block histograms for indexing and pooling. This architecture is thus called the PCA network (PCANet) and can be extremely easily and efficiently designed and learned. For comparison and to provide a better understanding, we also introduce and study two simple variations of PCANet: 1) RandNet and 2) LDANet. They share the same topology as PCANet, but their cascaded filters are either randomly selected or learned from linear discriminant analysis. We have extensively tested these basic networks on many benchmark visual data sets for different tasks, including Labeled Faces in the Wild (LFW) for face verification; the MultiPIE, Extended Yale B, AR, Facial Recognition Technology (FERET) data sets for face recognition; and MNIST for hand-written digit recognition. Surprisingly, for all tasks, such a seemingly naive PCANet model is on par with the state-of-the-art features either prefixed, highly hand-crafted, or carefully learned [by deep neural networks (DNNs)]. Even more surprisingly, the model sets new records for many classification tasks on the Extended Yale B, AR, and FERET data sets and on MNIST variations. Additional experiments on other public data sets also demonstrate the potential of PCANet to serve as a simple but highly competitive baseline for texture classification and object recognition.

Beschreibung: Provides a listing of the editors, board members, and current staff for this issue of the publication.

Beschreibung: Provides a listing of the editors, board members, and current staff for this issue of the publication.

Beschreibung: The segmentation of brain MR images into different tissue classes is an important task for automatic image analysis technique, particularly due to the presence of intensity inhomogeneity artifact in MR images. In this regard, this paper presents a novel approach for simultaneous segmentation and bias field correction in brain MR images. It integrates judiciously the concept of rough sets and the merit of a novel probability distribution, called stomped normal (SN) distribution. The intensity distribution of a tissue class is represented by SN distribution, where each tissue class consists of a crisp lower approximation and a probabilistic boundary region. The intensity distribution of brain MR image is modeled as a mixture of finite number of SN distributions and one uniform distribution. The proposed method incorporates both the expectation-maximization and hidden Markov random field frameworks to provide an accurate and robust segmentation. The performance of the proposed approach, along with a comparison with related methods, is demonstrated on a set of synthetic and real brain MR images for different bias fields and noise levels.

Beschreibung: People know and care for personal objects, which can be different for individuals. Automatically discovering personal objects is thus of great practical importance. We, in this paper, pursue this task with wearable cameras based on the common sense that personal objects generally company us in various scenes. With this clue, we exploit a new object-scene distribution for robust detection. Two technical challenges involved in estimating this distribution, i.e., scene extraction and unsupervised object discovery, are tackled. For scene extraction, we learn the latent representation instead of simply selecting a few frames from the videos. In object discovery, we build an interaction model to select frame-level objects and use nonparametric Bayesian clustering. Experiments verify the usefulness of our approach.

Beschreibung: Single object tracking, in which a target is often initialized manually in the first frame and then is tracked and located automatically in the subsequent frames, is a hot topic in computer vision. The traditional tracking-by-detection framework, which often formulates tracking as a binary classification problem, has been widely applied and achieved great success in single object tracking. However, there are some potential issues in this formulation. For instance, the boundary between the positive and negative training samples is fuzzy, and the objectives of tracking and classification are inconsistent. In this paper, we attempt to address the above issues from the fuzzy system perspective and propose a novel tracking method by formulating tracking as a fuzzy classification problem. First, we introduce the fuzzy strategy into tracking and propose a novel fuzzy tracking framework, which can measure the importance of the training samples by assigning different memberships to them and offer more strict spatial constraints. Second, we develop a fuzzy least squares support vector machine (FLS-SVM) approach and employ it to implement a concrete tracker. In particular, the primal form, dual form, and kernel form of FLS-SVM are analyzed and the corresponding closed-form solutions are derived for efficient realizations. Besides, a least squares regression model is built to control the update adaptively, retaining the robustness of the appearance model. The experimental results demonstrate that our method can achieve comparable or superior performance to many state-of-the-art methods.

Beschreibung: Slow feature analysis (SFA) is a dimensionality reduction technique which has been linked to how visual brain cells work. In recent years, the SFA was adopted for computer vision tasks. In this paper, we propose an exact kernel SFA (KSFA) framework for positive definite and indefinite kernels in Krein space. We then formulate an online KSFA which employs a reduced set expansion. Finally, by utilizing a special kind of kernel family, we formulate exact online KSFA for which no reduced set is required. We apply the proposed system to develop a SFA-based change detection algorithm for stream data. This framework is employed for temporal video segmentation and tracking. We test our setup on synthetic and real data streams. When combined with an online learning tracking system, the proposed change detection approach improves upon tracking setups that do not utilize change detection.

Beschreibung: Quality assessment of 3D images encounters more challenges than its 2D counterparts. Directly applying 2D image quality metrics is not the solution. In this paper, we propose a new full-reference quality assessment for stereoscopic images by learning binocular receptive field properties to be more in line with human visual perception. To be more specific, in the training phase, we learn a multiscale dictionary from the training database, so that the latent structure of images can be represented as a set of basis vectors. In the quality estimation phase, we compute sparse feature similarity index based on the estimated sparse coefficient vectors by considering their phase difference and amplitude difference, and compute global luminance similarity index by considering luminance changes. The final quality score is obtained by incorporating binocular combination based on sparse energy and sparse complexity. Experimental results on five public 3D image quality assessment databases demonstrate that in comparison with the most related existing methods, the devised algorithm achieves high consistency with subjective assessment.

Beschreibung: Our goal is to detect and group different kinds of local symmetries in images in a scale- and rotation-invariant way. We propose an efficient wavelet-based method to determine the order of local symmetry at each location. Our algorithm relies on circular harmonic wavelets which are used to generate steerable wavelet channels corresponding to different symmetry orders. To give a measure of local symmetry, we use the F-test to examine the distribution of the energy across different channels. We provide experimental results on synthetic images, biological micrographs, and electron-microscopy images to demonstrate the performance of the algorithm.

Beschreibung: In this paper, we study a novel problem of classifying covert photos, whose acquisition processes are intentionally concealed from the subjects being photographed. Covert photos are often privacy invasive and, if distributed over Internet, can cause serious consequences. Automatic identification of such photos, therefore, serves as an important initial step toward further privacy protection operations. The problem is, however, very challenging due to the large semantic similarity between covert and noncovert photos, the enormous diversity in the photographing process and environment of cover photos, and the difficulty to collect an effective data set for the study. Attacking these challenges, we make three consecutive contributions. First, we collect a large data set containing 2500 covert photos, each of them is verified rigorously and carefully. Second, we conduct a user study on how humans distinguish covert photos from noncovert ones. The user study not only provides an important evaluation baseline, but also suggests fusing heterogeneous information for an automatic solution. Our third contribution is a covert photo classification algorithm that fuses various image features and visual attributes in the multiple kernel learning framework. We evaluate the proposed approach on the collected data set in comparison with other modern image classifiers. The results show that our approach achieves an average classification rate (1–EER) of 0.8940, which significantly outperforms other competitors as well as human’s performance.

Beschreibung: Driven by recent vision and graphics applications such as image segmentation and object recognition, computing pixel-accurate saliency values to uniformly highlight foreground objects becomes increasingly important. In this paper, we propose a unified framework called pixelwise image saliency aggregating (PISA) various bottom-up cues and priors. It generates spatially coherent yet detail-preserving, pixel-accurate, and fine-grained saliency, and overcomes the limitations of previous methods, which use homogeneous superpixel based and color only treatment. PISA aggregates multiple saliency cues in a global context, such as complementary color and structure contrast measures, with their spatial priors in the image domain. The saliency confidence is further jointly modeled with a neighborhood consistence constraint into an energy minimization formulation, in which each pixel will be evaluated with multiple hypothetical saliency levels. Instead of using global discrete optimization methods, we employ the cost-volume filtering technique to solve our formulation, assigning the saliency levels smoothly while preserving the edge-aware structure details. In addition, a faster version of PISA is developed using a gradient-driven image subsampling strategy to greatly improve the runtime efficiency while keeping comparable detection accuracy. Extensive experiments on a number of public data sets suggest that PISA convincingly outperforms other state-of-the-art approaches. In addition, with this work, we also create a new data set containing 800 commodity images for evaluating saliency detection.

Beschreibung: In this paper, we propose a new video inpainting method which applies to both static or free-moving camera videos. The method can be used for object removal, error concealment, and background reconstruction applications. To limit the computational time, a frame is inpainted by considering a small number of neighboring pictures which are grouped into a group of pictures (GoP). More specifically, to inpaint a frame, the method starts by aligning all the frames of the GoP. This is achieved by a region-based homography computation method which allows us to strengthen the spatial consistency of aligned frames. Then, from the stack of aligned frames, an energy function based on both spatial and temporal coherency terms is globally minimized. This energy function is efficient enough to provide high quality results even when the number of pictures in the GoP is rather small, e.g. 20 neighboring frames. This drastically reduces the algorithm complexity and makes the approach well suited for near real-time video editing applications as well as for loss concealment applications. Experiments with several challenging video sequences show that the proposed method provides visually pleasing results for object removal, error concealment, and background reconstruction context.

Beschreibung: Single-sensor imaging using the Bayer color filter array (CFA) and demosaicking is well established for current compact and low-cost color digital cameras. An extension from the CFA to a multispectral filter array (MSFA) enables us to acquire a multispectral image in one shot without increased size or cost. However, multispectral demosaicking for the MSFA has been a challenging problem because of very sparse sampling of each spectral band in the MSFA. In this paper, we propose a high-performance multispectral demosaicking algorithm, and at the same time, a novel MSFA pattern that is suitable for our proposed algorithm. Our key idea is the use of the guided filter to interpolate each spectral band. To generate an effective guide image, in our proposed MSFA pattern, we maintain the sampling density of the $G$ -band as high as the Bayer CFA, and we array each spectral band so that an adaptive kernel can be estimated directly from raw MSFA data. Given these two advantages, we effectively generate the guide image from the most densely sampled $G$ -band using the adaptive kernel. In the experiments, we demonstrate that our proposed algorithm with our proposed MSFA pattern outperforms existing algorithms and provides better color fidelity compared with a conventional color imaging system with the Bayer CFA. We also show some real applications using a multispectral camera prototype we built.

Beschreibung: Blind motion deblurring from a single image is a highly under-constrained problem with many degenerate solutions. A good approximation of the intrinsic image can, therefore, only be obtained with the help of prior information in the form of (often nonconvex) regularization terms for both the intrinsic image and the kernel. While the best choice of image priors is still a topic of ongoing investigation, this research is made more complicated by the fact that historically each new prior requires the development of a custom optimization method. In this paper, we develop a stochastic optimization method for blind deconvolution. Since this stochastic solver does not require the explicit computation of the gradient of the objective function and uses only efficient local evaluation of the objective, new priors can be implemented and tested very quickly. We demonstrate that this framework, in combination with different image priors produces results with Peak Signal-to-Noise Ratio (PSNR) values that match or exceed the results obtained by much more complex state-of-the-art blind motion deblurring algorithms.

Beschreibung: Recent advances in object detection have led to the development of segmentation by detection approaches that integrate top-down geometric priors for multiclass object segmentation. A key yet under-addressed issue in utilizing top-down cues for the problem of multiclass object segmentation by detection is efficiently generating robust and accurate geometric priors. In this paper, we propose a random geometric prior forest scheme to obtain object-adaptive geometric priors efficiently and robustly. In the scheme, a testing object first searches for training neighbors with similar geometries using the random geometric prior forest, and then the geometry of the testing object is reconstructed by linearly combining the geometries of its neighbors. Our scheme enjoys several favorable properties when compared with conventional methods. First, it is robust and very fast because its inference does not suffer from bad initializations, poor local minimums or complex optimization. Second, the figure/ground geometries of training samples are utilized in a multitask manner. Third, our scheme is object-adaptive but does not require the labeling of parts or poselets, and thus, it is quite easy to implement. To demonstrate the effectiveness of the proposed scheme, we integrate the obtained top-down geometric priors with conventional bottom-up color cues in the frame of graph cut. The proposed random geometric prior forest achieves the best segmentation results of all of the methods tested on VOC2010/2012 and is 90 times faster than the current state-of-the-art method.

Beschreibung: Tone mapping operators (TMOs) aim to compress high dynamic range (HDR) images to low dynamic range (LDR) ones so as to visualize HDR images on standard displays. Most existing TMOs were demonstrated on specific examples without being thoroughly evaluated using well-designed and subject-validated image quality assessment models. A recently proposed tone mapped image quality index (TMQI) made one of the first attempts on objective quality assessment of tone mapped images. Here, we propose a substantially different approach to design TMO. Instead of using any predefined systematic computational structure for tone mapping (such as analytic image transformations and/or explicit contrast/edge enhancement), we directly navigate in the space of all images, searching for the image that optimizes an improved TMQI. In particular, we first improve the two building blocks in TMQI—structural fidelity and statistical naturalness components—leading to a TMQI-II metric. We then propose an iterative algorithm that alternatively improves the structural fidelity and statistical naturalness of the resulting image. Numerical and subjective experiments demonstrate that the proposed algorithm consistently produces better quality tone mapped images even when the initial images of the iteration are created by the most competitive TMOs. Meanwhile, these results also validate the superiority of TMQI-II over TMQI. 1 1 Partial preliminary results of this work were presented at ICASSP 2013 and ICME 2014.

Beschreibung: Extracting the pixel-level 3D layout from a single image is important for different applications, such as object localization, image, and video categorization. Traditionally, the 3D layout is derived by solving a pixel-level classification problem. However, the image-level 3D structure can be very beneficial for extracting pixel-level 3D layout since it implies the way how pixels in the image are organized. In this paper, we propose an approach that first predicts the global image structure, and then we use the global structure for fine-grained pixel-level 3D layout extraction. In particular, image features are extracted based on multiple layout templates. We then learn a discriminative model for classifying the global layout at the image-level. Using latent variables, we implicitly model the sublevel semantics of the image, which enrich the expressiveness of our model. After the image-level structure is obtained, it is used as the prior knowledge to infer pixel-wise 3D layout. Experiments show that the results of our model outperform the state-of-the-art methods by 11.7% for 3D structure classification. Moreover, we show that employing the 3D structure prior information yields accurate 3D scene layout segmentation.

Beschreibung: Recognizing human activities from videos is a fundamental research problem in computer vision. Recently, there has been a growing interest in analyzing human behavior from data collected with wearable cameras. First-person cameras continuously record several hours of their wearers’ life. To cope with this vast amount of unlabeled and heterogeneous data, novel algorithmic solutions are required. In this paper, we propose a multitask clustering framework for activity of daily living analysis from visual data gathered from wearable cameras. Our intuition is that, even if the data are not annotated, it is possible to exploit the fact that the tasks of recognizing everyday activities of multiple individuals are related, since typically people perform the same actions in similar environments, e.g., people working in an office often read and write documents). In our framework, rather than clustering data from different users separately, we propose to look for clustering partitions which are coherent among related tasks. In particular, two novel multitask clustering algorithms, derived from a common optimization problem, are introduced. Our experimental evaluation, conducted both on synthetic data and on publicly available first-person vision data sets, shows that the proposed approach outperforms several single-task and multitask learning methods.

Beschreibung: The inverse problem of synthetic aperture imaging radiometers (SAIRs) has been demonstrated to be not well posed. The regularization methods are crucial for providing unique and stable solutions in the reconstruction of radiometric brightness temperature (BT) maps. Different to deterministic ones, a new approach is presented by referring to the rule of Bayesian inference, providing a probability model of regularized constraints to combat the ill-posedness of finite-dimensional discrete inverse problems. In addition, the SAIR inverse problem can be converted into the probability estimation of the reconstructed BT. Furthermore, in application to both uniformly and nonuniformly spaced arrays, our method can obtain the optimal solution adaptively and avoid the dilemma of choosing the optimal regularization parameter. Finally, simulation results illustrating the effectiveness and performance of the proposed method are provided.

Beschreibung: A range-cell-focusing algorithm is proposed in order to improve the quality of the target image. In a synthetic aperture radar (SAR) imaging system, the range resolution depends on the frequency bandwidth and determines the ability to distinguish between targets that are very close to each other. In cases where the resolution and the SNR from the environment are not adequate, targets cannot be accurately visualized. In order to successively classify targets that are close, we are combining an enhanced-multiple-signal-classification spectrum as a weighting function to reproduce the raw data. The proposed algorithm improves classification and separation for close targets while suppressing artifacts in the final images. The targets of interest are stationary point scatterers. The results are obtained from both simulated and experimental data to demonstrate that the proposed algorithm provides better performance than a conventional SAR imaging algorithm, the range migration algorithm.

Beschreibung: This letter presents a multiscale edge detection method for multilook polarimetric synthetic aperture radar (PolSAR) images based on the nonsubsampled contourlet transform (NSCT). The NSCT can provide flexible multiscale and directional decomposition. In the multiscale decomposition, the coefficients of the nonsubsampled pyramid in the NSCT are calculated via maximizing the polarimetric contrast between the adjacent subband levels, instead of using the difference of the adjacent subbands as used in the additive noise model. By this way, we make the NSCT applicable to PolSAR data and multiband data. Then, the edges are detected in the NSCT domain based on a fusion of the directional subband coefficients at different scales. Experimental results with both simulated and real PolSAR data show that the present approach is robust to noise and the extracted edges are complete and continuous.

Beschreibung: A method for defining the spatial resolution of a Global Navigation Satellite System reflectometry delay–Doppler map (DDM) and of any derived geophysical product is proposed. An effective spatial resolution is derived as a function of measurement geometry and delay–Doppler (DD) interval, and as a more appropriate representation of resolution than the geometric resolution previously used in the literature. The definition more accurately accounts for variations in the scattered power across different pixels of the DDM and more accurately includes the power spreading effect caused by the Woodward ambiguity function. The dependence of the effective resolution on incidence angle, receiver altitude, and DD interval is analyzed and compared with the dependence of the geometric resolution with similar parameters.

Beschreibung: Laser scanner-captured 3-D point cloud data analysis is becoming more commonly used for remote sensing and plant science applications. Because of nonrigidity and complexity, reconstructing a 3-D model of a plant is extremely challenging. Existing algorithms often fail to find correct correspondences for plantlike thin structures. We address the problem of finding 3-D junction points in plant point cloud data as a first step of this correspondence matching process. Temporarily, we transform the 3-D problem into 2-D by performing appropriate coordinate transformations to the neighborhood of each 3-D point. Our proposed method has two steps. First, a statistical dip test of multimodality is performed to detect the nonlinearity of the local 2D structure. Then, each branch is approximated by sequential random-sample-consensus line fitting and a Euclidean clustering technique. The straight line parameters of each branch are extracted using total-least-squares estimation. Finally, the straight line equations are solved to determine if they intersect in the local neighborhood. Such junction points are good candidates for subsequent correspondence algorithms. Using these detected junction points, we formulate a correspondence algorithm as a subgraph matching problem and show that, without using traditional descriptor similarity-based matching, good correspondences can be obtained by simply considering geodesic distances among graph nodes. Experiments on synthetic and real ( Arabidopsis plant) data show that the proposed method outperforms the state of the art.

Beschreibung: Repeat-track analysis is commonly utilized to generate elevation change time series from satellite radar altimetry over ice sheets. It requires surface gradient (SG) correction due primarily to orbital drifts and radar-related empirical corrections caused by radar scatters from ice surface and potential subsurface. In this letter, two approaches, namely, the use of a digital elevation model (DEM) and the modified repeat-track analysis, which uses the accumulated Envisat altimetry profiles, are applied to correct the SG over both Greenland ice sheet (GrIS) and Antarctic ice sheet (AIS). By comparing the root mean square (rms) of elevation change time series after SG correction, the percentage of data (rms< 1 m) obtained by using modified repeat-track analysis is found to be 85% and 88% for the GrIS and AIS, respectively, as opposed to 45% and 44% if the DEM method is used. Furthermore, three cases are studied to assess empirical corrections for elevation retrieved from both ice-1 and ice-2 algorithms over the AIS. We conclude that the modified repeat-track analysis is more effective to remove topographic induced error. For the ice-2 algorithm, waveform shape parameters are needed in addition to applying corrections from changes in backscatter coefficients. The trend of elevation changes from the ice-1 algorithm with only backscatter analysis agrees with that from the ice-2 algorithm with corrections from backscatter coefficient changes and waveform shape parameters. This study could provide a potential data processing recipe for generating improved satellite radar altimetry elevation time series over ice sheets.