ISSN:
1573-7683
Keywords:
Bayesian interpolation
;
regularization
;
hyperparameters
Source:
Springer Online Journal Archives 1860-2000
Topics:
Mathematics
Notes:
Abstract When interpolating incomplete data, one can choose a parametric model, or opt for a more general approach and use a non-parametric model which allows a very large class of interpolants. A popular non-parametric model for interpolating various types of data is based on regularization, which looks for an interpolant that is both close to the data and also “smooth” in some sense. Formally, this interpolant is obtained by minimizing an error functional which is the weighted sum of a “fidelity term” and a “smoothness term”. The classical approach to regularization is: select “optimal” weights (also called hyperparameters) that should be assigned to these two terms, and minimize the resulting error functional. However, using only the “optimal weights” does not guarantee that the chosen function will be optimal in some sense, such as the maximum likelihood criterion, or the minimal square error criterion. For that, we have to consider all possible weights. The approach suggested here is to use the full probability distribution on the space of admissible functions, as opposed to the probability induced by using a single combination of weights. The reason is as follows: the weight actually determines the probability space in which we are working. For a given weight λ, the probability of a function f is proportional to exp(− λ ∫ f2 uu du) (for the case of a function with one variable). For each different λ, there is a different solution to the restoration problem; denote it by fλ. Now, if we had known λ, it would not be necessary to use all the weights; however, all we are given are some noisy measurements of f, and we do not know the correct λ. Therefore, the mathematically correct solution is to calculate, for every λ, the probability that f was sampled from a space whose probability is determined by λ, and average the different fλ's weighted by these probabilities. The same argument holds for the noise variance, which is also unknown. Three basic problems are addressed is this work: • Computing the MAP estimate, that is, the function f maximizing Pr(f/D) when the data D is given. This problem is reduced to a one-dimensional optimization problem. • Computing the MSE estimate. This function is defined at each point x as ∫f(x)Pr(f/D) Ũf. This problem is reduced to computing a one-dimensional integral. In the general setting, the MAP estimate is not equal to the MSE estimate. • Computing the pointwise uncertainty associated with the MSE solution. This problem is reduced to computing three one-dimensional integrals.
Type of Medium:
Electronic Resource
URL:
http://dx.doi.org/10.1023/A:1008317210576
Permalink
