ALBERT

Note: Contents Part I Solution of Nonlinear Systems 1 Solution of Algebraic Polynomial Systems 1.1 Zeros of Polynomial Systems 1.2 Resultant Methods 1.2.1 Sylvester Resultant 1.2.2 Dixon Resultant 1.3 Gröbner Basis 1.3.1 Greatest Common Divisor of Polynomials 1.3.2 Reduced Gröbner Basis 1.3.3 Polynomials with Inexact Coefficients 1.4 Using Dixon-EDF for Symbolic Solution of Polynomial Systems 1.4.1 Explanation of Dixon-EDF 1.4.2 Distance from a Point to a Standard Ellipsoid 1.4.3 Distance from a Point to Any 3D Conic 1.4.4 Pose Estimation 1.4.5 How to Run Dixon-EDF 1.5 Applications 1.5.1 Common Points of Geometrical Objects 1.5.2 Nonlinear Heat Transfer 1.5.3 Helmert Transformation 1.6 Exercises 1.6.1 Solving a System with Different Techniques 1.6.2 Planar Ranging 1.6.3 3D Resection 1.6.4 Pose Estimation References 2 Homotopy Solution of Nonlinear Systems 2.1 The Concept of Homotopy 2.2 Solving Nonlinear Equation via Homotopy 2.3 Tracing Homotopy Path as Initial Value Problem 2.4 Types of Linear Homotopy 2.4.1 General Linear Homotopy 2.4.2 Fixed-Point Homotopy 2.4.3 Newton Homotopy 2.4.4 Affine Homotopy 2.4.5 Mixed Homotopy 2.5 Regularization of the Homotopy Function 2.6 Start System in Case of Algebraic Polynomial Systems 2.7 Homotopy Methods in Mathematica 2.8 Parallel Computation 2.9 General Nonlinear System 2.10 Nonlinear Homotopy 2.10.1 Quadratic Bezier Homotopy Function 2.10.2 Implementation in Mathematica 2.10.3 Comparing Linear and Quadratic Homotopy 2.11 Applications 2.11.1 Nonlinear Heat Conduction 2.11.2 Local Coordinates via GNSS 2.12 Exercises 2.12.1 GNSS Positioning N-Point Problem References 3 Overdetermined and Underdetermined Systems 3.1 Concept of the Over and Underdetermined Systems 3.1.1 Overdetermined Systems 3.1.2 Underdetermined Systems 3.2 Gauss–Jacobi Combinatorial Solution 3.3 Gauss–Jacobi Solution in Case of Nonlinear Systems 3.4 Transforming Overdetermined System into a Determined System 3.5 Extended Newton–Raphson Method 3.6 Solution of Underdetermined Systems 3.6.1 Direct Minimization 3.6.2 Method of Lagrange Multipliers 3.6.3 Method of Penalty Function 3.6.4 Extended Newton–Raphson 3.7 Applications 3.7.1 Geodetic Application—The Minimum Distance Problem 3.7.2 Global Navigation Satellite System (GNSS) Application 3.7.3 Geometric Application 3.8 Exercises 3.8.1 Solution of Overdetermined System 3.8.2 Solution of Underdetermined System Part II Optimization of Systems 4 Simulated Annealing 4.1 Metropolis Algorithm 4.2 Realization of the Metropolis Algorithm 4.2.1 Representation of a State 4.2.2 The Free Energy of a State 4.2.3 Perturbation of a State 4.2.4 Accepting a New State 4.2.5 Implementation of the Algorithm 4.3 Algorithm of the Simulated Annealing 4.4 Implementation of the Algorithm 4.5 Application to Computing Minimum of a Real Function 4.6 Generalization of the Algorithm 4.7 Applications 4.7.1 A Packing Problem 4.7.2 The Traveling Salesman Problem 4.8 Exercise 5 Genetic Algorithms 5.1 The Genetic Evolution Concept 5.2 Mutation of the Best Individual 5.3 Solving a Puzzle 5.4 Application to a Real Function 5.5 Employing Sexual Reproduction 5.5.1 Selection of Parents 5.5.2 Sexual Reproduction: Crossover and Mutation 5.6 The Basic Genetic Algorithm (BGA) 5.7 Applications 5.7.1 Nonlinear Parameter Estimation 5.7.2 Packing Spheres with Different Sizes 5.7.3 Finding All the Real Solutions of a Non-algebraic System 5.8 Exercises 5.8.1 Foxhole Problem References 6 Particle Swarm Optimization 6.1 The Concept of Social Behavior of Groups of Animals 6.2 Basic Algorithm 6.3 The Pseudo Code of the Algorithm 6.4 Applications 6.4.1 1D Example 6.4.2 2D Example 6.4.3 Solution of Nonlinear Non-algebraic System 6.5 Exercise Reference 7 Integer Programming 7.1 Integer Problem 7.2 Discrete Value Problems 7.3 Simple Logical Conditions 7.4 Some Typical Problems of Binary Programming 7.4.1 Knapsack Problem 7.4.2 Nonlinear Knapsack Problem 7.4.3 Set-Covering Problem 7.5 Solution Methods 7.5.1 Binary Countdown Method 7.5.2 Branch and Bound Method 7.6 Mixed–Integer Programming 7.7 Applications 7.7.1 Integer Least Squares 7.7.2 Optimal Number of Oil Wells 7.8 Exercises 7.8.1 Study of Mixed Integer Programming 7.8.2 Mixed Integer Least Square References 8 Multiobjective Optimization 8.1 Concept of Multiobjective Problem 8.1.1 Problem Definition 8.1.2 Interpretation of the Solution 8.2 Pareto Optimum 8.2.1 Nonlinear Problems 8.2.2 Pareto-Front and Pareto-Set 8.3 Computation of Pareto Optimum 8.3.1 Pareto Filter 8.3.2 Reducing the Problem to the Case of a Single Objective 8.3.3 Weighted Objective Functions 8.3.4 Ideal Point in the Function Space 8.3.5 Pareto Balanced Optimum 8.3.6 Non-convex Pareto-Front 8.4 Employing Genetic Algorithms 8.5 Application 8.5.1 Nonlinear Gauss-Helmert Model 8.6 Exercise References Part III Approximation of Functions and Data 9 Approximation with Radial Bases Functions 9.1 Basic Idea of RBF Interpolation 9.2 Positive Definite RBF Function 9.3 Compactly Supported Functions 9.4 Some Positive Definite RBF Function 9.4.1 Laguerre-Gauss Function 9.4.2 Generalized Multi-quadratic RBF 9.4.3 Wendland Function 9.4.4 Buchmann-Type RBF 9.5 Generic Derivatives of RBF Functions 9.6 Least Squares Approximation with RBF 9.7 Applications 9.7.1 Image Compression 9.7.2 RBF Collocation Solution of Partial Differential Equation 9.8 Exercise 9.8.1 Nonlinear Heat Transfer References 10 Support Vector Machines (SVM) 10.1 Concept of Machine Learning 10.2 Optimal Hyperplane Classifier 10.2.1 Linear Separability 10.2.2 Computation of the Optimal Parameters 10.2.3 Dual Optimization Problem 10.3 Nonlinear Separability 10.4 Feature Spaces and Kernels 10.5 Application of the Algorithm 10.5.1 Computation Step by Step 10.5.2 Implementation of the Algorithm 10.6 Two Nonlinear Test Problems 10.6.1 Learning a Chess Board 10.6.2 Two Intertwined Spirals 10.7 Concept of SVM Regression 10.7.1 e-Insensitive Loss Function 10.7.2 Concept of the Support Vector Machine Regression (SVMR) 10.7.3 The Algorithm of the SVMR 10.8 Employing Different Kernels 10.8.1 Gaussian Kernel 10.8.2 Polynomial Kernel 10.8.3 Wavelet Kernel 10.8.4 Universal Fourier Kernel 10.9 Applications 10.9.1 Image Classification 10.9.2 Maximum Flooding Level 10.10 Exercise 10.10.1 Noise Filtration References 11 Symbolic Regression 11.1 Concept of Symbolic Regression 11.2 Problem of Kepler 11.2.1 Polynomial Regression 11.2.2 Neural Network 11.2.3 Support Vector Machine Regression 11.2.4 RBF Interpolation 11.2.5 Random Models 11.2.6 Symbolic Regression 11.3 Applications 11.3.1 Correcting Gravimetric Geoid Using GPS Ellipsoidal Heights 11.3.2 Geometric Transformation 11.4 Exercise 11.4.1 Bremerton Data References 12 Quantile Regression 12.1 Problems with the Ordinary Least Squares 12.1.1 Correlation Height and Age 12.1.2 Engel’s Problem 12.2 Concept of Quantile 12.2.1 Quantile as a Generalization of Median 12.2.2 Quantile for Probability Distributions 12.3 Linear Quantile Regression 12.3.1 Ordinary Least Square (OLS) 12.3.2 Median Regression (MR) 12.3.3 Quantile Regression (QR) 12.4 Computing Quantile Regression 12.4.1 Quantile Regression via Linear Programming 12.4.2 Boscovich’s Problem 12.4.3 Extension to Linear Combination of Nonlinear Functions 12.4.4 B-Spline Application 12.5 Applications 12.5.1 Separate Outliers in Cloud Points 12.5.2 Modelling Time-Series 12.6 Exercise 12.6.1 Regression of Implicit-Functions References 13 Robust Regression 13.1 Basic Methods in Robust Regression 13.1.1 Concept of Robust Regression 13.1.2 Maximum Likelihood Method 13.1.3 Danish Algorithm 13.1.4 Danish Algorithm with PCA 13.1.5 RANSAC Algorithm 13.2 Application Examples 13.2.1 Fitting a Sphere to Point Cloud Data 13.2.2 Fitting a Cylinder 13.3 Problem 13.3.1 Fitting a Plane to a Slope References 14 Stochastic Modeling 14.1 Basic Stochastic Processes 14.1.1 Concept of Stochastic Processes 14.1.2 Examples for Stochastic Processes 14.1.3 Features of Stochastic Processes 14.2 Time Series 14.2.1 Concept of Time