Linear Curve Fitting
In the context of curve fitting, a linear curve is a curve that has a linear dependence on the curve parameters. Examples of linear curves are: lines, polynomials, Chebyshev series, and any linear combination of a set of curves. The linearity greatly simplifies the calculations, as the solution can be expressed in terms of simple linear algebra.
Mathematical background
Most curve fitting is done using linear combinations of a set of basis functions. A linear combination of a set of functions, f_{1}, f_{2}..., f_{n} is a function whose value is given by
f(x) = a_{1}f_{1}(x) + a_{2}f_{2}(x) + ... + a_{n}f_{n}(x)
where a_{1}, a_{2}..., a_{n} are the coefficients. The advantage of using this type of approximation is that the math is relatively simple. The solution can be found by solving a standard linear algebra problem. In many cases, this can be done very efficiently.
The basis functions f_{1}, f_{2}..., f_{n} that are combined to create the approximation together form a function basis. A linear combination is therefore defined by the function basis and the coefficients of the basis functions in the combination.
Perhaps the most well-known example of a linear combination of basis functions is provided by polynomials. A polynomial is a function that is a combination of the argument raised to different integer powers:
f(x) = a_{0} + a_{1}x + ... + a_{n}x^{n}
The integer powers are the basis functions. Chebyshev polynomials form an alternate basis for the polynomials. They can be defined mathematically by the identity
T_{n} ( cos x) = cos nx
Their mathematical properties make them particularly suited for approximating functions.
In Extreme Numerics.NET, function bases are implemented by the abstract FunctionBasis class and its derived classes. Linear combinations are implemented using the LinearCombination class and its derived classes, including Polynomial and ChebyshevSeries. The table below summarizes the classes that implement function bases and the corresponding linear combinations.
Basis | Linear combination | Basis functions |
---|---|---|
Any set of functions defined as Func<T, TResult> delegates. | ||
Positive integer powers. | ||
Chebyshev polynomials over an interval. |
Static curve fitting methods
Many classes expose static methods that allow you to obtain a least squares fit. The main drawback of the methods discussed in this section is that they don't offer much control over the calculation of the least squares fit. There is also no easy way to obtain information about the quality of the fit. However, they may be useful because they offer a very quick way to get a result.
The LeastSquaresFit method of the Polynomial class constructs a Polynomial that is the least squares fit of a specified degree through a given set of points.
Vector<double> xValues = Vector.Create(7, i => Math.Cos(i * Constants.Pi / 6));
Vector<double> yValues = Vector.Create(7, i => -Math.Sin(i * Constants.Pi / 6));
Polynomial p = Polynomial.LeastSquaresFit(xValues, yValues, 4);
To fit a line, set the degree equal to 1.
You can also create a function basis, and obtain the least squares fit using the functions in the function basis for a set of data points using the LeastSquaresFit method. It returns a LinearCombination curve. This allows you to fit data to an arbitrary set of functions. You can also specify weights for each of the data points. The weights specify how much weight each data point should have in the sum of squares that is to be minimized.
The following example fits a set of data points to a combination of the sine, cosine, and exponential functions.
Vector<double> xValues = Vector.Create(0.0, 1.0, 2.0, 3.0, 4.0);
Vector<double> yValues = Vector.Create(4.0, 1.0, 4.0, 10.0, 8.0);
GeneralFunctionBasis basis = new GeneralFunctionBasis(Math.Sin, Math.Cos, Math.Exp);
LinearCombination f = basis.LeastSquaresFit(xValues, yValues);
The LinearCurveFitter class
The LinearCurveFitter class performs a linear least squares fit. It offers greater control over the procedure, and gives more extensive results.
To perform the fit, a LinearCurveFitter needs data points, and a curve to fit. You must set the Curve property to an instance of a LinearCombination object. A LinearCombination object can represent any combination of functions. Other objects, for example of type Polynomial or ChebyshevSeries are allowed, as they inherit from LinearCombination.
The data is supplied as Vector<T> objects. The XValues and YValues properties specify the X and Y values of the data points, respectively.
By default, the fit is unweighted. All weights are set equal to 1. Weights can be specified in one of two ways. The first way is to set the WeightVector property to a vector with as many elements as there are data points. Each component of the vector specifies the weight of the corresponding data point.
Alternatively, the WeightFunction can be used to calculate the weights based on the data points. The WeightFunctions class contains a number of predefined weight functions listed in the table below. In the descriptions, x and y stand for the x and y values of each data point.
Field | Description |
---|---|
OneOverX | The weight equals 1/x. |
OneOverXSquared | The weight equals 1/x^{2}. |
OneOverY | The weight equals 1/y. |
OneOverYSquared | The weight equals 1/y^{2}. |
The Fit method performs the actual least squares calculation, and adjusts the parameters of the Curve to correspond to the least squares fit. The BestFitParameters property returns a Vector<T> containing the parameters of the curve. The GetStandardDeviations returns a Vector<T> containing the standard deviations of each parameter as estimated in the least squares calculation.
Example 1: Fitting a Polynomial
In the first example, we fit some data to a quadratic polynomial.
Vector<double> loadData = Vector.Create<double>(/*...*/);
Vector<double> deflectionData = Vector.Create<double>(/*...*/);
Polynomial poly = new Polynomial(2);
LinearCurveFitter fitter = new LinearCurveFitter();
fitter.Curve = poly;
fitter.XValues = loadData;
fitter.YValues = deflectionData;
fitter.Fit();
First, the data is loaded into the loadData and deflectionData vectors. We then create the polynomial, poly. That gives us what we need to create the LinearCurveFitter objects. We set its properties, and perform the fit.
Example 2: Fitting arbitrary functions
In the next example, we fit measurements of the conductance of copper to a theoretical model:
We have measurements of k(T) for a series of temperatures. To estimate the parameters c_{1} and c_{2}, we need to take the following steps:
- Transform the dependent variable k(T) into y = 1/k(T). The expression for y is then linear in the basis functions 1/T and T^{2}.
- Create a function basis with these two basis functions.
- Create a LinearCombination curve using this function basis.
- Create the LinearCurveFitter object and set its properties.
- Perform the fit and report the results.
The code below performs these steps. It assumes we have created two vector variables, temperature and conductance, that contain the data of the observations.
Vector<double> y = Vector.Reciprocal(conductance);
Func<double,double>[] basisFunctions = new Func<double,double>[]
{ x => 1/x, x => x * x };
GeneralFunctionBasis basis = new GeneralFunctionBasis(basisFunctions);
LinearCombination curve = new LinearCombination(basis);
LinearCurveFitter fitter = new LinearCurveFitter();
fitter.Curve = curve;
fitter.XValues = temperature;
fitter.YValues = y;
fitter.Fit();
Vector<double> solution = fitter.BestFitParameters;
Vector<double> s = fitter.GetStandardDeviations();
Console.WriteLine("c1: {0,20:E10} {1,20:E10}", solution[0], s[0]);
Console.WriteLine("c2: {0,20:E10} {1,20:E10}", solution[1], s[1]);