What's New in Version 9.0

We’re thrilled to announce the release of version 9 of Numerics.NET.

A new name

Extreme Optimization is now Numerics.NET, affirming our position as the leading numerical library for .NET.

Version 9 completes our migration: Both the NuGet packages and the namespaces have been changed to reflect this.

This will require some changes in your projects, but migration should be pretty straightforward:

  1. Update the package references to Numerics.NET (see below).
  2. Rename all Extreme.Mathematics namespace references to Numerics.NET
  3. Rename all remaining Extreme namespace references to Numerics.NET

If you encounter any challenges, we’re always here to help.

NuGet packages

The packages are available on the NuGet Gallery.

All NuGet packages now start with the Numerics.NET prefix. We’ve also reorganized them in a more convenient way. The functionality is spread over several core packages, while a number of bundles offer a simple way to get everything you need.

The main bundle packages are:

  • Numerics.NET: bundles all managed packages. No native libraries are included.
  • Numerics.NET.win-x64: everything in Numerics.NET plus processor-optimized native libraries for Windows (x64).
  • Numerics.NET.linux-x64: everything in Numerics.NET plus processor-optimized native libraries for Linux (x64).

Of course you stil have the option of installing the individual packages if that is what you prefer.

Linear algebra

Tensors

A tensor is a generalization of vectors and matrices to arbitrary dimensions. With the growing popularity in areas such as machine learning and AI, the time has come to add support for tensors to Numerics.NET.

There is so much new here that we’ve written an entire post just on the topic of tensors in Numerics.NET: In Depth: Tensors.

Microsoft is introducing a new tensor type in .NET 9.0, System.Numerics.Tensors.Tensor<T>. Why should you use our tensor type instead?

Microsoft has stated that its tensor type “is not a replacement for existing AI and machine learning libraries. Instead, it’s intended to provide a common set of APIs to reduce code duplication and dependencies, and to achieve better performance by using the latest runtime features.”

In other words: the type is purposely limited in scope. Many of the features in Numerics.NET’s tensor type are not available in the Microsoft tensor type and will likely never be. For example

Some of the features in Numerics.NET’s tensor type are not available in other tensor libraries include:

  • Setting the destination of an operation, like Tensor.Add(left, right, result). This avoids unnecessary allocations and can dramatically improve performance.
  • Memory layouts other than contiguous C-style (last index varies fastest). Much of the power and expressiveness of tensor libraries derives from the ability to rearrange the logical layout of the data while leaving the memory layout unchanged.
  • Scalar tensors. A scalar tensor is a tensor with zero dimensions. This is useful in many algorithms, especially when working with tensors of different ranks.
  • Aggregation along a specific dimension.
  • Conditional evaluation. This is useful when you want to apply an operation only to certain elements of a tensor.
  • Support for GPU and other platforms. Our tensor type is designed to be extensible so that, when provided with a suitable implementation, the same code can work on any platform, including GPU.

Memory<T> based vector and matrix storage

In previous versions, vectors and matrices always used managed arrays to store their. This incurred a penalty when working with some native code because it was necessary to copy the data between managed and unmanaged memory.

In version 9, vectors and matrices now support using Memory<T> to store the elements. This means you can create vectors and matrices directly from native memory. No more copying needed!

Spanified core API

The core implementations of linear algebra methods have been refitted to use Span<T> instead of arrays. This was a major undertaking that enables better performance and more flexibility in the future. For example, it is what allows us to use Memory<T> for element storage.

The one drawback is that the new API is not compatible with C++/CLI, which doesn’t support ref structs like Span<T>. So, there are no more mixed-mode native assemblies.

Schur and QZ decompositions

Two new matrix decompositions make their debut in version 9. The Schur decomposition writes a square matrix A as

\[A = ZTZ^*\]

where Z is orthogonal (unitary) and T, the Schur form of A, is upper triangular. The diagonal elements of T are also the eigenvalues of A. For real matrices with complex eigenvalues, the real Schur form of A is returned. This is an upper quasi-triangular matrix where 2x2 blocks on the diagonal correspond to pairs of complex conjugate eigenvalues.

Similar to eigensystems, there is also a generalization of the Schur decomposition to two matrices. This generalized Schur decomposition, also called the QZ decomposition, rewrites two square matrices A and B as

\[A = QSZ^*, B = QTZ^*\]

where again S and T are upper triangular, and Q and Z are orthogonal (unitary).

Mathematics

Generic arithmetic

Numerics.NET has had facilities for the generic implementation of numerical algorithms for over a decade. Our linear algebra library has supported arbitrary numerical types since 2008!

We continue to build on this foundation. The new tensor library is of course generic over the element type, but there’s more.

Our generic arithmetic implementation is built on a static class, Operations<T>, that provides all common operations on numbers. Thanks to JIT optimizations, performance is on par with hard-coded types.

The Operations<T> class now fully supports complex numbers and nullable numeric types. We’ve also brought this class up to speed with all the latest updates to the System.Math class and numeric types in recent .NET versions.

All our special number types (Quad, BigInteger, BigRational, BigFloat) implement the INumber<T> interface introduced in .NET 7.0. We’re not really happy with Microsoft’s work here. While the introduction of these interfaces is clearly a step forward in some ways, the way this was designed has some serious problems.

For .NET 7.0+ only, we’ve added generic implementations of some of the most common algorithms. The class names are the same but there is an extra type argument, which must implement INumber<T>. To find the classes, add .Generic to the namespace of the original (non-generic) class.

The following algorithms have a generic implementation:

  • Numerical integration.
  • Polynomial curve fitting, including using Chebyshev polynomials.
  • Solving equations in one variable.

Random number generators

We’ve added a new random number generator, PCG32, that is fast and compact. It is the new default random number generator.

Some new methods were recently added to .NET’s built-in random number generator. This includes GetItems, NextInt64 and NextSingle.

We’ve added implementations of these methods to our extended random number generator class so they are available to earlier .NET versions as well.

We’ve even added a few more! The GetItems method (introduced in .NET 8.0) returns a set of items chosen at random from a set of choices with replacement. This means that the same value can appear multiple times in the output. Many applications require items that are chosen without replacement. This means that every choice can appear at most once in the output. The GetItemsWithoutReplacement method does just that.

Special functions

So-called special functions are a nearly limitless group of funcions that appear in science. Some are better known and more widely used than others.

In version 9, we’ve expanded our support for special functions considerably. We’ve added:

  • Airy functions for complex argument.
  • Bessel functions for complex argument.
  • Kelvin functions.
  • Monic Hermite polynomials.
  • Jacobi polynomials.
  • Bernoulli numbers and Bernoulli polynomials.
  • Hurwitz zeta function and generalized harmonic numbers.
  • Jacobi Epsilon and Zeta functions.

Other improvements

  • New optimized methods for computing norms of band matrices.
  • An improved Quadratic Programming solver based on the algorithm of Goldfarb and Idnani.
  • Performance improvements, including in 2D integration, BigFloat functions.
  • Take advantage of enhancements and new functionality in .NET whenever possible.

Data Analysis

Aggregators

The API around aggregators (like Sum, Variance) has been overhauled to be more consistent and much more powerful.

Previously, there were two types of aggregators: normal aggregators always returned a value of a specific type. For example, computing the mean of an array always returned a double.

Aggregators now have an As<T> method that lets you change that. The input elements are converted to the result type. The converted elements are then used to compute the aggregate.

This is a breaking change. If you have code that depends on the type of the result of an aggregator being fixed, you will need to update it. For example, instead of Aggregators.Mean, you now use Aggregators.Mean.As<double>().

All aggregators support skipping missing values in the input. More generally, you can apply any condition to the input values and only those values that meet the condition will be included in the aggregation.

Accumulators

Internally, many of the aggregators use accumulators. These are simple structures that incrementally accumulate the result of the aggregation.

These accumulators have now been made public in the Numerics.NET.DataAnalysis.Accumulators namespace.

One accumulator can be the basis for several different aggregators. For example, the VarianceAccumulator<T> can return the count, the mean, as well as both the population and sample variance and standard deviation.

Statistics

New probability distributions

We’ve added a number of probability distributions. The following contiunous distributions are new in version 9:

Also new is the log-series distribution, a discrete distribution with applications in biology and finance.

Improvements in statistical models

No major new features here.

  • When performing stepwise regression, you can now get detailed information about each step.
  • New diagnostics for linear regression: Predicted residual error sum of squares (PRESS) and Predicted R2.

Release Notes

You can find details about the latest updates since the release of version 9 in the release notes.