Accessing Tensor Elements

Indexing tensors is broadly similar to indexing arrays. Some of the differences are:

• Tensor indexers always return a tensor, even if the result is a single value. To get single values, use the GetValue method.

• In addition to integers and ranges, tensor indices may be boolean arrays, sets of integers, or certain special values.

• The number of indices does not have to be equal to the number of dimensions. When some indices are omitted, the remaining dimensions are assumed to be full.

• Whenever possible, tensor indexers return a view of the data. The main reason is performance: copying elements is often unnecessary.

All these points are expanded on below.

The Tensor<T> class supports standard indexer properties, both for getting parts of a tensor and assigning to parts of a tensor. In addition, the GetValue and SetValue allow getting and setting of individual elements.

There are six different types of tensor indices. Tensors can have 2, 3, 4 or more dimensions. To avoid a combinatorial explosion of overloads, indexing is performed using a TensorIndex structure. There is an implicit conversion to TensorIndex for every value that may serve as a tensor index.

Integers

An integer index selects a single value within a dimension. The presence of an integer index reduces the rank of the resulting tensor by one.

Passing all integer indices results in a scalar (rank 0) tensor.

System.Index

The Index type was introduced in C# 8.0. Indexes work exactly like integers, but allow for counting from the end of a dimension. As with integers, the rank of the resulting tensor is reduced by one.

System.Range

The Range type was also introduced in C# 8.0. The resulting tensor contains only the values along the dimension that fall within the specified range.

A range index does not change the rank of the resulting tensor, regardless of how many elements are in the range. If the range is empty, then the resulting tensor will be empty as well. If the range contains only one value, then the resulting tensor will have length 1 along the dimension.

All variations of ranges may be used: the start or the end of a range may be omitted. Completely open ranges are allowed as well.

Numerics.NET Ranges and Slices

The Range and Slice types in Numerics.NET were introduced before the introduction of range and index operators and the corresponding Range and Index types in C# 8.0. Their use is similar to the ranges discussed above. In addition, they allow for specifying the stride of the range.

Advanced Indices

Occasionally, it can be useful to have a more complex value as an index. The AdvancedTensorIndex structure enables the use of lists of integer and booleans as a tensor indices.

As with TensorIndex, implicit conversions do the heavy lifting.

When the index is a sequence of integers, the resulting tensor contains the entries whose index in the specified dimension is listed in the sequence in the order in which they appear in the sequence.

The sequence can be an array, an IEnumerable<T>, a Vector<T> or even a Tensor<T> with integer elements. It must have the same number of elements as the length of the dimension.

When the index is a sequence of booleans, the resulting tensor contains only those entries whose value at the corresponding position in the index is true. The length in the dimension is the total number of true values in the index sequence.

Indexers that include an advanced index always return a copy of the data. If the same index occurs two or more times, the same values are copied each time.

When the number of indices is equal to the rank of a tensor and all the indices are either integers or Index values, then the result is a scalar tensor.

var t = Tensor.CreateFromFunction((3, 4, 5), (i, j, k) => 100 * i + 10 * j + k);
var t123 = t[1, 2, 3];
// t123 -> [ 123 ]

Dim t = Tensor.CreateFromFunction((3, 4, 5), Function (i, j, k) 100 * i + 10 * j + k)
Dim t123 = t(1, 2, 3)
' t123 -> [ 123 ]

No code example is currently available or this language may not be supported.

let t = Tensor.CreateFromFunction(shape3 3 4 5, fun i j k -> 100 * i + 10 * j + k)
let t123 = t[1, 2, 3]
// t123 -> [ 123 ]

Starting with C# 8.0 (.NET Core 3.0/.NET Standard 2.1), it is possible to use Index values as indices, where indices are counted from the end. The following example selects the same element, counting all dimensions from the end:

var t_2_2_2 = t[^2, ^2, ^2];
// t_2_2_2 -> [ 123 ]

It is possible to assign to single elements, but the right-hand side must then be a scalar tensor:

t[1, 2, 3] = Tensor.CreateScalar(999);
t123 = t[1, 2, 3];
// t123 -> [ 999 ]

t(1, 2, 3) = Tensor.CreateScalar(999)
t123 = t(1, 2, 3)
' t123 -> [ 999 ]

No code example is currently available or this language may not be supported.

t[1, 2, 3] <- Tensor.CreateScalar(999)
let t123 = t[1, 2, 3]
// t123 -> [ 999 ]

In order to access a single element as a scalar value (not a scalar tensor), use the GetValue method. The number of indices must match the rank of the tensor. To set individual elements, use the SetValue method. The first argument to this method is the value that is to be assigned. The remaining arguments are the indices of the element to be set.

var tValue = t.GetValue(1, 2, 3);
// tValue -> [ 999 ]
t.SetValue(99, 1, 2, 3);
tValue = t.GetValue(1, 2, 3);
// tValue -> [ 99 ]

Dim tValue = t.GetValue(1, 2, 3)
' tValue -> [ 999 ]
t.SetValue(99, 1, 2, 3)
tValue = t.GetValue(1, 2, 3)
' tValue -> [ 99 ]

No code example is currently available or this language may not be supported.

let tValue = t.GetValue(1, 2, 3);
// tValue -> [ 999 ]
t.SetValue(99, 1, 2, 3);
let tValue = t.GetValue(1, 2, 3);
// tValue -> [ 99 ]

Ranges and slices are indices that select a range of values along a dimension. Range and Slice values can be used to select a range of values along the corresponding dimension.

var r12 = new Numerics.NET.Range(1, 2);
var trrr = t[r12, r12, r12];
// trrr -> [[[ 111, 112], [121, 122]], [211, 212], [221, 222]]]
trrr = t[1..3, 1..3, 1..3];

Dim r12 = New Range(1, 2)
Dim trrr = t(r12, r12, r12)
' trrr -> [[[ 111, 112], [121, 122]], [211, 212], [221, 222]]]

No code example is currently available or this language may not be supported.

let trrr = t[1..2, 1..2, 1..2];
// trrr -> [[[ 111, 112], [121, 122]], [211, 212], [221, 222]]]

As can be seen above, range expressions (introduced in C# 8.0) can be used as well. Note that Numerics.NET ranges are inclusive, meaning that the upper limit of the range is included in the range, while C# range expressions are end-exclusive.

Keep in mind that indexers return views of the data, so changes to the view will affect the original tensor. All kinds of combinations of indices are possible when getting parts of tensors:

var s = Tensor.CreateFromFunction((3, 3), (i, j) => 11 + 10 * i + j);
// s -> [[ 11 12 13 ]
//       [ 21 22 23 ]
//       [ 31 32 33 ]]
var row1 = s[0, ..];
// row1 -> [ 11 12 13 ]
var column1 = s[.., ^2];
// column1 -> [ 12 22 32 ]
var row2 = s[1, 1..];
// row2 -> [ 22 23 ]

Dim m = Tensor.CreateFromFunction((3, 3), Function (i, j) 11 + 10 * i + j)
' m -> [[ 11 12 13 ]
'       [ 21 22 23 ]
'       [ 31 32 33 ]]
Dim row1 = m(0, Range.All)
' row1 -> [ 11 12 13 ]
Dim column1 = m(Range.All, 1)
' column1 -> [ 12 22 32 ]
Dim row2 = m(1, New Range(1,2))
' row2 -> [ 22 23 ]

No code example is currently available or this language may not be supported.

let m = Tensor.CreateFromFunction(TensorShape(3, 3), fun i j -> 11 + 10 * i + j)
// m -> [[ 11 12 13 ]
//       [ 21 22 23 ]
//       [ 31 32 33 ]]
let row1 = m[0, *]
// row1 -> [ 11 12 13 ]
let column1 = m[*, 1]
// column1 -> [ 12 22 32 ]
let row2 = m[1, 1..3]
// row2 -> [ 22 23 ]

The same is true when setting parts of tensors:

s[1, 1..3] = Tensor.CreateFromArray([ 88, 99 ]);
// s -> [[ 11 12 13 ]
//       [ 21 88 99 ]
//       [ 31 32 33 ]]
s[..^1, ^2] = Tensor.CreateFromArray([ 1, 2 ]);
// s -> [[ 11  1 13 ]
//       [ 21  2 99 ]
//       [ 31 32 33 ]]

m(1, New Range(1,3)) = Tensor.CreateFromArray({ 88, 99 })
' m -> [[ 11 12 13 ]
'       [ 21 88 99 ]
'       [ 31 32 33 ]]
m(Range.All, 1) = Tensor.CreateFromArray({ 1, 2, 3 })
' m -> [[ 11 1 13 ]
'       [ 21 2 99 ]
'       [ 31 3 33 ]]

No code example is currently available or this language may not be supported.

m[1, 1..] <- Tensor.CreateFromArray([| 88; 99 |])
// m -> [[ 11 12 13 ]
//       [ 21 88 99 ]
//       [ 31 32 33 ]]
m[*, 1] <- Tensor.Create([| 1; 2; 3 |])
// m -> [[ 11 1 13 ]
//       [ 21 2 99 ]
//       [ 31 3 33 ]]

The values selected by a range are most often contiguous, but this is not a requirement. The distance between consecutive elements is called the stride. So most ranges have a stride equal to 1.

C# ranges don't support setting the stride, so you need to use either the Range or Slice type to use ranges with a stride different from 1:

var row3 = s[1, new Range(0, 2, 2)];
// row3 -> [ 21 23 ]
row3 = s[1, new Slice(2, 0, 2)];
// row3 -> [ 21 23 ]

Dim row3 = m(1, New Range(0, 2, 2))
' row3 -> [ 21 23 ]

No code example is currently available or this language may not be supported.

let i1 = TensorIndex.op_Implicit(1);
let i2 = TensorIndex.op_Implicit(Range(0, 2, 2));
let row3 = m[i1, i2]
// row3 -> [ 21 23 ]

Strides can even be negative:

var x = Tensor.CreateRange(3);
// x -> [ 0 1 2 ]
var reverse = x[new Range(2, 0, -1)];
// reverse -> [ 2 1 0 ]
reverse = x[new Slice(2, 2, -1)];
// reverse -> [ 2 1 0 ]

Dim x = Tensor.CreateRange(3)
' x -> [ 0 1 2 ]
Dim reverse = x(New Range(2, 0, -1))
' reverse -> [ 2 1 0 ]
reverse = x(New Slice(2, 2, -1))
' reverse -> [ 2 1 0 ]

No code example is currently available or this language may not be supported.

let x = Tensor.CreateRange(3)
// x -> [ 0 1 2 ]
let iRange = TensorIndex.op_Implicit(Range(2, 0, -1))
let reverse = x[iRange]
// reverse -> [ 2 1 0 ]
let iSlice = TensorIndex.op_Implicit(Slice(2, 2, -1))
let reverse = x[iSlice]
// reverse -> [ 2 1 0 ]

Ranges and Slices can represent the same set of indices but they specify them differently. A range specifies the start, the end, and the stride of the range, while a slice specifies the number of elements, the start, and stride.

Setting a range with a stride different from 1 is also possible:

s[1, new Range(0, 2, 2)] = Tensor.CreateFromArray([77, 66]);
// s -> [[ 11  1 13 ]
//       [ 77  2 66 ]
//       [ 31 32 33 ]]

m(1, New Range(0, 2, 2)) = Tensor.CreateFromArray({ 77, 66 })
' m -> [[ 11  1 13 ]
'       [ 77 88 66 ]
'       [ 31  2 33 ]]

No code example is currently available or this language may not be supported.

let i1 = TensorIndex.op_Implicit(1);
let i2 = TensorIndex.op_Implicit(Range(0, 2, 2));
m[i1, i2] <- Tensor.Create([| 77; 66 |])
// m -> [[ 11 12 13 ]
//       [ 77 88 66 ]
//       [ 31 32 33 ]]

When there are fewer indexers than dimensions, the same rule applies that the remaining dimensions are assumed to be full.

Sometimes, ranges and slices are not enough to select the desired elements. For example, you may want to select elements that satisfy a certain condition. For this purpose, advanced indexing using the AdvancedTensorIndex type is available.

Advanced indexing always makes a copy of the values selected.

There are two basic forms of advanced indexing: integer and boolean. An integer advanced index selects elements by their index in the dimension. The index is a list of integers that specify the index of the elements to select. The size of the dimension in the resulting tensor is the same as the number of indices. The order of the elements is determined by the order of the indices in the index. When an index is repeated, the same elements are copied multiple times.

var t = Tensor.CreateFromFunction((3, 4, 5), (i, j, k) => 100 * i + 10 * j + k);
int[] indexes = [ 0, 3 ];
var t1 = t[1, indexes, 3..5];
// t1 -> [[ 103 133 ]
//        [ 104 134 ]]

Dim t = Tensor.CreateFromFunction((3, 4, 5), Function (i, j, k) 100 * i + 10 * j + k)
Dim indexes = { 0, 3 }
Dim t2 = t(1, indexes, New Range(3, 4))
' t1 -> [[ 103 133 ]
'        [ 104 134 ]]

No code example is currently available or this language may not be supported.

let t = Tensor.CreateFromFunction(TensorShape(3, 4, 5), fun i j k -> 100 * i + 10 * j + k)
let i1 = AdvancedTensorIndex.op_Implicit(1)
let i3 = AdvancedTensorIndex.op_Implicit(Range(3, 4))
let indexes = [| 0; 3 |]
let i2 = AdvancedTensorIndex.op_Implicit(indexes)
let t1 = t[i1, i2, i3]
// t1 -> [[ 103 133 ]
//        [ 104 134 ]]

A boolean advanced index selects elements by a boolean value. The index is a list of booleans of the same size as the dimension being indexed. An element is selected if the corresponding element in the index is true. The size of the dimension in the resulting tensor is the same as the number of indices that are true in the list . The order of the elements is the same as the order of the original tensor.

bool[] mask = [ true, false, false, true ];
var t2 = t[1, mask, 3..5];
// t2 -> [[ 103 133 ]
//        [ 104 134 ]]

Dim mask = { true, false, false, true }
Dim t1 = t(1, mask, New Range(3, 4))
' t1 -> [[ 103 133 ]
'        [ 104 134 ]]

No code example is currently available or this language may not be supported.

let mask = [| true; false; false; true |]
let i2 = AdvancedTensorIndex.op_Implicit(mask)
let t2 = t[i1, i2, i3]
// t1 -> [[ 103 133 ]
//        [ 104 134 ]]

Assigning using advanced indexing works the same way as getting. The size of the dimension corresponding to the index in the right-hand side must match the number of elements selected by the advanced index.

As mentioned earlier, indexing relies heavily on implicit conversions to avoid a combinatorial explosion of overloads to support every possible combination of indices. F# has very limited support for implicit conversions. Without additional help, even the simplest indexing expressions will not compile and F# users would have to resort to calling the implicit conversion methods explicitly.

The compatibility assembly Numerics.NET.FSharp defines type augmentations for tensors that support the most common indexing operations with integers and F# range operators in up to 3 dimensions. F# ranges are end-inclusive. Reverse indexing using the ^ operator is also supported.