spekk.spec.Spec

spekk.spec.Spec#

Bases: TreeLens

In a nested tree of arrays, a Spec describes the dimensions of the arrays. Spec is a subclass of TreeLens which takes the tree as an argument when constructing an object.

The tree of a Spec is a nested data-structure consisting of dictionaries and sequences, where the leaves are sequences of strings. An example of a Spec is as follows:

>>> spec = Spec({"foo": ["a", "b"], "bar": ["b"]})

The above spec describes a dictionary of arrays. As data, it could look something like this:

>>> import numpy as np
>>> data = {"foo": np.ones([2, 3]), "bar": np.ones([3])}

Note that the structure of the spec mirrors the structure of the data, but where each array has been replaced with a list of strings, representing the dimensions of the arrays. Note also that the second dimension of the "foo" array share the same name as the first dimension of the "bar" array, meaning that they are semantically the same dimension. This is better understood with a more concrete example:

>>> spec = Spec({"image":   ["batch", "width", "height", "channels"],
...              "caption": ["batch", "tokens"]})

In the above example, both the "image" and the "caption" has the same "batch" dimension so we know that if we loop over the batch-items we must loop over both the images and captions.

__init__(tree: Mapping[Any, Mapping[Any, Tree] | Sequence[Tree] | Any] | Sequence[Mapping[Any, Tree] | Sequence[Tree] | Any] | Any = ())#

Methods

`__init__`([tree])
`add_dimension`(dimension[, path, index])	Add the dimension to the list of dimensions at the specified path and at the specified index in the list.
`copy_with`(tree)	Return a copy of this object with the given tree.
`get`(path)	Get the value or subtree at the given path.
`has_dimension`(*dimensions)	Return True if the spec has the given dimension(s).
`has_subtree`(path)	Return True if the given path exists in the tree.
`index_for`(dimension[, path])	Return the indices of the given dimension in the spec with the same structure as the spec.
`is_leaf`([tree])	Return True if this spec object represents the dimensions of an array (i.e.: not a nested data-structure of arrays).
`prune_empty_branches`([is_leaf])	Remove all empty subtrees.
`remove_dimension`(dimension[, path])	Remove the given dimension from everywhere in the spec.
`remove_subtree`(path)	Remove the value or subtree at the given path.
`replace`(replacements)	Update the spec by replacing subtrees with corresponding subtrees in the replacements tree.
`set`(value, path)	Set the value or subtree at the given path.
`size`(data[, dimension])	Get the size of dimensions (or a single dimension) in the data.
`update_leaves`(f[, path])	See `update_leaves()`.
`update_subtree`(f, path)	Update the value or subtree at the given path.
`validate`(data)	Validate that the data conforms to the spec, raising a `ValidationError` if not.

Attributes

`at`	Interface for working on subtrees.
`dimensions`	Return all dimensions in the spec.

add_dimension(dimension: str, path: Sequence = (), index: int = 0) → Spec[source]#

Add the dimension to the list of dimensions at the specified path and at the specified index in the list.

>>> spec = Spec({"foo": {"baz": ["a", "b"]}, "bar": ["b"]})
>>> spec.add_dimension("c", ["foo", "baz"], 0)
Spec({'foo': {'baz': ['c', 'a', 'b']}, 'bar': ['b']})
>>> spec.add_dimension("c", ["foo", "baz"], 1)
Spec({'foo': {'baz': ['a', 'c', 'b']}, 'bar': ['b']})
>>> spec.add_dimension("c", ["bar"], 0)
Spec({'foo': {'baz': ['a', 'b']}, 'bar': ['c', 'b']})

property at: _TreeNavigator#

Interface for working on subtrees. This is a convenience method that provides the same functionality as set(…) and update_subtree(…), but with a (potentially) more readable syntax.

>>> tree = TreeLens({"a": {"b": [1, 2, 3]}, "d": [3]})

It can be used to set the value at the given path: >>> tree.at[“a”, “b”, 1].set(5) TreeLens({‘a’: {‘b’: [1, 5, 3]}, ‘d’: [3]})

Or update it, given a function: >>> tree.at[“a”, “b”, 1].update(lambda x: x+10) TreeLens({‘a’: {‘b’: [1, 12, 3]}, ‘d’: [3]})

copy_with(tree: Mapping[Any, Mapping[Any, Tree] | Sequence[Tree] | Any] | Sequence[Mapping[Any, Tree] | Sequence[Tree] | Any] | Any) → TSelf#: Return a copy of this object with the given tree.

property dimensions: Set[str]#

Return all dimensions in the spec.

>>> spec = Spec({"signal": ["transmits", "receivers"],
...              "receiver": {"position": ["receivers"], "direction": []},
...              "point_position": ["transmits", "points"]})
>>> sorted(spec.dimensions)
['points', 'receivers', 'transmits']

get(path: Sequence[Any]) → TSelf#

Get the value or subtree at the given path.

>>> tree = TreeLens({"a": {"b": [1, 2, 3]}, "d": [3]})
>>> tree.get(["a", "b"])
TreeLens([1, 2, 3])
>>> tree.get(["a", "b", 1])
TreeLens(2)

has_dimension(*dimensions: str) → bool[source]#

Return True if the spec has the given dimension(s).

>>> spec = Spec({"signal": ["transmits", "receivers"],
...              "receiver": {"position": ["receivers"], "direction": []}})
>>> spec.has_dimension("transmits", "receivers")
True
>>> spec.has_dimension("frames", "transmits", "receivers")
False

has_subtree(path: Sequence[Any]) → bool#

Return True if the given path exists in the tree.

>>> tree = TreeLens({"a": {"b": [1, 2, 3]}})
>>> tree.has_subtree(["a", "b", 1])
True
>>> tree.has_subtree(["a", "c"])
False

Return the indices of the given dimension in the spec with the same structure as the spec.

>>> spec = Spec({"signal": ["transmits", "receivers"],
...              "receiver": {"position": ["receivers"], "direction": []}})
>>> spec.index_for("receivers")
{'signal': 1, 'receiver': {'position': 0, 'direction': None}}

Return True if this spec object represents the dimensions of an array (i.e.: not a nested data-structure of arrays).

May optionally be called as a static method where self is a tree, or with an explicitly given tree.

See also

func:._is_spec_leaf).

Remove all empty subtrees.

May be called as a static method where self is a Tree.

remove_dimension(dimension: str | Sequence[str], path: Sequence = ()) → Spec[source]#

Remove the given dimension from everywhere in the spec.

>>> spec = Spec({"signal": ["transmits", "receivers"],
...              "receiver": {"position": ["receivers"], "direction": []}})
>>> spec.remove_dimension("receivers")
Spec({'signal': ['transmits'], 'receiver': {'position': [], 'direction': []}})

You can also remove multiple dimensions at once:

>>> spec.remove_dimension(["transmits", "receivers"])
Spec({'signal': [], 'receiver': {'position': [], 'direction': []}})

remove_subtree(path: Sequence[Any]) → TSelf#

Remove the value or subtree at the given path.

>>> tree = TreeLens({"a": {"b": [1, 2, 3]}, "d": [3]})
>>> tree.remove_subtree(["a", "b", 1])
TreeLens({'a': {'b': [1, 3]}, 'd': [3]})

Update the spec by replacing subtrees with corresponding subtrees in the replacements tree.

A value of None in the replacements tree removes the subtree at the corresponding path.
A leaf (list of dimensions, see Spec.is_leaf) in the replacements tree always replaces the leaf (or subtree) at the corresponding path.
Keys present in the replacements tree but not in the spec are added to the spec at the corresponding path.

>>> spec = Spec({"foo": {"baz": ["a", "b"]}, "bar": ["b"]})

Replacing a path with None removes the subtree at that path:

>>> spec.replace({"foo": None})
Spec({'bar': ['b']})

Removing a subtree such that its parent becomes an empty collection also removes the parent:

>>> spec.replace({"foo": {"baz": None}})
Spec({'bar': ['b']})

Replacing an existing path with a list of dimensions overwrites the path:

>>> spec.replace({"foo": ["c"]})
Spec({'foo': ['c'], 'bar': ['b']})

Other than that, it is assumed that the replacements tree structure mirrors the spec structure:

>>> spec.replace({"foo": {"baz": ["c"]}})
Spec({'foo': {'baz': ['c']}, 'bar': ['b']})

set(value: Any, path: Sequence[Any]) → TSelf#

Set the value or subtree at the given path.

>>> tree = TreeLens({"a": {"b": [1, 2, 3]}, "d": [3]})
>>> tree.set(5, ["a", "b", 1])
TreeLens({'a': {'b': [1, 5, 3]}, 'd': [3]})

Get the size of dimensions (or a single dimension) in the data.

>>> import numpy as np
>>> spec = Spec({"signal": ["transmits", "receivers"],
...              "receiver": {"position": ["receivers"], "direction": []}})
>>> data = {"signal": np.random.randn(10, 20),
...         "receiver": {"position": np.random.randn(20, 3),
...                      "direction": np.random.randn(20, 3)}}
>>> spec.size(data) == {'transmits': 10, 'receivers': 20}
True
>>> spec.size(data, "transmits")
10
>>> spec.size(data, "receivers")
20

update_leaves(f: Callable, path: Sequence[Any] = ()) → TSelf#: See update_leaves().

update_subtree(f: Callable, path: Sequence[Any]) → TSelf#

Update the value or subtree at the given path.

>>> tree = TreeLens({"a":{"b": [1, 2, 3]}, "d":[3]})
>>> tree.update_subtree(lambda x: x + 10, ["a", "b", 1])
TreeLens({'a': {'b': [1, 12, 3]}, 'd': [3]})

validate(data: Mapping[Any, Mapping[Any, Tree] | Sequence[Tree] | Any] | Sequence[Mapping[Any, Tree] | Sequence[Tree] | Any] | Any)[source]#: Validate that the data conforms to the spec, raising a ValidationError if not.

See also

validate()

spekk.spec.Spec

Contents

spekk.spec.Spec#