NumpyMap

class tfdatacompose.map.numpymap.NumpyMap(out_type: DType | Tuple[DType, ...], stateful: bool = False)

Base class for mapping operations that require numpy arrays.

Wraps the Tensorflow Map operation on the dataset where the filter function is wrapped in numpy_function. The return type of the operation must be specified in advance for Tensorflow to build the computation graph. Transformations should be implemented in the map method. This operation is useful if your mapping can not be implemented in with Tensorflow operations, for instance, if it needs to call an external library.

Parameters:

• out_type – The return type of the operation

• stateful – Whether the operation is stateless. Tensorflow can enable some optimizations on stateless functions to improve performance.

apply(dataset: DatasetV1) → DatasetV1

Applies the operation on the given dataset and returns the new dataset.

Parameters:

dataset – Tensorflow Dataset The dataset to change

Returns:

Tensorflow Dataset The changed dataset

abstract map(*args: Tensor) → Tuple[Tensor]

Implement your transformation in this method. The method receives a dataset element as input transformed into a numpy ndarray and should return a list of numpy ndarray . The implementation of the transformation can be arbitrary Python code.

Parameters:

args – the dataset element as a numpy ndarray

Returns:

the transformed element as a list of numpy ndarray