NumpyFilter

class tfdatacompose.filter.numpyfilter.NumpyFilter(stateful: bool = False)

Base class for filtering operations that require numpy arrays.

Wraps the Tensorflow Filter operation on the dataset where the filter function is wrapped in numpy_function. The filtering operation should be implemented in the filter method. This operation is useful if your filtering can not be implemented in with Tensorflow operations, for instance, if it needs to call an external library.

Parameters:

stateful – Whether the operation is stateless. Tensorflow can enable some optimizations on stateless functions to improve performance.

apply(dataset: DatasetV1) → DatasetV1

Applies the operation on the given dataset and returns the new dataset.

Parameters:

dataset – Tensorflow Dataset The dataset to change

Returns:

Tensorflow Dataset The changed dataset

abstract filter(*args: ndarray) → bool

Implement your filter in this method. The method receives a dataset element as input transformed into a numpy ndarray and should return a boolean. The implementation of the filter can be arbitrary Python code.

Parameters:

args – the dataset element as a numpy ndarray

Returns:

True to keep an element and False to remove it