mxnet.gluon.data.batchify¶

Batchify function.

Classes

`Append`([expand, batch_axis, use_shared_mem])	Loosely return list of the input data samples.
`AsList`()	Simply forward the list of input data.
`Group`(fn, *args)	Wrap multiple batchify functions together.
`Pad`([val, dtype, round_to, use_shared_mem])	Pad the input ndarrays along the specific padding axis and stack them to get the output.
`Stack`([use_shared_mem])	Stack the input data samples to construct the batch.

class mxnet.gluon.data.batchify.Append(expand=True, batch_axis=0, use_shared_mem=False)[source]¶

Bases: object

Loosely return list of the input data samples. There is no constraint of shape for any of the input samples, however, you will only be able to apply single batch operations since the output have different shapes.

Examples

>>> a = [1, 2, 3, 4]
>>> b = [4, 5, 6]
>>> c = [8, 2]
>>> batchify.Append()([a, b, c])
[
[[1. 2. 3. 4.]]
<NDArray 1x4 @cpu_shared(0)>,
[[4. 5. 6.]]
<NDArray 1x3 @cpu_shared(0)>,
[[8. 2.]]
<NDArray 1x2 @cpu_shared(0)>
]

class mxnet.gluon.data.batchify.AsList[source]¶

Bases: object

Simply forward the list of input data. This is particularly useful when the Dataset contains textual data and in conjonction with the Group batchify function.

Examples

>>> a = ([1, 2, 3, 4], "I am using MXNet")
>>> b = ([5, 7, 2, 5], "Gluon rocks!")
>>> c = ([1, 2, 3, 4], "Batchification!")
>>> _, l = Group(Stack(), AsList())([a, b, c])
>>> l
['I am using MXNet', 'Gluon rocks!', 'Batchification!']

class mxnet.gluon.data.batchify.Group(fn, *args)[source]¶

Bases: object

Wrap multiple batchify functions together. The input functions will be applied to the corresponding input fields. Each data sample should be a list or tuple containing multiple attributes. The i`th batchify function stored in `Group will be applied on the i`th attribute. For example, each data sample is (nd_data, label). You can wrap two batchify functions using `Group(DataBatchify, LabelBatchify) to batchify nd_data and label correspondingly.

Parameters:

fn (list or tuple or callable) – The batchify functions to wrap.
*args (tuple of callable) – The additional batchify functions to wrap.

Examples

>>> a = ([1, 2, 3, 4], 0)
>>> b = ([5, 7], 1)
>>> c = ([1, 2, 3, 4, 5, 6, 7], 0)
>>> f1, f2 = Group(Pad(val=0),
...                Stack())([a, b])
>>> f1

[[1. 2. 3. 4.]
 [5. 7. 0. 0.]]
<NDArray 2x4 @cpu_shared(0)>
>>> f2

[0 1]
<NDArray 2 @cpu_shared(0)>

class mxnet.gluon.data.batchify.Pad(val=None, dtype=None, round_to=None, use_shared_mem=False)[source]¶

Bases: object

Pad the input ndarrays along the specific padding axis and stack them to get the output. Input of the function will be N samples. Each sample should contain a single element that can be 1) numpy.ndarray, 2) mxnet.nd.NDArray, 3) list of numbers. You can set the pad_val to determine the padding value.

The arrays will be padded to the largest dimensions(at most 5 dimensions to pad) and then stacked to form the final output.

Parameters:

val (float or int, default None) – The padding value.
dtype (str or numpy.dtype, default None) – The value type of the output. If it is set to None, the input data type is used.
round_to (int, default None) – If specified, the padded dimension will be rounded to be multiple of this argument.

Examples

>>> from mxnet.gluon.data import batchify
>>> # Inputs are multiple lists
>>> a = [1, 2, 3, 4]
>>> b = [4, 5, 6]
>>> c = [8, 2]
>>> batchify.Pad()([a, b, c])
[[ 1  2  3  4]
 [ 4  5  6  0]
 [ 8  2  0  0]]
<NDArray 3x4 @cpu(0)>
>>> # Also output the lengths
>>> a = [1, 2, 3, 4]
>>> b = [4, 5, 6]
>>> c = [8, 2]
>>> # Inputs are multiple ndarrays
>>> import numpy as np
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
>>> b = np.array([[5, 8], [1, 2]])
>>> batchify.Pad(val=-1)([a, b])
[[[ 1  2  3  4]
  [ 5  6  7  8]]
 [[ 5  8 -1 -1]
  [ 1  2 -1 -1]]]
<NDArray 2x2x4 @cpu(0)>
>>> # Inputs are multiple NDArrays
>>> import mxnet as mx
>>> a = nd.array([[1, 2, 3, 4], [5, 6, 7, 8]])
>>> b = nd.array([[5, 8], [1, 2]])
>>> batchify.Pad(val=-1)([a, b])
[[[ 1.  2.  3.  4.]
  [ 5.  6.  7.  8.]]
 [[ 5.  8. -1. -1.]
  [ 1.  2. -1. -1.]]]
<NDArray 2x2x4 @cpu(0)>

class mxnet.gluon.data.batchify.Stack(use_shared_mem=False)[source]¶

Bases: object

Stack the input data samples to construct the batch. The N input samples must have the same shape/length and will be stacked to construct a batch.

Examples

>>> from mxnet.gluon.data import batchify
>>> # Stack multiple lists
>>> a = [1, 2, 3, 4]
>>> b = [4, 5, 6, 8]
>>> c = [8, 9, 1, 2]
>>> batchify.Stack()([a, b, c])
[[1. 2. 3. 4.]
 [4. 5. 6. 8.]
 [8. 9. 1. 2.]]
<NDArray 3x4 @cpu(0)>
>>> # Stack multiple numpy.ndarrays
>>> import numpy as np
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
>>> b = np.array([[5, 6, 7, 8], [1, 2, 3, 4]])
>>> batchify.Stack()([a, b])
[[[1. 2. 3. 4.]
  [5. 6. 7. 8.]]
 [[5. 6. 7. 8.]
  [1. 2. 3. 4.]]]
<NDArray 2x2x4 @cpu(0)>
>>> # Stack multiple NDArrays
>>> import mxnet as mx
>>> a = nd.array([[1, 2, 3, 4], [5, 6, 7, 8]])
>>> b = nd.array([[5, 6, 7, 8], [1, 2, 3, 4]])
>>> batchify.Stack()([a, b])
[[[1. 2. 3. 4.]
  [5. 6. 7. 8.]]
 [[5. 6. 7. 8.]
  [1. 2. 3. 4.]]]
<NDArray 2x2x4 @cpu(0)>