annbatch.types.LoadRequest

typeddict annbatch.types.LoadRequest

Load request from sampler.

This is the request format Loader will expect from the sampler. Not satisfying the constrains documented here may result in unexpected behavior.

requests

Either an array of indices to load that must be the same size as the sum of the split sizes, or a list of slices with a range of chunk_size except the last one which may be smaller but not empty.

Changed in version 0.2.0: Renamed from chunks to requests.

splits

How the in-memory data should be split into batches after it is read off disk and after all the chunks are loaded and concatenated in the order requested by chunks. A list of splits, last one may be partial but not empty i.e. 1 <= len(last_split) <= batch_size. If not provided, the sampler’s batch_size property will be used to automatically generate splits.

Notes

Warning

This is a behaviour change in 0.2.0. Before 0.2.0, splits had to index into the loader’s dataset-grouped physical layout (i.e. observations ordered by dataset index, not by chunk order), so custom samplers had to account for that reordering themselves. They now index in chunk order and the loader remaps them internally. Custom samplers written for earlier versions that compensated for the dataset reordering must drop that compensation.

Required Keys:

• requests (Union[list[slice], ndarray])

• splits (list[ndarray])