scaffold.data.iterstream.source
===============================

.. py:module:: scaffold.data.iterstream.source


Classes
-------

.. autoapisummary::

   scaffold.data.iterstream.source.IterableSamplerSource
   scaffold.data.iterstream.source.IterableSource


Module Contents
---------------

.. py:class:: IterableSamplerSource(iterables: List[Iterable], probs: Optional[List[float]] = None, rng: Optional[random.Random] = None, seed: Optional[int] = None)

   Bases: :py:obj:`scaffold.data.iterstream.base.Composable`


   A class that samples from iterables into an iterstream.

   Initialize IterableSamplerSource.

   :param iterables: List of iterables to sample from.
   :type iterables: List[Iterable]
   :param probs: [description]. Defaults to None.
   :type probs: Optional[List[float]], optional
   :param rng: Random number generator to use.
   :type rng: random.Random, optional
   :param seed: An int or other acceptable types that works for random.seed(). Will be used to seed
                `rng`. If None, a unique identifier will be used to seed.
   :type seed: Optional[int]


   .. py:method:: __iter__() -> Iterator

      Samples items from the iterables, returns all samples until all iterables are exhausted.



   .. py:attribute:: iterables


   .. py:attribute:: probs
      :value: None



   .. py:attribute:: rng
      :value: None



.. py:class:: IterableSource(source: Optional[Union[Iterable, Callable]] = ())

   Bases: :py:obj:`scaffold.data.iterstream.base.Composable`


   A class that turns an iterable to a source of a stream and provides stream manipulation functionalities on top,
   for instance:
   - map
   - map_async
   - filter
   - batched
   - shuffle
   - and more

   For the detailed description of each, please refer to the corresponding docstring in :py:class:`Composable`.

   Initialize IterableSource.

   :param source: An Iterable that the IterableSource is
                  built based on, or a callable that generates items when called.
   :type source: Union[Iterable, Callable], Optional


   .. py:method:: __iter__() -> Iterator

      Iterates over the items in the iterable