scaffold.system_monitor
=======================

.. py:module:: scaffold.system_monitor

.. autoapi-nested-parse::

   This file is strongly informed by the design of WandB's client published under MIT license

   A good overview and explanation of the different logged quantities can be found
   at: https://lambdalabs.com/blog/weights-and-bias-gpu-cpu-utilization/



Attributes
----------

.. autoapisummary::

   scaffold.system_monitor.GPUHandle
   scaffold.system_monitor.SamplerDict
   scaffold.system_monitor.StatsDict


Classes
-------

.. autoapisummary::

   scaffold.system_monitor.AbstractBackendSender
   scaffold.system_monitor.SystemMonitor


Module Contents
---------------

.. py:class:: AbstractBackendSender

   Abstract interface that works in conjunction with the system monitor.
   Should be used to configure where the system metrics are being sent to.


   .. py:method:: publish(event: StatsDict, **kwargs) -> None
      :abstractmethod:


      Implements sending `event` to the corresponding backend.



.. py:class:: SystemMonitor(pid: int, backend_sender: AbstractBackendSender, sample_rate_seconds: int = 2, rank: Optional[int] = None)

   Bases: :py:obj:`object`


   
   Initializes the SystemMonitor using the parent threads PID and a backend_sender that implements the
   logic to send the collectesd stats to the corresponding backend, e.g. CSV, Tensorboard, WandB.

   :param pid: Process id of the parent process.
   :type pid: int
   :param backend_sender: Class that publishes the stats, e.g. to WandB.
   :type backend_sender: AbstractBackendSender
   :param sample_rate_seconds: The rate of sampling system stats in seconds.
   :type sample_rate_seconds: int
   :param rank: Rank of the current process. Only needed in a distributed setting.
   :type rank: int


   .. py:method:: flush() -> None

      Used the backend senders publish method to sent the latest metrics to the configured backend



   .. py:method:: shutdown() -> None

      Closes the SystemMonitor's thread.



   .. py:method:: start() -> None

      Starts the the monitor by launching a separate thread.



   .. py:attribute:: gpu_count
      :type:  int


   .. py:attribute:: network_init
      :type:  Optional[Dict]


   .. py:property:: proc
      :type: psutil.Process


      Returns the process corresponding to the parents PID


   .. py:attribute:: sampler
      :type:  StatsDict


.. py:data:: GPUHandle

.. py:data:: SamplerDict

.. py:data:: StatsDict