scaffold.data.artifact_manager.filesystem ========================================= .. py:module:: scaffold.data.artifact_manager.filesystem Classes ------- .. autoapisummary:: scaffold.data.artifact_manager.filesystem.FileSystemArtifactManager Module Contents --------------- .. py:class:: FileSystemArtifactManager(url: str, collection: str = 'default', **fs_kwargs: Any) Bases: :py:obj:`scaffold.data.artifact_manager.base.ArtifactManager` Artifact manager backed by a file system using fsspec. This implementation logs and retrieves artifacts from a specified URL. Initialize a FileSystemArtifactManager. :param url: The base URL of the artifact store. :type url: str :param collection: The default collection name. Defaults to "default". :type collection: str :param \*\*fs_kwargs: Additional keyword arguments for the file system. :type \*\*fs_kwargs: Any .. py:method:: download_artifact(artifact_name: str, collection: Optional[str] = None, version: Optional[str] = None, to: Optional[str] = None) -> Union[scaffold.data.artifact_manager.base.Artifact, scaffold.data.artifact_manager.base.TmpArtifact] Download an artifact from the artifact store. If a destination path `to` is provided, the contents of the artifact version are copied there. Otherwise, a TmpArtifact context manager is returned. :param artifact_name: The artifact name. :type artifact_name: str :param collection: The collection name. Defaults to the active collection. :type collection: Optional[str] :param version: The version of the artifact. If None or "latest", the latest version is used. :type version: Optional[str] :param to: The destination path. If not provided, a temporary directory is used. :type to: Optional[str] :returns: If `to` is provided, returns an Artifact with metadata. Otherwise, returns a TmpArtifact context manager that also has an `artifact` property. :rtype: Union[Artifact, TmpArtifact] .. py:method:: exists_in_collection(artifact_name: str, collection: Optional[str] = None) -> bool Check if an artifact exists in a specific collection. :param artifact_name: The artifact name. :type artifact_name: str :param collection: The collection name. Defaults to the active collection. :type collection: Optional[str] :returns: True if the artifact exists, False otherwise. :rtype: bool .. py:method:: list_artifacts(collection: str = None) -> List[str] Get sorted versions for an artifact. :param collection: Collection name. :type collection: str :returns: List of artifact strings. :rtype: List[str] .. py:method:: list_collection_names() -> Iterable[str] List all collections in the artifact store. :returns: A list of collection names. :rtype: Iterable[str] .. py:method:: list_versions(artifact_name: str, collection: str = None) -> List[str] Get sorted versions for an artifact. :param artifact_name: Artifact name. :type artifact_name: str :param collection: Collection name. :type collection: str :returns: List of version strings sorted by version number (e.g., ["v0", "v1", "v2"]). :rtype: List[str] .. py:method:: log_files(artifact_name: str, local_path: str, description: str, collection: Optional[str] = None, artifact_path: Optional[str] = None) -> scaffold.data.artifact_manager.base.Artifact Log a file or folder as an artifact. This method uploads the file (or folder) located at `local_path` to the artifact store. If `artifact_path` is provided, the file is uploaded to a subpath within the artifact; otherwise, the entire folder is uploaded. :param artifact_name: The artifact name. :type artifact_name: str :param local_path: The local path to the file or folder. :type local_path: str :param description: Description of the artifact. Will be logged at /ARTIFACT_META_DIR/ARTIFACT_DESCRIPTION_FILE and serves to reduce undocumented artifact clutter. :param collection: The collection name. Defaults to the active collection. :type collection: Optional[str] :param artifact_path: The subpath within the artifact for single file uploads. :type artifact_path: Optional[str] :returns: The logged artifact with its metadata (name, collection, version). :rtype: Artifact .. py:attribute:: fs .. py:attribute:: url