gw.training.train_pipeline.copy_files_to_local

gw.training.train_pipeline.copy_files_to_local(
    file_path,
    local_dir,
    leave_keys_on_disk,
    is_condor=False,
)

Copy files to local node if local_dir is provided to minimize network traffic during training.

Parameters

Name	Type	Description	Default
file_path	str	Path to file that should be copied.	required
local_dir	Optional[str]	Directory where file should be copied. If None, file will not be copied.	required
leave_keys_on_disk	bool	Whether to leave keys on disk and load them during training. If dataset is not copied and leave_keys_on_disk is True, a warning will be raised.	required
is_condor	bool	Whether this is a condor job.	`False`

Name	Type	Description
local_file_path	str	Modified file path if file was copied to local node, else the original file path.