When downloading models through Torch Hub, models are automatically downloaded in
export TORCH_HOME=/my/path/
export TRANSFORMERS_CACHE=/my/path/ will update hugging face model path
python -m torch.distributed.run --nproc_per_node=16 will add WORLD_SIZE in os.environ