loadModel: local_files_only shaves off almost 2s on my Quadro RTX 5k

gadicc · gadicc · commit 68ad3ec0f9ea · 2022-11-08T13:05:19.000Z
diff --git a/loadModel.py b/loadModel.py
@@ -30,6 +30,7 @@ def loadModel(model_id: str, load=True):
         torch_dtype=torch.float16,
         use_auth_token=HF_AUTH_TOKEN,
         scheduler=scheduler,
+        local_files_only=load,
     )
 
     return model.to("cuda") if load else None

Original file line number	Diff line number	Diff line change
`@@ -30,6 +30,7 @@ def loadModel(model_id: str, load=True):`
`30`	`30`	`torch_dtype=torch.float16,`
`31`	`31`	`use_auth_token=HF_AUTH_TOKEN,`
`32`	`32`	`scheduler=scheduler,`
	`33`	`+ local_files_only=load,`
`33`	`34`	`)`
`34`	`35`
`35`	`36`	`return model.to("cuda") if load else None`