Skip to content

Make sure download stats work on Hugging Face, improve metadata - #6

Merged
wangbinDL merged 1 commit into
opendatalab:mainfrom
NielsRogge:feature/add_hf
Oct 22, 2024
Merged

wangbinDL merged 1 commit into
opendatalab:mainfrom
NielsRogge:feature/add_hf

Conversation

@NielsRogge

@NielsRogge NielsRogge commented Oct 19, 2024 •

Copy link
Copy Markdown
Contributor

Hi @wangbinDL,

Thanks for this nice work, great to see the models being released on the hub! Your paper was featured on the daily papers: https://huggingface.co/papers/2410.12628. Would be great to link the models to the paper.

To make download stats work, I wrote a PR similar to THU-MIG/yolov10#168 which I did for the YOLOv10 repository.

It leverages the PyTorchModelHubMixin developed by the 🤗 team to make sure a custom PyTorch model like yours:

  • automatically inherits from_pretrained and push_to_hub methods
  • have better metadata
  • make sure download stats work.

Usage is as follows:

from doclayout_yolo import YOLOv10
from huggingface_hub import hf_hub_download

# Load a pre-trained model
filepath = hf_hub_download(repo_id="juliozhao/DocLayout-YOLO-D4LA-from_scratch", filename="doclayout_yolo_d4la_imgsz1600_from_scratch.pt")
model = YOLOv10(filepath)

# One can optionally push this to the hub
model.push_to_hub("juliozhao/DocLayout-YOLO-D4LA-from_scratch")

# Can now be reloaded as follows (and will increment download count)
model = YOLOv10.from_pretrained("juliozhao/DocLayout-YOLO-D4LA-from_scratch")

Besides that, some more suggestions to improve the HF release:

Let me know whether you need any help!

Kind regards,

Niels from HF

@JulioZhao97

Copy link
Copy Markdown
Member

Thank you very much for your suggestions! If there are updates I will let you know immediately.

@wangbinDL

Copy link
Copy Markdown
Member

Hi @NielsRogge

Thanks for your valuable suggestions. We've now:

Your assistance is greatly appreciated!

Best,
Bin

@wangbinDL
wangbinDL merged commit 966f1bb into opendatalab:main Oct 22, 2024
@NielsRogge

Copy link
Copy Markdown
Contributor Author

Thanks, looking great!

Note that for download stats to work, you will need to run the code snippet above for each of the model repos (basically overwriting them). As the class now inherits from PyTorchModelHubMixin, a config.json along with safetensors weights will be pushed.

@JulioZhao97

Copy link
Copy Markdown
Member

Thanks, looking great!

Note that for download stats to work, you will need to run the code snippet above for each of the model repos (basically overwriting them). As the class now inherits from PyTorchModelHubMixin, a config.json along with safetensors weights will be pushed.

Noted, thanks for your contribution again!

@NielsRogge

Copy link
Copy Markdown
Contributor Author

Hi @JulioZhao97 are you planning to run the script above?

Cause I see downloads aren't working yet

@JulioZhao97

Copy link
Copy Markdown
Member

@NielsRogge Hello! I tested the script above at the time you submit the PR and I find

model = YOLOv10.from_pretrained("juliozhao/DocLayout-YOLO-D4LA-from_scratch")

requires yolov10n.pt, and the model weights I upload to huggingface reposity have not been re-uploaded. But other in the script above is fine. I will see to it in these two days.

@JulioZhao97

Copy link
Copy Markdown
Member

@NielsRogge Hello! I tried model.push_to_hub but no model weights found in the repository, could you please tell me where is wrong?
image

Follow is the target repository:
image

@NielsRogge

Copy link
Copy Markdown
Contributor Author

Hi @JulioZhao97 I just tried, it works for me: https://huggingface.co/nielsr/DocLayout-YOLO-D4LA-from_scratch/tree/main

Colab notebook: https://colab.research.google.com/drive/1N8Rfdu875lu5VCKG4uRUcM-Gi0o6qHom?usp=sharing.

Perhaps you might need to update your huggingface_hub version

@JulioZhao97 JulioZhao97 mentioned this pull request Nov 13, 2024
@agombert

Copy link
Copy Markdown

Hey @JulioZhao97 how did you solve the pb to push to the hub ? Do you need to download the repo to do so as in Niels example ?

@NielsRogge

Copy link
Copy Markdown
Contributor Author

@agombert which error are you encountering?

@agombert

Copy link
Copy Markdown

@NielsRogge thanks for the quick answer

model = YOLOv10.from_pretrained("juliozhao/DocLayout-YOLO-D4LA-from_scratch")
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[91], [line 1](vscode-notebook-cell:?execution_count=91&line=1)
----> [1](vscode-notebook-cell:?execution_count=91&line=1) model = YOLOv10.from_pretrained("juliozhao/DocLayout-YOLO-D4LA-from_scratch")

AttributeError: type object 'YOLOv10' has no attribute 'from_pretrained'

It happens when I install doclayout-yolo = "^0.0.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants