🚨Fix memory leaks caused by lru decorators in vision models by yonigozlan · Pull Request #45922 · huggingface/transformers

yonigozlan · 2026-05-12T20:59:12Z

Lru decorators were keeping references to entire models causing memory to not be freed when deleting the models

Breaking changes:

text_embeds input in sam3/edgetam/sam3_lite_text is now expected to be full text embeds and not just pooler outputs, to be coherent with other models and the existing doc .
change attr num_pos_feats to num_position_features in several models in an effort to standardize attr names across models

yonigozlan · 2026-05-12T21:06:47Z

run-slow: beit, conditional_detr, d_fine, data2vec, deformable_detr, deimv2, detr, edgetam, edgetam_video, mask2former, maskformer, oneformer, pp_doclayout_v2, pp_doclayout_v3, rt_detr, rt_detr_v2, sam2, sam2_video, sam3, sam3_lite_text, sam3_tracker, sam3_tracker_video

github-actions · 2026-05-12T21:08:16Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/beit", "models/conditional_detr", "models/d_fine", "models/data2vec", "models/deformable_detr", "models/deimv2", "models/detr", "models/edgetam", "models/edgetam_video", "models/mask2former", "models/maskformer", "models/oneformer", "models/pp_doclayout_v2", "models/pp_doclayout_v3", "models/rt_detr", "models/rt_detr_v2", "models/sam2", "models/sam2_video", "models/sam3", "models/sam3_lite_text", "models/sam3_tracker", "models/sam3_tracker_video"]
quantizations: []

HuggingFaceDocBuilderDev · 2026-05-12T21:13:05Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

github-actions · 2026-05-12T21:40:06Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	024b266b	workflow commit (merge commit)
PR	24b1cdde	branch commit (from PR)
main	7ee56fc2	base commit (on `main`)

⚠️ Model CI failed to report results

The test failure analysis could not be completed. Please check the workflow run for details.

yonigozlan · 2026-05-12T22:22:48Z

run-slow: beit, conditional_detr, d_fine, data2vec, deformable_detr, deimv2, detr, edgetam, edgetam_video, mask2former, maskformer, oneformer, pp_doclayout_v2, pp_doclayout_v3, rt_detr, rt_detr_v2, sam2, sam2_video, sam3, sam3_lite_text, sam3_tracker, sam3_tracker_video

github-actions · 2026-05-12T22:24:11Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/beit", "models/conditional_detr", "models/d_fine", "models/data2vec", "models/deformable_detr", "models/deimv2", "models/detr", "models/edgetam", "models/edgetam_video", "models/mask2former", "models/maskformer", "models/oneformer", "models/pp_doclayout_v2", "models/pp_doclayout_v3", "models/rt_detr", "models/rt_detr_v2", "models/sam2", "models/sam2_video", "models/sam3", "models/sam3_lite_text", "models/sam3_tracker", "models/sam3_tracker_video"]
quantizations: []

github-actions · 2026-05-12T22:43:57Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	14a5605c	workflow commit (merge commit)
PR	344c4c3a	branch commit (from PR)
main	7ee56fc2	base commit (on `main`)

⚠️ Model CI failed to report results

The test failure analysis could not be completed. Please check the workflow run for details.

vasqu

Just did a sanity check and went through a few files. I think this is fine but would like your opinion on my comments; currently we do have a few breaking spots so we should be careful

vasqu · 2026-05-13T12:37:06Z



+@compile_compatible_method_lru_cache(maxsize=10)
+def _cached_generate_relative_position_index(window_size: tuple[int, int]) -> torch.Tensor:


Just to be sure that I understood the root cause --> the lru cache saved a reference of self as it's a model's internal function?

Would it be possible to instead make these staticmethod? Might be a more natural move than all the private global fns

Anyways we do need to add 🚨 because it changes the behavior slightly and people might have relied on these internal functions

Just to be sure that I understood the root cause --> the lru cache saved a reference of self as it's a model's internal function?

Yes, I'm not sure about the exact inner workings of lru_cache but as a general rule it's not good practice to apply them to instance methods. That was my mistake

Would it be possible to instead make these staticmethod? Might be a more natural move than all the private global fns

That's a good point! I'll try to refactor with staticmethod instead, and make sure lru_cache doesn't cause issues when applied to static method (should be ok)

vasqu · 2026-05-13T12:40:11Z

+
+
+@compile_compatible_method_lru_cache(maxsize=1)
+def _cached_build_sine_position_embedding(*args, **kwargs) -> torch.Tensor:


Ah ok here we keep BC - we really need to decide what would be better ig but should be consistent then

vasqu · 2026-05-13T12:41:30Z

                text_embeds = model.get_text_features(
                    input_ids=inputs_dict["input_ids"], attention_mask=inputs_dict["attention_mask"], return_dict=True
-                ).pooler_output
+                )


That is definitely breaking 👀 Any reason we could not keep the same behavior there?

I think it was broken before this fix, as the docs were already using the behavior of this PR, but it's a niche feature so maybe why it was never flagged. I'll add a 🚨

yonigozlan · 2026-05-13T14:57:18Z

Thanks for the review @vasqu ! Indeed it might be better to use staticmethod, I'll make the changes and ping you when it's ready :)

yonigozlan · 2026-05-13T21:14:18Z

run-slow: beit, conditional_detr, d_fine, data2vec, deformable_detr, deimv2, detr, edgetam, edgetam_video, mask2former, maskformer, oneformer, pp_doclayout_v2, pp_doclayout_v3, rt_detr, rt_detr_v2, sam2, sam2_video, sam3, sam3_lite_text, sam3_tracker, sam3_tracker_video

github-actions · 2026-05-13T21:16:08Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/beit", "models/conditional_detr", "models/d_fine", "models/data2vec", "models/deformable_detr", "models/deimv2", "models/detr", "models/edgetam", "models/edgetam_video", "models/mask2former", "models/maskformer", "models/oneformer", "models/pp_doclayout_v2", "models/pp_doclayout_v3", "models/rt_detr", "models/rt_detr_v2", "models/sam2", "models/sam2_video", "models/sam3", "models/sam3_lite_text", "models/sam3_tracker", "models/sam3_tracker_video"]
quantizations: []

github-actions · 2026-05-13T21:56:51Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	98f1299d	workflow commit (merge commit)
PR	4c8136f3	branch commit (from PR)
main	98a25186	base commit (on `main`)

Model CI Report

❌ 29 new failed tests from this PR 😭

edgetam_video:
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_mask_generation_video_multi_objects_multi_points (✅ ⟹ ❌)
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_mask_generation_video_multi_points (✅ ⟹ ❌)
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_mask_generation_video_one_bb (✅ ⟹ ❌)
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_mask_generation_video_one_point (✅ ⟹ ❌)
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_mask_generation_video_one_point_one_bb (✅ ⟹ ❌)
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_mask_generation_video_one_point_propagate_in_video_directly (✅ ⟹ ❌)
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_propagate_on_streamed_video (✅ ⟹ ❌)
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_propagate_video_from_mask_input (✅ ⟹ ❌)
tests/models/edgetam_video/test_modeling_edgetam_video.py::EdgeTamVideoModelIntegrationTest::test_inference_with_different_dtypes (✅ ⟹ ❌)
sam2_video:
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_mask_generation_video_batched_bb (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_mask_generation_video_multi_objects_multi_points (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_mask_generation_video_multi_points (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_mask_generation_video_one_bb (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_mask_generation_video_one_point (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_mask_generation_video_one_point_one_bb (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_mask_generation_video_one_point_propagate_in_video_directly (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_propagate_on_streamed_video (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_propagate_video_from_mask_input (✅ ⟹ ❌)
tests/models/sam2_video/test_modeling_sam2_video.py::Sam2VideoModelIntegrationTest::test_inference_with_different_dtypes (✅ ⟹ ❌)
sam3_tracker_video:
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_mask_generation_video_batched_bb (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_mask_generation_video_multi_objects_multi_points (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_mask_generation_video_multi_points (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_mask_generation_video_one_bb (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_mask_generation_video_one_point (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_mask_generation_video_one_point_one_bb (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_mask_generation_video_one_point_propagate_in_video_directly (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_propagate_on_streamed_video (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_propagate_video_from_mask_input (✅ ⟹ ❌)
tests/models/sam3_tracker_video/test_modeling_sam3_tracker_video.py::Sam3TrackerVideoModelIntegrationTest::test_inference_with_different_dtypes (✅ ⟹ ❌)

yonigozlan · 2026-05-15T15:33:08Z

run-slow: beit, conditional_detr, d_fine, data2vec, deformable_detr, deimv2, detr, edgetam, edgetam_video, mask2former, maskformer, oneformer, pp_doclayout_v2, pp_doclayout_v3, rt_detr, rt_detr_v2, sam2, sam2_video, sam3, sam3_lite_text, sam3_tracker, sam3_tracker_video

github-actions · 2026-05-15T15:34:28Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/beit", "models/conditional_detr", "models/d_fine", "models/data2vec", "models/deformable_detr", "models/deimv2", "models/detr", "models/edgetam", "models/edgetam_video", "models/mask2former", "models/maskformer", "models/oneformer", "models/pp_doclayout_v2", "models/pp_doclayout_v3", "models/rt_detr", "models/rt_detr_v2", "models/sam2", "models/sam2_video", "models/sam3", "models/sam3_lite_text", "models/sam3_tracker", "models/sam3_tracker_video"]
quantizations: []

yonigozlan · 2026-05-15T15:44:31Z

Waiting to see if all slow tests pass but it should be good to go then @vasqu !

vasqu · 2026-05-15T15:53:57Z

Sounds good, keeping an eye here to review later today 🫡

github-actions · 2026-05-15T16:05:28Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	7fbd5dec	workflow commit (merge commit)
PR	c307ba63	branch commit (from PR)
main	331f1007	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

vasqu

Just a few smaller comments 🤗 Can you also add the potentially breaking stuff into the PR description, e.g. the sam3 pooler behavior, small renames for args (pos features)

vasqu · 2026-05-15T16:45:10Z

+    @staticmethod
    @compile_compatible_method_lru_cache(maxsize=32)
+    def _cached_build_2d_sinusoidal_position_embedding(*args, **kwargs) -> torch.Tensor:
+        return build_2d_sinusoidal_position_embedding(*args, **kwargs)


this wrapper structure is a bit awkward, could we move things here a bit to make it directly use build_2d_sinusoidal_position_embedding or is the inheritance too awkward?

Not sure we can do that and still have it standardized everywhere it's used, because some models call build_2d_sinusoidal_position_embedding at init and copy the result in their weights, so they don't need to lru_cache it at all.

vasqu · 2026-05-15T16:47:10Z

        self.num_feature_levels = 3
-        self.position_embedder = Mask2FormerSinePositionEmbedding(num_pos_feats=hidden_dim // 2, normalize=True)
+        self.position_embedder = Mask2FormerSinePositionEmbedding(
+            num_position_features=hidden_dim // 2, normalize=True


this is slightly breaking because of the rename

…transformers into fix-lru-memory-leaks

github-actions · 2026-05-15T19:49:57Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: beit, conditional_detr, d_fine, dab_detr, data2vec, deformable_detr, deimv2, detr, edgetam, edgetam_video, got_ocr2, mask2former, maskformer, oneformer, pp_doclayout_v2, pp_doclayout_v3

yonigozlan added 2 commits May 12, 2026 20:55

fix lru leaks and standardized sine pos embeds

1d8e780

Merge remote-tracking branch 'upstream/main' into fix-lru-memory-leaks

24b1cdd

yonigozlan requested a review from molbap May 12, 2026 21:08

yonigozlan requested a review from vasqu May 12, 2026 21:08

fix sam3 tests

344c4c3

vasqu reviewed May 13, 2026

View reviewed changes

yonigozlan and others added 2 commits May 13, 2026 19:58

use static methods

e4f9adc

Merge branch 'main' into fix-lru-memory-leaks

4c8136f

yonigozlan added 4 commits May 14, 2026 15:27

Merge remote-tracking branch 'upstream/main' into fix-lru-memory-leaks

db6c36a

temp fix

385f165

standardize SinePositionEmbedding output

dedc005

Merge remote-tracking branch 'upstream/main' into fix-lru-memory-leaks

c307ba6

yonigozlan changed the title ~~Fix memory leaks caused by lru decorators in vision models~~ 🚨Fix memory leaks caused by lru decorators in vision models May 15, 2026

fix sam3video

ec7712d

vasqu approved these changes May 15, 2026

View reviewed changes

yonigozlan and others added 4 commits May 15, 2026 18:28

permute -> transpose

374b70d

Merge branch 'main' into fix-lru-memory-leaks

c11f14d

remove print

8e00fe4

Merge branch 'fix-lru-memory-leaks' of https://github.com/yonigozlan/…

38d60b7

…transformers into fix-lru-memory-leaks

yonigozlan enabled auto-merge May 15, 2026 19:53

yonigozlan added this pull request to the merge queue May 15, 2026

Merged via the queue into huggingface:main with commit 3ef2781 May 15, 2026
92 of 93 checks passed

yonigozlan deleted the fix-lru-memory-leaks branch May 15, 2026 20:21



		@compile_compatible_method_lru_cache(maxsize=10)
		def _cached_generate_relative_position_index(window_size: tuple[int, int]) -> torch.Tensor:



		@compile_compatible_method_lru_cache(maxsize=1)
		def _cached_build_sine_position_embedding(args, *kwargs) -> torch.Tensor:

Conversation

yonigozlan commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yonigozlan commented May 12, 2026

Uh oh!

github-actions Bot commented May 12, 2026

Uh oh!

HuggingFaceDocBuilderDev commented May 12, 2026

Uh oh!

github-actions Bot commented May 12, 2026

CI Results

Commit Info

Uh oh!

yonigozlan commented May 12, 2026

Uh oh!

github-actions Bot commented May 12, 2026

Uh oh!

github-actions Bot commented May 12, 2026

CI Results

Commit Info

Uh oh!

vasqu left a comment

Choose a reason for hiding this comment

Uh oh!

vasqu May 13, 2026

Choose a reason for hiding this comment

Uh oh!

yonigozlan May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vasqu May 13, 2026

Choose a reason for hiding this comment

Uh oh!

vasqu May 13, 2026

Choose a reason for hiding this comment

Uh oh!

yonigozlan May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yonigozlan commented May 13, 2026

Uh oh!

yonigozlan commented May 13, 2026

Uh oh!

github-actions Bot commented May 13, 2026

Uh oh!

github-actions Bot commented May 13, 2026

CI Results

Commit Info

Model CI Report

Uh oh!

yonigozlan commented May 15, 2026

Uh oh!

github-actions Bot commented May 15, 2026

Uh oh!

yonigozlan commented May 15, 2026

Uh oh!

vasqu commented May 15, 2026

Uh oh!

github-actions Bot commented May 15, 2026

CI Results

Commit Info

Uh oh!

vasqu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vasqu May 15, 2026

Choose a reason for hiding this comment

Uh oh!

yonigozlan May 15, 2026

Choose a reason for hiding this comment

Uh oh!

vasqu May 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented May 15, 2026

yonigozlan commented May 12, 2026 •

edited

Loading

yonigozlan May 13, 2026 •

edited

Loading

yonigozlan May 13, 2026 •

edited

Loading