Skip to content

[attention backends] fix ring CP for flash and flash 3#13182

Merged
sayakpaul merged 12 commits intomainfrom
cp-attn-backend-tests
Apr 24, 2026
Merged

[attention backends] fix ring CP for flash and flash 3#13182
sayakpaul merged 12 commits intomainfrom
cp-attn-backend-tests

Conversation

@sayakpaul
Copy link
Copy Markdown
Member

What does this PR do?

  • Fixes support for ring attention for Flash and Flash 3
  • Adds a test suite for CP backends x attention backends
image (CP x FA3) image (CP x FA2)

@sayakpaul sayakpaul requested a review from DN6 February 24, 2026 11:39
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@sayakpaul
Copy link
Copy Markdown
Member Author

@DN6 a gentle ping.

@github-actions github-actions Bot added models tests size/M PR with diff < 200 LOC labels Apr 17, 2026
@sayakpaul
Copy link
Copy Markdown
Member Author

Tested with pytest tests/models/transformers/test_models_transformer_flux.py::TestFluxTransformerContextParallelAttnBackends -- everything passes.

Copy link
Copy Markdown
Collaborator

@DN6 DN6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Small request to add an attention marker to the Mixin.

Comment thread tests/models/testing_utils/parallelism.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
@github-actions github-actions Bot added size/M PR with diff < 200 LOC and removed size/M PR with diff < 200 LOC labels Apr 24, 2026
@github-actions github-actions Bot added size/M PR with diff < 200 LOC and removed size/M PR with diff < 200 LOC labels Apr 24, 2026
@github-actions github-actions Bot added size/M PR with diff < 200 LOC and removed size/M PR with diff < 200 LOC labels Apr 24, 2026
@github-actions github-actions Bot removed the size/M PR with diff < 200 LOC label Apr 24, 2026
@github-actions github-actions Bot added the size/M PR with diff < 200 LOC label Apr 24, 2026
@github-actions github-actions Bot added size/M PR with diff < 200 LOC and removed size/M PR with diff < 200 LOC labels Apr 24, 2026
@sayakpaul sayakpaul merged commit f7fd76a into main Apr 24, 2026
15 of 20 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Diffusers Roadmap 0.38 Apr 24, 2026
@sayakpaul sayakpaul deleted the cp-attn-backend-tests branch April 24, 2026 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models size/M PR with diff < 200 LOC tests

Projects

Development

Successfully merging this pull request may close these issues.

3 participants