Skip to content

Partition anisotropic lines onto one rank and make it the default - #2980

Open
bigfooted wants to merge 1 commit into
developfrom
feature_parmetis_anisotropy
Open

bigfooted wants to merge 1 commit into
developfrom
feature_parmetis_anisotropy

Conversation

@bigfooted

Copy link
Copy Markdown
Contributor

This PR splits off the parmetis improvements from #2931. It turns out that this parmetis improvement is 'almost' always better, and in the worst case performs equally well. It improves MPI performance for cases with inflation layers.

PARMETIS_ANISOTROPY_WEIGHT (default YES) weights the ParMETIS graph edges by cell aspect ratio, gives the edges that chain points into wall-normal lines the largest weight, and moves each line to the rank holding most of its points after partitioning. Lines are then not split between ranks, which keeps the block-Jacobi ILU and line-based agglomeration from seeing fragments of lines.

On stretched boundary-layer meshes this turned stalled or diverging parallel runs into converging ones (2D shear layer, 2D bump, 3D bump, flat plate with multigrid) and made iteration counts match the serial run more closely, with no case that converged without it failing with it. The load imbalance stays within 1.01-1.05 in 2D and up to about 1.3 on a large 3D mesh. The option has no effect on isotropic meshes or without ParMETIS; a warning is printed when it is set explicitly in a build without ParMETIS.

Proposed Changes

Give a brief overview of your contribution here in a few sentences.

Related Work

Resolve any issues (bug fix or feature request), note any related PRs, or mention interactions with the work of others, if any.

PR Checklist

Put an X by all that apply. You can fill this out after submitting the PR. If you have any questions, don't hesitate to ask! We want to help. These are a guide for you to know what the reviewers will be looking for in your contribution.

  • I am submitting my contribution to the develop branch.
  • My contribution generates no new compiler warnings (try with --warnlevel=3 when using meson).
  • My contribution is commented and consistent with SU2 style (https://su2code.github.io/docs_v7/Style-Guide/).
  • I used the pre-commit hook to prevent dirty commits and used pre-commit run --all to format old commits.
  • I have added a test case that demonstrates my contribution, if necessary.
  • I have updated appropriate documentation (Tutorials, Docs Page, config_template.cpp), if necessary.

PARMETIS_ANISOTROPY_WEIGHT (default YES) weights the ParMETIS graph edges by
cell aspect ratio, gives the edges that chain points into wall-normal lines
the largest weight, and moves each line to the rank holding most of its
points after partitioning. Lines are then not split between ranks, which
keeps the block-Jacobi ILU and line-based agglomeration from seeing
fragments of lines.

On stretched boundary-layer meshes this turned stalled or diverging parallel
runs into converging ones (2D shear layer, 2D bump, 3D bump, flat plate with
multigrid) and made iteration counts match the serial run more closely, with
no case that converged without it failing with it. The load imbalance stays
within 1.01-1.05 in 2D and up to about 1.3 on a large 3D mesh. The option has
no effect on isotropic meshes or without ParMETIS; a warning is printed when
it is set explicitly in a build without ParMETIS.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant