Skip to content

[VL] Rework option spark.gluten.sql.columnar.backend.velox.asyncTimeoutOnTaskStopping#11882

Open
zhztheplayer wants to merge 7 commits intoapache:mainfrom
zhztheplayer:wip-async
Open

[VL] Rework option spark.gluten.sql.columnar.backend.velox.asyncTimeoutOnTaskStopping#11882
zhztheplayer wants to merge 7 commits intoapache:mainfrom
zhztheplayer:wip-async

Conversation

@zhztheplayer
Copy link
Copy Markdown
Member

@zhztheplayer zhztheplayer commented Apr 7, 2026

  1. Ensures Velox async threads can be joined after Task ends;
  2. Removes the current looped-waiting code since no longer needed. Used new blocking-join mechanism instead.
  3. spark.gluten.sql.columnar.backend.velox.asyncTimeoutOnTaskStopping is still used as the joining timeout here.

@zhztheplayer zhztheplayer force-pushed the wip-async branch 4 times, most recently from 9dd1dc0 to 7fa72c4 Compare April 17, 2026 01:48
@zhztheplayer zhztheplayer marked this pull request as ready for review April 17, 2026 01:48
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

2 similar comments
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

2 similar comments
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

2 similar comments
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@zhztheplayer
Copy link
Copy Markdown
Member Author

Comment thread cpp/velox/compute/VeloxBackend.cc Outdated

initJolFilesystem();
initConnector(hiveConf);
initConnector(hiveConnectorConfig_);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to pass the argument?

Copy link
Copy Markdown
Member Author

@zhztheplayer zhztheplayer Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. initConnector is now misleading, and let me untangle this code a bit.

Comment thread cpp/velox/compute/VeloxBackend.cc Outdated
}

void VeloxBackend::initConnector(const std::shared_ptr<velox::config::ConfigBase>& hiveConf) {
(void)hiveConf;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this line?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to mark out an unused parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants