Skip to content

UN-3798 [FIX] Skip broken file-path task load for PG pluggable workers#2197

Merged
muhammad-ali-e merged 2 commits into
feat/UN-3445-pg-queue-aux-workersfrom
fix/UN-3798-pg-pluggable-loader
Jul 23, 2026
Merged

UN-3798 [FIX] Skip broken file-path task load for PG pluggable workers#2197
muhammad-ali-e merged 2 commits into
feat/UN-3445-pg-queue-aux-workersfrom
fix/UN-3798-pg-pluggable-loader

Conversation

@muhammad-ali-e

Copy link
Copy Markdown
Contributor

What

  • For pluggable WORKER_TYPEs, the top-level workers/worker.py no longer loads the plugin's tasks.py by file path under a bare "tasks" module name. Their tasks are already registered by WorkerBuilder.build_celery_app(); the file-path load is skipped.
  • Non-pluggable (top-level) workers keep the existing file-path load, unchanged.

Why

  • The file-path load used spec_from_file_location("tasks", ...) — a bare name with no parent package — so every plugin tasks.py relative import (from .clients import ...) failed with ImportError: attempted relative import with no known parent package, crash-looping the worker on startup under python -m pg_queue_consumer.
  • This blocks every cloud PG pluggable worker (bulk-download UN-3752, agentic-callback UN-3754, agentic_studio UN-3779). It was never caught because these workers are disabled in all base values and no test had started one via the PG consumer.
  • The Celery path is unaffected: Celery pluggable workers run via celery -A pluggable_worker.{type}.worker (a proper dotted package import), which resolves relative imports and never touches this loader. The is_pluggable() file-path branch never actually ran successfully.

How

  • build_celery_app() (called just above the loader) verifies a pluggable type via _verify_pluggable_worker_existsimportlib.import_module("pluggable_worker.{type}.worker") — a package import that runs the plugin's from . import tasks and registers the tasks on this same app. So the file-path load is redundant + broken; skip it for pluggable workers.

Can this PR break any existing features?

  • No. The Celery path and non-pluggable file-path path are untouched (non-pluggable load simply moved into the else branch). Removes a code path that never ran successfully. PG and Celery register identical tasks (both via build_celery_app).

Database Migrations

  • None.

Env Config

  • None.

Notes on Testing

  • Validated on a running dev stack via python -m pg_queue_consumer: agentic_callback and agentic_studio now start clean (tasks already registered … skipping file-path task loadready for Celery); general (non-pluggable) loads unchanged via the file-path path.
  • No unit test: worker.py is module-level startup code and this repo has no pluggable-worker fixture; the runtime worker-start is the validation.

Related Issues or PRs

  • Prerequisite for UN-3752, UN-3754, UN-3779.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3d6a6edd-5b28-4b9c-bf97-bae70c7f0412

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/UN-3798-pg-pluggable-loader

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Base automatically changed from feat/UN-3445-pg-queue-integration to main July 23, 2026 08:25
The top-level worker.py loaded a pluggable worker's tasks.py via
spec_from_file_location("tasks", ...) — a bare module name with no parent
package — so the plugin's relative imports (from .clients import ...) failed
with "attempted relative import with no known parent package", crash-looping
every cloud PG pluggable worker (agentic_callback/UN-3754, agentic_studio/
UN-3779, bulk_download/UN-3752) on startup under `python -m pg_queue_consumer`.

The tasks are already registered by that point: WorkerBuilder.build_celery_app()
(called just above) verifies a pluggable type by importing
pluggable_worker.{type}.worker as a proper package (_verify_pluggable_worker_exists
→ import_module), which runs the plugin's `from . import tasks` and registers the
tasks on this same app. So the file-path load is both redundant and broken.

Fix: skip the file-path load for pluggable workers; non-pluggable (top-level)
workers keep it unchanged. The Celery path is unaffected — it runs via
`celery -A pluggable_worker.{type}.worker` (a dotted package import) and never
touches this loader; the is_pluggable() file-path branch never ran successfully.

Validated on a running dev stack via `python -m pg_queue_consumer`:
agentic_callback and agentic_studio now start clean ("tasks already registered
… skipping file-path task load" → "ready for Celery"); general (non-pluggable)
loads unchanged. Prerequisite for UN-3752 / UN-3754 / UN-3779.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@muhammad-ali-e
muhammad-ali-e force-pushed the fix/UN-3798-pg-pluggable-loader branch from c407b45 to 9314a14 Compare July 23, 2026 09:12
@muhammad-ali-e
muhammad-ali-e changed the base branch from main to feat/UN-3445-pg-celery-decommission July 23, 2026 09:12

@muhammad-ali-e muhammad-ali-e left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR review (PR Review Toolkit: Code Reviewer, Silent-Failure Hunter, Type-Design Analyzer, Test Analyzer, Comment Analyzer, Code Simplifier).

Verdict: the fix itself is correct and behavior-preserving — the file-path load is cleanly narrowed to the non-pluggable branch, and the pluggable skip is safe (a missing/broken plugin already raises ImportError in build_celery_app before this code runs). No blocking bugs. The comments below are mostly comment-accuracy fixes plus optional hardening/refactor suggestions.

Comment thread workers/worker.py Outdated
# `importlib.import_module("pluggable_worker.{type}.worker")` — a proper
# PACKAGE import that runs the plugin's own `from . import tasks`, resolving
# its relative imports (`from .clients import ...`) and registering the tasks
# on this same app. The generic file-path load below loads tasks.py under a

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Comment accuracy — MEDIUM] "registering the tasks on this same app" is contradicted by call ordering. _verify_pluggable_worker_exists() performs the import_module at builder.py:60, before app = Celery(app_name) is created at builder.py:74. The app object this file uses does not exist yet when the plugin is imported, so registration must go through Celery's deferred binding (@shared_task / connect_on_app_finalize), not a direct attach to "this same app".

Suggested wording: "…runs the plugin's task-registration code. Celery binds those tasks to the worker's app when it finalizes, so the tasks are already registered by this point."

Comment thread workers/worker.py Outdated
# on this same app. The generic file-path load below loads tasks.py under a
# bare "tasks" spec (no parent package), which BREAKS those relative imports
# ("attempted relative import with no known parent package"), so it MUST be
# skipped for pluggable workers — otherwise every cloud PG pluggable worker

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Comment accuracy — LOW] "otherwise every cloud PG pluggable worker crashes on startup" overstates the guarantee. The attempted relative import with no known parent package crash only fires for plugins whose tasks.py actually uses relative imports; a plugin written with absolute imports would not crash. Suggest softening to the real precondition, e.g. "…which breaks any relative imports in the plugin's tasks.py, so the file-path load must be skipped for pluggable workers."

Comment thread workers/worker.py Outdated
# build_celery_app() above verifies the plugin via
# WorkerBuilder._verify_pluggable_worker_exists, which does
# `importlib.import_module("pluggable_worker.{type}.worker")` — a proper
# PACKAGE import that runs the plugin's own `from . import tasks`, resolving

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Maintainability / rot risk — LOW] This comment hard-codes the internal structure of cloud-only plugin code that is not in this repo (pluggable_worker/ is gitignored in OSS): from . import tasks, from .clients import .... Nobody editing the OSS side can validate these specifics, and they silently rot if a plugin registers tasks differently. Prefer stating the contract ("pluggable workers are proper packages; verification imports pluggable_worker.<type>.worker as a package, running its own task registration incl. relative imports") and marking the concrete from . import ... examples as illustrative ("e.g.").

Comment thread workers/worker.py Outdated
# Pluggable workers live inside workers/pluggable_worker/{worker_name}
worker_directory = os.path.join("pluggable_worker", worker_type.value)
worker_path = os.path.join(base_dir, worker_directory)
# Pluggable workers' tasks are ALREADY registered by this point:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Structure / testability — suggestion] Three agents (Simplifier, Test Analyzer, Type-Design) converged here: extract lines 445–490 into a load_worker_tasks(worker_type) helper. Benefits: (1) the giant explanatory comment becomes a proper docstring; (2) nesting collapses from depth 3 to depth 1 via guard-clause returns (impossible at module scope today); (3) it creates the seam for the single highest-value regression test — assert spec_from_file_location is never called when is_pluggable() is True — which permanently pins this fix against a future re-merge of the branches. All locals (base_dir, worker_directory, worker_path, tasks_file, spec, tasks_module) are used only in this block, so extraction leaks nothing. Reasonable to defer as a fast-follow given this is a hotfix.

Comment thread workers/worker.py
logger.warning(f"⚠️ No tasks.py found at: {tasks_file}")
else:
logger.error(f"❌ Worker directory not found: {worker_path}")
logger.error(f"❌ Worker directory not found: {worker_path}")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Silent failure — MEDIUM] Two zero-task boot paths survive here: (a) this else logs an error but execution continues to the ✅ Successfully loaded line at 492 — a misleading success after a fatal misconfig (pre-existing, just re-indented); (b) after removing the file-path load for pluggable workers, plugin-side import is now the sole task-registration mechanism with no post-check. Celery does not error on an empty task registry — the worker starts and silently processes nothing. Consider one guard after the if/else that covers both paths:

registered = [n for n in app.tasks if not n.startswith("celery.")]
if not registered:
    raise RuntimeError(
        f"No tasks registered for worker '{worker_type.value}' "
        f"(pluggable={worker_type.is_pluggable()}) — worker would start but process nothing."
    )

Comment thread workers/worker.py Outdated
# Non-pluggable (top-level) workers register tasks via a file-path load of
# their tasks.py — they use absolute imports, with their dir on sys.path.
# Enum values use underscores (Python module names); a few on-disk dirs still
# use hyphens (e.g. api-deployment). Derive the directory from the

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Type design — LOW] Deriving the on-disk directory as worker_type.to_import_path().rsplit(".", 1)[0] (line 468) reuses an import-path string as a filesystem path — it only works because every path is "{dir}.tasks". Consider an explicit WorkerType.to_directory() -> str that owns the hyphen/underscore mapping, so intent is named and the mapping lives in one place. (Keep the task-loading logic itself out of the enum — the other WorkerType methods are pure derivations; a load_tasks() would drag sys.path/importlib/the Celery app into a lightweight str, Enum.)

Comment thread workers/worker.py Outdated
if os.path.exists(tasks_file):
logger.info(f"📋 Loading tasks from: {tasks_file}")
# Import the tasks module to register tasks with the app
import importlib.util

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] import importlib.util is done inside the innermost if. Since os/sys are already imported at module top, moving this up with them is more consistent. Style-only.

…tory(), zero-task guard

- Extract the task-load block into load_worker_tasks(worker_type) with guard-clause
  returns (depth 3 -> 1) and an accurate docstring: pluggable tasks register via
  WorkerBuilder's package import and Celery binds them on app finalize; the
  file-path load is skipped for them because it breaks any relative imports in the
  plugin's tasks.py. Softened the overstated "every worker crashes" and marked the
  cloud-plugin `from . import tasks` example as illustrative (contract, not internals).
- Add WorkerType.to_directory() as the single source for the underscore->hyphen dir
  mapping; to_import_path() and the file-path loader both read it (no more slicing
  the import path). + tests.
- Add a post-load zero-task check as a WARNING (not a hard raise): pluggable tasks
  bind on app finalize which can be after this point, so a raise would false-positive
  on a correctly-configured pluggable worker (the exact regression this PR fixes).
- Move `import importlib.util` to the module-top imports.

Deferred (per reviewer's hotfix note): the "spec_from_file_location never called for
pluggable" regression test — worker.py runs infra init + builds the Celery app at
import, so there is no clean seam to exercise the loader in isolation without a
larger main()-guard refactor. The extraction creates that seam for a fast-follow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@muhammad-ali-e

Copy link
Copy Markdown
Contributor Author

Addressed all 7 threads in f7905a3e9.

  • Deferred-binding comment accuracy — reworded: pluggable tasks register via WorkerBuilder's package import and Celery binds them to the app on finalize, so they're registered by this point (dropped the "on this same app" phrasing that contradicted the import_module-before-Celery() ordering).
  • "every worker crashes" overstated — softened to the real precondition: the file-path load "breaks any relative imports in the plugin's tasks.py", so it's skipped for pluggable workers.
  • Hard-coded cloud-plugin internals — now states the contract (pluggable workers are proper packages; verification imports the package and runs its own registration) and marks from . import tasks as illustrative (e.g.).
  • Extract to a helper — extracted load_worker_tasks(worker_type) with guard-clause returns (nesting depth 3 → 1) and the explanatory comment moved into a docstring.
  • Silent zero-task boot — added a post-load check, but as a WARNING, not a hard raise: pluggable tasks bind on app finalize (@shared_task/connect_on_app_finalize), which can be after this point, so a raise here would false-positive on a correctly-configured pluggable worker — i.e. re-introduce the very crash this PR fixes. The warning surfaces the misconfig without that risk.
  • to_directory() type design — added WorkerType.to_directory() as the single source of the underscore→hyphen mapping; to_import_path() and the loader both read it (no more slicing the import path). Pinned by tests.
  • import importlib.util nit — moved to the module-top imports.

Deferred (per your hotfix note): the "spec_from_file_location never called when is_pluggable()" regression test. worker.py runs infra init + builds the Celery app at import, so there's no clean seam to call the loader in isolation without a larger main()-guard refactor. The extraction creates that seam for a fast-follow; in the meantime to_directory() is covered by tests/test_worker_enums_directory.py (3 tests pass).

@sonarqubecloud

Copy link
Copy Markdown

@muhammad-ali-e
muhammad-ali-e changed the base branch from feat/UN-3445-pg-celery-decommission to feat/UN-3445-pg-queue-aux-workers July 23, 2026 14:27
@muhammad-ali-e
muhammad-ali-e marked this pull request as ready for review July 23, 2026 14:36
@muhammad-ali-e
muhammad-ali-e merged commit 74f72df into feat/UN-3445-pg-queue-aux-workers Jul 23, 2026
6 checks passed
@muhammad-ali-e
muhammad-ali-e deleted the fix/UN-3798-pg-pluggable-loader branch July 23, 2026 14:37
@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR prevents PostgreSQL pluggable workers from reloading task modules through an invalid bare file-path import.

  • Skips the redundant file-path task load for pluggable workers after package-based registration by WorkerBuilder.
  • Extracts WorkerType.to_directory() as the shared source for worker filesystem-directory naming.
  • Preserves file-path loading for non-pluggable workers and adds focused directory-mapping tests.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code-triggered failures identified.

Pluggable workers retain package-based shared-task registration through WorkerBuilder, while all existing non-pluggable worker directory names remain correctly resolved and loaded through the preserved file-path path.

Important Files Changed

Filename Overview
workers/worker.py Separates task loading by worker type, skipping the broken bare-module import for pluggable workers while preserving non-pluggable loading.
workers/shared/enums/worker_enums_base.py Extracts filesystem-directory resolution into to_directory() and keeps existing import-path behavior derived from it.
workers/tests/test_worker_enums_directory.py Verifies the hyphenated API-deployment mapping, normal passthrough behavior, and consistency between directory and import paths.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Build Celery app] --> B{Pluggable worker?}
    B -->|Yes| C[WorkerBuilder imports plugin worker package]
    C --> D[Plugin tasks register through package imports]
    D --> E[Skip bare file-path task load]
    B -->|No| F[Resolve WorkerType.to_directory]
    F --> G[Load tasks.py from worker directory]
    E --> H[Inspect application task registry]
    G --> H
Loading

Reviews (1): Last reviewed commit: "UN-3798 [FIX] Address #2197 review: accu..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant