Fix RESOURCE_ASSET compatibility with Airflow 2.x in common-compat#64933
Fix RESOURCE_ASSET compatibility with Airflow 2.x in common-compat#64933jedcunningham merged 4 commits intoapache:mainfrom
Conversation
PR apache#63335 hardcoded RESOURCE_ASSET = "Assets" in this compat module, which broke Airflow 2.x deployments. In Airflow 2.x, the equivalent resource is named "Datasets" (RESOURCE_DATASET), not "Assets". apache-airflow-providers-fab imports RESOURCE_ASSET from this module at runtime (via the `else` branch of an `if TYPE_CHECKING` block). When RESOURCE_ASSET resolves to "Assets" instead of "Datasets", it creates duplicate "Assets" and "Datasets" resource types for upgraded instances. And worse "Assets", which dont even exist in AF2, are used for auth checks. This also breaks any custom roles. Fix: use AIRFLOW_V_3_0_PLUS to return the correct value for each version, falling back to RESOURCE_DATASET from airflow.security.permissions (which is not deprecated in Airflow 2.x) for the Airflow 2.x case. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
I guess that means rc2 for common.compat and the two providers that depend on it's new version from the current vote. Thanks for finding and fixing! |
|
BTW. I do think that the Also we will have to find out why this was not detected by compatibility tests - it should have been, but possibly this kind of issue was masked by something. To be diagnosed. |
|
I suspect it wasn't found in the compatibility tests because it caused "Assets" resources and permissions to be created alongside the "Dataset" ones that should have been used. And if you didn't have a custom role (or maybe modified the existing roles) without the dataset perms, its hidden because it "just works", even if its fundamentally wrong. |
Suppress attr-defined (RESOURCE_DATASET doesn't exist in Airflow 3 stubs, where mypy runs) and no-redef (RESOURCE_ASSET defined in both if/else branches) on the Airflow 2.x fallback import. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
An example after the upgrade (all default roles get this) (this is before this fix of course): 
|
|
There is a mypy issue for providers :( |
There was a problem hiding this comment.
Pull request overview
Fixes a compatibility regression in the common-compat provider where RESOURCE_ASSET was hardcoded to the Airflow 3 resource name, which breaks permission/role behavior on Airflow 2.x.
Changes:
- Introduces
AIRFLOW_V_3_0_PLUSgating to setRESOURCE_ASSETto"Assets"on Airflow 3. - For Airflow 2.x, maps
RESOURCE_ASSETback toairflow.security.permissions.RESOURCE_DATASETto preserve the"Datasets"permission resource name.
Replace the `from ... import RESOURCE_DATASET as RESOURCE_ASSET` form with a module import + attribute assignment. The `as`-alias import form triggers mypy `no-redef` when the name is already defined in the if-branch; a plain attribute access does not. Only `# type: ignore[attr-defined]` is needed since Airflow 3.x stubs don't expose `RESOURCE_DATASET`.
Summary
Fixes a regression introduced by #63335, which hardcoded
RESOURCE_ASSET = "Assets"in
providers/common/compat/security/permissions.py. This value is correct forAirflow 3, but breaks Airflow 2.x in two ways:
Wrong resource name in Airflow 2.x.
apache-airflow-providers-fabimportsRESOURCE_ASSETfrom this module at runtime. FAB's role sync updates stock rolesto grant the
"Assets"resource, but the correct Airflow 2.x name is"Datasets".This creates an inconsistency and pollutes the permission model with a resource that
doesn't belong in Airflow 2.x.
Custom roles break. Any role with
"Datasets"permissions stops working, sinceauth checks now look for
"Assets". While you could re-grant permissions against"Assets"after upgrading, existing roles are silently broken.Fix
Gate on
AIRFLOW_V_3_0_PLUS: return"Assets"for Airflow 3, and fall back toRESOURCE_DATASETfromairflow.security.permissions(not deprecated in Airflow 2.x)for Airflow 2.x.
Related
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code