Migrate IAM AWS managed policies#13836
Conversation
d914d4a to
d315aa5
Compare
LocalStack Community integration with Pro 2 files ± 0 2 suites ±0 2m 47s ⏱️ - 2h 8m 32s For more details on these failures, see this check. Results for commit 940032f. ± Comparison against base commit daa78fb. ♻️ This comment has been updated with latest results. |
Test Results (amd64) - Integration, Bootstrap 5 files ± 0 5 suites ±0 11m 33s ⏱️ - 2h 30m 11s For more details on these failures, see this check. Results for commit 940032f. ± Comparison against base commit daa78fb. ♻️ This comment has been updated with latest results. |
| def _generate_aws_managed_policy_id(self, name: str) -> str: | ||
| """Generate a deterministic, stable PolicyId for an AWS managed policy. | ||
|
|
||
| The format mirrors real AWS IDs (``ANPA`` + 17 upper-hex chars). The value | ||
| is derived from a SHA-256 hash of the policy name so it is consistent across | ||
| restarts without needing to be persisted. | ||
| """ | ||
| hash_hex = hashlib.sha256(name.encode()).hexdigest()[:17].upper() | ||
| return f"ANPA{hash_hex}" |
There was a problem hiding this comment.
suggestion: we could add this to the utils file.
There was a problem hiding this comment.
Same as with the other PR - let's do them all at once!
dc8bb42 to
f890220
Compare
Motivation
With this PR, we are migrating AWS managed IAM policies.
This does not yet include an automatic github action to update those policies.
Changes
Static data, not persisted in the store
AWS managed policies are fixed and identical for every account. Persisting them in the store would waste space and add noise to state snapshots, and we would not easily be able to update it with new LocalStack versions. Instead:
aws_managed_policies.json) copied from moto's dataset — a JSON file containing ~1,400 policies keyed by policy name.provider.pyparses the JSON once and builds a policy cache, a dict keyed by normalized ARN (arn:aws:iam::aws:policy/<Path><Name>).Attachment counts are the only per-account state
The only account-specific data for AWS managed policies is how many principals have them attached. This is stored in
IamStore.AWS_MANAGED_POLICIES— aCrossRegionAttributedict mapping non-normalized ARN →AwsManagedPolicy(attachment_count). A key is only present whenattachment_count > 0; it is removed when the count drops back to zero.ARN normalization
AWS managed policy ARNs can use any partition (
aws,aws-cn,aws-us-gov). The static index uses theawspartition. All lookups normalize the incoming ARN toarn:aws:iam::aws:policy/...before consulting the cache (but not the in-store attributes), so China and GovCloud work transparently.Policy IDs are deterministic
Real AWS managed policy IDs (e.g.
ANPAIWMBCKSKIEE64ZLYK) are not in moto's dataset. LocalStack generates a stable, deterministic ID for each policy:ANPA+ the first 17 hex characters ofSHA-256(policy_name). The value is consistent across restarts without needing to be persisted. In the future we can copy those IDs from AWS as well.Files Changed
localstack/services/iam/aws_managed_policies.jsonmoto/iam/aws_managed_policies.pylocalstack/services/iam/models.pyAwsManagedPolicydataclass; addedAWS_MANAGED_POLICIESfield toIamStorelocalstack/services/iam/provider.pyget_policy,get_policy_version,list_policies,attach_role_policy,detach_role_policy,attach_user_policy,detach_user_policy,_validate_permissions_boundarytests/aws/services/iam/test_iam_managed_policies.pypytestmark = pytest.mark.skipData Limitations
The static dataset is sourced from moto and reflects AWS policy versions at the time moto last updated its data. Policies may have been updated in real AWS since then (e.g. a policy at
DefaultVersionId: "v8"locally may be at"v12"in real AWS). Only the latest version document is available; older historical versions cannot be retrieved.Tests
All tests for managed policies are enabled and passing
Related
Closes UNC-273