OOTB generated python CF resource cannot be used with cfn test  #247

Activity

1. carpnick commented on Jan 17, 2023

   

   carpnick

   on Jan 17, 2023

   Author

   More actions

   Full trace:

   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/var/task/dd_dd_test1/handlers.py", line 3, in <module>
       from cloudformation_cli_python_lib import (
     File "/var/task/cloudformation_cli_python_lib/__init__.py", line 4, in <module>
       from .hook import Hook  # noqa: F401
     File "/var/task/cloudformation_cli_python_lib/hook.py", line 9, in <module>
       from .cipher import Cipher, KmsCipher
     File "/var/task/cloudformation_cli_python_lib/cipher.py", line 4, in <module>
       import aws_encryption_sdk  # type: ignore
     File "/var/task/aws_encryption_sdk/__init__.py", line 19, in <module>
       from aws_encryption_sdk.caches.local import LocalCryptoMaterialsCache  # noqa
     File "/var/task/aws_encryption_sdk/caches/__init__.py", line 25, in <module>
       from ..internal.formatting.encryption_context import serialize_encryption_context
     File "/var/task/aws_encryption_sdk/internal/formatting/__init__.py", line 14, in <module>
       from .serialize import serialize_header
     File "/var/task/aws_encryption_sdk/internal/formatting/serialize.py", line 17, in <module>
       import aws_encryption_sdk.internal.defaults
     File "/var/task/aws_encryption_sdk/internal/defaults.py", line 16, in <module>
       import aws_encryption_sdk.identifiers
     File "/var/task/aws_encryption_sdk/identifiers.py", line 18, in <module>
       from cryptography.hazmat.primitives.asymmetric import ec, padding, rsa
     File "/var/task/cryptography/hazmat/primitives/asymmetric/ec.py", line 10, in <module>
       from cryptography.hazmat._oid import ObjectIdentifier
     File "/var/task/cryptography/hazmat/_oid.py", line 7, in <module>
       from cryptography.hazmat.bindings._rust import (
   ImportError: cannot import name 'ObjectIdentifier' from 'cryptography.hazmat.bindings._rust' (unknown location)
   
   
2. carpnick commented on Jan 17, 2023

   

   carpnick

   on Jan 17, 2023

   Last edited by carpnick

   Author

   More actions

   Found out issue:

   1st issue:
   Template assumes X86. If you are on an Apple Mac M1, you have to modify the SAM template to use arm64 using the Architectures property. User error.

   Enhancement Request - With cfn init ask user architecture and support arm64 and x86/AMD64 and modify templating of new resources as required.

   2nd issue:
   Fundamentally we have a broken workflow I think. Any python package that requires native install cannot support the copysource workflow that SAM supports since it doesnt copy over all files. It excludes certain files we need for native packages. Broken because of this line more than likely.

   Confirmed:

   2023-01-17 20:04:34,335 | File (_padding.abi3.so) is in ignored set, skipping it
   2023-01-17 20:04:34,335 | File (_openssl.abi3.so) is in ignored set, skipping it
   
   

   This fundamentally breaks native zipping with SAM CLI to allow contract testing locally. Posting workaround below.

   AWS team - is there any plans to make it so we can ship native docker container images as an integration instead? If not, how will we overcome the native dependency install? SAM CLI Change? aws-lambda-builders change? In this case we cannot say not supported since cryptography is in the cloudformation-cli-python-plugin dependency tree, let alone customers having their own dependencies.

   As I see it - this completely blocks cfn test use cases running locally with sam local start-lambda using the documented way of running sam build and the templated default that comes from cfn init.

   ---

   ---

   Complete workaround:

   • run cfn submit --dry-run
   • Change generated template CodeUri line to CodeUri: src/. This allows the sam build call to rebuild for the SAM context.
   • instead of running sam build run sam build -u -m requirements.txt
   • This will reinstall the requirements but within the SAM context.
   • It also still copies the source over.
   • This is more of a hack workaround workflow to get it working properly. It was over 2 days of effort to get to the point I could run the hello world contract tests for python locally and start making code changes.
3. mmaeng

   added

   investigating

   on Jan 19, 2023