Does the policy tag feature require adding the CI project to a list of allowed projects for the feature?

The second failure (in TestBigQuery.test_dbapi_connection_does_not_leak_sockets) is actually a "negative" leak (3 connections open at the start, 1 open at the end). I'm assuming GC cleaned up after the policy tag test failed, but while the leak test was running.

Does the policy tag feature require adding the CI project to a list of allowed projects for the feature?

If I understood correctly, then no, but apparently there's still some missing piece to make this work. @shollyman ?

The second failure (in TestBigQuery.test_dbapi_connection_does_not_leak_sockets) is actually a "negative" leak (3 connections open at the start, 1 open at the end). I'm assuming GC cleaned up after the policy tag test failed, but while the leak test was running.

It's the first time I'm seeing this, but this explanation sounds plausible. It's slightly worrying that the test is not completely isolated, but it to date it hasn't caused problems. One option would be to change the assertion to "less or equal", although that could also be masking the tests interference.

I'd say we keep an eye on it, but don't try to fix it, unless the flakiness becomes problematic.

I seem to be able to use the client without issue.

$ pip freeze
cachetools==4.2.2
certifi==2021.5.30
chardet==4.0.0
google-api-core==1.30.0
google-auth==1.31.0
google-cloud-datacatalog==3.2.1
googleapis-common-protos==1.53.0
grpc-google-iam-v1==0.12.3
grpcio==1.38.0
idna==2.10
libcst==0.3.19
mypy-extensions==0.4.3
packaging==20.9
proto-plus==1.18.1
protobuf==3.17.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyparsing==2.4.7
pytz==2021.1
PyYAML==5.4.1
requests==2.25.1
rsa==4.7.2
six==1.16.0
typing-extensions==3.10.0.0
typing-inspect==0.7.1
urllib3==1.26.5

sample code:

from google.cloud.datacatalog_v1 import PolicyTagManagerClient
from google.cloud.datacatalog_v1.types import Taxonomy

def main():

  client = PolicyTagManagerClient()


  new_taxonomy = Taxonomy(
                          display_name="Custom test taxonomy",
                          description="This taxonomy is ony used for a test.",
                          activated_policy_types=[Taxonomy.PolicyType.FINE_GRAINED_ACCESS_CONTROL])

  createresp = client.create_taxonomy(parent="projects/shollyman-demo-test/locations/us", taxonomy=new_taxonomy)

  resp = client.list_taxonomies(parent="projects/shollyman-demo-test/locations/us")

  for t in resp:
    print("{} -> {}".format(t.display_name, t.name))


if __name__ == "__main__":
  main()

invocation:

$ python catalogtest.py
Custom test taxonomy -> projects/shollyman-demo-test/locations/us/taxonomies/408338391262190617
Sample Taxonomy -> projects/shollyman-demo-test/locations/us/taxonomies/6484488580374782575
test2 -> projects/shollyman-demo-test/locations/us/taxonomies/6247855699646749172
testing -> projects/shollyman-demo-test/locations/us/taxonomies/3363827134841403621

Edit: Could it be a GCP project issue? This is the one thing that differs.

Generally, if the service isn't enabled it's generally clear about the error (and the text usually has a link to enable the service) as this is handled centrally. I don't appear to have access to the python test project with my personal credentials so I'm only getting permission denied responses when I attempt to use it.

and apparently I mangled Peter's previous comment somehow instead of quoting it in my own.

I see the full project path is used for the parent argument in the working sample, i.e. "projects/shollyman-demo-test/locations/us", while the test here uses just the project id.

If I change that to a full project path, I start getting a permission denied error, but that actually seems OK, as my test service account does not have any DataCatalog roles granted. After fixing that, things actually started working.

Is there any way to make the backend error message more informative?

Edit: Same result if providing a completely random non-existing project ID.

Edit 2: However, providing a project path of a non-existing project results in 403 Permission denied.

Edit 3: Providing a project without location, e.g. projects/project-foo also results in UNIMPLEMENTED.