Skip to content

Autogen: fix coverage landmines. #8174

Closed

Description

@tseaver

The unit_cov_level numbers in the various noxfile.py files are landmines (e.g., PR #8147 adds a # pragma: NO COVER and actually trips the check, because the total number of lines checked in BQDT drops!) The numbers don't represent any real goals (why 79% in one library or 95% in another?), but are merely "high water marks." The real goal is 100% coverage across all unit test runs, which we don't hit for many of the autogen-only libraries.

To address the first issue, I would argue that we should just drop the coverage checks inside the unit / default sessions.

To address the second issue, we need to apply a change like the one in #7413 which actually tests the "shim" modules for the autogen-only libraries (and excludes the namespace package files).

Both of these changes need to happen in the generator / templates (dropping the --cov-fail-under bit in the unit / default session, tweaking the .coveragerc, adding a testcase module for each generated shim module).

Activity

  1. added
    api: datastoreIssues related to the Datastore API.
    api: pubsubIssues related to the Pub/Sub API.
    api: bigtableIssues related to the Bigtable API.
    api: loggingIssues related to the Cloud Logging API.
    api: visionIssues related to the Cloud Vision API.
    api: monitoringIssues related to the Cloud Monitoring API.
    api: translationIssues related to the Cloud Translation API API.
    api: speechIssues related to the Speech-to-Text API.
    api: spannerIssues related to the Spanner API.
    api: languageIssues related to the Cloud Natural Language API API.
    api: cloudtraceIssues related to the Cloud Trace API.
    api: videointelligenceIssues related to the Video Intelligence API API.
    api: firestoreIssues related to the Firestore API.
    type: processA process-related concern. May include testing, release, or the like.
    on May 25, 2019
  2. 14 remaining items

  3. added
    api: iamIssues related to the Identity and Access Management API.
    api: webriskIssues related to the Web Risk API.
    api: datacatalogIssues related to the Data Catalog API.
    api: phishingprotectionIssues related to the Phishing Protection API.
    api: datalabelingIssues related to the AI Platform Data Labeling Service API.
    on May 25, 2019
  4. tseaver commented on May 28, 2019

    @tseaver
    ContributorAuthor

    @lukesneeringer asserts that the new gapic-generator-python will produce fully-tested code, including the shim modules.

  5. busunkim96 commented on Nov 3, 2021

    @busunkim96
    Contributor

    The generator produces tests which cover 100% of the autogen code. If coverage falls below 100% an issue should be opened: https://github.com/googleapis/gapic-generator-python/issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: automlIssues related to the AutoML API.api: bigquerydatatransferIssues related to the BigQuery Data Transfer Service API.api: bigtableIssues related to the Bigtable API.api: cloudassetIssues related to the Cloud Asset Inventory API.api: clouderrorreportingIssues related to the Error Reporting API.api: cloudiotIssues related to the IoT Core API.api: cloudkmsIssues related to the Cloud Key Management Service API.api: cloudschedulerIssues related to the Cloud Scheduler API.api: cloudtasksIssues related to the Cloud Tasks API.api: cloudtraceIssues related to the Cloud Trace API.api: containerIssues related to the Kubernetes Engine API API.api: datacatalogIssues related to the Data Catalog API.api: datalabelingIssues related to the AI Platform Data Labeling Service API.api: dataprocIssues related to the Managed Service for Apache Spark API.api: datastoreIssues related to the Datastore API.api: dlpIssues related to the Sensitive Data Protection API.api: firestoreIssues related to the Firestore API.api: iamIssues related to the Identity and Access Management API.api: irmapi: languageIssues related to the Cloud Natural Language API API.api: loggingIssues related to the Cloud Logging API.api: monitoringIssues related to the Cloud Monitoring API.api: osloginIssues related to the Cloud OS Login API API.api: phishingprotectionIssues related to the Phishing Protection API.api: pubsubIssues related to the Pub/Sub API.api: redisIssues related to the Memorystore for Redis API.api: securitycenterIssues related to the Security Command Center API.api: spannerIssues related to the Spanner API.api: speechIssues related to the Speech-to-Text API.api: talentapi: texttospeechIssues related to the Text-to-Speech API.api: translationIssues related to the Cloud Translation API API.api: videointelligenceIssues related to the Video Intelligence API API.api: visionIssues related to the Cloud Vision API.api: webriskIssues related to the Web Risk API.api: websecurityscannerIssues related to the Web Security Scanner API.type: processA process-related concern. May include testing, release, or the like.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions