Skip to content

Dataflow monitoring by core is failing with incorrect job id #153

Description

@tims

Expected Behavior

Dataflow should be monitored for metrics and status by Core

Current Behavior

The monitoring check is failing, because it is looking up the the wrong job id:

{"timeMillis":1551867055865,"thread":"pool-4-thread-1","level":"ERROR","loggerName":"feast.core.job.dataflow.DataflowJobMonitor","message":"Unable to retrieve metrics for job with id: 2019-03-06_01_41_55-12356597275160718110\ncause: com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found\n{\n  \"code\" : 404,\n  \"errors\" : [ {\n    \"domain\" : \"global\",\n    \"message\" : \"(7d9ea65839310b88): Information about job 2019-03-06_01_41_55-12356597275160718110 could not be found in our system. Please double check the id is correct. If it is please contact customer support.\",\n    \"reason\" : \"notFound\"\n  } ],\n  \"message\" : \"(7d9ea65839310b88): Information about job 2019-03-06_01_41_55-12356597275160718110 could not be found in our system. Please double check the id is correct. If it is please contact customer support.\",\n  \"status\" : \"NOT_FOUND\"\n}","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":29,"threadPriority":5}

The job name that dataflow UI is reporting is: feastimport1551865294794

Steps to reproduce

Start a dataflow job.

Possible Solution

This is likely related to PR #147

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions