|
| 1 | +# Audit Logging |
| 2 | + |
| 3 | +### Introduction |
| 4 | + |
| 5 | +Feast provides audit logging functionality in order to debug problems and to trace the lineage of events. |
| 6 | + |
| 7 | +### Audit Log Types |
| 8 | + |
| 9 | +Audit Logs produced by Feast come in three favors: |
| 10 | + |
| 11 | +| Audit Log Type | Description | |
| 12 | +| :--- | :--- | |
| 13 | +| Message Audit Log | Logs service calls that can be used to track Feast request handling. Currently only gRPC request/response is supported. Enabling Message Audit Logs can be resource intensive and significantly increase latency, as such is not recommended on Online Serving. | |
| 14 | +| Transition Audit Log | Logs transitions in status in resources managed by Feast \(ie an Ingestion Job becoming RUNNING\). | |
| 15 | +| Action Audit Log | Logs actions performed on a specific resource managed by Feast \(ie an Ingestion Job is aborted\). | |
| 16 | + |
| 17 | +### Configuration |
| 18 | + |
| 19 | +| Audit Log Type | Description | |
| 20 | +| :--- | :--- | |
| 21 | +| Message Audit Log | Enabled when both `feast.logging.audit.enabled` and `feast.logging.audit.messageLogging.enabled` is set to `true` | |
| 22 | +| Transition Audit Log | Enabled when `feast.logging.audit.enabled` is set to `true` | |
| 23 | +| Action Audit Log | Enabled when `feast.logging.audit.enabled` is set to `true` | |
| 24 | + |
| 25 | +### JSON Format |
| 26 | + |
| 27 | +Audit Logs produced by Feast are written to the console similar to normal logs but in a structured, machine parsable JSON. Example of a Message Audit Log JSON entry produced: |
| 28 | + |
| 29 | +```text |
| 30 | +{ |
| 31 | + "message": { |
| 32 | + "logType": "FeastAuditLogEntry", |
| 33 | + "kind": "MESSAGE", |
| 34 | + "statusCode": "OK", |
| 35 | + "request": { |
| 36 | + "filter": { |
| 37 | + "project": "dummy", |
| 38 | + "featureSetName": "*" |
| 39 | + } |
| 40 | + }, |
| 41 | + "application": "Feast", |
| 42 | + "response": {}, |
| 43 | + "method": "ListFeatureSets", |
| 44 | + "identity": "105960238928959148073", |
| 45 | + "service": "CoreService", |
| 46 | + "component": "feast-core", |
| 47 | + "id": "45329ea9-0d48-46c5-b659-4604f6193711", |
| 48 | + "version": "0.6.2" |
| 49 | + }, |
| 50 | + "hostname": "feast.core" |
| 51 | + "timestamp": "2020-08-16T04:45:24Z", |
| 52 | + "severity": "INFO", |
| 53 | +} |
| 54 | +``` |
| 55 | + |
| 56 | +### Log Entry Schema |
| 57 | + |
| 58 | +Fields common to all Audit Log Types: |
| 59 | + |
| 60 | +| Field | Description | |
| 61 | +| :--- | :--- | |
| 62 | +| `logType` | Log Type. Always set to `FeastAuditLogEntry`. Useful for filtering out Feast audit logs. | |
| 63 | +| `application` | Application. Always set to `Feast`. | |
| 64 | +| `component` | Feast Component producing the Audit Log. Set to `feast-core` for Feast Core and `feast-serving` for Feast Serving. Use to filtering out Audit Logs by component. | |
| 65 | +| `version` | Version of Feast producing this Audit Log. Use to filtering out Audit Logs by version. | |
| 66 | + |
| 67 | +Fields in Message Audit Log Type |
| 68 | + |
| 69 | +| Field | Description | |
| 70 | +| :--- | :--- | |
| 71 | +| `id` | Generated UUID that uniquely identifies the service call. | |
| 72 | +| `service` | Name of the Service that handled the service call. | |
| 73 | +| `method` | Name of the Method that handled the service call. Useful for filtering Audit Logs by method \(ie `ApplyFeatureSet` calls\) | |
| 74 | +| `request` | Full request submitted by client in the service call as JSON. | |
| 75 | +| `response` | Full response returned to client by the service after handling the service call as JSON. | |
| 76 | +| `identity` | Identity of the client making the service call as an user Id. Only set when Authentication is enabled. | |
| 77 | +| `statusCode` | The status code returned by the service handling the service call \(ie `OK` if service call handled without error\). | |
| 78 | + |
| 79 | +Fields in Action Audit Log Type |
| 80 | + |
| 81 | +| Field | Description | |
| 82 | +| :--- | :--- | |
| 83 | +| `action` | Name of the action taken on the resource. | |
| 84 | +| `resource.type` | Type of resource of which the action was taken on \(ie `FEATURE_SET`\) | |
| 85 | +| resource.id | Identifier specifying the specific resource of which the action was taken on. | |
| 86 | + |
| 87 | +Fields in Transition Audit Log Type |
| 88 | + |
| 89 | +| Field | Description | |
| 90 | +| :--- | :--- | |
| 91 | +| `status` | The new status that the resource transitioned to | |
| 92 | +| `resource.type` | Type of resource of which the transition occurred \(ie `FeatureSet`\) | |
| 93 | +| `resource.id` | Identifier specifying the specific resource of which the transition occurred. | |
| 94 | + |
| 95 | +### Log Forwarder |
| 96 | + |
| 97 | +Feast currently only supports forwarding Request/Response \(Message Audit Log Type\) logs to an external fluentD service with `feast.**` Fluentd tag. |
| 98 | + |
| 99 | +#### Configuration |
| 100 | + |
| 101 | +The Fluentd Log Forwarder configured with the with the following configuration options in `application.yml`: |
| 102 | + |
| 103 | +| Settings | Description | |
| 104 | +| :--- | :--- | |
| 105 | +| `feast.logging.audit.messageLogging.destination` | `fluentd` | |
| 106 | +| `feast.logging.audit.messageLogging.fluentdHost` | `localhost` | |
| 107 | +| `feast.logging.audit.messageLogging.fluentdPort` | `24224` | |
| 108 | + |
0 commit comments