Skip to content

Commit 0bf3b68

Browse files
dhermesdhermes
authored
Merge pull request #2151 from dhermes/fix-2102
Merging logging-stdlib-handler-feature branch back into master
2 parents 20eea35 + c9c761d commit 0bf3b68

21 files changed

Lines changed: 1048 additions & 10 deletions

‎CONTRIBUTING.rst‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,8 @@ Running System Tests
160160
can be downloaded directly from the developer's console by clicking
161161
"Generate new JSON key". See private key
162162
`docs <https://cloud.google.com/storage/docs/authentication#generating-a-private-key>`__
163-
for more details.
163+
for more details. In order for Logging system tests to work, the Service Account
164+
will also have to be made a project Owner. This can be changed under "IAM & Admin".
164165

165166
- Examples of these can be found in ``system_tests/local_test_setup.sample``. We
166167
recommend copying this to ``system_tests/local_test_setup``, editing the

‎docs/index.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,10 @@
106106
logging-entries
107107
logging-metric
108108
logging-sink
109+
logging-handlers
110+
logging-transports-sync
111+
logging-transports-thread
112+
logging-transports-base
109113

110114
.. toctree::
111115
:maxdepth: 0

‎docs/logging-handlers.rst‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Python Logging Module Handler
2+
==============================
3+
4+
.. automodule:: gcloud.logging.handlers.handlers
5+
:members:
6+
:show-inheritance:

‎docs/logging-transports-base.rst‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Python Logging Handler Sync Transport
2+
======================================
3+
4+
.. automodule:: gcloud.logging.handlers.transports.base
5+
:members:
6+
:show-inheritance:

‎docs/logging-transports-sync.rst‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Python Logging Handler Sync Transport
2+
======================================
3+
4+
.. automodule:: gcloud.logging.handlers.transports.sync
5+
:members:
6+
:show-inheritance:

‎docs/logging-transports-thread.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Python Logging Handler Threaded Transport
2+
=========================================
3+
4+
5+
.. automodule:: gcloud.logging.handlers.transports.background_thread
6+
:members:
7+
:show-inheritance:

‎docs/logging-usage.rst‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -383,3 +383,74 @@ Delete a sink:
383383
>>> sink.delete() # API call
384384
>>> sink.exists() # API call
385385
False
386+
387+
Integration with Python logging module
388+
---------------------------------------------
389+
390+
391+
It's possible to tie the Python :mod:`logging` module directly into Google Cloud Logging. To use it,
392+
create a :class:`CloudLoggingHandler <gcloud.logging.CloudLoggingHandler>` instance from your
393+
Logging client.
394+
395+
.. doctest::
396+
397+
>>> import logging
398+
>>> import gcloud.logging # Don't conflict with standard logging
399+
>>> from gcloud.logging.handlers import CloudLoggingHandler
400+
>>> client = gcloud.logging.Client()
401+
>>> handler = CloudLoggingHandler(client)
402+
>>> cloud_logger = logging.getLogger('cloudLogger')
403+
>>> cloud_logger.setLevel(logging.INFO) # defaults to WARN
404+
>>> cloud_logger.addHandler(handler)
405+
>>> cloud_logger.error('bad news')
406+
407+
.. note::
408+
409+
This handler by default uses an asynchronous transport that sends log entries on a background
410+
thread. However, the API call will still be made in the same process. For other transport
411+
options, see the transports section.
412+
413+
All logs will go to a single custom log, which defaults to "python". The name of the Python
414+
logger will be included in the structured log entry under the "python_logger" field. You can
415+
change it by providing a name to the handler:
416+
417+
.. doctest::
418+
419+
>>> handler = CloudLoggingHandler(client, name="mycustomlog")
420+
421+
It is also possible to attach the handler to the root Python logger, so that for example a plain
422+
`logging.warn` call would be sent to Cloud Logging, as well as any other loggers created. However,
423+
you must avoid infinite recursion from the logging calls the client itself makes. A helper
424+
method :meth:`setup_logging <gcloud.logging.handlers.setup_logging>` is provided to configure
425+
this automatically:
426+
427+
.. doctest::
428+
429+
>>> import logging
430+
>>> import gcloud.logging # Don't conflict with standard logging
431+
>>> from gcloud.logging.handlers import CloudLoggingHandler, setup_logging
432+
>>> client = gcloud.logging.Client()
433+
>>> handler = CloudLoggingHandler(client)
434+
>>> logging.getLogger().setLevel(logging.INFO) # defaults to WARN
435+
>>> setup_logging(handler)
436+
>>> logging.error('bad news')
437+
438+
You can also exclude certain loggers:
439+
440+
.. doctest::
441+
442+
>>> setup_logging(handler, excluded_loggers=('werkzeug',)))
443+
444+
445+
446+
Python logging handler transports
447+
==================================
448+
449+
The Python logging handler can use different transports. The default is
450+
:class:`gcloud.logging.handlers.BackgroundThreadTransport`.
451+
452+
1. :class:`gcloud.logging.handlers.BackgroundThreadTransport` this is the default. It writes
453+
entries on a background :class:`python.threading.Thread`.
454+
455+
1. :class:`gcloud.logging.handlers.SyncTransport` this handler does a direct API call on each
456+
logging statement to write the entry.

‎gcloud/logging/handlers/__init__.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Copyright 2016 Google Inc. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Python :mod:`logging` handlers for Google Cloud Logging."""
16+
17+
from gcloud.logging.handlers.handlers import CloudLoggingHandler
18+
from gcloud.logging.handlers.handlers import setup_logging

‎gcloud/logging/handlers/handlers.py‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
# Copyright 2016 Google Inc. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Python :mod:`logging` handlers for Google Cloud Logging."""
16+
17+
import logging
18+
19+
from gcloud.logging.handlers.transports import BackgroundThreadTransport
20+
21+
22+
EXCLUDE_LOGGER_DEFAULTS = (
23+
'gcloud',
24+
'oauth2client'
25+
)
26+
27+
DEFAULT_LOGGER_NAME = 'python'
28+
29+
30+
class CloudLoggingHandler(logging.StreamHandler):
31+
"""Python standard ``logging`` handler.
32+
33+
This handler can be used to route Python standard logging messages
34+
directly to the Google Cloud Logging API.
35+
36+
Note that this handler currently only supports a synchronous API call,
37+
which means each logging statement that uses this handler will require
38+
an API call.
39+
40+
:type client: :class:`gcloud.logging.client`
41+
:param client: the authenticated gcloud logging client for this handler
42+
to use
43+
44+
:type name: str
45+
:param name: the name of the custom log in Stackdriver Logging. Defaults
46+
to 'python'. The name of the Python logger will be represented
47+
in the ``python_logger`` field.
48+
49+
:type transport: type
50+
:param transport: Class for creating new transport objects. It should
51+
extend from the base :class:`.Transport` type and
52+
implement :meth`.Transport.send`. Defaults to
53+
:class:`.BackgroundThreadTransport`. The other
54+
option is :class:`.SyncTransport`.
55+
56+
Example:
57+
58+
.. doctest::
59+
60+
import gcloud.logging
61+
from gcloud.logging.handlers import CloudLoggingHandler
62+
63+
client = gcloud.logging.Client()
64+
handler = CloudLoggingHandler(client)
65+
66+
cloud_logger = logging.getLogger('cloudLogger')
67+
cloud_logger.setLevel(logging.INFO)
68+
cloud_logger.addHandler(handler)
69+
70+
cloud.logger.error('bad news') # API call
71+
72+
"""
73+
74+
def __init__(self, client,
75+
name=DEFAULT_LOGGER_NAME,
76+
transport=BackgroundThreadTransport):
77+
super(CloudLoggingHandler, self).__init__()
78+
self.name = name
79+
self.client = client
80+
self.transport = transport(client, name)
81+
82+
def emit(self, record):
83+
"""Actually log the specified logging record.
84+
85+
Overrides the default emit behavior of ``StreamHandler``.
86+
87+
See: https://docs.python.org/2/library/logging.html#handler-objects
88+
89+
:type record: :class:`logging.LogRecord`
90+
:param record: The record to be logged.
91+
"""
92+
message = super(CloudLoggingHandler, self).format(record)
93+
self.transport.send(record, message)
94+
95+
96+
def setup_logging(handler, excluded_loggers=EXCLUDE_LOGGER_DEFAULTS):
97+
"""Attach the ``CloudLogging`` handler to the Python root logger
98+
99+
Excludes loggers that this library itself uses to avoid
100+
infinite recursion.
101+
102+
:type handler: :class:`logging.handler`
103+
:param handler: the handler to attach to the global handler
104+
105+
:type excluded_loggers: tuple
106+
:param excluded_loggers: The loggers to not attach the handler to. This
107+
will always include the loggers in the path of
108+
the logging client itself.
109+
110+
Example:
111+
112+
.. doctest::
113+
114+
import logging
115+
import gcloud.logging
116+
from gcloud.logging.handlers import CloudLoggingHandler
117+
118+
client = gcloud.logging.Client()
119+
handler = CloudLoggingHandler(client)
120+
gcloud.logging.setup_logging(handler)
121+
logging.getLogger().setLevel(logging.DEBUG)
122+
123+
logging.error('bad news') # API call
124+
125+
"""
126+
all_excluded_loggers = set(excluded_loggers + EXCLUDE_LOGGER_DEFAULTS)
127+
logger = logging.getLogger()
128+
logger.addHandler(handler)
129+
logger.addHandler(logging.StreamHandler())
130+
for logger_name in all_excluded_loggers:
131+
logger = logging.getLogger(logger_name)
132+
logger.propagate = False
133+
logger.addHandler(logging.StreamHandler())

0 commit comments

Comments
 (0)