|
| 1 | +Python Client for Dataproc Metastore API |
| 2 | +======================================== |
| 3 | + |
| 4 | +|stable| |pypi| |versions| |
| 5 | + |
| 6 | +`Dataproc Metastore API`_: The Dataproc Metastore API is used to manage the lifecycle and configuration of metastore services. |
| 7 | + |
| 8 | +- `Client Library Documentation`_ |
| 9 | +- `Product Documentation`_ |
| 10 | + |
| 11 | +.. |stable| image:: https://img.shields.io/badge/support-stable-gold.svg |
| 12 | + :target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#stability-levels |
| 13 | +.. |pypi| image:: https://img.shields.io/pypi/v/google-cloud-dataproc-metastore.svg |
| 14 | + :target: https://pypi.org/project/google-cloud-dataproc-metastore/ |
| 15 | +.. |versions| image:: https://img.shields.io/pypi/pyversions/google-cloud-dataproc-metastore.svg |
| 16 | + :target: https://pypi.org/project/google-cloud-dataproc-metastore/ |
| 17 | +.. _Dataproc Metastore API: https://cloud.google.com/dataproc-metastore/ |
| 18 | +.. _Client Library Documentation: https://cloud.google.com/python/docs/reference/metastore/latest/summary_overview |
| 19 | +.. _Product Documentation: https://cloud.google.com/dataproc-metastore/ |
| 20 | + |
| 21 | +Quick Start |
| 22 | +----------- |
| 23 | + |
| 24 | +In order to use this library, you first need to go through the following steps: |
| 25 | + |
| 26 | +1. `Select or create a Cloud Platform project.`_ |
| 27 | +2. `Enable billing for your project.`_ |
| 28 | +3. `Enable the Dataproc Metastore API.`_ |
| 29 | +4. `Set up Authentication.`_ |
| 30 | + |
| 31 | +.. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project |
| 32 | +.. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project |
| 33 | +.. _Enable the Dataproc Metastore API.: https://cloud.google.com/dataproc-metastore/ |
| 34 | +.. _Set up Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html |
| 35 | + |
| 36 | +Installation |
| 37 | +~~~~~~~~~~~~ |
| 38 | + |
| 39 | +Install this library in a virtual environment using `venv`_. `venv`_ is a tool that |
| 40 | +creates isolated Python environments. These isolated environments can have separate |
| 41 | +versions of Python packages, which allows you to isolate one project's dependencies |
| 42 | +from the dependencies of other projects. |
| 43 | + |
| 44 | +With `venv`_, it's possible to install this library without needing system |
| 45 | +install permissions, and without clashing with the installed system |
| 46 | +dependencies. |
| 47 | + |
| 48 | +.. _`venv`: https://docs.python.org/3/library/venv.html |
| 49 | + |
| 50 | + |
| 51 | +Code samples and snippets |
| 52 | +~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 53 | + |
| 54 | +Code samples and snippets live in the `samples/`_ folder. |
| 55 | + |
| 56 | +.. _samples/: https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-dataproc-metastore/samples |
| 57 | + |
| 58 | + |
| 59 | +Supported Python Versions |
| 60 | +^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 61 | +Our client libraries are compatible with all current `active`_ and `maintenance`_ versions of |
| 62 | +Python. |
| 63 | + |
| 64 | +Python >= 3.7 |
| 65 | + |
| 66 | +.. _active: https://devguide.python.org/devcycle/#in-development-main-branch |
| 67 | +.. _maintenance: https://devguide.python.org/devcycle/#maintenance-branches |
| 68 | + |
| 69 | +Unsupported Python Versions |
| 70 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 71 | +Python <= 3.6 |
| 72 | + |
| 73 | +If you are using an `end-of-life`_ |
| 74 | +version of Python, we recommend that you update as soon as possible to an actively supported version. |
| 75 | + |
| 76 | +.. _end-of-life: https://devguide.python.org/devcycle/#end-of-life-branches |
| 77 | + |
| 78 | +Mac/Linux |
| 79 | +^^^^^^^^^ |
| 80 | + |
| 81 | +.. code-block:: console |
| 82 | +
|
| 83 | + python3 -m venv <your-env> |
| 84 | + source <your-env>/bin/activate |
| 85 | + pip install google-cloud-dataproc-metastore |
| 86 | +
|
| 87 | +
|
| 88 | +Windows |
| 89 | +^^^^^^^ |
| 90 | + |
| 91 | +.. code-block:: console |
| 92 | +
|
| 93 | + py -m venv <your-env> |
| 94 | + .\<your-env>\Scripts\activate |
| 95 | + pip install google-cloud-dataproc-metastore |
| 96 | +
|
| 97 | +Next Steps |
| 98 | +~~~~~~~~~~ |
| 99 | + |
| 100 | +- Read the `Client Library Documentation`_ for Dataproc Metastore API |
| 101 | + to see other available methods on the client. |
| 102 | +- Read the `Dataproc Metastore API Product documentation`_ to learn |
| 103 | + more about the product and see How-to Guides. |
| 104 | +- View this `README`_ to see the full list of Cloud |
| 105 | + APIs that we cover. |
| 106 | + |
| 107 | +.. _Dataproc Metastore API Product documentation: https://cloud.google.com/dataproc-metastore/ |
| 108 | +.. _README: https://github.com/googleapis/google-cloud-python/blob/main/README.rst |
| 109 | + |
| 110 | +Logging |
| 111 | +------- |
| 112 | + |
| 113 | +This library uses the standard Python :code:`logging` functionality to log some RPC events that could be of interest for debugging and monitoring purposes. |
| 114 | +Note the following: |
| 115 | + |
| 116 | +#. Logs may contain sensitive information. Take care to **restrict access to the logs** if they are saved, whether it be on local storage or on Google Cloud Logging. |
| 117 | +#. Google may refine the occurrence, level, and content of various log messages in this library without flagging such changes as breaking. **Do not depend on immutability of the logging events**. |
| 118 | +#. By default, the logging events from this library are not handled. You must **explicitly configure log handling** using one of the mechanisms below. |
| 119 | + |
| 120 | +Simple, environment-based configuration |
| 121 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 122 | + |
| 123 | +To enable logging for this library without any changes in your code, set the :code:`GOOGLE_SDK_PYTHON_LOGGING_SCOPE` environment variable to a valid Google |
| 124 | +logging scope. This configures handling of logging events (at level :code:`logging.DEBUG` or higher) from this library in a default manner, emitting the logged |
| 125 | +messages in a structured format. It does not currently allow customizing the logging levels captured nor the handlers, formatters, etc. used for any logging |
| 126 | +event. |
| 127 | + |
| 128 | +A logging scope is a period-separated namespace that begins with :code:`google`, identifying the Python module or package to log. |
| 129 | + |
| 130 | +- Valid logging scopes: :code:`google`, :code:`google.cloud.asset.v1`, :code:`google.api`, :code:`google.auth`, etc. |
| 131 | +- Invalid logging scopes: :code:`foo`, :code:`123`, etc. |
| 132 | + |
| 133 | +**NOTE**: If the logging scope is invalid, the library does not set up any logging handlers. |
| 134 | + |
| 135 | +Environment-Based Examples |
| 136 | +^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 137 | + |
| 138 | +- Enabling the default handler for all Google-based loggers |
| 139 | + |
| 140 | +.. code-block:: console |
| 141 | +
|
| 142 | + export GOOGLE_SDK_PYTHON_LOGGING_SCOPE=google |
| 143 | +
|
| 144 | +- Enabling the default handler for a specific Google module (for a client library called :code:`library_v1`): |
| 145 | + |
| 146 | +.. code-block:: console |
| 147 | +
|
| 148 | + export GOOGLE_SDK_PYTHON_LOGGING_SCOPE=google.cloud.library_v1 |
| 149 | +
|
| 150 | +
|
| 151 | +Advanced, code-based configuration |
| 152 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 153 | + |
| 154 | +You can also configure a valid logging scope using Python's standard `logging` mechanism. |
| 155 | + |
| 156 | +Code-Based Examples |
| 157 | +^^^^^^^^^^^^^^^^^^^ |
| 158 | + |
| 159 | +- Configuring a handler for all Google-based loggers |
| 160 | + |
| 161 | +.. code-block:: python |
| 162 | +
|
| 163 | + import logging |
| 164 | +
|
| 165 | + from google.cloud import library_v1 |
| 166 | +
|
| 167 | + base_logger = logging.getLogger("google") |
| 168 | + base_logger.addHandler(logging.StreamHandler()) |
| 169 | + base_logger.setLevel(logging.DEBUG) |
| 170 | +
|
| 171 | +- Configuring a handler for a specific Google module (for a client library called :code:`library_v1`): |
| 172 | + |
| 173 | +.. code-block:: python |
| 174 | +
|
| 175 | + import logging |
| 176 | +
|
| 177 | + from google.cloud import library_v1 |
| 178 | +
|
| 179 | + base_logger = logging.getLogger("google.cloud.library_v1") |
| 180 | + base_logger.addHandler(logging.StreamHandler()) |
| 181 | + base_logger.setLevel(logging.DEBUG) |
| 182 | +
|
| 183 | +Logging details |
| 184 | +~~~~~~~~~~~~~~~ |
| 185 | + |
| 186 | +#. Regardless of which of the mechanisms above you use to configure logging for this library, by default logging events are not propagated up to the root |
| 187 | + logger from the `google`-level logger. If you need the events to be propagated to the root logger, you must explicitly set |
| 188 | + :code:`logging.getLogger("google").propagate = True` in your code. |
| 189 | +#. You can mix the different logging configurations above for different Google modules. For example, you may want use a code-based logging configuration for |
| 190 | + one library, but decide you need to also set up environment-based logging configuration for another library. |
| 191 | + |
| 192 | + #. If you attempt to use both code-based and environment-based configuration for the same module, the environment-based configuration will be ineffectual |
| 193 | + if the code -based configuration gets applied first. |
| 194 | + |
| 195 | +#. The Google-specific logging configurations (default handlers for environment-based configuration; not propagating logging events to the root logger) get |
| 196 | + executed the first time *any* client library is instantiated in your application, and only if the affected loggers have not been previously configured. |
| 197 | + (This is the reason for 2.i. above.) |
0 commit comments