From 1bbaf6d01b86464c7dbb81cfd3d3b1254d2cfee2 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Fri, 5 Jun 2026 16:44:35 -0400 Subject: [PATCH 01/18] docs: convert restructuredText sources to MyST markdown MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2 of the documentation-site refresh. Run `rst2myst convert` over every human-authored .rst file under docs/source/ and remove the originals. The result: - 33 .rst files become 33 .md files (user guide, contributor guide, index, links). - Headings, paragraphs, hyperlinks, code blocks, admonitions, and toctree directives all map cleanly to MyST syntax. - Cross-reference anchors round-trip through MyST as `(label)=` blocks. The converter kebab-cased the labels (e.g. `(io-csv)=`), but every `{ref}` target in the corpus still uses the underscore form from the original RST (`{ref}\`CSV \``) and so do the Python docstrings that AutoAPI pulls in. Rewrite the anchors back to the underscore form so the existing references resolve. - 86 `{eval-rst}` blocks remain — they all wrap `.. ipython::` directives, which have no first-class MyST equivalent. They render identically and don't block the build. conf.py changes: - Enable `colon_fence` and `deflist` MyST extensions (rst-to-myst emits these on a few files, particularly execution-metrics.md). - Keep `.rst` in `source_suffix` even though no human-authored RST remains: sphinx-autoapi generates RST under autoapi/ at build time and Sphinx needs the suffix registered to parse it. AGENTS.md: update the two .rst paths called out under "Aggregate and Window Function Documentation" to point at the .md equivalents. Verified by building locally — `build succeeded`, no warnings, all internal cross-references resolve, the ipython examples on the landing page and basics page still execute. Co-Authored-By: Claude Opus 4.7 (1M context) --- AGENTS.md | 4 +- docs/source/conf.py | 12 +- docs/source/contributor-guide/ffi.md | 268 +++++++++++ docs/source/contributor-guide/ffi.rst | 265 ----------- docs/source/contributor-guide/index.md | 38 ++ docs/source/contributor-guide/index.rst | 28 -- docs/source/contributor-guide/introduction.md | 158 +++++++ .../source/contributor-guide/introduction.rst | 154 ------- docs/source/index.md | 72 +++ docs/source/index.rst | 62 --- docs/source/links.md | 40 ++ docs/source/links.rst | 30 -- .../source/user-guide/ai-coding-assistants.md | 90 ++++ .../user-guide/ai-coding-assistants.rst | 82 ---- docs/source/user-guide/basics.md | 107 +++++ docs/source/user-guide/basics.rst | 98 ---- .../{aggregations.rst => aggregations.md} | 369 ++++++++------- .../common-operations/basic-info.md | 80 ++++ .../common-operations/basic-info.rst | 61 --- .../{expressions.rst => expressions.md} | 199 ++++---- .../{functions.rst => functions.md} | 126 ++--- .../user-guide/common-operations/index.md | 45 ++ .../user-guide/common-operations/index.rst | 34 -- .../common-operations/{joins.rst => joins.md} | 95 ++-- .../common-operations/select-and-filter.md | 80 ++++ .../common-operations/select-and-filter.rst | 64 --- .../{udf-and-udfa.rst => udf-and-udfa.md} | 429 +++++++++--------- .../user-guide/common-operations/views.md | 67 +++ .../user-guide/common-operations/views.rst | 58 --- .../{windows.rst => windows.md} | 157 ++++--- docs/source/user-guide/configuration.md | 194 ++++++++ docs/source/user-guide/configuration.rst | 188 -------- docs/source/user-guide/data-sources.md | 290 ++++++++++++ docs/source/user-guide/data-sources.rst | 286 ------------ .../user-guide/dataframe/execution-metrics.md | 219 +++++++++ .../dataframe/execution-metrics.rst | 215 --------- docs/source/user-guide/dataframe/index.md | 380 ++++++++++++++++ docs/source/user-guide/dataframe/index.rst | 380 ---------------- docs/source/user-guide/dataframe/rendering.md | 236 ++++++++++ .../source/user-guide/dataframe/rendering.rst | 240 ---------- docs/source/user-guide/distributing-work.md | 364 +++++++++++++++ docs/source/user-guide/distributing-work.rst | 368 --------------- docs/source/user-guide/index.md | 48 ++ docs/source/user-guide/index.rst | 38 -- docs/source/user-guide/introduction.md | 91 ++++ docs/source/user-guide/introduction.rst | 77 ---- docs/source/user-guide/io/arrow.md | 85 ++++ docs/source/user-guide/io/arrow.rst | 75 --- docs/source/user-guide/io/avro.md | 41 ++ docs/source/user-guide/io/avro.rst | 32 -- docs/source/user-guide/io/csv.md | 69 +++ docs/source/user-guide/io/csv.rst | 60 --- docs/source/user-guide/io/index.md | 40 ++ docs/source/user-guide/io/index.rst | 29 -- docs/source/user-guide/io/json.md | 41 ++ docs/source/user-guide/io/json.rst | 31 -- docs/source/user-guide/io/parquet.md | 47 ++ docs/source/user-guide/io/parquet.rst | 37 -- docs/source/user-guide/io/table_provider.md | 72 +++ docs/source/user-guide/io/table_provider.rst | 62 --- docs/source/user-guide/{sql.rst => sql.md} | 99 ++-- docs/source/user-guide/upgrade-guides.md | 172 +++++++ docs/source/user-guide/upgrade-guides.rst | 166 ------- 63 files changed, 4256 insertions(+), 3888 deletions(-) create mode 100644 docs/source/contributor-guide/ffi.md delete mode 100644 docs/source/contributor-guide/ffi.rst create mode 100644 docs/source/contributor-guide/index.md delete mode 100644 docs/source/contributor-guide/index.rst create mode 100644 docs/source/contributor-guide/introduction.md delete mode 100644 docs/source/contributor-guide/introduction.rst create mode 100644 docs/source/index.md delete mode 100644 docs/source/index.rst create mode 100644 docs/source/links.md delete mode 100644 docs/source/links.rst create mode 100644 docs/source/user-guide/ai-coding-assistants.md delete mode 100644 docs/source/user-guide/ai-coding-assistants.rst create mode 100644 docs/source/user-guide/basics.md delete mode 100644 docs/source/user-guide/basics.rst rename docs/source/user-guide/common-operations/{aggregations.rst => aggregations.md} (51%) create mode 100644 docs/source/user-guide/common-operations/basic-info.md delete mode 100644 docs/source/user-guide/common-operations/basic-info.rst rename docs/source/user-guide/common-operations/{expressions.rst => expressions.md} (68%) rename docs/source/user-guide/common-operations/{functions.rst => functions.md} (52%) create mode 100644 docs/source/user-guide/common-operations/index.md delete mode 100644 docs/source/user-guide/common-operations/index.rst rename docs/source/user-guide/common-operations/{joins.rst => joins.md} (67%) create mode 100644 docs/source/user-guide/common-operations/select-and-filter.md delete mode 100644 docs/source/user-guide/common-operations/select-and-filter.rst rename docs/source/user-guide/common-operations/{udf-and-udfa.rst => udf-and-udfa.md} (54%) create mode 100644 docs/source/user-guide/common-operations/views.md delete mode 100644 docs/source/user-guide/common-operations/views.rst rename docs/source/user-guide/common-operations/{windows.rst => windows.md} (59%) create mode 100644 docs/source/user-guide/configuration.md delete mode 100644 docs/source/user-guide/configuration.rst create mode 100644 docs/source/user-guide/data-sources.md delete mode 100644 docs/source/user-guide/data-sources.rst create mode 100644 docs/source/user-guide/dataframe/execution-metrics.md delete mode 100644 docs/source/user-guide/dataframe/execution-metrics.rst create mode 100644 docs/source/user-guide/dataframe/index.md delete mode 100644 docs/source/user-guide/dataframe/index.rst create mode 100644 docs/source/user-guide/dataframe/rendering.md delete mode 100644 docs/source/user-guide/dataframe/rendering.rst create mode 100644 docs/source/user-guide/distributing-work.md delete mode 100644 docs/source/user-guide/distributing-work.rst create mode 100644 docs/source/user-guide/index.md delete mode 100644 docs/source/user-guide/index.rst create mode 100644 docs/source/user-guide/introduction.md delete mode 100644 docs/source/user-guide/introduction.rst create mode 100644 docs/source/user-guide/io/arrow.md delete mode 100644 docs/source/user-guide/io/arrow.rst create mode 100644 docs/source/user-guide/io/avro.md delete mode 100644 docs/source/user-guide/io/avro.rst create mode 100644 docs/source/user-guide/io/csv.md delete mode 100644 docs/source/user-guide/io/csv.rst create mode 100644 docs/source/user-guide/io/index.md delete mode 100644 docs/source/user-guide/io/index.rst create mode 100644 docs/source/user-guide/io/json.md delete mode 100644 docs/source/user-guide/io/json.rst create mode 100644 docs/source/user-guide/io/parquet.md delete mode 100644 docs/source/user-guide/io/parquet.rst create mode 100644 docs/source/user-guide/io/table_provider.md delete mode 100644 docs/source/user-guide/io/table_provider.rst rename docs/source/user-guide/{sql.rst => sql.md} (54%) create mode 100644 docs/source/user-guide/upgrade-guides.md delete mode 100644 docs/source/user-guide/upgrade-guides.rst diff --git a/AGENTS.md b/AGENTS.md index 632d6ebc0..fda08b23c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -84,9 +84,9 @@ Every Python function must include a docstring with usage examples. When adding or updating an aggregate or window function, ensure the corresponding site documentation is kept in sync: -- **Aggregations**: `docs/source/user-guide/common-operations/aggregations.rst` — +- **Aggregations**: `docs/source/user-guide/common-operations/aggregations.md` — add new aggregate functions to the "Aggregate Functions" list and include usage examples if appropriate. -- **Window functions**: `docs/source/user-guide/common-operations/windows.rst` — +- **Window functions**: `docs/source/user-guide/common-operations/windows.md` — add new window functions to the "Available Functions" list and include usage examples if appropriate. diff --git a/docs/source/conf.py b/docs/source/conf.py index bb1473546..e10862388 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -53,6 +53,10 @@ "autoapi.extension", ] +# NOTE: .rst stays alongside .md because sphinx-autoapi generates RST +# under autoapi/ and Sphinx needs the suffix to parse it. The human- +# authored docs are all MyST .md now; the .rst entry is only for the +# autoapi build artifacts. source_suffix = { ".rst": "restructuredtext", ".md": "markdown", @@ -171,5 +175,9 @@ def setup(sphinx) -> None: # tell myst_parser to auto-generate anchor links for headers h1, h2, h3 myst_heading_anchors = 3 -# enable nice rendering of checkboxes for the task lists -myst_enable_extensions = ["tasklist"] +# MyST extensions: +# - tasklist: GitHub-style `- [x]` checkboxes +# - colon_fence: `:::{directive}` blocks (needed by execution-metrics.md +# after the RST -> MyST conversion) +# - deflist: definition lists (used in a couple of converted pages) +myst_enable_extensions = ["tasklist", "colon_fence", "deflist"] diff --git a/docs/source/contributor-guide/ffi.md b/docs/source/contributor-guide/ffi.md new file mode 100644 index 000000000..403cdf40e --- /dev/null +++ b/docs/source/contributor-guide/ffi.md @@ -0,0 +1,268 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(ffi)= + +# Python Extensions + +The DataFusion in Python project is designed to allow users to extend its functionality in a few core +areas. Ideally many users would like to package their extensions as a Python package and easily +integrate that package with this project. This page serves to describe some of the challenges we face +when doing these integrations and the approach our project uses. + +## The Primary Issue + +Suppose you wish to use DataFusion and you have a custom data source that can produce tables that +can then be queried against, similar to how you can register a {ref}`CSV ` or +{ref}`Parquet ` file. In DataFusion terminology, you likely want to implement a +{ref}`Custom Table Provider `. In an effort to make your data source +as performant as possible and to utilize the features of DataFusion, you may decide to write +your source in Rust and then expose it through [PyO3](https://pyo3.rs) as a Python library. + +At first glance, it may appear the best way to do this is to add the `datafusion-python` +crate as a dependency, provide a `PyTable`, and then to register it with the +`SessionContext`. Unfortunately, this will not work. + +When you produce your code as a Python library and it needs to interact with the DataFusion +library, at the lowest level they communicate through an Application Binary Interface (ABI). +The acronym sounds similar to API (Application Programming Interface), but it is distinctly +different. + +The ABI sets the standard for how these libraries can share data and functions between each +other. One of the key differences between Rust and other programming languages is that Rust +does not have a stable ABI. What this means in practice is that if you compile a Rust library +with one version of the `rustc` compiler and I compile another library to interface with it +but I use a different version of the compiler, there is no guarantee the interface will be +the same. + +In practice, this means that a Python library built with `datafusion-python` as a Rust +dependency will generally **not** be compatible with the DataFusion Python package, even +if they reference the same version of `datafusion-python`. If you attempt to do this, it may +work on your local computer if you have built both packages with the same optimizations. +This can sometimes lead to a false expectation that the code will work, but it frequently +breaks the moment you try to use your package against the released packages. + +You can find more information about the Rust ABI in their +[online documentation](https://doc.rust-lang.org/reference/abi.html). + +## The FFI Approach + +Rust supports interacting with other programming languages through it's Foreign Function +Interface (FFI). The advantage of using the FFI is that it enables you to write data structures +and functions that have a stable ABI. The allows you to use Rust code with C, Python, and +other languages. In fact, the [PyO3](https://pyo3.rs) library uses the FFI to share data +and functions between Python and Rust. + +The approach we are taking in the DataFusion in Python project is to incrementally expose +more portions of the DataFusion project via FFI interfaces. This allows users to write Rust +code that does **not** require the `datafusion-python` crate as a dependency, expose their +code in Python via PyO3, and have it interact with the DataFusion Python package. + +Early adopters of this approach include [delta-rs](https://delta-io.github.io/delta-rs/) +who has adapted their Table Provider for use in `` `datafusion-python` `` with only a few lines +of code. Also, the DataFusion Python project uses the existing definitions from +[Apache Arrow CStream Interface](https://arrow.apache.org/docs/format/CStreamInterface.html) +to support importing **and** exporting tables. Any Python package that supports reading +the Arrow C Stream interface can work with DataFusion Python out of the box! You can read +more about working with Arrow sources in the {ref}`Data Sources ` +page. + +To learn more about the Foreign Function Interface in Rust, the +[Rustonomicon](https://doc.rust-lang.org/nomicon/ffi.html) is a good resource. + +## Inspiration from Arrow + +DataFusion is built upon [Apache Arrow](https://arrow.apache.org/). The canonical Python +Arrow implementation, [pyarrow](https://arrow.apache.org/docs/python/index.html) provides +an excellent way to share Arrow data between Python projects without performing any copy +operations on the data. They do this by using a well defined set of interfaces. You can +find the details about their stream interface +[here](https://arrow.apache.org/docs/format/CStreamInterface.html). The +[Rust Arrow Implementation](https://github.com/apache/arrow-rs) also supports these +`C` style definitions via the Foreign Function Interface. + +In addition to using these interfaces to transfer Arrow data between libraries, `pyarrow` +goes one step further to make sharing the interfaces easier in Python. They do this +by exposing PyCapsules that contain the expected functionality. + +You can learn more about PyCapsules from the official +[Python online documentation](https://docs.python.org/3/c-api/capsule.html). PyCapsules +have excellent support in PyO3 already. The +[PyO3 online documentation](https://pyo3.rs/main/doc/pyo3/types/struct.pycapsule) is a good source +for more details on using PyCapsules in Rust. + +Two lessons we leverage from the Arrow project in DataFusion Python are: + +- We reuse the existing Arrow FFI functionality wherever possible. +- We expose PyCapsules that contain a FFI stable struct. + +## Implementation Details + +The bulk of the code necessary to perform our FFI operations is in the upstream +[DataFusion](https://datafusion.apache.org/) core repository. You can review the code and +documentation in the [datafusion-ffi] crate. + +Our FFI implementation is narrowly focused at sharing data and functions with Rust backed +libraries. This allows us to use the [abi_stable crate](https://crates.io/crates/abi_stable). +This is an excellent crate that allows for easy conversion between Rust native types +and FFI-safe alternatives. For example, if you needed to pass a `Vec` via FFI, +you can simply convert it to a `RVec` in an intuitive manner. It also supports +features like `RResult` and `ROption` that do not have an obvious translation to a +C equivalent. + +The [datafusion-ffi] crate has been designed to make it easy to convert from DataFusion +traits into their FFI counterparts. For example, if you have defined a custom +[TableProvider](https://docs.rs/datafusion/45.0.0/datafusion/catalog/trait.TableProvider.html) +and you want to create a sharable FFI counterpart, you could write: + +```rust +let my_provider = MyTableProvider::default(); +let ffi_provider = FFI_TableProvider::new(Arc::new(my_provider), false, None); +``` + +(ffi_pyclass_mutability)= + +## PyO3 class mutability guidelines + +PyO3 bindings should present immutable wrappers whenever a struct stores shared or +interior-mutable state. In practice this means that any `#[pyclass]` containing an +`Arc>` or similar synchronized primitive must opt into `#[pyclass(frozen)]` +unless there is a compelling reason not to. + +The execution context illustrates the preferred pattern. `PySessionContext` in +{file}`src/context.rs` stays frozen even though it shares mutable state internally via +`SessionContext`. This ensures PyO3 tracks borrows correctly while Python-facing APIs +clone the inner `SessionContext` or return new wrappers instead of mutating the +existing instance in place: + +```rust +#[pyclass(from_py_object, frozen, name = "SessionContext", module = "datafusion", subclass)] +#[derive(Clone)] +pub struct PySessionContext { + pub ctx: SessionContext, +} +``` + +Occasionally a type must remain mutable—for example when PyO3 attribute setters need to +update fields directly. In these rare cases add an inline justification so reviewers and +future contributors understand why `frozen` is unsafe to enable. `DataTypeMap` in +{file}`src/common/data_type.rs` includes such a comment because PyO3 still needs to track +field updates: + +```rust +// TODO: This looks like this needs pyo3 tracking so leaving unfrozen for now +#[derive(Debug, Clone)] +#[pyclass(from_py_object, name = "DataTypeMap", module = "datafusion.common", subclass)] +pub struct DataTypeMap { + #[pyo3(get, set)] + pub arrow_type: PyDataType, + #[pyo3(get, set)] + pub python_type: PythonType, + #[pyo3(get, set)] + pub sql_type: SqlType, +} +``` + +When reviewers encounter a mutable `#[pyclass]` without a comment, they should request +an explanation or ask that `frozen` be added. Keeping these wrappers frozen by default +helps avoid subtle bugs stemming from PyO3's interior mutability tracking. + +If you were interfacing with a library that provided the above `FFI_TableProvider` and +you needed to turn it back into an `TableProvider`, you can turn it into a +`ForeignTableProvider` with implements the `TableProvider` trait. + +```rust +let foreign_provider: ForeignTableProvider = ffi_provider.into(); +``` + +If you review the code in [datafusion-ffi] you will find that each of the traits we share +across the boundary has two portions, one with a `FFI_` prefix and one with a `Foreign` +prefix. This is used to distinguish which side of the FFI boundary that struct is +designed to be used on. The structures with the `FFI_` prefix are to be used on the +**provider** of the structure. In the example we're showing, this means the code that has +written the underlying `TableProvider` implementation to access your custom data source. +The structures with the `Foreign` prefix are to be used by the receiver. In this case, +it is the `datafusion-python` library. + +In order to share these FFI structures, we need to wrap them in some kind of Python object +that can be used to interface from one package to another. As described in the above +section on our inspiration from Arrow, we use `PyCapsule`. We can create a `PyCapsule` +for our provider thusly: + +```rust +let name = CString::new("datafusion_table_provider")?; +let my_capsule = PyCapsule::new_bound(py, provider, Some(name))?; +``` + +On the receiving side, turn this pycapsule object into the `FFI_TableProvider`, which +can then be turned into a `ForeignTableProvider` the associated code is: + +```rust +let capsule = capsule.cast::()?; +let data: NonNull = capsule + .pointer_checked(Some(name))? + .cast(); +let codec = unsafe { data.as_ref() }; +``` + +By convention the `datafusion-python` library expects a Python object that has a +`TableProvider` PyCapsule to have this capsule accessible by calling a function named +`__datafusion_table_provider__`. You can see a complete working example of how to +share a `TableProvider` from one python library to DataFusion Python in the +[repository examples folder](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example). + +This section has been written using `TableProvider` as an example. It is the first +extension that has been written using this approach and the most thoroughly implemented. +As we continue to expose more of the DataFusion features, we intend to follow this same +design pattern. + +## Alternative Approach + +Suppose you needed to expose some other features of DataFusion and you could not wait +for the upstream repository to implement the FFI approach we describe. In this case +you decide to create your dependency on the `datafusion-python` crate instead. + +As we discussed, this is not guaranteed to work across different compiler versions and +optimization levels. If you wish to go down this route, there are two approaches we +have identified you can use. + +1. Re-export all of `datafusion-python` yourself with your extensions built in. +2. Carefully synchronize your software releases with the `datafusion-python` CI build + system so that your libraries use the exact same compiler, features, and + optimization level. + +We currently do not recommend either of these approaches as they are difficult to +maintain over a long period. Additionally, they require a tight version coupling +between libraries. + +## Status of Work + +At the time of this writing, the FFI features are under active development. To see +the latest status, we recommend reviewing the code in the [datafusion-ffi] crate. + +[datafusion-ffi]: https://crates.io/crates/datafusion-ffi diff --git a/docs/source/contributor-guide/ffi.rst b/docs/source/contributor-guide/ffi.rst deleted file mode 100644 index c89b99849..000000000 --- a/docs/source/contributor-guide/ffi.rst +++ /dev/null @@ -1,265 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _ffi: - -Python Extensions -================= - -The DataFusion in Python project is designed to allow users to extend its functionality in a few core -areas. Ideally many users would like to package their extensions as a Python package and easily -integrate that package with this project. This page serves to describe some of the challenges we face -when doing these integrations and the approach our project uses. - -The Primary Issue ------------------ - -Suppose you wish to use DataFusion and you have a custom data source that can produce tables that -can then be queried against, similar to how you can register a :ref:`CSV ` or -:ref:`Parquet ` file. In DataFusion terminology, you likely want to implement a -:ref:`Custom Table Provider `. In an effort to make your data source -as performant as possible and to utilize the features of DataFusion, you may decide to write -your source in Rust and then expose it through `PyO3 `_ as a Python library. - -At first glance, it may appear the best way to do this is to add the ``datafusion-python`` -crate as a dependency, provide a ``PyTable``, and then to register it with the -``SessionContext``. Unfortunately, this will not work. - -When you produce your code as a Python library and it needs to interact with the DataFusion -library, at the lowest level they communicate through an Application Binary Interface (ABI). -The acronym sounds similar to API (Application Programming Interface), but it is distinctly -different. - -The ABI sets the standard for how these libraries can share data and functions between each -other. One of the key differences between Rust and other programming languages is that Rust -does not have a stable ABI. What this means in practice is that if you compile a Rust library -with one version of the ``rustc`` compiler and I compile another library to interface with it -but I use a different version of the compiler, there is no guarantee the interface will be -the same. - -In practice, this means that a Python library built with ``datafusion-python`` as a Rust -dependency will generally **not** be compatible with the DataFusion Python package, even -if they reference the same version of ``datafusion-python``. If you attempt to do this, it may -work on your local computer if you have built both packages with the same optimizations. -This can sometimes lead to a false expectation that the code will work, but it frequently -breaks the moment you try to use your package against the released packages. - -You can find more information about the Rust ABI in their -`online documentation `_. - -The FFI Approach ----------------- - -Rust supports interacting with other programming languages through it's Foreign Function -Interface (FFI). The advantage of using the FFI is that it enables you to write data structures -and functions that have a stable ABI. The allows you to use Rust code with C, Python, and -other languages. In fact, the `PyO3 `_ library uses the FFI to share data -and functions between Python and Rust. - -The approach we are taking in the DataFusion in Python project is to incrementally expose -more portions of the DataFusion project via FFI interfaces. This allows users to write Rust -code that does **not** require the ``datafusion-python`` crate as a dependency, expose their -code in Python via PyO3, and have it interact with the DataFusion Python package. - -Early adopters of this approach include `delta-rs `_ -who has adapted their Table Provider for use in ```datafusion-python``` with only a few lines -of code. Also, the DataFusion Python project uses the existing definitions from -`Apache Arrow CStream Interface `_ -to support importing **and** exporting tables. Any Python package that supports reading -the Arrow C Stream interface can work with DataFusion Python out of the box! You can read -more about working with Arrow sources in the :ref:`Data Sources ` -page. - -To learn more about the Foreign Function Interface in Rust, the -`Rustonomicon `_ is a good resource. - -Inspiration from Arrow ----------------------- - -DataFusion is built upon `Apache Arrow `_. The canonical Python -Arrow implementation, `pyarrow `_ provides -an excellent way to share Arrow data between Python projects without performing any copy -operations on the data. They do this by using a well defined set of interfaces. You can -find the details about their stream interface -`here `_. The -`Rust Arrow Implementation `_ also supports these -``C`` style definitions via the Foreign Function Interface. - -In addition to using these interfaces to transfer Arrow data between libraries, ``pyarrow`` -goes one step further to make sharing the interfaces easier in Python. They do this -by exposing PyCapsules that contain the expected functionality. - -You can learn more about PyCapsules from the official -`Python online documentation `_. PyCapsules -have excellent support in PyO3 already. The -`PyO3 online documentation `_ is a good source -for more details on using PyCapsules in Rust. - -Two lessons we leverage from the Arrow project in DataFusion Python are: - -- We reuse the existing Arrow FFI functionality wherever possible. -- We expose PyCapsules that contain a FFI stable struct. - -Implementation Details ----------------------- - -The bulk of the code necessary to perform our FFI operations is in the upstream -`DataFusion `_ core repository. You can review the code and -documentation in the `datafusion-ffi`_ crate. - -Our FFI implementation is narrowly focused at sharing data and functions with Rust backed -libraries. This allows us to use the `abi_stable crate `_. -This is an excellent crate that allows for easy conversion between Rust native types -and FFI-safe alternatives. For example, if you needed to pass a ``Vec`` via FFI, -you can simply convert it to a ``RVec`` in an intuitive manner. It also supports -features like ``RResult`` and ``ROption`` that do not have an obvious translation to a -C equivalent. - -The `datafusion-ffi`_ crate has been designed to make it easy to convert from DataFusion -traits into their FFI counterparts. For example, if you have defined a custom -`TableProvider `_ -and you want to create a sharable FFI counterpart, you could write: - -.. code-block:: rust - - let my_provider = MyTableProvider::default(); - let ffi_provider = FFI_TableProvider::new(Arc::new(my_provider), false, None); - -.. _ffi_pyclass_mutability: - -PyO3 class mutability guidelines --------------------------------- - -PyO3 bindings should present immutable wrappers whenever a struct stores shared or -interior-mutable state. In practice this means that any ``#[pyclass]`` containing an -``Arc>`` or similar synchronized primitive must opt into ``#[pyclass(frozen)]`` -unless there is a compelling reason not to. - -The execution context illustrates the preferred pattern. ``PySessionContext`` in -:file:`src/context.rs` stays frozen even though it shares mutable state internally via -``SessionContext``. This ensures PyO3 tracks borrows correctly while Python-facing APIs -clone the inner ``SessionContext`` or return new wrappers instead of mutating the -existing instance in place: - -.. code-block:: rust - - #[pyclass(from_py_object, frozen, name = "SessionContext", module = "datafusion", subclass)] - #[derive(Clone)] - pub struct PySessionContext { - pub ctx: SessionContext, - } - -Occasionally a type must remain mutable—for example when PyO3 attribute setters need to -update fields directly. In these rare cases add an inline justification so reviewers and -future contributors understand why ``frozen`` is unsafe to enable. ``DataTypeMap`` in -:file:`src/common/data_type.rs` includes such a comment because PyO3 still needs to track -field updates: - -.. code-block:: rust - - // TODO: This looks like this needs pyo3 tracking so leaving unfrozen for now - #[derive(Debug, Clone)] - #[pyclass(from_py_object, name = "DataTypeMap", module = "datafusion.common", subclass)] - pub struct DataTypeMap { - #[pyo3(get, set)] - pub arrow_type: PyDataType, - #[pyo3(get, set)] - pub python_type: PythonType, - #[pyo3(get, set)] - pub sql_type: SqlType, - } - -When reviewers encounter a mutable ``#[pyclass]`` without a comment, they should request -an explanation or ask that ``frozen`` be added. Keeping these wrappers frozen by default -helps avoid subtle bugs stemming from PyO3's interior mutability tracking. - -If you were interfacing with a library that provided the above ``FFI_TableProvider`` and -you needed to turn it back into an ``TableProvider``, you can turn it into a -``ForeignTableProvider`` with implements the ``TableProvider`` trait. - -.. code-block:: rust - - let foreign_provider: ForeignTableProvider = ffi_provider.into(); - -If you review the code in `datafusion-ffi`_ you will find that each of the traits we share -across the boundary has two portions, one with a ``FFI_`` prefix and one with a ``Foreign`` -prefix. This is used to distinguish which side of the FFI boundary that struct is -designed to be used on. The structures with the ``FFI_`` prefix are to be used on the -**provider** of the structure. In the example we're showing, this means the code that has -written the underlying ``TableProvider`` implementation to access your custom data source. -The structures with the ``Foreign`` prefix are to be used by the receiver. In this case, -it is the ``datafusion-python`` library. - -In order to share these FFI structures, we need to wrap them in some kind of Python object -that can be used to interface from one package to another. As described in the above -section on our inspiration from Arrow, we use ``PyCapsule``. We can create a ``PyCapsule`` -for our provider thusly: - -.. code-block:: rust - - let name = CString::new("datafusion_table_provider")?; - let my_capsule = PyCapsule::new_bound(py, provider, Some(name))?; - -On the receiving side, turn this pycapsule object into the ``FFI_TableProvider``, which -can then be turned into a ``ForeignTableProvider`` the associated code is: - -.. code-block:: rust - - let capsule = capsule.cast::()?; - let data: NonNull = capsule - .pointer_checked(Some(name))? - .cast(); - let codec = unsafe { data.as_ref() }; - -By convention the ``datafusion-python`` library expects a Python object that has a -``TableProvider`` PyCapsule to have this capsule accessible by calling a function named -``__datafusion_table_provider__``. You can see a complete working example of how to -share a ``TableProvider`` from one python library to DataFusion Python in the -`repository examples folder `_. - -This section has been written using ``TableProvider`` as an example. It is the first -extension that has been written using this approach and the most thoroughly implemented. -As we continue to expose more of the DataFusion features, we intend to follow this same -design pattern. - -Alternative Approach --------------------- - -Suppose you needed to expose some other features of DataFusion and you could not wait -for the upstream repository to implement the FFI approach we describe. In this case -you decide to create your dependency on the ``datafusion-python`` crate instead. - -As we discussed, this is not guaranteed to work across different compiler versions and -optimization levels. If you wish to go down this route, there are two approaches we -have identified you can use. - -#. Re-export all of ``datafusion-python`` yourself with your extensions built in. -#. Carefully synchronize your software releases with the ``datafusion-python`` CI build - system so that your libraries use the exact same compiler, features, and - optimization level. - -We currently do not recommend either of these approaches as they are difficult to -maintain over a long period. Additionally, they require a tight version coupling -between libraries. - -Status of Work --------------- - -At the time of this writing, the FFI features are under active development. To see -the latest status, we recommend reviewing the code in the `datafusion-ffi`_ crate. - -.. _datafusion-ffi: https://crates.io/crates/datafusion-ffi diff --git a/docs/source/contributor-guide/index.md b/docs/source/contributor-guide/index.md new file mode 100644 index 000000000..df528ed54 --- /dev/null +++ b/docs/source/contributor-guide/index.md @@ -0,0 +1,38 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Contributor Guide + +Guides for contributors to the DataFusion in Python project. + +```{toctree} +:maxdepth: 2 + +introduction +ffi +``` diff --git a/docs/source/contributor-guide/index.rst b/docs/source/contributor-guide/index.rst deleted file mode 100644 index b32e08878..000000000 --- a/docs/source/contributor-guide/index.rst +++ /dev/null @@ -1,28 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -================= -Contributor Guide -================= - -Guides for contributors to the DataFusion in Python project. - -.. toctree:: - :maxdepth: 2 - - introduction - ffi diff --git a/docs/source/contributor-guide/introduction.md b/docs/source/contributor-guide/introduction.md new file mode 100644 index 000000000..fa87c57a2 --- /dev/null +++ b/docs/source/contributor-guide/introduction.md @@ -0,0 +1,158 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Introduction + +We welcome and encourage contributions of all kinds, such as: + +1. Tickets with issue reports of feature requests +2. Documentation improvements +3. Code, both PR and (especially) PR Review. + +In addition to submitting new PRs, we have a healthy tradition of community members reviewing each other’s PRs. +Doing so is a great way to help the community as well as get more familiar with Rust and the relevant codebases. + +Before opening a pull request that touches PyO3 bindings, please review the +{ref}`PyO3 class mutability guidelines ` so you can flag missing +`#[pyclass(frozen)]` annotations during development and review. + +## How to develop + +This assumes that you have rust and cargo installed. We use the workflow recommended by +[pyo3](https://github.com/PyO3/pyo3) and [maturin](https://github.com/PyO3/maturin). We recommend using +[uv](https://docs.astral.sh/uv/) for python package management. + +By default `uv` will attempt to build the datafusion python package. For our development we prefer to build manually. This means +that when creating your virtual environment using `uv sync` you need to pass in the additional `--no-install-package datafusion` +and for `uv run` commands the additional parameter `--no-project` + +Bootstrap: + +```shell +# fetch this repo +git clone git@github.com:apache/datafusion-python.git +# create the virtual environment +uv sync --dev --no-install-package datafusion +# activate the environment +source .venv/bin/activate +``` + +The tests rely on test data in git submodules. + +```shell +git submodule init +git submodule update +``` + +Whenever rust code changes (your changes or via `git pull`): + +```shell +# make sure you activate the venv using "source .venv/bin/activate" first +maturin develop -uv +python -m pytest +``` + +## Running & Installing pre-commit hooks + +arrow-datafusion-python takes advantage of [pre-commit](https://pre-commit.com/) to assist developers with code linting to help reduce the number of commits that ultimately fail in CI due to linter errors. Using the pre-commit hooks is optional for the developer but certainly helpful for keeping PRs clean and concise. + +Our pre-commit hooks can be installed by running {code}`pre-commit install`, which will install the configurations in your ARROW_DATAFUSION_PYTHON_ROOT/.github directory and run each time you perform a commit, failing to complete the commit if an offending lint is found allowing you to make changes locally before pushing. + +The pre-commit hooks can also be run adhoc without installing them by simply running {code}`pre-commit run --all-files` + +## Guidelines for Separating Python and Rust Code + +Version 40 of `datafusion-python` introduced `python` wrappers around the `pyo3` generated code to vastly improve the user experience. (See the [blog post](https://datafusion.apache.org/blog/2024/08/20/python-datafusion-40.0.0/) and [pull request](https://github.com/apache/datafusion-python/pull/750) for more details.) + +Mostly, the `python` code is limited to pure wrappers with type hints and good docstrings, but there are a few reasons for when the code does more: + +1. Trivial aliases like {py:func}`~datafusion.functions.array_append` and {py:func}`~datafusion.functions.list_append`. +2. Simple type conversion, like from a `path` to a `string` of the path or from `number` to `lit(number)`. +3. The additional code makes an API **much** more pythonic, like we do for {py:func}`~datafusion.functions.named_struct` (see [source code](https://github.com/apache/datafusion-python/blob/a0913c728f5f323c1eb4913e614c9d996083e274/python/datafusion/functions.py#L1040-L1046)). + +## Update Dependencies + +To change test dependencies, change the `pyproject.toml` and run + +To update dependencies, run + +```shell +uv sync --dev --no-install-package datafusion +``` + +## Improving Build Speed + +The [pyo3](https://github.com/PyO3/pyo3) dependency of this project contains a `build.rs` file which +can cause it to rebuild frequently. You can prevent this from happening by defining a `PYO3_CONFIG_FILE` +environment variable that points to a file with your build configuration. Whenever your build configuration +changes, such as during some major version updates, you will need to regenerate this file. This variable +should point to a fully resolved path on your build machine. + +To generate this file, use the following command: + +```shell +PYO3_PRINT_CONFIG=1 cargo build +``` + +This will generate some output that looks like the following. You will want to copy these contents intro +a file. If you place this file in your project directory with filename `.pyo3_build_config` it will +be ignored by `git`. + +``` +implementation=CPython +version=3.9 +shared=true +abi3=true +lib_name=python3.12 +lib_dir=/opt/homebrew/opt/python@3.12/Frameworks/Python.framework/Versions/3.12/lib +executable=/Users/myusername/src/datafusion-python/.venv/bin/python +pointer_width=64 +build_flags= +suppress_build_script_link_lines=false +``` + +Add the environment variable to your system. + +```shell +export PYO3_CONFIG_FILE="/Users//myusername/src/datafusion-python/.pyo3_build_config" +``` + +If you are on a Mac and you use VS Code for your IDE, you will want to add these variables +to your settings. You can find the appropriate rust flags by looking in the +`.cargo/config.toml` file. + +``` +"rust-analyzer.cargo.extraEnv": { + "RUSTFLAGS": "-C link-arg=-undefined -C link-arg=dynamic_lookup", + "PYO3_CONFIG_FILE": "/Users/myusername/src/datafusion-python/.pyo3_build_config" +}, +"rust-analyzer.runnables.extraEnv": { + "RUSTFLAGS": "-C link-arg=-undefined -C link-arg=dynamic_lookup", + "PYO3_CONFIG_FILE": "/Users/myusername/src/personal/datafusion-python/.pyo3_build_config" +} +``` diff --git a/docs/source/contributor-guide/introduction.rst b/docs/source/contributor-guide/introduction.rst deleted file mode 100644 index 33c2b274c..000000000 --- a/docs/source/contributor-guide/introduction.rst +++ /dev/null @@ -1,154 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Introduction -============ -We welcome and encourage contributions of all kinds, such as: - -1. Tickets with issue reports of feature requests -2. Documentation improvements -3. Code, both PR and (especially) PR Review. - -In addition to submitting new PRs, we have a healthy tradition of community members reviewing each other’s PRs. -Doing so is a great way to help the community as well as get more familiar with Rust and the relevant codebases. - -Before opening a pull request that touches PyO3 bindings, please review the -:ref:`PyO3 class mutability guidelines ` so you can flag missing -``#[pyclass(frozen)]`` annotations during development and review. - -How to develop --------------- - -This assumes that you have rust and cargo installed. We use the workflow recommended by -`pyo3 `_ and `maturin `_. We recommend using -`uv `_ for python package management. - -By default `uv` will attempt to build the datafusion python package. For our development we prefer to build manually. This means -that when creating your virtual environment using `uv sync` you need to pass in the additional `--no-install-package datafusion` -and for `uv run` commands the additional parameter `--no-project` - -Bootstrap: - -.. code-block:: shell - - # fetch this repo - git clone git@github.com:apache/datafusion-python.git - # create the virtual environment - uv sync --dev --no-install-package datafusion - # activate the environment - source .venv/bin/activate - -The tests rely on test data in git submodules. - -.. code-block:: shell - - git submodule init - git submodule update - - -Whenever rust code changes (your changes or via `git pull`): - -.. code-block:: shell - - # make sure you activate the venv using "source .venv/bin/activate" first - maturin develop -uv - python -m pytest - -Running & Installing pre-commit hooks -------------------------------------- - -arrow-datafusion-python takes advantage of `pre-commit `_ to assist developers with code linting to help reduce the number of commits that ultimately fail in CI due to linter errors. Using the pre-commit hooks is optional for the developer but certainly helpful for keeping PRs clean and concise. - -Our pre-commit hooks can be installed by running :code:`pre-commit install`, which will install the configurations in your ARROW_DATAFUSION_PYTHON_ROOT/.github directory and run each time you perform a commit, failing to complete the commit if an offending lint is found allowing you to make changes locally before pushing. - -The pre-commit hooks can also be run adhoc without installing them by simply running :code:`pre-commit run --all-files` - -Guidelines for Separating Python and Rust Code ----------------------------------------------- - -Version 40 of ``datafusion-python`` introduced ``python`` wrappers around the ``pyo3`` generated code to vastly improve the user experience. (See the `blog post `_ and `pull request `_ for more details.) - -Mostly, the ``python`` code is limited to pure wrappers with type hints and good docstrings, but there are a few reasons for when the code does more: - -1. Trivial aliases like :py:func:`~datafusion.functions.array_append` and :py:func:`~datafusion.functions.list_append`. -2. Simple type conversion, like from a ``path`` to a ``string`` of the path or from ``number`` to ``lit(number)``. -3. The additional code makes an API **much** more pythonic, like we do for :py:func:`~datafusion.functions.named_struct` (see `source code `_). - - -Update Dependencies -------------------- - -To change test dependencies, change the ``pyproject.toml`` and run - -To update dependencies, run - -.. code-block:: shell - - uv sync --dev --no-install-package datafusion - -Improving Build Speed ---------------------- - -The `pyo3 `_ dependency of this project contains a ``build.rs`` file which -can cause it to rebuild frequently. You can prevent this from happening by defining a ``PYO3_CONFIG_FILE`` -environment variable that points to a file with your build configuration. Whenever your build configuration -changes, such as during some major version updates, you will need to regenerate this file. This variable -should point to a fully resolved path on your build machine. - -To generate this file, use the following command: - -.. code-block:: shell - - PYO3_PRINT_CONFIG=1 cargo build - -This will generate some output that looks like the following. You will want to copy these contents intro -a file. If you place this file in your project directory with filename ``.pyo3_build_config`` it will -be ignored by ``git``. - -.. code-block:: - - implementation=CPython - version=3.9 - shared=true - abi3=true - lib_name=python3.12 - lib_dir=/opt/homebrew/opt/python@3.12/Frameworks/Python.framework/Versions/3.12/lib - executable=/Users/myusername/src/datafusion-python/.venv/bin/python - pointer_width=64 - build_flags= - suppress_build_script_link_lines=false - -Add the environment variable to your system. - -.. code-block:: shell - - export PYO3_CONFIG_FILE="/Users//myusername/src/datafusion-python/.pyo3_build_config" - -If you are on a Mac and you use VS Code for your IDE, you will want to add these variables -to your settings. You can find the appropriate rust flags by looking in the -``.cargo/config.toml`` file. - -.. code-block:: - - "rust-analyzer.cargo.extraEnv": { - "RUSTFLAGS": "-C link-arg=-undefined -C link-arg=dynamic_lookup", - "PYO3_CONFIG_FILE": "/Users/myusername/src/datafusion-python/.pyo3_build_config" - }, - "rust-analyzer.runnables.extraEnv": { - "RUSTFLAGS": "-C link-arg=-undefined -C link-arg=dynamic_lookup", - "PYO3_CONFIG_FILE": "/Users/myusername/src/personal/datafusion-python/.pyo3_build_config" - } diff --git a/docs/source/index.md b/docs/source/index.md new file mode 100644 index 000000000..5b1f0f53b --- /dev/null +++ b/docs/source/index.md @@ -0,0 +1,72 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# DataFusion in Python + +This is a Python library that binds to [Apache Arrow](https://arrow.apache.org/) in-memory query engine [DataFusion](https://github.com/apache/datafusion). + +Like pyspark, it allows you to build a plan through SQL or a DataFrame API against in-memory data, parquet or CSV files, run it in a multi-threaded environment, and obtain the result back in Python. + +It also allows you to use UDFs and UDAFs for complex operations. + +The major advantage of this library over other execution engines is that this library achieves zero-copy between Python and its execution engine: there is no cost in using UDFs, UDAFs, and collecting the results to Python apart from having to lock the GIL when running those operations. + +Its query engine, DataFusion, is written in [Rust](https://www.rust-lang.org), which makes strong assumptions about thread safety and lack of memory leaks. + +Technically, zero-copy is achieved via the [c data interface](https://arrow.apache.org/docs/format/CDataInterface.html). + +## Install + +```shell +pip install datafusion +``` + +## Example + +```{eval-rst} +.. ipython:: python + + from datafusion import SessionContext + + ctx = SessionContext() + + df = ctx.read_csv("pokemon.csv") + + df.show() + +``` + +```{toctree} +:hidden: true +:maxdepth: 1 + +user-guide/index +contributor-guide/index +API Reference +links +``` diff --git a/docs/source/index.rst b/docs/source/index.rst deleted file mode 100644 index 6b72537da..000000000 --- a/docs/source/index.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -==================== -DataFusion in Python -==================== - -This is a Python library that binds to `Apache Arrow `_ in-memory query engine `DataFusion `_. - -Like pyspark, it allows you to build a plan through SQL or a DataFrame API against in-memory data, parquet or CSV files, run it in a multi-threaded environment, and obtain the result back in Python. - -It also allows you to use UDFs and UDAFs for complex operations. - -The major advantage of this library over other execution engines is that this library achieves zero-copy between Python and its execution engine: there is no cost in using UDFs, UDAFs, and collecting the results to Python apart from having to lock the GIL when running those operations. - -Its query engine, DataFusion, is written in `Rust `_, which makes strong assumptions about thread safety and lack of memory leaks. - -Technically, zero-copy is achieved via the `c data interface `_. - -Install -------- - -.. code-block:: shell - - pip install datafusion - -Example -------- - -.. ipython:: python - - from datafusion import SessionContext - - ctx = SessionContext() - - df = ctx.read_csv("pokemon.csv") - - df.show() - - -.. toctree:: - :hidden: - :maxdepth: 1 - - user-guide/index - contributor-guide/index - API Reference - links diff --git a/docs/source/links.md b/docs/source/links.md new file mode 100644 index 000000000..fbcde343e --- /dev/null +++ b/docs/source/links.md @@ -0,0 +1,40 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Links + +External resources for the DataFusion in Python project. + +```{toctree} +:maxdepth: 1 + +GitHub and Issue Tracker +Rust API Docs +Code of Conduct +Examples +``` diff --git a/docs/source/links.rst b/docs/source/links.rst deleted file mode 100644 index 10473f31b..000000000 --- a/docs/source/links.rst +++ /dev/null @@ -1,30 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -===== -Links -===== - -External resources for the DataFusion in Python project. - -.. toctree:: - :maxdepth: 1 - - GitHub and Issue Tracker - Rust API Docs - Code of Conduct - Examples diff --git a/docs/source/user-guide/ai-coding-assistants.md b/docs/source/user-guide/ai-coding-assistants.md new file mode 100644 index 000000000..90335837b --- /dev/null +++ b/docs/source/user-guide/ai-coding-assistants.md @@ -0,0 +1,90 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Using AI Coding Assistants + +If you write DataFusion Python code with an AI coding assistant, this +project ships machine-readable guidance so the assistant produces +idiomatic code rather than guessing from its training data. + +## What is published + +- [SKILL.md](https://github.com/apache/datafusion-python/blob/main/skills/datafusion_python/SKILL.md) — + a dense, skill-oriented reference covering imports, data loading, + DataFrame operations, expression building, SQL-to-DataFrame mappings, + idiomatic patterns, and common pitfalls. Follows the + [Agent Skills](https://agentskills.io) open standard. +- [llms.txt](https://datafusion.apache.org/python/llms.txt) — an entry point for LLM-based tools following the + [llmstxt.org](https://llmstxt.org) convention. Categorized links to the + skill, user guide, API reference, and examples. + +Both files live at stable URLs so an agent can discover them without a +checkout of the repo. + +## Installing the skill + +**Preferred:** run + +```shell +npx skills add apache/datafusion-python +``` + +This installs the skill in any supported agent on your machine (Claude +Code, Cursor, Windsurf, Cline, Codex, Copilot, Gemini CLI, and others). +The command writes the pointer into the agent's configuration so that any +project you open that uses DataFusion Python picks up the skill +automatically. + +**Manual:** if you are not using the `skills` registry, paste this +single line into your project's `AGENTS.md` or `CLAUDE.md`: + +``` +For DataFusion Python code, see https://github.com/apache/datafusion-python/blob/main/skills/datafusion_python/SKILL.md +``` + +Most assistants resolve that pointer the first time they see a +DataFusion-related prompt in the project. + +## What the skill covers + +Writing DataFusion Python code has a handful of conventions that are easy +for a model to miss — bitwise `&` / `|` / `~` instead of Python +`and` / `or` / `not`, the lazy-DataFrame immutability model, how +window functions replace SQL correlated subqueries, the `case` / +`when` builder syntax, and the `in_list` / `array_position` options +for membership tests. The skill enumerates each of these with short, +copyable examples. + +It is *not* a replacement for this user guide. Think of it as a distilled +reference the assistant keeps open while it writes code for you. + +## If you are an agent author + +The skill file and `llms.txt` are the two supported integration +points. Both are versioned along with the release and follow open +standards — no project-specific handshake is required. diff --git a/docs/source/user-guide/ai-coding-assistants.rst b/docs/source/user-guide/ai-coding-assistants.rst deleted file mode 100644 index fb7998c6d..000000000 --- a/docs/source/user-guide/ai-coding-assistants.rst +++ /dev/null @@ -1,82 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Using AI Coding Assistants -========================== - -If you write DataFusion Python code with an AI coding assistant, this -project ships machine-readable guidance so the assistant produces -idiomatic code rather than guessing from its training data. - -What is published ------------------ - -- `SKILL.md `_ — - a dense, skill-oriented reference covering imports, data loading, - DataFrame operations, expression building, SQL-to-DataFrame mappings, - idiomatic patterns, and common pitfalls. Follows the - `Agent Skills `_ open standard. -- `llms.txt `_ — an entry point for LLM-based tools following the - `llmstxt.org `_ convention. Categorized links to the - skill, user guide, API reference, and examples. - -Both files live at stable URLs so an agent can discover them without a -checkout of the repo. - -Installing the skill --------------------- - -**Preferred:** run - -.. code-block:: shell - - npx skills add apache/datafusion-python - -This installs the skill in any supported agent on your machine (Claude -Code, Cursor, Windsurf, Cline, Codex, Copilot, Gemini CLI, and others). -The command writes the pointer into the agent's configuration so that any -project you open that uses DataFusion Python picks up the skill -automatically. - -**Manual:** if you are not using the ``skills`` registry, paste this -single line into your project's ``AGENTS.md`` or ``CLAUDE.md``:: - - For DataFusion Python code, see https://github.com/apache/datafusion-python/blob/main/skills/datafusion_python/SKILL.md - -Most assistants resolve that pointer the first time they see a -DataFusion-related prompt in the project. - -What the skill covers ---------------------- - -Writing DataFusion Python code has a handful of conventions that are easy -for a model to miss — bitwise ``&`` / ``|`` / ``~`` instead of Python -``and`` / ``or`` / ``not``, the lazy-DataFrame immutability model, how -window functions replace SQL correlated subqueries, the ``case`` / -``when`` builder syntax, and the ``in_list`` / ``array_position`` options -for membership tests. The skill enumerates each of these with short, -copyable examples. - -It is *not* a replacement for this user guide. Think of it as a distilled -reference the assistant keeps open while it writes code for you. - -If you are an agent author --------------------------- - -The skill file and ``llms.txt`` are the two supported integration -points. Both are versioned along with the release and follow open -standards — no project-specific handshake is required. diff --git a/docs/source/user-guide/basics.md b/docs/source/user-guide/basics.md new file mode 100644 index 000000000..800b6a67c --- /dev/null +++ b/docs/source/user-guide/basics.md @@ -0,0 +1,107 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(user_guide_concepts)= + +# Concepts + +In this section, we will cover a basic example to introduce a few key concepts. We will use the +2021 Yellow Taxi Trip Records ([download](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet)), +from the [TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page). + +```{eval-rst} +.. ipython:: python + + from datafusion import SessionContext, col, lit, functions as f + + ctx = SessionContext() + + df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") + + df = df.select( + "trip_distance", + col("total_amount").alias("total"), + (f.round(lit(100.0) * col("tip_amount") / col("total_amount"), lit(1))).alias("tip_percent"), + ) + + df.show() +``` + +## Session Context + +The first statement group creates a {py:class}`~datafusion.context.SessionContext`. + +```python +# create a context +ctx = datafusion.SessionContext() +``` + +A Session Context is the main interface for executing queries with DataFusion. It maintains the state +of the connection between a user and an instance of the DataFusion engine. Additionally it provides +the following functionality: + +- Create a DataFrame from a data source. +- Register a data source as a table that can be referenced from a SQL query. +- Execute a SQL query + +## DataFrame + +The second statement group creates a {code}`DataFrame`, + +```python +# Create a DataFrame from a file +df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") +``` + +A DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html). +DataFrames are typically created by calling a method on {py:class}`~datafusion.context.SessionContext`, such as {code}`read_csv`, and can then be modified by +calling the transformation methods, such as {py:func}`~datafusion.dataframe.DataFrame.filter`, {py:func}`~datafusion.dataframe.DataFrame.select`, {py:func}`~datafusion.dataframe.DataFrame.aggregate`, +and {py:func}`~datafusion.dataframe.DataFrame.limit` to build up a query definition. + +For more details on working with DataFrames, including visualization options and conversion to other formats, see {doc}`dataframe/index`. + +## Expressions + +The third statement uses {code}`Expressions` to build up a query definition. You can find +explanations for what the functions below do in the user documentation for +{py:func}`~datafusion.col`, {py:func}`~datafusion.lit`, {py:func}`~datafusion.functions.round`, +and {py:func}`~datafusion.expr.Expr.alias`. + +```python +df = df.select( + "trip_distance", + col("total_amount").alias("total"), + (f.round(lit(100.0) * col("tip_amount") / col("total_amount"), lit(1))).alias("tip_percent"), +) +``` + +Finally the {py:func}`~datafusion.dataframe.DataFrame.show` method converts the logical plan +represented by the DataFrame into a physical plan and execute it, collecting all results and +displaying them to the user. It is important to note that DataFusion performs lazy evaluation +of the DataFrame. Until you call a method such as {py:func}`~datafusion.dataframe.DataFrame.show` +or {py:func}`~datafusion.dataframe.DataFrame.collect`, DataFusion will not perform the query. diff --git a/docs/source/user-guide/basics.rst b/docs/source/user-guide/basics.rst deleted file mode 100644 index 7c6820461..000000000 --- a/docs/source/user-guide/basics.rst +++ /dev/null @@ -1,98 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _user_guide_concepts: - -Concepts -======== - -In this section, we will cover a basic example to introduce a few key concepts. We will use the -2021 Yellow Taxi Trip Records (`download `_), -from the `TLC Trip Record Data `_. - -.. ipython:: python - - from datafusion import SessionContext, col, lit, functions as f - - ctx = SessionContext() - - df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") - - df = df.select( - "trip_distance", - col("total_amount").alias("total"), - (f.round(lit(100.0) * col("tip_amount") / col("total_amount"), lit(1))).alias("tip_percent"), - ) - - df.show() - -Session Context ---------------- - -The first statement group creates a :py:class:`~datafusion.context.SessionContext`. - -.. code-block:: python - - # create a context - ctx = datafusion.SessionContext() - -A Session Context is the main interface for executing queries with DataFusion. It maintains the state -of the connection between a user and an instance of the DataFusion engine. Additionally it provides -the following functionality: - -- Create a DataFrame from a data source. -- Register a data source as a table that can be referenced from a SQL query. -- Execute a SQL query - -DataFrame ---------- - -The second statement group creates a :code:`DataFrame`, - -.. code-block:: python - - # Create a DataFrame from a file - df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") - -A DataFrame refers to a (logical) set of rows that share the same column names, similar to a `Pandas DataFrame `_. -DataFrames are typically created by calling a method on :py:class:`~datafusion.context.SessionContext`, such as :code:`read_csv`, and can then be modified by -calling the transformation methods, such as :py:func:`~datafusion.dataframe.DataFrame.filter`, :py:func:`~datafusion.dataframe.DataFrame.select`, :py:func:`~datafusion.dataframe.DataFrame.aggregate`, -and :py:func:`~datafusion.dataframe.DataFrame.limit` to build up a query definition. - -For more details on working with DataFrames, including visualization options and conversion to other formats, see :doc:`dataframe/index`. - -Expressions ------------ - -The third statement uses :code:`Expressions` to build up a query definition. You can find -explanations for what the functions below do in the user documentation for -:py:func:`~datafusion.col`, :py:func:`~datafusion.lit`, :py:func:`~datafusion.functions.round`, -and :py:func:`~datafusion.expr.Expr.alias`. - -.. code-block:: python - - df = df.select( - "trip_distance", - col("total_amount").alias("total"), - (f.round(lit(100.0) * col("tip_amount") / col("total_amount"), lit(1))).alias("tip_percent"), - ) - -Finally the :py:func:`~datafusion.dataframe.DataFrame.show` method converts the logical plan -represented by the DataFrame into a physical plan and execute it, collecting all results and -displaying them to the user. It is important to note that DataFusion performs lazy evaluation -of the DataFrame. Until you call a method such as :py:func:`~datafusion.dataframe.DataFrame.show` -or :py:func:`~datafusion.dataframe.DataFrame.collect`, DataFusion will not perform the query. diff --git a/docs/source/user-guide/common-operations/aggregations.rst b/docs/source/user-guide/common-operations/aggregations.md similarity index 51% rename from docs/source/user-guide/common-operations/aggregations.rst rename to docs/source/user-guide/common-operations/aggregations.md index b1e43a32f..7a59390a2 100644 --- a/docs/source/user-guide/common-operations/aggregations.rst +++ b/docs/source/user-guide/common-operations/aggregations.md @@ -1,29 +1,40 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at +% Licensed to the Apache Software Foundation (ASF) under one -.. http://www.apache.org/licenses/LICENSE-2.0 +% or more contributor license agreements. See the NOTICE file -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. +% distributed with this work for additional information -.. _aggregation: +% regarding copyright ownership. The ASF licenses this file -Aggregation -============ +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(aggregation)= + +# Aggregation An aggregate or aggregation is a function where the values of multiple rows are processed together to form a single summary value. For performing an aggregation, DataFusion provides the -:py:func:`~datafusion.dataframe.DataFrame.aggregate` +{py:func}`~datafusion.dataframe.DataFrame.aggregate` +```{eval-rst} .. ipython:: python from datafusion import SessionContext, col, lit, functions as f @@ -40,19 +51,23 @@ to form a single summary value. For performing an aggregation, DataFusion provid f.approx_distinct(col_speed).alias("Count"), f.approx_median(col_speed).alias("Median Speed"), f.approx_percentile_cont(col_speed, 0.9).alias("90% Speed")]) +``` -When :code:`group_by` is :code:`None` or an empty list, the aggregation is done over the whole -:class:`.DataFrame`. For grouping the :code:`group_by` list must contain at least one column. +When {code}`group_by` is {code}`None` or an empty list, the aggregation is done over the whole +{class}`.DataFrame`. For grouping the {code}`group_by` list must contain at least one column. +```{eval-rst} .. ipython:: python df.aggregate([col_type_1], [ f.max(col_speed).alias("Max Speed"), f.avg(col_speed).alias("Avg Speed"), f.min(col_speed).alias("Min Speed")]) +``` More than one column can be used for grouping +```{eval-rst} .. ipython:: python df.aggregate([col_type_1, col_type_2], [ @@ -61,28 +76,30 @@ More than one column can be used for grouping f.min(col_speed).alias("Min Speed")]) +``` -Setting Parameters ------------------- +## Setting Parameters Each of the built in aggregate functions provides arguments for the parameters that affect their operation. These can also be overridden using the builder approach to setting any of the following -parameters. When you use the builder, you must call ``build()`` to finish. For example, these two +parameters. When you use the builder, you must call `build()` to finish. For example, these two expressions are equivalent. +```{eval-rst} .. ipython:: python first_1 = f.first_value(col("a"), order_by=[col("a")]) first_2 = f.first_value(col("a")).order_by(col("a")).build() +``` -Ordering -^^^^^^^^ +### Ordering You can control the order in which rows are processed by window functions by providing -a list of ``order_by`` functions for the ``order_by`` parameter. In the following example, we +a list of `order_by` functions for the `order_by` parameter. In the following example, we sort the Pokemon by their attack in increasing order and take the first value, which gives us the -Pokemon with the smallest attack value in each ``Type 1``. +Pokemon with the smallest attack value in each `Type 1`. +```{eval-rst} .. ipython:: python df.aggregate( @@ -92,33 +109,36 @@ Pokemon with the smallest attack value in each ``Type 1``. order_by=[col('"Attack"').sort(ascending=True)] ).alias("Smallest Attack") ]) +``` -Distinct -^^^^^^^^ +### Distinct -When you set the parameter ``distinct`` to ``True``, then unique values will only be evaluated one -time each. Suppose we want to create an array of all of the ``Type 2`` for each ``Type 1`` of our -Pokemon set. Since there will be many entries of ``Type 2`` we only one each distinct value. +When you set the parameter `distinct` to `True`, then unique values will only be evaluated one +time each. Suppose we want to create an array of all of the `Type 2` for each `Type 1` of our +Pokemon set. Since there will be many entries of `Type 2` we only one each distinct value. +```{eval-rst} .. ipython:: python df.aggregate([col_type_1], [f.array_agg(col_type_2, distinct=True).alias("Type 2 List")]) +``` -In the output of the above we can see that there are some ``Type 1`` for which the ``Type 2`` entry -is ``null``. In reality, we probably want to filter those out. We can do this in two ways. First, -we can filter DataFrame rows that have no ``Type 2``. If we do this, we might have some ``Type 1`` -entries entirely removed. The second is we can use the ``filter`` argument described below. +In the output of the above we can see that there are some `Type 1` for which the `Type 2` entry +is `null`. In reality, we probably want to filter those out. We can do this in two ways. First, +we can filter DataFrame rows that have no `Type 2`. If we do this, we might have some `Type 1` +entries entirely removed. The second is we can use the `filter` argument described below. +```{eval-rst} .. ipython:: python df.filter(col_type_2.is_not_null()).aggregate([col_type_1], [f.array_agg(col_type_2, distinct=True).alias("Type 2 List")]) df.aggregate([col_type_1], [f.array_agg(col_type_2, distinct=True, filter=col_type_2.is_not_null()).alias("Type 2 List")]) +``` Which approach you take should depend on your use case. -Null Treatment -^^^^^^^^^^^^^^ +### Null Treatment This option allows you to either respect or ignore null values. @@ -126,7 +146,7 @@ One common usage for handling nulls is the case where you want to find the first partition. By setting the null treatment to ignore nulls, we can find the first non-null value in our partition. - +```{eval-rst} .. ipython:: python from datafusion.common import NullTreatment @@ -144,9 +164,9 @@ in our partition. order_by=[col_attack], null_treatment=NullTreatment.IGNORE_NULLS ).alias("Lowest Attack Type 2")]) +``` -Filter -^^^^^^ +### Filter Using the filter option is useful for filtering results to include in the aggregate function. It can be seen in the example above on how this can be useful to only filter rows evaluated by the @@ -156,24 +176,25 @@ Filter takes a single expression. Suppose we want to find the speed values for only Pokemon that have low Attack values. +```{eval-rst} .. ipython:: python df.aggregate([col_type_1], [ f.avg(col_speed).alias("Avg Speed All"), f.avg(col_speed, filter=col_attack < lit(50)).alias("Avg Speed Low Attack")]) +``` -Comparing subsets within a group -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +### Comparing subsets within a group Sometimes you need to compare the full membership of a group against a subset that meets some condition — for example, "which groups have at least -one failure, but not every member failed?". The ``filter`` argument on an +one failure, but not every member failed?". The `filter` argument on an aggregate restricts the rows that contribute to *that* aggregate without dropping the group, so a single pass can produce both the full set and the filtered subset side by side. Pairing -:py:func:`~datafusion.functions.array_agg` with ``distinct=True`` and -``filter=`` is a compact way to express this: collect the distinct values +{py:func}`~datafusion.functions.array_agg` with `distinct=True` and +`filter=` is a compact way to express this: collect the distinct values of the group, collect the distinct values that satisfy the condition, then compare the two arrays. @@ -182,6 +203,7 @@ a flag for whether that supplier met the commit date. We want to identify *partially failed* orders — orders where at least one supplier failed but not every supplier failed: +```{eval-rst} .. ipython:: python orders_df = ctx.from_pydict( @@ -208,13 +230,13 @@ not every supplier failed: (f.array_length(col("failed_suppliers")) > lit(0)) & (f.array_length(col("failed_suppliers")) < f.array_length(col("all_suppliers"))) ).select(col("order_id"), col("failed_suppliers")) +``` Order 1 is partial (one of three suppliers failed). Order 2 is excluded because no supplier failed, order 3 because its only supplier failed, and order 4 because both of its suppliers failed. -Grouping Sets -------------- +## Grouping Sets The default style of aggregation produces one row per group. Sometimes you want a single query to produce rows at multiple levels of detail — for example, totals per type *and* an overall grand @@ -223,28 +245,28 @@ separate queries and concatenating them is tedious and runs the data multiple ti solve this by letting you specify several grouping levels in one pass. DataFusion supports three grouping set styles through the -:py:class:`~datafusion.expr.GroupingSet` class: +{py:class}`~datafusion.expr.GroupingSet` class: -- :py:meth:`~datafusion.expr.GroupingSet.rollup` — hierarchical subtotals, like a drill-down report -- :py:meth:`~datafusion.expr.GroupingSet.cube` — every possible subtotal combination, like a pivot table -- :py:meth:`~datafusion.expr.GroupingSet.grouping_sets` — explicitly list exactly which grouping levels you want +- {py:meth}`~datafusion.expr.GroupingSet.rollup` — hierarchical subtotals, like a drill-down report +- {py:meth}`~datafusion.expr.GroupingSet.cube` — every possible subtotal combination, like a pivot table +- {py:meth}`~datafusion.expr.GroupingSet.grouping_sets` — explicitly list exactly which grouping levels you want Because result rows come from different grouping levels, a column that is *not* part of a -particular level will be ``null`` in that row. Use :py:func:`~datafusion.functions.grouping` to -distinguish a real ``null`` in the data from one that means "this column was aggregated across." -It returns ``0`` when the column is a grouping key for that row, and ``1`` when it is not. +particular level will be `null` in that row. Use {py:func}`~datafusion.functions.grouping` to +distinguish a real `null` in the data from one that means "this column was aggregated across." +It returns `0` when the column is a grouping key for that row, and `1` when it is not. -Rollup -^^^^^^ +### Rollup -:py:meth:`~datafusion.expr.GroupingSet.rollup` creates a hierarchy. ``rollup(a, b)`` produces -grouping sets ``(a, b)``, ``(a)``, and ``()`` — like nested subtotals in a report. This is useful +{py:meth}`~datafusion.expr.GroupingSet.rollup` creates a hierarchy. `rollup(a, b)` produces +grouping sets `(a, b)`, `(a)`, and `()` — like nested subtotals in a report. This is useful when your columns have a natural hierarchy, such as region → city or type → subtype. -Suppose we want to summarize Pokemon stats by ``Type 1`` with subtotals and a grand total. With -the default aggregation style we would need two separate queries. With ``rollup`` we get it all at +Suppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With +the default aggregation style we would need two separate queries. With `rollup` we get it all at once: +```{eval-rst} .. ipython:: python from datafusion.expr import GroupingSet @@ -255,25 +277,27 @@ once: f.avg(col_speed).alias("Avg Speed"), f.max(col_speed).alias("Max Speed")] ).sort(col_type_1.sort(ascending=True, nulls_first=True)) - -The first row — where ``Type 1`` is ``null`` — is the grand total across all types. But how do you -tell a grand-total ``null`` apart from a Pokemon that genuinely has no type? The -:py:func:`~datafusion.functions.grouping` function returns ``0`` when the column is a grouping key -for that row and ``1`` when it is aggregated across. - -.. note:: - - Due to an upstream DataFusion limitation - (`apache/datafusion#21411 `_), - ``.alias()`` cannot be applied directly to a ``grouping()`` expression — it will raise an - error at execution time. Instead, use - :py:meth:`~datafusion.dataframe.DataFrame.with_column_renamed` on the result DataFrame to - give the column a readable name. Once the upstream issue is resolved, you will be able to - use ``.alias()`` directly and the workaround below will no longer be necessary. - -The raw column name generated by ``grouping()`` contains internal identifiers, so we use -:py:meth:`~datafusion.dataframe.DataFrame.with_column_renamed` to clean it up: - +``` + +The first row — where `Type 1` is `null` — is the grand total across all types. But how do you +tell a grand-total `null` apart from a Pokemon that genuinely has no type? The +{py:func}`~datafusion.functions.grouping` function returns `0` when the column is a grouping key +for that row and `1` when it is aggregated across. + +:::{note} +Due to an upstream DataFusion limitation +([apache/datafusion#21411](https://github.com/apache/datafusion/issues/21411)), +`.alias()` cannot be applied directly to a `grouping()` expression — it will raise an +error at execution time. Instead, use +{py:meth}`~datafusion.dataframe.DataFrame.with_column_renamed` on the result DataFrame to +give the column a readable name. Once the upstream issue is resolved, you will be able to +use `.alias()` directly and the workaround below will no longer be necessary. +::: + +The raw column name generated by `grouping()` contains internal identifiers, so we use +{py:meth}`~datafusion.dataframe.DataFrame.with_column_renamed` to clean it up: + +```{eval-rst} .. ipython:: python result = df.aggregate( @@ -286,13 +310,15 @@ The raw column name generated by ``grouping()`` contains internal identifiers, s if field.name.startswith("grouping("): result = result.with_column_renamed(field.name, "Is Total") result.sort(col_type_1.sort(ascending=True, nulls_first=True)) +``` -With two columns the hierarchy becomes more apparent. ``rollup(Type 1, Type 2)`` produces: +With two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces: -- one row per ``(Type 1, Type 2)`` pair — the most detailed level -- one row per ``Type 1`` — subtotals +- one row per `(Type 1, Type 2)` pair — the most detailed level +- one row per `Type 1` — subtotals - one grand total row +```{eval-rst} .. ipython:: python df.aggregate( @@ -303,18 +329,19 @@ With two columns the hierarchy becomes more apparent. ``rollup(Type 1, Type 2)`` col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True) ) +``` -Cube -^^^^ +### Cube -:py:meth:`~datafusion.expr.GroupingSet.cube` produces every possible subset. ``cube(a, b)`` -produces grouping sets ``(a, b)``, ``(a)``, ``(b)``, and ``()`` — one more than ``rollup`` because -it also includes ``(b)`` alone. This is useful when neither column is "above" the other in a +{py:meth}`~datafusion.expr.GroupingSet.cube` produces every possible subset. `cube(a, b)` +produces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` — one more than `rollup` because +it also includes `(b)` alone. This is useful when neither column is "above" the other in a hierarchy and you want all cross-tabulations. -For our Pokemon data, ``cube(Type 1, Type 2)`` gives us stats broken down by the type pair, -by ``Type 1`` alone, by ``Type 2`` alone, and a grand total — all in one query: +For our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair, +by `Type 1` alone, by `Type 2` alone, and a grand total — all in one query: +```{eval-rst} .. ipython:: python df.aggregate( @@ -325,20 +352,21 @@ by ``Type 1`` alone, by ``Type 2`` alone, and a grand total — all in one query col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True) ) +``` -Compared to the ``rollup`` example above, notice the extra rows where ``Type 1`` is ``null`` but -``Type 2`` has a value — those are the per-``Type 2`` subtotals that ``rollup`` does not include. +Compared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but +`Type 2` has a value — those are the per-`Type 2` subtotals that `rollup` does not include. -Explicit Grouping Sets -^^^^^^^^^^^^^^^^^^^^^^ +### Explicit Grouping Sets -:py:meth:`~datafusion.expr.GroupingSet.grouping_sets` lets you list exactly which grouping levels -you need when ``rollup`` or ``cube`` would produce too many or too few. Each argument is a list of +{py:meth}`~datafusion.expr.GroupingSet.grouping_sets` lets you list exactly which grouping levels +you need when `rollup` or `cube` would produce too many or too few. Each argument is a list of columns forming one grouping set. -For example, if we want only the per-``Type 1`` totals and per-``Type 2`` totals — but *not* the -full ``(Type 1, Type 2)`` detail rows or the grand total — we can ask for exactly that: +For example, if we want only the per-`Type 1` totals and per-`Type 2` totals — but *not* the +full `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that: +```{eval-rst} .. ipython:: python df.aggregate( @@ -349,10 +377,12 @@ full ``(Type 1, Type 2)`` detail rows or the grand total — we can ask for exac col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True) ) +``` -Each row belongs to exactly one grouping level. The :py:func:`~datafusion.functions.grouping` +Each row belongs to exactly one grouping level. The {py:func}`~datafusion.functions.grouping` function tells you which level each row comes from: +```{eval-rst} .. ipython:: python result = df.aggregate( @@ -370,85 +400,84 @@ function tells you which level each row comes from: col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True) ) +``` -Where ``grouping(Type 1)`` is ``0`` the row is a per-``Type 1`` total (and ``Type 2`` is ``null``). -Where ``grouping(Type 2)`` is ``0`` the row is a per-``Type 2`` total (and ``Type 1`` is ``null``). - +Where `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`). +Where `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`). -Aggregate Functions -------------------- +## Aggregate Functions The available aggregate functions are: -1. Comparison Functions - - :py:func:`datafusion.functions.min` - - :py:func:`datafusion.functions.max` -2. Math Functions - - :py:func:`datafusion.functions.sum` - - :py:func:`datafusion.functions.avg` - - :py:func:`datafusion.functions.median` -3. Array Functions - - :py:func:`datafusion.functions.array_agg` -4. Logical Functions - - :py:func:`datafusion.functions.bit_and` - - :py:func:`datafusion.functions.bit_or` - - :py:func:`datafusion.functions.bit_xor` - - :py:func:`datafusion.functions.bool_and` - - :py:func:`datafusion.functions.bool_or` -5. Statistical Functions - - :py:func:`datafusion.functions.count` - - :py:func:`datafusion.functions.corr` - - :py:func:`datafusion.functions.covar_samp` - - :py:func:`datafusion.functions.covar_pop` - - :py:func:`datafusion.functions.stddev` - - :py:func:`datafusion.functions.stddev_pop` - - :py:func:`datafusion.functions.var_samp` - - :py:func:`datafusion.functions.var_pop` - - :py:func:`datafusion.functions.var_population` -6. Linear Regression Functions - - :py:func:`datafusion.functions.regr_count` - - :py:func:`datafusion.functions.regr_slope` - - :py:func:`datafusion.functions.regr_intercept` - - :py:func:`datafusion.functions.regr_r2` - - :py:func:`datafusion.functions.regr_avgx` - - :py:func:`datafusion.functions.regr_avgy` - - :py:func:`datafusion.functions.regr_sxx` - - :py:func:`datafusion.functions.regr_syy` - - :py:func:`datafusion.functions.regr_slope` -7. Positional Functions - - :py:func:`datafusion.functions.first_value` - - :py:func:`datafusion.functions.last_value` - - :py:func:`datafusion.functions.nth_value` -8. String Functions - - :py:func:`datafusion.functions.string_agg` -9. Percentile Functions - - :py:func:`datafusion.functions.percentile_cont` - - :py:func:`datafusion.functions.quantile_cont` - - :py:func:`datafusion.functions.approx_distinct` - - :py:func:`datafusion.functions.approx_median` - - :py:func:`datafusion.functions.approx_percentile_cont` - - :py:func:`datafusion.functions.approx_percentile_cont_with_weight` +01. Comparison Functions + : - {py:func}`datafusion.functions.min` + - {py:func}`datafusion.functions.max` +02. Math Functions + : - {py:func}`datafusion.functions.sum` + - {py:func}`datafusion.functions.avg` + - {py:func}`datafusion.functions.median` +03. Array Functions + : - {py:func}`datafusion.functions.array_agg` +04. Logical Functions + : - {py:func}`datafusion.functions.bit_and` + - {py:func}`datafusion.functions.bit_or` + - {py:func}`datafusion.functions.bit_xor` + - {py:func}`datafusion.functions.bool_and` + - {py:func}`datafusion.functions.bool_or` +05. Statistical Functions + : - {py:func}`datafusion.functions.count` + - {py:func}`datafusion.functions.corr` + - {py:func}`datafusion.functions.covar_samp` + - {py:func}`datafusion.functions.covar_pop` + - {py:func}`datafusion.functions.stddev` + - {py:func}`datafusion.functions.stddev_pop` + - {py:func}`datafusion.functions.var_samp` + - {py:func}`datafusion.functions.var_pop` + - {py:func}`datafusion.functions.var_population` +06. Linear Regression Functions + : - {py:func}`datafusion.functions.regr_count` + - {py:func}`datafusion.functions.regr_slope` + - {py:func}`datafusion.functions.regr_intercept` + - {py:func}`datafusion.functions.regr_r2` + - {py:func}`datafusion.functions.regr_avgx` + - {py:func}`datafusion.functions.regr_avgy` + - {py:func}`datafusion.functions.regr_sxx` + - {py:func}`datafusion.functions.regr_syy` + - {py:func}`datafusion.functions.regr_slope` +07. Positional Functions + : - {py:func}`datafusion.functions.first_value` + - {py:func}`datafusion.functions.last_value` + - {py:func}`datafusion.functions.nth_value` +08. String Functions + : - {py:func}`datafusion.functions.string_agg` +09. Percentile Functions + : - {py:func}`datafusion.functions.percentile_cont` + - {py:func}`datafusion.functions.quantile_cont` + - {py:func}`datafusion.functions.approx_distinct` + - {py:func}`datafusion.functions.approx_median` + - {py:func}`datafusion.functions.approx_percentile_cont` + - {py:func}`datafusion.functions.approx_percentile_cont_with_weight` 10. Grouping Set Functions - - :py:func:`datafusion.functions.grouping` - - :py:meth:`datafusion.expr.GroupingSet.rollup` - - :py:meth:`datafusion.expr.GroupingSet.cube` - - :py:meth:`datafusion.expr.GroupingSet.grouping_sets` + \- {py:func}`datafusion.functions.grouping` + \- {py:meth}`datafusion.expr.GroupingSet.rollup` + \- {py:meth}`datafusion.expr.GroupingSet.cube` + \- {py:meth}`datafusion.expr.GroupingSet.grouping_sets` -User-Defined Aggregate Functions --------------------------------- +## User-Defined Aggregate Functions You can ship custom aggregations to the engine by subclassing -:py:class:`~datafusion.user_defined.Accumulator` and registering it via -:py:func:`~datafusion.udaf`. See :py:mod:`datafusion.user_defined` for +{py:class}`~datafusion.user_defined.Accumulator` and registering it via +{py:func}`~datafusion.udaf`. See {py:mod}`datafusion.user_defined` for the accumulator interface and worked examples. -.. note:: Serialization - - Python aggregate UDFs travel inline inside pickled or - :py:meth:`~datafusion.expr.Expr.to_bytes`-serialized expressions — - the accumulator class is captured by value via :mod:`cloudpickle`, - so worker processes do not need to pre-register the UDF. Any names - the accumulator resolves via ``import`` are captured **by reference** - and must be importable on the receiving worker. See - :py:mod:`datafusion.ipc` for the full IPC model and security caveats. - +:::{note} +Serialization + +Python aggregate UDFs travel inline inside pickled or +{py:meth}`~datafusion.expr.Expr.to_bytes`-serialized expressions — +the accumulator class is captured by value via {mod}`cloudpickle`, +so worker processes do not need to pre-register the UDF. Any names +the accumulator resolves via `import` are captured **by reference** +and must be importable on the receiving worker. See +{py:mod}`datafusion.ipc` for the full IPC model and security caveats. +::: diff --git a/docs/source/user-guide/common-operations/basic-info.md b/docs/source/user-guide/common-operations/basic-info.md new file mode 100644 index 000000000..ed4816338 --- /dev/null +++ b/docs/source/user-guide/common-operations/basic-info.md @@ -0,0 +1,80 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Basic Operations + +In this section, you will learn how to display essential details of DataFrames using specific functions. + +```{eval-rst} +.. ipython:: python + + from datafusion import SessionContext + import random + + ctx = SessionContext() + df = ctx.from_pydict({ + "nrs": [1, 2, 3, 4, 5], + "names": ["python", "ruby", "java", "haskell", "go"], + "random": random.sample(range(1000), 5), + "groups": ["A", "A", "B", "C", "B"], + }) + df +``` + +Use {py:func}`~datafusion.dataframe.DataFrame.limit` to view the top rows of the frame: + +```{eval-rst} +.. ipython:: python + + df.limit(2) +``` + +Display the columns of the DataFrame using {py:func}`~datafusion.dataframe.DataFrame.schema`: + +```{eval-rst} +.. ipython:: python + + df.schema() +``` + +The method {py:func}`~datafusion.dataframe.DataFrame.to_pandas` uses pyarrow to convert to pandas DataFrame, by collecting the batches, +passing them to an Arrow table, and then converting them to a pandas DataFrame. + +```{eval-rst} +.. ipython:: python + + df.to_pandas() +``` + +{py:func}`~datafusion.dataframe.DataFrame.describe` shows a quick statistic summary of your data: + +```{eval-rst} +.. ipython:: python + + df.describe() +``` diff --git a/docs/source/user-guide/common-operations/basic-info.rst b/docs/source/user-guide/common-operations/basic-info.rst deleted file mode 100644 index d48b49d5c..000000000 --- a/docs/source/user-guide/common-operations/basic-info.rst +++ /dev/null @@ -1,61 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Basic Operations -================ - -In this section, you will learn how to display essential details of DataFrames using specific functions. - -.. ipython:: python - - from datafusion import SessionContext - import random - - ctx = SessionContext() - df = ctx.from_pydict({ - "nrs": [1, 2, 3, 4, 5], - "names": ["python", "ruby", "java", "haskell", "go"], - "random": random.sample(range(1000), 5), - "groups": ["A", "A", "B", "C", "B"], - }) - df - -Use :py:func:`~datafusion.dataframe.DataFrame.limit` to view the top rows of the frame: - -.. ipython:: python - - df.limit(2) - -Display the columns of the DataFrame using :py:func:`~datafusion.dataframe.DataFrame.schema`: - -.. ipython:: python - - df.schema() - -The method :py:func:`~datafusion.dataframe.DataFrame.to_pandas` uses pyarrow to convert to pandas DataFrame, by collecting the batches, -passing them to an Arrow table, and then converting them to a pandas DataFrame. - -.. ipython:: python - - df.to_pandas() - -:py:func:`~datafusion.dataframe.DataFrame.describe` shows a quick statistic summary of your data: - -.. ipython:: python - - df.describe() - diff --git a/docs/source/user-guide/common-operations/expressions.rst b/docs/source/user-guide/common-operations/expressions.md similarity index 68% rename from docs/source/user-guide/common-operations/expressions.rst rename to docs/source/user-guide/common-operations/expressions.md index f52c79ddb..008f1d75f 100644 --- a/docs/source/user-guide/common-operations/expressions.rst +++ b/docs/source/user-guide/common-operations/expressions.md @@ -1,74 +1,84 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at +% Licensed to the Apache Software Foundation (ASF) under one -.. http://www.apache.org/licenses/LICENSE-2.0 +% or more contributor license agreements. See the NOTICE file -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. +% distributed with this work for additional information -.. _expressions: +% regarding copyright ownership. The ASF licenses this file -Expressions -=========== +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(expressions)= + +# Expressions In DataFusion an expression is an abstraction that represents a computation. Expressions are used as the primary inputs and outputs for most functions within DataFusion. As such, expressions can be combined to create expression trees, a concept shared across most compilers and databases. -Column ------- +## Column -The first expression most new users will interact with is the Column, which is created by calling :py:func:`~datafusion.col`. -This expression represents a column within a DataFrame. The function :py:func:`~datafusion.col` takes as in input a string +The first expression most new users will interact with is the Column, which is created by calling {py:func}`~datafusion.col`. +This expression represents a column within a DataFrame. The function {py:func}`~datafusion.col` takes as in input a string and returns an expression as it's output. -Literal -------- +## Literal Literal expressions represent a single value. These are helpful in a wide range of operations where -a specific, known value is of interest. You can create a literal expression using the function :py:func:`~datafusion.lit`. -The type of the object passed to the :py:func:`~datafusion.lit` function will be used to convert it to a known data type. +a specific, known value is of interest. You can create a literal expression using the function {py:func}`~datafusion.lit`. +The type of the object passed to the {py:func}`~datafusion.lit` function will be used to convert it to a known data type. In the following example we create expressions for the column named `color` and the literal scalar string `red`. The resultant variable `red_units` is itself also an expression. +```{eval-rst} .. ipython:: python red_units = col("color") == lit("red") +``` -Boolean -------- +## Boolean When combining expressions that evaluate to a boolean value, you can combine these expressions using boolean operators. It is important to note that in order to combine these expressions, you *must* use bitwise operators. See the following examples for the and, or, and not operations. - +```{eval-rst} .. ipython:: python red_or_green_units = (col("color") == lit("red")) | (col("color") == lit("green")) heavy_red_units = (col("color") == lit("red")) & (col("weight") > lit(42)) not_red_units = ~(col("color") == lit("red")) +``` -Arrays ------- +## Arrays For columns that contain arrays of values, you can access individual elements of the array by index using bracket indexing. This is similar to calling the function -:py:func:`datafusion.functions.array_element`, except that array indexing using brackets is 0 based, -similar to Python arrays and ``array_element`` is 1 based indexing to be compatible with other SQL +{py:func}`datafusion.functions.array_element`, except that array indexing using brackets is 0 based, +similar to Python arrays and `array_element` is 1 based indexing to be compatible with other SQL approaches. +```{eval-rst} .. ipython:: python from datafusion import SessionContext, col @@ -76,22 +86,26 @@ approaches. ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5, 6]]}) df.select(col("a")[0].alias("a0")) +``` -.. warning:: - - Indexing an element of an array via ``[]`` starts at index 0 whereas - :py:func:`~datafusion.functions.array_element` starts at index 1. +:::{warning} +Indexing an element of an array via `[]` starts at index 0 whereas +{py:func}`~datafusion.functions.array_element` starts at index 1. +::: Starting in DataFusion 49.0.0 you can also create slices of array elements using slice syntax from Python. +```{eval-rst} .. ipython:: python df.select(col("a")[1:3].alias("second_two_elements")) +``` -To check if an array is empty, you can use the function :py:func:`datafusion.functions.array_empty` or `datafusion.functions.empty`. +To check if an array is empty, you can use the function {py:func}`datafusion.functions.array_empty` or `datafusion.functions.empty`. This function returns a boolean indicating whether the array is empty. +```{eval-rst} .. ipython:: python from datafusion import SessionContext, col @@ -100,12 +114,14 @@ This function returns a boolean indicating whether the array is empty. ctx = SessionContext() df = ctx.from_pydict({"a": [[], [1, 2, 3]]}) df.select(array_empty(col("a")).alias("is_empty")) +``` In this example, the `is_empty` column will contain `True` for the first row and `False` for the second row. -To get the total number of elements in an array, you can use the function :py:func:`datafusion.functions.cardinality`. +To get the total number of elements in an array, you can use the function {py:func}`datafusion.functions.cardinality`. This function returns an integer indicating the total number of elements in the array. +```{eval-rst} .. ipython:: python from datafusion import SessionContext, col @@ -114,12 +130,14 @@ This function returns an integer indicating the total number of elements in the ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5, 6]]}) df.select(cardinality(col("a")).alias("num_elements")) +``` In this example, the `num_elements` column will contain `3` for both rows. -To concatenate two arrays, you can use the function :py:func:`datafusion.functions.array_cat` or :py:func:`datafusion.functions.array_concat`. +To concatenate two arrays, you can use the function {py:func}`datafusion.functions.array_cat` or {py:func}`datafusion.functions.array_concat`. These functions return a new array that is the concatenation of the input arrays. +```{eval-rst} .. ipython:: python from datafusion import SessionContext, col @@ -128,12 +146,14 @@ These functions return a new array that is the concatenation of the input arrays ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3]], "b": [[4, 5, 6]]}) df.select(array_cat(col("a"), col("b")).alias("concatenated_array")) +``` In this example, the `concatenated_array` column will contain `[1, 2, 3, 4, 5, 6]`. -To repeat the elements of an array a specified number of times, you can use the function :py:func:`datafusion.functions.array_repeat`. +To repeat the elements of an array a specified number of times, you can use the function {py:func}`datafusion.functions.array_repeat`. This function returns a new array with the elements repeated. +```{eval-rst} .. ipython:: python from datafusion import SessionContext, col, literal @@ -142,23 +162,24 @@ This function returns a new array with the elements repeated. ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3]]}) df.select(array_repeat(col("a"), literal(2)).alias("repeated_array")) +``` In this example, the `repeated_array` column will contain `[[1, 2, 3], [1, 2, 3]]`. -Lambda functions ----------------- +## Lambda functions Some array functions take a *lambda function*: a small function that runs once -per element. :py:func:`~datafusion.functions.array_transform` maps a lambda over -every element, :py:func:`~datafusion.functions.array_filter` keeps the elements +per element. {py:func}`~datafusion.functions.array_transform` maps a lambda over +every element, {py:func}`~datafusion.functions.array_filter` keeps the elements for which a predicate lambda is true, and -:py:func:`~datafusion.functions.array_any_match` returns whether any element +{py:func}`~datafusion.functions.array_any_match` returns whether any element satisfies a predicate lambda. (Functions that take another function as an argument are sometimes called *higher-order* functions.) -The simplest way to supply a lambda is a Python ``lambda``. Its parameter names +The simplest way to supply a lambda is a Python `lambda`. Its parameter names become the lambda parameters, and its return value becomes the body. +```{eval-rst} .. ipython:: python from datafusion import SessionContext, col @@ -169,46 +190,48 @@ become the lambda parameters, and its return value becomes the body. df.select(f.array_transform(col("a"), lambda v: v * 2).alias("doubled")) df.select(f.array_filter(col("a"), lambda v: v > 2).alias("big_only")) df.select(f.array_any_match(col("a"), lambda v: v > 3).alias("has_big")) +``` If you need explicit control over parameter names, build the lambda with -:py:func:`~datafusion.functions.lambda_` and reference its parameters with -:py:func:`~datafusion.functions.lambda_var`. The following is equivalent to the -``array_transform`` call above. +{py:func}`~datafusion.functions.lambda_` and reference its parameters with +{py:func}`~datafusion.functions.lambda_var`. The following is equivalent to the +`array_transform` call above. +```{eval-rst} .. ipython:: python from datafusion import lit double_fn = f.lambda_(["v"], f.lambda_var("v") * lit(2)) df.select(f.array_transform(col("a"), double_fn).alias("doubled")) - -.. note:: - - Lambda expressions cannot yet be serialized: calling - :py:meth:`~datafusion.expr.Expr.to_bytes` or pickling an expression that - contains a lambda raises ``Lambda not implemented``. SQL lambda syntax is - only parsed by dialects that support lambdas; set - ``datafusion.sql_parser.dialect`` to one of ``DuckDB``, ``ClickHouse``, - ``Snowflake``, or ``Databricks``. Both arrow syntax (``x -> x * 2``) and - keyword syntax (``lambda x: x * 2``) parse. DuckDB will drop the arrow - form in v2.1, so prefer ``lambda x: x * 2`` for forward compatibility. - The Python expression builder shown above works regardless of dialect. - - -Testing membership in a list ----------------------------- +``` + +:::{note} +Lambda expressions cannot yet be serialized: calling +{py:meth}`~datafusion.expr.Expr.to_bytes` or pickling an expression that +contains a lambda raises `Lambda not implemented`. SQL lambda syntax is +only parsed by dialects that support lambdas; set +`datafusion.sql_parser.dialect` to one of `DuckDB`, `ClickHouse`, +`Snowflake`, or `Databricks`. Both arrow syntax (`x -> x * 2`) and +keyword syntax (`lambda x: x * 2`) parse. DuckDB will drop the arrow +form in v2.1, so prefer `lambda x: x * 2` for forward compatibility. +The Python expression builder shown above works regardless of dialect. +::: + +## Testing membership in a list A common need is filtering rows where a column equals *any* of a small set of values. DataFusion offers three forms; they differ in readability and in how they scale: -1. A compound boolean using ``|`` across explicit equalities. -2. :py:func:`~datafusion.functions.in_list`, which accepts a list of +1. A compound boolean using `|` across explicit equalities. +2. {py:func}`~datafusion.functions.in_list`, which accepts a list of expressions and tests equality against all of them in one call. -3. A trick with :py:func:`~datafusion.functions.array_position` and - :py:func:`~datafusion.functions.make_array`, which returns the 1-based +3. A trick with {py:func}`~datafusion.functions.array_position` and + {py:func}`~datafusion.functions.make_array`, which returns the 1-based index of the value in a constructed array, or null if it is not present. +```{eval-rst} .. ipython:: python from datafusion import SessionContext, col, lit @@ -230,22 +253,23 @@ they scale: f.make_array(lit("MAIL"), lit("SHIP")), col("shipmode") ).is_null() ) +``` -Use ``in_list`` as the default. It is explicit, readable, and matches the -semantics users expect from SQL's ``IN (...)``. Reach for the -``array_position`` form only when the membership set is itself an array +Use `in_list` as the default. It is explicit, readable, and matches the +semantics users expect from SQL's `IN (...)`. Reach for the +`array_position` form only when the membership set is itself an array column rather than a literal list. -Conditional expressions ------------------------ +## Conditional expressions -DataFusion provides :py:func:`~datafusion.functions.case` for the SQL -``CASE`` expression in both its switched and searched forms, along with -:py:func:`~datafusion.functions.when` as a standalone builder for the +DataFusion provides {py:func}`~datafusion.functions.case` for the SQL +`CASE` expression in both its switched and searched forms, along with +{py:func}`~datafusion.functions.when` as a standalone builder for the searched form. **Switched CASE** (one expression compared against several literal values): +```{eval-rst} .. ipython:: python df = ctx.from_pydict( @@ -260,11 +284,13 @@ searched form. .otherwise(lit(0)) .alias("is_high_priority"), ) +``` **Searched CASE** (an independent boolean predicate per branch). Use this form whenever a branch tests more than simple equality — for example, -checking whether a joined column is ``NULL`` to gate a computed value: +checking whether a joined column is `NULL` to gate a computed value: +```{eval-rst} .. ipython:: python df = ctx.from_pydict( @@ -278,21 +304,22 @@ checking whether a joined column is ``NULL`` to gate a computed value: .otherwise(lit(0.0)) .alias("attributed_volume"), ) +``` This searched-CASE pattern is idiomatic for "attribute the measure to the matching side of a left join, otherwise contribute zero" — a shape that appears in TPC-H Q08 and similar market-share calculations. If a switched CASE only groups several equality matches into one bucket, -``f.when(f.in_list(col(...), [...]), value).otherwise(default)`` is often -simpler than the full ``case`` builder. +`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often +simpler than the full `case` builder. -Structs -------- +## Structs Columns that contain struct elements can be accessed using the bracket notation as if they were Python dictionary style objects. This expects a string key as the parameter passed. +```{eval-rst} .. ipython:: python ctx = SessionContext() @@ -300,16 +327,17 @@ Python dictionary style objects. This expects a string key as the parameter pass df = ctx.from_pydict(data) df.select(col("a")["size"].alias("a_size")) +``` -Functions ---------- +## Functions As mentioned before, most functions in DataFusion return an expression at their output. This allows us to create -a wide variety of expressions built up from other expressions. For example, :py:func:`~datafusion.expr.Expr.alias` is a function that takes +a wide variety of expressions built up from other expressions. For example, {py:func}`~datafusion.expr.Expr.alias` is a function that takes as it input a single expression and returns an expression in which the name of the expression has changed. The following example shows a series of expressions that are built up from functions operating on expressions. +```{eval-rst} .. ipython:: python from datafusion import SessionContext @@ -335,3 +363,4 @@ The following example shows a series of expressions that are built up from funct long_timer = started_young & can_retire df.filter(long_timer).select(col("name"), renamed_age, col("years_in_position")) +``` diff --git a/docs/source/user-guide/common-operations/functions.rst b/docs/source/user-guide/common-operations/functions.md similarity index 52% rename from docs/source/user-guide/common-operations/functions.rst rename to docs/source/user-guide/common-operations/functions.md index ccb47a4e7..f57e53ecd 100644 --- a/docs/source/user-guide/common-operations/functions.rst +++ b/docs/source/user-guide/common-operations/functions.md @@ -1,28 +1,39 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Functions -========= +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Functions DataFusion provides a large number of built-in functions for performing complex queries without requiring user-defined functions. -In here we will cover some of the more popular use cases. If you want to view all the functions go to the :py:mod:`Functions ` API Reference. +In here we will cover some of the more popular use cases. If you want to view all the functions go to the {py:mod}`Functions ` API Reference. We'll use the pokemon dataset in the following examples. +```{eval-rst} .. ipython:: python from datafusion import SessionContext @@ -30,12 +41,13 @@ We'll use the pokemon dataset in the following examples. ctx = SessionContext() ctx.register_csv("pokemon", "pokemon.csv") df = ctx.table("pokemon") +``` -Mathematical ------------- +## Mathematical -DataFusion offers mathematical functions such as :py:func:`~datafusion.functions.pow` or :py:func:`~datafusion.functions.log` +DataFusion offers mathematical functions such as {py:func}`~datafusion.functions.pow` or {py:func}`~datafusion.functions.log` +```{eval-rst} .. ipython:: python from datafusion import col, literal, string_literal, str_lit @@ -45,48 +57,55 @@ DataFusion offers mathematical functions such as :py:func:`~datafusion.functions f.pow(col('"Attack"'), literal(2)) - f.pow(col('"Defense"'), literal(2)) ).limit(10) +``` -Conditional ------------ +## Conditional -There 3 conditional functions in DataFusion :py:func:`~datafusion.functions.coalesce`, :py:func:`~datafusion.functions.nullif` and :py:func:`~datafusion.functions.case`. +There 3 conditional functions in DataFusion {py:func}`~datafusion.functions.coalesce`, {py:func}`~datafusion.functions.nullif` and {py:func}`~datafusion.functions.case`. +```{eval-rst} .. ipython:: python df.select( f.coalesce(col('"Type 1"'), col('"Type 2"')).alias("dominant_type") ).limit(10) +``` -Temporal --------- +## Temporal -For selecting the current time use :py:func:`~datafusion.functions.now` +For selecting the current time use {py:func}`~datafusion.functions.now` +```{eval-rst} .. ipython:: python df.select(f.now()) +``` -Convert to timestamps using :py:func:`~datafusion.functions.to_timestamp` +Convert to timestamps using {py:func}`~datafusion.functions.to_timestamp` +```{eval-rst} .. ipython:: python df.select(f.to_timestamp(col('"Total"')).alias("timestamp")) +``` -Extracting parts of a date using :py:func:`~datafusion.functions.date_part` (alias :py:func:`~datafusion.functions.extract`) +Extracting parts of a date using {py:func}`~datafusion.functions.date_part` (alias {py:func}`~datafusion.functions.extract`) +```{eval-rst} .. ipython:: python df.select( f.date_part(literal("month"), f.to_timestamp(col('"Total"'))).alias("month"), f.extract(literal("day"), f.to_timestamp(col('"Total"'))).alias("day") ) - -String ------- +``` + +## String In the field of data science, working with textual data is a common task. To make string manipulation easier, DataFusion offers a range of helpful options. +```{eval-rst} .. ipython:: python df.select( @@ -94,33 +113,37 @@ DataFusion offers a range of helpful options. f.lower(col('"Name"')).alias("lower"), f.left(col('"Name"'), literal(4)).alias("code") ) +``` -This also includes the functions for regular expressions like :py:func:`~datafusion.functions.regexp_replace` and :py:func:`~datafusion.functions.regexp_match` +This also includes the functions for regular expressions like {py:func}`~datafusion.functions.regexp_replace` and {py:func}`~datafusion.functions.regexp_match` +```{eval-rst} .. ipython:: python df.select( f.regexp_match(col('"Name"'), literal("Char")).alias("dragons"), f.regexp_replace(col('"Name"'), literal("saur"), literal("fleur")).alias("flowers") ) +``` -Casting -------- +## Casting -Casting expressions to different data types using :py:func:`~datafusion.functions.arrow_cast` +Casting expressions to different data types using {py:func}`~datafusion.functions.arrow_cast` +```{eval-rst} .. ipython:: python df.select( f.arrow_cast(col('"Total"'), string_literal("Float64")).alias("total_as_float"), f.arrow_cast(col('"Total"'), str_lit("Int32")).alias("total_as_int") ) +``` -Other ------ +## Other -The function :py:func:`~datafusion.functions.in_list` allows to check a column for the presence of multiple values: +The function {py:func}`~datafusion.functions.in_list` allows to check a column for the presence of multiple values: +```{eval-rst} .. ipython:: python types = [literal("Grass"), literal("Fire"), literal("Water")] @@ -130,23 +153,22 @@ The function :py:func:`~datafusion.functions.in_list` allows to check a column f .to_pandas() ) +``` -Handling Missing Values -======================= +# Handling Missing Values DataFusion provides methods to handle missing values in DataFrames: -fill_null ---------- - -The ``fill_null()`` method replaces NULL values in specified columns with a provided value: +## fill_null -.. code-block:: python +The `fill_null()` method replaces NULL values in specified columns with a provided value: - # Fill all NULL values with 0 where possible - df = df.fill_null(0) +```python +# Fill all NULL values with 0 where possible +df = df.fill_null(0) - # Fill NULL values only in specific string columns - df = df.fill_null("missing", subset=["name", "category"]) +# Fill NULL values only in specific string columns +df = df.fill_null("missing", subset=["name", "category"]) +``` The fill value will be cast to match each column's type. If casting fails for a column, that column remains unchanged. diff --git a/docs/source/user-guide/common-operations/index.md b/docs/source/user-guide/common-operations/index.md new file mode 100644 index 000000000..58947844c --- /dev/null +++ b/docs/source/user-guide/common-operations/index.md @@ -0,0 +1,45 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Common Operations + +The contents of this section are designed to guide a new user through how to use DataFusion. + +```{toctree} +:maxdepth: 2 + +views +basic-info +select-and-filter +expressions +joins +functions +aggregations +windows +udf-and-udfa +``` diff --git a/docs/source/user-guide/common-operations/index.rst b/docs/source/user-guide/common-operations/index.rst deleted file mode 100644 index 7abd1f138..000000000 --- a/docs/source/user-guide/common-operations/index.rst +++ /dev/null @@ -1,34 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Common Operations -================= - -The contents of this section are designed to guide a new user through how to use DataFusion. - -.. toctree:: - :maxdepth: 2 - - views - basic-info - select-and-filter - expressions - joins - functions - aggregations - windows - udf-and-udfa diff --git a/docs/source/user-guide/common-operations/joins.rst b/docs/source/user-guide/common-operations/joins.md similarity index 67% rename from docs/source/user-guide/common-operations/joins.rst rename to docs/source/user-guide/common-operations/joins.md index a289c9377..bcbd63613 100644 --- a/docs/source/user-guide/common-operations/joins.rst +++ b/docs/source/user-guide/common-operations/joins.md @@ -1,24 +1,34 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at +% Licensed to the Apache Software Foundation (ASF) under one -.. http://www.apache.org/licenses/LICENSE-2.0 +% or more contributor license agreements. See the NOTICE file -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. +% distributed with this work for additional information -Joins -===== +% regarding copyright ownership. The ASF licenses this file -DataFusion supports the following join variants via the method :py:func:`~datafusion.dataframe.DataFrame.join` +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Joins + +DataFusion supports the following join variants via the method {py:func}`~datafusion.dataframe.DataFrame.join` - Inner Join - Left Join @@ -29,6 +39,7 @@ DataFusion supports the following join variants via the method :py:func:`~datafu For the examples in this section we'll use the following two DataFrames +```{eval-rst} .. ipython:: python from datafusion import SessionContext @@ -47,71 +58,77 @@ For the examples in this section we'll use the following two DataFrames {"id": 2, "name": "MetroRide"}, {"id": 5, "name": "UrbanGo"}, ]) +``` -Inner Join ----------- +## Inner Join When using an inner join, only rows containing the common values between the two join columns present in both DataFrames will be included in the resulting DataFrame. +```{eval-rst} .. ipython:: python left.join(right, left_on="customer_id", right_on="id", how="inner") +``` -The parameter ``join_keys`` specifies the columns from the left DataFrame and right DataFrame that contains the values +The parameter `join_keys` specifies the columns from the left DataFrame and right DataFrame that contains the values that should match. -Left Join ---------- +## Left Join A left join combines rows from two DataFrames using the key columns. It returns all rows from the left DataFrame and matching rows from the right DataFrame. If there's no match in the right DataFrame, it returns null values for the corresponding columns. +```{eval-rst} .. ipython:: python left.join(right, left_on="customer_id", right_on="id", how="left") +``` -Full Join ---------- +## Full Join A full join merges rows from two tables based on a related column, returning all rows from both tables, even if there is no match. Unmatched rows will have null values. +```{eval-rst} .. ipython:: python left.join(right, left_on="customer_id", right_on="id", how="full") +``` -Left Semi Join --------------- +## Left Semi Join A left semi join retrieves matching rows from the left table while omitting duplicates with multiple matches in the right table. +```{eval-rst} .. ipython:: python left.join(right, left_on="customer_id", right_on="id", how="semi") +``` -Left Anti Join --------------- +## Left Anti Join A left anti join shows all rows from the left table without any matching rows in the right table, based on a the specified matching columns. It excludes rows from the left table that have at least one matching row in the right table. +```{eval-rst} .. ipython:: python left.join(right, left_on="customer_id", right_on="id", how="anti") +``` -Duplicate Keys --------------- +## Duplicate Keys It is common to join two DataFrames on a common column name. Starting in -version 51.0.0, ``datafusion-python``` will now coalesce on column with identical names by +version 51.0.0, `` datafusion-python` `` will now coalesce on column with identical names by default. This reduces problems with ambiguous column selection after joins. -You can disable this feature by setting the parameter ``coalesce_duplicate_keys`` -to ``False``. +You can disable this feature by setting the parameter `coalesce_duplicate_keys` +to `False`. +```{eval-rst} .. ipython:: python left = ctx.from_pydict( @@ -128,24 +145,27 @@ to ``False``. ]) left.join(right, "id", how="inner") +``` In contrast to the above example, if we wish to get both columns: +```{eval-rst} .. ipython:: python left.join(right, "id", how="inner", coalesce_duplicate_keys=False) +``` -Disambiguating Columns with ``DataFrame.col()`` ------------------------------------------------- +## Disambiguating Columns with `DataFrame.col()` When both DataFrames contain non-key columns with the same name, you can use -:py:meth:`~datafusion.dataframe.DataFrame.col` on each DataFrame **before** the +{py:meth}`~datafusion.dataframe.DataFrame.col` on each DataFrame **before** the join to create fully qualified column references. These references can then be used in the join predicate and when selecting from the result. -This is especially useful with :py:meth:`~datafusion.dataframe.DataFrame.join_on`, +This is especially useful with {py:meth}`~datafusion.dataframe.DataFrame.join_on`, which accepts expression-based predicates. +```{eval-rst} .. ipython:: python left = ctx.from_pydict( @@ -167,3 +187,4 @@ which accepts expression-based predicates. ) joined.select(left.col("id"), left.col("val"), right.col("val")) +``` diff --git a/docs/source/user-guide/common-operations/select-and-filter.md b/docs/source/user-guide/common-operations/select-and-filter.md new file mode 100644 index 000000000..61de45814 --- /dev/null +++ b/docs/source/user-guide/common-operations/select-and-filter.md @@ -0,0 +1,80 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Column Selections + +Use {py:func}`~datafusion.dataframe.DataFrame.select` for basic column selection. + +DataFusion can work with several file types, to start simple we can use a subset of the +[TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page), +which you can download [here](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet). + +```{eval-rst} +.. ipython:: python + + from datafusion import SessionContext + + ctx = SessionContext() + df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") + df.select("trip_distance", "passenger_count") +``` + +For mathematical or logical operations use {py:func}`~datafusion.col` to select columns, and give meaningful names to the resulting +operations using {py:func}`~datafusion.expr.Expr.alias` + +```{eval-rst} +.. ipython:: python + + from datafusion import col, lit + df.select((col("tip_amount") + col("tolls_amount")).alias("tips_plus_tolls")) +``` + +:::{warning} +Please be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters +(ex: Name) you must put your column name in double quotes or the selection won’t work. As an alternative for simple +column selection use {py:func}`~datafusion.dataframe.DataFrame.select` without double quotes +::: + +For selecting columns with capital letters use `'"VendorID"'` + +```{eval-rst} +.. ipython:: python + + df.select(col('"VendorID"')) + +``` + +To combine it with literal values use the {py:func}`~datafusion.lit` + +```{eval-rst} +.. ipython:: python + + large_trip_distance = col("trip_distance") > lit(5.0) + low_passenger_count = col("passenger_count") < lit(4) + df.select((large_trip_distance & low_passenger_count).alias("lonely_trips")) +``` diff --git a/docs/source/user-guide/common-operations/select-and-filter.rst b/docs/source/user-guide/common-operations/select-and-filter.rst deleted file mode 100644 index 083bcbbd2..000000000 --- a/docs/source/user-guide/common-operations/select-and-filter.rst +++ /dev/null @@ -1,64 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Column Selections -================= - -Use :py:func:`~datafusion.dataframe.DataFrame.select` for basic column selection. - -DataFusion can work with several file types, to start simple we can use a subset of the -`TLC Trip Record Data `_, -which you can download `here `_. - -.. ipython:: python - - from datafusion import SessionContext - - ctx = SessionContext() - df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") - df.select("trip_distance", "passenger_count") - -For mathematical or logical operations use :py:func:`~datafusion.col` to select columns, and give meaningful names to the resulting -operations using :py:func:`~datafusion.expr.Expr.alias` - - -.. ipython:: python - - from datafusion import col, lit - df.select((col("tip_amount") + col("tolls_amount")).alias("tips_plus_tolls")) - -.. warning:: - - Please be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters - (ex: Name) you must put your column name in double quotes or the selection won’t work. As an alternative for simple - column selection use :py:func:`~datafusion.dataframe.DataFrame.select` without double quotes - -For selecting columns with capital letters use ``'"VendorID"'`` - -.. ipython:: python - - df.select(col('"VendorID"')) - - -To combine it with literal values use the :py:func:`~datafusion.lit` - -.. ipython:: python - - large_trip_distance = col("trip_distance") > lit(5.0) - low_passenger_count = col("passenger_count") < lit(4) - df.select((large_trip_distance & low_passenger_count).alias("lonely_trips")) - diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.rst b/docs/source/user-guide/common-operations/udf-and-udfa.md similarity index 54% rename from docs/source/user-guide/common-operations/udf-and-udfa.rst rename to docs/source/user-guide/common-operations/udf-and-udfa.md index 918c2e29e..d673aaa28 100644 --- a/docs/source/user-guide/common-operations/udf-and-udfa.rst +++ b/docs/source/user-guide/common-operations/udf-and-udfa.md @@ -1,39 +1,49 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -User-Defined Functions -====================== +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# User-Defined Functions DataFusion provides powerful expressions and functions, reducing the need for custom Python functions. However you can still incorporate your own functions, i.e. User-Defined Functions (UDFs). -Scalar Functions ----------------- +## Scalar Functions When writing a user-defined function that can operate on a row by row basis, these are called Scalar Functions. You can define your own scalar function by calling -:py:func:`~datafusion.user_defined.ScalarUDF.udf` . +{py:func}`~datafusion.user_defined.ScalarUDF.udf` . The basic definition of a scalar UDF is a python function that takes one or more -`pyarrow `_ arrays and returns a single array as +[pyarrow](https://arrow.apache.org/docs/python/index.html) arrays and returns a single array as output. DataFusion scalar UDFs operate on an entire batch of records at a time, though the evaluation of those records should be on a row by row basis. In the following example, we compute if the input array contains null values. +```{eval-rst} .. ipython:: python import pyarrow @@ -54,10 +64,11 @@ if the input array contains null values. df = ctx.create_dataframe([[batch]], name="batch_array") df.select(col("a"), is_null_arr(col("a")).alias("is_null")).show() +``` In the previous example, we used the fact that pyarrow provides a variety of built in array -functions such as ``is_null()``. There are additional pyarrow -`compute functions `_ available. When possible, +functions such as `is_null()`. There are additional pyarrow +[compute functions](https://arrow.apache.org/docs/python/compute.html) available. When possible, it is highly recommended to use these functions because they can perform computations without doing any copy operations from the original arrays. This leads to greatly improved performance. @@ -66,9 +77,10 @@ functions, you will need to convert the record batch into python values, perform and construct an array. This operation of converting the built in data type of the array into a python object can be one of the slowest operations in DataFusion, so it should be done sparingly. -The following example performs the same operation as before with ``is_null`` but demonstrates +The following example performs the same operation as before with `is_null` but demonstrates converting to Python objects to do the evaluation. +```{eval-rst} .. ipython:: python import pyarrow @@ -89,24 +101,24 @@ converting to Python objects to do the evaluation. df = ctx.create_dataframe([[batch]], name="batch_array") df.select(col("a"), is_null_arr(col("a")).alias("is_null")).show() +``` -In this example we passed the PyArrow ``DataType`` when we defined the function -by calling ``udf()``. If you need additional control, such as specifying +In this example we passed the PyArrow `DataType` when we defined the function +by calling `udf()`. If you need additional control, such as specifying metadata or nullability of the input or output, you can instead specify a -PyArrow ``Field``. +PyArrow `Field`. If you need to write a custom function but do not want to incur the performance cost of converting to Python objects and back, a more advanced approach is to write Rust based UDFs and to expose them to Python. There is an example in the -`DataFusion blog `_ +[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/) describing how to do this. -When not to use a UDF -^^^^^^^^^^^^^^^^^^^^^ +### When not to use a UDF A UDF is the right tool when the per-row computation genuinely cannot be expressed with DataFusion's built-in expressions. It is often the *wrong* -tool for a predicate that *can* be written as an ``Expr`` tree but feels +tool for a predicate that *can* be written as an `Expr` tree but feels easier to write as a Python function — for example, a filter that keeps a row if it matches any one of several rule sets, where each rule set checks its own combination of columns (the worked example at the end of @@ -127,6 +139,7 @@ ways: first with a native expression, then with a UDF that computes the same result. The filter itself is simple on purpose so we can compare the plans side by side. +```{eval-rst} .. ipython:: python import tempfile, os @@ -147,31 +160,35 @@ the plans side by side. ctx = SessionContext() items = ctx.read_parquet(parquet_path) +``` **Native-expression predicate.** The filter is a plain boolean tree over column references and literals, so the optimizer can analyze it: +```{eval-rst} .. ipython:: python native_filtered = items.filter( (col("brand") == lit("A")) & (col("qty") >= lit(150)) ) print(native_filtered.execution_plan().display_indent()) +``` -Notice the ``DataSourceExec`` line. It carries three annotations the +Notice the `DataSourceExec` line. It carries three annotations the optimizer computed from the predicate: -- ``predicate=brand@1 = A AND qty@2 >= 150`` — the filter is pushed +- `predicate=brand@1 = A AND qty@2 >= 150` — the filter is pushed into the Parquet scan itself, so the scan only reads matching rows. -- ``pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ... - qty_max@4 >= 150`` — the scan prunes whole row groups by consulting +- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ... + qty_max@4 >= 150` — the scan prunes whole row groups by consulting the Parquet min/max statistics in the footer *before* reading any column data. -- ``required_guarantees=[brand in (A)]`` — the scan uses this when a +- `required_guarantees=[brand in (A)]` — the scan uses this when a bloom filter or dictionary is available to skip pages. **UDF predicate.** Now wrap the same logic in a Python UDF: +```{eval-rst} .. ipython:: python def brand_qty_filter(brand_arr: pa.Array, qty_arr: pa.Array) -> pa.Array: @@ -185,9 +202,10 @@ optimizer computed from the predicate: ) udf_filtered = items.filter(pred_udf(col("brand"), col("qty"))) print(udf_filtered.execution_plan().display_indent()) +``` -The ``DataSourceExec`` now carries only ``predicate=brand_qty_filter(...)``. -There is no ``pruning_predicate`` and no ``required_guarantees``: the +The `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`. +There is no `pruning_predicate` and no `required_guarantees`: the scan has to materialize every row group and hand each row to the Python callback just to decide whether to keep it. @@ -199,106 +217,104 @@ reads all of it. **Takeaway.** Reach for a UDF when the per-row computation is genuinely not expressible as a tree of built-in functions (custom numerical work, external lookups, complex business rules). When it *is* expressible — -even if the native form is a little more verbose — build the ``Expr`` +even if the native form is a little more verbose — build the `Expr` tree directly so the optimizer can see through it. For disjunctive predicates the idiom is to produce one clause per bucket and combine -them with ``|``: - -.. code-block:: python - - from functools import reduce - from operator import or_ - from datafusion import col, lit, functions as f - - buckets = { - "Brand#12": {"containers": ["SM CASE", "SM BOX"], "min_qty": 1, "max_size": 5}, - "Brand#23": {"containers": ["MED BAG", "MED BOX"], "min_qty": 10, "max_size": 10}, - } - - def bucket_clause(brand, spec): - return ( - (col("brand") == lit(brand)) - & f.in_list(col("container"), [lit(c) for c in spec["containers"]]) - & (col("quantity") >= lit(spec["min_qty"])) - & (col("quantity") <= lit(spec["min_qty"] + 10)) - & (col("size") >= lit(1)) - & (col("size") <= lit(spec["max_size"])) - ) +them with `|`: + +```python +from functools import reduce +from operator import or_ +from datafusion import col, lit, functions as f + +buckets = { + "Brand#12": {"containers": ["SM CASE", "SM BOX"], "min_qty": 1, "max_size": 5}, + "Brand#23": {"containers": ["MED BAG", "MED BOX"], "min_qty": 10, "max_size": 10}, +} + +def bucket_clause(brand, spec): + return ( + (col("brand") == lit(brand)) + & f.in_list(col("container"), [lit(c) for c in spec["containers"]]) + & (col("quantity") >= lit(spec["min_qty"])) + & (col("quantity") <= lit(spec["min_qty"] + 10)) + & (col("size") >= lit(1)) + & (col("size") <= lit(spec["max_size"])) + ) - predicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items())) - df = df.filter(predicate) +predicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items())) +df = df.filter(predicate) +``` -Aggregate Functions -------------------- +## Aggregate Functions -The :py:func:`~datafusion.user_defined.AggregateUDF.udaf` function allows you to define User-Defined +The {py:func}`~datafusion.user_defined.AggregateUDF.udaf` function allows you to define User-Defined Aggregate Functions (UDAFs). To use this you must implement an -:py:class:`~datafusion.user_defined.Accumulator` that determines how the aggregation is performed. +{py:class}`~datafusion.user_defined.Accumulator` that determines how the aggregation is performed. -When defining a UDAF there are four methods you need to implement. The ``update`` function takes the +When defining a UDAF there are four methods you need to implement. The `update` function takes the array(s) of input and updates the internal state of the accumulator. You should define this function to have as many input arguments as you will pass when calling the UDAF. Since aggregation may be split into multiple batches, we must have a method to combine multiple batches. For this, we have -two functions, ``state`` and ``merge``. ``state`` will return an array of scalar values that contain -the current state of a single batch accumulation. Then we must ``merge`` the results of these -different states. Finally ``evaluate`` is the call that will return the final result after the -``merge`` is complete. +two functions, `state` and `merge`. `state` will return an array of scalar values that contain +the current state of a single batch accumulation. Then we must `merge` the results of these +different states. Finally `evaluate` is the call that will return the final result after the +`merge` is complete. In the following example we want to define a custom aggregate function that will return the difference between the sum of two columns. The state can be represented by a single value and we can -also see how the inputs to ``update`` and ``merge`` differ. - -.. code-block:: python - - import pyarrow as pa - import pyarrow.compute - import datafusion - from datafusion import col, udaf, Accumulator - from typing import List - - class MyAccumulator(Accumulator): - """ - Interface of a user-defined accumulation. - """ - def __init__(self): - self._sum = 0.0 - - def update(self, values_a: pa.Array, values_b: pa.Array) -> None: - self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py() - - def merge(self, states: list[pa.Array]) -> None: - self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py() - - def state(self) -> list[pa.Scalar]: - return [pyarrow.scalar(self._sum)] - - def evaluate(self) -> pa.Scalar: - return pyarrow.scalar(self._sum) - - ctx = datafusion.SessionContext() - df = ctx.from_pydict( - { - "a": [4, 5, 6], - "b": [1, 2, 3], - } - ) +also see how the inputs to `update` and `merge` differ. + +```python +import pyarrow as pa +import pyarrow.compute +import datafusion +from datafusion import col, udaf, Accumulator +from typing import List + +class MyAccumulator(Accumulator): + """ + Interface of a user-defined accumulation. + """ + def __init__(self): + self._sum = 0.0 + + def update(self, values_a: pa.Array, values_b: pa.Array) -> None: + self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py() + + def merge(self, states: list[pa.Array]) -> None: + self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py() + + def state(self) -> list[pa.Scalar]: + return [pyarrow.scalar(self._sum)] + + def evaluate(self) -> pa.Scalar: + return pyarrow.scalar(self._sum) + +ctx = datafusion.SessionContext() +df = ctx.from_pydict( + { + "a": [4, 5, 6], + "b": [1, 2, 3], + } +) - my_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable') +my_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable') - df.aggregate([], [my_udaf(col("a"), col("b")).alias("col_diff")]) +df.aggregate([], [my_udaf(col("a"), col("b")).alias("col_diff")]) +``` -FAQ -^^^ +### FAQ **How do I return a list from a UDAF?** -Both the ``evaluate`` and the ``state`` functions expect to return scalar values. +Both the `evaluate` and the `state` functions expect to return scalar values. If you wish to return a list array as a scalar value, the best practice is to -wrap the values in a ``pyarrow.Scalar`` object. For example, you can return a -timestamp list with ``pa.scalar([...], type=pa.list_(pa.timestamp("ms")))`` and +wrap the values in a `pyarrow.Scalar` object. For example, you can return a +timestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp("ms")))` and register the appropriate return or state types as -``return_type=pa.list_(pa.timestamp("ms"))`` and -``state_type=[pa.list_(pa.timestamp("ms"))]``, respectively. +`return_type=pa.list_(pa.timestamp("ms"))` and +`state_type=[pa.list_(pa.timestamp("ms"))]`, respectively. As of DataFusion 52.0.0 , you can pass return any Python object, including a PyArrow array, as the return value(s) for these functions and DataFusion will @@ -306,23 +322,23 @@ attempt to create a scalar type from the value. DataFusion has been tested to convert PyArrow, nanoarrow, and arro3 objects as well as primitive data types like integers, strings, and so on. -Window Functions ----------------- +## Window Functions To implement a User-Defined Window Function (UDWF) you must call the -:py:func:`~datafusion.user_defined.WindowUDF.udwf` function using a class that implements the abstract -class :py:class:`~datafusion.user_defined.WindowEvaluator`. +{py:func}`~datafusion.user_defined.WindowUDF.udwf` function using a class that implements the abstract +class {py:class}`~datafusion.user_defined.WindowEvaluator`. There are three methods of evaluation of UDWFs. -- ``evaluate`` is the simplest case, where you are given an array and are expected to calculate the +- `evaluate` is the simplest case, where you are given an array and are expected to calculate the value for a single row of that array. This is the simplest case, but also the least performant. -- ``evaluate_all`` computes the values for all rows for an input array at a single time. -- ``evaluate_all_with_rank`` computes the values for all rows, but you only have the rank +- `evaluate_all` computes the values for all rows for an input array at a single time. +- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank information for the rows. Which methods you implement are based upon which of these options are set. +```{eval-rst} .. list-table:: :header-rows: 1 @@ -346,62 +362,60 @@ Which methods you implement are based upon which of these options are set. - True/False - True/False - ``evaluate`` +``` -UDWF options -^^^^^^^^^^^^ +### UDWF options When you define your UDWF you can override the functions that return these values. They will determine which evaluate functions are called. -- ``uses_window_frame`` is set for functions that compute based on the specified window frame. If - your function depends upon the specified frame, set this to ``True``. -- ``supports_bounded_execution`` specifies if your function can be incrementally computed. -- ``include_rank`` is set to ``True`` for window functions that can be computed only using the rank +- `uses_window_frame` is set for functions that compute based on the specified window frame. If + your function depends upon the specified frame, set this to `True`. +- `supports_bounded_execution` specifies if your function can be incrementally computed. +- `include_rank` is set to `True` for window functions that can be computed only using the rank information. +```python +import pyarrow as pa +from datafusion import udwf, col, SessionContext +from datafusion.user_defined import WindowEvaluator -.. code-block:: python +class ExponentialSmooth(WindowEvaluator): + def __init__(self, alpha: float) -> None: + self.alpha = alpha - import pyarrow as pa - from datafusion import udwf, col, SessionContext - from datafusion.user_defined import WindowEvaluator - - class ExponentialSmooth(WindowEvaluator): - def __init__(self, alpha: float) -> None: - self.alpha = alpha - - def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array: - results = [] - curr_value = 0.0 - values = values[0] - for idx in range(num_rows): - if idx == 0: - curr_value = values[idx].as_py() - else: - curr_value = values[idx].as_py() * self.alpha + curr_value * ( - 1.0 - self.alpha - ) - results.append(curr_value) - - return pa.array(results) - - exp_smooth = udwf( - ExponentialSmooth(0.9), - pa.float64(), - pa.float64(), - volatility="immutable", - ) + def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array: + results = [] + curr_value = 0.0 + values = values[0] + for idx in range(num_rows): + if idx == 0: + curr_value = values[idx].as_py() + else: + curr_value = values[idx].as_py() * self.alpha + curr_value * ( + 1.0 - self.alpha + ) + results.append(curr_value) - ctx = SessionContext() + return pa.array(results) + +exp_smooth = udwf( + ExponentialSmooth(0.9), + pa.float64(), + pa.float64(), + volatility="immutable", +) - df = ctx.from_pydict({ - "a": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0] - }) +ctx = SessionContext() - df.select("a", exp_smooth(col("a")).alias("smooth_a")).show() +df = ctx.from_pydict({ + "a": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0] +}) -Table Functions ---------------- +df.select("a", exp_smooth(col("a")).alias("smooth_a")).show() +``` + +## Table Functions User Defined Table Functions are slightly different than the other functions described here. These functions take any number of `Expr` arguments, but only @@ -409,61 +423,60 @@ literal expressions are supported. Table functions must return a Table Provider as described in the ref:`_io_custom_table_provider` page. Once you have a table function, you can register it with the session context -by using :py:func:`datafusion.context.SessionContext.register_udtf`. +by using {py:func}`datafusion.context.SessionContext.register_udtf`. There are examples of both rust backed and python based table functions in the examples folder of the repository. If you have a rust backed table function -that you wish to expose via PyO3, you need to expose it as a ``PyCapsule``. - -.. code-block:: rust +that you wish to expose via PyO3, you need to expose it as a `PyCapsule`. - #[pymethods] - impl MyTableFunction { - fn __datafusion_table_function__<'py>( - &self, - py: Python<'py>, - ) -> PyResult> { - let name = cr"datafusion_table_function".into(); +```rust +#[pymethods] +impl MyTableFunction { + fn __datafusion_table_function__<'py>( + &self, + py: Python<'py>, + ) -> PyResult> { + let name = cr"datafusion_table_function".into(); - let func = self.clone(); - let provider = FFI_TableFunction::new(Arc::new(func), None); + let func = self.clone(); + let provider = FFI_TableFunction::new(Arc::new(func), None); - PyCapsule::new(py, provider, Some(name)) - } + PyCapsule::new(py, provider, Some(name)) } +} +``` -Accessing the Calling Session -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +### Accessing the Calling Session Pure-Python UDTFs can opt into receiving the calling -:py:class:`~datafusion.SessionContext` by registering with -``with_session=True``. The context is passed as a ``session`` keyword +{py:class}`~datafusion.SessionContext` by registering with +`with_session=True`. The context is passed as a `session` keyword argument on every invocation. Use it to look up registered tables, UDFs, or session configuration from inside the callback. -.. code-block:: python - - from datafusion import SessionContext, Table, udtf - from datafusion.context import TableProviderExportable - import pyarrow as pa - import pyarrow.dataset as ds - - @udtf("list_tables", with_session=True) - def list_tables(*, session: SessionContext) -> TableProviderExportable: - names = sorted(session.catalog().schema().names()) - batch = pa.RecordBatch.from_pydict({"name": names}) - return Table(ds.dataset([batch])) - - ctx = SessionContext() - ctx.register_batch("t1", pa.RecordBatch.from_pydict({"x": [1]})) - ctx.register_udtf(list_tables) - ctx.sql("SELECT * FROM list_tables()").show() - -Without ``with_session=True``, the callback receives only the positional +```python +from datafusion import SessionContext, Table, udtf +from datafusion.context import TableProviderExportable +import pyarrow as pa +import pyarrow.dataset as ds + +@udtf("list_tables", with_session=True) +def list_tables(*, session: SessionContext) -> TableProviderExportable: + names = sorted(session.catalog().schema().names()) + batch = pa.RecordBatch.from_pydict({"name": names}) + return Table(ds.dataset([batch])) + +ctx = SessionContext() +ctx.register_batch("t1", pa.RecordBatch.from_pydict({"x": [1]})) +ctx.register_udtf(list_tables) +ctx.sql("SELECT * FROM list_tables()").show() +``` + +Without `with_session=True`, the callback receives only the positional expression arguments. The flag is opt-in so existing UDTFs keep working unchanged. -The injected ``session`` is a fresh :py:class:`~datafusion.SessionContext` +The injected `session` is a fresh {py:class}`~datafusion.SessionContext` wrapper backed by the same underlying state as the caller, so registries (tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering a new table or UDF) propagate to the live session because the registries diff --git a/docs/source/user-guide/common-operations/views.md b/docs/source/user-guide/common-operations/views.md new file mode 100644 index 000000000..be00e25a2 --- /dev/null +++ b/docs/source/user-guide/common-operations/views.md @@ -0,0 +1,67 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Registering Views + +You can use the context's `register_view` method to register a DataFrame as a view + +```python +from datafusion import SessionContext, col, literal + +# Create a DataFusion context +ctx = SessionContext() + +# Create sample data +data = {"a": [1, 2, 3, 4, 5], "b": [10, 20, 30, 40, 50]} + +# Create a DataFrame from the dictionary +df = ctx.from_pydict(data, "my_table") + +# Filter the DataFrame (for example, keep rows where a > 2) +df_filtered = df.filter(col("a") > literal(2)) + +# Register the dataframe as a view with the context +ctx.register_view("view1", df_filtered) + +# Now run a SQL query against the registered view +df_view = ctx.sql("SELECT * FROM view1") + +# Collect the results +results = df_view.collect() + +# Convert results to a list of dictionaries for display +result_dicts = [batch.to_pydict() for batch in results] + +print(result_dicts) +``` + +This will output: + +```python +[{'a': [3, 4, 5], 'b': [30, 40, 50]}] +``` diff --git a/docs/source/user-guide/common-operations/views.rst b/docs/source/user-guide/common-operations/views.rst deleted file mode 100644 index df11e3abe..000000000 --- a/docs/source/user-guide/common-operations/views.rst +++ /dev/null @@ -1,58 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -====================== -Registering Views -====================== - -You can use the context's ``register_view`` method to register a DataFrame as a view - -.. code-block:: python - - from datafusion import SessionContext, col, literal - - # Create a DataFusion context - ctx = SessionContext() - - # Create sample data - data = {"a": [1, 2, 3, 4, 5], "b": [10, 20, 30, 40, 50]} - - # Create a DataFrame from the dictionary - df = ctx.from_pydict(data, "my_table") - - # Filter the DataFrame (for example, keep rows where a > 2) - df_filtered = df.filter(col("a") > literal(2)) - - # Register the dataframe as a view with the context - ctx.register_view("view1", df_filtered) - - # Now run a SQL query against the registered view - df_view = ctx.sql("SELECT * FROM view1") - - # Collect the results - results = df_view.collect() - - # Convert results to a list of dictionaries for display - result_dicts = [batch.to_pydict() for batch in results] - - print(result_dicts) - -This will output: - -.. code-block:: python - - [{'a': [3, 4, 5], 'b': [30, 40, 50]}] diff --git a/docs/source/user-guide/common-operations/windows.rst b/docs/source/user-guide/common-operations/windows.md similarity index 59% rename from docs/source/user-guide/common-operations/windows.rst rename to docs/source/user-guide/common-operations/windows.md index 127f691b5..e7e45178a 100644 --- a/docs/source/user-guide/common-operations/windows.rst +++ b/docs/source/user-guide/common-operations/windows.md @@ -1,33 +1,44 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at +% Licensed to the Apache Software Foundation (ASF) under one -.. http://www.apache.org/licenses/LICENSE-2.0 +% or more contributor license agreements. See the NOTICE file -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. +% distributed with this work for additional information -.. _window_functions: +% regarding copyright ownership. The ASF licenses this file -Window Functions -================ +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(window_functions)= + +# Window Functions In this section you will learn about window functions. A window function utilizes values from one or multiple rows to produce a result for each individual row, unlike an aggregate function that provides a single value for multiple rows. -The window functions are available in the :py:mod:`~datafusion.functions` module. +The window functions are available in the {py:mod}`~datafusion.functions` module. We'll use the pokemon dataset (from Ritchie Vink) in the following examples. +```{eval-rst} .. ipython:: python from datafusion import SessionContext @@ -36,10 +47,12 @@ We'll use the pokemon dataset (from Ritchie Vink) in the following examples. ctx = SessionContext() df = ctx.read_csv("pokemon.csv") +``` Here is an example that shows how you can compare each pokemon's speed to the speed of the previous row in the DataFrame. +```{eval-rst} .. ipython:: python df.select( @@ -47,17 +60,16 @@ previous row in the DataFrame. col('"Speed"'), f.lag(col('"Speed"')).alias("Previous Speed") ) +``` -Setting Parameters ------------------- - +## Setting Parameters -Ordering -^^^^^^^^ +### Ordering You can control the order in which rows are processed by window functions by providing -a list of ``order_by`` functions for the ``order_by`` parameter. +a list of `order_by` functions for the `order_by` parameter. +```{eval-rst} .. ipython:: python df.select( @@ -69,16 +81,17 @@ a list of ``order_by`` functions for the ``order_by`` parameter. order_by=[col('"Attack"').sort(ascending=True)], ).alias("rank"), ).sort(col('"Type 1"'), col('"Attack"')) +``` -Partitions -^^^^^^^^^^ +### Partitions -A window function can take a list of ``partition_by`` columns similar to an -:ref:`Aggregation Function`. This will cause the window values to be evaluated +A window function can take a list of `partition_by` columns similar to an +{ref}`Aggregation Function`. This will cause the window values to be evaluated independently for each of the partitions. In the example above, we found the rank of each -Pokemon per ``Type 1`` partitions. We can see the first couple of each partition if we do +Pokemon per `Type 1` partitions. We can see the first couple of each partition if we do the following: +```{eval-rst} .. ipython:: python df.select( @@ -90,34 +103,35 @@ the following: order_by=[col('"Attack"').sort(ascending=True)], ).alias("rank"), ).filter(col("rank") < lit(3)).sort(col('"Type 1"'), col("rank")) +``` -Window Frame -^^^^^^^^^^^^ +### Window Frame When using aggregate functions, the Window Frame of defines the rows over which it operates. If you do not specify a Window Frame, the frame will be set depending on the following criteria. -* If an ``order_by`` clause is set, the default window frame is defined as the rows between +- If an `order_by` clause is set, the default window frame is defined as the rows between unbounded preceding and the current row. -* If an ``order_by`` is not set, the default frame is defined as the rows between unbounded +- If an `order_by` is not set, the default frame is defined as the rows between unbounded and unbounded following (the entire partition). Window Frames are defined by three parameters: unit type, starting bound, and ending bound. The unit types available are: -* Rows: The starting and ending boundaries are defined by the number of rows relative to the +- Rows: The starting and ending boundaries are defined by the number of rows relative to the current row. -* Range: When using Range, the ``order_by`` clause must have exactly one term. The boundaries - are defined bow how close the rows are to the value of the expression in the ``order_by`` +- Range: When using Range, the `order_by` clause must have exactly one term. The boundaries + are defined bow how close the rows are to the value of the expression in the `order_by` parameter. -* Groups: A "group" is the set of all rows that have equivalent values for all terms in the - ``order_by`` clause. +- Groups: A "group" is the set of all rows that have equivalent values for all terms in the + `order_by` clause. In this example we perform a "rolling average" of the speed of the current Pokemon and the two preceding rows. +```{eval-rst} .. ipython:: python from datafusion.expr import Window, WindowFrame @@ -129,9 +143,9 @@ two preceding rows. .over(Window(window_frame=WindowFrame("rows", 2, 0), order_by=[col('"Speed"')])) .alias("Previous Speed"), ) +``` -Null Treatment -^^^^^^^^^^^^^^ +### Null Treatment When using aggregate functions as window functions, it is often useful to specify how null values should be treated. In order to do this you need to use the builder function. In future releases @@ -143,8 +157,9 @@ nulls will fill in with the value of the most recent non-null row. To do this, w the window frame so that we only process up to the current row. In this example, we filter down to one specific type of Pokemon that does have some entries in -it's ``Type 2`` column that are null. +it's `Type 2` column that are null. +```{eval-rst} .. ipython:: python from datafusion.common import NullTreatment @@ -171,14 +186,15 @@ it's ``Type 2`` column that are null. ) .alias("last_with_null"), ) +``` -Aggregate Functions -------------------- +## Aggregate Functions -You can use any :ref:`Aggregation Function` as a window function. Here +You can use any {ref}`Aggregation Function` as a window function. Here is an example that shows how to compare each pokemons’s attack power with the average attack -power in its ``"Type 1"`` using the :py:func:`datafusion.functions.avg` function. +power in its `"Type 1"` using the {py:func}`datafusion.functions.avg` function. +```{eval-rst} .. ipython:: python :okwarning: @@ -193,41 +209,40 @@ power in its ``"Type 1"`` using the :py:func:`datafusion.functions.avg` function ) ).alias("Average Attack"), ) +``` -Available Functions -------------------- +## Available Functions The possible window functions are: 1. Rank Functions - - :py:func:`datafusion.functions.rank` - - :py:func:`datafusion.functions.dense_rank` - - :py:func:`datafusion.functions.ntile` - - :py:func:`datafusion.functions.row_number` - + : - {py:func}`datafusion.functions.rank` + - {py:func}`datafusion.functions.dense_rank` + - {py:func}`datafusion.functions.ntile` + - {py:func}`datafusion.functions.row_number` 2. Analytical Functions - - :py:func:`datafusion.functions.cume_dist` - - :py:func:`datafusion.functions.percent_rank` - - :py:func:`datafusion.functions.lag` - - :py:func:`datafusion.functions.lead` - + : - {py:func}`datafusion.functions.cume_dist` + - {py:func}`datafusion.functions.percent_rank` + - {py:func}`datafusion.functions.lag` + - {py:func}`datafusion.functions.lead` 3. Aggregate Functions - - All :ref:`Aggregation Functions` can be used as window functions. + : - All {ref}`Aggregation Functions` can be used as window functions. -User-Defined Window Functions ------------------------------ +## User-Defined Window Functions You can ship custom window functions to the engine by subclassing -:py:class:`~datafusion.user_defined.WindowEvaluator` and registering it -via :py:func:`~datafusion.udwf`. See :py:mod:`datafusion.user_defined` +{py:class}`~datafusion.user_defined.WindowEvaluator` and registering it +via {py:func}`~datafusion.udwf`. See {py:mod}`datafusion.user_defined` for the evaluator interface and worked examples. -.. note:: Serialization - - Python window UDFs travel inline inside pickled or - :py:meth:`~datafusion.expr.Expr.to_bytes`-serialized expressions — - the evaluator class is captured by value via :mod:`cloudpickle`, so - worker processes do not need to pre-register the UDF. Any names the - evaluator resolves via ``import`` are captured **by reference** and - must be importable on the receiving worker. See - :py:mod:`datafusion.ipc` for the full IPC model and security caveats. +:::{note} +Serialization + +Python window UDFs travel inline inside pickled or +{py:meth}`~datafusion.expr.Expr.to_bytes`-serialized expressions — +the evaluator class is captured by value via {mod}`cloudpickle`, so +worker processes do not need to pre-register the UDF. Any names the +evaluator resolves via `import` are captured **by reference** and +must be importable on the receiving worker. See +{py:mod}`datafusion.ipc` for the full IPC model and security caveats. +::: diff --git a/docs/source/user-guide/configuration.md b/docs/source/user-guide/configuration.md new file mode 100644 index 000000000..21a06da18 --- /dev/null +++ b/docs/source/user-guide/configuration.md @@ -0,0 +1,194 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(configuration)= + +# Configuration + +Let's look at how we can configure DataFusion. When creating a {py:class}`~datafusion.context.SessionContext`, you can pass in +a {py:class}`~datafusion.context.SessionConfig` and {py:class}`~datafusion.context.RuntimeEnvBuilder` object. These two cover a wide range of options. + +```python +from datafusion import RuntimeEnvBuilder, SessionConfig, SessionContext + +# create a session context with default settings +ctx = SessionContext() +print(ctx) + +# create a session context with explicit runtime and config settings +runtime = RuntimeEnvBuilder().with_disk_manager_os().with_fair_spill_pool(10000000) +config = ( + SessionConfig() + .with_create_default_catalog_and_schema(True) + .with_default_catalog_and_schema("foo", "bar") + .with_target_partitions(8) + .with_information_schema(True) + .with_repartition_joins(False) + .with_repartition_aggregations(False) + .with_repartition_windows(False) + .with_parquet_pruning(False) + .set("datafusion.execution.parquet.pushdown_filters", "true") +) +ctx = SessionContext(config, runtime) +print(ctx) +``` + +## Maximizing CPU Usage + +DataFusion uses partitions to parallelize work. For small queries the +default configuration (number of CPU cores) is often sufficient, but to +fully utilize available hardware you can tune how many partitions are +created and when DataFusion will repartition data automatically. + +Configure a `SessionContext` with a higher partition count: + +```python +from datafusion import SessionConfig, SessionContext + +# allow up to 16 concurrent partitions +config = SessionConfig().with_target_partitions(16) +ctx = SessionContext(config) +``` + +Automatic repartitioning for joins, aggregations, window functions and +other operations can be enabled to increase parallelism: + +```python +config = ( + SessionConfig() + .with_target_partitions(16) + .with_repartition_joins(True) + .with_repartition_aggregations(True) + .with_repartition_windows(True) +) +``` + +Manual repartitioning is available on DataFrames when you need precise +control: + +```python +from datafusion import col + +df = ctx.read_parquet("data.parquet") + +# Evenly divide into 16 partitions +df = df.repartition(16) + +# Or partition by the hash of a column +df = df.repartition_by_hash(col("a"), num=16) + +result = df.collect() +``` + +### Benchmark Example + +The repository includes a benchmark script that demonstrates how to maximize CPU usage +with DataFusion. The {code}`benchmarks/max_cpu_usage.py` script shows a practical example +of configuring DataFusion for optimal parallelism. + +You can run the benchmark script to see the impact of different configuration settings: + +```bash +# Run with default settings (uses all CPU cores) +python benchmarks/max_cpu_usage.py + +# Run with specific number of rows and partitions +python benchmarks/max_cpu_usage.py --rows 5000000 --partitions 16 + +# See all available options +python benchmarks/max_cpu_usage.py --help +``` + +Here's an example showing the performance difference between single and multiple partitions: + +```bash +# Single partition - slower processing +$ python benchmarks/max_cpu_usage.py --rows=10000000 --partitions 1 +Processed 10000000 rows using 1 partitions in 0.107s + +# Multiple partitions - faster processing +$ python benchmarks/max_cpu_usage.py --rows=10000000 --partitions 10 +Processed 10000000 rows using 10 partitions in 0.038s +``` + +This example demonstrates nearly 3x performance improvement (0.107s vs 0.038s) when using +10 partitions instead of 1, showcasing how proper partitioning can significantly improve +CPU utilization and query performance. + +The script demonstrates several key optimization techniques: + +1. **Higher target partition count**: Uses {code}`with_target_partitions()` to set the number of concurrent partitions +2. **Automatic repartitioning**: Enables repartitioning for joins, aggregations, and window functions +3. **Manual repartitioning**: Uses {code}`repartition()` to ensure all partitions are utilized +4. **CPU-intensive operations**: Performs aggregations that can benefit from parallelization + +The benchmark creates synthetic data and measures the time taken to perform a sum aggregation +across the specified number of partitions. This helps you understand how partition configuration +affects performance on your specific hardware. + +#### Important Considerations + +The provided benchmark script demonstrates partitioning concepts using synthetic in-memory data +and simple aggregation operations. While useful for understanding basic configuration principles, +actual performance in production environments may vary significantly based on numerous factors: + +**Data Sources and I/O Characteristics:** + +- **Table providers**: Performance differs greatly between Parquet files, CSV files, databases, and cloud storage +- **Storage type**: Local SSD, network-attached storage, and cloud storage have vastly different characteristics +- **Network latency**: Remote data sources introduce additional latency considerations +- **File sizes and distribution**: Large files may benefit differently from partitioning than many small files + +**Query and Workload Characteristics:** + +- **Operation complexity**: Simple aggregations versus complex joins, window functions, or nested queries +- **Data distribution**: Skewed data may not partition evenly, affecting parallel efficiency +- **Memory usage**: Large datasets may require different memory management strategies +- **Concurrent workloads**: Multiple queries running simultaneously affect resource allocation + +**Hardware and Environment Factors:** + +- **CPU architecture**: Different processors have varying parallel processing capabilities +- **Available memory**: Limited RAM may require different optimization strategies +- **System load**: Other applications competing for resources affect DataFusion performance + +**Recommendations for Production Use:** + +To optimize DataFusion for your specific use case, it is strongly recommended to: + +1. **Create custom benchmarks** using your actual data sources, formats, and query patterns +2. **Test with representative data volumes** that match your production workloads +3. **Measure end-to-end performance** including data loading, processing, and result handling +4. **Evaluate different configuration combinations** for your specific hardware and workload +5. **Monitor resource utilization** (CPU, memory, I/O) to identify bottlenecks in your environment + +This approach will provide more accurate insights into how DataFusion configuration options +will impact your particular applications and infrastructure. + +For more information about available {py:class}`~datafusion.context.SessionConfig` options, see the [rust DataFusion Configuration guide](https://arrow.apache.org/datafusion/user-guide/configs.html), +and about {code}`RuntimeEnvBuilder` options in the rust [online API documentation](https://docs.rs/datafusion/latest/datafusion/execution/runtime_env/struct.RuntimeEnvBuilder.html). diff --git a/docs/source/user-guide/configuration.rst b/docs/source/user-guide/configuration.rst deleted file mode 100644 index f8e613cd4..000000000 --- a/docs/source/user-guide/configuration.rst +++ /dev/null @@ -1,188 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _configuration: - -Configuration -============= - -Let's look at how we can configure DataFusion. When creating a :py:class:`~datafusion.context.SessionContext`, you can pass in -a :py:class:`~datafusion.context.SessionConfig` and :py:class:`~datafusion.context.RuntimeEnvBuilder` object. These two cover a wide range of options. - -.. code-block:: python - - from datafusion import RuntimeEnvBuilder, SessionConfig, SessionContext - - # create a session context with default settings - ctx = SessionContext() - print(ctx) - - # create a session context with explicit runtime and config settings - runtime = RuntimeEnvBuilder().with_disk_manager_os().with_fair_spill_pool(10000000) - config = ( - SessionConfig() - .with_create_default_catalog_and_schema(True) - .with_default_catalog_and_schema("foo", "bar") - .with_target_partitions(8) - .with_information_schema(True) - .with_repartition_joins(False) - .with_repartition_aggregations(False) - .with_repartition_windows(False) - .with_parquet_pruning(False) - .set("datafusion.execution.parquet.pushdown_filters", "true") - ) - ctx = SessionContext(config, runtime) - print(ctx) - -Maximizing CPU Usage --------------------- - -DataFusion uses partitions to parallelize work. For small queries the -default configuration (number of CPU cores) is often sufficient, but to -fully utilize available hardware you can tune how many partitions are -created and when DataFusion will repartition data automatically. - -Configure a ``SessionContext`` with a higher partition count: - -.. code-block:: python - - from datafusion import SessionConfig, SessionContext - - # allow up to 16 concurrent partitions - config = SessionConfig().with_target_partitions(16) - ctx = SessionContext(config) - -Automatic repartitioning for joins, aggregations, window functions and -other operations can be enabled to increase parallelism: - -.. code-block:: python - - config = ( - SessionConfig() - .with_target_partitions(16) - .with_repartition_joins(True) - .with_repartition_aggregations(True) - .with_repartition_windows(True) - ) - -Manual repartitioning is available on DataFrames when you need precise -control: - -.. code-block:: python - - from datafusion import col - - df = ctx.read_parquet("data.parquet") - - # Evenly divide into 16 partitions - df = df.repartition(16) - - # Or partition by the hash of a column - df = df.repartition_by_hash(col("a"), num=16) - - result = df.collect() - - -Benchmark Example -^^^^^^^^^^^^^^^^^ - -The repository includes a benchmark script that demonstrates how to maximize CPU usage -with DataFusion. The :code:`benchmarks/max_cpu_usage.py` script shows a practical example -of configuring DataFusion for optimal parallelism. - -You can run the benchmark script to see the impact of different configuration settings: - -.. code-block:: bash - - # Run with default settings (uses all CPU cores) - python benchmarks/max_cpu_usage.py - - # Run with specific number of rows and partitions - python benchmarks/max_cpu_usage.py --rows 5000000 --partitions 16 - - # See all available options - python benchmarks/max_cpu_usage.py --help - -Here's an example showing the performance difference between single and multiple partitions: - -.. code-block:: bash - - # Single partition - slower processing - $ python benchmarks/max_cpu_usage.py --rows=10000000 --partitions 1 - Processed 10000000 rows using 1 partitions in 0.107s - - # Multiple partitions - faster processing - $ python benchmarks/max_cpu_usage.py --rows=10000000 --partitions 10 - Processed 10000000 rows using 10 partitions in 0.038s - -This example demonstrates nearly 3x performance improvement (0.107s vs 0.038s) when using -10 partitions instead of 1, showcasing how proper partitioning can significantly improve -CPU utilization and query performance. - -The script demonstrates several key optimization techniques: - -1. **Higher target partition count**: Uses :code:`with_target_partitions()` to set the number of concurrent partitions -2. **Automatic repartitioning**: Enables repartitioning for joins, aggregations, and window functions -3. **Manual repartitioning**: Uses :code:`repartition()` to ensure all partitions are utilized -4. **CPU-intensive operations**: Performs aggregations that can benefit from parallelization - -The benchmark creates synthetic data and measures the time taken to perform a sum aggregation -across the specified number of partitions. This helps you understand how partition configuration -affects performance on your specific hardware. - -Important Considerations -"""""""""""""""""""""""" - -The provided benchmark script demonstrates partitioning concepts using synthetic in-memory data -and simple aggregation operations. While useful for understanding basic configuration principles, -actual performance in production environments may vary significantly based on numerous factors: - -**Data Sources and I/O Characteristics:** - -- **Table providers**: Performance differs greatly between Parquet files, CSV files, databases, and cloud storage -- **Storage type**: Local SSD, network-attached storage, and cloud storage have vastly different characteristics -- **Network latency**: Remote data sources introduce additional latency considerations -- **File sizes and distribution**: Large files may benefit differently from partitioning than many small files - -**Query and Workload Characteristics:** - -- **Operation complexity**: Simple aggregations versus complex joins, window functions, or nested queries -- **Data distribution**: Skewed data may not partition evenly, affecting parallel efficiency -- **Memory usage**: Large datasets may require different memory management strategies -- **Concurrent workloads**: Multiple queries running simultaneously affect resource allocation - -**Hardware and Environment Factors:** - -- **CPU architecture**: Different processors have varying parallel processing capabilities -- **Available memory**: Limited RAM may require different optimization strategies -- **System load**: Other applications competing for resources affect DataFusion performance - -**Recommendations for Production Use:** - -To optimize DataFusion for your specific use case, it is strongly recommended to: - -1. **Create custom benchmarks** using your actual data sources, formats, and query patterns -2. **Test with representative data volumes** that match your production workloads -3. **Measure end-to-end performance** including data loading, processing, and result handling -4. **Evaluate different configuration combinations** for your specific hardware and workload -5. **Monitor resource utilization** (CPU, memory, I/O) to identify bottlenecks in your environment - -This approach will provide more accurate insights into how DataFusion configuration options -will impact your particular applications and infrastructure. - -For more information about available :py:class:`~datafusion.context.SessionConfig` options, see the `rust DataFusion Configuration guide `_, -and about :code:`RuntimeEnvBuilder` options in the rust `online API documentation `_. diff --git a/docs/source/user-guide/data-sources.md b/docs/source/user-guide/data-sources.md new file mode 100644 index 000000000..cab7c3897 --- /dev/null +++ b/docs/source/user-guide/data-sources.md @@ -0,0 +1,290 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(user_guide_data_sources)= + +# Data Sources + +DataFusion provides a wide variety of ways to get data into a DataFrame to perform operations. + +## Local file + +DataFusion has the ability to read from a variety of popular file formats, such as {ref}`Parquet `, +{ref}`CSV `, {ref}`JSON `, and {ref}`AVRO `. + +```{eval-rst} +.. ipython:: python + + from datafusion import SessionContext + ctx = SessionContext() + df = ctx.read_csv("pokemon.csv") + df.show() +``` + +## Create in-memory + +Sometimes it can be convenient to create a small DataFrame from a Python list or dictionary object. +To do this in DataFusion, you can use one of the three functions +{py:func}`~datafusion.context.SessionContext.from_pydict`, +{py:func}`~datafusion.context.SessionContext.from_pylist`, or +{py:func}`~datafusion.context.SessionContext.create_dataframe`. + +As their names suggest, `from_pydict` and `from_pylist` will create DataFrames from Python +dictionary and list objects, respectively. `create_dataframe` assumes you will pass in a list +of list of [PyArrow Record Batches](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html). + +The following three examples all will create identical DataFrames: + +```{eval-rst} +.. ipython:: python + + import pyarrow as pa + + ctx.from_pylist([ + { "a": 1, "b": 10.0, "c": "alpha" }, + { "a": 2, "b": 20.0, "c": "beta" }, + { "a": 3, "b": 30.0, "c": "gamma" }, + ]).show() + + ctx.from_pydict({ + "a": [1, 2, 3], + "b": [10.0, 20.0, 30.0], + "c": ["alpha", "beta", "gamma"], + }).show() + + batch = pa.RecordBatch.from_arrays( + [ + pa.array([1, 2, 3]), + pa.array([10.0, 20.0, 30.0]), + pa.array(["alpha", "beta", "gamma"]), + ], + names=["a", "b", "c"], + ) + + ctx.create_dataframe([[batch]]).show() + +``` + +## Object Store + +DataFusion has support for multiple storage options in addition to local files. +The example below requires an appropriate S3 account with access credentials. + +Supported Object Stores are + +- {py:class}`~datafusion.object_store.AmazonS3` +- {py:class}`~datafusion.object_store.GoogleCloud` +- {py:class}`~datafusion.object_store.Http` +- {py:class}`~datafusion.object_store.LocalFileSystem` +- {py:class}`~datafusion.object_store.MicrosoftAzure` + +```python +from datafusion.object_store import AmazonS3 + +region = "us-east-1" +bucket_name = "yellow-trips" + +s3 = AmazonS3( + bucket_name=bucket_name, + region=region, + access_key_id=os.getenv("AWS_ACCESS_KEY_ID"), + secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"), +) + +path = f"s3://{bucket_name}/" +ctx.register_object_store("s3://", s3, None) + +ctx.register_parquet("trips", path) + +ctx.table("trips").show() +``` + +## Other DataFrame Libraries + +DataFusion can import DataFrames directly from other libraries, such as +[Polars](https://pola.rs/) and [Pandas](https://pandas.pydata.org/). +Since DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule +interface can be imported to DataFusion using the +{py:func}`~datafusion.context.SessionContext.from_arrow` function. Older versions of Polars may +not support the arrow interface. In those cases, you can still import via the +{py:func}`~datafusion.context.SessionContext.from_polars` function. + +```python +import pandas as pd + +data = { "a": [1, 2, 3], "b": [10.0, 20.0, 30.0], "c": ["alpha", "beta", "gamma"] } +pandas_df = pd.DataFrame(data) + +datafusion_df = ctx.from_arrow(pandas_df) +datafusion_df.show() +``` + +```python +import polars as pl +polars_df = pl.DataFrame(data) + +datafusion_df = ctx.from_arrow(polars_df) +datafusion_df.show() +``` + +## Delta Lake + +DataFusion 43.0.0 and later support the ability to register table providers from sources such +as Delta Lake. This will require a recent version of +[deltalake](https://delta-io.github.io/delta-rs/) to provide the required interfaces. + +```python +from deltalake import DeltaTable + +delta_table = DeltaTable("path_to_table") +ctx.register_table("my_delta_table", delta_table) +df = ctx.table("my_delta_table") +df.show() +``` + +On older versions of `deltalake` (prior to 0.22) you can use the +[Arrow DataSet](https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Dataset.html) +interface to import to DataFusion, but this does not support features such as filter push down +which can lead to a significant performance difference. + +```python +from deltalake import DeltaTable + +delta_table = DeltaTable("path_to_table") +ctx.register_dataset("my_delta_table", delta_table.to_pyarrow_dataset()) +df = ctx.table("my_delta_table") +df.show() +``` + +## Apache Iceberg + +DataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface. + +This requires either the [pyiceberg](https://pypi.org/project/pyiceberg/) library (>=0.10.0) or the [pyiceberg-core](https://pypi.org/project/pyiceberg-core/) library (>=0.5.0). + +- The `pyiceberg-core` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings. +- The `pyiceberg` library utilizes the `pyiceberg-core` python bindings under the hood and provides a native way for Python users to interact with the DataFusion. + +```python +from datafusion import SessionContext +from pyiceberg.catalog import load_catalog +import pyarrow as pa + +# Load catalog and create/load a table +catalog = load_catalog("catalog", type="in-memory") +catalog.create_namespace_if_not_exists("default") + +# Create some sample data +data = pa.table({"x": [1, 2, 3], "y": [4, 5, 6]}) +iceberg_table = catalog.create_table("default.test", schema=data.schema) +iceberg_table.append(data) + +# Register the table with DataFusion +ctx = SessionContext() +ctx.register_table_provider("test", iceberg_table) + +# Query the table using DataFusion +ctx.table("test").show() +``` + +Note that the Datafusion integration rely on features from the [Iceberg Rust](https://github.com/apache/iceberg-rust/) implementation instead of the [PyIceberg](https://github.com/apache/iceberg-python/) implementation. +Features that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion. + +## Custom Table Provider + +You can implement a custom Data Provider in Rust and expose it to DataFusion through the +the interface as describe in the {ref}`Custom Table Provider ` +section. This is an advanced topic, but a +[user example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example) +is provided in the DataFusion repository. + +# Catalog + +A common technique for organizing tables is using a three level hierarchical approach. DataFusion +supports this form of organizing using the {py:class}`~datafusion.catalog.Catalog`, +{py:class}`~datafusion.catalog.Schema`, and {py:class}`~datafusion.catalog.Table`. By default, +a {py:class}`~datafusion.context.SessionContext` comes with a single Catalog and a single Schema +with the names `datafusion` and `public`, respectively. + +The default implementation uses an in-memory approach to the catalog and schema. We have support +for adding additional in-memory catalogs and schemas. You can access tables registered in a schema +either through the Dataframe API or via sql commands. This can be done like in the following +example: + +```python +import pyarrow as pa +from datafusion.catalog import Catalog, Schema +from datafusion import SessionContext + +ctx = SessionContext() + +my_catalog = Catalog.memory_catalog() +my_schema = Schema.memory_schema() +my_catalog.register_schema('my_schema_name', my_schema) +ctx.register_catalog_provider('my_catalog_name', my_catalog) + +# Create an in-memory table +table = pa.table({ + 'name': ['Bulbasaur', 'Charmander', 'Squirtle'], + 'type': ['Grass', 'Fire', 'Water'], + 'hp': [45, 39, 44], +}) +df = ctx.create_dataframe([table.to_batches()], name='pokemon') + +my_schema.register_table('pokemon', df) + +ctx.sql('SELECT * FROM my_catalog_name.my_schema_name.pokemon').show() +``` + +## User Defined Catalog and Schema + +If the in-memory catalogs are insufficient for your uses, there are two approaches you can take +to implementing a custom catalog and/or schema. In the below discussion, we describe how to +implement these for a Catalog, but the approach to implementing for a Schema is nearly +identical. + +DataFusion supports Catalogs written in either Rust or Python. If you write a Catalog in Rust, +you will need to export it as a Python library via PyO3. There is a complete example of a +catalog implemented this way in the +[examples folder](https://github.com/apache/datafusion-python/tree/main/examples/) +of our repository. Writing catalog providers in Rust provides typically can lead to significant +performance improvements over the Python based approach. + +To implement a Catalog in Python, you will need to inherit from the abstract base class +{py:class}`~datafusion.catalog.CatalogProvider`. There are examples in the +[unit tests](https://github.com/apache/datafusion-python/tree/main/python/tests) of +implementing a basic Catalog in Python where we simply keep a dictionary of the +registered Schemas. + +One important note for developers is that when we have a Catalog defined in Python, we have +two different ways of accessing this Catalog. First, we register the catalog with a Rust +wrapper. This allows for any rust based code to call the Python functions as necessary. +Second, if the user access the Catalog via the Python API, we identify this and return back +the original Python object that implements the Catalog. This is an important distinction +for developers because we do *not* return a Python wrapper around the Rust wrapper of the +original Python object. diff --git a/docs/source/user-guide/data-sources.rst b/docs/source/user-guide/data-sources.rst deleted file mode 100644 index 48ff4c014..000000000 --- a/docs/source/user-guide/data-sources.rst +++ /dev/null @@ -1,286 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _user_guide_data_sources: - -Data Sources -============ - -DataFusion provides a wide variety of ways to get data into a DataFrame to perform operations. - -Local file ----------- - -DataFusion has the ability to read from a variety of popular file formats, such as :ref:`Parquet `, -:ref:`CSV `, :ref:`JSON `, and :ref:`AVRO `. - -.. ipython:: python - - from datafusion import SessionContext - ctx = SessionContext() - df = ctx.read_csv("pokemon.csv") - df.show() - -Create in-memory ----------------- - -Sometimes it can be convenient to create a small DataFrame from a Python list or dictionary object. -To do this in DataFusion, you can use one of the three functions -:py:func:`~datafusion.context.SessionContext.from_pydict`, -:py:func:`~datafusion.context.SessionContext.from_pylist`, or -:py:func:`~datafusion.context.SessionContext.create_dataframe`. - -As their names suggest, ``from_pydict`` and ``from_pylist`` will create DataFrames from Python -dictionary and list objects, respectively. ``create_dataframe`` assumes you will pass in a list -of list of `PyArrow Record Batches `_. - -The following three examples all will create identical DataFrames: - -.. ipython:: python - - import pyarrow as pa - - ctx.from_pylist([ - { "a": 1, "b": 10.0, "c": "alpha" }, - { "a": 2, "b": 20.0, "c": "beta" }, - { "a": 3, "b": 30.0, "c": "gamma" }, - ]).show() - - ctx.from_pydict({ - "a": [1, 2, 3], - "b": [10.0, 20.0, 30.0], - "c": ["alpha", "beta", "gamma"], - }).show() - - batch = pa.RecordBatch.from_arrays( - [ - pa.array([1, 2, 3]), - pa.array([10.0, 20.0, 30.0]), - pa.array(["alpha", "beta", "gamma"]), - ], - names=["a", "b", "c"], - ) - - ctx.create_dataframe([[batch]]).show() - - -Object Store ------------- - -DataFusion has support for multiple storage options in addition to local files. -The example below requires an appropriate S3 account with access credentials. - -Supported Object Stores are - -- :py:class:`~datafusion.object_store.AmazonS3` -- :py:class:`~datafusion.object_store.GoogleCloud` -- :py:class:`~datafusion.object_store.Http` -- :py:class:`~datafusion.object_store.LocalFileSystem` -- :py:class:`~datafusion.object_store.MicrosoftAzure` - -.. code-block:: python - - from datafusion.object_store import AmazonS3 - - region = "us-east-1" - bucket_name = "yellow-trips" - - s3 = AmazonS3( - bucket_name=bucket_name, - region=region, - access_key_id=os.getenv("AWS_ACCESS_KEY_ID"), - secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"), - ) - - path = f"s3://{bucket_name}/" - ctx.register_object_store("s3://", s3, None) - - ctx.register_parquet("trips", path) - - ctx.table("trips").show() - -Other DataFrame Libraries -------------------------- - -DataFusion can import DataFrames directly from other libraries, such as -`Polars `_ and `Pandas `_. -Since DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule -interface can be imported to DataFusion using the -:py:func:`~datafusion.context.SessionContext.from_arrow` function. Older versions of Polars may -not support the arrow interface. In those cases, you can still import via the -:py:func:`~datafusion.context.SessionContext.from_polars` function. - -.. code-block:: python - - import pandas as pd - - data = { "a": [1, 2, 3], "b": [10.0, 20.0, 30.0], "c": ["alpha", "beta", "gamma"] } - pandas_df = pd.DataFrame(data) - - datafusion_df = ctx.from_arrow(pandas_df) - datafusion_df.show() - -.. code-block:: python - - import polars as pl - polars_df = pl.DataFrame(data) - - datafusion_df = ctx.from_arrow(polars_df) - datafusion_df.show() - -Delta Lake ----------- - -DataFusion 43.0.0 and later support the ability to register table providers from sources such -as Delta Lake. This will require a recent version of -`deltalake `_ to provide the required interfaces. - -.. code-block:: python - - from deltalake import DeltaTable - - delta_table = DeltaTable("path_to_table") - ctx.register_table("my_delta_table", delta_table) - df = ctx.table("my_delta_table") - df.show() - -On older versions of ``deltalake`` (prior to 0.22) you can use the -`Arrow DataSet `_ -interface to import to DataFusion, but this does not support features such as filter push down -which can lead to a significant performance difference. - -.. code-block:: python - - from deltalake import DeltaTable - - delta_table = DeltaTable("path_to_table") - ctx.register_dataset("my_delta_table", delta_table.to_pyarrow_dataset()) - df = ctx.table("my_delta_table") - df.show() - -Apache Iceberg --------------- - -DataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface. - -This requires either the `pyiceberg `__ library (>=0.10.0) or the `pyiceberg-core `__ library (>=0.5.0). - -* The ``pyiceberg-core`` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings. -* The ``pyiceberg`` library utilizes the ``pyiceberg-core`` python bindings under the hood and provides a native way for Python users to interact with the DataFusion. - -.. code-block:: python - - from datafusion import SessionContext - from pyiceberg.catalog import load_catalog - import pyarrow as pa - - # Load catalog and create/load a table - catalog = load_catalog("catalog", type="in-memory") - catalog.create_namespace_if_not_exists("default") - - # Create some sample data - data = pa.table({"x": [1, 2, 3], "y": [4, 5, 6]}) - iceberg_table = catalog.create_table("default.test", schema=data.schema) - iceberg_table.append(data) - - # Register the table with DataFusion - ctx = SessionContext() - ctx.register_table_provider("test", iceberg_table) - - # Query the table using DataFusion - ctx.table("test").show() - - -Note that the Datafusion integration rely on features from the `Iceberg Rust `_ implementation instead of the `PyIceberg `_ implementation. -Features that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion. - -Custom Table Provider ---------------------- - -You can implement a custom Data Provider in Rust and expose it to DataFusion through the -the interface as describe in the :ref:`Custom Table Provider ` -section. This is an advanced topic, but a -`user example `_ -is provided in the DataFusion repository. - -Catalog -======= - -A common technique for organizing tables is using a three level hierarchical approach. DataFusion -supports this form of organizing using the :py:class:`~datafusion.catalog.Catalog`, -:py:class:`~datafusion.catalog.Schema`, and :py:class:`~datafusion.catalog.Table`. By default, -a :py:class:`~datafusion.context.SessionContext` comes with a single Catalog and a single Schema -with the names ``datafusion`` and ``public``, respectively. - -The default implementation uses an in-memory approach to the catalog and schema. We have support -for adding additional in-memory catalogs and schemas. You can access tables registered in a schema -either through the Dataframe API or via sql commands. This can be done like in the following -example: - -.. code-block:: python - - import pyarrow as pa - from datafusion.catalog import Catalog, Schema - from datafusion import SessionContext - - ctx = SessionContext() - - my_catalog = Catalog.memory_catalog() - my_schema = Schema.memory_schema() - my_catalog.register_schema('my_schema_name', my_schema) - ctx.register_catalog_provider('my_catalog_name', my_catalog) - - # Create an in-memory table - table = pa.table({ - 'name': ['Bulbasaur', 'Charmander', 'Squirtle'], - 'type': ['Grass', 'Fire', 'Water'], - 'hp': [45, 39, 44], - }) - df = ctx.create_dataframe([table.to_batches()], name='pokemon') - - my_schema.register_table('pokemon', df) - - ctx.sql('SELECT * FROM my_catalog_name.my_schema_name.pokemon').show() - -User Defined Catalog and Schema -------------------------------- - -If the in-memory catalogs are insufficient for your uses, there are two approaches you can take -to implementing a custom catalog and/or schema. In the below discussion, we describe how to -implement these for a Catalog, but the approach to implementing for a Schema is nearly -identical. - -DataFusion supports Catalogs written in either Rust or Python. If you write a Catalog in Rust, -you will need to export it as a Python library via PyO3. There is a complete example of a -catalog implemented this way in the -`examples folder `_ -of our repository. Writing catalog providers in Rust provides typically can lead to significant -performance improvements over the Python based approach. - -To implement a Catalog in Python, you will need to inherit from the abstract base class -:py:class:`~datafusion.catalog.CatalogProvider`. There are examples in the -`unit tests `_ of -implementing a basic Catalog in Python where we simply keep a dictionary of the -registered Schemas. - -One important note for developers is that when we have a Catalog defined in Python, we have -two different ways of accessing this Catalog. First, we register the catalog with a Rust -wrapper. This allows for any rust based code to call the Python functions as necessary. -Second, if the user access the Catalog via the Python API, we identify this and return back -the original Python object that implements the Catalog. This is an important distinction -for developers because we do *not* return a Python wrapper around the Rust wrapper of the -original Python object. diff --git a/docs/source/user-guide/dataframe/execution-metrics.md b/docs/source/user-guide/dataframe/execution-metrics.md new file mode 100644 index 000000000..e66ea1100 --- /dev/null +++ b/docs/source/user-guide/dataframe/execution-metrics.md @@ -0,0 +1,219 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(execution_metrics)= + +# Execution Metrics + +## Overview + +When DataFusion executes a query it compiles the logical plan into a tree of +*physical plan operators* (e.g. `FilterExec`, `ProjectionExec`, +`HashAggregateExec`). Each operator can record runtime statistics while it +runs. These statistics are called **execution metrics**. + +Typical metrics include: + +- **output_rows** – number of rows produced by the operator +- **elapsed_compute** – total CPU time (nanoseconds) spent inside the operator +- **spill_count** – number of times the operator spilled data to disk +- **spilled_bytes** – total bytes written to disk during spills +- **spilled_rows** – total rows written to disk during spills + +Metrics are collected *per-partition*: DataFusion may execute each operator +in parallel across several partitions. The convenience properties on +{py:class}`~datafusion.MetricsSet` (e.g. `output_rows`, `elapsed_compute`) +automatically sum the named metric across **all** partitions, giving a single +aggregate value for the operator as a whole. You can also access the raw +per-partition {py:class}`~datafusion.Metric` objects via +{py:meth}`~datafusion.MetricsSet.metrics`. + +## When Are Metrics Available? + +Some operators (for example `DataSourceExec`) eagerly create a +{py:class}`~datafusion.MetricsSet` when the physical plan is built, so +{py:meth}`~datafusion.ExecutionPlan.metrics` may return a set even before any +rows have been processed. However, metric **values** such as `output_rows` +are only meaningful **after** the DataFrame has been executed via one of the +terminal operations: + +- {py:meth}`~datafusion.DataFrame.collect` +- {py:meth}`~datafusion.DataFrame.collect_partitioned` +- {py:meth}`~datafusion.DataFrame.execute_stream` + (metrics are available once the stream has been fully consumed) +- {py:meth}`~datafusion.DataFrame.execute_stream_partitioned` + (metrics are available once all partition streams have been fully consumed) + +Before execution, metric values will be `0` or `None`. + +:::{note} +**display() does not populate metrics.** +When a DataFrame is displayed in a notebook (e.g. via `display(df)` or +automatic `repr` output), DataFusion runs a *limited* internal execution +to fetch preview rows. This internal execution does **not** cache the +physical plan used, so {py:meth}`~datafusion.ExecutionPlan.collect_metrics` +will not reflect the display execution. To access metrics you must call +one of the terminal operations listed above. +::: + +If you call {py:meth}`~datafusion.DataFrame.collect` (or another terminal +operation) multiple times on the same DataFrame, each call creates a fresh +physical plan. Metrics from {py:meth}`~datafusion.DataFrame.execution_plan` +always reflect the **most recent** execution. + +## Reading the Physical Plan Tree + +{py:meth}`~datafusion.DataFrame.execution_plan` returns the root +{py:class}`~datafusion.ExecutionPlan` node of the physical plan tree. The tree +mirrors the operator pipeline: the root is typically a projection or +coalescing node; its children are filters, aggregates, scans, etc. + +The `operator_name` string returned by +{py:meth}`~datafusion.ExecutionPlan.collect_metrics` is the *display* name of +the node, for example `"FilterExec: column1@0 > 1"`. This is the same string +you would see when calling `plan.display()`. + +## Aggregated vs Per-Partition Metrics + +DataFusion executes each operator across one or more **partitions** in +parallel. The {py:class}`~datafusion.MetricsSet` convenience properties +(`output_rows`, `elapsed_compute`, etc.) automatically **sum** the named +metric across all partitions, giving a single aggregate value. + +To inspect individual partitions — for example to detect data skew where one +partition processes far more rows than others — iterate over the raw +{py:class}`~datafusion.Metric` objects: + +```python +for metric in metrics_set.metrics(): + print(f" partition={metric.partition} {metric.name}={metric.value}") +``` + +The `partition` property is a 0-based index (`0`, `1`, …) identifying +which parallel slot processed this metric. It is `None` for metrics that +apply globally (not tied to a specific partition). + +## Available Metrics + +The following metrics are directly accessible as properties on +{py:class}`~datafusion.MetricsSet`: + +```{eval-rst} +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Property + - Description + * - ``output_rows`` + - Number of rows emitted by the operator (summed across partitions). + * - ``elapsed_compute`` + - Wall-clock CPU time **in nanoseconds** spent inside the operator's + compute loop, excluding I/O wait. Useful for identifying which + operators are most expensive (summed across partitions). + * - ``spill_count`` + - Number of spill-to-disk events triggered by memory pressure. This is + a unitless count of events, not a measure of data volume (summed across + partitions). + * - ``spilled_bytes`` + - Total bytes written to disk during spill events (summed across + partitions). + * - ``spilled_rows`` + - Total rows written to disk during spill events (summed across + partitions). +``` + +Any metric not listed above can be accessed via +{py:meth}`~datafusion.MetricsSet.sum_by_name`, or by iterating over the raw +{py:class}`~datafusion.Metric` objects returned by +{py:meth}`~datafusion.MetricsSet.metrics`. + +## Labels + +A {py:class}`~datafusion.Metric` may carry *labels*: key/value pairs that +provide additional context. Labels are operator-specific; most metrics have +an empty label dict. + +Some operators tag their metrics with labels to distinguish variants. For +example, a `HashAggregateExec` may record separate `output_rows` metrics +for intermediate and final output: + +```python +for metric in metrics_set.metrics(): + print(metric.name, metric.labels()) +# output_rows {'output_type': 'final'} +# output_rows {'output_type': 'intermediate'} +``` + +When summing by name (via {py:attr}`~datafusion.MetricsSet.output_rows` or +{py:meth}`~datafusion.MetricsSet.sum_by_name`), **all** metrics with that +name are summed regardless of labels. To filter by label, iterate over the +raw {py:class}`~datafusion.Metric` objects directly. + +## End-to-End Example + +```python +from datafusion import SessionContext + +ctx = SessionContext() +ctx.sql("CREATE TABLE sales AS VALUES (1, 100), (2, 200), (3, 50)") + +df = ctx.sql("SELECT * FROM sales WHERE column1 > 1") + +# Execute the query — this populates the metrics +results = df.collect() + +# Retrieve the physical plan with metrics +plan = df.execution_plan() + +# Walk every operator and print its metrics +for operator_name, ms in plan.collect_metrics(): + if ms.output_rows is not None: + print(f"{operator_name}") + print(f" output_rows = {ms.output_rows}") + print(f" elapsed_compute = {ms.elapsed_compute} ns") + +# Access raw per-partition metrics +for operator_name, ms in plan.collect_metrics(): + for metric in ms.metrics(): + print( + f" partition={metric.partition} " + f"{metric.name}={metric.value} " + f"labels={metric.labels()}" + ) +``` + +## API Reference + +- {py:class}`datafusion.ExecutionPlan` — physical plan node +- {py:meth}`datafusion.ExecutionPlan.collect_metrics` — walk the tree and + return `(operator_name, MetricsSet)` pairs +- {py:meth}`datafusion.ExecutionPlan.metrics` — return the + {py:class}`~datafusion.MetricsSet` for a single node +- {py:class}`datafusion.MetricsSet` — aggregated metrics for one operator +- {py:class}`datafusion.Metric` — a single per-partition metric value diff --git a/docs/source/user-guide/dataframe/execution-metrics.rst b/docs/source/user-guide/dataframe/execution-metrics.rst deleted file mode 100644 index 764fa76ef..000000000 --- a/docs/source/user-guide/dataframe/execution-metrics.rst +++ /dev/null @@ -1,215 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _execution_metrics: - -Execution Metrics -================= - -Overview --------- - -When DataFusion executes a query it compiles the logical plan into a tree of -*physical plan operators* (e.g. ``FilterExec``, ``ProjectionExec``, -``HashAggregateExec``). Each operator can record runtime statistics while it -runs. These statistics are called **execution metrics**. - -Typical metrics include: - -- **output_rows** – number of rows produced by the operator -- **elapsed_compute** – total CPU time (nanoseconds) spent inside the operator -- **spill_count** – number of times the operator spilled data to disk -- **spilled_bytes** – total bytes written to disk during spills -- **spilled_rows** – total rows written to disk during spills - -Metrics are collected *per-partition*: DataFusion may execute each operator -in parallel across several partitions. The convenience properties on -:py:class:`~datafusion.MetricsSet` (e.g. ``output_rows``, ``elapsed_compute``) -automatically sum the named metric across **all** partitions, giving a single -aggregate value for the operator as a whole. You can also access the raw -per-partition :py:class:`~datafusion.Metric` objects via -:py:meth:`~datafusion.MetricsSet.metrics`. - -When Are Metrics Available? ---------------------------- - -Some operators (for example ``DataSourceExec``) eagerly create a -:py:class:`~datafusion.MetricsSet` when the physical plan is built, so -:py:meth:`~datafusion.ExecutionPlan.metrics` may return a set even before any -rows have been processed. However, metric **values** such as ``output_rows`` -are only meaningful **after** the DataFrame has been executed via one of the -terminal operations: - -- :py:meth:`~datafusion.DataFrame.collect` -- :py:meth:`~datafusion.DataFrame.collect_partitioned` -- :py:meth:`~datafusion.DataFrame.execute_stream` - (metrics are available once the stream has been fully consumed) -- :py:meth:`~datafusion.DataFrame.execute_stream_partitioned` - (metrics are available once all partition streams have been fully consumed) - -Before execution, metric values will be ``0`` or ``None``. - -.. note:: - - **display() does not populate metrics.** - When a DataFrame is displayed in a notebook (e.g. via ``display(df)`` or - automatic ``repr`` output), DataFusion runs a *limited* internal execution - to fetch preview rows. This internal execution does **not** cache the - physical plan used, so :py:meth:`~datafusion.ExecutionPlan.collect_metrics` - will not reflect the display execution. To access metrics you must call - one of the terminal operations listed above. - -If you call :py:meth:`~datafusion.DataFrame.collect` (or another terminal -operation) multiple times on the same DataFrame, each call creates a fresh -physical plan. Metrics from :py:meth:`~datafusion.DataFrame.execution_plan` -always reflect the **most recent** execution. - -Reading the Physical Plan Tree --------------------------------- - -:py:meth:`~datafusion.DataFrame.execution_plan` returns the root -:py:class:`~datafusion.ExecutionPlan` node of the physical plan tree. The tree -mirrors the operator pipeline: the root is typically a projection or -coalescing node; its children are filters, aggregates, scans, etc. - -The ``operator_name`` string returned by -:py:meth:`~datafusion.ExecutionPlan.collect_metrics` is the *display* name of -the node, for example ``"FilterExec: column1@0 > 1"``. This is the same string -you would see when calling ``plan.display()``. - -Aggregated vs Per-Partition Metrics ------------------------------------- - -DataFusion executes each operator across one or more **partitions** in -parallel. The :py:class:`~datafusion.MetricsSet` convenience properties -(``output_rows``, ``elapsed_compute``, etc.) automatically **sum** the named -metric across all partitions, giving a single aggregate value. - -To inspect individual partitions — for example to detect data skew where one -partition processes far more rows than others — iterate over the raw -:py:class:`~datafusion.Metric` objects: - -.. code-block:: python - - for metric in metrics_set.metrics(): - print(f" partition={metric.partition} {metric.name}={metric.value}") - -The ``partition`` property is a 0-based index (``0``, ``1``, …) identifying -which parallel slot processed this metric. It is ``None`` for metrics that -apply globally (not tied to a specific partition). - -Available Metrics ------------------ - -The following metrics are directly accessible as properties on -:py:class:`~datafusion.MetricsSet`: - -.. list-table:: - :header-rows: 1 - :widths: 25 75 - - * - Property - - Description - * - ``output_rows`` - - Number of rows emitted by the operator (summed across partitions). - * - ``elapsed_compute`` - - Wall-clock CPU time **in nanoseconds** spent inside the operator's - compute loop, excluding I/O wait. Useful for identifying which - operators are most expensive (summed across partitions). - * - ``spill_count`` - - Number of spill-to-disk events triggered by memory pressure. This is - a unitless count of events, not a measure of data volume (summed across - partitions). - * - ``spilled_bytes`` - - Total bytes written to disk during spill events (summed across - partitions). - * - ``spilled_rows`` - - Total rows written to disk during spill events (summed across - partitions). - -Any metric not listed above can be accessed via -:py:meth:`~datafusion.MetricsSet.sum_by_name`, or by iterating over the raw -:py:class:`~datafusion.Metric` objects returned by -:py:meth:`~datafusion.MetricsSet.metrics`. - -Labels ------- - -A :py:class:`~datafusion.Metric` may carry *labels*: key/value pairs that -provide additional context. Labels are operator-specific; most metrics have -an empty label dict. - -Some operators tag their metrics with labels to distinguish variants. For -example, a ``HashAggregateExec`` may record separate ``output_rows`` metrics -for intermediate and final output: - -.. code-block:: python - - for metric in metrics_set.metrics(): - print(metric.name, metric.labels()) - # output_rows {'output_type': 'final'} - # output_rows {'output_type': 'intermediate'} - -When summing by name (via :py:attr:`~datafusion.MetricsSet.output_rows` or -:py:meth:`~datafusion.MetricsSet.sum_by_name`), **all** metrics with that -name are summed regardless of labels. To filter by label, iterate over the -raw :py:class:`~datafusion.Metric` objects directly. - -End-to-End Example ------------------- - -.. code-block:: python - - from datafusion import SessionContext - - ctx = SessionContext() - ctx.sql("CREATE TABLE sales AS VALUES (1, 100), (2, 200), (3, 50)") - - df = ctx.sql("SELECT * FROM sales WHERE column1 > 1") - - # Execute the query — this populates the metrics - results = df.collect() - - # Retrieve the physical plan with metrics - plan = df.execution_plan() - - # Walk every operator and print its metrics - for operator_name, ms in plan.collect_metrics(): - if ms.output_rows is not None: - print(f"{operator_name}") - print(f" output_rows = {ms.output_rows}") - print(f" elapsed_compute = {ms.elapsed_compute} ns") - - # Access raw per-partition metrics - for operator_name, ms in plan.collect_metrics(): - for metric in ms.metrics(): - print( - f" partition={metric.partition} " - f"{metric.name}={metric.value} " - f"labels={metric.labels()}" - ) - -API Reference -------------- - -- :py:class:`datafusion.ExecutionPlan` — physical plan node -- :py:meth:`datafusion.ExecutionPlan.collect_metrics` — walk the tree and - return ``(operator_name, MetricsSet)`` pairs -- :py:meth:`datafusion.ExecutionPlan.metrics` — return the - :py:class:`~datafusion.MetricsSet` for a single node -- :py:class:`datafusion.MetricsSet` — aggregated metrics for one operator -- :py:class:`datafusion.Metric` — a single per-partition metric value diff --git a/docs/source/user-guide/dataframe/index.md b/docs/source/user-guide/dataframe/index.md new file mode 100644 index 000000000..dd7d949e1 --- /dev/null +++ b/docs/source/user-guide/dataframe/index.md @@ -0,0 +1,380 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# DataFrames + +## Overview + +The `DataFrame` class is the core abstraction in DataFusion that represents tabular data and operations +on that data. DataFrames provide a flexible API for transforming data through various operations such as +filtering, projection, aggregation, joining, and more. + +A DataFrame represents a logical plan that is lazily evaluated. The actual execution occurs only when +terminal operations like `collect()`, `show()`, or `to_pandas()` are called. + +## Creating DataFrames + +DataFrames can be created in several ways: + +- From SQL queries via a `SessionContext`: + + ```python + from datafusion import SessionContext + + ctx = SessionContext() + df = ctx.sql("SELECT * FROM your_table") + ``` + +- From registered tables: + + ```python + df = ctx.table("your_table") + ``` + +- From various data sources: + + ```python + # From CSV files (see :ref:`io_csv` for detailed options) + df = ctx.read_csv("path/to/data.csv") + + # From Parquet files (see :ref:`io_parquet` for detailed options) + df = ctx.read_parquet("path/to/data.parquet") + + # From JSON files (see :ref:`io_json` for detailed options) + df = ctx.read_json("path/to/data.json") + + # From Avro files (see :ref:`io_avro` for detailed options) + df = ctx.read_avro("path/to/data.avro") + + # From Pandas DataFrame + import pandas as pd + pandas_df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}) + df = ctx.from_pandas(pandas_df) + + # From Arrow data + import pyarrow as pa + batch = pa.RecordBatch.from_arrays( + [pa.array([1, 2, 3]), pa.array([4, 5, 6])], + names=["a", "b"] + ) + df = ctx.from_arrow(batch) + ``` + +For detailed information about reading from different data sources, see the {doc}`I/O Guide <../io/index>`. +For custom data sources, see {ref}`io_custom_table_provider`. + +## Common DataFrame Operations + +DataFusion's DataFrame API offers a wide range of operations: + +```python +from datafusion import column, literal + +# Select specific columns +df = df.select("col1", "col2") + +# Select with expressions +df = df.select(column("a") + column("b"), column("a") - column("b")) + +# Filter rows (expressions or SQL strings) +df = df.filter(column("age") > literal(25)) +df = df.filter("age > 25") + +# Add computed columns +df = df.with_column("full_name", column("first_name") + literal(" ") + column("last_name")) + +# Multiple column additions +df = df.with_columns( + (column("a") + column("b")).alias("sum"), + (column("a") * column("b")).alias("product") +) + +# Sort data +df = df.sort(column("age").sort(ascending=False)) + +# Join DataFrames +df = df1.join(df2, on="user_id", how="inner") + +# Aggregate data +from datafusion import functions as f +df = df.aggregate( + [], # Group by columns (empty for global aggregation) + [f.sum(column("amount")).alias("total_amount")] +) + +# Limit rows +df = df.limit(100) + +# Drop columns +df = df.drop("temporary_column") +``` + +## Column Names as Function Arguments + +Some `DataFrame` methods accept column names when an argument refers to an +existing column. These include: + +- {py:meth}`~datafusion.DataFrame.select` +- {py:meth}`~datafusion.DataFrame.sort` +- {py:meth}`~datafusion.DataFrame.drop` +- {py:meth}`~datafusion.DataFrame.join` (`on` argument) +- {py:meth}`~datafusion.DataFrame.aggregate` (grouping columns) + +See the full function documentation for details on any specific function. + +Note that {py:meth}`~datafusion.DataFrame.join_on` expects `col()`/`column()` expressions rather than plain strings. + +For such methods, you can pass column names directly: + +```python +from datafusion import col, functions as f + +df.sort('id') +df.aggregate('id', [f.count(col('value'))]) +``` + +The same operation can also be written with explicit column expressions, using either `col()` or `column()`: + +```python +from datafusion import col, column, functions as f + +df.sort(col('id')) +df.aggregate(column('id'), [f.count(col('value'))]) +``` + +Note that `column()` is an alias of `col()`, so you can use either name; the example above shows both in action. + +Whenever an argument represents an expression—such as in +{py:meth}`~datafusion.DataFrame.filter` or +{py:meth}`~datafusion.DataFrame.with_column`—use `col()` to reference +columns. The comparison and arithmetic operators on `Expr` will automatically +convert any non-`Expr` value into a literal expression, so writing + +```python +from datafusion import col +df.filter(col("age") > 21) +``` + +is equivalent to using `lit(21)` explicitly. Use `lit()` (also available +as `literal()`) when you need to construct a literal expression directly. + +## Terminal Operations + +To materialize the results of your DataFrame operations: + +```python +# Collect all data as PyArrow RecordBatches +result_batches = df.collect() + +# Convert to various formats +pandas_df = df.to_pandas() # Pandas DataFrame +polars_df = df.to_polars() # Polars DataFrame +arrow_table = df.to_arrow_table() # PyArrow Table +py_dict = df.to_pydict() # Python dictionary +py_list = df.to_pylist() # Python list of dictionaries + +# Display results +df.show() # Print tabular format to console + +# Count rows +count = df.count() + +# Collect a single column of data as a PyArrow Array +arr = df.collect_column("age") +``` + +## Zero-copy streaming to Arrow-based Python libraries + +DataFusion DataFrames implement the `__arrow_c_stream__` protocol, enabling +zero-copy, lazy streaming into Arrow-based Python libraries. With the streaming +protocol, batches are produced on demand. + +:::{note} +The protocol is implementation-agnostic and works with any Python library +that understands the Arrow C streaming interface (for example, PyArrow +or other Arrow-compatible implementations). The sections below provide a +short PyArrow-specific example and general guidance for other +implementations. +::: + +## PyArrow + +```python +import pyarrow as pa + +# Create a PyArrow RecordBatchReader without materializing all batches +reader = pa.RecordBatchReader.from_stream(df) +for batch in reader: + ... # process each batch as it is produced +``` + +DataFrames are also iterable, yielding {class}`datafusion.RecordBatch` +objects lazily so you can loop over results directly without importing +PyArrow: + +```python +for batch in df: + ... # each batch is a ``datafusion.RecordBatch`` +``` + +Each batch exposes `to_pyarrow()`, allowing conversion to a PyArrow +table. `pa.table(df)` collects the entire DataFrame eagerly into a +PyArrow table: + +```python +import pyarrow as pa +table = pa.table(df) +``` + +Asynchronous iteration is supported as well, allowing integration with +`asyncio` event loops: + +```python +async for batch in df: + ... # process each batch as it is produced +``` + +To work with the stream directly, use `execute_stream()`, which returns a +{class}`~datafusion.RecordBatchStream`. + +```python +stream = df.execute_stream() +for batch in stream: + ... +``` + +### Execute as Stream + +For finer control over streaming execution, use +{py:meth}`~datafusion.DataFrame.execute_stream` to obtain a +{py:class}`datafusion.RecordBatchStream`: + +```python +stream = df.execute_stream() +for batch in stream: + ... # process each batch as it is produced +``` + +:::{tip} +To get a PyArrow reader instead, call + +`pa.RecordBatchReader.from_stream(df)`. +::: + +When partition boundaries are important, +{py:meth}`~datafusion.DataFrame.execute_stream_partitioned` +returns an iterable of {py:class}`datafusion.RecordBatchStream` objects, one per +partition: + +```python +for stream in df.execute_stream_partitioned(): + for batch in stream: + ... # each stream yields RecordBatches +``` + +To process partitions concurrently, first collect the streams into a list +and then poll each one in a separate `asyncio` task: + +```python +import asyncio + +async def consume(stream): + async for batch in stream: + ... + +streams = list(df.execute_stream_partitioned()) +await asyncio.gather(*(consume(s) for s in streams)) +``` + +See {doc}`../io/arrow` for additional details on the Arrow interface. + +## HTML Rendering + +When working in Jupyter notebooks or other environments that support HTML rendering, DataFrames will +automatically display as formatted HTML tables. For detailed information about customizing HTML +rendering, formatting options, and advanced styling, see {doc}`rendering`. + +## Core Classes + +**DataFrame** + +: The main DataFrame class for building and executing queries. + + See: {py:class}`datafusion.DataFrame` + +**SessionContext** + +: The primary entry point for creating DataFrames from various data sources. + + Key methods for DataFrame creation: + + - {py:meth}`~datafusion.SessionContext.read_csv` - Read CSV files + - {py:meth}`~datafusion.SessionContext.read_parquet` - Read Parquet files + - {py:meth}`~datafusion.SessionContext.read_json` - Read JSON files + - {py:meth}`~datafusion.SessionContext.read_avro` - Read Avro files + - {py:meth}`~datafusion.SessionContext.table` - Access registered tables + - {py:meth}`~datafusion.SessionContext.sql` - Execute SQL queries + - {py:meth}`~datafusion.SessionContext.from_pandas` - Create from Pandas DataFrame + - {py:meth}`~datafusion.SessionContext.from_arrow` - Create from Arrow data + + See: {py:class}`datafusion.SessionContext` + +## Expression Classes + +**Expr** + +: Represents expressions that can be used in DataFrame operations. + + See: {py:class}`datafusion.Expr` + +**Functions for creating expressions:** + +- {py:func}`datafusion.column` - Reference a column by name +- {py:func}`datafusion.literal` - Create a literal value expression + +## Built-in Functions + +DataFusion provides many built-in functions for data manipulation: + +- {py:mod}`datafusion.functions` - Mathematical, string, date/time, and aggregation functions + +For a complete list of available functions, see the {py:mod}`datafusion.functions` module documentation. + +## Execution Metrics + +After executing a DataFrame (via `collect()`, `execute_stream()`, etc.), +DataFusion populates per-operator runtime statistics such as row counts and +compute time. See {doc}`execution-metrics` for a full explanation and +worked example. + +```{toctree} +:maxdepth: 1 + +rendering +execution-metrics +``` diff --git a/docs/source/user-guide/dataframe/index.rst b/docs/source/user-guide/dataframe/index.rst deleted file mode 100644 index 8475a7bd7..000000000 --- a/docs/source/user-guide/dataframe/index.rst +++ /dev/null @@ -1,380 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -DataFrames -========== - -Overview --------- - -The ``DataFrame`` class is the core abstraction in DataFusion that represents tabular data and operations -on that data. DataFrames provide a flexible API for transforming data through various operations such as -filtering, projection, aggregation, joining, and more. - -A DataFrame represents a logical plan that is lazily evaluated. The actual execution occurs only when -terminal operations like ``collect()``, ``show()``, or ``to_pandas()`` are called. - -Creating DataFrames -------------------- - -DataFrames can be created in several ways: - -* From SQL queries via a ``SessionContext``: - - .. code-block:: python - - from datafusion import SessionContext - - ctx = SessionContext() - df = ctx.sql("SELECT * FROM your_table") - -* From registered tables: - - .. code-block:: python - - df = ctx.table("your_table") - -* From various data sources: - - .. code-block:: python - - # From CSV files (see :ref:`io_csv` for detailed options) - df = ctx.read_csv("path/to/data.csv") - - # From Parquet files (see :ref:`io_parquet` for detailed options) - df = ctx.read_parquet("path/to/data.parquet") - - # From JSON files (see :ref:`io_json` for detailed options) - df = ctx.read_json("path/to/data.json") - - # From Avro files (see :ref:`io_avro` for detailed options) - df = ctx.read_avro("path/to/data.avro") - - # From Pandas DataFrame - import pandas as pd - pandas_df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}) - df = ctx.from_pandas(pandas_df) - - # From Arrow data - import pyarrow as pa - batch = pa.RecordBatch.from_arrays( - [pa.array([1, 2, 3]), pa.array([4, 5, 6])], - names=["a", "b"] - ) - df = ctx.from_arrow(batch) - -For detailed information about reading from different data sources, see the :doc:`I/O Guide <../io/index>`. -For custom data sources, see :ref:`io_custom_table_provider`. - -Common DataFrame Operations ---------------------------- - -DataFusion's DataFrame API offers a wide range of operations: - -.. code-block:: python - - from datafusion import column, literal - - # Select specific columns - df = df.select("col1", "col2") - - # Select with expressions - df = df.select(column("a") + column("b"), column("a") - column("b")) - - # Filter rows (expressions or SQL strings) - df = df.filter(column("age") > literal(25)) - df = df.filter("age > 25") - - # Add computed columns - df = df.with_column("full_name", column("first_name") + literal(" ") + column("last_name")) - - # Multiple column additions - df = df.with_columns( - (column("a") + column("b")).alias("sum"), - (column("a") * column("b")).alias("product") - ) - - # Sort data - df = df.sort(column("age").sort(ascending=False)) - - # Join DataFrames - df = df1.join(df2, on="user_id", how="inner") - - # Aggregate data - from datafusion import functions as f - df = df.aggregate( - [], # Group by columns (empty for global aggregation) - [f.sum(column("amount")).alias("total_amount")] - ) - - # Limit rows - df = df.limit(100) - - # Drop columns - df = df.drop("temporary_column") - -Column Names as Function Arguments ----------------------------------- - -Some ``DataFrame`` methods accept column names when an argument refers to an -existing column. These include: - -* :py:meth:`~datafusion.DataFrame.select` -* :py:meth:`~datafusion.DataFrame.sort` -* :py:meth:`~datafusion.DataFrame.drop` -* :py:meth:`~datafusion.DataFrame.join` (``on`` argument) -* :py:meth:`~datafusion.DataFrame.aggregate` (grouping columns) - -See the full function documentation for details on any specific function. - -Note that :py:meth:`~datafusion.DataFrame.join_on` expects ``col()``/``column()`` expressions rather than plain strings. - -For such methods, you can pass column names directly: - -.. code-block:: python - - from datafusion import col, functions as f - - df.sort('id') - df.aggregate('id', [f.count(col('value'))]) - -The same operation can also be written with explicit column expressions, using either ``col()`` or ``column()``: - -.. code-block:: python - - from datafusion import col, column, functions as f - - df.sort(col('id')) - df.aggregate(column('id'), [f.count(col('value'))]) - -Note that ``column()`` is an alias of ``col()``, so you can use either name; the example above shows both in action. - -Whenever an argument represents an expression—such as in -:py:meth:`~datafusion.DataFrame.filter` or -:py:meth:`~datafusion.DataFrame.with_column`—use ``col()`` to reference -columns. The comparison and arithmetic operators on ``Expr`` will automatically -convert any non-``Expr`` value into a literal expression, so writing - -.. code-block:: python - - from datafusion import col - df.filter(col("age") > 21) - -is equivalent to using ``lit(21)`` explicitly. Use ``lit()`` (also available -as ``literal()``) when you need to construct a literal expression directly. - -Terminal Operations -------------------- - -To materialize the results of your DataFrame operations: - -.. code-block:: python - - # Collect all data as PyArrow RecordBatches - result_batches = df.collect() - - # Convert to various formats - pandas_df = df.to_pandas() # Pandas DataFrame - polars_df = df.to_polars() # Polars DataFrame - arrow_table = df.to_arrow_table() # PyArrow Table - py_dict = df.to_pydict() # Python dictionary - py_list = df.to_pylist() # Python list of dictionaries - - # Display results - df.show() # Print tabular format to console - - # Count rows - count = df.count() - - # Collect a single column of data as a PyArrow Array - arr = df.collect_column("age") - -Zero-copy streaming to Arrow-based Python libraries ---------------------------------------------------- - -DataFusion DataFrames implement the ``__arrow_c_stream__`` protocol, enabling -zero-copy, lazy streaming into Arrow-based Python libraries. With the streaming -protocol, batches are produced on demand. - -.. note:: - - The protocol is implementation-agnostic and works with any Python library - that understands the Arrow C streaming interface (for example, PyArrow - or other Arrow-compatible implementations). The sections below provide a - short PyArrow-specific example and general guidance for other - implementations. - -PyArrow -------- - -.. code-block:: python - - import pyarrow as pa - - # Create a PyArrow RecordBatchReader without materializing all batches - reader = pa.RecordBatchReader.from_stream(df) - for batch in reader: - ... # process each batch as it is produced - -DataFrames are also iterable, yielding :class:`datafusion.RecordBatch` -objects lazily so you can loop over results directly without importing -PyArrow: - -.. code-block:: python - - for batch in df: - ... # each batch is a ``datafusion.RecordBatch`` - -Each batch exposes ``to_pyarrow()``, allowing conversion to a PyArrow -table. ``pa.table(df)`` collects the entire DataFrame eagerly into a -PyArrow table: - -.. code-block:: python - - import pyarrow as pa - table = pa.table(df) - -Asynchronous iteration is supported as well, allowing integration with -``asyncio`` event loops: - -.. code-block:: python - - async for batch in df: - ... # process each batch as it is produced - -To work with the stream directly, use ``execute_stream()``, which returns a -:class:`~datafusion.RecordBatchStream`. - -.. code-block:: python - - stream = df.execute_stream() - for batch in stream: - ... - -Execute as Stream -^^^^^^^^^^^^^^^^^ - -For finer control over streaming execution, use -:py:meth:`~datafusion.DataFrame.execute_stream` to obtain a -:py:class:`datafusion.RecordBatchStream`: - -.. code-block:: python - - stream = df.execute_stream() - for batch in stream: - ... # process each batch as it is produced - -.. tip:: - - To get a PyArrow reader instead, call - - ``pa.RecordBatchReader.from_stream(df)``. - -When partition boundaries are important, -:py:meth:`~datafusion.DataFrame.execute_stream_partitioned` -returns an iterable of :py:class:`datafusion.RecordBatchStream` objects, one per -partition: - -.. code-block:: python - - for stream in df.execute_stream_partitioned(): - for batch in stream: - ... # each stream yields RecordBatches - -To process partitions concurrently, first collect the streams into a list -and then poll each one in a separate ``asyncio`` task: - -.. code-block:: python - - import asyncio - - async def consume(stream): - async for batch in stream: - ... - - streams = list(df.execute_stream_partitioned()) - await asyncio.gather(*(consume(s) for s in streams)) - -See :doc:`../io/arrow` for additional details on the Arrow interface. - -HTML Rendering --------------- - -When working in Jupyter notebooks or other environments that support HTML rendering, DataFrames will -automatically display as formatted HTML tables. For detailed information about customizing HTML -rendering, formatting options, and advanced styling, see :doc:`rendering`. - -Core Classes ------------- - -**DataFrame** - The main DataFrame class for building and executing queries. - - See: :py:class:`datafusion.DataFrame` - -**SessionContext** - The primary entry point for creating DataFrames from various data sources. - - Key methods for DataFrame creation: - - * :py:meth:`~datafusion.SessionContext.read_csv` - Read CSV files - * :py:meth:`~datafusion.SessionContext.read_parquet` - Read Parquet files - * :py:meth:`~datafusion.SessionContext.read_json` - Read JSON files - * :py:meth:`~datafusion.SessionContext.read_avro` - Read Avro files - * :py:meth:`~datafusion.SessionContext.table` - Access registered tables - * :py:meth:`~datafusion.SessionContext.sql` - Execute SQL queries - * :py:meth:`~datafusion.SessionContext.from_pandas` - Create from Pandas DataFrame - * :py:meth:`~datafusion.SessionContext.from_arrow` - Create from Arrow data - - See: :py:class:`datafusion.SessionContext` - -Expression Classes ------------------- - -**Expr** - Represents expressions that can be used in DataFrame operations. - - See: :py:class:`datafusion.Expr` - -**Functions for creating expressions:** - -* :py:func:`datafusion.column` - Reference a column by name -* :py:func:`datafusion.literal` - Create a literal value expression - -Built-in Functions ------------------- - -DataFusion provides many built-in functions for data manipulation: - -* :py:mod:`datafusion.functions` - Mathematical, string, date/time, and aggregation functions - -For a complete list of available functions, see the :py:mod:`datafusion.functions` module documentation. - - -Execution Metrics ------------------ - -After executing a DataFrame (via ``collect()``, ``execute_stream()``, etc.), -DataFusion populates per-operator runtime statistics such as row counts and -compute time. See :doc:`execution-metrics` for a full explanation and -worked example. - -.. toctree:: - :maxdepth: 1 - - rendering - execution-metrics diff --git a/docs/source/user-guide/dataframe/rendering.md b/docs/source/user-guide/dataframe/rendering.md new file mode 100644 index 000000000..d92d9b386 --- /dev/null +++ b/docs/source/user-guide/dataframe/rendering.md @@ -0,0 +1,236 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# DataFrame Rendering + +DataFusion provides configurable rendering for DataFrames in both plain text and HTML +formats. The `datafusion.dataframe_formatter` module controls how DataFrames are +displayed in Jupyter notebooks (via `_repr_html_`), in the terminal (via `__repr__`), +and anywhere else a string or HTML representation is needed. + +## Basic Rendering + +In a Jupyter environment, displaying a DataFrame triggers HTML rendering: + +```python +# Will display as HTML table in Jupyter +df + +# Explicit display also uses HTML rendering +display(df) +``` + +In a terminal or when converting to string, plain text rendering is used: + +```python +# Plain text table output +print(df) +``` + +## Configuring the Formatter + +You can customize how DataFrames are rendered by configuring the global formatter: + +```python +from datafusion.dataframe_formatter import configure_formatter + +configure_formatter( + max_cell_length=25, # Maximum characters in a cell before truncation + max_width=1000, # Maximum width in pixels (HTML only) + max_height=300, # Maximum height in pixels (HTML only) + max_memory_bytes=2097152, # Maximum memory for rendering (2MB) + min_rows=10, # Minimum number of rows to display + max_rows=10, # Maximum rows to display + enable_cell_expansion=True, # Allow expanding truncated cells (HTML only) + custom_css=None, # Additional custom CSS (HTML only) + show_truncation_message=True, # Show message when data is truncated + style_provider=None, # Custom styling provider (HTML only) + use_shared_styles=True, # Share styles across tables (HTML only) +) +``` + +The formatter settings affect all DataFrames displayed after configuration. + +## Custom Style Providers + +For HTML styling, you can create a custom style provider that implements the +`StyleProvider` protocol: + +```python +from datafusion.dataframe_formatter import configure_formatter + +class MyStyleProvider: + def get_cell_style(self): + """Return CSS style string for table data cells.""" + return "border: 1px solid #ddd; padding: 8px; text-align: left;" + + def get_header_style(self): + """Return CSS style string for table header cells.""" + return ( + "background-color: #007bff; color: white; " + "padding: 8px; text-align: left;" + ) + +# Apply the custom style provider +configure_formatter(style_provider=MyStyleProvider()) +``` + +## Custom Cell Formatters + +You can register custom formatters for specific Python types. A cell formatter is any +callable that takes a value and returns a string: + +```python +from datafusion.dataframe_formatter import get_formatter + +formatter = get_formatter() + +# Format floats to 2 decimal places +formatter.register_formatter(float, lambda v: f"{v:.2f}") + +# Format dates in a custom way +from datetime import date +formatter.register_formatter(date, lambda v: v.strftime("%B %d, %Y")) +``` + +## Custom Cell and Header Builders + +For full control over the HTML of individual cells or headers, you can set custom +builder functions: + +```python +from datafusion.dataframe_formatter import get_formatter + +formatter = get_formatter() + +# Custom cell builder receives (value, row, col, table_id) and returns HTML +def my_cell_builder(value, row, col, table_id): + color = "red" if isinstance(value, (int, float)) and value < 0 else "black" + return f"{value}" + +formatter.set_custom_cell_builder(my_cell_builder) + +# Custom header builder receives a schema field and returns HTML +def my_header_builder(field): + return f"{field.name}" + +formatter.set_custom_header_builder(my_header_builder) +``` + +## Performance Optimization with Shared Styles + +The `use_shared_styles` parameter (enabled by default) optimizes performance when +displaying multiple DataFrames in notebook environments: + +```python +from datafusion.dataframe_formatter import configure_formatter + +# Default: Use shared styles (recommended for notebooks) +configure_formatter(use_shared_styles=True) + +# Disable shared styles (each DataFrame includes its own styles) +configure_formatter(use_shared_styles=False) +``` + +When `use_shared_styles=True`: + +- CSS styles and JavaScript are included only once per notebook session +- This reduces HTML output size and prevents style duplication +- Improves rendering performance with many DataFrames +- Applies consistent styling across all DataFrames + +## Working with the Formatter Directly + +You can use `get_formatter()` and `set_formatter()` for direct access to the global +formatter instance: + +```python +from datafusion.dataframe_formatter import ( + DataFrameHtmlFormatter, + get_formatter, + set_formatter, +) + +# Get and modify the current formatter +formatter = get_formatter() +print(formatter.max_rows) +print(formatter.max_cell_length) + +# Create and set a fully custom formatter +custom_formatter = DataFrameHtmlFormatter( + max_cell_length=50, + max_rows=20, + enable_cell_expansion=False, +) +set_formatter(custom_formatter) +``` + +Reset to default formatting: + +```python +from datafusion.dataframe_formatter import reset_formatter + +# Reset to default settings +reset_formatter() +``` + +## Memory and Display Controls + +You can control how much data is displayed and how much memory is used for rendering: + +```python +from datafusion.dataframe_formatter import configure_formatter + +configure_formatter( + max_memory_bytes=4 * 1024 * 1024, # 4MB maximum memory for display + min_rows=20, # Always show at least 20 rows + max_rows=50, # Show up to 50 rows in output +) +``` + +These parameters help balance comprehensive data display against performance considerations. + +## Best Practices + +1. **Global Configuration**: Use `configure_formatter()` at the beginning of your notebook to set up consistent formatting for all DataFrames. +2. **Memory Management**: Set appropriate `max_memory_bytes` limits to prevent performance issues with large datasets. +3. **Shared Styles**: Keep `use_shared_styles=True` (default) for better performance in notebooks with multiple DataFrames. +4. **Reset When Needed**: Call `reset_formatter()` when you want to start fresh with default settings. +5. **Cell Expansion**: Use `enable_cell_expansion=True` when cells might contain longer content that users may want to see in full. + +## Additional Resources + +- {doc}`../dataframe/index` - Complete guide to using DataFrames +- {doc}`../io/index` - I/O Guide for reading data from various sources +- {doc}`../data-sources` - Comprehensive data sources guide +- {ref}`io_csv` - CSV file reading +- {ref}`io_parquet` - Parquet file reading +- {ref}`io_json` - JSON file reading +- {ref}`io_avro` - Avro file reading +- {ref}`io_custom_table_provider` - Custom table providers +- [API Reference](https://arrow.apache.org/datafusion-python/api/index.html) - Full API reference diff --git a/docs/source/user-guide/dataframe/rendering.rst b/docs/source/user-guide/dataframe/rendering.rst deleted file mode 100644 index dc61a422f..000000000 --- a/docs/source/user-guide/dataframe/rendering.rst +++ /dev/null @@ -1,240 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -DataFrame Rendering -=================== - -DataFusion provides configurable rendering for DataFrames in both plain text and HTML -formats. The ``datafusion.dataframe_formatter`` module controls how DataFrames are -displayed in Jupyter notebooks (via ``_repr_html_``), in the terminal (via ``__repr__``), -and anywhere else a string or HTML representation is needed. - -Basic Rendering ---------------- - -In a Jupyter environment, displaying a DataFrame triggers HTML rendering: - -.. code-block:: python - - # Will display as HTML table in Jupyter - df - - # Explicit display also uses HTML rendering - display(df) - -In a terminal or when converting to string, plain text rendering is used: - -.. code-block:: python - - # Plain text table output - print(df) - -Configuring the Formatter -------------------------- - -You can customize how DataFrames are rendered by configuring the global formatter: - -.. code-block:: python - - from datafusion.dataframe_formatter import configure_formatter - - configure_formatter( - max_cell_length=25, # Maximum characters in a cell before truncation - max_width=1000, # Maximum width in pixels (HTML only) - max_height=300, # Maximum height in pixels (HTML only) - max_memory_bytes=2097152, # Maximum memory for rendering (2MB) - min_rows=10, # Minimum number of rows to display - max_rows=10, # Maximum rows to display - enable_cell_expansion=True, # Allow expanding truncated cells (HTML only) - custom_css=None, # Additional custom CSS (HTML only) - show_truncation_message=True, # Show message when data is truncated - style_provider=None, # Custom styling provider (HTML only) - use_shared_styles=True, # Share styles across tables (HTML only) - ) - -The formatter settings affect all DataFrames displayed after configuration. - -Custom Style Providers ----------------------- - -For HTML styling, you can create a custom style provider that implements the -``StyleProvider`` protocol: - -.. code-block:: python - - from datafusion.dataframe_formatter import configure_formatter - - class MyStyleProvider: - def get_cell_style(self): - """Return CSS style string for table data cells.""" - return "border: 1px solid #ddd; padding: 8px; text-align: left;" - - def get_header_style(self): - """Return CSS style string for table header cells.""" - return ( - "background-color: #007bff; color: white; " - "padding: 8px; text-align: left;" - ) - - # Apply the custom style provider - configure_formatter(style_provider=MyStyleProvider()) - -Custom Cell Formatters ----------------------- - -You can register custom formatters for specific Python types. A cell formatter is any -callable that takes a value and returns a string: - -.. code-block:: python - - from datafusion.dataframe_formatter import get_formatter - - formatter = get_formatter() - - # Format floats to 2 decimal places - formatter.register_formatter(float, lambda v: f"{v:.2f}") - - # Format dates in a custom way - from datetime import date - formatter.register_formatter(date, lambda v: v.strftime("%B %d, %Y")) - -Custom Cell and Header Builders -------------------------------- - -For full control over the HTML of individual cells or headers, you can set custom -builder functions: - -.. code-block:: python - - from datafusion.dataframe_formatter import get_formatter - - formatter = get_formatter() - - # Custom cell builder receives (value, row, col, table_id) and returns HTML - def my_cell_builder(value, row, col, table_id): - color = "red" if isinstance(value, (int, float)) and value < 0 else "black" - return f"{value}" - - formatter.set_custom_cell_builder(my_cell_builder) - - # Custom header builder receives a schema field and returns HTML - def my_header_builder(field): - return f"{field.name}" - - formatter.set_custom_header_builder(my_header_builder) - -Performance Optimization with Shared Styles --------------------------------------------- - -The ``use_shared_styles`` parameter (enabled by default) optimizes performance when -displaying multiple DataFrames in notebook environments: - -.. code-block:: python - - from datafusion.dataframe_formatter import configure_formatter - - # Default: Use shared styles (recommended for notebooks) - configure_formatter(use_shared_styles=True) - - # Disable shared styles (each DataFrame includes its own styles) - configure_formatter(use_shared_styles=False) - -When ``use_shared_styles=True``: - -- CSS styles and JavaScript are included only once per notebook session -- This reduces HTML output size and prevents style duplication -- Improves rendering performance with many DataFrames -- Applies consistent styling across all DataFrames - -Working with the Formatter Directly ------------------------------------- - -You can use ``get_formatter()`` and ``set_formatter()`` for direct access to the global -formatter instance: - -.. code-block:: python - - from datafusion.dataframe_formatter import ( - DataFrameHtmlFormatter, - get_formatter, - set_formatter, - ) - - # Get and modify the current formatter - formatter = get_formatter() - print(formatter.max_rows) - print(formatter.max_cell_length) - - # Create and set a fully custom formatter - custom_formatter = DataFrameHtmlFormatter( - max_cell_length=50, - max_rows=20, - enable_cell_expansion=False, - ) - set_formatter(custom_formatter) - -Reset to default formatting: - -.. code-block:: python - - from datafusion.dataframe_formatter import reset_formatter - - # Reset to default settings - reset_formatter() - -Memory and Display Controls ---------------------------- - -You can control how much data is displayed and how much memory is used for rendering: - -.. code-block:: python - - from datafusion.dataframe_formatter import configure_formatter - - configure_formatter( - max_memory_bytes=4 * 1024 * 1024, # 4MB maximum memory for display - min_rows=20, # Always show at least 20 rows - max_rows=50, # Show up to 50 rows in output - ) - -These parameters help balance comprehensive data display against performance considerations. - -Best Practices --------------- - -1. **Global Configuration**: Use ``configure_formatter()`` at the beginning of your notebook to set up consistent formatting for all DataFrames. - -2. **Memory Management**: Set appropriate ``max_memory_bytes`` limits to prevent performance issues with large datasets. - -3. **Shared Styles**: Keep ``use_shared_styles=True`` (default) for better performance in notebooks with multiple DataFrames. - -4. **Reset When Needed**: Call ``reset_formatter()`` when you want to start fresh with default settings. - -5. **Cell Expansion**: Use ``enable_cell_expansion=True`` when cells might contain longer content that users may want to see in full. - -Additional Resources --------------------- - -* :doc:`../dataframe/index` - Complete guide to using DataFrames -* :doc:`../io/index` - I/O Guide for reading data from various sources -* :doc:`../data-sources` - Comprehensive data sources guide -* :ref:`io_csv` - CSV file reading -* :ref:`io_parquet` - Parquet file reading -* :ref:`io_json` - JSON file reading -* :ref:`io_avro` - Avro file reading -* :ref:`io_custom_table_provider` - Custom table providers -* `API Reference `_ - Full API reference diff --git a/docs/source/user-guide/distributing-work.md b/docs/source/user-guide/distributing-work.md new file mode 100644 index 000000000..8634cf24d --- /dev/null +++ b/docs/source/user-guide/distributing-work.md @@ -0,0 +1,364 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Distributing work + +DataFusion supports splitting work across processes by shipping +serialized expressions to workers: the driver builds an +{py:class}`~datafusion.Expr`, each worker evaluates it against its +own slice of data. This pattern suits embarrassingly-parallel +workloads where the driver decides partitioning up front. + +Query-level distribution — where the runtime partitions a single +logical or physical plan across worker nodes — is in progress +upstream via [datafusion-distributed](https://github.com/apache/datafusion-distributed) and [Apache +Ballista](https://github.com/apache/datafusion-ballista). Both +have short sections at the end of this page; integration details +will land as those projects become usable from datafusion-python. + +## Expression-level distribution + +DataFusion expressions support distribution directly: pass one to a +worker process and Python's standard +[pickle](https://docs.python.org/3/library/pickle.html) machinery +serializes it transparently — the same machinery +{py:meth}`multiprocessing.pool.Pool.map`, Ray's `@ray.remote`, and +similar libraries already use to ship function arguments. Python UDFs +— scalar, aggregate, and window — travel inside the serialized +expression; the receiver does not need to pre-register them. + +### Basic worker-pool example + +Define a worker function that takes the expression plus a batch and +returns the evaluated result: + +```python +import pyarrow as pa +from datafusion import SessionContext + + +def evaluate(expr, batch): + # `expr` arrived here via the pool's automatic pickling — + # no manual serialization needed in user code. + ctx = SessionContext() + df = ctx.from_pydict({"a": batch}) + return df.with_column("result", expr).select("result").to_pydict()["result"] +``` + +Then build the expression in the driver and fan it out: + +```python +import multiprocessing as mp +from datafusion import col, udf + +double = udf( + lambda arr: pa.array([(v.as_py() or 0) * 2 for v in arr]), + [pa.int64()], pa.int64(), volatility="immutable", name="double", +) +expr = double(col("a")) + +mp_ctx = mp.get_context("forkserver") +with mp_ctx.Pool(processes=4) as pool: + results = pool.starmap( + evaluate, + [(expr, [1, 2, 3]), (expr, [10, 20, 30])], + ) +print(results) # [[2, 4, 6], [20, 40, 60]] +``` + +:::{note} +When saved to a `.py` file and executed with the `spawn` or +`forkserver` start method, wrap the driver block in +`if __name__ == "__main__":` so worker processes can re-import +the module without re-running it. This is a standard Python +{py:mod}`multiprocessing` requirement, not DataFusion-specific — +see [Safe importing of main module](https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods) +in the Python docs. +::: + +### What travels with the expression + +- **Built-in functions** (`abs`, `length`, arithmetic, comparisons, + etc.) — fully portable. Worker needs nothing pre-registered. + +- **Python UDFs** — travel inline (subject to the two portability + requirements below). The callable, its signature, and any state + captured in closures travel inside the serialized expression and are + reconstructed on the worker automatically. Applies equally to: + + - **scalar UDFs** ({py:func}`datafusion.udf`) + - **aggregate UDFs** ({py:func}`datafusion.udaf`) + - **window UDFs** ({py:func}`datafusion.udwf`) + +- **UDFs imported via the FFI capsule protocol** — travel **by name + only**. The worker must already have a matching registration on its + {py:class}`SessionContext`. Without that registration, evaluation + raises an error. + +### Portability requirements for inline Python UDFs + +Inline Python UDFs ride on [cloudpickle](https://github.com/cloudpipe/cloudpickle), which imposes two +requirements on the worker environment: + +- **Matching Python minor version.** cloudpickle serializes Python + bytecode, which is not stable across minor versions. A UDF pickled + on 3.12 cannot be reconstructed on 3.11 or 3.13. The wire format + stamps the sender's `(major, minor)`; mismatches raise a clear + error naming both versions. Align the Python version on driver and + workers. +- **Imported modules must be importable on the worker.** cloudpickle + captures the callable *by value* (bytecode and closure cells travel + whole), but names resolved through `import` are captured *by + reference* — module path only. A UDF doing + `from mylib import transform` requires `mylib` installed on the + worker. Same applies to bound methods of imported classes. + Self-contained UDFs (no imports beyond what the worker already has, + e.g. `pyarrow`) avoid this entirely. + +### Registering shared UDFs on workers + +When an expression references an FFI capsule UDF (or any UDF the +worker must resolve from its registered functions), set up the +worker's {py:class}`SessionContext` once per process and install it +as the *worker context*: + +```python +from datafusion import SessionContext +from datafusion.ipc import set_worker_ctx + + +def init_worker(): + ctx = SessionContext() + ctx.register_udaf(my_ffi_aggregate) + set_worker_ctx(ctx) + + +with mp.get_context("forkserver").Pool( + processes=4, initializer=init_worker +) as pool: + ... +``` + +Inside a worker, expressions arriving from the driver resolve their +by-name references against the installed worker context. If no worker +context is installed, the global {py:class}`SessionContext` is used — +fine for expressions that only reference built-ins and Python UDFs, +but FFI-capsule-backed registrations must be installed on the global +context to resolve. + +### Python 3.14 default change + +Python 3.14 changed the Linux default start method for +{py:mod}`multiprocessing` from `fork` to `forkserver` (macOS has +defaulted to `spawn` since Python 3.8; Windows has always used +`spawn`). With `fork`, any state set in the parent was visible in +workers via copy-on-write; with `forkserver` and `spawn` it is +not. The {py:func}`~datafusion.ipc.set_worker_ctx` pattern works on +every start method — prefer it over relying on inherited state. + +### Practical considerations + +- **Serialized size scales with what travels inline.** A serialized + expression of just built-ins is small (tens of bytes). An + expression carrying a Python UDF is hundreds of bytes (the callable + and its signature). When the same UDF is shipped many times, + registering an equivalent FFI-capsule UDF on each worker via + {py:func}`~datafusion.ipc.set_worker_ctx` and referring to it by + name cuts the per-trip overhead. +- **Closure capture.** When a Python UDF closes over surrounding + state — local variables, module-level objects, file paths — that + state is captured at serialization time. Surprises are possible if + the captured state is large, mutable, or not portable to the + worker's environment. See [Portability requirements for inline + Python UDFs][portability requirements for inline python udfs] for the Python-version and imported-module rules. + +### Disabling Python UDF inlining + +For a stricter wire format, call +{py:meth}`SessionContext.with_python_udf_inlining(enabled=False) +` on the session +producing or consuming the bytes. With inlining disabled, Python +UDFs travel by name only — the same way FFI-capsule UDFs do — and +the receiver must have a matching registration. + +Two use cases: + +- **Cross-language portability.** A non-Python decoder cannot + reconstruct a cloudpickled payload. Senders aimed at Java, C++, + or another Rust binary disable inlining and rely on the receiver + having compatible UDF registrations. +- **Untrusted-source decode.** With inlining disabled, + {py:meth}`Expr.from_bytes` never calls `cloudpickle.loads` on + the incoming bytes — an inline payload from a misbehaving sender + raises a clear error instead of executing arbitrary Python code. + +Mismatched configurations raise a descriptive error: an inline blob +fed to a strict receiver fails fast rather than silently dropping +into `cloudpickle.loads`. + +To make the toggle apply through {py:func}`pickle.dumps` (which +calls {py:meth}`Expr.to_bytes` with no context), install the strict +session as the driver's *sender context*: + +```python +from datafusion import SessionContext +from datafusion.ipc import set_sender_ctx + +set_sender_ctx(SessionContext().with_python_udf_inlining(enabled=False)) +# Every subsequent pickle.dumps(expr) on this thread encodes +# without inlining the Python callable. +``` + +Pair with a matching strict worker context +({py:func}`~datafusion.ipc.set_worker_ctx`) so the `pickle.loads` +side also refuses inline payloads. Explicit +{py:meth}`Expr.to_bytes(ctx) ` and +{py:meth}`Expr.from_bytes(blob, ctx=ctx) ` calls +honor the supplied `ctx` directly and ignore the sender / worker +contexts. + +The toggle only narrows the {py:meth}`Expr.from_bytes` surface; +{py:func}`pickle.loads` on untrusted bytes remains unsafe regardless +of this setting. See the [Security] section below for the full +threat model. + +### Security + +:::{warning} +Reconstructing an expression containing a Python UDF executes +arbitrary Python code on the receiver — pickle is doing the work +under the hood and pickle is unsafe on untrusted input (see the +[pickle module security warning](https://docs.python.org/3/library/pickle.html#module-pickle) +in the Python standard library docs). Only accept expressions +from trusted sources. For untrusted-source workflows, disable +Python UDF inlining (see above), restrict senders to built-in +functions and pre-registered Rust-side UDFs, and avoid +{py:func}`pickle.loads` on externally supplied bytes entirely. +::: + +### Reference: session context slots + +There is only one type — {py:class}`SessionContext`. It can occupy +up to four *slots* in a running program: + +```{eval-rst} +.. list-table:: + :header-rows: 1 + :widths: 12 18 40 30 + + * - Slot + - Lifetime + - Purpose + - Set how + * - User-held + - Local variable / attribute + - Build and run queries + - ``ctx = SessionContext(...)`` + * - Global + - Process singleton (lazy-init) + - Backs module-level + :py:func:`~datafusion.io.read_parquet`, + :py:func:`~datafusion.io.read_csv`, + :py:func:`~datafusion.io.read_json`, + :py:func:`~datafusion.io.read_avro`; final fallback for + :py:meth:`Expr.from_bytes` + - Implicit; access via + :py:meth:`SessionContext.global_ctx` + * - Sender + - Thread-local on the driver + - Codec settings for outbound :py:func:`pickle.dumps` / + :py:meth:`Expr.to_bytes` without ``ctx`` + - :py:func:`~datafusion.ipc.set_sender_ctx` + * - Worker + - Thread-local on the worker + - Function registry for inbound :py:func:`pickle.loads` / + :py:meth:`Expr.from_bytes` without ``ctx`` + - :py:func:`~datafusion.ipc.set_worker_ctx` +``` + +The same {py:class}`SessionContext` object may occupy more than one +slot simultaneously — installing it into a slot is a reference, not +a copy. A non-distributed program only ever uses the user-held slot; +the global slot is invisible unless you call top-level `read_*` +helpers. + +Resolution order on the worker side is *explicit argument → +worker context → global context.* Explicit `ctx=` on +{py:meth}`Expr.from_bytes` always wins; the sender slot is ignored +on decode and the worker slot is ignored on encode. + +Sharp edges: + +- Sender and worker slots are **thread-local**. Background threads + on either side see `None` until they install their own. +- Under the `fork` start method, the parent's `threading.local()` + values are copied into the child by copy-on-write — a forked + worker initially observes whatever sender / worker slot the parent + had set, until the worker writes its own value (or calls the + matching `clear_*_ctx`). `spawn` and `forkserver` workers + start with empty thread-local slots. Treat the slot as + uninitialized on worker entry and install (or clear) it explicitly + in the worker initializer; do not rely on inherited state. +- The global slot persists across `fork` workers (copy-on-write + memory inherit) but not across `spawn` / `forkserver` workers + (fresh process — register or install a worker context on + start-up). +- The inlining toggle is per-context state, not a global switch. + Two contexts with different toggles can coexist in one process. + +## Query-level distribution via datafusion-distributed + +🚧 *Work in progress upstream — not yet usable from datafusion-python.* + +[datafusion-distributed](https://github.com/apache/datafusion-distributed) +splits a single physical plan into stages and runs each stage on a +different worker node. The driver writes a SQL or DataFrame query +once; the runtime handles partitioning, shuffles, and reassembly. + +A datafusion-python integration is in development. This section will +document the integration once it lands. In the meantime, the +expression-level approach above covers most use cases that do not +require automatic plan partitioning. + +## Query-level distribution via Apache Ballista + +🚧 *Work in progress upstream — not yet usable from datafusion-python.* + +[Apache Ballista](https://github.com/apache/datafusion-ballista) +provides distributed query execution on top of DataFusion with a +scheduler / executor model better suited to long-lived cluster +deployments. A datafusion-python integration is on the roadmap; this +section will fill in once the integration is usable. + +## See also + +- {py:mod}`datafusion.ipc` — worker context API. +- `examples/multiprocessing_pickle_expr.py` — runnable + `multiprocessing.Pool` example that ships a different parametric + expression to each worker and collects results back. +- `examples/ray_pickle_expr.py` — runnable Ray actor example. diff --git a/docs/source/user-guide/distributing-work.rst b/docs/source/user-guide/distributing-work.rst deleted file mode 100644 index 03b5ca0b9..000000000 --- a/docs/source/user-guide/distributing-work.rst +++ /dev/null @@ -1,368 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Distributing work -================= - -DataFusion supports splitting work across processes by shipping -serialized expressions to workers: the driver builds an -:py:class:`~datafusion.Expr`, each worker evaluates it against its -own slice of data. This pattern suits embarrassingly-parallel -workloads where the driver decides partitioning up front. - -Query-level distribution — where the runtime partitions a single -logical or physical plan across worker nodes — is in progress -upstream via `datafusion-distributed -`_ and `Apache -Ballista `_. Both -have short sections at the end of this page; integration details -will land as those projects become usable from datafusion-python. - -Expression-level distribution ------------------------------ - -DataFusion expressions support distribution directly: pass one to a -worker process and Python's standard -`pickle `_ machinery -serializes it transparently — the same machinery -:py:meth:`multiprocessing.pool.Pool.map`, Ray's ``@ray.remote``, and -similar libraries already use to ship function arguments. Python UDFs -— scalar, aggregate, and window — travel inside the serialized -expression; the receiver does not need to pre-register them. - -Basic worker-pool example -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Define a worker function that takes the expression plus a batch and -returns the evaluated result: - -.. code-block:: python - - import pyarrow as pa - from datafusion import SessionContext - - - def evaluate(expr, batch): - # `expr` arrived here via the pool's automatic pickling — - # no manual serialization needed in user code. - ctx = SessionContext() - df = ctx.from_pydict({"a": batch}) - return df.with_column("result", expr).select("result").to_pydict()["result"] - -Then build the expression in the driver and fan it out: - -.. code-block:: python - - import multiprocessing as mp - from datafusion import col, udf - - double = udf( - lambda arr: pa.array([(v.as_py() or 0) * 2 for v in arr]), - [pa.int64()], pa.int64(), volatility="immutable", name="double", - ) - expr = double(col("a")) - - mp_ctx = mp.get_context("forkserver") - with mp_ctx.Pool(processes=4) as pool: - results = pool.starmap( - evaluate, - [(expr, [1, 2, 3]), (expr, [10, 20, 30])], - ) - print(results) # [[2, 4, 6], [20, 40, 60]] - -.. note:: - - When saved to a ``.py`` file and executed with the ``spawn`` or - ``forkserver`` start method, wrap the driver block in - ``if __name__ == "__main__":`` so worker processes can re-import - the module without re-running it. This is a standard Python - :py:mod:`multiprocessing` requirement, not DataFusion-specific — - see `Safe importing of main module - `_ - in the Python docs. - - -What travels with the expression -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* **Built-in functions** (``abs``, ``length``, arithmetic, comparisons, - etc.) — fully portable. Worker needs nothing pre-registered. -* **Python UDFs** — travel inline (subject to the two portability - requirements below). The callable, its signature, and any state - captured in closures travel inside the serialized expression and are - reconstructed on the worker automatically. Applies equally to: - - * **scalar UDFs** (:py:func:`datafusion.udf`) - * **aggregate UDFs** (:py:func:`datafusion.udaf`) - * **window UDFs** (:py:func:`datafusion.udwf`) -* **UDFs imported via the FFI capsule protocol** — travel **by name - only**. The worker must already have a matching registration on its - :py:class:`SessionContext`. Without that registration, evaluation - raises an error. - -Portability requirements for inline Python UDFs -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Inline Python UDFs ride on `cloudpickle -`_, which imposes two -requirements on the worker environment: - -* **Matching Python minor version.** cloudpickle serializes Python - bytecode, which is not stable across minor versions. A UDF pickled - on 3.12 cannot be reconstructed on 3.11 or 3.13. The wire format - stamps the sender's ``(major, minor)``; mismatches raise a clear - error naming both versions. Align the Python version on driver and - workers. -* **Imported modules must be importable on the worker.** cloudpickle - captures the callable *by value* (bytecode and closure cells travel - whole), but names resolved through ``import`` are captured *by - reference* — module path only. A UDF doing - ``from mylib import transform`` requires ``mylib`` installed on the - worker. Same applies to bound methods of imported classes. - Self-contained UDFs (no imports beyond what the worker already has, - e.g. ``pyarrow``) avoid this entirely. - -Registering shared UDFs on workers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -When an expression references an FFI capsule UDF (or any UDF the -worker must resolve from its registered functions), set up the -worker's :py:class:`SessionContext` once per process and install it -as the *worker context*: - -.. code-block:: python - - from datafusion import SessionContext - from datafusion.ipc import set_worker_ctx - - - def init_worker(): - ctx = SessionContext() - ctx.register_udaf(my_ffi_aggregate) - set_worker_ctx(ctx) - - - with mp.get_context("forkserver").Pool( - processes=4, initializer=init_worker - ) as pool: - ... - -Inside a worker, expressions arriving from the driver resolve their -by-name references against the installed worker context. If no worker -context is installed, the global :py:class:`SessionContext` is used — -fine for expressions that only reference built-ins and Python UDFs, -but FFI-capsule-backed registrations must be installed on the global -context to resolve. - -Python 3.14 default change -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Python 3.14 changed the Linux default start method for -:py:mod:`multiprocessing` from ``fork`` to ``forkserver`` (macOS has -defaulted to ``spawn`` since Python 3.8; Windows has always used -``spawn``). With ``fork``, any state set in the parent was visible in -workers via copy-on-write; with ``forkserver`` and ``spawn`` it is -not. The :py:func:`~datafusion.ipc.set_worker_ctx` pattern works on -every start method — prefer it over relying on inherited state. - -Practical considerations -~~~~~~~~~~~~~~~~~~~~~~~~ - -* **Serialized size scales with what travels inline.** A serialized - expression of just built-ins is small (tens of bytes). An - expression carrying a Python UDF is hundreds of bytes (the callable - and its signature). When the same UDF is shipped many times, - registering an equivalent FFI-capsule UDF on each worker via - :py:func:`~datafusion.ipc.set_worker_ctx` and referring to it by - name cuts the per-trip overhead. -* **Closure capture.** When a Python UDF closes over surrounding - state — local variables, module-level objects, file paths — that - state is captured at serialization time. Surprises are possible if - the captured state is large, mutable, or not portable to the - worker's environment. See `Portability requirements for inline - Python UDFs`_ for the Python-version and imported-module rules. - -Disabling Python UDF inlining -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For a stricter wire format, call -:py:meth:`SessionContext.with_python_udf_inlining(enabled=False) -` on the session -producing or consuming the bytes. With inlining disabled, Python -UDFs travel by name only — the same way FFI-capsule UDFs do — and -the receiver must have a matching registration. - -Two use cases: - -* **Cross-language portability.** A non-Python decoder cannot - reconstruct a cloudpickled payload. Senders aimed at Java, C++, - or another Rust binary disable inlining and rely on the receiver - having compatible UDF registrations. -* **Untrusted-source decode.** With inlining disabled, - :py:meth:`Expr.from_bytes` never calls ``cloudpickle.loads`` on - the incoming bytes — an inline payload from a misbehaving sender - raises a clear error instead of executing arbitrary Python code. - -Mismatched configurations raise a descriptive error: an inline blob -fed to a strict receiver fails fast rather than silently dropping -into ``cloudpickle.loads``. - -To make the toggle apply through :py:func:`pickle.dumps` (which -calls :py:meth:`Expr.to_bytes` with no context), install the strict -session as the driver's *sender context*: - -.. code-block:: python - - from datafusion import SessionContext - from datafusion.ipc import set_sender_ctx - - set_sender_ctx(SessionContext().with_python_udf_inlining(enabled=False)) - # Every subsequent pickle.dumps(expr) on this thread encodes - # without inlining the Python callable. - -Pair with a matching strict worker context -(:py:func:`~datafusion.ipc.set_worker_ctx`) so the ``pickle.loads`` -side also refuses inline payloads. Explicit -:py:meth:`Expr.to_bytes(ctx) ` and -:py:meth:`Expr.from_bytes(blob, ctx=ctx) ` calls -honor the supplied ``ctx`` directly and ignore the sender / worker -contexts. - -The toggle only narrows the :py:meth:`Expr.from_bytes` surface; -:py:func:`pickle.loads` on untrusted bytes remains unsafe regardless -of this setting. See the `Security`_ section below for the full -threat model. - -Security -~~~~~~~~ - -.. warning:: - - Reconstructing an expression containing a Python UDF executes - arbitrary Python code on the receiver — pickle is doing the work - under the hood and pickle is unsafe on untrusted input (see the - `pickle module security warning - `_ - in the Python standard library docs). Only accept expressions - from trusted sources. For untrusted-source workflows, disable - Python UDF inlining (see above), restrict senders to built-in - functions and pre-registered Rust-side UDFs, and avoid - :py:func:`pickle.loads` on externally supplied bytes entirely. - -Reference: session context slots -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -There is only one type — :py:class:`SessionContext`. It can occupy -up to four *slots* in a running program: - -.. list-table:: - :header-rows: 1 - :widths: 12 18 40 30 - - * - Slot - - Lifetime - - Purpose - - Set how - * - User-held - - Local variable / attribute - - Build and run queries - - ``ctx = SessionContext(...)`` - * - Global - - Process singleton (lazy-init) - - Backs module-level - :py:func:`~datafusion.io.read_parquet`, - :py:func:`~datafusion.io.read_csv`, - :py:func:`~datafusion.io.read_json`, - :py:func:`~datafusion.io.read_avro`; final fallback for - :py:meth:`Expr.from_bytes` - - Implicit; access via - :py:meth:`SessionContext.global_ctx` - * - Sender - - Thread-local on the driver - - Codec settings for outbound :py:func:`pickle.dumps` / - :py:meth:`Expr.to_bytes` without ``ctx`` - - :py:func:`~datafusion.ipc.set_sender_ctx` - * - Worker - - Thread-local on the worker - - Function registry for inbound :py:func:`pickle.loads` / - :py:meth:`Expr.from_bytes` without ``ctx`` - - :py:func:`~datafusion.ipc.set_worker_ctx` - -The same :py:class:`SessionContext` object may occupy more than one -slot simultaneously — installing it into a slot is a reference, not -a copy. A non-distributed program only ever uses the user-held slot; -the global slot is invisible unless you call top-level ``read_*`` -helpers. - -Resolution order on the worker side is *explicit argument → -worker context → global context.* Explicit ``ctx=`` on -:py:meth:`Expr.from_bytes` always wins; the sender slot is ignored -on decode and the worker slot is ignored on encode. - -Sharp edges: - -* Sender and worker slots are **thread-local**. Background threads - on either side see ``None`` until they install their own. -* Under the ``fork`` start method, the parent's ``threading.local()`` - values are copied into the child by copy-on-write — a forked - worker initially observes whatever sender / worker slot the parent - had set, until the worker writes its own value (or calls the - matching ``clear_*_ctx``). ``spawn`` and ``forkserver`` workers - start with empty thread-local slots. Treat the slot as - uninitialized on worker entry and install (or clear) it explicitly - in the worker initializer; do not rely on inherited state. -* The global slot persists across ``fork`` workers (copy-on-write - memory inherit) but not across ``spawn`` / ``forkserver`` workers - (fresh process — register or install a worker context on - start-up). -* The inlining toggle is per-context state, not a global switch. - Two contexts with different toggles can coexist in one process. - -Query-level distribution via datafusion-distributed ---------------------------------------------------- - -🚧 *Work in progress upstream — not yet usable from datafusion-python.* - -`datafusion-distributed `_ -splits a single physical plan into stages and runs each stage on a -different worker node. The driver writes a SQL or DataFrame query -once; the runtime handles partitioning, shuffles, and reassembly. - -A datafusion-python integration is in development. This section will -document the integration once it lands. In the meantime, the -expression-level approach above covers most use cases that do not -require automatic plan partitioning. - -Query-level distribution via Apache Ballista --------------------------------------------- - -🚧 *Work in progress upstream — not yet usable from datafusion-python.* - -`Apache Ballista `_ -provides distributed query execution on top of DataFusion with a -scheduler / executor model better suited to long-lived cluster -deployments. A datafusion-python integration is on the roadmap; this -section will fill in once the integration is usable. - -See also --------- - -* :py:mod:`datafusion.ipc` — worker context API. -* ``examples/multiprocessing_pickle_expr.py`` — runnable - ``multiprocessing.Pool`` example that ships a different parametric - expression to each worker and collects results back. -* ``examples/ray_pickle_expr.py`` — runnable Ray actor example. diff --git a/docs/source/user-guide/index.md b/docs/source/user-guide/index.md new file mode 100644 index 000000000..509c850c8 --- /dev/null +++ b/docs/source/user-guide/index.md @@ -0,0 +1,48 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# User Guide + +The user guide walks through installing DataFusion in Python, building queries +with the DataFrame API or SQL, reading and writing data, and tuning execution. + +```{toctree} +:maxdepth: 2 + +introduction +basics +data-sources +dataframe/index +common-operations/index +io/index +configuration +distributing-work +sql +upgrade-guides +ai-coding-assistants +``` diff --git a/docs/source/user-guide/index.rst b/docs/source/user-guide/index.rst deleted file mode 100644 index 2d6b94392..000000000 --- a/docs/source/user-guide/index.rst +++ /dev/null @@ -1,38 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -========== -User Guide -========== - -The user guide walks through installing DataFusion in Python, building queries -with the DataFrame API or SQL, reading and writing data, and tuning execution. - -.. toctree:: - :maxdepth: 2 - - introduction - basics - data-sources - dataframe/index - common-operations/index - io/index - configuration - distributing-work - sql - upgrade-guides - ai-coding-assistants diff --git a/docs/source/user-guide/introduction.md b/docs/source/user-guide/introduction.md new file mode 100644 index 000000000..1abe55a34 --- /dev/null +++ b/docs/source/user-guide/introduction.md @@ -0,0 +1,91 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(guide)= + +# Introduction + +Welcome to the User Guide for the Python bindings of Arrow DataFusion. This guide aims to provide an introduction to +DataFusion through various examples and highlight the most effective ways of using it. + +## Installation + +DataFusion is a Python library and, as such, can be installed via pip from [PyPI](https://pypi.org/project/datafusion). + +```shell +pip install datafusion +``` + +You can verify the installation by running: + +```{eval-rst} +.. ipython:: python + + import datafusion + datafusion.__version__ +``` + +In this documentation we will also show some examples for how DataFusion integrates +with Jupyter notebooks. To install and start a Jupyter labs session use + +```shell +pip install jupyterlab +jupyter lab +``` + +To demonstrate working with DataFusion, we need a data source. Later in the tutorial we will show +options for data sources. For our first example, we demonstrate using a Pokemon dataset that you +can download +[here](https://gist.githubusercontent.com/ritchie46/cac6b337ea52281aa23c049250a4ff03/raw/89a957ff3919d90e6ef2d34235e6bf22304f3366/pokemon.csv). + +With that file in place you can use the following python example to view the DataFrame in +DataFusion. + +```{eval-rst} +.. ipython:: python + + from datafusion import SessionContext + + ctx = SessionContext() + + df = ctx.read_csv("pokemon.csv") + + df.show() +``` + +If you are working in a Jupyter notebook, you can also use the following to give you a table +display that may be easier to read. + +```shell +display(df) +``` + +```{image} ../images/jupyter_lab_df_view.png +:alt: Rendered table showing Pokemon DataFrame +:width: 800 +``` diff --git a/docs/source/user-guide/introduction.rst b/docs/source/user-guide/introduction.rst deleted file mode 100644 index 7b30ef2b2..000000000 --- a/docs/source/user-guide/introduction.rst +++ /dev/null @@ -1,77 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _guide: - -Introduction -============ - -Welcome to the User Guide for the Python bindings of Arrow DataFusion. This guide aims to provide an introduction to -DataFusion through various examples and highlight the most effective ways of using it. - -Installation ------------- - -DataFusion is a Python library and, as such, can be installed via pip from `PyPI `__. - -.. code-block:: shell - - pip install datafusion - -You can verify the installation by running: - -.. ipython:: python - - import datafusion - datafusion.__version__ - -In this documentation we will also show some examples for how DataFusion integrates -with Jupyter notebooks. To install and start a Jupyter labs session use - -.. code-block:: shell - - pip install jupyterlab - jupyter lab - -To demonstrate working with DataFusion, we need a data source. Later in the tutorial we will show -options for data sources. For our first example, we demonstrate using a Pokemon dataset that you -can download -`here `_. - -With that file in place you can use the following python example to view the DataFrame in -DataFusion. - -.. ipython:: python - - from datafusion import SessionContext - - ctx = SessionContext() - - df = ctx.read_csv("pokemon.csv") - - df.show() - -If you are working in a Jupyter notebook, you can also use the following to give you a table -display that may be easier to read. - -.. code-block:: shell - - display(df) - -.. image:: ../images/jupyter_lab_df_view.png - :width: 800 - :alt: Rendered table showing Pokemon DataFrame diff --git a/docs/source/user-guide/io/arrow.md b/docs/source/user-guide/io/arrow.md new file mode 100644 index 000000000..3d35e83f8 --- /dev/null +++ b/docs/source/user-guide/io/arrow.md @@ -0,0 +1,85 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Arrow + +DataFusion implements the +[Apache Arrow PyCapsule interface](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) +for importing and exporting DataFrames with zero copy. With this feature, any Python +project that implements this interface can share data back and forth with DataFusion +with zero copy. + +We can demonstrate using [pyarrow](https://arrow.apache.org/docs/python/index.html). + +## Importing to DataFusion + +Here we will create an Arrow table and import it to DataFusion. + +To import an Arrow table, use {py:func}`datafusion.context.SessionContext.from_arrow`. +This will accept any Python object that implements +[\_\_arrow_c_stream\_\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowstream-export) +or [\_\_arrow_c_array\_\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowarray-export) +and returns a `StructArray`. Common pyarrow sources you can use are: + +- [Array](https://arrow.apache.org/docs/python/generated/pyarrow.Array.html) (but it must return a Struct Array) +- [Record Batch](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html) +- [Record Batch Reader](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatchReader.html) +- [Table](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html) + +```{eval-rst} +.. ipython:: python + + from datafusion import SessionContext + import pyarrow as pa + + data = {"a": [1, 2, 3], "b": [4, 5, 6]} + table = pa.Table.from_pydict(data) + + ctx = SessionContext() + df = ctx.from_arrow(table) + df +``` + +## Exporting from DataFusion + +DataFusion DataFrames implement `__arrow_c_stream__` PyCapsule interface, so any +Python library that accepts these can import a DataFusion DataFrame directly. + +Invoking `__arrow_c_stream__` triggers execution of the underlying query, but +batches are yielded incrementally rather than materialized all at once in memory. +Consumers can process the stream as it arrives. The stream executes lazily, +letting downstream readers pull batches on demand. + +```{eval-rst} +.. ipython:: python + + from datafusion import col, lit + + df = df.select((col("a") * lit(1.5)).alias("c"), lit("df").alias("d")) + pa.table(df) +``` diff --git a/docs/source/user-guide/io/arrow.rst b/docs/source/user-guide/io/arrow.rst deleted file mode 100644 index 9196fcea7..000000000 --- a/docs/source/user-guide/io/arrow.rst +++ /dev/null @@ -1,75 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Arrow -===== - -DataFusion implements the -`Apache Arrow PyCapsule interface `_ -for importing and exporting DataFrames with zero copy. With this feature, any Python -project that implements this interface can share data back and forth with DataFusion -with zero copy. - -We can demonstrate using `pyarrow `_. - -Importing to DataFusion ------------------------ - -Here we will create an Arrow table and import it to DataFusion. - -To import an Arrow table, use :py:func:`datafusion.context.SessionContext.from_arrow`. -This will accept any Python object that implements -`__arrow_c_stream__ `_ -or `__arrow_c_array__ `_ -and returns a ``StructArray``. Common pyarrow sources you can use are: - -- `Array `_ (but it must return a Struct Array) -- `Record Batch `_ -- `Record Batch Reader `_ -- `Table `_ - -.. ipython:: python - - from datafusion import SessionContext - import pyarrow as pa - - data = {"a": [1, 2, 3], "b": [4, 5, 6]} - table = pa.Table.from_pydict(data) - - ctx = SessionContext() - df = ctx.from_arrow(table) - df - -Exporting from DataFusion -------------------------- - -DataFusion DataFrames implement ``__arrow_c_stream__`` PyCapsule interface, so any -Python library that accepts these can import a DataFusion DataFrame directly. - -Invoking ``__arrow_c_stream__`` triggers execution of the underlying query, but -batches are yielded incrementally rather than materialized all at once in memory. -Consumers can process the stream as it arrives. The stream executes lazily, -letting downstream readers pull batches on demand. - - -.. ipython:: python - - from datafusion import col, lit - - df = df.select((col("a") * lit(1.5)).alias("c"), lit("df").alias("d")) - pa.table(df) - diff --git a/docs/source/user-guide/io/avro.md b/docs/source/user-guide/io/avro.md new file mode 100644 index 000000000..92a63e6b0 --- /dev/null +++ b/docs/source/user-guide/io/avro.md @@ -0,0 +1,41 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(io_avro)= + +# Avro + +[Avro](https://avro.apache.org/) is a serialization format for record data. Reading an avro file is very straightforward +with {py:func}`~datafusion.context.SessionContext.read_avro` + +```python +from datafusion import SessionContext + +ctx = SessionContext() +df = ctx.read_avro("file.avro") +``` diff --git a/docs/source/user-guide/io/avro.rst b/docs/source/user-guide/io/avro.rst deleted file mode 100644 index 66398ac7f..000000000 --- a/docs/source/user-guide/io/avro.rst +++ /dev/null @@ -1,32 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _io_avro: - -Avro -==== - -`Avro `_ is a serialization format for record data. Reading an avro file is very straightforward -with :py:func:`~datafusion.context.SessionContext.read_avro` - -.. code-block:: python - - - from datafusion import SessionContext - - ctx = SessionContext() - df = ctx.read_avro("file.avro") \ No newline at end of file diff --git a/docs/source/user-guide/io/csv.md b/docs/source/user-guide/io/csv.md new file mode 100644 index 000000000..4c541c57d --- /dev/null +++ b/docs/source/user-guide/io/csv.md @@ -0,0 +1,69 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(io_csv)= + +# CSV + +Reading a csv is very straightforward with {py:func}`~datafusion.context.SessionContext.read_csv` + +```python +from datafusion import SessionContext + +ctx = SessionContext() +df = ctx.read_csv("file.csv") +``` + +An alternative is to use {py:func}`~datafusion.context.SessionContext.register_csv` + +```python +ctx.register_csv("file", "file.csv") +df = ctx.table("file") +``` + +If you require additional control over how to read the CSV file, you can use +{py:class}`~datafusion.options.CsvReadOptions` to set a variety of options. + +```python +from datafusion import CsvReadOptions +options = ( + CsvReadOptions() + .with_has_header(True) # File contains a header row + .with_delimiter(";") # Use ; as the delimiter instead of , + .with_comment("#") # Skip lines starting with # + .with_escape("\\") # Escape character + .with_null_regex(r"^(null|NULL|N/A)$") # Treat these as NULL + .with_truncated_rows(True) # Allow rows to have incomplete columns + .with_file_compression_type("gzip") # Read gzipped CSV + .with_file_extension(".gz") # File extension other than .csv +) +df = ctx.read_csv("data.csv.gz", options=options) +``` + +Details for all CSV reading options can be found on the +[DataFusion documentation site](https://datafusion.apache.org/library-user-guide/custom-table-providers.html). diff --git a/docs/source/user-guide/io/csv.rst b/docs/source/user-guide/io/csv.rst deleted file mode 100644 index 9c23c291b..000000000 --- a/docs/source/user-guide/io/csv.rst +++ /dev/null @@ -1,60 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _io_csv: - -CSV -=== - -Reading a csv is very straightforward with :py:func:`~datafusion.context.SessionContext.read_csv` - -.. code-block:: python - - - from datafusion import SessionContext - - ctx = SessionContext() - df = ctx.read_csv("file.csv") - -An alternative is to use :py:func:`~datafusion.context.SessionContext.register_csv` - -.. code-block:: python - - ctx.register_csv("file", "file.csv") - df = ctx.table("file") - -If you require additional control over how to read the CSV file, you can use -:py:class:`~datafusion.options.CsvReadOptions` to set a variety of options. - -.. code-block:: python - - from datafusion import CsvReadOptions - options = ( - CsvReadOptions() - .with_has_header(True) # File contains a header row - .with_delimiter(";") # Use ; as the delimiter instead of , - .with_comment("#") # Skip lines starting with # - .with_escape("\\") # Escape character - .with_null_regex(r"^(null|NULL|N/A)$") # Treat these as NULL - .with_truncated_rows(True) # Allow rows to have incomplete columns - .with_file_compression_type("gzip") # Read gzipped CSV - .with_file_extension(".gz") # File extension other than .csv - ) - df = ctx.read_csv("data.csv.gz", options=options) - -Details for all CSV reading options can be found on the -`DataFusion documentation site `_. diff --git a/docs/source/user-guide/io/index.md b/docs/source/user-guide/io/index.md new file mode 100644 index 000000000..a2da64989 --- /dev/null +++ b/docs/source/user-guide/io/index.md @@ -0,0 +1,40 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# IO + +```{toctree} +:maxdepth: 2 + +arrow +avro +csv +json +parquet +table_provider +``` diff --git a/docs/source/user-guide/io/index.rst b/docs/source/user-guide/io/index.rst deleted file mode 100644 index b885cfeda..000000000 --- a/docs/source/user-guide/io/index.rst +++ /dev/null @@ -1,29 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -IO -== - -.. toctree:: - :maxdepth: 2 - - arrow - avro - csv - json - parquet - table_provider diff --git a/docs/source/user-guide/io/json.md b/docs/source/user-guide/io/json.md new file mode 100644 index 000000000..bcf60dfe3 --- /dev/null +++ b/docs/source/user-guide/io/json.md @@ -0,0 +1,41 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(io_json)= + +# JSON + +[JSON](https://www.json.org/json-en.html) (JavaScript Object Notation) is a lightweight data-interchange format. +When it comes to reading a JSON file, using {py:func}`~datafusion.context.SessionContext.read_json` is a simple and easy + +```python +from datafusion import SessionContext + +ctx = SessionContext() +df = ctx.read_json("file.json") +``` diff --git a/docs/source/user-guide/io/json.rst b/docs/source/user-guide/io/json.rst deleted file mode 100644 index 39030db7f..000000000 --- a/docs/source/user-guide/io/json.rst +++ /dev/null @@ -1,31 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _io_json: - -JSON -==== -`JSON `_ (JavaScript Object Notation) is a lightweight data-interchange format. -When it comes to reading a JSON file, using :py:func:`~datafusion.context.SessionContext.read_json` is a simple and easy - -.. code-block:: python - - - from datafusion import SessionContext - - ctx = SessionContext() - df = ctx.read_json("file.json") diff --git a/docs/source/user-guide/io/parquet.md b/docs/source/user-guide/io/parquet.md new file mode 100644 index 000000000..ca2187409 --- /dev/null +++ b/docs/source/user-guide/io/parquet.md @@ -0,0 +1,47 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(io_parquet)= + +# Parquet + +It is quite simple to read a parquet file using the {py:func}`~datafusion.context.SessionContext.read_parquet` function. + +```python +from datafusion import SessionContext + +ctx = SessionContext() +df = ctx.read_parquet("file.parquet") +``` + +An alternative is to use {py:func}`~datafusion.context.SessionContext.register_parquet` + +```python +ctx.register_parquet("file", "file.parquet") +df = ctx.table("file") +``` diff --git a/docs/source/user-guide/io/parquet.rst b/docs/source/user-guide/io/parquet.rst deleted file mode 100644 index c5b9ca3d4..000000000 --- a/docs/source/user-guide/io/parquet.rst +++ /dev/null @@ -1,37 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _io_parquet: - -Parquet -======= - -It is quite simple to read a parquet file using the :py:func:`~datafusion.context.SessionContext.read_parquet` function. - -.. code-block:: python - - from datafusion import SessionContext - - ctx = SessionContext() - df = ctx.read_parquet("file.parquet") - -An alternative is to use :py:func:`~datafusion.context.SessionContext.register_parquet` - -.. code-block:: python - - ctx.register_parquet("file", "file.parquet") - df = ctx.table("file") diff --git a/docs/source/user-guide/io/table_provider.md b/docs/source/user-guide/io/table_provider.md new file mode 100644 index 000000000..0116ccf3f --- /dev/null +++ b/docs/source/user-guide/io/table_provider.md @@ -0,0 +1,72 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +(io_custom_table_provider)= + +# Custom Table Provider + +If you have a custom data source that you want to integrate with DataFusion, you can do so by +implementing the [TableProvider](https://datafusion.apache.org/library-user-guide/custom-table-providers.html) +interface in Rust and then exposing it in Python. To do so, +you must use DataFusion 43.0.0 or later and expose a [FFI_TableProvider](https://crates.io/crates/datafusion-ffi) +via [PyCapsule](https://pyo3.rs/main/doc/pyo3/types/struct.pycapsule). + +A complete example can be found in the [examples folder](https://github.com/apache/datafusion-python/tree/main/examples). + +```rust +#[pymethods] +impl MyTableProvider { + + fn __datafusion_table_provider__<'py>( + &self, + py: Python<'py>, + ) -> PyResult> { + let name = cr"datafusion_table_provider".into(); + + let provider = Arc::new(self.clone()); + let provider = FFI_TableProvider::new(provider, false, None); + + PyCapsule::new_bound(py, provider, Some(name.clone())) + } +} +``` + +Once you have this library available, you can construct a +{py:class}`~datafusion.Table` in Python and register it with the +`SessionContext`. + +```python +from datafusion import SessionContext, Table + +ctx = SessionContext() +provider = MyTableProvider() + +ctx.register_table("capsule_table", provider) + +ctx.table("capsule_table").show() +``` diff --git a/docs/source/user-guide/io/table_provider.rst b/docs/source/user-guide/io/table_provider.rst deleted file mode 100644 index 29e5d9880..000000000 --- a/docs/source/user-guide/io/table_provider.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -.. _io_custom_table_provider: - -Custom Table Provider -===================== - -If you have a custom data source that you want to integrate with DataFusion, you can do so by -implementing the `TableProvider `_ -interface in Rust and then exposing it in Python. To do so, -you must use DataFusion 43.0.0 or later and expose a `FFI_TableProvider `_ -via `PyCapsule `_. - -A complete example can be found in the `examples folder `_. - -.. code-block:: rust - - #[pymethods] - impl MyTableProvider { - - fn __datafusion_table_provider__<'py>( - &self, - py: Python<'py>, - ) -> PyResult> { - let name = cr"datafusion_table_provider".into(); - - let provider = Arc::new(self.clone()); - let provider = FFI_TableProvider::new(provider, false, None); - - PyCapsule::new_bound(py, provider, Some(name.clone())) - } - } - -Once you have this library available, you can construct a -:py:class:`~datafusion.Table` in Python and register it with the -``SessionContext``. - -.. code-block:: python - - from datafusion import SessionContext, Table - - ctx = SessionContext() - provider = MyTableProvider() - - ctx.register_table("capsule_table", provider) - - ctx.table("capsule_table").show() diff --git a/docs/source/user-guide/sql.rst b/docs/source/user-guide/sql.md similarity index 54% rename from docs/source/user-guide/sql.rst rename to docs/source/user-guide/sql.md index b4bfb9611..20ae8bc27 100644 --- a/docs/source/user-guide/sql.rst +++ b/docs/source/user-guide/sql.md @@ -1,25 +1,36 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at +% Licensed to the Apache Software Foundation (ASF) under one -.. http://www.apache.org/licenses/LICENSE-2.0 +% or more contributor license agreements. See the NOTICE file -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. +% distributed with this work for additional information -SQL -=== +% regarding copyright ownership. The ASF licenses this file -DataFusion also offers a SQL API, read the full reference `here `_ +% to you under the Apache License, Version 2.0 (the +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# SQL + +DataFusion also offers a SQL API, read the full reference [here](https://arrow.apache.org/datafusion/user-guide/sql/index.html) + +```{eval-rst} .. ipython:: python import datafusion @@ -36,16 +47,17 @@ DataFusion also offers a SQL API, read the full reference `here `_, +[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html), but allow passing named parameters into a SQL query. Consider this simple example. +```{eval-rst} .. ipython:: python def show_attacks(ctx: SessionContext, threshold: int) -> None: @@ -53,34 +65,36 @@ example. 'SELECT "Name", "Attack" FROM pokemon WHERE "Attack" > $val', val=threshold ).show(num=5) show_attacks(ctx, 75) +``` When passing parameters like the example above we convert the Python objects into their string representation. We also have special case handling -for :py:class:`~datafusion.dataframe.DataFrame` objects, since they cannot simply +for {py:class}`~datafusion.dataframe.DataFrame` objects, since they cannot simply be turned into string representations for an SQL query. In these cases we -will register a temporary view in the :py:class:`~datafusion.context.SessionContext` +will register a temporary view in the {py:class}`~datafusion.context.SessionContext` using a generated table name. The formatting for passing string replacement objects is to precede the -variable name with a single ``$``. This works for all dialects in -the SQL parser except ``hive`` and ``mysql``. Since these dialects do not +variable name with a single `$`. This works for all dialects in +the SQL parser except `hive` and `mysql`. Since these dialects do not support named placeholders, we are unable to do this type of replacement. We recommend either switching to another dialect or using Python f-string style replacement. -.. warning:: - - To support DataFrame parameterized queries, your session must support - registration of temporary views. The default - :py:class:`~datafusion.catalog.CatalogProvider` and - :py:class:`~datafusion.catalog.SchemaProvider` do have this capability. - If you have implemented custom providers, it is important that temporary - views do not persist across :py:class:`~datafusion.context.SessionContext` - or you may get unintended consequences. - -The following example shows passing in both a :py:class:`~datafusion.dataframe.DataFrame` +:::{warning} +To support DataFrame parameterized queries, your session must support +registration of temporary views. The default +{py:class}`~datafusion.catalog.CatalogProvider` and +{py:class}`~datafusion.catalog.SchemaProvider` do have this capability. +If you have implemented custom providers, it is important that temporary +views do not persist across {py:class}`~datafusion.context.SessionContext` +or you may get unintended consequences. +::: + +The following example shows passing in both a {py:class}`~datafusion.dataframe.DataFrame` object as well as a Python object to be used in parameterized replacement. +```{eval-rst} .. ipython:: python def show_column( @@ -94,24 +108,26 @@ object as well as a Python object to be used in parameterized replacement. ).show(num=5) df = ctx.table("pokemon") show_column(ctx, '"Defense"', df, 75) +``` The approach implemented for conversion of variables into a SQL query relies on string conversion. This has the potential for data loss, specifically for cases like floating point numbers. If you need to pass variables into a parameterized query and it is important to maintain the original value without conversion to a string, then you can use the -optional parameter ``param_values`` to specify these. This parameter +optional parameter `param_values` to specify these. This parameter expects a dictionary mapping from the parameter name to a Python object. Those objects will be cast into a -`PyArrow Scalar Value `_. +[PyArrow Scalar Value](https://arrow.apache.org/docs/python/generated/pyarrow.Scalar.html). -Using ``param_values`` will rely on the SQL dialect you have configured -for your session. This can be set using the :ref:`configuration options ` -of your :py:class:`~datafusion.context.SessionContext`. Similar to how -`prepared statements `_ +Using `param_values` will rely on the SQL dialect you have configured +for your session. This can be set using the {ref}`configuration options ` +of your {py:class}`~datafusion.context.SessionContext`. Similar to how +[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html) work, these parameters are limited to places where you would pass in a scalar value, such as a comparison. +```{eval-rst} .. ipython:: python def param_attacks(ctx: SessionContext, threshold: int) -> None: @@ -120,3 +136,4 @@ scalar value, such as a comparison. param_values={"val": threshold}, ).show(num=5) param_attacks(ctx, 75) +``` diff --git a/docs/source/user-guide/upgrade-guides.md b/docs/source/user-guide/upgrade-guides.md new file mode 100644 index 000000000..5db5fa8fd --- /dev/null +++ b/docs/source/user-guide/upgrade-guides.md @@ -0,0 +1,172 @@ +% Licensed to the Apache Software Foundation (ASF) under one + +% or more contributor license agreements. See the NOTICE file + +% distributed with this work for additional information + +% regarding copyright ownership. The ASF licenses this file + +% to you under the Apache License, Version 2.0 (the + +% "License"); you may not use this file except in compliance + +% with the License. You may obtain a copy of the License at + +% http://www.apache.org/licenses/LICENSE-2.0 + +% Unless required by applicable law or agreed to in writing, + +% software distributed under the License is distributed on an + +% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + +% KIND, either express or implied. See the License for the + +% specific language governing permissions and limitations + +% under the License. + +# Upgrade Guides + +## DataFusion 54.0.0 + +The `Config` class has been removed. It was a standalone wrapper around +`ConfigOptions` that could not be connected to a `SessionContext`, making it +effectively unusable. Use {py:class}`~datafusion.context.SessionConfig` instead, +which is passed directly to `SessionContext`. + +Before: + +```python +from datafusion import Config + +config = Config() +config.set("datafusion.execution.batch_size", "4096") +# config could not be passed to SessionContext +``` + +After: + +```python +from datafusion import SessionConfig, SessionContext + +config = SessionConfig().set("datafusion.execution.batch_size", "4096") +ctx = SessionContext(config) +``` + +The aggregate functions {py:func}`~datafusion.functions.sum` and +{py:func}`~datafusion.functions.avg` now accept a `distinct` argument, matching +the other aggregate functions. `distinct` is inserted *before* `filter` in the +argument list, so any code that passed `filter` positionally must be updated to +pass it as a keyword argument. The types are distinct so a type checker should flag this. + +Before: + +```python +f.sum(column("a"), my_filter) +f.avg(column("a"), my_filter) +``` + +Now: + +```python +f.sum(column("a"), filter=my_filter) +f.avg(column("a"), filter=my_filter) +``` + +## DataFusion 53.0.0 + +This version includes an upgraded version of `pyo3`, which changed the way to extract an FFI +object. Example: + +Before: + +```rust +let codec = unsafe { capsule.reference::() }; +``` + +Now: + +```rust +let data: NonNull = capsule + .pointer_checked(Some(c_str!("datafusion_logical_extension_codec")))? + .cast(); +let codec = unsafe { data.as_ref() }; +``` + +## DataFusion 52.0.0 + +This version includes a major update to the {ref}`ffi` due to upgrades +to the [Foreign Function Interface](https://doc.rust-lang.org/nomicon/ffi.html). +Users who contribute their own `CatalogProvider`, `SchemaProvider`, +`TableProvider` or `TableFunction` via FFI must now provide access to a +`LogicalExtensionCodec` and a `TaskContextProvider`. The function signatures +for the methods to get these `PyCapsule` objects now requires an additional +parameter, which is a Python object that can be used to extract the +`FFI_LogicalExtensionCodec` that is necessary. + +A complete example can be found in the [FFI example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example). +Your FFI hook methods — `__datafusion_catalog_provider__`, +`__datafusion_schema_provider__`, `__datafusion_table_provider__`, and +`__datafusion_table_function__` — need to be updated to accept an additional +`session: Bound` parameter, as shown in this example. + +```rust +#[pymethods] +impl MyCatalogProvider { + pub fn __datafusion_catalog_provider__<'py>( + &self, + py: Python<'py>, + session: Bound, + ) -> PyResult> { + let name = cr"datafusion_catalog_provider".into(); + + let provider = Arc::clone(&self.inner) as Arc; + + let codec = ffi_logical_codec_from_pycapsule(session)?; + let provider = FFI_CatalogProvider::new_with_ffi_codec(provider, None, codec); + + PyCapsule::new(py, provider, Some(name)) + } +} +``` + +To extract the logical extension codec FFI object from the provided object you +can implement a helper method such as: + +```rust +pub(crate) fn ffi_logical_codec_from_pycapsule( + obj: Bound, +) -> PyResult { + let attr_name = "__datafusion_logical_extension_codec__"; + let capsule = if obj.hasattr(attr_name)? { + obj.getattr(attr_name)?.call0()? + } else { + obj + }; + + let capsule = capsule.downcast::()?; + validate_pycapsule(capsule, "datafusion_logical_extension_codec")?; + + let codec = unsafe { capsule.reference::() }; + + Ok(codec.clone()) +} +``` + +The DataFusion FFI interface updates no longer depend directly on the +`datafusion` core crate. You can improve your build times and potentially +reduce your library binary size by removing this dependency and instead +using the specific datafusion project crates. + +For example, instead of including expressions like: + +```rust +use datafusion::catalog::MemTable; +``` + +Instead you can now write: + +```rust +use datafusion_catalog::MemTable; +``` diff --git a/docs/source/user-guide/upgrade-guides.rst b/docs/source/user-guide/upgrade-guides.rst deleted file mode 100644 index 9671594b8..000000000 --- a/docs/source/user-guide/upgrade-guides.rst +++ /dev/null @@ -1,166 +0,0 @@ -.. Licensed to the Apache Software Foundation (ASF) under one -.. or more contributor license agreements. See the NOTICE file -.. distributed with this work for additional information -.. regarding copyright ownership. The ASF licenses this file -.. to you under the Apache License, Version 2.0 (the -.. "License"); you may not use this file except in compliance -.. with the License. You may obtain a copy of the License at - -.. http://www.apache.org/licenses/LICENSE-2.0 - -.. Unless required by applicable law or agreed to in writing, -.. software distributed under the License is distributed on an -.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -.. KIND, either express or implied. See the License for the -.. specific language governing permissions and limitations -.. under the License. - -Upgrade Guides -============== - -DataFusion 54.0.0 ------------------ - -The ``Config`` class has been removed. It was a standalone wrapper around -``ConfigOptions`` that could not be connected to a ``SessionContext``, making it -effectively unusable. Use :py:class:`~datafusion.context.SessionConfig` instead, -which is passed directly to ``SessionContext``. - -Before: - -.. code-block:: python - - from datafusion import Config - - config = Config() - config.set("datafusion.execution.batch_size", "4096") - # config could not be passed to SessionContext - -After: - -.. code-block:: python - - from datafusion import SessionConfig, SessionContext - - config = SessionConfig().set("datafusion.execution.batch_size", "4096") - ctx = SessionContext(config) - -The aggregate functions :py:func:`~datafusion.functions.sum` and -:py:func:`~datafusion.functions.avg` now accept a ``distinct`` argument, matching -the other aggregate functions. ``distinct`` is inserted *before* ``filter`` in the -argument list, so any code that passed ``filter`` positionally must be updated to -pass it as a keyword argument. The types are distinct so a type checker should flag this. - -Before: - -.. code-block:: python - - f.sum(column("a"), my_filter) - f.avg(column("a"), my_filter) - -Now: - -.. code-block:: python - - f.sum(column("a"), filter=my_filter) - f.avg(column("a"), filter=my_filter) - -DataFusion 53.0.0 ------------------ - -This version includes an upgraded version of ``pyo3``, which changed the way to extract an FFI -object. Example: - -Before: - -.. code-block:: rust - - let codec = unsafe { capsule.reference::() }; - -Now: - -.. code-block:: rust - - let data: NonNull = capsule - .pointer_checked(Some(c_str!("datafusion_logical_extension_codec")))? - .cast(); - let codec = unsafe { data.as_ref() }; - -DataFusion 52.0.0 ------------------ - -This version includes a major update to the :ref:`ffi` due to upgrades -to the `Foreign Function Interface `_. -Users who contribute their own ``CatalogProvider``, ``SchemaProvider``, -``TableProvider`` or ``TableFunction`` via FFI must now provide access to a -``LogicalExtensionCodec`` and a ``TaskContextProvider``. The function signatures -for the methods to get these ``PyCapsule`` objects now requires an additional -parameter, which is a Python object that can be used to extract the -``FFI_LogicalExtensionCodec`` that is necessary. - -A complete example can be found in the `FFI example `_. -Your FFI hook methods — ``__datafusion_catalog_provider__``, -``__datafusion_schema_provider__``, ``__datafusion_table_provider__``, and -``__datafusion_table_function__`` — need to be updated to accept an additional -``session: Bound`` parameter, as shown in this example. - -.. code-block:: rust - - #[pymethods] - impl MyCatalogProvider { - pub fn __datafusion_catalog_provider__<'py>( - &self, - py: Python<'py>, - session: Bound, - ) -> PyResult> { - let name = cr"datafusion_catalog_provider".into(); - - let provider = Arc::clone(&self.inner) as Arc; - - let codec = ffi_logical_codec_from_pycapsule(session)?; - let provider = FFI_CatalogProvider::new_with_ffi_codec(provider, None, codec); - - PyCapsule::new(py, provider, Some(name)) - } - } - -To extract the logical extension codec FFI object from the provided object you -can implement a helper method such as: - -.. code-block:: rust - - pub(crate) fn ffi_logical_codec_from_pycapsule( - obj: Bound, - ) -> PyResult { - let attr_name = "__datafusion_logical_extension_codec__"; - let capsule = if obj.hasattr(attr_name)? { - obj.getattr(attr_name)?.call0()? - } else { - obj - }; - - let capsule = capsule.downcast::()?; - validate_pycapsule(capsule, "datafusion_logical_extension_codec")?; - - let codec = unsafe { capsule.reference::() }; - - Ok(codec.clone()) - } - - -The DataFusion FFI interface updates no longer depend directly on the -``datafusion`` core crate. You can improve your build times and potentially -reduce your library binary size by removing this dependency and instead -using the specific datafusion project crates. - -For example, instead of including expressions like: - -.. code-block:: rust - - use datafusion::catalog::MemTable; - -Instead you can now write: - -.. code-block:: rust - - use datafusion_catalog::MemTable; From 30efd76687178ff1599abeff34afe6835ff0d4a2 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Sun, 7 Jun 2026 15:27:24 +0200 Subject: [PATCH 02/18] docs: fix Apache license header format in converted markdown files RST-to-MD conversion emitted MyST `%` comment syntax with blank line between each header line, which renders as visible text. Replace with canonical `` HTML comment block matching upstream apache/datafusion and this repo's existing markdown files. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/source/contributor-guide/ffi.md | 45 ++++++++----------- docs/source/contributor-guide/index.md | 45 ++++++++----------- docs/source/contributor-guide/introduction.md | 45 ++++++++----------- docs/source/index.md | 45 ++++++++----------- docs/source/links.md | 45 ++++++++----------- .../source/user-guide/ai-coding-assistants.md | 45 ++++++++----------- docs/source/user-guide/basics.md | 45 ++++++++----------- .../common-operations/aggregations.md | 45 ++++++++----------- .../common-operations/basic-info.md | 45 ++++++++----------- .../common-operations/expressions.md | 45 ++++++++----------- .../user-guide/common-operations/functions.md | 45 ++++++++----------- .../user-guide/common-operations/index.md | 45 ++++++++----------- .../user-guide/common-operations/joins.md | 45 ++++++++----------- .../common-operations/select-and-filter.md | 45 ++++++++----------- .../common-operations/udf-and-udfa.md | 45 ++++++++----------- .../user-guide/common-operations/views.md | 45 ++++++++----------- .../user-guide/common-operations/windows.md | 45 ++++++++----------- docs/source/user-guide/configuration.md | 45 ++++++++----------- docs/source/user-guide/data-sources.md | 45 ++++++++----------- .../user-guide/dataframe/execution-metrics.md | 45 ++++++++----------- docs/source/user-guide/dataframe/index.md | 45 ++++++++----------- docs/source/user-guide/dataframe/rendering.md | 45 ++++++++----------- docs/source/user-guide/distributing-work.md | 45 ++++++++----------- docs/source/user-guide/index.md | 45 ++++++++----------- docs/source/user-guide/introduction.md | 45 ++++++++----------- docs/source/user-guide/io/arrow.md | 45 ++++++++----------- docs/source/user-guide/io/avro.md | 45 ++++++++----------- docs/source/user-guide/io/csv.md | 45 ++++++++----------- docs/source/user-guide/io/index.md | 45 ++++++++----------- docs/source/user-guide/io/json.md | 45 ++++++++----------- docs/source/user-guide/io/parquet.md | 45 ++++++++----------- docs/source/user-guide/io/table_provider.md | 45 ++++++++----------- docs/source/user-guide/sql.md | 45 ++++++++----------- docs/source/user-guide/upgrade-guides.md | 45 ++++++++----------- 34 files changed, 612 insertions(+), 918 deletions(-) diff --git a/docs/source/contributor-guide/ffi.md b/docs/source/contributor-guide/ffi.md index 403cdf40e..bf65cad2a 100644 --- a/docs/source/contributor-guide/ffi.md +++ b/docs/source/contributor-guide/ffi.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (ffi)= diff --git a/docs/source/contributor-guide/index.md b/docs/source/contributor-guide/index.md index df528ed54..a989a068d 100644 --- a/docs/source/contributor-guide/index.md +++ b/docs/source/contributor-guide/index.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Contributor Guide diff --git a/docs/source/contributor-guide/introduction.md b/docs/source/contributor-guide/introduction.md index fa87c57a2..683b27bf2 100644 --- a/docs/source/contributor-guide/introduction.md +++ b/docs/source/contributor-guide/introduction.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Introduction diff --git a/docs/source/index.md b/docs/source/index.md index 5b1f0f53b..8f92dd9be 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # DataFusion in Python diff --git a/docs/source/links.md b/docs/source/links.md index fbcde343e..9ad3ff305 100644 --- a/docs/source/links.md +++ b/docs/source/links.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Links diff --git a/docs/source/user-guide/ai-coding-assistants.md b/docs/source/user-guide/ai-coding-assistants.md index 90335837b..2057ec410 100644 --- a/docs/source/user-guide/ai-coding-assistants.md +++ b/docs/source/user-guide/ai-coding-assistants.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Using AI Coding Assistants diff --git a/docs/source/user-guide/basics.md b/docs/source/user-guide/basics.md index 800b6a67c..42d8432d9 100644 --- a/docs/source/user-guide/basics.md +++ b/docs/source/user-guide/basics.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (user_guide_concepts)= diff --git a/docs/source/user-guide/common-operations/aggregations.md b/docs/source/user-guide/common-operations/aggregations.md index 7a59390a2..e578a3fa5 100644 --- a/docs/source/user-guide/common-operations/aggregations.md +++ b/docs/source/user-guide/common-operations/aggregations.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (aggregation)= diff --git a/docs/source/user-guide/common-operations/basic-info.md b/docs/source/user-guide/common-operations/basic-info.md index ed4816338..263068b4e 100644 --- a/docs/source/user-guide/common-operations/basic-info.md +++ b/docs/source/user-guide/common-operations/basic-info.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Basic Operations diff --git a/docs/source/user-guide/common-operations/expressions.md b/docs/source/user-guide/common-operations/expressions.md index 008f1d75f..0679febab 100644 --- a/docs/source/user-guide/common-operations/expressions.md +++ b/docs/source/user-guide/common-operations/expressions.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (expressions)= diff --git a/docs/source/user-guide/common-operations/functions.md b/docs/source/user-guide/common-operations/functions.md index f57e53ecd..2fbd0c85f 100644 --- a/docs/source/user-guide/common-operations/functions.md +++ b/docs/source/user-guide/common-operations/functions.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Functions diff --git a/docs/source/user-guide/common-operations/index.md b/docs/source/user-guide/common-operations/index.md index 58947844c..cf1559e8c 100644 --- a/docs/source/user-guide/common-operations/index.md +++ b/docs/source/user-guide/common-operations/index.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Common Operations diff --git a/docs/source/user-guide/common-operations/joins.md b/docs/source/user-guide/common-operations/joins.md index bcbd63613..6f4ccf3ad 100644 --- a/docs/source/user-guide/common-operations/joins.md +++ b/docs/source/user-guide/common-operations/joins.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Joins diff --git a/docs/source/user-guide/common-operations/select-and-filter.md b/docs/source/user-guide/common-operations/select-and-filter.md index 61de45814..8126dfc3a 100644 --- a/docs/source/user-guide/common-operations/select-and-filter.md +++ b/docs/source/user-guide/common-operations/select-and-filter.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Column Selections diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.md b/docs/source/user-guide/common-operations/udf-and-udfa.md index d673aaa28..e2d55ee17 100644 --- a/docs/source/user-guide/common-operations/udf-and-udfa.md +++ b/docs/source/user-guide/common-operations/udf-and-udfa.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # User-Defined Functions diff --git a/docs/source/user-guide/common-operations/views.md b/docs/source/user-guide/common-operations/views.md index be00e25a2..4feaac028 100644 --- a/docs/source/user-guide/common-operations/views.md +++ b/docs/source/user-guide/common-operations/views.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Registering Views diff --git a/docs/source/user-guide/common-operations/windows.md b/docs/source/user-guide/common-operations/windows.md index e7e45178a..267218003 100644 --- a/docs/source/user-guide/common-operations/windows.md +++ b/docs/source/user-guide/common-operations/windows.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (window_functions)= diff --git a/docs/source/user-guide/configuration.md b/docs/source/user-guide/configuration.md index 21a06da18..d1c5c9b44 100644 --- a/docs/source/user-guide/configuration.md +++ b/docs/source/user-guide/configuration.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (configuration)= diff --git a/docs/source/user-guide/data-sources.md b/docs/source/user-guide/data-sources.md index cab7c3897..425e9402f 100644 --- a/docs/source/user-guide/data-sources.md +++ b/docs/source/user-guide/data-sources.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (user_guide_data_sources)= diff --git a/docs/source/user-guide/dataframe/execution-metrics.md b/docs/source/user-guide/dataframe/execution-metrics.md index e66ea1100..ede3339e0 100644 --- a/docs/source/user-guide/dataframe/execution-metrics.md +++ b/docs/source/user-guide/dataframe/execution-metrics.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (execution_metrics)= diff --git a/docs/source/user-guide/dataframe/index.md b/docs/source/user-guide/dataframe/index.md index dd7d949e1..a3af4e3cd 100644 --- a/docs/source/user-guide/dataframe/index.md +++ b/docs/source/user-guide/dataframe/index.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # DataFrames diff --git a/docs/source/user-guide/dataframe/rendering.md b/docs/source/user-guide/dataframe/rendering.md index d92d9b386..8b3019016 100644 --- a/docs/source/user-guide/dataframe/rendering.md +++ b/docs/source/user-guide/dataframe/rendering.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # DataFrame Rendering diff --git a/docs/source/user-guide/distributing-work.md b/docs/source/user-guide/distributing-work.md index 8634cf24d..7d5aa475a 100644 --- a/docs/source/user-guide/distributing-work.md +++ b/docs/source/user-guide/distributing-work.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Distributing work diff --git a/docs/source/user-guide/index.md b/docs/source/user-guide/index.md index 509c850c8..0155c4e91 100644 --- a/docs/source/user-guide/index.md +++ b/docs/source/user-guide/index.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # User Guide diff --git a/docs/source/user-guide/introduction.md b/docs/source/user-guide/introduction.md index 1abe55a34..15a403bf4 100644 --- a/docs/source/user-guide/introduction.md +++ b/docs/source/user-guide/introduction.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (guide)= diff --git a/docs/source/user-guide/io/arrow.md b/docs/source/user-guide/io/arrow.md index 3d35e83f8..644f74dc5 100644 --- a/docs/source/user-guide/io/arrow.md +++ b/docs/source/user-guide/io/arrow.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Arrow diff --git a/docs/source/user-guide/io/avro.md b/docs/source/user-guide/io/avro.md index 92a63e6b0..5654547ac 100644 --- a/docs/source/user-guide/io/avro.md +++ b/docs/source/user-guide/io/avro.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (io_avro)= diff --git a/docs/source/user-guide/io/csv.md b/docs/source/user-guide/io/csv.md index 4c541c57d..3022a6db8 100644 --- a/docs/source/user-guide/io/csv.md +++ b/docs/source/user-guide/io/csv.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (io_csv)= diff --git a/docs/source/user-guide/io/index.md b/docs/source/user-guide/io/index.md index a2da64989..764e05cb9 100644 --- a/docs/source/user-guide/io/index.md +++ b/docs/source/user-guide/io/index.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # IO diff --git a/docs/source/user-guide/io/json.md b/docs/source/user-guide/io/json.md index bcf60dfe3..45df4c6ce 100644 --- a/docs/source/user-guide/io/json.md +++ b/docs/source/user-guide/io/json.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (io_json)= diff --git a/docs/source/user-guide/io/parquet.md b/docs/source/user-guide/io/parquet.md index ca2187409..da79360e8 100644 --- a/docs/source/user-guide/io/parquet.md +++ b/docs/source/user-guide/io/parquet.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (io_parquet)= diff --git a/docs/source/user-guide/io/table_provider.md b/docs/source/user-guide/io/table_provider.md index 0116ccf3f..3c436ba1d 100644 --- a/docs/source/user-guide/io/table_provider.md +++ b/docs/source/user-guide/io/table_provider.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + (io_custom_table_provider)= diff --git a/docs/source/user-guide/sql.md b/docs/source/user-guide/sql.md index 20ae8bc27..339eb4949 100644 --- a/docs/source/user-guide/sql.md +++ b/docs/source/user-guide/sql.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # SQL diff --git a/docs/source/user-guide/upgrade-guides.md b/docs/source/user-guide/upgrade-guides.md index 5db5fa8fd..360e0533c 100644 --- a/docs/source/user-guide/upgrade-guides.md +++ b/docs/source/user-guide/upgrade-guides.md @@ -1,30 +1,21 @@ -% Licensed to the Apache Software Foundation (ASF) under one - -% or more contributor license agreements. See the NOTICE file - -% distributed with this work for additional information - -% regarding copyright ownership. The ASF licenses this file - -% to you under the Apache License, Version 2.0 (the - -% "License"); you may not use this file except in compliance - -% with the License. You may obtain a copy of the License at - -% http://www.apache.org/licenses/LICENSE-2.0 - -% Unless required by applicable law or agreed to in writing, - -% software distributed under the License is distributed on an - -% "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - -% KIND, either express or implied. See the License for the - -% specific language governing permissions and limitations - -% under the License. + # Upgrade Guides From 20707b80c7040fd740b38e35b7030723e8eb14bb Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Sun, 7 Jun 2026 17:36:13 +0200 Subject: [PATCH 03/18] docs: migrate from Sphinx+MyST to MkDocs+mkdocstrings Replaces the Sphinx + MyST + sphinx-autoapi + pydata-sphinx-theme stack with MkDocs + Material + mkdocstrings + mkdocs-jupyter. The MyST conversion from #1579 forms the base; this commit removes the remaining RST/MyST artifacts and rebuilds the docs around MkDocs. Build tooling * Replace `docs/source/conf.py` and the Sphinx Makefile with a root-level `mkdocs.yml` and a thin Makefile wrapper that runs `mkdocs build`. * Swap the `[dependency-groups] docs` deps in `pyproject.toml`: out go sphinx / sphinx-autoapi / sphinx-reredirects / myst-parser / pydata-sphinx-theme; in come mkdocs<2, mkdocs-material, mkdocstrings, mkdocs-jupyter, mkdocs-redirects. The `mkdocs<2` cap reflects upstream uncertainty around the announced MkDocs 2.0 rewrite. * Update the `build-docs` job in `.github/workflows/build.yml` to run `mkdocs build` and stage data files (pokemon.csv, taxi parquet) at `docs/source/` so notebooks can resolve relative paths during execution. The asf-staging / asf-site publish logic is preserved byte-for-byte. API reference * Replace sphinx-autoapi with one mkdocstrings page per top-level class or module under `docs/source/reference/`. The deprecated members previously hidden via `autoapi_skip_member_fn` are reproduced via mkdocstrings `filters:` / `members:` options. * Add `dev/check_api_coverage.py`, a CI guard that fails if any `datafusion.__all__` entry lacks a heading in `docs/source/reference/`. User-guide notebooks * Convert the 14 `.md` pages that contained `{eval-rst}` `.. ipython::` blocks into executable Jupyter notebooks rendered by `mkdocs-jupyter`. Each notebook starts with a small setup cell that locates the docs root (so `pokemon.csv` etc. resolve regardless of nesting depth) and imports the common `SessionContext` / `col` / `lit` / `functions` symbols. * Convert two remaining `{eval-rst}` `.. list-table::` blocks (distributing-work, execution-metrics) into Markdown tables. * Rewrite MyST `:::{note}` / `:::{warning}` / `:::{tip}` admonitions to Material's `!!! note` syntax. Cross-reference rewrite * Rewrite Sphinx/MyST roles to Markdown links across user-guide pages, notebooks, and `python/datafusion/*.py` docstrings via `dev/rewrite_doc_roles.py`. Patterns covered: `:py:class:` / `:py:func:` / `:py:meth:` / `:py:mod:` / `{py:class}` / `{py:func}` / `{py:meth}` / `{py:mod}` / `{code}` / `{doc}` / `{ref}`, plus stray `(label)=` anchors. * Add `inventories:` for python and pyarrow under the mkdocstrings python handler so links to stdlib and Arrow symbols resolve. * Collapse intra-module and over-long markdown links to inline code in docstrings to keep lines within the 88-character ruff limit. Theme * Port the prior theme to a Material `custom_dir` overrides directory: Apache trademark footer (`_overrides/partials/copyright.html`) and a slimmed `theme_overrides.css` that keeps the `#D74633` accent applied to links and inline code while leaving the header palette white (light) / black (dark) to match `datafusion-comet`. The result builds with all 14 notebooks executing successfully; remaining cross-reference warnings are doc-quality follow-ups, not migration blockers. Co-Authored-By: Claude Opus 4.7 --- .github/workflows/build.yml | 13 +- .gitignore | 1 + dev/check_api_coverage.py | 82 ++ dev/rewrite_doc_roles.py | 143 ++ docs/.gitignore | 5 +- docs/Makefile | 32 +- .../source/_overrides/partials/copyright.html | 31 + docs/source/_static/theme_overrides.css | 56 +- docs/source/_templates/layout.html | 22 - docs/source/_templates/sidebar-globaltoc.html | 30 - docs/source/conf.py | 183 --- docs/source/contributor-guide/ffi.md | 10 +- docs/source/contributor-guide/introduction.md | 10 +- docs/source/index.ipynb | 73 + docs/source/index.md | 63 - docs/source/reference/catalog.md | 17 + docs/source/reference/common.md | 5 + docs/source/reference/context.md | 21 + docs/source/reference/dataframe.md | 33 + docs/source/reference/expr.md | 25 + docs/source/reference/functions.md | 10 + docs/source/reference/index.md | 65 + docs/source/reference/io.md | 19 + docs/source/reference/ipc.md | 3 + docs/source/reference/object_store.md | 3 + docs/source/reference/options.md | 5 + docs/source/reference/plan.md | 17 + docs/source/reference/record_batch.md | 9 + docs/source/reference/substrait.md | 10 + docs/source/reference/unparser.md | 3 + docs/source/reference/user_defined.md | 37 + docs/source/user-guide/basics.ipynb | 81 ++ docs/source/user-guide/basics.md | 98 -- .../common-operations/aggregations.ipynb | 462 ++++++ .../common-operations/aggregations.md | 474 ------- .../common-operations/basic-info.ipynb | 138 ++ .../common-operations/basic-info.md | 71 - .../common-operations/expressions.ipynb | 382 +++++ .../common-operations/expressions.md | 357 ----- .../common-operations/functions.ipynb | 237 ++++ .../user-guide/common-operations/functions.md | 165 --- .../user-guide/common-operations/joins.ipynb | 237 ++++ .../user-guide/common-operations/joins.md | 181 --- .../common-operations/select-and-filter.ipynb | 115 ++ .../common-operations/select-and-filter.md | 71 - .../common-operations/udf-and-udfa.ipynb | 205 +++ .../common-operations/udf-and-udfa.md | 476 ------- .../common-operations/windows.ipynb | 225 +++ .../user-guide/common-operations/windows.md | 239 ---- docs/source/user-guide/configuration.md | 15 +- docs/source/user-guide/data-sources.ipynb | 114 ++ docs/source/user-guide/data-sources.md | 281 ---- .../user-guide/dataframe/execution-metrics.md | 114 +- docs/source/user-guide/dataframe/index.md | 86 +- docs/source/user-guide/dataframe/rendering.md | 16 +- docs/source/user-guide/distributing-work.md | 129 +- docs/source/user-guide/introduction.ipynb | 89 ++ docs/source/user-guide/introduction.md | 82 -- docs/source/user-guide/io/arrow.ipynb | 87 ++ docs/source/user-guide/io/arrow.md | 76 - docs/source/user-guide/io/avro.md | 3 +- docs/source/user-guide/io/csv.md | 7 +- docs/source/user-guide/io/json.md | 3 +- docs/source/user-guide/io/parquet.md | 5 +- docs/source/user-guide/io/table_provider.md | 3 +- docs/source/user-guide/sql.ipynb | 148 ++ docs/source/user-guide/sql.md | 130 -- docs/source/user-guide/upgrade-guides.md | 8 +- mkdocs.yml | 167 +++ pyproject.toml | 26 +- python/datafusion/__init__.py | 6 +- python/datafusion/context.py | 174 +-- python/datafusion/dataframe.py | 150 +- python/datafusion/expr.py | 70 +- python/datafusion/functions.py | 269 ++-- python/datafusion/io.py | 4 +- python/datafusion/plan.py | 16 +- python/datafusion/record_batch.py | 16 +- python/datafusion/substrait.py | 2 +- python/datafusion/user_defined.py | 50 +- uv.lock | 1242 ++++++++++++++--- 81 files changed, 4935 insertions(+), 3903 deletions(-) create mode 100644 dev/check_api_coverage.py create mode 100644 dev/rewrite_doc_roles.py create mode 100644 docs/source/_overrides/partials/copyright.html delete mode 100644 docs/source/_templates/layout.html delete mode 100644 docs/source/_templates/sidebar-globaltoc.html delete mode 100644 docs/source/conf.py create mode 100644 docs/source/index.ipynb delete mode 100644 docs/source/index.md create mode 100644 docs/source/reference/catalog.md create mode 100644 docs/source/reference/common.md create mode 100644 docs/source/reference/context.md create mode 100644 docs/source/reference/dataframe.md create mode 100644 docs/source/reference/expr.md create mode 100644 docs/source/reference/functions.md create mode 100644 docs/source/reference/index.md create mode 100644 docs/source/reference/io.md create mode 100644 docs/source/reference/ipc.md create mode 100644 docs/source/reference/object_store.md create mode 100644 docs/source/reference/options.md create mode 100644 docs/source/reference/plan.md create mode 100644 docs/source/reference/record_batch.md create mode 100644 docs/source/reference/substrait.md create mode 100644 docs/source/reference/unparser.md create mode 100644 docs/source/reference/user_defined.md create mode 100644 docs/source/user-guide/basics.ipynb delete mode 100644 docs/source/user-guide/basics.md create mode 100644 docs/source/user-guide/common-operations/aggregations.ipynb delete mode 100644 docs/source/user-guide/common-operations/aggregations.md create mode 100644 docs/source/user-guide/common-operations/basic-info.ipynb delete mode 100644 docs/source/user-guide/common-operations/basic-info.md create mode 100644 docs/source/user-guide/common-operations/expressions.ipynb delete mode 100644 docs/source/user-guide/common-operations/expressions.md create mode 100644 docs/source/user-guide/common-operations/functions.ipynb delete mode 100644 docs/source/user-guide/common-operations/functions.md create mode 100644 docs/source/user-guide/common-operations/joins.ipynb delete mode 100644 docs/source/user-guide/common-operations/joins.md create mode 100644 docs/source/user-guide/common-operations/select-and-filter.ipynb delete mode 100644 docs/source/user-guide/common-operations/select-and-filter.md create mode 100644 docs/source/user-guide/common-operations/udf-and-udfa.ipynb delete mode 100644 docs/source/user-guide/common-operations/udf-and-udfa.md create mode 100644 docs/source/user-guide/common-operations/windows.ipynb delete mode 100644 docs/source/user-guide/common-operations/windows.md create mode 100644 docs/source/user-guide/data-sources.ipynb delete mode 100644 docs/source/user-guide/data-sources.md create mode 100644 docs/source/user-guide/introduction.ipynb delete mode 100644 docs/source/user-guide/introduction.md create mode 100644 docs/source/user-guide/io/arrow.ipynb delete mode 100644 docs/source/user-guide/io/arrow.md create mode 100644 docs/source/user-guide/sql.ipynb delete mode 100644 docs/source/user-guide/sql.md create mode 100644 mkdocs.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 593a343e1..013f7ea45 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -523,7 +523,7 @@ jobs: enable-cache: true # Download the Linux wheel built in the previous job. - # Docs only need the abi3 wheel — interpreter doesn't matter for sphinx. + # Docs only need the abi3 wheel — interpreter doesn't matter for mkdocs. - name: Download pre-built Linux wheel uses: actions/download-artifact@v8 with: @@ -549,12 +549,19 @@ jobs: fi - name: Build docs + env: + DISABLE_MKDOCS_2_WARNING: "true" run: | set -x - cd docs + # Stage notebook data files at docs_dir root so notebooks can + # resolve relative paths like "pokemon.csv" during execution. + cd docs/source curl -O https://gist.githubusercontent.com/ritchie46/cac6b337ea52281aa23c049250a4ff03/raw/89a957ff3919d90e6ef2d34235e6bf22304f3366/pokemon.csv curl -O https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet - uv run --no-project make html + cd ../.. + # Verify every datafusion.__all__ entry is documented. + uv run --no-project python dev/check_api_coverage.py + uv run --no-project mkdocs build - name: Copy & push the generated HTML if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref_type == 'tag') diff --git a/.gitignore b/.gitignore index 614d82327..198a0b041 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,7 @@ target .idea /docs/temp /docs/build +/.cache .DS_Store .vscode diff --git a/dev/check_api_coverage.py b/dev/check_api_coverage.py new file mode 100644 index 000000000..d18f659cf --- /dev/null +++ b/dev/check_api_coverage.py @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Check that every symbol in datafusion.__all__ is documented. + +Walks every Markdown file under docs/source/reference/ and collects: + +1. The dotted target of every ``::: `` mkdocstrings directive. +2. Every Markdown heading (``##``, ``###``, etc.). + +A ``__all__`` entry is considered documented if its name appears as: + +- The leaf of a ``::: <...>`` directive, OR +- The leaf of a ``### name`` heading. + +Run from the repo root:: + + python dev/check_api_coverage.py +""" + +from __future__ import annotations + +import re +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[1] +REFERENCE_DIR = REPO_ROOT / "docs" / "source" / "reference" + + +def collect_documented_names() -> set[str]: + documented: set[str] = set() + directive_re = re.compile(r"^:::\s+([A-Za-z0-9_.]+)") + heading_re = re.compile(r"^#{1,6}\s+([A-Za-z0-9_]+)") + for md in REFERENCE_DIR.rglob("*.md"): + if md.stem != "index": + documented.add(md.stem) + for line in md.read_text().splitlines(): + m = directive_re.match(line.strip()) + if m: + dotted = m.group(1) + documented.add(dotted.split(".")[-1]) + documented.add(dotted) + continue + m = heading_re.match(line) + if m: + documented.add(m.group(1)) + return documented + + +def main() -> int: + sys.path.insert(0, str(REPO_ROOT / "python")) + import datafusion # noqa: PLC0415 + + documented = collect_documented_names() + missing = sorted(name for name in datafusion.__all__ if name not in documented) + if missing: + print("Undocumented entries in datafusion.__all__:") + for name in missing: + print(f" - {name}") + print(f"\n{len(missing)} symbol(s) missing from docs/source/reference/") + return 1 + print(f"All {len(datafusion.__all__)} __all__ entries are documented.") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/dev/rewrite_doc_roles.py b/dev/rewrite_doc_roles.py new file mode 100644 index 000000000..dc5346933 --- /dev/null +++ b/dev/rewrite_doc_roles.py @@ -0,0 +1,143 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Rewrite Sphinx / MyST cross-reference roles to Markdown links. + +Operates on: +- python/datafusion/*.py docstrings +- docs/source/**/*.md +- docs/source/**/*.ipynb (markdown cells) + +Conversions: + + :py:class:`~datafusion.x.Y` -> [`Y`][datafusion.x.Y] + :py:func:`~mod.fn` -> [`fn`][mod.fn] + :py:meth:`X.do ` -> [`X.do`][X.do] + {py:class}`~datafusion.x.Y` -> [`Y`][datafusion.x.Y] + {py:func}`mod.fn` -> [`mod.fn`][mod.fn] + {py:mod}`mod` -> [`mod`][mod] + {code}`text` -> `text` + {doc}`path/to/page` -> [path/to/page](path/to/page.md) + {doc}`Label ` -> [Label](path/to/page.md) + {ref}`anchor` -> [anchor](anchor) (best-effort) + {ref}`Label ` -> [Label](anchor) + (label)= (alone on a line) -> removed +""" + +from __future__ import annotations + +import json +import re +import sys +from pathlib import Path + +REPO = Path(__file__).resolve().parents[1] + +ROLE_PATTERNS = [ + # Sphinx RST roles: :py:class:`~mod.Name` or :py:class:`Name ` + ( + re.compile(r":py:(?:class|func|meth|mod|attr|obj|data):`~?([\w.]+)`"), + lambda m: f"[`{m.group(1).split('.')[-1]}`][{m.group(1)}]", + ), + ( + re.compile( + r":py:(?:class|func|meth|mod|attr|obj|data):`([^<`]+)\s*<([\w.]+)>`" + ), + lambda m: f"[`{m.group(1).strip()}`][{m.group(2)}]", + ), + # MyST roles: {py:class}`~mod.Name` + ( + re.compile(r"\{py:(?:class|func|meth|mod|attr|obj|data)\}`~?([\w.]+)`"), + lambda m: f"[`{m.group(1).split('.')[-1]}`][{m.group(1)}]", + ), + ( + re.compile( + r"\{py:(?:class|func|meth|mod|attr|obj|data)\}`([^<`]+)\s*<([\w.]+)>`" + ), + lambda m: f"[`{m.group(1).strip()}`][{m.group(2)}]", + ), + # {code}`text` -> `text` + (re.compile(r"\{code\}`([^`]+)`"), lambda m: f"`{m.group(1)}`"), + # {doc}`Label ` -> [Label](path.md) + ( + re.compile(r"\{doc\}`([^<`]+)\s*<([^>]+)>`"), + lambda m: f"[{m.group(1).strip()}]({m.group(2)}.md)", + ), + # {doc}`path` -> [path](path.md) + (re.compile(r"\{doc\}`([^`<]+)`"), lambda m: f"[{m.group(1)}]({m.group(1)}.md)"), + # {ref}`Label ` -> [Label](anchor) + ( + re.compile(r"\{ref\}`([^<`]+)\s*<([^>]+)>`"), + lambda m: f"[{m.group(1).strip()}]({m.group(2)})", + ), + # {ref}`anchor` -> [anchor](anchor) + (re.compile(r"\{ref\}`([^`<]+)`"), lambda m: f"[{m.group(1)}]({m.group(1)})"), +] + +# Drop standalone (label)= anchor lines (MyST cross-reference targets) +ANCHOR_LINE = re.compile(r"^\([a-zA-Z0-9_-]+\)=\s*$", re.MULTILINE) + + +def rewrite(text: str) -> str: + for pattern, repl in ROLE_PATTERNS: + text = pattern.sub(repl, text) + return ANCHOR_LINE.sub("", text) + + +def process_file(path: Path, *, dry_run: bool = False) -> int: + if path.suffix == ".ipynb": + original = path.read_text() + nb = json.loads(original) + changed = False + for cell in nb.get("cells", []): + if cell.get("cell_type") != "markdown": + continue + old = cell["source"] + text = "".join(old) if isinstance(old, list) else old + new = rewrite(text) + if new != text: + cell["source"] = new + changed = True + if changed and not dry_run: + path.write_text(json.dumps(nb, indent=1) + "\n") + return 1 if changed else 0 + + original = path.read_text() + new = rewrite(original) + if new != original: + if not dry_run: + path.write_text(new) + return 1 + return 0 + + +def main() -> int: + dry = "--dry-run" in sys.argv + paths = ( + list((REPO / "python" / "datafusion").rglob("*.py")) + + list((REPO / "docs" / "source").rglob("*.md")) + + list((REPO / "docs" / "source").rglob("*.ipynb")) + ) + changed = 0 + for p in paths: + changed += process_file(p, dry_run=dry) + print(f"changed: {changed} files" + (" (dry run)" if dry else "")) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/docs/.gitignore b/docs/.gitignore index 6e8a53b6f..6f2465e60 100644 --- a/docs/.gitignore +++ b/docs/.gitignore @@ -1,4 +1,7 @@ pokemon.csv yellow_trip_data.parquet yellow_tripdata_2021-01.parquet - +source/pokemon.csv +source/yellow_trip_data.parquet +source/yellow_tripdata_2021-01.parquet +build/ diff --git a/docs/Makefile b/docs/Makefile index 49ebae372..8ad0ae144 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -15,24 +15,24 @@ # specific language governing permissions and limitations # under the License. -# -# Minimal makefile for Sphinx documentation -# +# Thin wrapper. The mkdocs.yml lives at the repo root; run `mkdocs build` +# from one directory up. -# You can set these variables from the command line, and also -# from the environment for the first two. -SPHINXOPTS ?= -SPHINXBUILD ?= sphinx-build -SOURCEDIR = source -BUILDDIR = build +MKDOCS ?= mkdocs + +.PHONY: help html serve clean -# Put it first so that "make" without argument is like "make help". help: - @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + @echo "Targets:" + @echo " html - build site to docs/build/html" + @echo " serve - serve site at http://localhost:8000" + @echo " clean - remove docs/build/" + +html: + cd .. && DISABLE_MKDOCS_2_WARNING=true $(MKDOCS) build --strict -.PHONY: help Makefile +serve: + cd .. && DISABLE_MKDOCS_2_WARNING=true $(MKDOCS) serve -# Catch-all target: route all unknown targets to Sphinx using the new -# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). -%: Makefile - @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) --fail-on-warning \ No newline at end of file +clean: + rm -rf build/ diff --git a/docs/source/_overrides/partials/copyright.html b/docs/source/_overrides/partials/copyright.html new file mode 100644 index 000000000..2b034896b --- /dev/null +++ b/docs/source/_overrides/partials/copyright.html @@ -0,0 +1,31 @@ +{#- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +-#} +
+ {% if config.copyright %} +
+ {{ config.copyright }} +
+ {% endif %} +
+ Apache Arrow DataFusion, Arrow DataFusion, Apache, the Apache feather logo, + and the Apache Arrow DataFusion project logo are either registered + trademarks or trademarks of The Apache Software Foundation in the United + States and other countries. +
+
diff --git a/docs/source/_static/theme_overrides.css b/docs/source/_static/theme_overrides.css index 661454b12..596404ad9 100644 --- a/docs/source/_static/theme_overrides.css +++ b/docs/source/_static/theme_overrides.css @@ -17,49 +17,39 @@ * under the License. */ - -/* Customizing with theme CSS variables */ +/* Apache DataFusion accent color: rgb(215, 70, 51) == #D74633. + * + * Header bar is white in light mode / black in dark mode (Material palette + * primary: white|black). Red is applied only to links, code, and accents + * to match the pydata-sphinx-theme look used by datafusion-comet. + */ :root { - --pst-color-link-hover: 215, 70, 51; - --pst-color-headerlink: 215, 70, 51; - /* Softer blue from bootstrap's default info color */ - --pst-color-info: 23, 162, 184; + --md-accent-fg-color: #D74633; + --md-accent-fg-color--transparent: rgba(215, 70, 51, 0.1); + --md-typeset-a-color: #D74633; } -code { - color: rgb(215, 70, 51); +[data-md-color-scheme="slate"] { + --md-accent-fg-color: #FF8A75; + --md-typeset-a-color: #FF8A75; } -html[data-theme="dark"] code { - color: rgb(255, 138, 117); -} - -.footer { - text-align: center; +/* Inline code styled with accent color */ +.md-typeset code { + color: #D74633; } - -/* Bootstrap "table-striped" applied globally so individual tables in - user-guide pages don't need ":class: table-striped" added one by one. */ - -.table tbody tr:nth-of-type(odd) { - background-color: rgba(0, 0, 0, 0.05); +[data-md-color-scheme="slate"] .md-typeset code { + color: #FF8A75; } -html[data-theme="dark"] .table tbody tr:nth-of-type(odd) { - background-color: rgba(255, 255, 255, 0.05); +/* Center the footer copyright/trademark block */ +.md-copyright { + text-align: center; } - -/* Fix table text wrapping in RTD theme, - * see https://rackerlabs.github.io/docs-rackspace/tools/rtd-tables.html - */ - -@media screen { - table.docutils td { - /* !important prevents the common CSS stylesheets from overriding - this as on RTD they are loaded after this stylesheet */ - white-space: normal !important; - } +.md-copyright__trademark { + margin-top: 0.4em; + opacity: 0.7; } diff --git a/docs/source/_templates/layout.html b/docs/source/_templates/layout.html deleted file mode 100644 index d83d283c7..000000000 --- a/docs/source/_templates/layout.html +++ /dev/null @@ -1,22 +0,0 @@ -{% extends "pydata_sphinx_theme/layout.html" %} - - -{% block footer %} - -
-
- {% for footer_item in theme_footer_items %} -
- {% include footer_item %} -
- {% endfor %} -
-

Apache Arrow DataFusion, Arrow DataFusion, Apache, the Apache feather logo, and the Apache Arrow DataFusion project logo

-

are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.

-
-
-
- -{% endblock %} diff --git a/docs/source/_templates/sidebar-globaltoc.html b/docs/source/_templates/sidebar-globaltoc.html deleted file mode 100644 index f4aa2051f..000000000 --- a/docs/source/_templates/sidebar-globaltoc.html +++ /dev/null @@ -1,30 +0,0 @@ -{# Renders the global document toctree on every page (including the - landing page) with pydata-sphinx-theme's collapsible chevrons. - - The stock sidebar-nav-bs.html starts at the current section and is - stripped from the sidebar list by suppress_sidebar_toctree() on the - root page (no parent section). Using generate_toctree_html with - startdepth=0 renders the whole tree from root with the bootstrap - classes the theme's JS uses for expand/collapse toggles. Naming the - template "sidebar-globaltoc" sidesteps the suppress filter, which - matches on "sidebar-nav-bs.html" specifically. #} - diff --git a/docs/source/conf.py b/docs/source/conf.py deleted file mode 100644 index e10862388..000000000 --- a/docs/source/conf.py +++ /dev/null @@ -1,183 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -"""Documentation generation.""" - -# Configuration file for the Sphinx documentation builder. -# -# This file only contains a selection of the most common options. For a full -# list see the documentation: -# https://www.sphinx-doc.org/en/master/usage/configuration.html - -# -- Path setup -------------------------------------------------------------- - -# If extensions (or modules to document with autodoc) are in another directory, -# add these directories to sys.path here. If the directory is relative to the -# documentation root, use os.path.abspath to make it absolute, like shown here. -# -# import os -# import sys -# sys.path.insert(0, os.path.abspath('.')) - -# -- Project information ----------------------------------------------------- - -project = "Apache DataFusion in Python" -copyright = "2019-2026, Apache Software Foundation" -author = "Apache Software Foundation" - - -# -- General configuration --------------------------------------------------- - -# Add any Sphinx extension module names here, as strings. They can be -# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom -# ones. -extensions = [ - "sphinx.ext.mathjax", - "sphinx.ext.napoleon", - "myst_parser", - "IPython.sphinxext.ipython_directive", - "autoapi.extension", -] - -# NOTE: .rst stays alongside .md because sphinx-autoapi generates RST -# under autoapi/ and Sphinx needs the suffix to parse it. The human- -# authored docs are all MyST .md now; the .rst entry is only for the -# autoapi build artifacts. -source_suffix = { - ".rst": "restructuredtext", - ".md": "markdown", -} - -# Add any paths that contain templates here, relative to this directory. -templates_path = ["_templates"] - -# List of patterns, relative to source directory, that match files and -# directories to ignore when looking for source files. -# This pattern also affects html_static_path and html_extra_path. -exclude_patterns = [] - -autoapi_dirs = ["../../python"] -autoapi_ignore = ["*tests*"] -autoapi_member_order = "groupwise" -suppress_warnings = ["autoapi.python_import_resolution"] -autoapi_python_class_content = "both" -autoapi_keep_files = False # set to True for debugging generated files - - -def autoapi_skip_member_fn(app, what, name, obj, skip, options) -> bool: # noqa: ARG001 - skip_contents = [ - # Re-exports - ("class", "datafusion.DataFrame"), - ("class", "datafusion.SessionContext"), - ("module", "datafusion.common"), - # Duplicate modules (skip module-level docs to avoid duplication) - ("module", "datafusion.col"), - ("module", "datafusion.udf"), - # Deprecated - ("class", "datafusion.substrait.serde"), - ("class", "datafusion.substrait.plan"), - ("class", "datafusion.substrait.producer"), - ("class", "datafusion.substrait.consumer"), - ("method", "datafusion.context.SessionContext.tables"), - ("method", "datafusion.dataframe.DataFrame.unnest_column"), - ] - # Explicitly skip certain members listed above. These are either - # re-exports, duplicate module-level documentation, deprecated - # API surfaces, or private variables that would otherwise appear - # in the generated docs and cause confusing duplication. - # Keeping this explicit list avoids surprising entries in the - # AutoAPI output and gives us a single place to opt-out items - # when we intentionally hide them from the docs. - if (what, name) in skip_contents: - skip = True - - return skip - - -def setup(sphinx) -> None: - sphinx.connect("autoapi-skip-member", autoapi_skip_member_fn) - - -# -- Options for HTML output ------------------------------------------------- - -# The theme to use for HTML and HTML Help pages. See the documentation for -# a list of builtin themes. -# -html_theme = "pydata_sphinx_theme" - -html_theme_options = { - "use_edit_page_button": False, - "show_toc_level": 2, - "logo": { - "image_light": "_static/images/original.svg", - "image_dark": "_static/images/original.svg", - "alt_text": "Apache DataFusion in Python", - }, - "navbar_start": ["navbar-logo"], - "navbar_center": ["navbar-nav"], - "navbar_end": ["navbar-icon-links", "theme-switcher"], - "icon_links": [ - { - "name": "GitHub", - "url": "https://github.com/apache/datafusion-python", - "icon": "fa-brands fa-github", - }, - { - "name": "Rust API docs (docs.rs)", - "url": "https://docs.rs/datafusion/latest/datafusion/", - "icon": "fa-brands fa-rust", - }, - ], - "secondary_sidebar_items": [], - "collapse_navigation": True, - "show_nav_level": 2, -} - -html_context = { - "github_user": "apache", - "github_repo": "datafusion-python", - "github_version": "main", - "doc_path": "docs/source", - "default_mode": "auto", -} - -# Add any paths that contain custom static files (such as style sheets) here, -# relative to this directory. They are copied after the builtin static files, -# so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ["_static"] - -html_favicon = "_static/favicon.svg" - -# Copy agent-facing files (llms.txt) verbatim to the site root so they -# resolve at conventional URLs like `https://.../python/llms.txt`. -html_extra_path = ["llms.txt"] - -html_css_files = ["theme_overrides.css"] - -html_sidebars = { - "**": ["sidebar-globaltoc.html"], -} - -# tell myst_parser to auto-generate anchor links for headers h1, h2, h3 -myst_heading_anchors = 3 - -# MyST extensions: -# - tasklist: GitHub-style `- [x]` checkboxes -# - colon_fence: `:::{directive}` blocks (needed by execution-metrics.md -# after the RST -> MyST conversion) -# - deflist: definition lists (used in a couple of converted pages) -myst_enable_extensions = ["tasklist", "colon_fence", "deflist"] diff --git a/docs/source/contributor-guide/ffi.md b/docs/source/contributor-guide/ffi.md index bf65cad2a..def0b906f 100644 --- a/docs/source/contributor-guide/ffi.md +++ b/docs/source/contributor-guide/ffi.md @@ -17,7 +17,6 @@ under the License. --> -(ffi)= # Python Extensions @@ -29,9 +28,9 @@ when doing these integrations and the approach our project uses. ## The Primary Issue Suppose you wish to use DataFusion and you have a custom data source that can produce tables that -can then be queried against, similar to how you can register a {ref}`CSV ` or -{ref}`Parquet ` file. In DataFusion terminology, you likely want to implement a -{ref}`Custom Table Provider `. In an effort to make your data source +can then be queried against, similar to how you can register a [CSV](io_csv) or +[Parquet](io_parquet) file. In DataFusion terminology, you likely want to implement a +[Custom Table Provider](io_custom_table_provider). In an effort to make your data source as performant as possible and to utilize the features of DataFusion, you may decide to write your source in Rust and then expose it through [PyO3](https://pyo3.rs) as a Python library. @@ -80,7 +79,7 @@ of code. Also, the DataFusion Python project uses the existing definitions from [Apache Arrow CStream Interface](https://arrow.apache.org/docs/format/CStreamInterface.html) to support importing **and** exporting tables. Any Python package that supports reading the Arrow C Stream interface can work with DataFusion Python out of the box! You can read -more about working with Arrow sources in the {ref}`Data Sources ` +more about working with Arrow sources in the [Data Sources](user_guide_data_sources) page. To learn more about the Foreign Function Interface in Rust, the @@ -136,7 +135,6 @@ let my_provider = MyTableProvider::default(); let ffi_provider = FFI_TableProvider::new(Arc::new(my_provider), false, None); ``` -(ffi_pyclass_mutability)= ## PyO3 class mutability guidelines diff --git a/docs/source/contributor-guide/introduction.md b/docs/source/contributor-guide/introduction.md index 683b27bf2..691a0d2d2 100644 --- a/docs/source/contributor-guide/introduction.md +++ b/docs/source/contributor-guide/introduction.md @@ -29,7 +29,7 @@ In addition to submitting new PRs, we have a healthy tradition of community memb Doing so is a great way to help the community as well as get more familiar with Rust and the relevant codebases. Before opening a pull request that touches PyO3 bindings, please review the -{ref}`PyO3 class mutability guidelines ` so you can flag missing +[PyO3 class mutability guidelines](ffi_pyclass_mutability) so you can flag missing `#[pyclass(frozen)]` annotations during development and review. ## How to develop @@ -72,9 +72,9 @@ python -m pytest arrow-datafusion-python takes advantage of [pre-commit](https://pre-commit.com/) to assist developers with code linting to help reduce the number of commits that ultimately fail in CI due to linter errors. Using the pre-commit hooks is optional for the developer but certainly helpful for keeping PRs clean and concise. -Our pre-commit hooks can be installed by running {code}`pre-commit install`, which will install the configurations in your ARROW_DATAFUSION_PYTHON_ROOT/.github directory and run each time you perform a commit, failing to complete the commit if an offending lint is found allowing you to make changes locally before pushing. +Our pre-commit hooks can be installed by running `pre-commit install`, which will install the configurations in your ARROW_DATAFUSION_PYTHON_ROOT/.github directory and run each time you perform a commit, failing to complete the commit if an offending lint is found allowing you to make changes locally before pushing. -The pre-commit hooks can also be run adhoc without installing them by simply running {code}`pre-commit run --all-files` +The pre-commit hooks can also be run adhoc without installing them by simply running `pre-commit run --all-files` ## Guidelines for Separating Python and Rust Code @@ -82,9 +82,9 @@ Version 40 of `datafusion-python` introduced `python` wrappers around the `pyo3` Mostly, the `python` code is limited to pure wrappers with type hints and good docstrings, but there are a few reasons for when the code does more: -1. Trivial aliases like {py:func}`~datafusion.functions.array_append` and {py:func}`~datafusion.functions.list_append`. +1. Trivial aliases like [`array_append`][datafusion.functions.array_append] and [`list_append`][datafusion.functions.list_append]. 2. Simple type conversion, like from a `path` to a `string` of the path or from `number` to `lit(number)`. -3. The additional code makes an API **much** more pythonic, like we do for {py:func}`~datafusion.functions.named_struct` (see [source code](https://github.com/apache/datafusion-python/blob/a0913c728f5f323c1eb4913e614c9d996083e274/python/datafusion/functions.py#L1040-L1046)). +3. The additional code makes an API **much** more pythonic, like we do for [`named_struct`][datafusion.functions.named_struct] (see [source code](https://github.com/apache/datafusion-python/blob/a0913c728f5f323c1eb4913e614c9d996083e274/python/datafusion/functions.py#L1040-L1046)). ## Update Dependencies diff --git a/docs/source/index.ipynb b/docs/source/index.ipynb new file mode 100644 index 000000000..deb5a4304 --- /dev/null +++ b/docs/source/index.ipynb @@ -0,0 +1,73 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n# DataFusion in Python\n\nThis is a Python library that binds to [Apache Arrow](https://arrow.apache.org/) in-memory query engine [DataFusion](https://github.com/apache/datafusion).\n\nLike pyspark, it allows you to build a plan through SQL or a DataFrame API against in-memory data, parquet or CSV files, run it in a multi-threaded environment, and obtain the result back in Python.\n\nIt also allows you to use UDFs and UDAFs for complex operations.\n\nThe major advantage of this library over other execution engines is that this library achieves zero-copy between Python and its execution engine: there is no cost in using UDFs, UDAFs, and collecting the results to Python apart from having to lock the GIL when running those operations.\n\nIts query engine, DataFusion, is written in [Rust](https://www.rust-lang.org), which makes strong assumptions about thread safety and lack of memory leaks.\n\nTechnically, zero-copy is achieved via the [c data interface](https://arrow.apache.org/docs/format/CDataInterface.html).\n\n## Install\n\n```shell\npip install datafusion\n```\n\n## Example\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "\n", + "df = ctx.read_csv(\"pokemon.csv\")\n", + "\n", + "df.show()" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\n```{toctree}\n:hidden: true\n:maxdepth: 1\n\nuser-guide/index\ncontributor-guide/index\nAPI Reference \nlinks\n```\n" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/index.md b/docs/source/index.md deleted file mode 100644 index 8f92dd9be..000000000 --- a/docs/source/index.md +++ /dev/null @@ -1,63 +0,0 @@ - - -# DataFusion in Python - -This is a Python library that binds to [Apache Arrow](https://arrow.apache.org/) in-memory query engine [DataFusion](https://github.com/apache/datafusion). - -Like pyspark, it allows you to build a plan through SQL or a DataFrame API against in-memory data, parquet or CSV files, run it in a multi-threaded environment, and obtain the result back in Python. - -It also allows you to use UDFs and UDAFs for complex operations. - -The major advantage of this library over other execution engines is that this library achieves zero-copy between Python and its execution engine: there is no cost in using UDFs, UDAFs, and collecting the results to Python apart from having to lock the GIL when running those operations. - -Its query engine, DataFusion, is written in [Rust](https://www.rust-lang.org), which makes strong assumptions about thread safety and lack of memory leaks. - -Technically, zero-copy is achieved via the [c data interface](https://arrow.apache.org/docs/format/CDataInterface.html). - -## Install - -```shell -pip install datafusion -``` - -## Example - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - - ctx = SessionContext() - - df = ctx.read_csv("pokemon.csv") - - df.show() - -``` - -```{toctree} -:hidden: true -:maxdepth: 1 - -user-guide/index -contributor-guide/index -API Reference -links -``` diff --git a/docs/source/reference/catalog.md b/docs/source/reference/catalog.md new file mode 100644 index 000000000..04b6e6680 --- /dev/null +++ b/docs/source/reference/catalog.md @@ -0,0 +1,17 @@ +# Catalog + +## Catalog + +::: datafusion.catalog.Catalog + +## Table + +::: datafusion.catalog.Table + +## TableProviderFactory + +::: datafusion.catalog.TableProviderFactory + +## TableProviderFactoryExportable + +::: datafusion.catalog.TableProviderFactoryExportable diff --git a/docs/source/reference/common.md b/docs/source/reference/common.md new file mode 100644 index 000000000..d611d2db1 --- /dev/null +++ b/docs/source/reference/common.md @@ -0,0 +1,5 @@ +# Common + +## DFSchema + +::: datafusion.common.DFSchema diff --git a/docs/source/reference/context.md b/docs/source/reference/context.md new file mode 100644 index 000000000..7acd6164b --- /dev/null +++ b/docs/source/reference/context.md @@ -0,0 +1,21 @@ +# SessionContext + +## SessionContext + +::: datafusion.context.SessionContext + options: + filters: + - "!^_" + - "!^tables$" + +## SessionConfig + +::: datafusion.context.SessionConfig + +## SQLOptions + +::: datafusion.context.SQLOptions + +## RuntimeEnvBuilder + +::: datafusion.context.RuntimeEnvBuilder diff --git a/docs/source/reference/dataframe.md b/docs/source/reference/dataframe.md new file mode 100644 index 000000000..c278ec538 --- /dev/null +++ b/docs/source/reference/dataframe.md @@ -0,0 +1,33 @@ +# DataFrame + +## DataFrame + +::: datafusion.dataframe.DataFrame + options: + filters: + - "!^_" + - "!^unnest_column$" + +## DataFrameWriteOptions + +::: datafusion.dataframe.DataFrameWriteOptions + +## ParquetWriterOptions + +::: datafusion.dataframe.ParquetWriterOptions + +## ParquetColumnOptions + +::: datafusion.dataframe.ParquetColumnOptions + +## InsertOp + +::: datafusion.dataframe.InsertOp + +## ExplainFormat + +::: datafusion.dataframe.ExplainFormat + +## configure_formatter + +::: datafusion.dataframe_formatter.configure_formatter diff --git a/docs/source/reference/expr.md b/docs/source/reference/expr.md new file mode 100644 index 000000000..acea40c80 --- /dev/null +++ b/docs/source/reference/expr.md @@ -0,0 +1,25 @@ +# Expr + +## Expr + +::: datafusion.expr.Expr + +## WindowFrame + +::: datafusion.expr.WindowFrame + +## col + +::: datafusion.col.col + +## column + +::: datafusion.col.column + +## lit + +::: datafusion.lit + +## literal + +::: datafusion.literal diff --git a/docs/source/reference/functions.md b/docs/source/reference/functions.md new file mode 100644 index 000000000..17c3f0e1a --- /dev/null +++ b/docs/source/reference/functions.md @@ -0,0 +1,10 @@ +# Functions + +The `datafusion.functions` module provides 290+ scalar, aggregate, and window +functions. Import as: + +```python +from datafusion import functions as F +``` + +::: datafusion.functions diff --git a/docs/source/reference/index.md b/docs/source/reference/index.md new file mode 100644 index 000000000..0eab6977f --- /dev/null +++ b/docs/source/reference/index.md @@ -0,0 +1,65 @@ + + +# API Reference + +The public API of `datafusion` is exported from the top-level package. Every +symbol below is importable directly: `from datafusion import SessionContext`. + +| Symbol | Page | +|---|---| +| `Accumulator` | [User-Defined Functions](user_defined.md) | +| `AggregateUDF` | [User-Defined Functions](user_defined.md) | +| `Catalog` | [Catalog](catalog.md) | +| `CsvReadOptions` | [Options](options.md) | +| `DFSchema` | [Common](common.md) | +| `DataFrame` | [DataFrame](dataframe.md) | +| `DataFrameWriteOptions` | [DataFrame](dataframe.md) | +| `ExecutionPlan` | [Plan](plan.md) | +| `ExplainFormat` | [DataFrame](dataframe.md) | +| `Expr` | [Expr](expr.md) | +| `InsertOp` | [DataFrame](dataframe.md) | +| `LogicalPlan` | [Plan](plan.md) | +| `Metric` | [Plan](plan.md) | +| `MetricsSet` | [Plan](plan.md) | +| `ParquetColumnOptions` | [DataFrame](dataframe.md) | +| `ParquetWriterOptions` | [DataFrame](dataframe.md) | +| `RecordBatch` | [RecordBatch](record_batch.md) | +| `RecordBatchStream` | [RecordBatch](record_batch.md) | +| `RuntimeEnvBuilder` | [SessionContext](context.md) | +| `SQLOptions` | [SessionContext](context.md) | +| `ScalarUDF` | [User-Defined Functions](user_defined.md) | +| `SessionConfig` | [SessionContext](context.md) | +| `SessionContext` | [SessionContext](context.md) | +| `Table` | [Catalog](catalog.md) | +| `TableFunction` | [User-Defined Functions](user_defined.md) | +| `TableProviderFactory` | [Catalog](catalog.md) | +| `TableProviderFactoryExportable` | [Catalog](catalog.md) | +| `WindowFrame` | [Expr](expr.md) | +| `WindowUDF` | [User-Defined Functions](user_defined.md) | +| `col`, `column` | [Expr](expr.md) | +| `configure_formatter` | [DataFrame](dataframe.md) | +| `functions` | [Functions](functions.md) | +| `ipc` | [IPC](ipc.md) | +| `lit`, `literal` | [Expr](expr.md) | +| `object_store` | [Object Store](object_store.md) | +| `read_avro`, `read_csv`, `read_json`, `read_parquet` | [I/O](io.md) | +| `substrait` | [Substrait](substrait.md) | +| `udaf`, `udf`, `udtf`, `udwf` | [User-Defined Functions](user_defined.md) | +| `unparser` | [Unparser](unparser.md) | diff --git a/docs/source/reference/io.md b/docs/source/reference/io.md new file mode 100644 index 000000000..235d138e3 --- /dev/null +++ b/docs/source/reference/io.md @@ -0,0 +1,19 @@ +# I/O + +Top-level reader functions for loading data into a DataFrame. + +## read_csv + +::: datafusion.io.read_csv + +## read_parquet + +::: datafusion.io.read_parquet + +## read_json + +::: datafusion.io.read_json + +## read_avro + +::: datafusion.io.read_avro diff --git a/docs/source/reference/ipc.md b/docs/source/reference/ipc.md new file mode 100644 index 000000000..e8e1297bf --- /dev/null +++ b/docs/source/reference/ipc.md @@ -0,0 +1,3 @@ +# IPC + +::: datafusion.ipc diff --git a/docs/source/reference/object_store.md b/docs/source/reference/object_store.md new file mode 100644 index 000000000..2b5b6794d --- /dev/null +++ b/docs/source/reference/object_store.md @@ -0,0 +1,3 @@ +# Object Store + +::: datafusion.object_store diff --git a/docs/source/reference/options.md b/docs/source/reference/options.md new file mode 100644 index 000000000..36c9a2f6c --- /dev/null +++ b/docs/source/reference/options.md @@ -0,0 +1,5 @@ +# Options + +## CsvReadOptions + +::: datafusion.options.CsvReadOptions diff --git a/docs/source/reference/plan.md b/docs/source/reference/plan.md new file mode 100644 index 000000000..3460e5eb1 --- /dev/null +++ b/docs/source/reference/plan.md @@ -0,0 +1,17 @@ +# Plan + +## LogicalPlan + +::: datafusion.plan.LogicalPlan + +## ExecutionPlan + +::: datafusion.plan.ExecutionPlan + +## Metric + +::: datafusion.plan.Metric + +## MetricsSet + +::: datafusion.plan.MetricsSet diff --git a/docs/source/reference/record_batch.md b/docs/source/reference/record_batch.md new file mode 100644 index 000000000..727cdb464 --- /dev/null +++ b/docs/source/reference/record_batch.md @@ -0,0 +1,9 @@ +# RecordBatch + +## RecordBatch + +::: datafusion.record_batch.RecordBatch + +## RecordBatchStream + +::: datafusion.record_batch.RecordBatchStream diff --git a/docs/source/reference/substrait.md b/docs/source/reference/substrait.md new file mode 100644 index 000000000..f615bb463 --- /dev/null +++ b/docs/source/reference/substrait.md @@ -0,0 +1,10 @@ +# Substrait + +::: datafusion.substrait + options: + filters: + - "!^_" + - "!^serde$" + - "!^plan$" + - "!^producer$" + - "!^consumer$" diff --git a/docs/source/reference/unparser.md b/docs/source/reference/unparser.md new file mode 100644 index 000000000..c26b0d403 --- /dev/null +++ b/docs/source/reference/unparser.md @@ -0,0 +1,3 @@ +# Unparser + +::: datafusion.unparser diff --git a/docs/source/reference/user_defined.md b/docs/source/reference/user_defined.md new file mode 100644 index 000000000..966a2b0e0 --- /dev/null +++ b/docs/source/reference/user_defined.md @@ -0,0 +1,37 @@ +# User-Defined Functions + +## ScalarUDF + +::: datafusion.user_defined.ScalarUDF + +## AggregateUDF + +::: datafusion.user_defined.AggregateUDF + +## WindowUDF + +::: datafusion.user_defined.WindowUDF + +## TableFunction + +::: datafusion.user_defined.TableFunction + +## Accumulator + +::: datafusion.user_defined.Accumulator + +## udf + +::: datafusion.user_defined.udf + +## udaf + +::: datafusion.user_defined.udaf + +## udwf + +::: datafusion.user_defined.udwf + +## udtf + +::: datafusion.user_defined.udtf diff --git a/docs/source/user-guide/basics.ipynb b/docs/source/user-guide/basics.ipynb new file mode 100644 index 000000000..e2cc9603e --- /dev/null +++ b/docs/source/user-guide/basics.ipynb @@ -0,0 +1,81 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n\n# Concepts\n\nIn this section, we will cover a basic example to introduce a few key concepts. We will use the\n2021 Yellow Taxi Trip Records ([download](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet)),\nfrom the [TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page).\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "\n", + "df = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n", + "\n", + "df = df.select(\n", + " \"trip_distance\",\n", + " col(\"total_amount\").alias(\"total\"),\n", + " (f.round(lit(100.0) * col(\"tip_amount\") / col(\"total_amount\"), lit(1))).alias(\n", + " \"tip_percent\"\n", + " ),\n", + ")\n", + "\n", + "df.show()" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\n## Session Context\n\nThe first statement group creates a [`SessionContext`][datafusion.context.SessionContext].\n\n```python\n# create a context\nctx = datafusion.SessionContext()\n```\n\nA Session Context is the main interface for executing queries with DataFusion. It maintains the state\nof the connection between a user and an instance of the DataFusion engine. Additionally it provides\nthe following functionality:\n\n- Create a DataFrame from a data source.\n- Register a data source as a table that can be referenced from a SQL query.\n- Execute a SQL query\n\n## DataFrame\n\nThe second statement group creates a `DataFrame`,\n\n```python\n# Create a DataFrame from a file\ndf = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n```\n\nA DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html).\nDataFrames are typically created by calling a method on [`SessionContext`][datafusion.context.SessionContext], such as `read_csv`, and can then be modified by\ncalling the transformation methods, such as [`filter`][datafusion.dataframe.DataFrame.filter], [`select`][datafusion.dataframe.DataFrame.select], [`aggregate`][datafusion.dataframe.DataFrame.aggregate],\nand [`limit`][datafusion.dataframe.DataFrame.limit] to build up a query definition.\n\nFor more details on working with DataFrames, including visualization options and conversion to other formats, see [dataframe/index](dataframe/index.md).\n\n## Expressions\n\nThe third statement uses `Expressions` to build up a query definition. You can find\nexplanations for what the functions below do in the user documentation for\n[`col`][datafusion.col], [`lit`][datafusion.lit], [`round`][datafusion.functions.round],\nand [`alias`][datafusion.expr.Expr.alias].\n\n```python\ndf = df.select(\n \"trip_distance\",\n col(\"total_amount\").alias(\"total\"),\n (f.round(lit(100.0) * col(\"tip_amount\") / col(\"total_amount\"), lit(1))).alias(\"tip_percent\"),\n)\n```\n\nFinally the [`show`][datafusion.dataframe.DataFrame.show] method converts the logical plan\nrepresented by the DataFrame into a physical plan and execute it, collecting all results and\ndisplaying them to the user. It is important to note that DataFusion performs lazy evaluation\nof the DataFrame. Until you call a method such as [`show`][datafusion.dataframe.DataFrame.show]\nor [`collect`][datafusion.dataframe.DataFrame.collect], DataFusion will not perform the query.\n" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/basics.md b/docs/source/user-guide/basics.md deleted file mode 100644 index 42d8432d9..000000000 --- a/docs/source/user-guide/basics.md +++ /dev/null @@ -1,98 +0,0 @@ - - -(user_guide_concepts)= - -# Concepts - -In this section, we will cover a basic example to introduce a few key concepts. We will use the -2021 Yellow Taxi Trip Records ([download](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet)), -from the [TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page). - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col, lit, functions as f - - ctx = SessionContext() - - df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") - - df = df.select( - "trip_distance", - col("total_amount").alias("total"), - (f.round(lit(100.0) * col("tip_amount") / col("total_amount"), lit(1))).alias("tip_percent"), - ) - - df.show() -``` - -## Session Context - -The first statement group creates a {py:class}`~datafusion.context.SessionContext`. - -```python -# create a context -ctx = datafusion.SessionContext() -``` - -A Session Context is the main interface for executing queries with DataFusion. It maintains the state -of the connection between a user and an instance of the DataFusion engine. Additionally it provides -the following functionality: - -- Create a DataFrame from a data source. -- Register a data source as a table that can be referenced from a SQL query. -- Execute a SQL query - -## DataFrame - -The second statement group creates a {code}`DataFrame`, - -```python -# Create a DataFrame from a file -df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") -``` - -A DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html). -DataFrames are typically created by calling a method on {py:class}`~datafusion.context.SessionContext`, such as {code}`read_csv`, and can then be modified by -calling the transformation methods, such as {py:func}`~datafusion.dataframe.DataFrame.filter`, {py:func}`~datafusion.dataframe.DataFrame.select`, {py:func}`~datafusion.dataframe.DataFrame.aggregate`, -and {py:func}`~datafusion.dataframe.DataFrame.limit` to build up a query definition. - -For more details on working with DataFrames, including visualization options and conversion to other formats, see {doc}`dataframe/index`. - -## Expressions - -The third statement uses {code}`Expressions` to build up a query definition. You can find -explanations for what the functions below do in the user documentation for -{py:func}`~datafusion.col`, {py:func}`~datafusion.lit`, {py:func}`~datafusion.functions.round`, -and {py:func}`~datafusion.expr.Expr.alias`. - -```python -df = df.select( - "trip_distance", - col("total_amount").alias("total"), - (f.round(lit(100.0) * col("tip_amount") / col("total_amount"), lit(1))).alias("tip_percent"), -) -``` - -Finally the {py:func}`~datafusion.dataframe.DataFrame.show` method converts the logical plan -represented by the DataFrame into a physical plan and execute it, collecting all results and -displaying them to the user. It is important to note that DataFusion performs lazy evaluation -of the DataFrame. Until you call a method such as {py:func}`~datafusion.dataframe.DataFrame.show` -or {py:func}`~datafusion.dataframe.DataFrame.collect`, DataFusion will not perform the query. diff --git a/docs/source/user-guide/common-operations/aggregations.ipynb b/docs/source/user-guide/common-operations/aggregations.ipynb new file mode 100644 index 000000000..c16c4ff6e --- /dev/null +++ b/docs/source/user-guide/common-operations/aggregations.ipynb @@ -0,0 +1,462 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n\n# Aggregation\n\nAn aggregate or aggregation is a function where the values of multiple rows are processed together\nto form a single summary value. For performing an aggregation, DataFusion provides the\n[`aggregate`][datafusion.dataframe.DataFrame.aggregate]\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "df = ctx.read_csv(\"pokemon.csv\")\n", + "\n", + "col_type_1 = col('\"Type 1\"')\n", + "col_type_2 = col('\"Type 2\"')\n", + "col_speed = col('\"Speed\"')\n", + "col_attack = col('\"Attack\"')\n", + "\n", + "df.aggregate(\n", + " [col_type_1],\n", + " [\n", + " f.approx_distinct(col_speed).alias(\"Count\"),\n", + " f.approx_median(col_speed).alias(\"Median Speed\"),\n", + " f.approx_percentile_cont(col_speed, 0.9).alias(\"90% Speed\"),\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\nWhen `group_by` is `None` or an empty list, the aggregation is done over the whole\n{class}`.DataFrame`. For grouping the `group_by` list must contain at least one column.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "df.aggregate(\n", + " [col_type_1],\n", + " [\n", + " f.max(col_speed).alias(\"Max Speed\"),\n", + " f.avg(col_speed).alias(\"Avg Speed\"),\n", + " f.min(col_speed).alias(\"Min Speed\"),\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\nMore than one column can be used for grouping\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "df.aggregate(\n", + " [col_type_1, col_type_2],\n", + " [\n", + " f.max(col_speed).alias(\"Max Speed\"),\n", + " f.avg(col_speed).alias(\"Avg Speed\"),\n", + " f.min(col_speed).alias(\"Min Speed\"),\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\n## Setting Parameters\n\nEach of the built in aggregate functions provides arguments for the parameters that affect their\noperation. These can also be overridden using the builder approach to setting any of the following\nparameters. When you use the builder, you must call `build()` to finish. For example, these two\nexpressions are equivalent.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "first_1 = f.first_value(col(\"a\"), order_by=[col(\"a\")])\n", + "first_2 = f.first_value(col(\"a\")).order_by(col(\"a\")).build()" + ] + }, + { + "cell_type": "markdown", + "id": "7cdc8c89c7104fffa095e18ddfef8986", + "metadata": {}, + "source": "\n### Ordering\n\nYou can control the order in which rows are processed by window functions by providing\na list of `order_by` functions for the `order_by` parameter. In the following example, we\nsort the Pokemon by their attack in increasing order and take the first value, which gives us the\nPokemon with the smallest attack value in each `Type 1`.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b118ea5561624da68c537baed56e602f", + "metadata": {}, + "outputs": [], + "source": [ + "df.aggregate(\n", + " [col('\"Type 1\"')],\n", + " [\n", + " f.first_value(\n", + " col('\"Name\"'), order_by=[col('\"Attack\"').sort(ascending=True)]\n", + " ).alias(\"Smallest Attack\")\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "938c804e27f84196a10c8828c723f798", + "metadata": {}, + "source": "\n### Distinct\n\nWhen you set the parameter `distinct` to `True`, then unique values will only be evaluated one\ntime each. Suppose we want to create an array of all of the `Type 2` for each `Type 1` of our\nPokemon set. Since there will be many entries of `Type 2` we only one each distinct value.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "504fb2a444614c0babb325280ed9130a", + "metadata": {}, + "outputs": [], + "source": [ + "df.aggregate(\n", + " [col_type_1], [f.array_agg(col_type_2, distinct=True).alias(\"Type 2 List\")]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "59bbdb311c014d738909a11f9e486628", + "metadata": {}, + "source": "\nIn the output of the above we can see that there are some `Type 1` for which the `Type 2` entry\nis `null`. In reality, we probably want to filter those out. We can do this in two ways. First,\nwe can filter DataFrame rows that have no `Type 2`. If we do this, we might have some `Type 1`\nentries entirely removed. The second is we can use the `filter` argument described below.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b43b363d81ae4b689946ece5c682cd59", + "metadata": {}, + "outputs": [], + "source": [ + "df.filter(col_type_2.is_not_null()).aggregate(\n", + " [col_type_1], [f.array_agg(col_type_2, distinct=True).alias(\"Type 2 List\")]\n", + ")\n", + "\n", + "df.aggregate(\n", + " [col_type_1],\n", + " [\n", + " f.array_agg(col_type_2, distinct=True, filter=col_type_2.is_not_null()).alias(\n", + " \"Type 2 List\"\n", + " )\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "8a65eabff63a45729fe45fb5ade58bdc", + "metadata": {}, + "source": "\nWhich approach you take should depend on your use case.\n\n### Null Treatment\n\nThis option allows you to either respect or ignore null values.\n\nOne common usage for handling nulls is the case where you want to find the first value within a\npartition. By setting the null treatment to ignore nulls, we can find the first non-null value\nin our partition.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c3933fab20d04ec698c2621248eb3be0", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion.common import NullTreatment\n", + "\n", + "df.aggregate(\n", + " [col_type_1],\n", + " [\n", + " f.first_value(\n", + " col_type_2,\n", + " order_by=[col_attack],\n", + " null_treatment=NullTreatment.RESPECT_NULLS,\n", + " ).alias(\"Lowest Attack Type 2\")\n", + " ],\n", + ")\n", + "\n", + "df.aggregate(\n", + " [col_type_1],\n", + " [\n", + " f.first_value(\n", + " col_type_2, order_by=[col_attack], null_treatment=NullTreatment.IGNORE_NULLS\n", + " ).alias(\"Lowest Attack Type 2\")\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "4dd4641cc4064e0191573fe9c69df29b", + "metadata": {}, + "source": "\n### Filter\n\nUsing the filter option is useful for filtering results to include in the aggregate function. It can\nbe seen in the example above on how this can be useful to only filter rows evaluated by the\naggregate function without filtering rows from the entire DataFrame.\n\nFilter takes a single expression.\n\nSuppose we want to find the speed values for only Pokemon that have low Attack values.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8309879909854d7188b41380fd92a7c3", + "metadata": {}, + "outputs": [], + "source": [ + "df.aggregate(\n", + " [col_type_1],\n", + " [\n", + " f.avg(col_speed).alias(\"Avg Speed All\"),\n", + " f.avg(col_speed, filter=col_attack < lit(50)).alias(\"Avg Speed Low Attack\"),\n", + " ],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3ed186c9a28b402fb0bc4494df01f08d", + "metadata": {}, + "source": "\n### Comparing subsets within a group\n\nSometimes you need to compare the full membership of a group against a\nsubset that meets some condition — for example, \"which groups have at least\none failure, but not every member failed?\". The `filter` argument on an\naggregate restricts the rows that contribute to *that* aggregate without\ndropping the group, so a single pass can produce both the full set and the\nfiltered subset side by side. Pairing\n[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and\n`filter=` is a compact way to express this: collect the distinct values\nof the group, collect the distinct values that satisfy the condition, then\ncompare the two arrays.\n\nSuppose each row records a line item with the supplier that fulfilled it and\na flag for whether that supplier met the commit date. We want to identify\n*partially failed* orders — orders where at least one supplier failed but\nnot every supplier failed:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb1e1581032b452c9409d6c6813c49d1", + "metadata": {}, + "outputs": [], + "source": [ + "orders_df = ctx.from_pydict(\n", + " {\n", + " \"order_id\": [1, 1, 1, 2, 2, 3, 4, 4],\n", + " \"supplier_id\": [100, 101, 102, 200, 201, 300, 400, 401],\n", + " \"failed\": [False, True, False, False, False, True, True, True],\n", + " },\n", + ")\n", + "\n", + "grouped = orders_df.aggregate(\n", + " [col(\"order_id\")],\n", + " [\n", + " f.array_agg(col(\"supplier_id\"), distinct=True).alias(\"all_suppliers\"),\n", + " f.array_agg(\n", + " col(\"supplier_id\"),\n", + " filter=col(\"failed\"),\n", + " distinct=True,\n", + " ).alias(\"failed_suppliers\"),\n", + " ],\n", + ")\n", + "\n", + "grouped.filter(\n", + " (f.array_length(col(\"failed_suppliers\")) > lit(0))\n", + " & (f.array_length(col(\"failed_suppliers\")) < f.array_length(col(\"all_suppliers\")))\n", + ").select(col(\"order_id\"), col(\"failed_suppliers\"))" + ] + }, + { + "cell_type": "markdown", + "id": "379cbbc1e968416e875cc15c1202d7eb", + "metadata": {}, + "source": "\nOrder 1 is partial (one of three suppliers failed). Order 2 is excluded\nbecause no supplier failed, order 3 because its only supplier failed, and\norder 4 because both of its suppliers failed.\n\n## Grouping Sets\n\nThe default style of aggregation produces one row per group. Sometimes you want a single query to\nproduce rows at multiple levels of detail — for example, totals per type *and* an overall grand\ntotal, or subtotals for every combination of two columns plus the individual column totals. Writing\nseparate queries and concatenating them is tedious and runs the data multiple times. Grouping sets\nsolve this by letting you specify several grouping levels in one pass.\n\nDataFusion supports three grouping set styles through the\n[`GroupingSet`][datafusion.expr.GroupingSet] class:\n\n- [`rollup`][datafusion.expr.GroupingSet.rollup] — hierarchical subtotals, like a drill-down report\n- [`cube`][datafusion.expr.GroupingSet.cube] — every possible subtotal combination, like a pivot table\n- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] — explicitly list exactly which grouping levels you want\n\nBecause result rows come from different grouping levels, a column that is *not* part of a\nparticular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to\ndistinguish a real `null` in the data from one that means \"this column was aggregated across.\"\nIt returns `0` when the column is a grouping key for that row, and `1` when it is not.\n\n### Rollup\n\n[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces\ngrouping sets `(a, b)`, `(a)`, and `()` — like nested subtotals in a report. This is useful\nwhen your columns have a natural hierarchy, such as region → city or type → subtype.\n\nSuppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With\nthe default aggregation style we would need two separate queries. With `rollup` we get it all at\nonce:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "277c27b1587741f2af2001be3712ef0d", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion.expr import GroupingSet\n", + "\n", + "df.aggregate(\n", + " [GroupingSet.rollup(col_type_1)],\n", + " [\n", + " f.count(col_speed).alias(\"Count\"),\n", + " f.avg(col_speed).alias(\"Avg Speed\"),\n", + " f.max(col_speed).alias(\"Max Speed\"),\n", + " ],\n", + ").sort(col_type_1.sort(ascending=True, nulls_first=True))" + ] + }, + { + "cell_type": "markdown", + "id": "db7b79bc585a40fcaf58bf750017e135", + "metadata": {}, + "source": "\nThe first row — where `Type 1` is `null` — is the grand total across all types. But how do you\ntell a grand-total `null` apart from a Pokemon that genuinely has no type? The\n[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key\nfor that row and `1` when it is aggregated across.\n\n!!! note\n\n Due to an upstream DataFusion limitation\n ([apache/datafusion#21411](https://github.com/apache/datafusion/issues/21411)),\n `.alias()` cannot be applied directly to a `grouping()` expression — it will raise an\n error at execution time. Instead, use\n [`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] on the result DataFrame to\n give the column a readable name. Once the upstream issue is resolved, you will be able to\n use `.alias()` directly and the workaround below will no longer be necessary.\n\nThe raw column name generated by `grouping()` contains internal identifiers, so we use\n[`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] to clean it up:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "916684f9a58a4a2aa5f864670399430d", + "metadata": {}, + "outputs": [], + "source": [ + "result = df.aggregate(\n", + " [GroupingSet.rollup(col_type_1)],\n", + " [\n", + " f.count(col_speed).alias(\"Count\"),\n", + " f.avg(col_speed).alias(\"Avg Speed\"),\n", + " f.grouping(col_type_1),\n", + " ],\n", + ")\n", + "for field in result.schema():\n", + " if field.name.startswith(\"grouping(\"):\n", + " result = result.with_column_renamed(field.name, \"Is Total\")\n", + "result.sort(col_type_1.sort(ascending=True, nulls_first=True))" + ] + }, + { + "cell_type": "markdown", + "id": "1671c31a24314836a5b85d7ef7fbf015", + "metadata": {}, + "source": "\nWith two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces:\n\n- one row per `(Type 1, Type 2)` pair — the most detailed level\n- one row per `Type 1` — subtotals\n- one grand total row\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "33b0902fd34d4ace834912fa1002cf8e", + "metadata": {}, + "outputs": [], + "source": [ + "df.aggregate(\n", + " [GroupingSet.rollup(col_type_1, col_type_2)],\n", + " [f.count(col_speed).alias(\"Count\"), f.avg(col_speed).alias(\"Avg Speed\")],\n", + ").sort(\n", + " col_type_1.sort(ascending=True, nulls_first=True),\n", + " col_type_2.sort(ascending=True, nulls_first=True),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f6fa52606d8c4a75a9b52967216f8f3f", + "metadata": {}, + "source": "\n### Cube\n\n[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)`\nproduces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` — one more than `rollup` because\nit also includes `(b)` alone. This is useful when neither column is \"above\" the other in a\nhierarchy and you want all cross-tabulations.\n\nFor our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair,\nby `Type 1` alone, by `Type 2` alone, and a grand total — all in one query:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f5a1fa73e5044315a093ec459c9be902", + "metadata": {}, + "outputs": [], + "source": [ + "df.aggregate(\n", + " [GroupingSet.cube(col_type_1, col_type_2)],\n", + " [f.count(col_speed).alias(\"Count\"), f.avg(col_speed).alias(\"Avg Speed\")],\n", + ").sort(\n", + " col_type_1.sort(ascending=True, nulls_first=True),\n", + " col_type_2.sort(ascending=True, nulls_first=True),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "cdf66aed5cc84ca1b48e60bad68798a8", + "metadata": {}, + "source": "\nCompared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but\n`Type 2` has a value — those are the per-`Type 2` subtotals that `rollup` does not include.\n\n### Explicit Grouping Sets\n\n[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels\nyou need when `rollup` or `cube` would produce too many or too few. Each argument is a list of\ncolumns forming one grouping set.\n\nFor example, if we want only the per-`Type 1` totals and per-`Type 2` totals — but *not* the\nfull `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "28d3efd5258a48a79c179ea5c6759f01", + "metadata": {}, + "outputs": [], + "source": [ + "df.aggregate(\n", + " [GroupingSet.grouping_sets([col_type_1], [col_type_2])],\n", + " [f.count(col_speed).alias(\"Count\"), f.avg(col_speed).alias(\"Avg Speed\")],\n", + ").sort(\n", + " col_type_1.sort(ascending=True, nulls_first=True),\n", + " col_type_2.sort(ascending=True, nulls_first=True),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3f9bc0b9dd2c44919cc8dcca39b469f8", + "metadata": {}, + "source": "\nEach row belongs to exactly one grouping level. The [`grouping`][datafusion.functions.grouping]\nfunction tells you which level each row comes from:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0e382214b5f147d187d36a2058b9c724", + "metadata": {}, + "outputs": [], + "source": [ + "result = df.aggregate(\n", + " [GroupingSet.grouping_sets([col_type_1], [col_type_2])],\n", + " [\n", + " f.count(col_speed).alias(\"Count\"),\n", + " f.avg(col_speed).alias(\"Avg Speed\"),\n", + " f.grouping(col_type_1),\n", + " f.grouping(col_type_2),\n", + " ],\n", + ")\n", + "for field in result.schema():\n", + " if field.name.startswith(\"grouping(\"):\n", + " clean = field.name.split(\".\")[-1].rstrip(\")\")\n", + " result = result.with_column_renamed(field.name, f\"grouping({clean})\")\n", + "result.sort(\n", + " col_type_1.sort(ascending=True, nulls_first=True),\n", + " col_type_2.sort(ascending=True, nulls_first=True),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "5b09d5ef5b5e4bb6ab9b829b10b6a29f", + "metadata": {}, + "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions —\n the accumulator class is captured by value via {mod}`cloudpickle`,\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/common-operations/aggregations.md b/docs/source/user-guide/common-operations/aggregations.md deleted file mode 100644 index e578a3fa5..000000000 --- a/docs/source/user-guide/common-operations/aggregations.md +++ /dev/null @@ -1,474 +0,0 @@ - - -(aggregation)= - -# Aggregation - -An aggregate or aggregation is a function where the values of multiple rows are processed together -to form a single summary value. For performing an aggregation, DataFusion provides the -{py:func}`~datafusion.dataframe.DataFrame.aggregate` - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col, lit, functions as f - - ctx = SessionContext() - df = ctx.read_csv("pokemon.csv") - - col_type_1 = col('"Type 1"') - col_type_2 = col('"Type 2"') - col_speed = col('"Speed"') - col_attack = col('"Attack"') - - df.aggregate([col_type_1], [ - f.approx_distinct(col_speed).alias("Count"), - f.approx_median(col_speed).alias("Median Speed"), - f.approx_percentile_cont(col_speed, 0.9).alias("90% Speed")]) -``` - -When {code}`group_by` is {code}`None` or an empty list, the aggregation is done over the whole -{class}`.DataFrame`. For grouping the {code}`group_by` list must contain at least one column. - -```{eval-rst} -.. ipython:: python - - df.aggregate([col_type_1], [ - f.max(col_speed).alias("Max Speed"), - f.avg(col_speed).alias("Avg Speed"), - f.min(col_speed).alias("Min Speed")]) -``` - -More than one column can be used for grouping - -```{eval-rst} -.. ipython:: python - - df.aggregate([col_type_1, col_type_2], [ - f.max(col_speed).alias("Max Speed"), - f.avg(col_speed).alias("Avg Speed"), - f.min(col_speed).alias("Min Speed")]) - - -``` - -## Setting Parameters - -Each of the built in aggregate functions provides arguments for the parameters that affect their -operation. These can also be overridden using the builder approach to setting any of the following -parameters. When you use the builder, you must call `build()` to finish. For example, these two -expressions are equivalent. - -```{eval-rst} -.. ipython:: python - - first_1 = f.first_value(col("a"), order_by=[col("a")]) - first_2 = f.first_value(col("a")).order_by(col("a")).build() -``` - -### Ordering - -You can control the order in which rows are processed by window functions by providing -a list of `order_by` functions for the `order_by` parameter. In the following example, we -sort the Pokemon by their attack in increasing order and take the first value, which gives us the -Pokemon with the smallest attack value in each `Type 1`. - -```{eval-rst} -.. ipython:: python - - df.aggregate( - [col('"Type 1"')], - [f.first_value( - col('"Name"'), - order_by=[col('"Attack"').sort(ascending=True)] - ).alias("Smallest Attack") - ]) -``` - -### Distinct - -When you set the parameter `distinct` to `True`, then unique values will only be evaluated one -time each. Suppose we want to create an array of all of the `Type 2` for each `Type 1` of our -Pokemon set. Since there will be many entries of `Type 2` we only one each distinct value. - -```{eval-rst} -.. ipython:: python - - df.aggregate([col_type_1], [f.array_agg(col_type_2, distinct=True).alias("Type 2 List")]) -``` - -In the output of the above we can see that there are some `Type 1` for which the `Type 2` entry -is `null`. In reality, we probably want to filter those out. We can do this in two ways. First, -we can filter DataFrame rows that have no `Type 2`. If we do this, we might have some `Type 1` -entries entirely removed. The second is we can use the `filter` argument described below. - -```{eval-rst} -.. ipython:: python - - df.filter(col_type_2.is_not_null()).aggregate([col_type_1], [f.array_agg(col_type_2, distinct=True).alias("Type 2 List")]) - - df.aggregate([col_type_1], [f.array_agg(col_type_2, distinct=True, filter=col_type_2.is_not_null()).alias("Type 2 List")]) -``` - -Which approach you take should depend on your use case. - -### Null Treatment - -This option allows you to either respect or ignore null values. - -One common usage for handling nulls is the case where you want to find the first value within a -partition. By setting the null treatment to ignore nulls, we can find the first non-null value -in our partition. - -```{eval-rst} -.. ipython:: python - - from datafusion.common import NullTreatment - - df.aggregate([col_type_1], [ - f.first_value( - col_type_2, - order_by=[col_attack], - null_treatment=NullTreatment.RESPECT_NULLS - ).alias("Lowest Attack Type 2")]) - - df.aggregate([col_type_1], [ - f.first_value( - col_type_2, - order_by=[col_attack], - null_treatment=NullTreatment.IGNORE_NULLS - ).alias("Lowest Attack Type 2")]) -``` - -### Filter - -Using the filter option is useful for filtering results to include in the aggregate function. It can -be seen in the example above on how this can be useful to only filter rows evaluated by the -aggregate function without filtering rows from the entire DataFrame. - -Filter takes a single expression. - -Suppose we want to find the speed values for only Pokemon that have low Attack values. - -```{eval-rst} -.. ipython:: python - - df.aggregate([col_type_1], [ - f.avg(col_speed).alias("Avg Speed All"), - f.avg(col_speed, filter=col_attack < lit(50)).alias("Avg Speed Low Attack")]) - -``` - -### Comparing subsets within a group - -Sometimes you need to compare the full membership of a group against a -subset that meets some condition — for example, "which groups have at least -one failure, but not every member failed?". The `filter` argument on an -aggregate restricts the rows that contribute to *that* aggregate without -dropping the group, so a single pass can produce both the full set and the -filtered subset side by side. Pairing -{py:func}`~datafusion.functions.array_agg` with `distinct=True` and -`filter=` is a compact way to express this: collect the distinct values -of the group, collect the distinct values that satisfy the condition, then -compare the two arrays. - -Suppose each row records a line item with the supplier that fulfilled it and -a flag for whether that supplier met the commit date. We want to identify -*partially failed* orders — orders where at least one supplier failed but -not every supplier failed: - -```{eval-rst} -.. ipython:: python - - orders_df = ctx.from_pydict( - { - "order_id": [1, 1, 1, 2, 2, 3, 4, 4], - "supplier_id": [100, 101, 102, 200, 201, 300, 400, 401], - "failed": [False, True, False, False, False, True, True, True], - }, - ) - - grouped = orders_df.aggregate( - [col("order_id")], - [ - f.array_agg(col("supplier_id"), distinct=True).alias("all_suppliers"), - f.array_agg( - col("supplier_id"), - filter=col("failed"), - distinct=True, - ).alias("failed_suppliers"), - ], - ) - - grouped.filter( - (f.array_length(col("failed_suppliers")) > lit(0)) - & (f.array_length(col("failed_suppliers")) < f.array_length(col("all_suppliers"))) - ).select(col("order_id"), col("failed_suppliers")) -``` - -Order 1 is partial (one of three suppliers failed). Order 2 is excluded -because no supplier failed, order 3 because its only supplier failed, and -order 4 because both of its suppliers failed. - -## Grouping Sets - -The default style of aggregation produces one row per group. Sometimes you want a single query to -produce rows at multiple levels of detail — for example, totals per type *and* an overall grand -total, or subtotals for every combination of two columns plus the individual column totals. Writing -separate queries and concatenating them is tedious and runs the data multiple times. Grouping sets -solve this by letting you specify several grouping levels in one pass. - -DataFusion supports three grouping set styles through the -{py:class}`~datafusion.expr.GroupingSet` class: - -- {py:meth}`~datafusion.expr.GroupingSet.rollup` — hierarchical subtotals, like a drill-down report -- {py:meth}`~datafusion.expr.GroupingSet.cube` — every possible subtotal combination, like a pivot table -- {py:meth}`~datafusion.expr.GroupingSet.grouping_sets` — explicitly list exactly which grouping levels you want - -Because result rows come from different grouping levels, a column that is *not* part of a -particular level will be `null` in that row. Use {py:func}`~datafusion.functions.grouping` to -distinguish a real `null` in the data from one that means "this column was aggregated across." -It returns `0` when the column is a grouping key for that row, and `1` when it is not. - -### Rollup - -{py:meth}`~datafusion.expr.GroupingSet.rollup` creates a hierarchy. `rollup(a, b)` produces -grouping sets `(a, b)`, `(a)`, and `()` — like nested subtotals in a report. This is useful -when your columns have a natural hierarchy, such as region → city or type → subtype. - -Suppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With -the default aggregation style we would need two separate queries. With `rollup` we get it all at -once: - -```{eval-rst} -.. ipython:: python - - from datafusion.expr import GroupingSet - - df.aggregate( - [GroupingSet.rollup(col_type_1)], - [f.count(col_speed).alias("Count"), - f.avg(col_speed).alias("Avg Speed"), - f.max(col_speed).alias("Max Speed")] - ).sort(col_type_1.sort(ascending=True, nulls_first=True)) -``` - -The first row — where `Type 1` is `null` — is the grand total across all types. But how do you -tell a grand-total `null` apart from a Pokemon that genuinely has no type? The -{py:func}`~datafusion.functions.grouping` function returns `0` when the column is a grouping key -for that row and `1` when it is aggregated across. - -:::{note} -Due to an upstream DataFusion limitation -([apache/datafusion#21411](https://github.com/apache/datafusion/issues/21411)), -`.alias()` cannot be applied directly to a `grouping()` expression — it will raise an -error at execution time. Instead, use -{py:meth}`~datafusion.dataframe.DataFrame.with_column_renamed` on the result DataFrame to -give the column a readable name. Once the upstream issue is resolved, you will be able to -use `.alias()` directly and the workaround below will no longer be necessary. -::: - -The raw column name generated by `grouping()` contains internal identifiers, so we use -{py:meth}`~datafusion.dataframe.DataFrame.with_column_renamed` to clean it up: - -```{eval-rst} -.. ipython:: python - - result = df.aggregate( - [GroupingSet.rollup(col_type_1)], - [f.count(col_speed).alias("Count"), - f.avg(col_speed).alias("Avg Speed"), - f.grouping(col_type_1)] - ) - for field in result.schema(): - if field.name.startswith("grouping("): - result = result.with_column_renamed(field.name, "Is Total") - result.sort(col_type_1.sort(ascending=True, nulls_first=True)) -``` - -With two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces: - -- one row per `(Type 1, Type 2)` pair — the most detailed level -- one row per `Type 1` — subtotals -- one grand total row - -```{eval-rst} -.. ipython:: python - - df.aggregate( - [GroupingSet.rollup(col_type_1, col_type_2)], - [f.count(col_speed).alias("Count"), - f.avg(col_speed).alias("Avg Speed")] - ).sort( - col_type_1.sort(ascending=True, nulls_first=True), - col_type_2.sort(ascending=True, nulls_first=True) - ) -``` - -### Cube - -{py:meth}`~datafusion.expr.GroupingSet.cube` produces every possible subset. `cube(a, b)` -produces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` — one more than `rollup` because -it also includes `(b)` alone. This is useful when neither column is "above" the other in a -hierarchy and you want all cross-tabulations. - -For our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair, -by `Type 1` alone, by `Type 2` alone, and a grand total — all in one query: - -```{eval-rst} -.. ipython:: python - - df.aggregate( - [GroupingSet.cube(col_type_1, col_type_2)], - [f.count(col_speed).alias("Count"), - f.avg(col_speed).alias("Avg Speed")] - ).sort( - col_type_1.sort(ascending=True, nulls_first=True), - col_type_2.sort(ascending=True, nulls_first=True) - ) -``` - -Compared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but -`Type 2` has a value — those are the per-`Type 2` subtotals that `rollup` does not include. - -### Explicit Grouping Sets - -{py:meth}`~datafusion.expr.GroupingSet.grouping_sets` lets you list exactly which grouping levels -you need when `rollup` or `cube` would produce too many or too few. Each argument is a list of -columns forming one grouping set. - -For example, if we want only the per-`Type 1` totals and per-`Type 2` totals — but *not* the -full `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that: - -```{eval-rst} -.. ipython:: python - - df.aggregate( - [GroupingSet.grouping_sets([col_type_1], [col_type_2])], - [f.count(col_speed).alias("Count"), - f.avg(col_speed).alias("Avg Speed")] - ).sort( - col_type_1.sort(ascending=True, nulls_first=True), - col_type_2.sort(ascending=True, nulls_first=True) - ) -``` - -Each row belongs to exactly one grouping level. The {py:func}`~datafusion.functions.grouping` -function tells you which level each row comes from: - -```{eval-rst} -.. ipython:: python - - result = df.aggregate( - [GroupingSet.grouping_sets([col_type_1], [col_type_2])], - [f.count(col_speed).alias("Count"), - f.avg(col_speed).alias("Avg Speed"), - f.grouping(col_type_1), - f.grouping(col_type_2)] - ) - for field in result.schema(): - if field.name.startswith("grouping("): - clean = field.name.split(".")[-1].rstrip(")") - result = result.with_column_renamed(field.name, f"grouping({clean})") - result.sort( - col_type_1.sort(ascending=True, nulls_first=True), - col_type_2.sort(ascending=True, nulls_first=True) - ) -``` - -Where `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`). -Where `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`). - -## Aggregate Functions - -The available aggregate functions are: - -01. Comparison Functions - : - {py:func}`datafusion.functions.min` - - {py:func}`datafusion.functions.max` -02. Math Functions - : - {py:func}`datafusion.functions.sum` - - {py:func}`datafusion.functions.avg` - - {py:func}`datafusion.functions.median` -03. Array Functions - : - {py:func}`datafusion.functions.array_agg` -04. Logical Functions - : - {py:func}`datafusion.functions.bit_and` - - {py:func}`datafusion.functions.bit_or` - - {py:func}`datafusion.functions.bit_xor` - - {py:func}`datafusion.functions.bool_and` - - {py:func}`datafusion.functions.bool_or` -05. Statistical Functions - : - {py:func}`datafusion.functions.count` - - {py:func}`datafusion.functions.corr` - - {py:func}`datafusion.functions.covar_samp` - - {py:func}`datafusion.functions.covar_pop` - - {py:func}`datafusion.functions.stddev` - - {py:func}`datafusion.functions.stddev_pop` - - {py:func}`datafusion.functions.var_samp` - - {py:func}`datafusion.functions.var_pop` - - {py:func}`datafusion.functions.var_population` -06. Linear Regression Functions - : - {py:func}`datafusion.functions.regr_count` - - {py:func}`datafusion.functions.regr_slope` - - {py:func}`datafusion.functions.regr_intercept` - - {py:func}`datafusion.functions.regr_r2` - - {py:func}`datafusion.functions.regr_avgx` - - {py:func}`datafusion.functions.regr_avgy` - - {py:func}`datafusion.functions.regr_sxx` - - {py:func}`datafusion.functions.regr_syy` - - {py:func}`datafusion.functions.regr_slope` -07. Positional Functions - : - {py:func}`datafusion.functions.first_value` - - {py:func}`datafusion.functions.last_value` - - {py:func}`datafusion.functions.nth_value` -08. String Functions - : - {py:func}`datafusion.functions.string_agg` -09. Percentile Functions - : - {py:func}`datafusion.functions.percentile_cont` - - {py:func}`datafusion.functions.quantile_cont` - - {py:func}`datafusion.functions.approx_distinct` - - {py:func}`datafusion.functions.approx_median` - - {py:func}`datafusion.functions.approx_percentile_cont` - - {py:func}`datafusion.functions.approx_percentile_cont_with_weight` -10. Grouping Set Functions - \- {py:func}`datafusion.functions.grouping` - \- {py:meth}`datafusion.expr.GroupingSet.rollup` - \- {py:meth}`datafusion.expr.GroupingSet.cube` - \- {py:meth}`datafusion.expr.GroupingSet.grouping_sets` - -## User-Defined Aggregate Functions - -You can ship custom aggregations to the engine by subclassing -{py:class}`~datafusion.user_defined.Accumulator` and registering it via -{py:func}`~datafusion.udaf`. See {py:mod}`datafusion.user_defined` for -the accumulator interface and worked examples. - -:::{note} -Serialization - -Python aggregate UDFs travel inline inside pickled or -{py:meth}`~datafusion.expr.Expr.to_bytes`-serialized expressions — -the accumulator class is captured by value via {mod}`cloudpickle`, -so worker processes do not need to pre-register the UDF. Any names -the accumulator resolves via `import` are captured **by reference** -and must be importable on the receiving worker. See -{py:mod}`datafusion.ipc` for the full IPC model and security caveats. -::: diff --git a/docs/source/user-guide/common-operations/basic-info.ipynb b/docs/source/user-guide/common-operations/basic-info.ipynb new file mode 100644 index 000000000..4b934710b --- /dev/null +++ b/docs/source/user-guide/common-operations/basic-info.ipynb @@ -0,0 +1,138 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n# Basic Operations\n\nIn this section, you will learn how to display essential details of DataFrames using specific functions.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "import random\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict(\n", + " {\n", + " \"nrs\": [1, 2, 3, 4, 5],\n", + " \"names\": [\"python\", \"ruby\", \"java\", \"haskell\", \"go\"],\n", + " \"random\": random.sample(range(1000), 5),\n", + " \"groups\": [\"A\", \"A\", \"B\", \"C\", \"B\"],\n", + " }\n", + ")\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\nUse [`limit`][datafusion.dataframe.DataFrame.limit] to view the top rows of the frame:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "df.limit(2)" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\nDisplay the columns of the DataFrame using [`schema`][datafusion.dataframe.DataFrame.schema]:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "df.schema()" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\nThe method [`to_pandas`][datafusion.dataframe.DataFrame.to_pandas] uses pyarrow to convert to pandas DataFrame, by collecting the batches,\npassing them to an Arrow table, and then converting them to a pandas DataFrame.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "df.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "7cdc8c89c7104fffa095e18ddfef8986", + "metadata": {}, + "source": "\n[`describe`][datafusion.dataframe.DataFrame.describe] shows a quick statistic summary of your data:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b118ea5561624da68c537baed56e602f", + "metadata": {}, + "outputs": [], + "source": [ + "df.describe()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/common-operations/basic-info.md b/docs/source/user-guide/common-operations/basic-info.md deleted file mode 100644 index 263068b4e..000000000 --- a/docs/source/user-guide/common-operations/basic-info.md +++ /dev/null @@ -1,71 +0,0 @@ - - -# Basic Operations - -In this section, you will learn how to display essential details of DataFrames using specific functions. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - import random - - ctx = SessionContext() - df = ctx.from_pydict({ - "nrs": [1, 2, 3, 4, 5], - "names": ["python", "ruby", "java", "haskell", "go"], - "random": random.sample(range(1000), 5), - "groups": ["A", "A", "B", "C", "B"], - }) - df -``` - -Use {py:func}`~datafusion.dataframe.DataFrame.limit` to view the top rows of the frame: - -```{eval-rst} -.. ipython:: python - - df.limit(2) -``` - -Display the columns of the DataFrame using {py:func}`~datafusion.dataframe.DataFrame.schema`: - -```{eval-rst} -.. ipython:: python - - df.schema() -``` - -The method {py:func}`~datafusion.dataframe.DataFrame.to_pandas` uses pyarrow to convert to pandas DataFrame, by collecting the batches, -passing them to an Arrow table, and then converting them to a pandas DataFrame. - -```{eval-rst} -.. ipython:: python - - df.to_pandas() -``` - -{py:func}`~datafusion.dataframe.DataFrame.describe` shows a quick statistic summary of your data: - -```{eval-rst} -.. ipython:: python - - df.describe() -``` diff --git a/docs/source/user-guide/common-operations/expressions.ipynb b/docs/source/user-guide/common-operations/expressions.ipynb new file mode 100644 index 000000000..988a916b2 --- /dev/null +++ b/docs/source/user-guide/common-operations/expressions.ipynb @@ -0,0 +1,382 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n\n# Expressions\n\nIn DataFusion an expression is an abstraction that represents a computation.\nExpressions are used as the primary inputs and outputs for most functions within\nDataFusion. As such, expressions can be combined to create expression trees, a\nconcept shared across most compilers and databases.\n\n## Column\n\nThe first expression most new users will interact with is the Column, which is created by calling [`col`][datafusion.col].\nThis expression represents a column within a DataFrame. The function [`col`][datafusion.col] takes as in input a string\nand returns an expression as it's output.\n\n## Literal\n\nLiteral expressions represent a single value. These are helpful in a wide range of operations where\na specific, known value is of interest. You can create a literal expression using the function [`lit`][datafusion.lit].\nThe type of the object passed to the [`lit`][datafusion.lit] function will be used to convert it to a known data type.\n\nIn the following example we create expressions for the column named `color` and the literal scalar string `red`.\nThe resultant variable `red_units` is itself also an expression.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "red_units = col(\"color\") == lit(\"red\")" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\n## Boolean\n\nWhen combining expressions that evaluate to a boolean value, you can combine these expressions using boolean operators.\nIt is important to note that in order to combine these expressions, you *must* use bitwise operators. See the following\nexamples for the and, or, and not operations.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "red_or_green_units = (col(\"color\") == lit(\"red\")) | (col(\"color\") == lit(\"green\"))\n", + "heavy_red_units = (col(\"color\") == lit(\"red\")) & (col(\"weight\") > lit(42))\n", + "not_red_units = ~(col(\"color\") == lit(\"red\"))" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\n## Arrays\n\nFor columns that contain arrays of values, you can access individual elements of the array by index\nusing bracket indexing. This is similar to calling the function\n[`array_element`][datafusion.functions.array_element], except that array indexing using brackets is 0 based,\nsimilar to Python arrays and `array_element` is 1 based indexing to be compatible with other SQL\napproaches.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import col\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict({\"a\": [[1, 2, 3], [4, 5, 6]]})\n", + "df.select(col(\"a\")[0].alias(\"a0\"))" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\n!!! warning\n\n Indexing an element of an array via `[]` starts at index 0 whereas\n [`array_element`][datafusion.functions.array_element] starts at index 1.\n\nStarting in DataFusion 49.0.0 you can also create slices of array elements using\nslice syntax from Python.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(col(\"a\")[1:3].alias(\"second_two_elements\"))" + ] + }, + { + "cell_type": "markdown", + "id": "7cdc8c89c7104fffa095e18ddfef8986", + "metadata": {}, + "source": "\nTo check if an array is empty, you can use the function [`array_empty`][datafusion.functions.array_empty] or `datafusion.functions.empty`.\nThis function returns a boolean indicating whether the array is empty.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b118ea5561624da68c537baed56e602f", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import SessionContext, col\n", + "from datafusion.functions import array_empty\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict({\"a\": [[], [1, 2, 3]]})\n", + "df.select(array_empty(col(\"a\")).alias(\"is_empty\"))" + ] + }, + { + "cell_type": "markdown", + "id": "938c804e27f84196a10c8828c723f798", + "metadata": {}, + "source": "\nIn this example, the `is_empty` column will contain `True` for the first row and `False` for the second row.\n\nTo get the total number of elements in an array, you can use the function [`cardinality`][datafusion.functions.cardinality].\nThis function returns an integer indicating the total number of elements in the array.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "504fb2a444614c0babb325280ed9130a", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import SessionContext, col\n", + "from datafusion.functions import cardinality\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict({\"a\": [[1, 2, 3], [4, 5, 6]]})\n", + "df.select(cardinality(col(\"a\")).alias(\"num_elements\"))" + ] + }, + { + "cell_type": "markdown", + "id": "59bbdb311c014d738909a11f9e486628", + "metadata": {}, + "source": "\nIn this example, the `num_elements` column will contain `3` for both rows.\n\nTo concatenate two arrays, you can use the function [`array_cat`][datafusion.functions.array_cat] or [`array_concat`][datafusion.functions.array_concat].\nThese functions return a new array that is the concatenation of the input arrays.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b43b363d81ae4b689946ece5c682cd59", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import SessionContext, col\n", + "from datafusion.functions import array_cat\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict({\"a\": [[1, 2, 3]], \"b\": [[4, 5, 6]]})\n", + "df.select(array_cat(col(\"a\"), col(\"b\")).alias(\"concatenated_array\"))" + ] + }, + { + "cell_type": "markdown", + "id": "8a65eabff63a45729fe45fb5ade58bdc", + "metadata": {}, + "source": "\nIn this example, the `concatenated_array` column will contain `[1, 2, 3, 4, 5, 6]`.\n\nTo repeat the elements of an array a specified number of times, you can use the function [`array_repeat`][datafusion.functions.array_repeat].\nThis function returns a new array with the elements repeated.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c3933fab20d04ec698c2621248eb3be0", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import SessionContext, col\n", + "from datafusion.functions import array_repeat\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict({\"a\": [[1, 2, 3]]})\n", + "df.select(array_repeat(col(\"a\"), literal(2)).alias(\"repeated_array\"))" + ] + }, + { + "cell_type": "markdown", + "id": "4dd4641cc4064e0191573fe9c69df29b", + "metadata": {}, + "source": "\nIn this example, the `repeated_array` column will contain `[[1, 2, 3], [1, 2, 3]]`.\n\n## Lambda functions\n\nSome array functions take a *lambda function*: a small function that runs once\nper element. [`array_transform`][datafusion.functions.array_transform] maps a lambda over\nevery element, [`array_filter`][datafusion.functions.array_filter] keeps the elements\nfor which a predicate lambda is true, and\n[`array_any_match`][datafusion.functions.array_any_match] returns whether any element\nsatisfies a predicate lambda. (Functions that take another function as an\nargument are sometimes called *higher-order* functions.)\n\nThe simplest way to supply a lambda is a Python `lambda`. Its parameter names\nbecome the lambda parameters, and its return value becomes the body.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8309879909854d7188b41380fd92a7c3", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import SessionContext, col\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict({\"a\": [[1, 2, 3], [4, 5]]})\n", + "df.select(f.array_transform(col(\"a\"), lambda v: v * 2).alias(\"doubled\"))\n", + "df.select(f.array_filter(col(\"a\"), lambda v: v > 2).alias(\"big_only\"))\n", + "df.select(f.array_any_match(col(\"a\"), lambda v: v > 3).alias(\"has_big\"))" + ] + }, + { + "cell_type": "markdown", + "id": "3ed186c9a28b402fb0bc4494df01f08d", + "metadata": {}, + "source": "\nIf you need explicit control over parameter names, build the lambda with\n[`lambda_`][datafusion.functions.lambda_] and reference its parameters with\n[`lambda_var`][datafusion.functions.lambda_var]. The following is equivalent to the\n`array_transform` call above.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb1e1581032b452c9409d6c6813c49d1", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import lit\n", + "\n", + "double_fn = f.lambda_([\"v\"], f.lambda_var(\"v\") * lit(2))\n", + "df.select(f.array_transform(col(\"a\"), double_fn).alias(\"doubled\"))" + ] + }, + { + "cell_type": "markdown", + "id": "379cbbc1e968416e875cc15c1202d7eb", + "metadata": {}, + "source": "\n!!! note\n\n Lambda expressions cannot yet be serialized: calling\n [`to_bytes`][datafusion.expr.Expr.to_bytes] or pickling an expression that\n contains a lambda raises `Lambda not implemented`. SQL lambda syntax is\n only parsed by dialects that support lambdas; set\n `datafusion.sql_parser.dialect` to one of `DuckDB`, `ClickHouse`,\n `Snowflake`, or `Databricks`. Both arrow syntax (`x -> x * 2`) and\n keyword syntax (`lambda x: x * 2`) parse. DuckDB will drop the arrow\n form in v2.1, so prefer `lambda x: x * 2` for forward compatibility.\n The Python expression builder shown above works regardless of dialect.\n\n## Testing membership in a list\n\nA common need is filtering rows where a column equals *any* of a small set of\nvalues. DataFusion offers three forms; they differ in readability and in how\nthey scale:\n\n1. A compound boolean using `|` across explicit equalities.\n2. [`in_list`][datafusion.functions.in_list], which accepts a list of\n expressions and tests equality against all of them in one call.\n3. A trick with [`array_position`][datafusion.functions.array_position] and\n [`make_array`][datafusion.functions.make_array], which returns the 1-based\n index of the value in a constructed array, or null if it is not present.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "277c27b1587741f2af2001be3712ef0d", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import SessionContext, col, lit\n", + "from datafusion import functions as f\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict({\"shipmode\": [\"MAIL\", \"SHIP\", \"AIR\", \"TRUCK\", \"RAIL\"]})\n", + "\n", + "# Option 1: compound boolean. Fine for two values; awkward past three.\n", + "df.filter((col(\"shipmode\") == lit(\"MAIL\")) | (col(\"shipmode\") == lit(\"SHIP\")))\n", + "\n", + "# Option 2: in_list. Preferred for readability as the set grows.\n", + "df.filter(f.in_list(col(\"shipmode\"), [lit(\"MAIL\"), lit(\"SHIP\")]))\n", + "\n", + "# Option 3: array_position / make_array. Useful when you already have the\n", + "# set as an array column and want \"is in that array\" semantics.\n", + "df.filter(\n", + " ~f.array_position(f.make_array(lit(\"MAIL\"), lit(\"SHIP\")), col(\"shipmode\")).is_null()\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "db7b79bc585a40fcaf58bf750017e135", + "metadata": {}, + "source": "\nUse `in_list` as the default. It is explicit, readable, and matches the\nsemantics users expect from SQL's `IN (...)`. Reach for the\n`array_position` form only when the membership set is itself an array\ncolumn rather than a literal list.\n\n## Conditional expressions\n\nDataFusion provides [`case`][datafusion.functions.case] for the SQL\n`CASE` expression in both its switched and searched forms, along with\n[`when`][datafusion.functions.when] as a standalone builder for the\nsearched form.\n\n**Switched CASE** (one expression compared against several literal values):\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "916684f9a58a4a2aa5f864670399430d", + "metadata": {}, + "outputs": [], + "source": [ + "df = ctx.from_pydict(\n", + " {\"priority\": [\"1-URGENT\", \"2-HIGH\", \"3-MEDIUM\", \"5-LOW\"]},\n", + ")\n", + "\n", + "df.select(\n", + " col(\"priority\"),\n", + " f.case(col(\"priority\"))\n", + " .when(lit(\"1-URGENT\"), lit(1))\n", + " .when(lit(\"2-HIGH\"), lit(1))\n", + " .otherwise(lit(0))\n", + " .alias(\"is_high_priority\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "1671c31a24314836a5b85d7ef7fbf015", + "metadata": {}, + "source": "\n**Searched CASE** (an independent boolean predicate per branch). Use this\nform whenever a branch tests more than simple equality — for example,\nchecking whether a joined column is `NULL` to gate a computed value:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "33b0902fd34d4ace834912fa1002cf8e", + "metadata": {}, + "outputs": [], + "source": [ + "df = ctx.from_pydict(\n", + " {\"volume\": [10.0, 20.0, 30.0], \"supplier_id\": [1, None, 2]},\n", + ")\n", + "\n", + "df.select(\n", + " col(\"volume\"),\n", + " col(\"supplier_id\"),\n", + " f.when(col(\"supplier_id\").is_not_null(), col(\"volume\"))\n", + " .otherwise(lit(0.0))\n", + " .alias(\"attributed_volume\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f6fa52606d8c4a75a9b52967216f8f3f", + "metadata": {}, + "source": "\nThis searched-CASE pattern is idiomatic for \"attribute the measure to the\nmatching side of a left join, otherwise contribute zero\" — a shape that\nappears in TPC-H Q08 and similar market-share calculations.\n\nIf a switched CASE only groups several equality matches into one bucket,\n`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often\nsimpler than the full `case` builder.\n\n## Structs\n\nColumns that contain struct elements can be accessed using the bracket notation as if they were\nPython dictionary style objects. This expects a string key as the parameter passed.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f5a1fa73e5044315a093ec459c9be902", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "data = {\"a\": [{\"size\": 15, \"color\": \"green\"}, {\"size\": 10, \"color\": \"blue\"}]}\n", + "df = ctx.from_pydict(data)\n", + "df.select(col(\"a\")[\"size\"].alias(\"a_size\"))" + ] + }, + { + "cell_type": "markdown", + "id": "cdf66aed5cc84ca1b48e60bad68798a8", + "metadata": {}, + "source": "\n## Functions\n\nAs mentioned before, most functions in DataFusion return an expression at their output. This allows us to create\na wide variety of expressions built up from other expressions. For example, [`alias`][datafusion.expr.Expr.alias] is a function that takes\nas it input a single expression and returns an expression in which the name of the expression has changed.\n\nThe following example shows a series of expressions that are built up from functions operating on expressions.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "28d3efd5258a48a79c179ea5c6759f01", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import SessionContext, lit\n", + "from datafusion import functions as f\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_pydict(\n", + " {\n", + " \"name\": [\"Albert\", \"Becca\", \"Carlos\", \"Dante\"],\n", + " \"age\": [42, 67, 27, 71],\n", + " \"years_in_position\": [13, 21, 10, 54],\n", + " },\n", + " name=\"employees\",\n", + ")\n", + "\n", + "age_col = col(\"age\")\n", + "renamed_age = age_col.alias(\"age_in_years\")\n", + "start_age = age_col - col(\"years_in_position\")\n", + "started_young = start_age < lit(18)\n", + "can_retire = age_col > lit(65)\n", + "long_timer = started_young & can_retire\n", + "\n", + "df.filter(long_timer).select(col(\"name\"), renamed_age, col(\"years_in_position\"))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/common-operations/expressions.md b/docs/source/user-guide/common-operations/expressions.md deleted file mode 100644 index 0679febab..000000000 --- a/docs/source/user-guide/common-operations/expressions.md +++ /dev/null @@ -1,357 +0,0 @@ - - -(expressions)= - -# Expressions - -In DataFusion an expression is an abstraction that represents a computation. -Expressions are used as the primary inputs and outputs for most functions within -DataFusion. As such, expressions can be combined to create expression trees, a -concept shared across most compilers and databases. - -## Column - -The first expression most new users will interact with is the Column, which is created by calling {py:func}`~datafusion.col`. -This expression represents a column within a DataFrame. The function {py:func}`~datafusion.col` takes as in input a string -and returns an expression as it's output. - -## Literal - -Literal expressions represent a single value. These are helpful in a wide range of operations where -a specific, known value is of interest. You can create a literal expression using the function {py:func}`~datafusion.lit`. -The type of the object passed to the {py:func}`~datafusion.lit` function will be used to convert it to a known data type. - -In the following example we create expressions for the column named `color` and the literal scalar string `red`. -The resultant variable `red_units` is itself also an expression. - -```{eval-rst} -.. ipython:: python - - red_units = col("color") == lit("red") -``` - -## Boolean - -When combining expressions that evaluate to a boolean value, you can combine these expressions using boolean operators. -It is important to note that in order to combine these expressions, you *must* use bitwise operators. See the following -examples for the and, or, and not operations. - -```{eval-rst} -.. ipython:: python - - red_or_green_units = (col("color") == lit("red")) | (col("color") == lit("green")) - heavy_red_units = (col("color") == lit("red")) & (col("weight") > lit(42)) - not_red_units = ~(col("color") == lit("red")) -``` - -## Arrays - -For columns that contain arrays of values, you can access individual elements of the array by index -using bracket indexing. This is similar to calling the function -{py:func}`datafusion.functions.array_element`, except that array indexing using brackets is 0 based, -similar to Python arrays and `array_element` is 1 based indexing to be compatible with other SQL -approaches. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col - - ctx = SessionContext() - df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5, 6]]}) - df.select(col("a")[0].alias("a0")) -``` - -:::{warning} -Indexing an element of an array via `[]` starts at index 0 whereas -{py:func}`~datafusion.functions.array_element` starts at index 1. -::: - -Starting in DataFusion 49.0.0 you can also create slices of array elements using -slice syntax from Python. - -```{eval-rst} -.. ipython:: python - - df.select(col("a")[1:3].alias("second_two_elements")) -``` - -To check if an array is empty, you can use the function {py:func}`datafusion.functions.array_empty` or `datafusion.functions.empty`. -This function returns a boolean indicating whether the array is empty. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col - from datafusion.functions import array_empty - - ctx = SessionContext() - df = ctx.from_pydict({"a": [[], [1, 2, 3]]}) - df.select(array_empty(col("a")).alias("is_empty")) -``` - -In this example, the `is_empty` column will contain `True` for the first row and `False` for the second row. - -To get the total number of elements in an array, you can use the function {py:func}`datafusion.functions.cardinality`. -This function returns an integer indicating the total number of elements in the array. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col - from datafusion.functions import cardinality - - ctx = SessionContext() - df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5, 6]]}) - df.select(cardinality(col("a")).alias("num_elements")) -``` - -In this example, the `num_elements` column will contain `3` for both rows. - -To concatenate two arrays, you can use the function {py:func}`datafusion.functions.array_cat` or {py:func}`datafusion.functions.array_concat`. -These functions return a new array that is the concatenation of the input arrays. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col - from datafusion.functions import array_cat, array_concat - - ctx = SessionContext() - df = ctx.from_pydict({"a": [[1, 2, 3]], "b": [[4, 5, 6]]}) - df.select(array_cat(col("a"), col("b")).alias("concatenated_array")) -``` - -In this example, the `concatenated_array` column will contain `[1, 2, 3, 4, 5, 6]`. - -To repeat the elements of an array a specified number of times, you can use the function {py:func}`datafusion.functions.array_repeat`. -This function returns a new array with the elements repeated. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col, literal - from datafusion.functions import array_repeat - - ctx = SessionContext() - df = ctx.from_pydict({"a": [[1, 2, 3]]}) - df.select(array_repeat(col("a"), literal(2)).alias("repeated_array")) -``` - -In this example, the `repeated_array` column will contain `[[1, 2, 3], [1, 2, 3]]`. - -## Lambda functions - -Some array functions take a *lambda function*: a small function that runs once -per element. {py:func}`~datafusion.functions.array_transform` maps a lambda over -every element, {py:func}`~datafusion.functions.array_filter` keeps the elements -for which a predicate lambda is true, and -{py:func}`~datafusion.functions.array_any_match` returns whether any element -satisfies a predicate lambda. (Functions that take another function as an -argument are sometimes called *higher-order* functions.) - -The simplest way to supply a lambda is a Python `lambda`. Its parameter names -become the lambda parameters, and its return value becomes the body. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col - from datafusion import functions as f - - ctx = SessionContext() - df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5]]}) - df.select(f.array_transform(col("a"), lambda v: v * 2).alias("doubled")) - df.select(f.array_filter(col("a"), lambda v: v > 2).alias("big_only")) - df.select(f.array_any_match(col("a"), lambda v: v > 3).alias("has_big")) -``` - -If you need explicit control over parameter names, build the lambda with -{py:func}`~datafusion.functions.lambda_` and reference its parameters with -{py:func}`~datafusion.functions.lambda_var`. The following is equivalent to the -`array_transform` call above. - -```{eval-rst} -.. ipython:: python - - from datafusion import lit - - double_fn = f.lambda_(["v"], f.lambda_var("v") * lit(2)) - df.select(f.array_transform(col("a"), double_fn).alias("doubled")) -``` - -:::{note} -Lambda expressions cannot yet be serialized: calling -{py:meth}`~datafusion.expr.Expr.to_bytes` or pickling an expression that -contains a lambda raises `Lambda not implemented`. SQL lambda syntax is -only parsed by dialects that support lambdas; set -`datafusion.sql_parser.dialect` to one of `DuckDB`, `ClickHouse`, -`Snowflake`, or `Databricks`. Both arrow syntax (`x -> x * 2`) and -keyword syntax (`lambda x: x * 2`) parse. DuckDB will drop the arrow -form in v2.1, so prefer `lambda x: x * 2` for forward compatibility. -The Python expression builder shown above works regardless of dialect. -::: - -## Testing membership in a list - -A common need is filtering rows where a column equals *any* of a small set of -values. DataFusion offers three forms; they differ in readability and in how -they scale: - -1. A compound boolean using `|` across explicit equalities. -2. {py:func}`~datafusion.functions.in_list`, which accepts a list of - expressions and tests equality against all of them in one call. -3. A trick with {py:func}`~datafusion.functions.array_position` and - {py:func}`~datafusion.functions.make_array`, which returns the 1-based - index of the value in a constructed array, or null if it is not present. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext, col, lit - from datafusion import functions as f - - ctx = SessionContext() - df = ctx.from_pydict({"shipmode": ["MAIL", "SHIP", "AIR", "TRUCK", "RAIL"]}) - - # Option 1: compound boolean. Fine for two values; awkward past three. - df.filter((col("shipmode") == lit("MAIL")) | (col("shipmode") == lit("SHIP"))) - - # Option 2: in_list. Preferred for readability as the set grows. - df.filter(f.in_list(col("shipmode"), [lit("MAIL"), lit("SHIP")])) - - # Option 3: array_position / make_array. Useful when you already have the - # set as an array column and want "is in that array" semantics. - df.filter( - ~f.array_position( - f.make_array(lit("MAIL"), lit("SHIP")), col("shipmode") - ).is_null() - ) -``` - -Use `in_list` as the default. It is explicit, readable, and matches the -semantics users expect from SQL's `IN (...)`. Reach for the -`array_position` form only when the membership set is itself an array -column rather than a literal list. - -## Conditional expressions - -DataFusion provides {py:func}`~datafusion.functions.case` for the SQL -`CASE` expression in both its switched and searched forms, along with -{py:func}`~datafusion.functions.when` as a standalone builder for the -searched form. - -**Switched CASE** (one expression compared against several literal values): - -```{eval-rst} -.. ipython:: python - - df = ctx.from_pydict( - {"priority": ["1-URGENT", "2-HIGH", "3-MEDIUM", "5-LOW"]}, - ) - - df.select( - col("priority"), - f.case(col("priority")) - .when(lit("1-URGENT"), lit(1)) - .when(lit("2-HIGH"), lit(1)) - .otherwise(lit(0)) - .alias("is_high_priority"), - ) -``` - -**Searched CASE** (an independent boolean predicate per branch). Use this -form whenever a branch tests more than simple equality — for example, -checking whether a joined column is `NULL` to gate a computed value: - -```{eval-rst} -.. ipython:: python - - df = ctx.from_pydict( - {"volume": [10.0, 20.0, 30.0], "supplier_id": [1, None, 2]}, - ) - - df.select( - col("volume"), - col("supplier_id"), - f.when(col("supplier_id").is_not_null(), col("volume")) - .otherwise(lit(0.0)) - .alias("attributed_volume"), - ) -``` - -This searched-CASE pattern is idiomatic for "attribute the measure to the -matching side of a left join, otherwise contribute zero" — a shape that -appears in TPC-H Q08 and similar market-share calculations. - -If a switched CASE only groups several equality matches into one bucket, -`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often -simpler than the full `case` builder. - -## Structs - -Columns that contain struct elements can be accessed using the bracket notation as if they were -Python dictionary style objects. This expects a string key as the parameter passed. - -```{eval-rst} -.. ipython:: python - - ctx = SessionContext() - data = {"a": [{"size": 15, "color": "green"}, {"size": 10, "color": "blue"}]} - df = ctx.from_pydict(data) - df.select(col("a")["size"].alias("a_size")) - -``` - -## Functions - -As mentioned before, most functions in DataFusion return an expression at their output. This allows us to create -a wide variety of expressions built up from other expressions. For example, {py:func}`~datafusion.expr.Expr.alias` is a function that takes -as it input a single expression and returns an expression in which the name of the expression has changed. - -The following example shows a series of expressions that are built up from functions operating on expressions. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - from datafusion import column, lit - from datafusion import functions as f - import random - - ctx = SessionContext() - df = ctx.from_pydict( - { - "name": ["Albert", "Becca", "Carlos", "Dante"], - "age": [42, 67, 27, 71], - "years_in_position": [13, 21, 10, 54], - }, - name="employees" - ) - - age_col = col("age") - renamed_age = age_col.alias("age_in_years") - start_age = age_col - col("years_in_position") - started_young = start_age < lit(18) - can_retire = age_col > lit(65) - long_timer = started_young & can_retire - - df.filter(long_timer).select(col("name"), renamed_age, col("years_in_position")) -``` diff --git a/docs/source/user-guide/common-operations/functions.ipynb b/docs/source/user-guide/common-operations/functions.ipynb new file mode 100644 index 000000000..bdbda3c7c --- /dev/null +++ b/docs/source/user-guide/common-operations/functions.ipynb @@ -0,0 +1,237 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n# Functions\n\nDataFusion provides a large number of built-in functions for performing complex queries without requiring user-defined functions.\nIn here we will cover some of the more popular use cases. If you want to view all the functions go to the [`Functions`][datafusion.functions] API Reference.\n\nWe'll use the pokemon dataset in the following examples.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "ctx.register_csv(\"pokemon\", \"pokemon.csv\")\n", + "df = ctx.table(\"pokemon\")" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\n## Mathematical\n\nDataFusion offers mathematical functions such as [`pow`][datafusion.functions.pow] or [`log`][datafusion.functions.log]\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import str_lit, string_literal\n", + "\n", + "df.select(\n", + " f.pow(col('\"Attack\"'), literal(2)) - f.pow(col('\"Defense\"'), literal(2))\n", + ").limit(10)" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\n## Conditional\n\nThere 3 conditional functions in DataFusion [`coalesce`][datafusion.functions.coalesce], [`nullif`][datafusion.functions.nullif] and [`case`][datafusion.functions.case].\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(f.coalesce(col('\"Type 1\"'), col('\"Type 2\"')).alias(\"dominant_type\")).limit(10)" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\n## Temporal\n\nFor selecting the current time use [`now`][datafusion.functions.now]\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(f.now())" + ] + }, + { + "cell_type": "markdown", + "id": "7cdc8c89c7104fffa095e18ddfef8986", + "metadata": {}, + "source": "\nConvert to timestamps using [`to_timestamp`][datafusion.functions.to_timestamp]\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b118ea5561624da68c537baed56e602f", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(f.to_timestamp(col('\"Total\"')).alias(\"timestamp\"))" + ] + }, + { + "cell_type": "markdown", + "id": "938c804e27f84196a10c8828c723f798", + "metadata": {}, + "source": "\nExtracting parts of a date using [`date_part`][datafusion.functions.date_part] (alias [`extract`][datafusion.functions.extract])\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "504fb2a444614c0babb325280ed9130a", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(\n", + " f.date_part(literal(\"month\"), f.to_timestamp(col('\"Total\"'))).alias(\"month\"),\n", + " f.extract(literal(\"day\"), f.to_timestamp(col('\"Total\"'))).alias(\"day\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "59bbdb311c014d738909a11f9e486628", + "metadata": {}, + "source": "\n## String\n\nIn the field of data science, working with textual data is a common task. To make string manipulation easier,\nDataFusion offers a range of helpful options.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b43b363d81ae4b689946ece5c682cd59", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(\n", + " f.char_length(col('\"Name\"')).alias(\"len\"),\n", + " f.lower(col('\"Name\"')).alias(\"lower\"),\n", + " f.left(col('\"Name\"'), literal(4)).alias(\"code\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "8a65eabff63a45729fe45fb5ade58bdc", + "metadata": {}, + "source": "\nThis also includes the functions for regular expressions like [`regexp_replace`][datafusion.functions.regexp_replace] and [`regexp_match`][datafusion.functions.regexp_match]\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c3933fab20d04ec698c2621248eb3be0", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(\n", + " f.regexp_match(col('\"Name\"'), literal(\"Char\")).alias(\"dragons\"),\n", + " f.regexp_replace(col('\"Name\"'), literal(\"saur\"), literal(\"fleur\")).alias(\"flowers\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "4dd4641cc4064e0191573fe9c69df29b", + "metadata": {}, + "source": "\n## Casting\n\nCasting expressions to different data types using [`arrow_cast`][datafusion.functions.arrow_cast]\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8309879909854d7188b41380fd92a7c3", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(\n", + " f.arrow_cast(col('\"Total\"'), string_literal(\"Float64\")).alias(\"total_as_float\"),\n", + " f.arrow_cast(col('\"Total\"'), str_lit(\"Int32\")).alias(\"total_as_int\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "3ed186c9a28b402fb0bc4494df01f08d", + "metadata": {}, + "source": "\n## Other\n\nThe function [`in_list`][datafusion.functions.in_list] allows to check a column for the presence of multiple values:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb1e1581032b452c9409d6c6813c49d1", + "metadata": {}, + "outputs": [], + "source": [ + "types = [literal(\"Grass\"), literal(\"Fire\"), literal(\"Water\")]\n", + "(\n", + " df.select(f.in_list(col('\"Type 1\"'), types, negated=False).alias(\"basic_types\"))\n", + " .limit(20)\n", + " .to_pandas()\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "379cbbc1e968416e875cc15c1202d7eb", + "metadata": {}, + "source": "\n# Handling Missing Values\n\nDataFusion provides methods to handle missing values in DataFrames:\n\n## fill_null\n\nThe `fill_null()` method replaces NULL values in specified columns with a provided value:\n\n```python\n# Fill all NULL values with 0 where possible\ndf = df.fill_null(0)\n\n# Fill NULL values only in specific string columns\ndf = df.fill_null(\"missing\", subset=[\"name\", \"category\"])\n```\n\nThe fill value will be cast to match each column's type. If casting fails for a column, that column remains unchanged.\n" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/common-operations/functions.md b/docs/source/user-guide/common-operations/functions.md deleted file mode 100644 index 2fbd0c85f..000000000 --- a/docs/source/user-guide/common-operations/functions.md +++ /dev/null @@ -1,165 +0,0 @@ - - -# Functions - -DataFusion provides a large number of built-in functions for performing complex queries without requiring user-defined functions. -In here we will cover some of the more popular use cases. If you want to view all the functions go to the {py:mod}`Functions ` API Reference. - -We'll use the pokemon dataset in the following examples. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - - ctx = SessionContext() - ctx.register_csv("pokemon", "pokemon.csv") - df = ctx.table("pokemon") -``` - -## Mathematical - -DataFusion offers mathematical functions such as {py:func}`~datafusion.functions.pow` or {py:func}`~datafusion.functions.log` - -```{eval-rst} -.. ipython:: python - - from datafusion import col, literal, string_literal, str_lit - from datafusion import functions as f - - df.select( - f.pow(col('"Attack"'), literal(2)) - f.pow(col('"Defense"'), literal(2)) - ).limit(10) - -``` - -## Conditional - -There 3 conditional functions in DataFusion {py:func}`~datafusion.functions.coalesce`, {py:func}`~datafusion.functions.nullif` and {py:func}`~datafusion.functions.case`. - -```{eval-rst} -.. ipython:: python - - df.select( - f.coalesce(col('"Type 1"'), col('"Type 2"')).alias("dominant_type") - ).limit(10) -``` - -## Temporal - -For selecting the current time use {py:func}`~datafusion.functions.now` - -```{eval-rst} -.. ipython:: python - - df.select(f.now()) -``` - -Convert to timestamps using {py:func}`~datafusion.functions.to_timestamp` - -```{eval-rst} -.. ipython:: python - - df.select(f.to_timestamp(col('"Total"')).alias("timestamp")) -``` - -Extracting parts of a date using {py:func}`~datafusion.functions.date_part` (alias {py:func}`~datafusion.functions.extract`) - -```{eval-rst} -.. ipython:: python - - df.select( - f.date_part(literal("month"), f.to_timestamp(col('"Total"'))).alias("month"), - f.extract(literal("day"), f.to_timestamp(col('"Total"'))).alias("day") - ) -``` - -## String - -In the field of data science, working with textual data is a common task. To make string manipulation easier, -DataFusion offers a range of helpful options. - -```{eval-rst} -.. ipython:: python - - df.select( - f.char_length(col('"Name"')).alias("len"), - f.lower(col('"Name"')).alias("lower"), - f.left(col('"Name"'), literal(4)).alias("code") - ) -``` - -This also includes the functions for regular expressions like {py:func}`~datafusion.functions.regexp_replace` and {py:func}`~datafusion.functions.regexp_match` - -```{eval-rst} -.. ipython:: python - - df.select( - f.regexp_match(col('"Name"'), literal("Char")).alias("dragons"), - f.regexp_replace(col('"Name"'), literal("saur"), literal("fleur")).alias("flowers") - ) -``` - -## Casting - -Casting expressions to different data types using {py:func}`~datafusion.functions.arrow_cast` - -```{eval-rst} -.. ipython:: python - - df.select( - f.arrow_cast(col('"Total"'), string_literal("Float64")).alias("total_as_float"), - f.arrow_cast(col('"Total"'), str_lit("Int32")).alias("total_as_int") - ) -``` - -## Other - -The function {py:func}`~datafusion.functions.in_list` allows to check a column for the presence of multiple values: - -```{eval-rst} -.. ipython:: python - - types = [literal("Grass"), literal("Fire"), literal("Water")] - ( - df.select(f.in_list(col('"Type 1"'), types, negated=False).alias("basic_types")) - .limit(20) - .to_pandas() - ) - -``` - -# Handling Missing Values - -DataFusion provides methods to handle missing values in DataFrames: - -## fill_null - -The `fill_null()` method replaces NULL values in specified columns with a provided value: - -```python -# Fill all NULL values with 0 where possible -df = df.fill_null(0) - -# Fill NULL values only in specific string columns -df = df.fill_null("missing", subset=["name", "category"]) -``` - -The fill value will be cast to match each column's type. If casting fails for a column, that column remains unchanged. diff --git a/docs/source/user-guide/common-operations/joins.ipynb b/docs/source/user-guide/common-operations/joins.ipynb new file mode 100644 index 000000000..3c0beac92 --- /dev/null +++ b/docs/source/user-guide/common-operations/joins.ipynb @@ -0,0 +1,237 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n# Joins\n\nDataFusion supports the following join variants via the method [`join`][datafusion.dataframe.DataFrame.join]\n\n- Inner Join\n- Left Join\n- Right Join\n- Full Join\n- Left Semi Join\n- Left Anti Join\n\nFor the examples in this section we'll use the following two DataFrames\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "\n", + "left = ctx.from_pydict(\n", + " {\n", + " \"customer_id\": [1, 2, 3],\n", + " \"customer\": [\"Alice\", \"Bob\", \"Charlie\"],\n", + " }\n", + ")\n", + "\n", + "right = ctx.from_pylist(\n", + " [\n", + " {\"id\": 1, \"name\": \"CityCabs\"},\n", + " {\"id\": 2, \"name\": \"MetroRide\"},\n", + " {\"id\": 5, \"name\": \"UrbanGo\"},\n", + " ]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\n## Inner Join\n\nWhen using an inner join, only rows containing the common values between the two join columns present in both DataFrames\nwill be included in the resulting DataFrame.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"inner\")" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\nThe parameter `join_keys` specifies the columns from the left DataFrame and right DataFrame that contains the values\nthat should match.\n\n## Left Join\n\nA left join combines rows from two DataFrames using the key columns. It returns all rows from the left DataFrame and\nmatching rows from the right DataFrame. If there's no match in the right DataFrame, it returns null\nvalues for the corresponding columns.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"left\")" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\n## Full Join\n\nA full join merges rows from two tables based on a related column, returning all rows from both tables, even if there\nis no match. Unmatched rows will have null values.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"full\")" + ] + }, + { + "cell_type": "markdown", + "id": "7cdc8c89c7104fffa095e18ddfef8986", + "metadata": {}, + "source": "\n## Left Semi Join\n\nA left semi join retrieves matching rows from the left table while\nomitting duplicates with multiple matches in the right table.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b118ea5561624da68c537baed56e602f", + "metadata": {}, + "outputs": [], + "source": [ + "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"semi\")" + ] + }, + { + "cell_type": "markdown", + "id": "938c804e27f84196a10c8828c723f798", + "metadata": {}, + "source": "\n## Left Anti Join\n\nA left anti join shows all rows from the left table without any matching rows in the right table,\nbased on a the specified matching columns. It excludes rows from the left table that have at least one matching row in\nthe right table.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "504fb2a444614c0babb325280ed9130a", + "metadata": {}, + "outputs": [], + "source": [ + "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"anti\")" + ] + }, + { + "cell_type": "markdown", + "id": "59bbdb311c014d738909a11f9e486628", + "metadata": {}, + "source": "\n## Duplicate Keys\n\nIt is common to join two DataFrames on a common column name. Starting in\nversion 51.0.0, `` datafusion-python` `` will now coalesce on column with identical names by\ndefault. This reduces problems with ambiguous column selection after joins.\nYou can disable this feature by setting the parameter `coalesce_duplicate_keys`\nto `False`.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b43b363d81ae4b689946ece5c682cd59", + "metadata": {}, + "outputs": [], + "source": [ + "left = ctx.from_pydict(\n", + " {\n", + " \"id\": [1, 2, 3],\n", + " \"customer\": [\"Alice\", \"Bob\", \"Charlie\"],\n", + " }\n", + ")\n", + "\n", + "right = ctx.from_pylist(\n", + " [\n", + " {\"id\": 1, \"name\": \"CityCabs\"},\n", + " {\"id\": 2, \"name\": \"MetroRide\"},\n", + " {\"id\": 5, \"name\": \"UrbanGo\"},\n", + " ]\n", + ")\n", + "\n", + "left.join(right, \"id\", how=\"inner\")" + ] + }, + { + "cell_type": "markdown", + "id": "8a65eabff63a45729fe45fb5ade58bdc", + "metadata": {}, + "source": "\nIn contrast to the above example, if we wish to get both columns:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c3933fab20d04ec698c2621248eb3be0", + "metadata": {}, + "outputs": [], + "source": [ + "left.join(right, \"id\", how=\"inner\", coalesce_duplicate_keys=False)" + ] + }, + { + "cell_type": "markdown", + "id": "4dd4641cc4064e0191573fe9c69df29b", + "metadata": {}, + "source": "\n## Disambiguating Columns with `DataFrame.col()`\n\nWhen both DataFrames contain non-key columns with the same name, you can use\n[`col`][datafusion.dataframe.DataFrame.col] on each DataFrame **before** the\njoin to create fully qualified column references. These references can then be\nused in the join predicate and when selecting from the result.\n\nThis is especially useful with [`join_on`][datafusion.dataframe.DataFrame.join_on],\nwhich accepts expression-based predicates.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8309879909854d7188b41380fd92a7c3", + "metadata": {}, + "outputs": [], + "source": [ + "left = ctx.from_pydict(\n", + " {\n", + " \"id\": [1, 2, 3],\n", + " \"val\": [10, 20, 30],\n", + " }\n", + ")\n", + "\n", + "right = ctx.from_pydict(\n", + " {\n", + " \"id\": [1, 2, 3],\n", + " \"val\": [40, 50, 60],\n", + " }\n", + ")\n", + "\n", + "joined = left.join_on(right, left.col(\"id\") == right.col(\"id\"), how=\"inner\")\n", + "\n", + "joined.select(left.col(\"id\"), left.col(\"val\"), right.col(\"val\"))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/common-operations/joins.md b/docs/source/user-guide/common-operations/joins.md deleted file mode 100644 index 6f4ccf3ad..000000000 --- a/docs/source/user-guide/common-operations/joins.md +++ /dev/null @@ -1,181 +0,0 @@ - - -# Joins - -DataFusion supports the following join variants via the method {py:func}`~datafusion.dataframe.DataFrame.join` - -- Inner Join -- Left Join -- Right Join -- Full Join -- Left Semi Join -- Left Anti Join - -For the examples in this section we'll use the following two DataFrames - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - - ctx = SessionContext() - - left = ctx.from_pydict( - { - "customer_id": [1, 2, 3], - "customer": ["Alice", "Bob", "Charlie"], - } - ) - - right = ctx.from_pylist([ - {"id": 1, "name": "CityCabs"}, - {"id": 2, "name": "MetroRide"}, - {"id": 5, "name": "UrbanGo"}, - ]) -``` - -## Inner Join - -When using an inner join, only rows containing the common values between the two join columns present in both DataFrames -will be included in the resulting DataFrame. - -```{eval-rst} -.. ipython:: python - - left.join(right, left_on="customer_id", right_on="id", how="inner") -``` - -The parameter `join_keys` specifies the columns from the left DataFrame and right DataFrame that contains the values -that should match. - -## Left Join - -A left join combines rows from two DataFrames using the key columns. It returns all rows from the left DataFrame and -matching rows from the right DataFrame. If there's no match in the right DataFrame, it returns null -values for the corresponding columns. - -```{eval-rst} -.. ipython:: python - - left.join(right, left_on="customer_id", right_on="id", how="left") -``` - -## Full Join - -A full join merges rows from two tables based on a related column, returning all rows from both tables, even if there -is no match. Unmatched rows will have null values. - -```{eval-rst} -.. ipython:: python - - left.join(right, left_on="customer_id", right_on="id", how="full") -``` - -## Left Semi Join - -A left semi join retrieves matching rows from the left table while -omitting duplicates with multiple matches in the right table. - -```{eval-rst} -.. ipython:: python - - left.join(right, left_on="customer_id", right_on="id", how="semi") -``` - -## Left Anti Join - -A left anti join shows all rows from the left table without any matching rows in the right table, -based on a the specified matching columns. It excludes rows from the left table that have at least one matching row in -the right table. - -```{eval-rst} -.. ipython:: python - - left.join(right, left_on="customer_id", right_on="id", how="anti") -``` - -## Duplicate Keys - -It is common to join two DataFrames on a common column name. Starting in -version 51.0.0, `` datafusion-python` `` will now coalesce on column with identical names by -default. This reduces problems with ambiguous column selection after joins. -You can disable this feature by setting the parameter `coalesce_duplicate_keys` -to `False`. - -```{eval-rst} -.. ipython:: python - - left = ctx.from_pydict( - { - "id": [1, 2, 3], - "customer": ["Alice", "Bob", "Charlie"], - } - ) - - right = ctx.from_pylist([ - {"id": 1, "name": "CityCabs"}, - {"id": 2, "name": "MetroRide"}, - {"id": 5, "name": "UrbanGo"}, - ]) - - left.join(right, "id", how="inner") -``` - -In contrast to the above example, if we wish to get both columns: - -```{eval-rst} -.. ipython:: python - - left.join(right, "id", how="inner", coalesce_duplicate_keys=False) -``` - -## Disambiguating Columns with `DataFrame.col()` - -When both DataFrames contain non-key columns with the same name, you can use -{py:meth}`~datafusion.dataframe.DataFrame.col` on each DataFrame **before** the -join to create fully qualified column references. These references can then be -used in the join predicate and when selecting from the result. - -This is especially useful with {py:meth}`~datafusion.dataframe.DataFrame.join_on`, -which accepts expression-based predicates. - -```{eval-rst} -.. ipython:: python - - left = ctx.from_pydict( - { - "id": [1, 2, 3], - "val": [10, 20, 30], - } - ) - - right = ctx.from_pydict( - { - "id": [1, 2, 3], - "val": [40, 50, 60], - } - ) - - joined = left.join_on( - right, left.col("id") == right.col("id"), how="inner" - ) - - joined.select(left.col("id"), left.col("val"), right.col("val")) -``` diff --git a/docs/source/user-guide/common-operations/select-and-filter.ipynb b/docs/source/user-guide/common-operations/select-and-filter.ipynb new file mode 100644 index 000000000..8143c83c9 --- /dev/null +++ b/docs/source/user-guide/common-operations/select-and-filter.ipynb @@ -0,0 +1,115 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n# Column Selections\n\nUse [`select`][datafusion.dataframe.DataFrame.select] for basic column selection.\n\nDataFusion can work with several file types, to start simple we can use a subset of the\n[TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page),\nwhich you can download [here](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet).\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "df = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n", + "df.select(\"trip_distance\", \"passenger_count\")" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\nFor mathematical or logical operations use [`col`][datafusion.col] to select columns, and give meaningful names to the resulting\noperations using [`alias`][datafusion.expr.Expr.alias]\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "df.select((col(\"tip_amount\") + col(\"tolls_amount\")).alias(\"tips_plus_tolls\"))" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\n!!! warning\n\n Please be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters\n (ex: Name) you must put your column name in double quotes or the selection won’t work. As an alternative for simple\n column selection use [`select`][datafusion.dataframe.DataFrame.select] without double quotes\n\nFor selecting columns with capital letters use `'\"VendorID\"'`\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(col('\"VendorID\"'))" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\nTo combine it with literal values use the [`lit`][datafusion.lit]\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "large_trip_distance = col(\"trip_distance\") > lit(5.0)\n", + "low_passenger_count = col(\"passenger_count\") < lit(4)\n", + "df.select((large_trip_distance & low_passenger_count).alias(\"lonely_trips\"))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/common-operations/select-and-filter.md b/docs/source/user-guide/common-operations/select-and-filter.md deleted file mode 100644 index 8126dfc3a..000000000 --- a/docs/source/user-guide/common-operations/select-and-filter.md +++ /dev/null @@ -1,71 +0,0 @@ - - -# Column Selections - -Use {py:func}`~datafusion.dataframe.DataFrame.select` for basic column selection. - -DataFusion can work with several file types, to start simple we can use a subset of the -[TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page), -which you can download [here](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet). - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - - ctx = SessionContext() - df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") - df.select("trip_distance", "passenger_count") -``` - -For mathematical or logical operations use {py:func}`~datafusion.col` to select columns, and give meaningful names to the resulting -operations using {py:func}`~datafusion.expr.Expr.alias` - -```{eval-rst} -.. ipython:: python - - from datafusion import col, lit - df.select((col("tip_amount") + col("tolls_amount")).alias("tips_plus_tolls")) -``` - -:::{warning} -Please be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters -(ex: Name) you must put your column name in double quotes or the selection won’t work. As an alternative for simple -column selection use {py:func}`~datafusion.dataframe.DataFrame.select` without double quotes -::: - -For selecting columns with capital letters use `'"VendorID"'` - -```{eval-rst} -.. ipython:: python - - df.select(col('"VendorID"')) - -``` - -To combine it with literal values use the {py:func}`~datafusion.lit` - -```{eval-rst} -.. ipython:: python - - large_trip_distance = col("trip_distance") > lit(5.0) - low_passenger_count = col("passenger_count") < lit(4) - df.select((large_trip_distance & low_passenger_count).alias("lonely_trips")) -``` diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb new file mode 100644 index 000000000..8b7a793a2 --- /dev/null +++ b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb @@ -0,0 +1,205 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n# User-Defined Functions\n\nDataFusion provides powerful expressions and functions, reducing the need for custom Python\nfunctions. However you can still incorporate your own functions, i.e. User-Defined Functions (UDFs).\n\n## Scalar Functions\n\nWhen writing a user-defined function that can operate on a row by row basis, these are called Scalar\nFunctions. You can define your own scalar function by calling\n[`udf`][datafusion.user_defined.ScalarUDF.udf] .\n\nThe basic definition of a scalar UDF is a python function that takes one or more\n[pyarrow](https://arrow.apache.org/docs/python/index.html) arrays and returns a single array as\noutput. DataFusion scalar UDFs operate on an entire batch of records at a time, though the\nevaluation of those records should be on a row by row basis. In the following example, we compute\nif the input array contains null values.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "import pyarrow\n", + "from datafusion import udf\n", + "\n", + "\n", + "def is_null(array: pyarrow.Array) -> pyarrow.Array:\n", + " return array.is_null()\n", + "\n", + "\n", + "is_null_arr = udf(is_null, [pyarrow.int64()], pyarrow.bool_(), \"stable\")\n", + "\n", + "ctx = datafusion.SessionContext()\n", + "\n", + "batch = pyarrow.RecordBatch.from_arrays(\n", + " [pyarrow.array([1, None, 3]), pyarrow.array([4, 5, 6])],\n", + " names=[\"a\", \"b\"],\n", + ")\n", + "df = ctx.create_dataframe([[batch]], name=\"batch_array\")\n", + "\n", + "df.select(col(\"a\"), is_null_arr(col(\"a\")).alias(\"is_null\")).show()" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\nIn the previous example, we used the fact that pyarrow provides a variety of built in array\nfunctions such as `is_null()`. There are additional pyarrow\n[compute functions](https://arrow.apache.org/docs/python/compute.html) available. When possible,\nit is highly recommended to use these functions because they can perform computations without doing\nany copy operations from the original arrays. This leads to greatly improved performance.\n\nIf you need to perform an operation in python that is not available with the pyarrow compute\nfunctions, you will need to convert the record batch into python values, perform your operation,\nand construct an array. This operation of converting the built in data type of the array into a\npython object can be one of the slowest operations in DataFusion, so it should be done sparingly.\n\nThe following example performs the same operation as before with `is_null` but demonstrates\nconverting to Python objects to do the evaluation.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "import datafusion\n", + "import pyarrow\n", + "from datafusion import col, udf\n", + "\n", + "\n", + "def is_null(array: pyarrow.Array) -> pyarrow.Array:\n", + " return pyarrow.array([value.as_py() is None for value in array])\n", + "\n", + "\n", + "is_null_arr = udf(is_null, [pyarrow.int64()], pyarrow.bool_(), \"stable\")\n", + "\n", + "ctx = datafusion.SessionContext()\n", + "\n", + "batch = pyarrow.RecordBatch.from_arrays(\n", + " [pyarrow.array([1, None, 3]), pyarrow.array([4, 5, 6])],\n", + " names=[\"a\", \"b\"],\n", + ")\n", + "df = ctx.create_dataframe([[batch]], name=\"batch_array\")\n", + "\n", + "df.select(col(\"a\"), is_null_arr(col(\"a\")).alias(\"is_null\")).show()" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\nIn this example we passed the PyArrow `DataType` when we defined the function\nby calling `udf()`. If you need additional control, such as specifying\nmetadata or nullability of the input or output, you can instead specify a\nPyArrow `Field`.\n\nIf you need to write a custom function but do not want to incur the performance\ncost of converting to Python objects and back, a more advanced approach is to\nwrite Rust based UDFs and to expose them to Python. There is an example in the\n[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/)\ndescribing how to do this.\n\n### When not to use a UDF\n\nA UDF is the right tool when the per-row computation genuinely cannot be\nexpressed with DataFusion's built-in expressions. It is often the *wrong*\ntool for a predicate that *can* be written as an `Expr` tree but feels\neasier to write as a Python function — for example, a filter that keeps\na row if it matches any one of several rule sets, where each rule set\nchecks its own combination of columns (the worked example at the end of\nthis section keeps a row when it matches any one of several brand-specific\nrules). Looping over the rules in Python and returning a boolean per row\nreads naturally and is tempting to wrap in a UDF, but a UDF is opaque to\nthe optimizer: filters expressed as UDFs lose several rewrites that the\nengine applies to filters built from native expressions. The most visible\nof these is **predicate pushdown into the table provider**: a native\npredicate can be handed to the source so it skips data before it is read,\nwhile a UDF predicate cannot. The example below uses Parquet, where\npushdown prunes whole row groups using the min/max statistics in the\nfooter, but the same mechanism applies to any table provider that\nadvertises filter support — including custom providers.\n\nThe following example writes a small Parquet file, then filters it two\nways: first with a native expression, then with a UDF that computes the\nsame result. The filter itself is simple on purpose so we can compare\nthe plans side by side.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import tempfile\n", + "\n", + "import pyarrow as pa\n", + "import pyarrow.parquet as pq\n", + "from datafusion import col, udf\n", + "\n", + "tmpdir = tempfile.mkdtemp()\n", + "parquet_path = os.path.join(tmpdir, \"items.parquet\")\n", + "pq.write_table(\n", + " pa.table(\n", + " {\n", + " \"id\": list(range(100)),\n", + " \"brand\": [\"A\", \"B\", \"C\", \"D\"] * 25,\n", + " \"qty\": [i * 10 for i in range(100)],\n", + " }\n", + " ),\n", + " parquet_path,\n", + ")\n", + "\n", + "ctx = SessionContext()\n", + "items = ctx.read_parquet(parquet_path)" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\n**Native-expression predicate.** The filter is a plain boolean tree\nover column references and literals, so the optimizer can analyze it:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "native_filtered = items.filter((col(\"brand\") == lit(\"A\")) & (col(\"qty\") >= lit(150)))\n", + "print(native_filtered.execution_plan().display_indent())" + ] + }, + { + "cell_type": "markdown", + "id": "7cdc8c89c7104fffa095e18ddfef8986", + "metadata": {}, + "source": "\nNotice the `DataSourceExec` line. It carries three annotations the\noptimizer computed from the predicate:\n\n- `predicate=brand@1 = A AND qty@2 >= 150` — the filter is pushed\n into the Parquet scan itself, so the scan only reads matching rows.\n- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ...\n qty_max@4 >= 150` — the scan prunes whole row groups by consulting\n the Parquet min/max statistics in the footer *before* reading any\n column data.\n- `required_guarantees=[brand in (A)]` — the scan uses this when a\n bloom filter or dictionary is available to skip pages.\n\n**UDF predicate.** Now wrap the same logic in a Python UDF:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b118ea5561624da68c537baed56e602f", + "metadata": {}, + "outputs": [], + "source": [ + "def brand_qty_filter(brand_arr: pa.Array, qty_arr: pa.Array) -> pa.Array:\n", + " return pa.array(\n", + " [b.as_py() == \"A\" and q.as_py() >= 150 for b, q in zip(brand_arr, qty_arr)]\n", + " )\n", + "\n", + "\n", + "pred_udf = udf(\n", + " brand_qty_filter,\n", + " [pa.string(), pa.int64()],\n", + " pa.bool_(),\n", + " \"stable\",\n", + ")\n", + "udf_filtered = items.filter(pred_udf(col(\"brand\"), col(\"qty\")))\n", + "print(udf_filtered.execution_plan().display_indent())" + ] + }, + { + "cell_type": "markdown", + "id": "938c804e27f84196a10c8828c723f798", + "metadata": {}, + "source": "\nThe `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`.\nThere is no `pruning_predicate` and no `required_guarantees`: the\nscan has to materialize every row group and hand each row to the\nPython callback just to decide whether to keep it.\n\nAt small scale the cost difference is invisible; on a Parquet file with\nmany row groups, or data whose min/max statistics line up well with\nthe predicate, the native form can skip most of the file. The UDF form\nreads all of it.\n\n**Takeaway.** Reach for a UDF when the per-row computation is genuinely\nnot expressible as a tree of built-in functions (custom numerical work,\nexternal lookups, complex business rules). When it *is* expressible —\neven if the native form is a little more verbose — build the `Expr`\ntree directly so the optimizer can see through it. For disjunctive\npredicates the idiom is to produce one clause per bucket and combine\nthem with `|`:\n\n```python\nfrom functools import reduce\nfrom operator import or_\nfrom datafusion import col, lit, functions as f\n\nbuckets = {\n \"Brand#12\": {\"containers\": [\"SM CASE\", \"SM BOX\"], \"min_qty\": 1, \"max_size\": 5},\n \"Brand#23\": {\"containers\": [\"MED BAG\", \"MED BOX\"], \"min_qty\": 10, \"max_size\": 10},\n}\n\ndef bucket_clause(brand, spec):\n return (\n (col(\"brand\") == lit(brand))\n & f.in_list(col(\"container\"), [lit(c) for c in spec[\"containers\"]])\n & (col(\"quantity\") >= lit(spec[\"min_qty\"]))\n & (col(\"quantity\") <= lit(spec[\"min_qty\"] + 10))\n & (col(\"size\") >= lit(1))\n & (col(\"size\") <= lit(spec[\"max_size\"]))\n )\n\npredicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items()))\ndf = df.filter(predicate)\n```\n\n## Aggregate Functions\n\nThe [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined\nAggregate Functions (UDAFs). To use this you must implement an\n[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed.\n\nWhen defining a UDAF there are four methods you need to implement. The `update` function takes the\narray(s) of input and updates the internal state of the accumulator. You should define this function\nto have as many input arguments as you will pass when calling the UDAF. Since aggregation may be\nsplit into multiple batches, we must have a method to combine multiple batches. For this, we have\ntwo functions, `state` and `merge`. `state` will return an array of scalar values that contain\nthe current state of a single batch accumulation. Then we must `merge` the results of these\ndifferent states. Finally `evaluate` is the call that will return the final result after the\n`merge` is complete.\n\nIn the following example we want to define a custom aggregate function that will return the\ndifference between the sum of two columns. The state can be represented by a single value and we can\nalso see how the inputs to `update` and `merge` differ.\n\n```python\nimport pyarrow as pa\nimport pyarrow.compute\nimport datafusion\nfrom datafusion import col, udaf, Accumulator\nfrom typing import List\n\nclass MyAccumulator(Accumulator):\n \"\"\"\n Interface of a user-defined accumulation.\n \"\"\"\n def __init__(self):\n self._sum = 0.0\n\n def update(self, values_a: pa.Array, values_b: pa.Array) -> None:\n self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py()\n\n def merge(self, states: list[pa.Array]) -> None:\n self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py()\n\n def state(self) -> list[pa.Scalar]:\n return [pyarrow.scalar(self._sum)]\n\n def evaluate(self) -> pa.Scalar:\n return pyarrow.scalar(self._sum)\n\nctx = datafusion.SessionContext()\ndf = ctx.from_pydict(\n {\n \"a\": [4, 5, 6],\n \"b\": [1, 2, 3],\n }\n)\n\nmy_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable')\n\ndf.aggregate([], [my_udaf(col(\"a\"), col(\"b\")).alias(\"col_diff\")])\n```\n\n### FAQ\n\n**How do I return a list from a UDAF?**\n\nBoth the `evaluate` and the `state` functions expect to return scalar values.\nIf you wish to return a list array as a scalar value, the best practice is to\nwrap the values in a `pyarrow.Scalar` object. For example, you can return a\ntimestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp(\"ms\")))` and\nregister the appropriate return or state types as\n`return_type=pa.list_(pa.timestamp(\"ms\"))` and\n`state_type=[pa.list_(pa.timestamp(\"ms\"))]`, respectively.\n\nAs of DataFusion 52.0.0 , you can pass return any Python object, including a\nPyArrow array, as the return value(s) for these functions and DataFusion will\nattempt to create a scalar type from the value. DataFusion has been tested to\nconvert PyArrow, nanoarrow, and arro3 objects as well as primitive data types\nlike integers, strings, and so on.\n\n## Window Functions\n\nTo implement a User-Defined Window Function (UDWF) you must call the\n[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract\nclass [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator].\n\nThere are three methods of evaluation of UDWFs.\n\n- `evaluate` is the simplest case, where you are given an array and are expected to calculate the\n value for a single row of that array. This is the simplest case, but also the least performant.\n- `evaluate_all` computes the values for all rows for an input array at a single time.\n- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank\n information for the rows.\n\nWhich methods you implement are based upon which of these options are set.\n\n| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement |\n|---|---|---|---|\n| False (default) | False (default) | False (default) | `evaluate_all` |\n| False | True | False | `evaluate` |\n| False | True | False | `evaluate_all_with_rank` |\n| True | True/False | True/False | `evaluate` |\n\n### UDWF options\n\nWhen you define your UDWF you can override the functions that return these values. They will\ndetermine which evaluate functions are called.\n\n- `uses_window_frame` is set for functions that compute based on the specified window frame. If\n your function depends upon the specified frame, set this to `True`.\n- `supports_bounded_execution` specifies if your function can be incrementally computed.\n- `include_rank` is set to `True` for window functions that can be computed only using the rank\n information.\n\n```python\nimport pyarrow as pa\nfrom datafusion import udwf, col, SessionContext\nfrom datafusion.user_defined import WindowEvaluator\n\nclass ExponentialSmooth(WindowEvaluator):\n def __init__(self, alpha: float) -> None:\n self.alpha = alpha\n\n def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array:\n results = []\n curr_value = 0.0\n values = values[0]\n for idx in range(num_rows):\n if idx == 0:\n curr_value = values[idx].as_py()\n else:\n curr_value = values[idx].as_py() * self.alpha + curr_value * (\n 1.0 - self.alpha\n )\n results.append(curr_value)\n\n return pa.array(results)\n\nexp_smooth = udwf(\n ExponentialSmooth(0.9),\n pa.float64(),\n pa.float64(),\n volatility=\"immutable\",\n)\n\nctx = SessionContext()\n\ndf = ctx.from_pydict({\n \"a\": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0]\n})\n\ndf.select(\"a\", exp_smooth(col(\"a\")).alias(\"smooth_a\")).show()\n```\n\n## Table Functions\n\nUser Defined Table Functions are slightly different than the other functions\ndescribed here. These functions take any number of `Expr` arguments, but only\nliteral expressions are supported. Table functions must return a Table\nProvider as described in the ref:`_io_custom_table_provider` page.\n\nOnce you have a table function, you can register it with the session context\nby using [`register_udtf`][datafusion.context.SessionContext.register_udtf].\n\nThere are examples of both rust backed and python based table functions in the\nexamples folder of the repository. If you have a rust backed table function\nthat you wish to expose via PyO3, you need to expose it as a `PyCapsule`.\n\n```rust\n#[pymethods]\nimpl MyTableFunction {\n fn __datafusion_table_function__<'py>(\n &self,\n py: Python<'py>,\n ) -> PyResult> {\n let name = cr\"datafusion_table_function\".into();\n\n let func = self.clone();\n let provider = FFI_TableFunction::new(Arc::new(func), None);\n\n PyCapsule::new(py, provider, Some(name))\n }\n}\n```\n\n### Accessing the Calling Session\n\nPure-Python UDTFs can opt into receiving the calling\n[`SessionContext`][datafusion.SessionContext] by registering with\n`with_session=True`. The context is passed as a `session` keyword\nargument on every invocation. Use it to look up registered tables,\nUDFs, or session configuration from inside the callback.\n\n```python\nfrom datafusion import SessionContext, Table, udtf\nfrom datafusion.context import TableProviderExportable\nimport pyarrow as pa\nimport pyarrow.dataset as ds\n\n@udtf(\"list_tables\", with_session=True)\ndef list_tables(*, session: SessionContext) -> TableProviderExportable:\n names = sorted(session.catalog().schema().names())\n batch = pa.RecordBatch.from_pydict({\"name\": names})\n return Table(ds.dataset([batch]))\n\nctx = SessionContext()\nctx.register_batch(\"t1\", pa.RecordBatch.from_pydict({\"x\": [1]}))\nctx.register_udtf(list_tables)\nctx.sql(\"SELECT * FROM list_tables()\").show()\n```\n\nWithout `with_session=True`, the callback receives only the positional\nexpression arguments. The flag is opt-in so existing UDTFs keep working\nunchanged.\n\nThe injected `session` is a fresh [`SessionContext`][datafusion.SessionContext]\nwrapper backed by the same underlying state as the caller, so registries\n(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering\na new table or UDF) propagate to the live session because the registries\nare reference-counted and shared. Configuration changes made through the\nwrapper (e.g. setting session options) do **not** propagate — the wrapper\nholds its own clone of the session config.\n" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.md b/docs/source/user-guide/common-operations/udf-and-udfa.md deleted file mode 100644 index e2d55ee17..000000000 --- a/docs/source/user-guide/common-operations/udf-and-udfa.md +++ /dev/null @@ -1,476 +0,0 @@ - - -# User-Defined Functions - -DataFusion provides powerful expressions and functions, reducing the need for custom Python -functions. However you can still incorporate your own functions, i.e. User-Defined Functions (UDFs). - -## Scalar Functions - -When writing a user-defined function that can operate on a row by row basis, these are called Scalar -Functions. You can define your own scalar function by calling -{py:func}`~datafusion.user_defined.ScalarUDF.udf` . - -The basic definition of a scalar UDF is a python function that takes one or more -[pyarrow](https://arrow.apache.org/docs/python/index.html) arrays and returns a single array as -output. DataFusion scalar UDFs operate on an entire batch of records at a time, though the -evaluation of those records should be on a row by row basis. In the following example, we compute -if the input array contains null values. - -```{eval-rst} -.. ipython:: python - - import pyarrow - import datafusion - from datafusion import udf, col - - def is_null(array: pyarrow.Array) -> pyarrow.Array: - return array.is_null() - - is_null_arr = udf(is_null, [pyarrow.int64()], pyarrow.bool_(), 'stable') - - ctx = datafusion.SessionContext() - - batch = pyarrow.RecordBatch.from_arrays( - [pyarrow.array([1, None, 3]), pyarrow.array([4, 5, 6])], - names=["a", "b"], - ) - df = ctx.create_dataframe([[batch]], name="batch_array") - - df.select(col("a"), is_null_arr(col("a")).alias("is_null")).show() -``` - -In the previous example, we used the fact that pyarrow provides a variety of built in array -functions such as `is_null()`. There are additional pyarrow -[compute functions](https://arrow.apache.org/docs/python/compute.html) available. When possible, -it is highly recommended to use these functions because they can perform computations without doing -any copy operations from the original arrays. This leads to greatly improved performance. - -If you need to perform an operation in python that is not available with the pyarrow compute -functions, you will need to convert the record batch into python values, perform your operation, -and construct an array. This operation of converting the built in data type of the array into a -python object can be one of the slowest operations in DataFusion, so it should be done sparingly. - -The following example performs the same operation as before with `is_null` but demonstrates -converting to Python objects to do the evaluation. - -```{eval-rst} -.. ipython:: python - - import pyarrow - import datafusion - from datafusion import udf, col - - def is_null(array: pyarrow.Array) -> pyarrow.Array: - return pyarrow.array([value.as_py() is None for value in array]) - - is_null_arr = udf(is_null, [pyarrow.int64()], pyarrow.bool_(), 'stable') - - ctx = datafusion.SessionContext() - - batch = pyarrow.RecordBatch.from_arrays( - [pyarrow.array([1, None, 3]), pyarrow.array([4, 5, 6])], - names=["a", "b"], - ) - df = ctx.create_dataframe([[batch]], name="batch_array") - - df.select(col("a"), is_null_arr(col("a")).alias("is_null")).show() -``` - -In this example we passed the PyArrow `DataType` when we defined the function -by calling `udf()`. If you need additional control, such as specifying -metadata or nullability of the input or output, you can instead specify a -PyArrow `Field`. - -If you need to write a custom function but do not want to incur the performance -cost of converting to Python objects and back, a more advanced approach is to -write Rust based UDFs and to expose them to Python. There is an example in the -[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/) -describing how to do this. - -### When not to use a UDF - -A UDF is the right tool when the per-row computation genuinely cannot be -expressed with DataFusion's built-in expressions. It is often the *wrong* -tool for a predicate that *can* be written as an `Expr` tree but feels -easier to write as a Python function — for example, a filter that keeps -a row if it matches any one of several rule sets, where each rule set -checks its own combination of columns (the worked example at the end of -this section keeps a row when it matches any one of several brand-specific -rules). Looping over the rules in Python and returning a boolean per row -reads naturally and is tempting to wrap in a UDF, but a UDF is opaque to -the optimizer: filters expressed as UDFs lose several rewrites that the -engine applies to filters built from native expressions. The most visible -of these is **predicate pushdown into the table provider**: a native -predicate can be handed to the source so it skips data before it is read, -while a UDF predicate cannot. The example below uses Parquet, where -pushdown prunes whole row groups using the min/max statistics in the -footer, but the same mechanism applies to any table provider that -advertises filter support — including custom providers. - -The following example writes a small Parquet file, then filters it two -ways: first with a native expression, then with a UDF that computes the -same result. The filter itself is simple on purpose so we can compare -the plans side by side. - -```{eval-rst} -.. ipython:: python - - import tempfile, os - import pyarrow as pa - import pyarrow.parquet as pq - from datafusion import SessionContext, col, lit, udf - - tmpdir = tempfile.mkdtemp() - parquet_path = os.path.join(tmpdir, "items.parquet") - pq.write_table( - pa.table({ - "id": list(range(100)), - "brand": ["A", "B", "C", "D"] * 25, - "qty": [i * 10 for i in range(100)], - }), - parquet_path, - ) - - ctx = SessionContext() - items = ctx.read_parquet(parquet_path) -``` - -**Native-expression predicate.** The filter is a plain boolean tree -over column references and literals, so the optimizer can analyze it: - -```{eval-rst} -.. ipython:: python - - native_filtered = items.filter( - (col("brand") == lit("A")) & (col("qty") >= lit(150)) - ) - print(native_filtered.execution_plan().display_indent()) -``` - -Notice the `DataSourceExec` line. It carries three annotations the -optimizer computed from the predicate: - -- `predicate=brand@1 = A AND qty@2 >= 150` — the filter is pushed - into the Parquet scan itself, so the scan only reads matching rows. -- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ... - qty_max@4 >= 150` — the scan prunes whole row groups by consulting - the Parquet min/max statistics in the footer *before* reading any - column data. -- `required_guarantees=[brand in (A)]` — the scan uses this when a - bloom filter or dictionary is available to skip pages. - -**UDF predicate.** Now wrap the same logic in a Python UDF: - -```{eval-rst} -.. ipython:: python - - def brand_qty_filter(brand_arr: pa.Array, qty_arr: pa.Array) -> pa.Array: - return pa.array([ - b.as_py() == "A" and q.as_py() >= 150 - for b, q in zip(brand_arr, qty_arr) - ]) - - pred_udf = udf( - brand_qty_filter, [pa.string(), pa.int64()], pa.bool_(), "stable", - ) - udf_filtered = items.filter(pred_udf(col("brand"), col("qty"))) - print(udf_filtered.execution_plan().display_indent()) -``` - -The `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`. -There is no `pruning_predicate` and no `required_guarantees`: the -scan has to materialize every row group and hand each row to the -Python callback just to decide whether to keep it. - -At small scale the cost difference is invisible; on a Parquet file with -many row groups, or data whose min/max statistics line up well with -the predicate, the native form can skip most of the file. The UDF form -reads all of it. - -**Takeaway.** Reach for a UDF when the per-row computation is genuinely -not expressible as a tree of built-in functions (custom numerical work, -external lookups, complex business rules). When it *is* expressible — -even if the native form is a little more verbose — build the `Expr` -tree directly so the optimizer can see through it. For disjunctive -predicates the idiom is to produce one clause per bucket and combine -them with `|`: - -```python -from functools import reduce -from operator import or_ -from datafusion import col, lit, functions as f - -buckets = { - "Brand#12": {"containers": ["SM CASE", "SM BOX"], "min_qty": 1, "max_size": 5}, - "Brand#23": {"containers": ["MED BAG", "MED BOX"], "min_qty": 10, "max_size": 10}, -} - -def bucket_clause(brand, spec): - return ( - (col("brand") == lit(brand)) - & f.in_list(col("container"), [lit(c) for c in spec["containers"]]) - & (col("quantity") >= lit(spec["min_qty"])) - & (col("quantity") <= lit(spec["min_qty"] + 10)) - & (col("size") >= lit(1)) - & (col("size") <= lit(spec["max_size"])) - ) - -predicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items())) -df = df.filter(predicate) -``` - -## Aggregate Functions - -The {py:func}`~datafusion.user_defined.AggregateUDF.udaf` function allows you to define User-Defined -Aggregate Functions (UDAFs). To use this you must implement an -{py:class}`~datafusion.user_defined.Accumulator` that determines how the aggregation is performed. - -When defining a UDAF there are four methods you need to implement. The `update` function takes the -array(s) of input and updates the internal state of the accumulator. You should define this function -to have as many input arguments as you will pass when calling the UDAF. Since aggregation may be -split into multiple batches, we must have a method to combine multiple batches. For this, we have -two functions, `state` and `merge`. `state` will return an array of scalar values that contain -the current state of a single batch accumulation. Then we must `merge` the results of these -different states. Finally `evaluate` is the call that will return the final result after the -`merge` is complete. - -In the following example we want to define a custom aggregate function that will return the -difference between the sum of two columns. The state can be represented by a single value and we can -also see how the inputs to `update` and `merge` differ. - -```python -import pyarrow as pa -import pyarrow.compute -import datafusion -from datafusion import col, udaf, Accumulator -from typing import List - -class MyAccumulator(Accumulator): - """ - Interface of a user-defined accumulation. - """ - def __init__(self): - self._sum = 0.0 - - def update(self, values_a: pa.Array, values_b: pa.Array) -> None: - self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py() - - def merge(self, states: list[pa.Array]) -> None: - self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py() - - def state(self) -> list[pa.Scalar]: - return [pyarrow.scalar(self._sum)] - - def evaluate(self) -> pa.Scalar: - return pyarrow.scalar(self._sum) - -ctx = datafusion.SessionContext() -df = ctx.from_pydict( - { - "a": [4, 5, 6], - "b": [1, 2, 3], - } -) - -my_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable') - -df.aggregate([], [my_udaf(col("a"), col("b")).alias("col_diff")]) -``` - -### FAQ - -**How do I return a list from a UDAF?** - -Both the `evaluate` and the `state` functions expect to return scalar values. -If you wish to return a list array as a scalar value, the best practice is to -wrap the values in a `pyarrow.Scalar` object. For example, you can return a -timestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp("ms")))` and -register the appropriate return or state types as -`return_type=pa.list_(pa.timestamp("ms"))` and -`state_type=[pa.list_(pa.timestamp("ms"))]`, respectively. - -As of DataFusion 52.0.0 , you can pass return any Python object, including a -PyArrow array, as the return value(s) for these functions and DataFusion will -attempt to create a scalar type from the value. DataFusion has been tested to -convert PyArrow, nanoarrow, and arro3 objects as well as primitive data types -like integers, strings, and so on. - -## Window Functions - -To implement a User-Defined Window Function (UDWF) you must call the -{py:func}`~datafusion.user_defined.WindowUDF.udwf` function using a class that implements the abstract -class {py:class}`~datafusion.user_defined.WindowEvaluator`. - -There are three methods of evaluation of UDWFs. - -- `evaluate` is the simplest case, where you are given an array and are expected to calculate the - value for a single row of that array. This is the simplest case, but also the least performant. -- `evaluate_all` computes the values for all rows for an input array at a single time. -- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank - information for the rows. - -Which methods you implement are based upon which of these options are set. - -```{eval-rst} -.. list-table:: - :header-rows: 1 - - * - ``uses_window_frame`` - - ``supports_bounded_execution`` - - ``include_rank`` - - function_to_implement - * - False (default) - - False (default) - - False (default) - - ``evaluate_all`` - * - False - - True - - False - - ``evaluate`` - * - False - - True - - False - - ``evaluate_all_with_rank`` - * - True - - True/False - - True/False - - ``evaluate`` -``` - -### UDWF options - -When you define your UDWF you can override the functions that return these values. They will -determine which evaluate functions are called. - -- `uses_window_frame` is set for functions that compute based on the specified window frame. If - your function depends upon the specified frame, set this to `True`. -- `supports_bounded_execution` specifies if your function can be incrementally computed. -- `include_rank` is set to `True` for window functions that can be computed only using the rank - information. - -```python -import pyarrow as pa -from datafusion import udwf, col, SessionContext -from datafusion.user_defined import WindowEvaluator - -class ExponentialSmooth(WindowEvaluator): - def __init__(self, alpha: float) -> None: - self.alpha = alpha - - def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array: - results = [] - curr_value = 0.0 - values = values[0] - for idx in range(num_rows): - if idx == 0: - curr_value = values[idx].as_py() - else: - curr_value = values[idx].as_py() * self.alpha + curr_value * ( - 1.0 - self.alpha - ) - results.append(curr_value) - - return pa.array(results) - -exp_smooth = udwf( - ExponentialSmooth(0.9), - pa.float64(), - pa.float64(), - volatility="immutable", -) - -ctx = SessionContext() - -df = ctx.from_pydict({ - "a": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0] -}) - -df.select("a", exp_smooth(col("a")).alias("smooth_a")).show() -``` - -## Table Functions - -User Defined Table Functions are slightly different than the other functions -described here. These functions take any number of `Expr` arguments, but only -literal expressions are supported. Table functions must return a Table -Provider as described in the ref:`_io_custom_table_provider` page. - -Once you have a table function, you can register it with the session context -by using {py:func}`datafusion.context.SessionContext.register_udtf`. - -There are examples of both rust backed and python based table functions in the -examples folder of the repository. If you have a rust backed table function -that you wish to expose via PyO3, you need to expose it as a `PyCapsule`. - -```rust -#[pymethods] -impl MyTableFunction { - fn __datafusion_table_function__<'py>( - &self, - py: Python<'py>, - ) -> PyResult> { - let name = cr"datafusion_table_function".into(); - - let func = self.clone(); - let provider = FFI_TableFunction::new(Arc::new(func), None); - - PyCapsule::new(py, provider, Some(name)) - } -} -``` - -### Accessing the Calling Session - -Pure-Python UDTFs can opt into receiving the calling -{py:class}`~datafusion.SessionContext` by registering with -`with_session=True`. The context is passed as a `session` keyword -argument on every invocation. Use it to look up registered tables, -UDFs, or session configuration from inside the callback. - -```python -from datafusion import SessionContext, Table, udtf -from datafusion.context import TableProviderExportable -import pyarrow as pa -import pyarrow.dataset as ds - -@udtf("list_tables", with_session=True) -def list_tables(*, session: SessionContext) -> TableProviderExportable: - names = sorted(session.catalog().schema().names()) - batch = pa.RecordBatch.from_pydict({"name": names}) - return Table(ds.dataset([batch])) - -ctx = SessionContext() -ctx.register_batch("t1", pa.RecordBatch.from_pydict({"x": [1]})) -ctx.register_udtf(list_tables) -ctx.sql("SELECT * FROM list_tables()").show() -``` - -Without `with_session=True`, the callback receives only the positional -expression arguments. The flag is opt-in so existing UDTFs keep working -unchanged. - -The injected `session` is a fresh {py:class}`~datafusion.SessionContext` -wrapper backed by the same underlying state as the caller, so registries -(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering -a new table or UDF) propagate to the live session because the registries -are reference-counted and shared. Configuration changes made through the -wrapper (e.g. setting session options) do **not** propagate — the wrapper -holds its own clone of the session config. diff --git a/docs/source/user-guide/common-operations/windows.ipynb b/docs/source/user-guide/common-operations/windows.ipynb new file mode 100644 index 000000000..dc3d2d584 --- /dev/null +++ b/docs/source/user-guide/common-operations/windows.ipynb @@ -0,0 +1,225 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n\n# Window Functions\n\nIn this section you will learn about window functions. A window function utilizes values from one or\nmultiple rows to produce a result for each individual row, unlike an aggregate function that\nprovides a single value for multiple rows.\n\nThe window functions are available in the [`functions`][datafusion.functions] module.\n\nWe'll use the pokemon dataset (from Ritchie Vink) in the following examples.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "df = ctx.read_csv(\"pokemon.csv\")" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\nHere is an example that shows how you can compare each pokemon's speed to the speed of the\nprevious row in the DataFrame.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(col('\"Name\"'), col('\"Speed\"'), f.lag(col('\"Speed\"')).alias(\"Previous Speed\"))" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\n## Setting Parameters\n\n### Ordering\n\nYou can control the order in which rows are processed by window functions by providing\na list of `order_by` functions for the `order_by` parameter.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(\n", + " col('\"Name\"'),\n", + " col('\"Attack\"'),\n", + " col('\"Type 1\"'),\n", + " f.rank(\n", + " partition_by=[col('\"Type 1\"')],\n", + " order_by=[col('\"Attack\"').sort(ascending=True)],\n", + " ).alias(\"rank\"),\n", + ").sort(col('\"Type 1\"'), col('\"Attack\"'))" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\n### Partitions\n\nA window function can take a list of `partition_by` columns similar to an\n[Aggregation Function](aggregation). This will cause the window values to be evaluated\nindependently for each of the partitions. In the example above, we found the rank of each\nPokemon per `Type 1` partitions. We can see the first couple of each partition if we do\nthe following:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(\n", + " col('\"Name\"'),\n", + " col('\"Attack\"'),\n", + " col('\"Type 1\"'),\n", + " f.rank(\n", + " partition_by=[col('\"Type 1\"')],\n", + " order_by=[col('\"Attack\"').sort(ascending=True)],\n", + " ).alias(\"rank\"),\n", + ").filter(col(\"rank\") < lit(3)).sort(col('\"Type 1\"'), col(\"rank\"))" + ] + }, + { + "cell_type": "markdown", + "id": "7cdc8c89c7104fffa095e18ddfef8986", + "metadata": {}, + "source": "\n### Window Frame\n\nWhen using aggregate functions, the Window Frame of defines the rows over which it operates.\nIf you do not specify a Window Frame, the frame will be set depending on the following\ncriteria.\n\n- If an `order_by` clause is set, the default window frame is defined as the rows between\n unbounded preceding and the current row.\n- If an `order_by` is not set, the default frame is defined as the rows between unbounded\n and unbounded following (the entire partition).\n\nWindow Frames are defined by three parameters: unit type, starting bound, and ending bound.\n\nThe unit types available are:\n\n- Rows: The starting and ending boundaries are defined by the number of rows relative to the\n current row.\n- Range: When using Range, the `order_by` clause must have exactly one term. The boundaries\n are defined bow how close the rows are to the value of the expression in the `order_by`\n parameter.\n- Groups: A \"group\" is the set of all rows that have equivalent values for all terms in the\n `order_by` clause.\n\nIn this example we perform a \"rolling average\" of the speed of the current Pokemon and the\ntwo preceding rows.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b118ea5561624da68c537baed56e602f", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion.expr import Window, WindowFrame\n", + "\n", + "df.select(\n", + " col('\"Name\"'),\n", + " col('\"Speed\"'),\n", + " f.avg(col('\"Speed\"'))\n", + " .over(Window(window_frame=WindowFrame(\"rows\", 2, 0), order_by=[col('\"Speed\"')]))\n", + " .alias(\"Previous Speed\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "938c804e27f84196a10c8828c723f798", + "metadata": {}, + "source": "\n### Null Treatment\n\nWhen using aggregate functions as window functions, it is often useful to specify how null values\nshould be treated. In order to do this you need to use the builder function. In future releases\nwe expect this to be simplified in the interface.\n\nOne common usage for handling nulls is the case where you want to find the last value up to the\ncurrent row. In the following example we demonstrate how setting the null treatment to ignore\nnulls will fill in with the value of the most recent non-null row. To do this, we also will set\nthe window frame so that we only process up to the current row.\n\nIn this example, we filter down to one specific type of Pokemon that does have some entries in\nit's `Type 2` column that are null.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "504fb2a444614c0babb325280ed9130a", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion.common import NullTreatment\n", + "\n", + "df.filter(col('\"Type 1\"') == lit(\"Bug\")).select(\n", + " '\"Name\"',\n", + " '\"Type 2\"',\n", + " f.last_value(col('\"Type 2\"'))\n", + " .over(\n", + " Window(\n", + " window_frame=WindowFrame(\"rows\", None, 0),\n", + " order_by=[col('\"Speed\"')],\n", + " null_treatment=NullTreatment.IGNORE_NULLS,\n", + " )\n", + " )\n", + " .alias(\"last_wo_null\"),\n", + " f.last_value(col('\"Type 2\"'))\n", + " .over(\n", + " Window(\n", + " window_frame=WindowFrame(\"rows\", None, 0),\n", + " order_by=[col('\"Speed\"')],\n", + " null_treatment=NullTreatment.RESPECT_NULLS,\n", + " )\n", + " )\n", + " .alias(\"last_with_null\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "59bbdb311c014d738909a11f9e486628", + "metadata": {}, + "source": "\n## Aggregate Functions\n\nYou can use any [Aggregation Function](aggregation) as a window function. Here\nis an example that shows how to compare each pokemons’s attack power with the average attack\npower in its `\"Type 1\"` using the [`avg`][datafusion.functions.avg] function.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b43b363d81ae4b689946ece5c682cd59", + "metadata": {}, + "outputs": [], + "source": [ + "df.select(\n", + " col('\"Name\"'),\n", + " col('\"Attack\"'),\n", + " col('\"Type 1\"'),\n", + " f.avg(col('\"Attack\"'))\n", + " .over(\n", + " Window(\n", + " window_frame=WindowFrame(\"rows\", None, None),\n", + " partition_by=[col('\"Type 1\"')],\n", + " )\n", + " )\n", + " .alias(\"Average Attack\"),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "8a65eabff63a45729fe45fb5ade58bdc", + "metadata": {}, + "source": "\n## Available Functions\n\nThe possible window functions are:\n\n1. Rank Functions\n : - [`rank`][datafusion.functions.rank]\n - [`dense_rank`][datafusion.functions.dense_rank]\n - [`ntile`][datafusion.functions.ntile]\n - [`row_number`][datafusion.functions.row_number]\n2. Analytical Functions\n : - [`cume_dist`][datafusion.functions.cume_dist]\n - [`percent_rank`][datafusion.functions.percent_rank]\n - [`lag`][datafusion.functions.lag]\n - [`lead`][datafusion.functions.lead]\n3. Aggregate Functions\n : - All [Aggregation Functions](aggregation) can be used as window functions.\n\n## User-Defined Window Functions\n\nYou can ship custom window functions to the engine by subclassing\n[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it\nvia [`udwf`][datafusion.udwf]. See [`user_defined`][datafusion.user_defined]\nfor the evaluator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python window UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions —\n the evaluator class is captured by value via {mod}`cloudpickle`, so\n worker processes do not need to pre-register the UDF. Any names the\n evaluator resolves via `import` are captured **by reference** and\n must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/common-operations/windows.md b/docs/source/user-guide/common-operations/windows.md deleted file mode 100644 index 267218003..000000000 --- a/docs/source/user-guide/common-operations/windows.md +++ /dev/null @@ -1,239 +0,0 @@ - - -(window_functions)= - -# Window Functions - -In this section you will learn about window functions. A window function utilizes values from one or -multiple rows to produce a result for each individual row, unlike an aggregate function that -provides a single value for multiple rows. - -The window functions are available in the {py:mod}`~datafusion.functions` module. - -We'll use the pokemon dataset (from Ritchie Vink) in the following examples. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - from datafusion import col, lit - from datafusion import functions as f - - ctx = SessionContext() - df = ctx.read_csv("pokemon.csv") -``` - -Here is an example that shows how you can compare each pokemon's speed to the speed of the -previous row in the DataFrame. - -```{eval-rst} -.. ipython:: python - - df.select( - col('"Name"'), - col('"Speed"'), - f.lag(col('"Speed"')).alias("Previous Speed") - ) -``` - -## Setting Parameters - -### Ordering - -You can control the order in which rows are processed by window functions by providing -a list of `order_by` functions for the `order_by` parameter. - -```{eval-rst} -.. ipython:: python - - df.select( - col('"Name"'), - col('"Attack"'), - col('"Type 1"'), - f.rank( - partition_by=[col('"Type 1"')], - order_by=[col('"Attack"').sort(ascending=True)], - ).alias("rank"), - ).sort(col('"Type 1"'), col('"Attack"')) -``` - -### Partitions - -A window function can take a list of `partition_by` columns similar to an -{ref}`Aggregation Function`. This will cause the window values to be evaluated -independently for each of the partitions. In the example above, we found the rank of each -Pokemon per `Type 1` partitions. We can see the first couple of each partition if we do -the following: - -```{eval-rst} -.. ipython:: python - - df.select( - col('"Name"'), - col('"Attack"'), - col('"Type 1"'), - f.rank( - partition_by=[col('"Type 1"')], - order_by=[col('"Attack"').sort(ascending=True)], - ).alias("rank"), - ).filter(col("rank") < lit(3)).sort(col('"Type 1"'), col("rank")) -``` - -### Window Frame - -When using aggregate functions, the Window Frame of defines the rows over which it operates. -If you do not specify a Window Frame, the frame will be set depending on the following -criteria. - -- If an `order_by` clause is set, the default window frame is defined as the rows between - unbounded preceding and the current row. -- If an `order_by` is not set, the default frame is defined as the rows between unbounded - and unbounded following (the entire partition). - -Window Frames are defined by three parameters: unit type, starting bound, and ending bound. - -The unit types available are: - -- Rows: The starting and ending boundaries are defined by the number of rows relative to the - current row. -- Range: When using Range, the `order_by` clause must have exactly one term. The boundaries - are defined bow how close the rows are to the value of the expression in the `order_by` - parameter. -- Groups: A "group" is the set of all rows that have equivalent values for all terms in the - `order_by` clause. - -In this example we perform a "rolling average" of the speed of the current Pokemon and the -two preceding rows. - -```{eval-rst} -.. ipython:: python - - from datafusion.expr import Window, WindowFrame - - df.select( - col('"Name"'), - col('"Speed"'), - f.avg(col('"Speed"')) - .over(Window(window_frame=WindowFrame("rows", 2, 0), order_by=[col('"Speed"')])) - .alias("Previous Speed"), - ) -``` - -### Null Treatment - -When using aggregate functions as window functions, it is often useful to specify how null values -should be treated. In order to do this you need to use the builder function. In future releases -we expect this to be simplified in the interface. - -One common usage for handling nulls is the case where you want to find the last value up to the -current row. In the following example we demonstrate how setting the null treatment to ignore -nulls will fill in with the value of the most recent non-null row. To do this, we also will set -the window frame so that we only process up to the current row. - -In this example, we filter down to one specific type of Pokemon that does have some entries in -it's `Type 2` column that are null. - -```{eval-rst} -.. ipython:: python - - from datafusion.common import NullTreatment - - df.filter(col('"Type 1"') == lit("Bug")).select( - '"Name"', - '"Type 2"', - f.last_value(col('"Type 2"')) - .over( - Window( - window_frame=WindowFrame("rows", None, 0), - order_by=[col('"Speed"')], - null_treatment=NullTreatment.IGNORE_NULLS, - ) - ) - .alias("last_wo_null"), - f.last_value(col('"Type 2"')) - .over( - Window( - window_frame=WindowFrame("rows", None, 0), - order_by=[col('"Speed"')], - null_treatment=NullTreatment.RESPECT_NULLS, - ) - ) - .alias("last_with_null"), - ) -``` - -## Aggregate Functions - -You can use any {ref}`Aggregation Function` as a window function. Here -is an example that shows how to compare each pokemons’s attack power with the average attack -power in its `"Type 1"` using the {py:func}`datafusion.functions.avg` function. - -```{eval-rst} -.. ipython:: python - :okwarning: - - df.select( - col('"Name"'), - col('"Attack"'), - col('"Type 1"'), - f.avg(col('"Attack"')).over( - Window( - window_frame=WindowFrame("rows", None, None), - partition_by=[col('"Type 1"')], - ) - ).alias("Average Attack"), - ) -``` - -## Available Functions - -The possible window functions are: - -1. Rank Functions - : - {py:func}`datafusion.functions.rank` - - {py:func}`datafusion.functions.dense_rank` - - {py:func}`datafusion.functions.ntile` - - {py:func}`datafusion.functions.row_number` -2. Analytical Functions - : - {py:func}`datafusion.functions.cume_dist` - - {py:func}`datafusion.functions.percent_rank` - - {py:func}`datafusion.functions.lag` - - {py:func}`datafusion.functions.lead` -3. Aggregate Functions - : - All {ref}`Aggregation Functions` can be used as window functions. - -## User-Defined Window Functions - -You can ship custom window functions to the engine by subclassing -{py:class}`~datafusion.user_defined.WindowEvaluator` and registering it -via {py:func}`~datafusion.udwf`. See {py:mod}`datafusion.user_defined` -for the evaluator interface and worked examples. - -:::{note} -Serialization - -Python window UDFs travel inline inside pickled or -{py:meth}`~datafusion.expr.Expr.to_bytes`-serialized expressions — -the evaluator class is captured by value via {mod}`cloudpickle`, so -worker processes do not need to pre-register the UDF. Any names the -evaluator resolves via `import` are captured **by reference** and -must be importable on the receiving worker. See -{py:mod}`datafusion.ipc` for the full IPC model and security caveats. -::: diff --git a/docs/source/user-guide/configuration.md b/docs/source/user-guide/configuration.md index d1c5c9b44..09dd70f6b 100644 --- a/docs/source/user-guide/configuration.md +++ b/docs/source/user-guide/configuration.md @@ -17,12 +17,11 @@ under the License. --> -(configuration)= # Configuration -Let's look at how we can configure DataFusion. When creating a {py:class}`~datafusion.context.SessionContext`, you can pass in -a {py:class}`~datafusion.context.SessionConfig` and {py:class}`~datafusion.context.RuntimeEnvBuilder` object. These two cover a wide range of options. +Let's look at how we can configure DataFusion. When creating a [`SessionContext`][datafusion.context.SessionContext], you can pass in +a [`SessionConfig`][datafusion.context.SessionConfig] and [`RuntimeEnvBuilder`][datafusion.context.RuntimeEnvBuilder] object. These two cover a wide range of options. ```python from datafusion import RuntimeEnvBuilder, SessionConfig, SessionContext @@ -99,7 +98,7 @@ result = df.collect() ### Benchmark Example The repository includes a benchmark script that demonstrates how to maximize CPU usage -with DataFusion. The {code}`benchmarks/max_cpu_usage.py` script shows a practical example +with DataFusion. The `benchmarks/max_cpu_usage.py` script shows a practical example of configuring DataFusion for optimal parallelism. You can run the benchmark script to see the impact of different configuration settings: @@ -133,9 +132,9 @@ CPU utilization and query performance. The script demonstrates several key optimization techniques: -1. **Higher target partition count**: Uses {code}`with_target_partitions()` to set the number of concurrent partitions +1. **Higher target partition count**: Uses `with_target_partitions()` to set the number of concurrent partitions 2. **Automatic repartitioning**: Enables repartitioning for joins, aggregations, and window functions -3. **Manual repartitioning**: Uses {code}`repartition()` to ensure all partitions are utilized +3. **Manual repartitioning**: Uses `repartition()` to ensure all partitions are utilized 4. **CPU-intensive operations**: Performs aggregations that can benefit from parallelization The benchmark creates synthetic data and measures the time taken to perform a sum aggregation @@ -181,5 +180,5 @@ To optimize DataFusion for your specific use case, it is strongly recommended to This approach will provide more accurate insights into how DataFusion configuration options will impact your particular applications and infrastructure. -For more information about available {py:class}`~datafusion.context.SessionConfig` options, see the [rust DataFusion Configuration guide](https://arrow.apache.org/datafusion/user-guide/configs.html), -and about {code}`RuntimeEnvBuilder` options in the rust [online API documentation](https://docs.rs/datafusion/latest/datafusion/execution/runtime_env/struct.RuntimeEnvBuilder.html). +For more information about available [`SessionConfig`][datafusion.context.SessionConfig] options, see the [rust DataFusion Configuration guide](https://arrow.apache.org/datafusion/user-guide/configs.html), +and about `RuntimeEnvBuilder` options in the rust [online API documentation](https://docs.rs/datafusion/latest/datafusion/execution/runtime_env/struct.RuntimeEnvBuilder.html). diff --git a/docs/source/user-guide/data-sources.ipynb b/docs/source/user-guide/data-sources.ipynb new file mode 100644 index 000000000..6374e4cf1 --- /dev/null +++ b/docs/source/user-guide/data-sources.ipynb @@ -0,0 +1,114 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n\n# Data Sources\n\nDataFusion provides a wide variety of ways to get data into a DataFrame to perform operations.\n\n## Local file\n\nDataFusion has the ability to read from a variety of popular file formats, such as [Parquet](io_parquet),\n[CSV](io_csv), [JSON](io_json), and [AVRO](io_avro).\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "df = ctx.read_csv(\"pokemon.csv\")\n", + "df.show()" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\n## Create in-memory\n\nSometimes it can be convenient to create a small DataFrame from a Python list or dictionary object.\nTo do this in DataFusion, you can use one of the three functions\n[`from_pydict`][datafusion.context.SessionContext.from_pydict],\n[`from_pylist`][datafusion.context.SessionContext.from_pylist], or\n[`create_dataframe`][datafusion.context.SessionContext.create_dataframe].\n\nAs their names suggest, `from_pydict` and `from_pylist` will create DataFrames from Python\ndictionary and list objects, respectively. `create_dataframe` assumes you will pass in a list\nof list of [PyArrow Record Batches](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html).\n\nThe following three examples all will create identical DataFrames:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "import pyarrow as pa\n", + "\n", + "ctx.from_pylist(\n", + " [\n", + " {\"a\": 1, \"b\": 10.0, \"c\": \"alpha\"},\n", + " {\"a\": 2, \"b\": 20.0, \"c\": \"beta\"},\n", + " {\"a\": 3, \"b\": 30.0, \"c\": \"gamma\"},\n", + " ]\n", + ").show()\n", + "\n", + "ctx.from_pydict(\n", + " {\n", + " \"a\": [1, 2, 3],\n", + " \"b\": [10.0, 20.0, 30.0],\n", + " \"c\": [\"alpha\", \"beta\", \"gamma\"],\n", + " }\n", + ").show()\n", + "\n", + "batch = pa.RecordBatch.from_arrays(\n", + " [\n", + " pa.array([1, 2, 3]),\n", + " pa.array([10.0, 20.0, 30.0]),\n", + " pa.array([\"alpha\", \"beta\", \"gamma\"]),\n", + " ],\n", + " names=[\"a\", \"b\", \"c\"],\n", + ")\n", + "\n", + "ctx.create_dataframe([[batch]]).show()" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\n## Object Store\n\nDataFusion has support for multiple storage options in addition to local files.\nThe example below requires an appropriate S3 account with access credentials.\n\nSupported Object Stores are\n\n- [`AmazonS3`][datafusion.object_store.AmazonS3]\n- [`GoogleCloud`][datafusion.object_store.GoogleCloud]\n- [`Http`][datafusion.object_store.Http]\n- [`LocalFileSystem`][datafusion.object_store.LocalFileSystem]\n- [`MicrosoftAzure`][datafusion.object_store.MicrosoftAzure]\n\n```python\nfrom datafusion.object_store import AmazonS3\n\nregion = \"us-east-1\"\nbucket_name = \"yellow-trips\"\n\ns3 = AmazonS3(\n bucket_name=bucket_name,\n region=region,\n access_key_id=os.getenv(\"AWS_ACCESS_KEY_ID\"),\n secret_access_key=os.getenv(\"AWS_SECRET_ACCESS_KEY\"),\n)\n\npath = f\"s3://{bucket_name}/\"\nctx.register_object_store(\"s3://\", s3, None)\n\nctx.register_parquet(\"trips\", path)\n\nctx.table(\"trips\").show()\n```\n\n## Other DataFrame Libraries\n\nDataFusion can import DataFrames directly from other libraries, such as\n[Polars](https://pola.rs/) and [Pandas](https://pandas.pydata.org/).\nSince DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule\ninterface can be imported to DataFusion using the\n[`from_arrow`][datafusion.context.SessionContext.from_arrow] function. Older versions of Polars may\nnot support the arrow interface. In those cases, you can still import via the\n[`from_polars`][datafusion.context.SessionContext.from_polars] function.\n\n```python\nimport pandas as pd\n\ndata = { \"a\": [1, 2, 3], \"b\": [10.0, 20.0, 30.0], \"c\": [\"alpha\", \"beta\", \"gamma\"] }\npandas_df = pd.DataFrame(data)\n\ndatafusion_df = ctx.from_arrow(pandas_df)\ndatafusion_df.show()\n```\n\n```python\nimport polars as pl\npolars_df = pl.DataFrame(data)\n\ndatafusion_df = ctx.from_arrow(polars_df)\ndatafusion_df.show()\n```\n\n## Delta Lake\n\nDataFusion 43.0.0 and later support the ability to register table providers from sources such\nas Delta Lake. This will require a recent version of\n[deltalake](https://delta-io.github.io/delta-rs/) to provide the required interfaces.\n\n```python\nfrom deltalake import DeltaTable\n\ndelta_table = DeltaTable(\"path_to_table\")\nctx.register_table(\"my_delta_table\", delta_table)\ndf = ctx.table(\"my_delta_table\")\ndf.show()\n```\n\nOn older versions of `deltalake` (prior to 0.22) you can use the\n[Arrow DataSet](https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Dataset.html)\ninterface to import to DataFusion, but this does not support features such as filter push down\nwhich can lead to a significant performance difference.\n\n```python\nfrom deltalake import DeltaTable\n\ndelta_table = DeltaTable(\"path_to_table\")\nctx.register_dataset(\"my_delta_table\", delta_table.to_pyarrow_dataset())\ndf = ctx.table(\"my_delta_table\")\ndf.show()\n```\n\n## Apache Iceberg\n\nDataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface.\n\nThis requires either the [pyiceberg](https://pypi.org/project/pyiceberg/) library (>=0.10.0) or the [pyiceberg-core](https://pypi.org/project/pyiceberg-core/) library (>=0.5.0).\n\n- The `pyiceberg-core` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings.\n- The `pyiceberg` library utilizes the `pyiceberg-core` python bindings under the hood and provides a native way for Python users to interact with the DataFusion.\n\n```python\nfrom datafusion import SessionContext\nfrom pyiceberg.catalog import load_catalog\nimport pyarrow as pa\n\n# Load catalog and create/load a table\ncatalog = load_catalog(\"catalog\", type=\"in-memory\")\ncatalog.create_namespace_if_not_exists(\"default\")\n\n# Create some sample data\ndata = pa.table({\"x\": [1, 2, 3], \"y\": [4, 5, 6]})\niceberg_table = catalog.create_table(\"default.test\", schema=data.schema)\niceberg_table.append(data)\n\n# Register the table with DataFusion\nctx = SessionContext()\nctx.register_table_provider(\"test\", iceberg_table)\n\n# Query the table using DataFusion\nctx.table(\"test\").show()\n```\n\nNote that the Datafusion integration rely on features from the [Iceberg Rust](https://github.com/apache/iceberg-rust/) implementation instead of the [PyIceberg](https://github.com/apache/iceberg-python/) implementation.\nFeatures that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion.\n\n## Custom Table Provider\n\nYou can implement a custom Data Provider in Rust and expose it to DataFusion through the\nthe interface as describe in the [Custom Table Provider](io_custom_table_provider)\nsection. This is an advanced topic, but a\n[user example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example)\nis provided in the DataFusion repository.\n\n# Catalog\n\nA common technique for organizing tables is using a three level hierarchical approach. DataFusion\nsupports this form of organizing using the [`Catalog`][datafusion.catalog.Catalog],\n[`Schema`][datafusion.catalog.Schema], and [`Table`][datafusion.catalog.Table]. By default,\na [`SessionContext`][datafusion.context.SessionContext] comes with a single Catalog and a single Schema\nwith the names `datafusion` and `public`, respectively.\n\nThe default implementation uses an in-memory approach to the catalog and schema. We have support\nfor adding additional in-memory catalogs and schemas. You can access tables registered in a schema\neither through the Dataframe API or via sql commands. This can be done like in the following\nexample:\n\n```python\nimport pyarrow as pa\nfrom datafusion.catalog import Catalog, Schema\nfrom datafusion import SessionContext\n\nctx = SessionContext()\n\nmy_catalog = Catalog.memory_catalog()\nmy_schema = Schema.memory_schema()\nmy_catalog.register_schema('my_schema_name', my_schema)\nctx.register_catalog_provider('my_catalog_name', my_catalog)\n\n# Create an in-memory table\ntable = pa.table({\n 'name': ['Bulbasaur', 'Charmander', 'Squirtle'],\n 'type': ['Grass', 'Fire', 'Water'],\n 'hp': [45, 39, 44],\n})\ndf = ctx.create_dataframe([table.to_batches()], name='pokemon')\n\nmy_schema.register_table('pokemon', df)\n\nctx.sql('SELECT * FROM my_catalog_name.my_schema_name.pokemon').show()\n```\n\n## User Defined Catalog and Schema\n\nIf the in-memory catalogs are insufficient for your uses, there are two approaches you can take\nto implementing a custom catalog and/or schema. In the below discussion, we describe how to\nimplement these for a Catalog, but the approach to implementing for a Schema is nearly\nidentical.\n\nDataFusion supports Catalogs written in either Rust or Python. If you write a Catalog in Rust,\nyou will need to export it as a Python library via PyO3. There is a complete example of a\ncatalog implemented this way in the\n[examples folder](https://github.com/apache/datafusion-python/tree/main/examples/)\nof our repository. Writing catalog providers in Rust provides typically can lead to significant\nperformance improvements over the Python based approach.\n\nTo implement a Catalog in Python, you will need to inherit from the abstract base class\n[`CatalogProvider`][datafusion.catalog.CatalogProvider]. There are examples in the\n[unit tests](https://github.com/apache/datafusion-python/tree/main/python/tests) of\nimplementing a basic Catalog in Python where we simply keep a dictionary of the\nregistered Schemas.\n\nOne important note for developers is that when we have a Catalog defined in Python, we have\ntwo different ways of accessing this Catalog. First, we register the catalog with a Rust\nwrapper. This allows for any rust based code to call the Python functions as necessary.\nSecond, if the user access the Catalog via the Python API, we identify this and return back\nthe original Python object that implements the Catalog. This is an important distinction\nfor developers because we do *not* return a Python wrapper around the Rust wrapper of the\noriginal Python object.\n" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/data-sources.md b/docs/source/user-guide/data-sources.md deleted file mode 100644 index 425e9402f..000000000 --- a/docs/source/user-guide/data-sources.md +++ /dev/null @@ -1,281 +0,0 @@ - - -(user_guide_data_sources)= - -# Data Sources - -DataFusion provides a wide variety of ways to get data into a DataFrame to perform operations. - -## Local file - -DataFusion has the ability to read from a variety of popular file formats, such as {ref}`Parquet `, -{ref}`CSV `, {ref}`JSON `, and {ref}`AVRO `. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - ctx = SessionContext() - df = ctx.read_csv("pokemon.csv") - df.show() -``` - -## Create in-memory - -Sometimes it can be convenient to create a small DataFrame from a Python list or dictionary object. -To do this in DataFusion, you can use one of the three functions -{py:func}`~datafusion.context.SessionContext.from_pydict`, -{py:func}`~datafusion.context.SessionContext.from_pylist`, or -{py:func}`~datafusion.context.SessionContext.create_dataframe`. - -As their names suggest, `from_pydict` and `from_pylist` will create DataFrames from Python -dictionary and list objects, respectively. `create_dataframe` assumes you will pass in a list -of list of [PyArrow Record Batches](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html). - -The following three examples all will create identical DataFrames: - -```{eval-rst} -.. ipython:: python - - import pyarrow as pa - - ctx.from_pylist([ - { "a": 1, "b": 10.0, "c": "alpha" }, - { "a": 2, "b": 20.0, "c": "beta" }, - { "a": 3, "b": 30.0, "c": "gamma" }, - ]).show() - - ctx.from_pydict({ - "a": [1, 2, 3], - "b": [10.0, 20.0, 30.0], - "c": ["alpha", "beta", "gamma"], - }).show() - - batch = pa.RecordBatch.from_arrays( - [ - pa.array([1, 2, 3]), - pa.array([10.0, 20.0, 30.0]), - pa.array(["alpha", "beta", "gamma"]), - ], - names=["a", "b", "c"], - ) - - ctx.create_dataframe([[batch]]).show() - -``` - -## Object Store - -DataFusion has support for multiple storage options in addition to local files. -The example below requires an appropriate S3 account with access credentials. - -Supported Object Stores are - -- {py:class}`~datafusion.object_store.AmazonS3` -- {py:class}`~datafusion.object_store.GoogleCloud` -- {py:class}`~datafusion.object_store.Http` -- {py:class}`~datafusion.object_store.LocalFileSystem` -- {py:class}`~datafusion.object_store.MicrosoftAzure` - -```python -from datafusion.object_store import AmazonS3 - -region = "us-east-1" -bucket_name = "yellow-trips" - -s3 = AmazonS3( - bucket_name=bucket_name, - region=region, - access_key_id=os.getenv("AWS_ACCESS_KEY_ID"), - secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"), -) - -path = f"s3://{bucket_name}/" -ctx.register_object_store("s3://", s3, None) - -ctx.register_parquet("trips", path) - -ctx.table("trips").show() -``` - -## Other DataFrame Libraries - -DataFusion can import DataFrames directly from other libraries, such as -[Polars](https://pola.rs/) and [Pandas](https://pandas.pydata.org/). -Since DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule -interface can be imported to DataFusion using the -{py:func}`~datafusion.context.SessionContext.from_arrow` function. Older versions of Polars may -not support the arrow interface. In those cases, you can still import via the -{py:func}`~datafusion.context.SessionContext.from_polars` function. - -```python -import pandas as pd - -data = { "a": [1, 2, 3], "b": [10.0, 20.0, 30.0], "c": ["alpha", "beta", "gamma"] } -pandas_df = pd.DataFrame(data) - -datafusion_df = ctx.from_arrow(pandas_df) -datafusion_df.show() -``` - -```python -import polars as pl -polars_df = pl.DataFrame(data) - -datafusion_df = ctx.from_arrow(polars_df) -datafusion_df.show() -``` - -## Delta Lake - -DataFusion 43.0.0 and later support the ability to register table providers from sources such -as Delta Lake. This will require a recent version of -[deltalake](https://delta-io.github.io/delta-rs/) to provide the required interfaces. - -```python -from deltalake import DeltaTable - -delta_table = DeltaTable("path_to_table") -ctx.register_table("my_delta_table", delta_table) -df = ctx.table("my_delta_table") -df.show() -``` - -On older versions of `deltalake` (prior to 0.22) you can use the -[Arrow DataSet](https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Dataset.html) -interface to import to DataFusion, but this does not support features such as filter push down -which can lead to a significant performance difference. - -```python -from deltalake import DeltaTable - -delta_table = DeltaTable("path_to_table") -ctx.register_dataset("my_delta_table", delta_table.to_pyarrow_dataset()) -df = ctx.table("my_delta_table") -df.show() -``` - -## Apache Iceberg - -DataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface. - -This requires either the [pyiceberg](https://pypi.org/project/pyiceberg/) library (>=0.10.0) or the [pyiceberg-core](https://pypi.org/project/pyiceberg-core/) library (>=0.5.0). - -- The `pyiceberg-core` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings. -- The `pyiceberg` library utilizes the `pyiceberg-core` python bindings under the hood and provides a native way for Python users to interact with the DataFusion. - -```python -from datafusion import SessionContext -from pyiceberg.catalog import load_catalog -import pyarrow as pa - -# Load catalog and create/load a table -catalog = load_catalog("catalog", type="in-memory") -catalog.create_namespace_if_not_exists("default") - -# Create some sample data -data = pa.table({"x": [1, 2, 3], "y": [4, 5, 6]}) -iceberg_table = catalog.create_table("default.test", schema=data.schema) -iceberg_table.append(data) - -# Register the table with DataFusion -ctx = SessionContext() -ctx.register_table_provider("test", iceberg_table) - -# Query the table using DataFusion -ctx.table("test").show() -``` - -Note that the Datafusion integration rely on features from the [Iceberg Rust](https://github.com/apache/iceberg-rust/) implementation instead of the [PyIceberg](https://github.com/apache/iceberg-python/) implementation. -Features that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion. - -## Custom Table Provider - -You can implement a custom Data Provider in Rust and expose it to DataFusion through the -the interface as describe in the {ref}`Custom Table Provider ` -section. This is an advanced topic, but a -[user example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example) -is provided in the DataFusion repository. - -# Catalog - -A common technique for organizing tables is using a three level hierarchical approach. DataFusion -supports this form of organizing using the {py:class}`~datafusion.catalog.Catalog`, -{py:class}`~datafusion.catalog.Schema`, and {py:class}`~datafusion.catalog.Table`. By default, -a {py:class}`~datafusion.context.SessionContext` comes with a single Catalog and a single Schema -with the names `datafusion` and `public`, respectively. - -The default implementation uses an in-memory approach to the catalog and schema. We have support -for adding additional in-memory catalogs and schemas. You can access tables registered in a schema -either through the Dataframe API or via sql commands. This can be done like in the following -example: - -```python -import pyarrow as pa -from datafusion.catalog import Catalog, Schema -from datafusion import SessionContext - -ctx = SessionContext() - -my_catalog = Catalog.memory_catalog() -my_schema = Schema.memory_schema() -my_catalog.register_schema('my_schema_name', my_schema) -ctx.register_catalog_provider('my_catalog_name', my_catalog) - -# Create an in-memory table -table = pa.table({ - 'name': ['Bulbasaur', 'Charmander', 'Squirtle'], - 'type': ['Grass', 'Fire', 'Water'], - 'hp': [45, 39, 44], -}) -df = ctx.create_dataframe([table.to_batches()], name='pokemon') - -my_schema.register_table('pokemon', df) - -ctx.sql('SELECT * FROM my_catalog_name.my_schema_name.pokemon').show() -``` - -## User Defined Catalog and Schema - -If the in-memory catalogs are insufficient for your uses, there are two approaches you can take -to implementing a custom catalog and/or schema. In the below discussion, we describe how to -implement these for a Catalog, but the approach to implementing for a Schema is nearly -identical. - -DataFusion supports Catalogs written in either Rust or Python. If you write a Catalog in Rust, -you will need to export it as a Python library via PyO3. There is a complete example of a -catalog implemented this way in the -[examples folder](https://github.com/apache/datafusion-python/tree/main/examples/) -of our repository. Writing catalog providers in Rust provides typically can lead to significant -performance improvements over the Python based approach. - -To implement a Catalog in Python, you will need to inherit from the abstract base class -{py:class}`~datafusion.catalog.CatalogProvider`. There are examples in the -[unit tests](https://github.com/apache/datafusion-python/tree/main/python/tests) of -implementing a basic Catalog in Python where we simply keep a dictionary of the -registered Schemas. - -One important note for developers is that when we have a Catalog defined in Python, we have -two different ways of accessing this Catalog. First, we register the catalog with a Rust -wrapper. This allows for any rust based code to call the Python functions as necessary. -Second, if the user access the Catalog via the Python API, we identify this and return back -the original Python object that implements the Catalog. This is an important distinction -for developers because we do *not* return a Python wrapper around the Rust wrapper of the -original Python object. diff --git a/docs/source/user-guide/dataframe/execution-metrics.md b/docs/source/user-guide/dataframe/execution-metrics.md index ede3339e0..d44a46bc6 100644 --- a/docs/source/user-guide/dataframe/execution-metrics.md +++ b/docs/source/user-guide/dataframe/execution-metrics.md @@ -17,7 +17,6 @@ under the License. --> -(execution_metrics)= # Execution Metrics @@ -38,67 +37,67 @@ Typical metrics include: Metrics are collected *per-partition*: DataFusion may execute each operator in parallel across several partitions. The convenience properties on -{py:class}`~datafusion.MetricsSet` (e.g. `output_rows`, `elapsed_compute`) +[`MetricsSet`][datafusion.MetricsSet] (e.g. `output_rows`, `elapsed_compute`) automatically sum the named metric across **all** partitions, giving a single aggregate value for the operator as a whole. You can also access the raw -per-partition {py:class}`~datafusion.Metric` objects via -{py:meth}`~datafusion.MetricsSet.metrics`. +per-partition [`Metric`][datafusion.Metric] objects via +[`metrics`][datafusion.MetricsSet.metrics]. ## When Are Metrics Available? Some operators (for example `DataSourceExec`) eagerly create a -{py:class}`~datafusion.MetricsSet` when the physical plan is built, so -{py:meth}`~datafusion.ExecutionPlan.metrics` may return a set even before any +[`MetricsSet`][datafusion.MetricsSet] when the physical plan is built, so +[`metrics`][datafusion.ExecutionPlan.metrics] may return a set even before any rows have been processed. However, metric **values** such as `output_rows` are only meaningful **after** the DataFrame has been executed via one of the terminal operations: -- {py:meth}`~datafusion.DataFrame.collect` -- {py:meth}`~datafusion.DataFrame.collect_partitioned` -- {py:meth}`~datafusion.DataFrame.execute_stream` +- [`collect`][datafusion.DataFrame.collect] +- [`collect_partitioned`][datafusion.DataFrame.collect_partitioned] +- [`execute_stream`][datafusion.DataFrame.execute_stream] (metrics are available once the stream has been fully consumed) -- {py:meth}`~datafusion.DataFrame.execute_stream_partitioned` +- [`execute_stream_partitioned`][datafusion.DataFrame.execute_stream_partitioned] (metrics are available once all partition streams have been fully consumed) Before execution, metric values will be `0` or `None`. -:::{note} -**display() does not populate metrics.** -When a DataFrame is displayed in a notebook (e.g. via `display(df)` or -automatic `repr` output), DataFusion runs a *limited* internal execution -to fetch preview rows. This internal execution does **not** cache the -physical plan used, so {py:meth}`~datafusion.ExecutionPlan.collect_metrics` -will not reflect the display execution. To access metrics you must call -one of the terminal operations listed above. -::: - -If you call {py:meth}`~datafusion.DataFrame.collect` (or another terminal +!!! note + + **display() does not populate metrics.** + When a DataFrame is displayed in a notebook (e.g. via `display(df)` or + automatic `repr` output), DataFusion runs a *limited* internal execution + to fetch preview rows. This internal execution does **not** cache the + physical plan used, so [`collect_metrics`][datafusion.ExecutionPlan.collect_metrics] + will not reflect the display execution. To access metrics you must call + one of the terminal operations listed above. + +If you call [`collect`][datafusion.DataFrame.collect] (or another terminal operation) multiple times on the same DataFrame, each call creates a fresh -physical plan. Metrics from {py:meth}`~datafusion.DataFrame.execution_plan` +physical plan. Metrics from [`execution_plan`][datafusion.DataFrame.execution_plan] always reflect the **most recent** execution. ## Reading the Physical Plan Tree -{py:meth}`~datafusion.DataFrame.execution_plan` returns the root -{py:class}`~datafusion.ExecutionPlan` node of the physical plan tree. The tree +[`execution_plan`][datafusion.DataFrame.execution_plan] returns the root +[`ExecutionPlan`][datafusion.ExecutionPlan] node of the physical plan tree. The tree mirrors the operator pipeline: the root is typically a projection or coalescing node; its children are filters, aggregates, scans, etc. The `operator_name` string returned by -{py:meth}`~datafusion.ExecutionPlan.collect_metrics` is the *display* name of +[`collect_metrics`][datafusion.ExecutionPlan.collect_metrics] is the *display* name of the node, for example `"FilterExec: column1@0 > 1"`. This is the same string you would see when calling `plan.display()`. ## Aggregated vs Per-Partition Metrics DataFusion executes each operator across one or more **partitions** in -parallel. The {py:class}`~datafusion.MetricsSet` convenience properties +parallel. The [`MetricsSet`][datafusion.MetricsSet] convenience properties (`output_rows`, `elapsed_compute`, etc.) automatically **sum** the named metric across all partitions, giving a single aggregate value. To inspect individual partitions — for example to detect data skew where one partition processes far more rows than others — iterate over the raw -{py:class}`~datafusion.Metric` objects: +[`Metric`][datafusion.Metric] objects: ```python for metric in metrics_set.metrics(): @@ -112,41 +111,24 @@ apply globally (not tied to a specific partition). ## Available Metrics The following metrics are directly accessible as properties on -{py:class}`~datafusion.MetricsSet`: - -```{eval-rst} -.. list-table:: - :header-rows: 1 - :widths: 25 75 - - * - Property - - Description - * - ``output_rows`` - - Number of rows emitted by the operator (summed across partitions). - * - ``elapsed_compute`` - - Wall-clock CPU time **in nanoseconds** spent inside the operator's - compute loop, excluding I/O wait. Useful for identifying which - operators are most expensive (summed across partitions). - * - ``spill_count`` - - Number of spill-to-disk events triggered by memory pressure. This is - a unitless count of events, not a measure of data volume (summed across - partitions). - * - ``spilled_bytes`` - - Total bytes written to disk during spill events (summed across - partitions). - * - ``spilled_rows`` - - Total rows written to disk during spill events (summed across - partitions). -``` +[`MetricsSet`][datafusion.MetricsSet]: + +| Property | Description | +|----------|-------------| +| `output_rows` | Number of rows emitted by the operator (summed across partitions). | +| `elapsed_compute` | Wall-clock CPU time **in nanoseconds** spent inside the operator's compute loop, excluding I/O wait. Useful for identifying which operators are most expensive (summed across partitions). | +| `spill_count` | Number of spill-to-disk events triggered by memory pressure. This is a unitless count of events, not a measure of data volume (summed across partitions). | +| `spilled_bytes` | Total bytes written to disk during spill events (summed across partitions). | +| `spilled_rows` | Total rows written to disk during spill events (summed across partitions). | Any metric not listed above can be accessed via -{py:meth}`~datafusion.MetricsSet.sum_by_name`, or by iterating over the raw -{py:class}`~datafusion.Metric` objects returned by -{py:meth}`~datafusion.MetricsSet.metrics`. +[`sum_by_name`][datafusion.MetricsSet.sum_by_name], or by iterating over the raw +[`Metric`][datafusion.Metric] objects returned by +[`metrics`][datafusion.MetricsSet.metrics]. ## Labels -A {py:class}`~datafusion.Metric` may carry *labels*: key/value pairs that +A [`Metric`][datafusion.Metric] may carry *labels*: key/value pairs that provide additional context. Labels are operator-specific; most metrics have an empty label dict. @@ -161,10 +143,10 @@ for metric in metrics_set.metrics(): # output_rows {'output_type': 'intermediate'} ``` -When summing by name (via {py:attr}`~datafusion.MetricsSet.output_rows` or -{py:meth}`~datafusion.MetricsSet.sum_by_name`), **all** metrics with that +When summing by name (via [`output_rows`][datafusion.MetricsSet.output_rows] or +[`sum_by_name`][datafusion.MetricsSet.sum_by_name]), **all** metrics with that name are summed regardless of labels. To filter by label, iterate over the -raw {py:class}`~datafusion.Metric` objects directly. +raw [`Metric`][datafusion.Metric] objects directly. ## End-to-End Example @@ -201,10 +183,10 @@ for operator_name, ms in plan.collect_metrics(): ## API Reference -- {py:class}`datafusion.ExecutionPlan` — physical plan node -- {py:meth}`datafusion.ExecutionPlan.collect_metrics` — walk the tree and +- [`ExecutionPlan`][datafusion.ExecutionPlan] — physical plan node +- [`collect_metrics`][datafusion.ExecutionPlan.collect_metrics] — walk the tree and return `(operator_name, MetricsSet)` pairs -- {py:meth}`datafusion.ExecutionPlan.metrics` — return the - {py:class}`~datafusion.MetricsSet` for a single node -- {py:class}`datafusion.MetricsSet` — aggregated metrics for one operator -- {py:class}`datafusion.Metric` — a single per-partition metric value +- [`metrics`][datafusion.ExecutionPlan.metrics] — return the + [`MetricsSet`][datafusion.MetricsSet] for a single node +- [`MetricsSet`][datafusion.MetricsSet] — aggregated metrics for one operator +- [`Metric`][datafusion.Metric] — a single per-partition metric value diff --git a/docs/source/user-guide/dataframe/index.md b/docs/source/user-guide/dataframe/index.md index a3af4e3cd..76d46b38f 100644 --- a/docs/source/user-guide/dataframe/index.md +++ b/docs/source/user-guide/dataframe/index.md @@ -76,8 +76,8 @@ DataFrames can be created in several ways: df = ctx.from_arrow(batch) ``` -For detailed information about reading from different data sources, see the {doc}`I/O Guide <../io/index>`. -For custom data sources, see {ref}`io_custom_table_provider`. +For detailed information about reading from different data sources, see the [I/O Guide](../io/index.md). +For custom data sources, see [io_custom_table_provider](io_custom_table_provider). ## Common DataFrame Operations @@ -130,15 +130,15 @@ df = df.drop("temporary_column") Some `DataFrame` methods accept column names when an argument refers to an existing column. These include: -- {py:meth}`~datafusion.DataFrame.select` -- {py:meth}`~datafusion.DataFrame.sort` -- {py:meth}`~datafusion.DataFrame.drop` -- {py:meth}`~datafusion.DataFrame.join` (`on` argument) -- {py:meth}`~datafusion.DataFrame.aggregate` (grouping columns) +- [`select`][datafusion.DataFrame.select] +- [`sort`][datafusion.DataFrame.sort] +- [`drop`][datafusion.DataFrame.drop] +- [`join`][datafusion.DataFrame.join] (`on` argument) +- [`aggregate`][datafusion.DataFrame.aggregate] (grouping columns) See the full function documentation for details on any specific function. -Note that {py:meth}`~datafusion.DataFrame.join_on` expects `col()`/`column()` expressions rather than plain strings. +Note that [`join_on`][datafusion.DataFrame.join_on] expects `col()`/`column()` expressions rather than plain strings. For such methods, you can pass column names directly: @@ -161,8 +161,8 @@ df.aggregate(column('id'), [f.count(col('value'))]) Note that `column()` is an alias of `col()`, so you can use either name; the example above shows both in action. Whenever an argument represents an expression—such as in -{py:meth}`~datafusion.DataFrame.filter` or -{py:meth}`~datafusion.DataFrame.with_column`—use `col()` to reference +[`filter`][datafusion.DataFrame.filter] or +[`with_column`][datafusion.DataFrame.with_column]—use `col()` to reference columns. The comparison and arithmetic operators on `Expr` will automatically convert any non-`Expr` value into a literal expression, so writing @@ -205,13 +205,13 @@ DataFusion DataFrames implement the `__arrow_c_stream__` protocol, enabling zero-copy, lazy streaming into Arrow-based Python libraries. With the streaming protocol, batches are produced on demand. -:::{note} -The protocol is implementation-agnostic and works with any Python library -that understands the Arrow C streaming interface (for example, PyArrow -or other Arrow-compatible implementations). The sections below provide a -short PyArrow-specific example and general guidance for other -implementations. -::: +!!! note + + The protocol is implementation-agnostic and works with any Python library + that understands the Arrow C streaming interface (for example, PyArrow + or other Arrow-compatible implementations). The sections below provide a + short PyArrow-specific example and general guidance for other + implementations. ## PyArrow @@ -262,8 +262,8 @@ for batch in stream: ### Execute as Stream For finer control over streaming execution, use -{py:meth}`~datafusion.DataFrame.execute_stream` to obtain a -{py:class}`datafusion.RecordBatchStream`: +[`execute_stream`][datafusion.DataFrame.execute_stream] to obtain a +[`RecordBatchStream`][datafusion.RecordBatchStream]: ```python stream = df.execute_stream() @@ -271,15 +271,15 @@ for batch in stream: ... # process each batch as it is produced ``` -:::{tip} -To get a PyArrow reader instead, call +!!! tip + + To get a PyArrow reader instead, call -`pa.RecordBatchReader.from_stream(df)`. -::: + `pa.RecordBatchReader.from_stream(df)`. When partition boundaries are important, -{py:meth}`~datafusion.DataFrame.execute_stream_partitioned` -returns an iterable of {py:class}`datafusion.RecordBatchStream` objects, one per +[`execute_stream_partitioned`][datafusion.DataFrame.execute_stream_partitioned] +returns an iterable of [`RecordBatchStream`][datafusion.RecordBatchStream] objects, one per partition: ```python @@ -302,13 +302,13 @@ streams = list(df.execute_stream_partitioned()) await asyncio.gather(*(consume(s) for s in streams)) ``` -See {doc}`../io/arrow` for additional details on the Arrow interface. +See [../io/arrow](../io/arrow.ipynb) for additional details on the Arrow interface. ## HTML Rendering When working in Jupyter notebooks or other environments that support HTML rendering, DataFrames will automatically display as formatted HTML tables. For detailed information about customizing HTML -rendering, formatting options, and advanced styling, see {doc}`rendering`. +rendering, formatting options, and advanced styling, see [rendering](rendering.md). ## Core Classes @@ -316,7 +316,7 @@ rendering, formatting options, and advanced styling, see {doc}`rendering`. : The main DataFrame class for building and executing queries. - See: {py:class}`datafusion.DataFrame` + See: [`DataFrame`][datafusion.DataFrame] **SessionContext** @@ -324,16 +324,16 @@ rendering, formatting options, and advanced styling, see {doc}`rendering`. Key methods for DataFrame creation: - - {py:meth}`~datafusion.SessionContext.read_csv` - Read CSV files - - {py:meth}`~datafusion.SessionContext.read_parquet` - Read Parquet files - - {py:meth}`~datafusion.SessionContext.read_json` - Read JSON files - - {py:meth}`~datafusion.SessionContext.read_avro` - Read Avro files - - {py:meth}`~datafusion.SessionContext.table` - Access registered tables - - {py:meth}`~datafusion.SessionContext.sql` - Execute SQL queries - - {py:meth}`~datafusion.SessionContext.from_pandas` - Create from Pandas DataFrame - - {py:meth}`~datafusion.SessionContext.from_arrow` - Create from Arrow data + - [`read_csv`][datafusion.SessionContext.read_csv] - Read CSV files + - [`read_parquet`][datafusion.SessionContext.read_parquet] - Read Parquet files + - [`read_json`][datafusion.SessionContext.read_json] - Read JSON files + - [`read_avro`][datafusion.SessionContext.read_avro] - Read Avro files + - [`table`][datafusion.SessionContext.table] - Access registered tables + - [`sql`][datafusion.SessionContext.sql] - Execute SQL queries + - [`from_pandas`][datafusion.SessionContext.from_pandas] - Create from Pandas DataFrame + - [`from_arrow`][datafusion.SessionContext.from_arrow] - Create from Arrow data - See: {py:class}`datafusion.SessionContext` + See: [`SessionContext`][datafusion.SessionContext] ## Expression Classes @@ -341,26 +341,26 @@ rendering, formatting options, and advanced styling, see {doc}`rendering`. : Represents expressions that can be used in DataFrame operations. - See: {py:class}`datafusion.Expr` + See: [`Expr`][datafusion.Expr] **Functions for creating expressions:** -- {py:func}`datafusion.column` - Reference a column by name -- {py:func}`datafusion.literal` - Create a literal value expression +- [`column`][datafusion.column] - Reference a column by name +- [`literal`][datafusion.literal] - Create a literal value expression ## Built-in Functions DataFusion provides many built-in functions for data manipulation: -- {py:mod}`datafusion.functions` - Mathematical, string, date/time, and aggregation functions +- [`functions`][datafusion.functions] - Mathematical, string, date/time, and aggregation functions -For a complete list of available functions, see the {py:mod}`datafusion.functions` module documentation. +For a complete list of available functions, see the [`functions`][datafusion.functions] module documentation. ## Execution Metrics After executing a DataFrame (via `collect()`, `execute_stream()`, etc.), DataFusion populates per-operator runtime statistics such as row counts and -compute time. See {doc}`execution-metrics` for a full explanation and +compute time. See [execution-metrics](execution-metrics.md) for a full explanation and worked example. ```{toctree} diff --git a/docs/source/user-guide/dataframe/rendering.md b/docs/source/user-guide/dataframe/rendering.md index 8b3019016..4ae9854e5 100644 --- a/docs/source/user-guide/dataframe/rendering.md +++ b/docs/source/user-guide/dataframe/rendering.md @@ -216,12 +216,12 @@ These parameters help balance comprehensive data display against performance con ## Additional Resources -- {doc}`../dataframe/index` - Complete guide to using DataFrames -- {doc}`../io/index` - I/O Guide for reading data from various sources -- {doc}`../data-sources` - Comprehensive data sources guide -- {ref}`io_csv` - CSV file reading -- {ref}`io_parquet` - Parquet file reading -- {ref}`io_json` - JSON file reading -- {ref}`io_avro` - Avro file reading -- {ref}`io_custom_table_provider` - Custom table providers +- [../dataframe/index](../dataframe/index.md) - Complete guide to using DataFrames +- [../io/index](../io/index.md) - I/O Guide for reading data from various sources +- [../data-sources](../data-sources.ipynb) - Comprehensive data sources guide +- [io_csv](io_csv) - CSV file reading +- [io_parquet](io_parquet) - Parquet file reading +- [io_json](io_json) - JSON file reading +- [io_avro](io_avro) - Avro file reading +- [io_custom_table_provider](io_custom_table_provider) - Custom table providers - [API Reference](https://arrow.apache.org/datafusion-python/api/index.html) - Full API reference diff --git a/docs/source/user-guide/distributing-work.md b/docs/source/user-guide/distributing-work.md index 7d5aa475a..6728de925 100644 --- a/docs/source/user-guide/distributing-work.md +++ b/docs/source/user-guide/distributing-work.md @@ -21,7 +21,7 @@ DataFusion supports splitting work across processes by shipping serialized expressions to workers: the driver builds an -{py:class}`~datafusion.Expr`, each worker evaluates it against its +[`Expr`][datafusion.Expr], each worker evaluates it against its own slice of data. This pattern suits embarrassingly-parallel workloads where the driver decides partitioning up front. @@ -38,7 +38,7 @@ DataFusion expressions support distribution directly: pass one to a worker process and Python's standard [pickle](https://docs.python.org/3/library/pickle.html) machinery serializes it transparently — the same machinery -{py:meth}`multiprocessing.pool.Pool.map`, Ray's `@ray.remote`, and +[`map`][multiprocessing.pool.Pool.map], Ray's `@ray.remote`, and similar libraries already use to ship function arguments. Python UDFs — scalar, aggregate, and window — travel inside the serialized expression; the receiver does not need to pre-register them. @@ -82,15 +82,15 @@ with mp_ctx.Pool(processes=4) as pool: print(results) # [[2, 4, 6], [20, 40, 60]] ``` -:::{note} -When saved to a `.py` file and executed with the `spawn` or -`forkserver` start method, wrap the driver block in -`if __name__ == "__main__":` so worker processes can re-import -the module without re-running it. This is a standard Python -{py:mod}`multiprocessing` requirement, not DataFusion-specific — -see [Safe importing of main module](https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods) -in the Python docs. -::: +!!! note + + When saved to a `.py` file and executed with the `spawn` or + `forkserver` start method, wrap the driver block in + `if __name__ == "__main__":` so worker processes can re-import + the module without re-running it. This is a standard Python + [`multiprocessing`][multiprocessing] requirement, not DataFusion-specific — + see [Safe importing of main module](https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods) + in the Python docs. ### What travels with the expression @@ -102,13 +102,13 @@ in the Python docs. captured in closures travel inside the serialized expression and are reconstructed on the worker automatically. Applies equally to: - - **scalar UDFs** ({py:func}`datafusion.udf`) - - **aggregate UDFs** ({py:func}`datafusion.udaf`) - - **window UDFs** ({py:func}`datafusion.udwf`) + - **scalar UDFs** ([`udf`][datafusion.udf]) + - **aggregate UDFs** ([`udaf`][datafusion.udaf]) + - **window UDFs** ([`udwf`][datafusion.udwf]) - **UDFs imported via the FFI capsule protocol** — travel **by name only**. The worker must already have a matching registration on its - {py:class}`SessionContext`. Without that registration, evaluation + [`SessionContext`][datafusion.context.SessionContext]. Without that registration, evaluation raises an error. ### Portability requirements for inline Python UDFs @@ -135,7 +135,7 @@ requirements on the worker environment: When an expression references an FFI capsule UDF (or any UDF the worker must resolve from its registered functions), set up the -worker's {py:class}`SessionContext` once per process and install it +worker's [`SessionContext`][datafusion.context.SessionContext] once per process and install it as the *worker context*: ```python @@ -157,7 +157,7 @@ with mp.get_context("forkserver").Pool( Inside a worker, expressions arriving from the driver resolve their by-name references against the installed worker context. If no worker -context is installed, the global {py:class}`SessionContext` is used — +context is installed, the global [`SessionContext`][datafusion.context.SessionContext] is used — fine for expressions that only reference built-ins and Python UDFs, but FFI-capsule-backed registrations must be installed on the global context to resolve. @@ -165,11 +165,11 @@ context to resolve. ### Python 3.14 default change Python 3.14 changed the Linux default start method for -{py:mod}`multiprocessing` from `fork` to `forkserver` (macOS has +[`multiprocessing`][multiprocessing] from `fork` to `forkserver` (macOS has defaulted to `spawn` since Python 3.8; Windows has always used `spawn`). With `fork`, any state set in the parent was visible in workers via copy-on-write; with `forkserver` and `spawn` it is -not. The {py:func}`~datafusion.ipc.set_worker_ctx` pattern works on +not. The [`set_worker_ctx`][datafusion.ipc.set_worker_ctx] pattern works on every start method — prefer it over relying on inherited state. ### Practical considerations @@ -179,7 +179,7 @@ every start method — prefer it over relying on inherited state. expression carrying a Python UDF is hundreds of bytes (the callable and its signature). When the same UDF is shipped many times, registering an equivalent FFI-capsule UDF on each worker via - {py:func}`~datafusion.ipc.set_worker_ctx` and referring to it by + [`set_worker_ctx`][datafusion.ipc.set_worker_ctx] and referring to it by name cuts the per-trip overhead. - **Closure capture.** When a Python UDF closes over surrounding state — local variables, module-level objects, file paths — that @@ -191,8 +191,7 @@ every start method — prefer it over relying on inherited state. ### Disabling Python UDF inlining For a stricter wire format, call -{py:meth}`SessionContext.with_python_udf_inlining(enabled=False) -` on the session +[`SessionContext.with_python_udf_inlining(enabled=False)`][datafusion.SessionContext.with_python_udf_inlining] on the session producing or consuming the bytes. With inlining disabled, Python UDFs travel by name only — the same way FFI-capsule UDFs do — and the receiver must have a matching registration. @@ -204,7 +203,7 @@ Two use cases: or another Rust binary disable inlining and rely on the receiver having compatible UDF registrations. - **Untrusted-source decode.** With inlining disabled, - {py:meth}`Expr.from_bytes` never calls `cloudpickle.loads` on + [`from_bytes`][datafusion.expr.Expr.from_bytes] never calls `cloudpickle.loads` on the incoming bytes — an inline payload from a misbehaving sender raises a clear error instead of executing arbitrary Python code. @@ -212,8 +211,8 @@ Mismatched configurations raise a descriptive error: an inline blob fed to a strict receiver fails fast rather than silently dropping into `cloudpickle.loads`. -To make the toggle apply through {py:func}`pickle.dumps` (which -calls {py:meth}`Expr.to_bytes` with no context), install the strict +To make the toggle apply through [`dumps`][pickle.dumps] (which +calls [`to_bytes`][datafusion.expr.Expr.to_bytes] with no context), install the strict session as the driver's *sender context*: ```python @@ -226,73 +225,45 @@ set_sender_ctx(SessionContext().with_python_udf_inlining(enabled=False)) ``` Pair with a matching strict worker context -({py:func}`~datafusion.ipc.set_worker_ctx`) so the `pickle.loads` +([`set_worker_ctx`][datafusion.ipc.set_worker_ctx]) so the `pickle.loads` side also refuses inline payloads. Explicit -{py:meth}`Expr.to_bytes(ctx) ` and -{py:meth}`Expr.from_bytes(blob, ctx=ctx) ` calls +[`Expr.to_bytes(ctx)`][Expr.to_bytes] and +[`Expr.from_bytes(blob, ctx=ctx)`][Expr.from_bytes] calls honor the supplied `ctx` directly and ignore the sender / worker contexts. -The toggle only narrows the {py:meth}`Expr.from_bytes` surface; -{py:func}`pickle.loads` on untrusted bytes remains unsafe regardless +The toggle only narrows the [`from_bytes`][datafusion.expr.Expr.from_bytes] surface; +[`loads`][pickle.loads] on untrusted bytes remains unsafe regardless of this setting. See the [Security] section below for the full threat model. ### Security -:::{warning} -Reconstructing an expression containing a Python UDF executes -arbitrary Python code on the receiver — pickle is doing the work -under the hood and pickle is unsafe on untrusted input (see the -[pickle module security warning](https://docs.python.org/3/library/pickle.html#module-pickle) -in the Python standard library docs). Only accept expressions -from trusted sources. For untrusted-source workflows, disable -Python UDF inlining (see above), restrict senders to built-in -functions and pre-registered Rust-side UDFs, and avoid -{py:func}`pickle.loads` on externally supplied bytes entirely. -::: +!!! warning + + Reconstructing an expression containing a Python UDF executes + arbitrary Python code on the receiver — pickle is doing the work + under the hood and pickle is unsafe on untrusted input (see the + [pickle module security warning](https://docs.python.org/3/library/pickle.html#module-pickle) + in the Python standard library docs). Only accept expressions + from trusted sources. For untrusted-source workflows, disable + Python UDF inlining (see above), restrict senders to built-in + functions and pre-registered Rust-side UDFs, and avoid + [`loads`][pickle.loads] on externally supplied bytes entirely. ### Reference: session context slots -There is only one type — {py:class}`SessionContext`. It can occupy +There is only one type — [`SessionContext`][datafusion.context.SessionContext]. It can occupy up to four *slots* in a running program: -```{eval-rst} -.. list-table:: - :header-rows: 1 - :widths: 12 18 40 30 - - * - Slot - - Lifetime - - Purpose - - Set how - * - User-held - - Local variable / attribute - - Build and run queries - - ``ctx = SessionContext(...)`` - * - Global - - Process singleton (lazy-init) - - Backs module-level - :py:func:`~datafusion.io.read_parquet`, - :py:func:`~datafusion.io.read_csv`, - :py:func:`~datafusion.io.read_json`, - :py:func:`~datafusion.io.read_avro`; final fallback for - :py:meth:`Expr.from_bytes` - - Implicit; access via - :py:meth:`SessionContext.global_ctx` - * - Sender - - Thread-local on the driver - - Codec settings for outbound :py:func:`pickle.dumps` / - :py:meth:`Expr.to_bytes` without ``ctx`` - - :py:func:`~datafusion.ipc.set_sender_ctx` - * - Worker - - Thread-local on the worker - - Function registry for inbound :py:func:`pickle.loads` / - :py:meth:`Expr.from_bytes` without ``ctx`` - - :py:func:`~datafusion.ipc.set_worker_ctx` -``` +| Slot | Lifetime | Purpose | Set how | +|------|----------|---------|---------| +| User-held | Local variable / attribute | Build and run queries | `ctx = SessionContext(...)` | +| Global | Process singleton (lazy-init) | Backs module-level [`read_parquet`][datafusion.io.read_parquet], [`read_csv`][datafusion.io.read_csv], [`read_json`][datafusion.io.read_json], [`read_avro`][datafusion.io.read_avro]; final fallback for [`Expr.from_bytes`][datafusion.expr.Expr.from_bytes] | Implicit; access via [`SessionContext.global_ctx`][datafusion.context.SessionContext.global_ctx] | +| Sender | Thread-local on the driver | Codec settings for outbound `pickle.dumps` / [`Expr.to_bytes`][datafusion.expr.Expr.to_bytes] without `ctx` | [`set_sender_ctx`][datafusion.ipc.set_sender_ctx] | +| Worker | Thread-local on the worker | Function registry for inbound `pickle.loads` / [`Expr.from_bytes`][datafusion.expr.Expr.from_bytes] without `ctx` | [`set_worker_ctx`][datafusion.ipc.set_worker_ctx] | -The same {py:class}`SessionContext` object may occupy more than one +The same [`SessionContext`][datafusion.context.SessionContext] object may occupy more than one slot simultaneously — installing it into a slot is a reference, not a copy. A non-distributed program only ever uses the user-held slot; the global slot is invisible unless you call top-level `read_*` @@ -300,7 +271,7 @@ helpers. Resolution order on the worker side is *explicit argument → worker context → global context.* Explicit `ctx=` on -{py:meth}`Expr.from_bytes` always wins; the sender slot is ignored +[`from_bytes`][datafusion.expr.Expr.from_bytes] always wins; the sender slot is ignored on decode and the worker slot is ignored on encode. Sharp edges: @@ -348,7 +319,7 @@ section will fill in once the integration is usable. ## See also -- {py:mod}`datafusion.ipc` — worker context API. +- [`ipc`][datafusion.ipc] — worker context API. - `examples/multiprocessing_pickle_expr.py` — runnable `multiprocessing.Pool` example that ships a different parametric expression to each worker and collects results back. diff --git a/docs/source/user-guide/introduction.ipynb b/docs/source/user-guide/introduction.ipynb new file mode 100644 index 000000000..9861d936d --- /dev/null +++ b/docs/source/user-guide/introduction.ipynb @@ -0,0 +1,89 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n\n# Introduction\n\nWelcome to the User Guide for the Python bindings of Arrow DataFusion. This guide aims to provide an introduction to\nDataFusion through various examples and highlight the most effective ways of using it.\n\n## Installation\n\nDataFusion is a Python library and, as such, can be installed via pip from [PyPI](https://pypi.org/project/datafusion).\n\n```shell\npip install datafusion\n```\n\nYou can verify the installation by running:\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "datafusion.__version__" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\nIn this documentation we will also show some examples for how DataFusion integrates\nwith Jupyter notebooks. To install and start a Jupyter labs session use\n\n```shell\npip install jupyterlab\njupyter lab\n```\n\nTo demonstrate working with DataFusion, we need a data source. Later in the tutorial we will show\noptions for data sources. For our first example, we demonstrate using a Pokemon dataset that you\ncan download\n[here](https://gist.githubusercontent.com/ritchie46/cac6b337ea52281aa23c049250a4ff03/raw/89a957ff3919d90e6ef2d34235e6bf22304f3366/pokemon.csv).\n\nWith that file in place you can use the following python example to view the DataFrame in\nDataFusion.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "ctx = SessionContext()\n", + "\n", + "df = ctx.read_csv(\"pokemon.csv\")\n", + "\n", + "df.show()" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\nIf you are working in a Jupyter notebook, you can also use the following to give you a table\ndisplay that may be easier to read.\n\n```shell\ndisplay(df)\n```\n\n```{image} ../images/jupyter_lab_df_view.png\n:alt: Rendered table showing Pokemon DataFrame\n:width: 800\n```\n" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/introduction.md b/docs/source/user-guide/introduction.md deleted file mode 100644 index 15a403bf4..000000000 --- a/docs/source/user-guide/introduction.md +++ /dev/null @@ -1,82 +0,0 @@ - - -(guide)= - -# Introduction - -Welcome to the User Guide for the Python bindings of Arrow DataFusion. This guide aims to provide an introduction to -DataFusion through various examples and highlight the most effective ways of using it. - -## Installation - -DataFusion is a Python library and, as such, can be installed via pip from [PyPI](https://pypi.org/project/datafusion). - -```shell -pip install datafusion -``` - -You can verify the installation by running: - -```{eval-rst} -.. ipython:: python - - import datafusion - datafusion.__version__ -``` - -In this documentation we will also show some examples for how DataFusion integrates -with Jupyter notebooks. To install and start a Jupyter labs session use - -```shell -pip install jupyterlab -jupyter lab -``` - -To demonstrate working with DataFusion, we need a data source. Later in the tutorial we will show -options for data sources. For our first example, we demonstrate using a Pokemon dataset that you -can download -[here](https://gist.githubusercontent.com/ritchie46/cac6b337ea52281aa23c049250a4ff03/raw/89a957ff3919d90e6ef2d34235e6bf22304f3366/pokemon.csv). - -With that file in place you can use the following python example to view the DataFrame in -DataFusion. - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - - ctx = SessionContext() - - df = ctx.read_csv("pokemon.csv") - - df.show() -``` - -If you are working in a Jupyter notebook, you can also use the following to give you a table -display that may be easier to read. - -```shell -display(df) -``` - -```{image} ../images/jupyter_lab_df_view.png -:alt: Rendered table showing Pokemon DataFrame -:width: 800 -``` diff --git a/docs/source/user-guide/io/arrow.ipynb b/docs/source/user-guide/io/arrow.ipynb new file mode 100644 index 000000000..548183e25 --- /dev/null +++ b/docs/source/user-guide/io/arrow.ipynb @@ -0,0 +1,87 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion # noqa: F401\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n# Arrow\n\nDataFusion implements the\n[Apache Arrow PyCapsule interface](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html)\nfor importing and exporting DataFrames with zero copy. With this feature, any Python\nproject that implements this interface can share data back and forth with DataFusion\nwith zero copy.\n\nWe can demonstrate using [pyarrow](https://arrow.apache.org/docs/python/index.html).\n\n## Importing to DataFusion\n\nHere we will create an Arrow table and import it to DataFusion.\n\nTo import an Arrow table, use [`from_arrow`][datafusion.context.SessionContext.from_arrow].\nThis will accept any Python object that implements\n[\\_\\_arrow_c_stream\\_\\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowstream-export)\nor [\\_\\_arrow_c_array\\_\\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowarray-export)\nand returns a `StructArray`. Common pyarrow sources you can use are:\n\n- [Array](https://arrow.apache.org/docs/python/generated/pyarrow.Array.html) (but it must return a Struct Array)\n- [Record Batch](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html)\n- [Record Batch Reader](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatchReader.html)\n- [Table](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html)\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "import pyarrow as pa\n", + "\n", + "data = {\"a\": [1, 2, 3], \"b\": [4, 5, 6]}\n", + "table = pa.Table.from_pydict(data)\n", + "\n", + "ctx = SessionContext()\n", + "df = ctx.from_arrow(table)\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\n## Exporting from DataFusion\n\nDataFusion DataFrames implement `__arrow_c_stream__` PyCapsule interface, so any\nPython library that accepts these can import a DataFusion DataFrame directly.\n\nInvoking `__arrow_c_stream__` triggers execution of the underlying query, but\nbatches are yielded incrementally rather than materialized all at once in memory.\nConsumers can process the stream as it arrives. The stream executes lazily,\nletting downstream readers pull batches on demand.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "df = df.select((col(\"a\") * lit(1.5)).alias(\"c\"), lit(\"df\").alias(\"d\"))\n", + "pa.table(df)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/io/arrow.md b/docs/source/user-guide/io/arrow.md deleted file mode 100644 index 644f74dc5..000000000 --- a/docs/source/user-guide/io/arrow.md +++ /dev/null @@ -1,76 +0,0 @@ - - -# Arrow - -DataFusion implements the -[Apache Arrow PyCapsule interface](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) -for importing and exporting DataFrames with zero copy. With this feature, any Python -project that implements this interface can share data back and forth with DataFusion -with zero copy. - -We can demonstrate using [pyarrow](https://arrow.apache.org/docs/python/index.html). - -## Importing to DataFusion - -Here we will create an Arrow table and import it to DataFusion. - -To import an Arrow table, use {py:func}`datafusion.context.SessionContext.from_arrow`. -This will accept any Python object that implements -[\_\_arrow_c_stream\_\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowstream-export) -or [\_\_arrow_c_array\_\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowarray-export) -and returns a `StructArray`. Common pyarrow sources you can use are: - -- [Array](https://arrow.apache.org/docs/python/generated/pyarrow.Array.html) (but it must return a Struct Array) -- [Record Batch](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html) -- [Record Batch Reader](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatchReader.html) -- [Table](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html) - -```{eval-rst} -.. ipython:: python - - from datafusion import SessionContext - import pyarrow as pa - - data = {"a": [1, 2, 3], "b": [4, 5, 6]} - table = pa.Table.from_pydict(data) - - ctx = SessionContext() - df = ctx.from_arrow(table) - df -``` - -## Exporting from DataFusion - -DataFusion DataFrames implement `__arrow_c_stream__` PyCapsule interface, so any -Python library that accepts these can import a DataFusion DataFrame directly. - -Invoking `__arrow_c_stream__` triggers execution of the underlying query, but -batches are yielded incrementally rather than materialized all at once in memory. -Consumers can process the stream as it arrives. The stream executes lazily, -letting downstream readers pull batches on demand. - -```{eval-rst} -.. ipython:: python - - from datafusion import col, lit - - df = df.select((col("a") * lit(1.5)).alias("c"), lit("df").alias("d")) - pa.table(df) -``` diff --git a/docs/source/user-guide/io/avro.md b/docs/source/user-guide/io/avro.md index 5654547ac..62c7c94d0 100644 --- a/docs/source/user-guide/io/avro.md +++ b/docs/source/user-guide/io/avro.md @@ -17,12 +17,11 @@ under the License. --> -(io_avro)= # Avro [Avro](https://avro.apache.org/) is a serialization format for record data. Reading an avro file is very straightforward -with {py:func}`~datafusion.context.SessionContext.read_avro` +with [`read_avro`][datafusion.context.SessionContext.read_avro] ```python from datafusion import SessionContext diff --git a/docs/source/user-guide/io/csv.md b/docs/source/user-guide/io/csv.md index 3022a6db8..0fa1369fa 100644 --- a/docs/source/user-guide/io/csv.md +++ b/docs/source/user-guide/io/csv.md @@ -17,11 +17,10 @@ under the License. --> -(io_csv)= # CSV -Reading a csv is very straightforward with {py:func}`~datafusion.context.SessionContext.read_csv` +Reading a csv is very straightforward with [`read_csv`][datafusion.context.SessionContext.read_csv] ```python from datafusion import SessionContext @@ -30,7 +29,7 @@ ctx = SessionContext() df = ctx.read_csv("file.csv") ``` -An alternative is to use {py:func}`~datafusion.context.SessionContext.register_csv` +An alternative is to use [`register_csv`][datafusion.context.SessionContext.register_csv] ```python ctx.register_csv("file", "file.csv") @@ -38,7 +37,7 @@ df = ctx.table("file") ``` If you require additional control over how to read the CSV file, you can use -{py:class}`~datafusion.options.CsvReadOptions` to set a variety of options. +[`CsvReadOptions`][datafusion.options.CsvReadOptions] to set a variety of options. ```python from datafusion import CsvReadOptions diff --git a/docs/source/user-guide/io/json.md b/docs/source/user-guide/io/json.md index 45df4c6ce..0b5d8d9d8 100644 --- a/docs/source/user-guide/io/json.md +++ b/docs/source/user-guide/io/json.md @@ -17,12 +17,11 @@ under the License. --> -(io_json)= # JSON [JSON](https://www.json.org/json-en.html) (JavaScript Object Notation) is a lightweight data-interchange format. -When it comes to reading a JSON file, using {py:func}`~datafusion.context.SessionContext.read_json` is a simple and easy +When it comes to reading a JSON file, using [`read_json`][datafusion.context.SessionContext.read_json] is a simple and easy ```python from datafusion import SessionContext diff --git a/docs/source/user-guide/io/parquet.md b/docs/source/user-guide/io/parquet.md index da79360e8..1e6e4a18e 100644 --- a/docs/source/user-guide/io/parquet.md +++ b/docs/source/user-guide/io/parquet.md @@ -17,11 +17,10 @@ under the License. --> -(io_parquet)= # Parquet -It is quite simple to read a parquet file using the {py:func}`~datafusion.context.SessionContext.read_parquet` function. +It is quite simple to read a parquet file using the [`read_parquet`][datafusion.context.SessionContext.read_parquet] function. ```python from datafusion import SessionContext @@ -30,7 +29,7 @@ ctx = SessionContext() df = ctx.read_parquet("file.parquet") ``` -An alternative is to use {py:func}`~datafusion.context.SessionContext.register_parquet` +An alternative is to use [`register_parquet`][datafusion.context.SessionContext.register_parquet] ```python ctx.register_parquet("file", "file.parquet") diff --git a/docs/source/user-guide/io/table_provider.md b/docs/source/user-guide/io/table_provider.md index 3c436ba1d..0462ca28d 100644 --- a/docs/source/user-guide/io/table_provider.md +++ b/docs/source/user-guide/io/table_provider.md @@ -17,7 +17,6 @@ under the License. --> -(io_custom_table_provider)= # Custom Table Provider @@ -48,7 +47,7 @@ impl MyTableProvider { ``` Once you have this library available, you can construct a -{py:class}`~datafusion.Table` in Python and register it with the +[`Table`][datafusion.Table] in Python and register it with the `SessionContext`. ```python diff --git a/docs/source/user-guide/sql.ipynb b/docs/source/user-guide/sql.ipynb new file mode 100644 index 000000000..1e2933ee1 --- /dev/null +++ b/docs/source/user-guide/sql.ipynb @@ -0,0 +1,148 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "7fb27b941602401d91542211134fc71a", + "metadata": { + "tags": [ + "nb-setup" + ] + }, + "outputs": [], + "source": [ + "import os\n", + "import pathlib\n", + "\n", + "import datafusion\n", + "from datafusion import ( # noqa: F401\n", + " SessionContext,\n", + " col,\n", + " column,\n", + " lit,\n", + " literal,\n", + ")\n", + "from datafusion import functions as f # noqa: F401\n", + "\n", + "_p = pathlib.Path.cwd()\n", + "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", + " _p = _p.parent\n", + "if (_p / \"pokemon.csv\").exists():\n", + " os.chdir(_p)" + ] + }, + { + "cell_type": "markdown", + "id": "acae54e37e7d407bbb7b55eff062a284", + "metadata": {}, + "source": "\n\n# SQL\n\nDataFusion also offers a SQL API, read the full reference [here](https://arrow.apache.org/datafusion/user-guide/sql/index.html)\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9a63283cbaf04dbcab1f6479b197f3a8", + "metadata": {}, + "outputs": [], + "source": [ + "from datafusion import DataFrame\n", + "\n", + "# create a context\n", + "ctx = datafusion.SessionContext()\n", + "\n", + "# register a CSV\n", + "ctx.register_csv(\"pokemon\", \"pokemon.csv\")\n", + "\n", + "# create a new statement via SQL\n", + "df = ctx.sql('SELECT \"Attack\"+\"Defense\", \"Attack\"-\"Defense\" FROM pokemon')\n", + "\n", + "# collect and convert to pandas DataFrame\n", + "df.to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "8dd0d8092fe74a7c96281538738b07e2", + "metadata": {}, + "source": "\n## Parameterized queries\n\nIn DataFusion-Python 51.0.0 we introduced the ability to pass parameters\nin a SQL query. These are similar in concept to\n[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html),\nbut allow passing named parameters into a SQL query. Consider this simple\nexample.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72eea5119410473aa328ad9291626812", + "metadata": {}, + "outputs": [], + "source": [ + "def show_attacks(ctx: SessionContext, threshold: int) -> None:\n", + " ctx.sql(\n", + " 'SELECT \"Name\", \"Attack\" FROM pokemon WHERE \"Attack\" > $val', val=threshold\n", + " ).show(num=5)\n", + "\n", + "\n", + "show_attacks(ctx, 75)" + ] + }, + { + "cell_type": "markdown", + "id": "8edb47106e1a46a883d545849b8ab81b", + "metadata": {}, + "source": "\nWhen passing parameters like the example above we convert the Python objects\ninto their string representation. We also have special case handling\nfor [`DataFrame`][datafusion.dataframe.DataFrame] objects, since they cannot simply\nbe turned into string representations for an SQL query. In these cases we\nwill register a temporary view in the [`SessionContext`][datafusion.context.SessionContext]\nusing a generated table name.\n\nThe formatting for passing string replacement objects is to precede the\nvariable name with a single `$`. This works for all dialects in\nthe SQL parser except `hive` and `mysql`. Since these dialects do not\nsupport named placeholders, we are unable to do this type of replacement.\nWe recommend either switching to another dialect or using Python\nf-string style replacement.\n\n!!! warning\n\n To support DataFrame parameterized queries, your session must support\n registration of temporary views. The default\n [`CatalogProvider`][datafusion.catalog.CatalogProvider] and\n [`SchemaProvider`][datafusion.catalog.SchemaProvider] do have this capability.\n If you have implemented custom providers, it is important that temporary\n views do not persist across [`SessionContext`][datafusion.context.SessionContext]\n or you may get unintended consequences.\n\nThe following example shows passing in both a [`DataFrame`][datafusion.dataframe.DataFrame]\nobject as well as a Python object to be used in parameterized replacement.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", + "metadata": {}, + "outputs": [], + "source": [ + "def show_column(\n", + " ctx: SessionContext, column: str, df: DataFrame, threshold: int\n", + ") -> None:\n", + " ctx.sql(\n", + " 'SELECT \"Name\", $col FROM $df WHERE $col > $val',\n", + " col=column,\n", + " df=df,\n", + " val=threshold,\n", + " ).show(num=5)\n", + "\n", + "\n", + "df = ctx.table(\"pokemon\")\n", + "show_column(ctx, '\"Defense\"', df, 75)" + ] + }, + { + "cell_type": "markdown", + "id": "8763a12b2bbd4a93a75aff182afb95dc", + "metadata": {}, + "source": "\nThe approach implemented for conversion of variables into a SQL query\nrelies on string conversion. This has the potential for data loss,\nspecifically for cases like floating point numbers. If you need to pass\nvariables into a parameterized query and it is important to maintain the\noriginal value without conversion to a string, then you can use the\noptional parameter `param_values` to specify these. This parameter\nexpects a dictionary mapping from the parameter name to a Python\nobject. Those objects will be cast into a\n[PyArrow Scalar Value](https://arrow.apache.org/docs/python/generated/pyarrow.Scalar.html).\n\nUsing `param_values` will rely on the SQL dialect you have configured\nfor your session. This can be set using the [configuration options](configuration)\nof your [`SessionContext`][datafusion.context.SessionContext]. Similar to how\n[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html)\nwork, these parameters are limited to places where you would pass in a\nscalar value, such as a comparison.\n\n" + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7623eae2785240b9bd12b16a66d81610", + "metadata": {}, + "outputs": [], + "source": [ + "def param_attacks(ctx: SessionContext, threshold: int) -> None:\n", + " ctx.sql(\n", + " 'SELECT \"Name\", \"Attack\" FROM pokemon WHERE \"Attack\" > $val',\n", + " param_values={\"val\": threshold},\n", + " ).show(num=5)\n", + "\n", + "\n", + "param_attacks(ctx, 75)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/source/user-guide/sql.md b/docs/source/user-guide/sql.md deleted file mode 100644 index 339eb4949..000000000 --- a/docs/source/user-guide/sql.md +++ /dev/null @@ -1,130 +0,0 @@ - - -# SQL - -DataFusion also offers a SQL API, read the full reference [here](https://arrow.apache.org/datafusion/user-guide/sql/index.html) - -```{eval-rst} -.. ipython:: python - - import datafusion - from datafusion import DataFrame, SessionContext - - # create a context - ctx = datafusion.SessionContext() - - # register a CSV - ctx.register_csv("pokemon", "pokemon.csv") - - # create a new statement via SQL - df = ctx.sql('SELECT "Attack"+"Defense", "Attack"-"Defense" FROM pokemon') - - # collect and convert to pandas DataFrame - df.to_pandas() -``` - -## Parameterized queries - -In DataFusion-Python 51.0.0 we introduced the ability to pass parameters -in a SQL query. These are similar in concept to -[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html), -but allow passing named parameters into a SQL query. Consider this simple -example. - -```{eval-rst} -.. ipython:: python - - def show_attacks(ctx: SessionContext, threshold: int) -> None: - ctx.sql( - 'SELECT "Name", "Attack" FROM pokemon WHERE "Attack" > $val', val=threshold - ).show(num=5) - show_attacks(ctx, 75) -``` - -When passing parameters like the example above we convert the Python objects -into their string representation. We also have special case handling -for {py:class}`~datafusion.dataframe.DataFrame` objects, since they cannot simply -be turned into string representations for an SQL query. In these cases we -will register a temporary view in the {py:class}`~datafusion.context.SessionContext` -using a generated table name. - -The formatting for passing string replacement objects is to precede the -variable name with a single `$`. This works for all dialects in -the SQL parser except `hive` and `mysql`. Since these dialects do not -support named placeholders, we are unable to do this type of replacement. -We recommend either switching to another dialect or using Python -f-string style replacement. - -:::{warning} -To support DataFrame parameterized queries, your session must support -registration of temporary views. The default -{py:class}`~datafusion.catalog.CatalogProvider` and -{py:class}`~datafusion.catalog.SchemaProvider` do have this capability. -If you have implemented custom providers, it is important that temporary -views do not persist across {py:class}`~datafusion.context.SessionContext` -or you may get unintended consequences. -::: - -The following example shows passing in both a {py:class}`~datafusion.dataframe.DataFrame` -object as well as a Python object to be used in parameterized replacement. - -```{eval-rst} -.. ipython:: python - - def show_column( - ctx: SessionContext, column: str, df: DataFrame, threshold: int - ) -> None: - ctx.sql( - 'SELECT "Name", $col FROM $df WHERE $col > $val', - col=column, - df=df, - val=threshold, - ).show(num=5) - df = ctx.table("pokemon") - show_column(ctx, '"Defense"', df, 75) -``` - -The approach implemented for conversion of variables into a SQL query -relies on string conversion. This has the potential for data loss, -specifically for cases like floating point numbers. If you need to pass -variables into a parameterized query and it is important to maintain the -original value without conversion to a string, then you can use the -optional parameter `param_values` to specify these. This parameter -expects a dictionary mapping from the parameter name to a Python -object. Those objects will be cast into a -[PyArrow Scalar Value](https://arrow.apache.org/docs/python/generated/pyarrow.Scalar.html). - -Using `param_values` will rely on the SQL dialect you have configured -for your session. This can be set using the {ref}`configuration options ` -of your {py:class}`~datafusion.context.SessionContext`. Similar to how -[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html) -work, these parameters are limited to places where you would pass in a -scalar value, such as a comparison. - -```{eval-rst} -.. ipython:: python - - def param_attacks(ctx: SessionContext, threshold: int) -> None: - ctx.sql( - 'SELECT "Name", "Attack" FROM pokemon WHERE "Attack" > $val', - param_values={"val": threshold}, - ).show(num=5) - param_attacks(ctx, 75) -``` diff --git a/docs/source/user-guide/upgrade-guides.md b/docs/source/user-guide/upgrade-guides.md index 360e0533c..09e5f3ba4 100644 --- a/docs/source/user-guide/upgrade-guides.md +++ b/docs/source/user-guide/upgrade-guides.md @@ -23,7 +23,7 @@ The `Config` class has been removed. It was a standalone wrapper around `ConfigOptions` that could not be connected to a `SessionContext`, making it -effectively unusable. Use {py:class}`~datafusion.context.SessionConfig` instead, +effectively unusable. Use [`SessionConfig`][datafusion.context.SessionConfig] instead, which is passed directly to `SessionContext`. Before: @@ -45,8 +45,8 @@ config = SessionConfig().set("datafusion.execution.batch_size", "4096") ctx = SessionContext(config) ``` -The aggregate functions {py:func}`~datafusion.functions.sum` and -{py:func}`~datafusion.functions.avg` now accept a `distinct` argument, matching +The aggregate functions [`sum`][datafusion.functions.sum] and +[`avg`][datafusion.functions.avg] now accept a `distinct` argument, matching the other aggregate functions. `distinct` is inserted *before* `filter` in the argument list, so any code that passed `filter` positionally must be updated to pass it as a keyword argument. The types are distinct so a type checker should flag this. @@ -87,7 +87,7 @@ let codec = unsafe { data.as_ref() }; ## DataFusion 52.0.0 -This version includes a major update to the {ref}`ffi` due to upgrades +This version includes a major update to the [ffi](ffi) due to upgrades to the [Foreign Function Interface](https://doc.rust-lang.org/nomicon/ffi.html). Users who contribute their own `CatalogProvider`, `SchemaProvider`, `TableProvider` or `TableFunction` via FFI must now provide access to a diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 000000000..1366d92c6 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,167 @@ +site_name: Apache DataFusion in Python +site_description: Python bindings for Apache DataFusion query engine +site_url: https://datafusion.apache.org/python/ +repo_url: https://github.com/apache/datafusion-python +repo_name: apache/datafusion-python +copyright: Copyright 2019-2026, Apache Software Foundation + +docs_dir: docs/source +site_dir: docs/build/html + +theme: + name: material + custom_dir: docs/source/_overrides + logo: _static/images/original.svg + favicon: _static/favicon.svg + features: + - navigation.sections + - navigation.indexes + - navigation.top + - navigation.instant + - navigation.tracking + - content.code.copy + - content.code.annotate + - search.highlight + - search.suggest + - toc.follow + palette: + - media: "(prefers-color-scheme: light)" + scheme: default + primary: white + accent: custom + toggle: + icon: material/brightness-7 + name: Switch to dark mode + - media: "(prefers-color-scheme: dark)" + scheme: slate + primary: black + accent: custom + toggle: + icon: material/brightness-4 + name: Switch to light mode + icon: + repo: fontawesome/brands/github + +extra: + social: + - icon: fontawesome/brands/github + link: https://github.com/apache/datafusion-python + - icon: fontawesome/brands/rust + link: https://docs.rs/datafusion/latest/datafusion/ + name: Rust API docs (docs.rs) + +extra_css: + - _static/theme_overrides.css + +plugins: + - search + - mkdocs-jupyter: + execute: true + allow_errors: false + include_source: false + ignore_h1_titles: true + - mkdocstrings: + default_handler: python + handlers: + python: + paths: [python] + inventories: + - https://docs.python.org/3/objects.inv + - https://arrow.apache.org/docs/objects.inv + options: + docstring_style: google + show_source: false + members_order: source + inherited_members: true + show_root_heading: true + show_signature_annotations: true + separate_signature: true + merge_init_into_class: true + docstring_section_style: spacy + filters: ["!^_"] + - redirects: + redirect_maps: {} + +markdown_extensions: + - admonition + - attr_list + - def_list + - md_in_html + - footnotes + - tables + - toc: + permalink: true + toc_depth: 3 + - pymdownx.details + - pymdownx.superfences + - pymdownx.tabbed: + alternate_style: true + - pymdownx.tasklist: + custom_checkbox: true + - pymdownx.highlight: + anchor_linenums: true + line_spans: __span + pygments_lang_class: true + - pymdownx.inlinehilite + - pymdownx.snippets + +watch: + - python/datafusion + +nav: + - Home: index.ipynb + - User Guide: + - user-guide/index.md + - Introduction: user-guide/introduction.ipynb + - Basics: user-guide/basics.ipynb + - Data Sources: user-guide/data-sources.ipynb + - DataFrame: + - user-guide/dataframe/index.md + - Rendering: user-guide/dataframe/rendering.md + - Execution Metrics: user-guide/dataframe/execution-metrics.md + - Common Operations: + - user-guide/common-operations/index.md + - Views: user-guide/common-operations/views.md + - Basic Info: user-guide/common-operations/basic-info.ipynb + - Select and Filter: user-guide/common-operations/select-and-filter.ipynb + - Expressions: user-guide/common-operations/expressions.ipynb + - Joins: user-guide/common-operations/joins.ipynb + - Functions: user-guide/common-operations/functions.ipynb + - Aggregations: user-guide/common-operations/aggregations.ipynb + - Windows: user-guide/common-operations/windows.ipynb + - UDF and UDAF: user-guide/common-operations/udf-and-udfa.ipynb + - I/O: + - user-guide/io/index.md + - Arrow: user-guide/io/arrow.ipynb + - Avro: user-guide/io/avro.md + - CSV: user-guide/io/csv.md + - JSON: user-guide/io/json.md + - Parquet: user-guide/io/parquet.md + - Table Provider: user-guide/io/table_provider.md + - Configuration: user-guide/configuration.md + - Distributing Work: user-guide/distributing-work.md + - SQL: user-guide/sql.ipynb + - Upgrade Guides: user-guide/upgrade-guides.md + - AI Coding Assistants: user-guide/ai-coding-assistants.md + - Contributor Guide: + - contributor-guide/index.md + - Introduction: contributor-guide/introduction.md + - FFI: contributor-guide/ffi.md + - API Reference: + - reference/index.md + - SessionContext: reference/context.md + - DataFrame: reference/dataframe.md + - Expr: reference/expr.md + - Functions: reference/functions.md + - User-Defined Functions: reference/user_defined.md + - Catalog: reference/catalog.md + - I/O: reference/io.md + - IPC: reference/ipc.md + - Object Store: reference/object_store.md + - Options: reference/options.md + - Plan: reference/plan.md + - RecordBatch: reference/record_batch.md + - Substrait: reference/substrait.md + - Unparser: reference/unparser.md + - Common: reference/common.md + - Links: links.md diff --git a/pyproject.toml b/pyproject.toml index e18c1d57c..c2f7e40b3 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -151,8 +151,10 @@ extend-allowed-calls = ["datafusion.lit", "lit"] "E", "ERA001", "EXE", + "INP001", "N817", "PLR", + "PTH", "S", "SIM", "T", @@ -177,7 +179,17 @@ extend-allowed-calls = ["datafusion.lit", "lit"] "UP", ] "docs/*" = ["D"] -"docs/source/conf.py" = ["ANN001", "ERA001", "INP001"] +# Notebook content cells originate from prose-driven user-guide pages +# where bare `print()` calls, magic comparison values, and per-cell +# re-imports are part of the explanation rather than production code. +"docs/source/**/*.ipynb" = [ + "B905", + "F811", + "ICN001", + "PLR2004", + "PTH118", + "T201", +] # CI and pre-commit invoke codespell with different paths, so we have a little # redundancy here, and we intentionally drop python in the path. @@ -215,13 +227,11 @@ dev = [ release = ["pygithub==2.5.0"] docs = [ "ipython>=8.12.3", - "jinja2>=3.1.5", - "myst-parser>=3.0.1", + "mkdocs>=1.6,<2", + "mkdocs-jupyter>=0.25", + "mkdocs-material>=9.5,<10", + "mkdocs-redirects>=1.2", + "mkdocstrings[python]>=0.27", "pandas>=2.0.3", "pickleshare>=0.7.5", - "pydata-sphinx-theme>=0.16,<0.17", - "setuptools>=75.3.0", - "sphinx-autoapi>=3.4.0", - "sphinx-reredirects>=0.1.5", - "sphinx>=7.1.2", ] diff --git a/python/datafusion/__init__.py b/python/datafusion/__init__.py index 9c55f446c..f4b50980d 100644 --- a/python/datafusion/__init__.py +++ b/python/datafusion/__init__.py @@ -18,7 +18,7 @@ """DataFusion: an in-process query engine built on Apache Arrow. DataFusion is not a database -- it has no server and no external dependencies. -You create a :py:class:`SessionContext`, point it at data sources (Parquet, CSV, +You create a `SessionContext`, point it at data sources (Parquet, CSV, JSON, Arrow IPC, Pandas, Polars, or raw Python dicts/lists), and run queries using either SQL or the DataFrame API. @@ -27,10 +27,10 @@ - **SessionContext** -- entry point for loading data, running SQL, and creating DataFrames. - **DataFrame** -- lazy query builder. Every method returns a new DataFrame; - call :py:meth:`~datafusion.dataframe.DataFrame.collect` or a ``to_*`` + call [`collect`][datafusion.dataframe.DataFrame.collect] or a ``to_*`` method to execute. - **Expr** -- expression tree node for column references, literals, and function - calls. Build with :py:func:`col` and :py:func:`lit`. + calls. Build with [`col`][datafusion.col.col] and [`lit`][datafusion.lit]. - **functions** -- 290+ built-in scalar, aggregate, and window functions. Quick start diff --git a/python/datafusion/context.py b/python/datafusion/context.py index 5dfeed719..6bdb68c60 100644 --- a/python/datafusion/context.py +++ b/python/datafusion/context.py @@ -15,22 +15,22 @@ # specific language governing permissions and limitations # under the License. -""":py:class:`SessionContext` — entry point for running DataFusion queries. +"""`SessionContext` — entry point for running DataFusion queries. -A :py:class:`SessionContext` holds registered tables, catalogs, and +A `SessionContext` holds registered tables, catalogs, and configuration for the current session. It is the first object most programs create: from it you register data, run SQL strings -(:py:meth:`SessionContext.sql`), read files -(:py:meth:`SessionContext.read_csv`, -:py:meth:`SessionContext.read_parquet`, ...), and construct -:py:class:`~datafusion.dataframe.DataFrame` objects in memory -(:py:meth:`SessionContext.from_pydict`, -:py:meth:`SessionContext.from_arrow`). +([`sql`][SessionContext.sql]), read files +([`read_csv`][SessionContext.read_csv], +[`read_parquet`][SessionContext.read_parquet], ...), and construct +[`DataFrame`][datafusion.dataframe.DataFrame] objects in memory +([`from_pydict`][SessionContext.from_pydict], +[`from_arrow`][SessionContext.from_arrow]). Session behavior (memory limits, batch size, configured optimizer passes, -...) is controlled by :py:class:`SessionConfig` and -:py:class:`RuntimeEnvBuilder`; SQL dialect limits are controlled by -:py:class:`SQLOptions`. +...) is controlled by [`SessionConfig`][datafusion.context.SessionConfig] and +`RuntimeEnvBuilder`; SQL dialect limits are controlled by +[`SQLOptions`][datafusion.context.SQLOptions]. Examples: >>> ctx = dfn.SessionContext() @@ -147,7 +147,7 @@ class SessionConfig: """Session configuration options.""" def __init__(self, config_options: dict[str, str] | None = None) -> None: - """Create a new :py:class:`SessionConfig` with the given configuration options. + """Create a new `SessionConfig` with the given configuration options. Args: config_options: Configuration options. @@ -164,7 +164,7 @@ def with_create_default_catalog_and_schema( automatically created. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = ( self.config_internal.with_create_default_catalog_and_schema(enabled) @@ -181,7 +181,7 @@ def with_default_catalog_and_schema( schema: Schema name. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_default_catalog_and_schema( catalog, schema @@ -195,7 +195,7 @@ def with_information_schema(self, enabled: bool = True) -> SessionConfig: enabled: Whether to include ``information_schema`` virtual tables. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_information_schema(enabled) return self @@ -207,7 +207,7 @@ def with_batch_size(self, batch_size: int) -> SessionConfig: batch_size: Batch size. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_batch_size(batch_size) return self @@ -221,7 +221,7 @@ def with_target_partitions(self, target_partitions: int) -> SessionConfig: target_partitions: Number of target partitions. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_target_partitions( target_partitions @@ -237,7 +237,7 @@ def with_repartition_aggregations(self, enabled: bool = True) -> SessionConfig: enabled: Whether to use repartitioning for aggregations. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_repartition_aggregations( enabled @@ -251,7 +251,7 @@ def with_repartition_joins(self, enabled: bool = True) -> SessionConfig: enabled: Whether to use repartitioning for joins. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_repartition_joins(enabled) return self @@ -265,7 +265,7 @@ def with_repartition_windows(self, enabled: bool = True) -> SessionConfig: enabled: Whether to use repartitioning for window functions. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_repartition_windows(enabled) return self @@ -279,7 +279,7 @@ def with_repartition_sorts(self, enabled: bool = True) -> SessionConfig: enabled: Whether to use repartitioning for window functions. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_repartition_sorts(enabled) return self @@ -291,7 +291,7 @@ def with_repartition_file_scans(self, enabled: bool = True) -> SessionConfig: enabled: Whether to use repartitioning for file scans. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_repartition_file_scans(enabled) return self @@ -303,7 +303,7 @@ def with_repartition_file_min_size(self, size: int) -> SessionConfig: size: Minimum file range size. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_repartition_file_min_size(size) return self @@ -317,7 +317,7 @@ def with_parquet_pruning(self, enabled: bool = True) -> SessionConfig: enabled: Whether to use pruning predicate for parquet readers. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_parquet_pruning(enabled) return self @@ -330,7 +330,7 @@ def set(self, key: str, value: str) -> SessionConfig: value: Option value. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.set(key, value) return self @@ -343,7 +343,7 @@ def with_extension(self, extension: Any) -> SessionConfig: shared from another DataFusion extension library. Returns: - A new :py:class:`SessionConfig` object with the updated setting. + A new `SessionConfig` object with the updated setting. """ self.config_internal = self.config_internal.with_extension(extension) return self @@ -353,14 +353,14 @@ class RuntimeEnvBuilder: """Runtime configuration options.""" def __init__(self) -> None: - """Create a new :py:class:`RuntimeEnvBuilder` with default values.""" + """Create a new `RuntimeEnvBuilder` with default values.""" self.config_internal = RuntimeEnvBuilderInternal() def with_disk_manager_disabled(self) -> RuntimeEnvBuilder: """Disable the disk manager, attempts to create temporary files will error. Returns: - A new :py:class:`RuntimeEnvBuilder` object with the updated setting. + A new `RuntimeEnvBuilder` object with the updated setting. """ self.config_internal = self.config_internal.with_disk_manager_disabled() return self @@ -369,7 +369,7 @@ def with_disk_manager_os(self) -> RuntimeEnvBuilder: """Use the operating system's temporary directory for disk manager. Returns: - A new :py:class:`RuntimeEnvBuilder` object with the updated setting. + A new `RuntimeEnvBuilder` object with the updated setting. """ self.config_internal = self.config_internal.with_disk_manager_os() return self @@ -383,7 +383,7 @@ def with_disk_manager_specified( paths: Paths to use for the disk manager's temporary files. Returns: - A new :py:class:`RuntimeEnvBuilder` object with the updated setting. + A new `RuntimeEnvBuilder` object with the updated setting. """ paths_list = [str(p) for p in paths] self.config_internal = self.config_internal.with_disk_manager_specified( @@ -395,7 +395,7 @@ def with_unbounded_memory_pool(self) -> RuntimeEnvBuilder: """Use an unbounded memory pool. Returns: - A new :py:class:`RuntimeEnvBuilder` object with the updated setting. + A new `RuntimeEnvBuilder` object with the updated setting. """ self.config_internal = self.config_internal.with_unbounded_memory_pool() return self @@ -421,7 +421,7 @@ def with_fair_spill_pool(self, size: int) -> RuntimeEnvBuilder: size: Size of the memory pool in bytes. Returns: - A new :py:class:`RuntimeEnvBuilder` object with the updated setting. + A new `RuntimeEnvBuilder` object with the updated setting. Examples: >>> config = dfn.RuntimeEnvBuilder().with_fair_spill_pool(1024) @@ -433,14 +433,14 @@ def with_greedy_memory_pool(self, size: int) -> RuntimeEnvBuilder: """Use a greedy memory pool with the specified size. This pool works well for queries that do not need to spill or have a single - spillable operator. See :py:func:`with_fair_spill_pool` if there are + spillable operator. See `with_fair_spill_pool` if there are multiple spillable operators that all will spill. Args: size: Size of the memory pool in bytes. Returns: - A new :py:class:`RuntimeEnvBuilder` object with the updated setting. + A new `RuntimeEnvBuilder` object with the updated setting. Examples: >>> config = dfn.RuntimeEnvBuilder().with_greedy_memory_pool(1024) @@ -455,7 +455,7 @@ def with_temp_file_path(self, path: str | pathlib.Path) -> RuntimeEnvBuilder: path: Path to use for temporary files. Returns: - A new :py:class:`RuntimeEnvBuilder` object with the updated setting. + A new `RuntimeEnvBuilder` object with the updated setting. Examples: >>> config = dfn.RuntimeEnvBuilder().with_temp_file_path("/tmp") @@ -468,7 +468,7 @@ class SQLOptions: """Options to be used when performing SQL queries.""" def __init__(self) -> None: - """Create a new :py:class:`SQLOptions` with default values. + """Create a new `SQLOptions` with default values. The default values are: - DDL commands are allowed @@ -486,7 +486,7 @@ def with_allow_ddl(self, allow: bool = True) -> SQLOptions: allow: Allow DDL commands to be run. Returns: - A new :py:class:`SQLOptions` object with the updated setting. + A new `SQLOptions` object with the updated setting. Examples: >>> options = dfn.SQLOptions().with_allow_ddl(True) @@ -503,7 +503,7 @@ def with_allow_dml(self, allow: bool = True) -> SQLOptions: allow: Allow DML commands to be run. Returns: - A new :py:class:`SQLOptions` object with the updated setting. + A new `SQLOptions` object with the updated setting. Examples: >>> options = dfn.SQLOptions().with_allow_dml(True) @@ -551,7 +551,7 @@ def __init__( Example usage: The following example demonstrates how to use the context to execute - a query against a CSV data source using the :py:class:`DataFrame` API:: + a query against a CSV data source using the `DataFrame` API:: from datafusion import SessionContext @@ -583,7 +583,7 @@ def enable_url_table(self) -> SessionContext: """Control if local files can be queried as tables. Returns: - A new :py:class:`SessionContext` object with url table enabled. + A new `SessionContext` object with url table enabled. """ klass = self.__class__ obj = klass.__new__(klass) @@ -597,7 +597,7 @@ def register_object_store( Args: schema: The data source schema. - store: The :py:class:`~datafusion.object_store.ObjectStore` to register. + store: The [`ObjectStore`][datafusion.object_store.ObjectStore] to register. host: URL for the host. """ self.ctx.register_object_store(schema, store, host) @@ -622,8 +622,8 @@ def register_listing_table( ) -> None: """Register multiple files as a single table. - Registers a :py:class:`~datafusion.catalog.Table` that can assemble multiple - files from locations in an :py:class:`~datafusion.object_store.ObjectStore` + Registers a [`Table`][datafusion.catalog.Table] that can assemble multiple + files from locations in an [`ObjectStore`][datafusion.object_store.ObjectStore] instance. Args: @@ -655,7 +655,7 @@ def sql( param_values: dict[str, Any] | None = None, **named_params: Any, ) -> DataFrame: - """Create a :py:class:`~datafusion.DataFrame` from SQL query text. + """Create a [`DataFrame`][datafusion.DataFrame] from SQL query text. See the online documentation for a description of how to perform parameterized substitution via either the ``param_values`` option @@ -664,7 +664,7 @@ def sql( Note: This API implements DDL statements such as ``CREATE TABLE`` and ``CREATE VIEW`` and DML statements such as ``INSERT INTO`` with in-memory default implementation.See - :py:func:`~datafusion.context.SessionContext.sql_with_options`. + [`sql_with_options`][datafusion.context.SessionContext.sql_with_options]. Args: query: SQL query text. @@ -720,7 +720,7 @@ def sql_with_options( param_values: dict[str, Any] | None = None, **named_params: Any, ) -> DataFrame: - """Create a :py:class:`~datafusion.dataframe.DataFrame` from SQL query text. + """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from SQL query text. This function will first validate that the query is allowed by the provided options. @@ -748,7 +748,7 @@ def create_dataframe( """Create and return a dataframe using the provided partitions. Args: - partitions: :py:class:`pa.RecordBatch` partitions to register. + partitions: [`RecordBatch`][pa.RecordBatch] partitions to register. name: Resultant dataframe name. schema: Schema for the partitions. @@ -758,7 +758,7 @@ def create_dataframe( return DataFrame(self.ctx.create_dataframe(partitions, name, schema)) def create_dataframe_from_logical_plan(self, plan: LogicalPlan) -> DataFrame: - """Create a :py:class:`~datafusion.dataframe.DataFrame` from an existing plan. + """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from an existing plan. Args: plan: Logical plan. @@ -771,7 +771,7 @@ def create_dataframe_from_logical_plan(self, plan: LogicalPlan) -> DataFrame: def from_pylist( self, data: list[dict[str, Any]], name: str | None = None ) -> DataFrame: - """Create a :py:class:`~datafusion.dataframe.DataFrame` from a list. + """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from a list. Args: data: List of dictionaries. @@ -785,7 +785,7 @@ def from_pylist( def from_pydict( self, data: dict[str, list[Any]], name: str | None = None ) -> DataFrame: - """Create a :py:class:`~datafusion.dataframe.DataFrame` from a dictionary. + """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from a dictionary. Args: data: Dictionary of lists. @@ -801,7 +801,7 @@ def from_arrow( data: ArrowStreamExportable | ArrowArrayExportable, name: str | None = None, ) -> DataFrame: - """Create a :py:class:`~datafusion.dataframe.DataFrame` from an Arrow source. + """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from an Arrow source. The Arrow data source can be any object that implements either ``__arrow_c_stream__`` or ``__arrow_c_array__``. For the latter, it must return @@ -819,7 +819,7 @@ def from_arrow( return DataFrame(self.ctx.from_arrow(data, name)) def from_pandas(self, data: pd.DataFrame, name: str | None = None) -> DataFrame: - """Create a :py:class:`~datafusion.dataframe.DataFrame` from a Pandas DataFrame. + """Create a `DataFrame` from a Pandas DataFrame. Args: data: Pandas DataFrame. @@ -831,7 +831,7 @@ def from_pandas(self, data: pd.DataFrame, name: str | None = None) -> DataFrame: return DataFrame(self.ctx.from_pandas(data, name)) def from_polars(self, data: pl.DataFrame, name: str | None = None) -> DataFrame: - """Create a :py:class:`~datafusion.dataframe.DataFrame` from a Polars DataFrame. + """Create a `DataFrame` from a Polars DataFrame. Args: data: Polars DataFrame. @@ -845,7 +845,7 @@ def from_polars(self, data: pl.DataFrame, name: str | None = None) -> DataFrame: # https://github.com/apache/datafusion-python/pull/1016#discussion_r1983239116 # is the discussion on how we arrived at adding register_view def register_view(self, name: str, df: DataFrame) -> None: - """Register a :py:class:`~datafusion.dataframe.DataFrame` as a view. + """Register a [`DataFrame`][datafusion.dataframe.DataFrame] as a view. Args: name (str): The name to register the view under. @@ -859,7 +859,7 @@ def register_table( name: str, table: Table | TableProviderExportable | DataFrame | pa.dataset.Dataset, ) -> None: - """Register a :py:class:`~datafusion.Table` with this context. + """Register a [`Table`][datafusion.Table] with this context. The registered table can be referenced from SQL statements executed against this context. @@ -879,7 +879,7 @@ def register_table_factory( format: str, factory: TableProviderFactory | TableProviderFactoryExportable, ) -> None: - """Register a :py:class:`~datafusion.TableProviderFactoryExportable`. + """Register a `TableProviderFactoryExportable`. The registered factory can be referenced from SQL DDL statements executed against this context. @@ -930,7 +930,7 @@ def register_udtf(self, func: TableFunction) -> None: self.ctx.register_udtf(func._udtf) def register_batch(self, name: str, batch: pa.RecordBatch) -> None: - """Register a single :py:class:`pa.RecordBatch` as a table. + """Register a single [`RecordBatch`][pa.RecordBatch] as a table. Args: name: Name of the resultant table. @@ -973,12 +973,12 @@ def register_record_batches( self.ctx.register_record_batches(name, partitions) def read_batch(self, batch: pa.RecordBatch) -> DataFrame: - """Return a :py:class:`~datafusion.DataFrame` reading a single batch. + """Return a [`DataFrame`][datafusion.DataFrame] reading a single batch. - Convenience wrapper around :py:meth:`read_batches` for the single-batch - case. Unlike :py:meth:`register_batch`, this does not register the + Convenience wrapper around [`read_batches`][read_batches] for the single-batch + case. Unlike [`register_batch`][register_batch], this does not register the batch as a named table; it returns an anonymous - :py:class:`~datafusion.DataFrame` directly. + [`DataFrame`][datafusion.DataFrame] directly. Args: batch: Record batch to wrap as a DataFrame. @@ -992,14 +992,14 @@ def read_batch(self, batch: pa.RecordBatch) -> DataFrame: return self.read_batches([batch]) def read_batches(self, batches: Iterable[pa.RecordBatch]) -> DataFrame: - """Return a :py:class:`~datafusion.DataFrame` reading the given batches. + """Return a [`DataFrame`][datafusion.DataFrame] reading the given batches. All batches must share the same schema. Any iterable of - :py:class:`pa.RecordBatch` is accepted (list, tuple, generator); + [`RecordBatch`][pa.RecordBatch] is accepted (list, tuple, generator); it is materialized into a list before being handed to the - underlying Rust binding. Unlike :py:meth:`register_record_batches`, + underlying Rust binding. Unlike `register_record_batches`, this does not register the batches as a named table; it returns - an anonymous :py:class:`~datafusion.DataFrame` directly. + an anonymous [`DataFrame`][datafusion.DataFrame] directly. Args: batches: Record batches to wrap as a DataFrame. @@ -1279,7 +1279,7 @@ def register_arrow( ) def register_dataset(self, name: str, dataset: pa.dataset.Dataset) -> None: - """Register a :py:class:`pa.dataset.Dataset` as a table. + """Register a [`Dataset`][pa.dataset.Dataset] as a table. Args: name: Name of the table to register. @@ -1326,9 +1326,9 @@ def deregister_udwf(self, name: str) -> None: def udf(self, name: str) -> ScalarUDF: """Look up a registered scalar UDF by name. - Returns the same ``ScalarUDF`` wrapper that :py:meth:`register_udf` + Returns the same ``ScalarUDF`` wrapper that [`register_udf`][register_udf] accepts, so it can be invoked as an expression in the DataFrame API - or re-registered into a different :py:class:`SessionContext`. + or re-registered into a different `SessionContext`. Built-in scalar functions from the session's function registry are also looked up. @@ -1372,9 +1372,9 @@ def udf(self, name: str) -> ScalarUDF: def udaf(self, name: str) -> AggregateUDF: """Look up a registered aggregate UDF by name. - Returns the same ``AggregateUDF`` wrapper that :py:meth:`register_udaf` + Returns the same ``AggregateUDF`` wrapper that [`register_udaf`][register_udaf] accepts. Built-in aggregate functions such as ``sum`` or ``avg`` are - also discoverable through this lookup. See :py:meth:`udf` for a worked + also discoverable through this lookup. See `udf` for a worked late-binding example; the pattern is identical for aggregates. Args: @@ -1385,7 +1385,7 @@ def udaf(self, name: str) -> AggregateUDF: Examples: Look up a built-in aggregate by name and use it in - :py:meth:`~datafusion.DataFrame.aggregate`: + [`aggregate`][datafusion.DataFrame.aggregate]: >>> ctx = dfn.SessionContext() >>> sum_fn = ctx.udaf("sum") @@ -1402,9 +1402,9 @@ def udaf(self, name: str) -> AggregateUDF: def udwf(self, name: str) -> WindowUDF: """Look up a registered window UDF by name. - Returns the same ``WindowUDF`` wrapper that :py:meth:`register_udwf` + Returns the same ``WindowUDF`` wrapper that [`register_udwf`][register_udwf] accepts. Built-in window functions such as ``row_number`` or ``rank`` - are also discoverable through this lookup. See :py:meth:`udf` for a + are also discoverable through this lookup. See `udf` for a worked late-binding example; the pattern is identical for window functions. @@ -1432,7 +1432,7 @@ def udfs(self) -> list[str]: """Return the sorted names of all registered scalar UDFs. Includes both user-registered and built-in scalar functions. Pair - with :py:meth:`udf` to drive discovery, validation, or config-based + with `udf` to drive discovery, validation, or config-based dispatch. Examples: @@ -1475,11 +1475,11 @@ def table_exist(self, name: str) -> bool: return self.ctx.table_exist(name) def empty_table(self) -> DataFrame: - """Create an empty :py:class:`~datafusion.dataframe.DataFrame`.""" + """Create an empty [`DataFrame`][datafusion.dataframe.DataFrame].""" return DataFrame(self.ctx.empty_table()) def session_id(self) -> str: - """Return an id that uniquely identifies this :py:class:`SessionContext`.""" + """Return an id that uniquely identifies this `SessionContext`.""" return self.ctx.session_id() def session_start_time(self) -> str: @@ -1503,7 +1503,7 @@ def enable_ident_normalization(self) -> bool: return self.ctx.enable_ident_normalization() def copied_config(self) -> SessionConfig: - """Return a copy of the active :py:class:`SessionConfig`. + """Return a copy of the active `SessionConfig`. Mutating the returned config does not affect this context; use the result when you need a starting point for a new context or @@ -1527,7 +1527,7 @@ def parse_capacity_limit(config_name: str, limit: str) -> int: ``"0"`` is accepted and returns 0. ``config_name`` is used purely for error messages and identifies which configuration setting the limit belongs to. Use this helper when constructing a - :py:class:`RuntimeEnvBuilder` from a human-friendly size string. + `RuntimeEnvBuilder` from a human-friendly size string. Examples: >>> SessionContext.parse_capacity_limit( @@ -1563,7 +1563,7 @@ def parse_sql_expr(self, sql: str, schema: DFSchema) -> Expr: return Expr(self.ctx.parse_sql_expr(sql, schema)) def execute_logical_plan(self, plan: LogicalPlan) -> DataFrame: - """Execute a :py:class:`~datafusion.plan.LogicalPlan` and return a DataFrame. + """Execute a `LogicalPlan` and return a DataFrame. Args: plan: Logical plan to execute. @@ -1636,7 +1636,7 @@ def add_physical_optimizer_rule( self.ctx.add_physical_optimizer_rule(rule) def table_provider(self, name: str) -> Table: - """Return the :py:class:`~datafusion.catalog.Table` for the given table name. + """Return the [`Table`][datafusion.catalog.Table] for the given table name. Args: name: Name of the table. @@ -1782,7 +1782,7 @@ def read_parquet( schema: pa.Schema | None = None, file_sort_order: Sequence[Sequence[SortKey]] | None = None, ) -> DataFrame: - """Read a Parquet source into a :py:class:`~datafusion.dataframe.Dataframe`. + """Read a Parquet source into a [`Dataframe`][datafusion.dataframe.Dataframe]. Args: path: Path to the Parquet file. @@ -1827,7 +1827,7 @@ def read_avro( file_partition_cols: list[tuple[str, str | pa.DataType]] | None = None, file_extension: str = ".avro", ) -> DataFrame: - """Create a :py:class:`DataFrame` for reading Avro data source. + """Create a `DataFrame` for reading Avro data source. Args: path: Path to the Avro file. @@ -1852,7 +1852,7 @@ def read_arrow( file_extension: str = ".arrow", file_partition_cols: list[tuple[str, str | pa.DataType]] | None = None, ) -> DataFrame: - """Create a :py:class:`DataFrame` for reading an Arrow IPC data source. + """Create a `DataFrame` for reading an Arrow IPC data source. Args: path: Path to the Arrow IPC file. @@ -1918,7 +1918,7 @@ def read_arrow( ) def read_empty(self) -> DataFrame: - """Create an empty :py:class:`DataFrame` with no columns or rows. + """Create an empty `DataFrame` with no columns or rows. See Also: This is an alias for :meth:`empty_table`. @@ -1928,7 +1928,7 @@ def read_empty(self) -> DataFrame: def read_table( self, table: Table | TableProviderExportable | DataFrame | pa.dataset.Dataset ) -> DataFrame: - """Creates a :py:class:`~datafusion.dataframe.DataFrame` from a table.""" + """Creates a [`DataFrame`][datafusion.dataframe.DataFrame] from a table.""" return DataFrame(self.ctx.read_table(table)) def execute(self, plan: ExecutionPlan, partitions: int) -> RecordBatchStream: @@ -2006,7 +2006,7 @@ def with_logical_extension_codec( Only FFI codecs are supported. Pass any object implementing ``__datafusion_logical_extension_codec__`` (see - :py:class:`~datafusion.user_defined.LogicalExtensionCodecExportable`). + `LogicalExtensionCodecExportable`). """ new_internal = self.ctx.with_logical_extension_codec(codec) new = SessionContext.__new__(SessionContext) @@ -2024,7 +2024,7 @@ def with_physical_extension_codec( Only FFI codecs are supported. Pass any object implementing ``__datafusion_physical_extension_codec__`` (see - :py:class:`~datafusion.user_defined.PhysicalExtensionCodecExportable`). + `PhysicalExtensionCodecExportable`). """ new_internal = self.ctx.with_physical_extension_codec(codec) new = SessionContext.__new__(SessionContext) diff --git a/python/datafusion/dataframe.py b/python/datafusion/dataframe.py index de00ff474..9baaed6b9 100644 --- a/python/datafusion/dataframe.py +++ b/python/datafusion/dataframe.py @@ -14,23 +14,23 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. -""":py:class:`DataFrame` — lazy, chainable query representation. +"""[`DataFrame`][datafusion.dataframe.DataFrame] — lazy, chainable query representation. -A :py:class:`DataFrame` is a logical plan over one or more data sources. -Methods that reshape the plan (:py:meth:`DataFrame.select`, -:py:meth:`DataFrame.filter`, :py:meth:`DataFrame.aggregate`, -:py:meth:`DataFrame.sort`, :py:meth:`DataFrame.join`, -:py:meth:`DataFrame.limit`, the set-operation methods, ...) return a new -:py:class:`DataFrame` and do no work until a terminal method such as -:py:meth:`DataFrame.collect`, :py:meth:`DataFrame.to_pydict`, -:py:meth:`DataFrame.show`, or one of the ``write_*`` methods is called. +A `DataFrame` is a logical plan over one or more data sources. +Methods that reshape the plan ([`select`][datafusion.dataframe.DataFrame.select], +`filter`, `aggregate`, +[`sort`][DataFrame.sort], [`join`][DataFrame.join], +`limit`, the set-operation methods, ...) return a new +`DataFrame` and do no work until a terminal method such as +[`collect`][datafusion.dataframe.DataFrame.collect], [`to_pydict`][DataFrame.to_pydict], +[`show`][DataFrame.show], or one of the ``write_*`` methods is called. DataFrames are produced from a -:py:class:`~datafusion.context.SessionContext`, typically via -:py:meth:`~datafusion.context.SessionContext.sql`, -:py:meth:`~datafusion.context.SessionContext.read_csv`, -:py:meth:`~datafusion.context.SessionContext.read_parquet`, or -:py:meth:`~datafusion.context.SessionContext.from_pydict`. +[`SessionContext`][datafusion.context.SessionContext], typically via +[`sql`][datafusion.context.SessionContext.sql], +[`read_csv`][datafusion.context.SessionContext.read_csv], +[`read_parquet`][datafusion.context.SessionContext.read_parquet], or +[`from_pydict`][datafusion.context.SessionContext.from_pydict]. Examples: >>> ctx = dfn.SessionContext() @@ -92,7 +92,7 @@ class ExplainFormat(Enum): """Output format for explain plans. - Controls how the query plan is rendered in :py:meth:`DataFrame.explain`. + Controls how the query plan is rendered in [`explain`][DataFrame.explain]. """ INDENT = "indent" @@ -356,8 +356,8 @@ class DataFrame: def __init__(self, df: DataFrameInternal) -> None: """This constructor is not to be used by the end user. - See :py:class:`~datafusion.context.SessionContext` for methods to - create a :py:class:`DataFrame`. + See [`SessionContext`][datafusion.context.SessionContext] for methods to + create a [`DataFrame`][datafusion.dataframe.DataFrame]. """ self.df = df @@ -379,7 +379,7 @@ def into_view(self, temporary: bool = False) -> Table: return _Table(self.df.into_view(temporary)) def __getitem__(self, key: str | list[str]) -> DataFrame: - """Return a new :py:class:`DataFrame` with the specified column or columns. + """Return a new `DataFrame` with the specified column or columns. Args: key: Column name or list of column names to select. @@ -428,7 +428,7 @@ def describe(self) -> DataFrame: return DataFrame(self.df.describe()) def schema(self) -> pa.Schema: - """Return the :py:class:`pyarrow.Schema` of this DataFrame. + """Return the [`Schema`][pyarrow.Schema] of this DataFrame. The output schema contains information on the name, data type, and nullability for each column. @@ -442,7 +442,7 @@ def column(self, name: str) -> Expr: """Return a fully qualified column expression for ``name``. Resolves an unqualified column name against this DataFrame's schema - and returns an :py:class:`Expr` whose underlying column reference + and returns an [`Expr`][datafusion.expr.Expr] whose underlying column reference includes the table qualifier. This is especially useful after joins, where the same column name may appear in multiple relations. @@ -477,17 +477,17 @@ def column(self, name: str) -> Expr: return self.find_qualified_columns(name)[0] def col(self, name: str) -> Expr: - """Alias for :py:meth:`column`. + """Alias for [`column`][datafusion.col.column]. See Also: - :py:meth:`column` + [`column`][datafusion.col.column] """ return self.column(name) def find_qualified_columns(self, *names: str) -> list[Expr]: """Return fully qualified column expressions for the given names. - This is a batch version of :py:meth:`column` — it resolves each + This is a batch version of [`column`][datafusion.col.column] — it resolves each unqualified name against the DataFrame's schema and returns a list of qualified column expressions. @@ -524,7 +524,7 @@ def select_exprs(self, *args: str) -> DataFrame: return self.df.select_exprs(*args) def alias(self, alias: str) -> DataFrame: - """Assign a table alias to this :py:class:`DataFrame`. + """Assign a table alias to this [`DataFrame`][datafusion.dataframe.DataFrame]. Replaces the qualifiers of the output columns with ``alias``. Useful for self-joins and any situation that needs an unambiguous table-style @@ -550,13 +550,13 @@ def alias(self, alias: str) -> DataFrame: return DataFrame(self.df.alias(alias)) def select(self, *exprs: Expr | str) -> DataFrame: - """Project arbitrary expressions into a new :py:class:`DataFrame`. + """Project arbitrary expressions into a new `DataFrame`. - String arguments are treated as column names; :py:class:`~datafusion.expr.Expr` + String arguments are treated as column names; [`Expr`][datafusion.expr.Expr] arguments can reshape, rename, or compute new columns. Args: - exprs: Either column names or :py:class:`~datafusion.expr.Expr` to select. + exprs: Either column names or [`Expr`][datafusion.expr.Expr] to select. Returns: DataFrame after projection. It has one column for each expression. @@ -647,7 +647,7 @@ def filter(self, *predicates: Expr | str) -> DataFrame: :class:`~datafusion.expr.Expr` created using helper functions such as :func:`datafusion.col` or :func:`datafusion.lit`, or a SQL expression string that will be parsed against the DataFrame schema. If more complex logic is - required, see the logical operations in :py:mod:`~datafusion.functions`. + required, see the logical operations in [`functions`][datafusion.functions]. Examples: >>> ctx = dfn.SessionContext() @@ -806,18 +806,18 @@ def aggregate( By default each unique combination of the ``group_by`` columns produces one row. To get multiple levels of subtotals in a single pass, pass a - :py:class:`~datafusion.expr.GroupingSet` expression + [`GroupingSet`][datafusion.expr.GroupingSet] expression (created via - :py:meth:`~datafusion.expr.GroupingSet.rollup`, - :py:meth:`~datafusion.expr.GroupingSet.cube`, or - :py:meth:`~datafusion.expr.GroupingSet.grouping_sets`) + [`rollup`][datafusion.expr.GroupingSet.rollup], + [`cube`][datafusion.expr.GroupingSet.cube], or + [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]) as the ``group_by`` argument. See the :ref:`aggregation` user guide for detailed examples. Args: group_by: Sequence of expressions or column names to group by, or ``None`` for aggregation over the whole DataFrame. - A :py:class:`~datafusion.expr.GroupingSet` expression may + A [`GroupingSet`][datafusion.expr.GroupingSet] expression may be included to produce multiple grouping levels (rollup, cube, or explicit grouping sets). aggs: Sequence of expressions to aggregate. @@ -867,7 +867,7 @@ def sort(self, *exprs: SortKey) -> DataFrame: Note that any expression can be turned into a sort expression by calling its ``sort`` method. For ascending-only sorts, the shorter - :py:meth:`sort_by` is usually more convenient. + [`sort_by`][sort_by] is usually more convenient. Args: exprs: Sort expressions or column names, applied in order. @@ -883,7 +883,7 @@ def sort(self, *exprs: SortKey) -> DataFrame: >>> df.sort("a").to_pydict() {'a': [1, 2, 3], 'b': [20, 30, 10]} - Sort descending using :py:meth:`Expr.sort`: + Sort descending using [`sort`][Expr.sort]: >>> df.sort(col("a").sort(ascending=False)).to_pydict() {'a': [3, 2, 1], 'b': [10, 30, 20]} @@ -904,10 +904,10 @@ def cast(self, mapping: dict[str, pa.DataType[Any]]) -> DataFrame: return self.with_columns(exprs) def limit(self, count: int, offset: int = 0) -> DataFrame: - """Return a new :py:class:`DataFrame` with a limited number of rows. + """Return a new `DataFrame` with a limited number of rows. Results are returned in unspecified order unless the DataFrame is - explicitly sorted first via :py:meth:`sort` or :py:meth:`sort_by`. + explicitly sorted first via [`sort`][sort] or [`sort_by`][sort_by]. Args: count: Number of rows to limit the DataFrame to. @@ -932,7 +932,7 @@ def limit(self, count: int, offset: int = 0) -> DataFrame: return DataFrame(self.df.limit(count, offset)) def head(self, n: int = 5) -> DataFrame: - """Return a new :py:class:`DataFrame` with a limited number of rows. + """Return a new `DataFrame` with a limited number of rows. Args: n: Number of rows to take from the head of the DataFrame. @@ -943,7 +943,7 @@ def head(self, n: int = 5) -> DataFrame: return DataFrame(self.df.limit(n, 0)) def tail(self, n: int = 5) -> DataFrame: - """Return a new :py:class:`DataFrame` with a limited number of rows. + """Return a new `DataFrame` with a limited number of rows. Be aware this could be potentially expensive since the row size needs to be determined of the dataframe. This is done by collecting it. @@ -957,19 +957,19 @@ def tail(self, n: int = 5) -> DataFrame: return DataFrame(self.df.limit(n, max(0, self.count() - n))) def collect(self) -> list[pa.RecordBatch]: - """Execute this :py:class:`DataFrame` and collect results into memory. + """Execute this `DataFrame` and collect results into memory. Prior to calling ``collect``, modifying a DataFrame simply updates a plan (no actual computation is performed). Calling ``collect`` triggers the computation. Returns: - List of :py:class:`pyarrow.RecordBatch` collected from the DataFrame. + List of [`RecordBatch`][pyarrow.RecordBatch] collected from the DataFrame. """ return self.df.collect() def collect_column(self, column_name: str) -> pa.Array | pa.ChunkedArray: - """Executes this :py:class:`DataFrame` for a single column.""" + """Executes this `DataFrame` for a single column.""" return self.df.collect_column(column_name) def cache(self) -> DataFrame: @@ -983,11 +983,11 @@ def cache(self) -> DataFrame: def collect_partitioned(self) -> list[list[pa.RecordBatch]]: """Execute this DataFrame and collect all partitioned results. - This operation returns :py:class:`pyarrow.RecordBatch` maintaining the input + This operation returns `RecordBatch` maintaining the input partitioning. Returns: - List of list of :py:class:`RecordBatch` collected from the + List of list of `RecordBatch` collected from the DataFrame. """ return self.df.collect_partitioned() @@ -1001,7 +1001,7 @@ def show(self, num: int = 20) -> None: self.df.show(num) def distinct(self) -> DataFrame: - """Return a new :py:class:`DataFrame` with all duplicated rows removed. + """Return a new `DataFrame` with all duplicated rows removed. Returns: DataFrame after removing duplicates. @@ -1058,15 +1058,15 @@ def join( join_keys: tuple[list[str], list[str]] | None = None, coalesce_duplicate_keys: bool = True, ) -> DataFrame: - """Join this :py:class:`DataFrame` with another :py:class:`DataFrame`. + """Join this `DataFrame` with another `DataFrame`. ``on`` has to be provided or both ``left_on`` and ``right_on`` in conjunction. When non-key columns share the same name in both DataFrames, use - :py:meth:`DataFrame.col` on each DataFrame **before** the join to + [`col`][DataFrame.col] on each DataFrame **before** the join to obtain fully qualified column references that can disambiguate them. - See :py:meth:`join_on` for an example. + See [`join_on`][join_on] for an example. Args: right: Other DataFrame to join with. @@ -1156,13 +1156,13 @@ def join_on( *on_exprs: Expr, how: Literal["inner", "left", "right", "full", "semi", "anti"] = "inner", ) -> DataFrame: - """Join two :py:class:`DataFrame` using the specified expressions. + """Join two `DataFrame` using the specified expressions. Join predicates must be :class:`~datafusion.expr.Expr` objects, typically built with :func:`datafusion.col`. On expressions are used to support in-equality predicates. Equality predicates are correctly optimized. - Use :py:meth:`DataFrame.col` on each DataFrame **before** the join to + Use [`col`][DataFrame.col] on each DataFrame **before** the join to obtain fully qualified column references. These qualified references can then be used in the join predicate and to disambiguate columns with the same name when selecting from the result. @@ -1178,7 +1178,7 @@ def join_on( ... ).sort(col("x")).to_pydict() {'a': [1, 2], 'x': ['a', 'b'], 'b': [1, 2], 'y': ['c', 'd']} - Use :py:meth:`col` to disambiguate shared column names: + Use [`col`][datafusion.col.col] to disambiguate shared column names: >>> left = ctx.from_pydict({"id": [1, 2], "val": [10, 20]}) >>> right = ctx.from_pydict({"id": [1, 2], "val": [30, 40]}) @@ -1216,7 +1216,7 @@ def explain( verbose: If ``True``, more details will be included. analyze: If ``True``, the plan will run and metrics reported. format: Output format for the plan. Defaults to - :py:attr:`ExplainFormat.INDENT`. + [`INDENT`][ExplainFormat.INDENT]. Examples: Show the plan in tree format: @@ -1287,9 +1287,9 @@ def repartition_by_hash(self, *exprs: Expr | str, num: int) -> DataFrame: return DataFrame(self.df.repartition_by_hash(*exprs, num=num)) def union(self, other: DataFrame, distinct: bool = False) -> DataFrame: - """Calculate the union of two :py:class:`DataFrame`. + """Calculate the union of two [`DataFrame`][datafusion.dataframe.DataFrame]. - The two :py:class:`DataFrame` must have exactly the same schema. + The two `DataFrame` must have exactly the same schema. Args: other: DataFrame to union with. @@ -1318,17 +1318,17 @@ def union(self, other: DataFrame, distinct: bool = False) -> DataFrame: "union_distinct() is deprecated. Use union(other, distinct=True) instead." ) def union_distinct(self, other: DataFrame) -> DataFrame: - """Calculate the distinct union of two :py:class:`DataFrame`. + """Calculate the distinct union of two `DataFrame`. See Also: - :py:meth:`union` + [`union`][union] """ return self.union(other, distinct=True) def intersect(self, other: DataFrame, distinct: bool = False) -> DataFrame: - """Calculate the intersection of two :py:class:`DataFrame`. + """Calculate the intersection of two `DataFrame`. - The two :py:class:`DataFrame` must have exactly the same schema. + The two `DataFrame` must have exactly the same schema. Args: other: DataFrame to intersect with. @@ -1356,11 +1356,11 @@ def intersect(self, other: DataFrame, distinct: bool = False) -> DataFrame: return DataFrame(self.df.intersect(other.df, distinct)) def except_all(self, other: DataFrame, distinct: bool = False) -> DataFrame: - """Calculate the set difference of two :py:class:`DataFrame`. + """Calculate the set difference of two `DataFrame`. Returns rows that are in this DataFrame but not in ``other``. - The two :py:class:`DataFrame` must have exactly the same schema. + The two `DataFrame` must have exactly the same schema. Args: other: DataFrame to calculate exception with. @@ -1386,9 +1386,9 @@ def except_all(self, other: DataFrame, distinct: bool = False) -> DataFrame: return DataFrame(self.df.except_all(other.df, distinct)) def union_by_name(self, other: DataFrame, distinct: bool = False) -> DataFrame: - """Union two :py:class:`DataFrame` matching columns by name. + """Union two `DataFrame` matching columns by name. - Unlike :py:meth:`union` which matches columns by position, this method + Unlike [`union`][union] which matches columns by position, this method matches columns by their names, allowing DataFrames with different column orders to be combined. @@ -1460,7 +1460,7 @@ def sort_by(self, *exprs: Expr | str) -> DataFrame: This is a convenience method that sorts the DataFrame by the given expressions in ascending order with nulls last. For more control over - sort direction and null ordering, use :py:meth:`sort` instead. + sort direction and null ordering, use [`sort`][sort] instead. Args: exprs: Expressions or column names to sort by. @@ -1485,7 +1485,7 @@ def write_csv( with_header: bool = False, write_options: DataFrameWriteOptions | None = None, ) -> None: - """Execute the :py:class:`DataFrame` and write the results to a CSV file. + """Execute the `DataFrame` and write the results to a CSV file. Args: path: Path of the CSV file to write. @@ -1531,7 +1531,7 @@ def write_parquet( compression_level: int | None = None, write_options: DataFrameWriteOptions | None = None, ) -> None: - """Execute the :py:class:`DataFrame` and write the results to a Parquet file. + """Execute the `DataFrame` and write the results to a Parquet file. Available compression types are: @@ -1586,7 +1586,7 @@ def write_parquet_with_options( options: ParquetWriterOptions, write_options: DataFrameWriteOptions | None = None, ) -> None: - """Execute the :py:class:`DataFrame` and write the results to a Parquet file. + """Execute the `DataFrame` and write the results to a Parquet file. Allows advanced writer options to be set with `ParquetWriterOptions`. @@ -1645,7 +1645,7 @@ def write_json( path: str | pathlib.Path, write_options: DataFrameWriteOptions | None = None, ) -> None: - """Execute the :py:class:`DataFrame` and write the results to a JSON file. + """Execute the `DataFrame` and write the results to a JSON file. Args: path: Path of the JSON file to write. @@ -1659,7 +1659,7 @@ def write_json( def write_table( self, table_name: str, write_options: DataFrameWriteOptions | None = None ) -> None: - """Execute the :py:class:`DataFrame` and write the results to a table. + """Execute the `DataFrame` and write the results to a table. The table must be registered with the session to perform this operation. Not all table providers support writing operations. See the individual @@ -1671,7 +1671,7 @@ def write_table( self.df.write_table(table_name, raw_write_options) def to_arrow_table(self) -> pa.Table: - """Execute the :py:class:`DataFrame` and convert it into an Arrow Table. + """Execute the `DataFrame` and convert it into an Arrow Table. Returns: Arrow Table. @@ -1696,7 +1696,7 @@ def execute_stream_partitioned(self) -> list[RecordBatchStream]: return [RecordBatchStream(rbs) for rbs in streams] def to_pandas(self) -> pd.DataFrame: - """Execute the :py:class:`DataFrame` and convert it into a Pandas DataFrame. + """Execute the `DataFrame` and convert it into a Pandas DataFrame. Returns: Pandas DataFrame. @@ -1704,7 +1704,7 @@ def to_pandas(self) -> pd.DataFrame: return self.df.to_pandas() def to_pylist(self) -> list[dict[str, Any]]: - """Execute the :py:class:`DataFrame` and convert it into a list of dictionaries. + """Execute the `DataFrame` and convert it into a list of dictionaries. Returns: List of dictionaries. @@ -1712,7 +1712,7 @@ def to_pylist(self) -> list[dict[str, Any]]: return self.df.to_pylist() def to_pydict(self) -> dict[str, list[Any]]: - """Execute the :py:class:`DataFrame` and convert it into a dictionary of lists. + """Execute the `DataFrame` and convert it into a dictionary of lists. Returns: Dictionary of lists. @@ -1720,7 +1720,7 @@ def to_pydict(self) -> dict[str, list[Any]]: return self.df.to_pydict() def to_polars(self) -> pl.DataFrame: - """Execute the :py:class:`DataFrame` and convert it into a Polars DataFrame. + """Execute the `DataFrame` and convert it into a Polars DataFrame. Returns: Polars DataFrame. @@ -1728,7 +1728,7 @@ def to_polars(self) -> pl.DataFrame: return self.df.to_polars() def count(self) -> int: - """Return the total number of rows in this :py:class:`DataFrame`. + """Return the total number of rows in this `DataFrame`. Note that this method will actually run a plan to calculate the count, which may be slow for large or complicated DataFrames. @@ -1790,7 +1790,7 @@ def __arrow_c_stream__(self, requested_schema: object | None = None) -> object: supported through this interface. Args: - requested_schema: Either a :py:class:`pyarrow.Schema` or an Arrow C + requested_schema: Either a [`Schema`][pyarrow.Schema] or an Arrow C Schema capsule (``PyCapsule``) produced by ``schema._export_to_c_capsule()``. The DataFrame will attempt to align its output with the fields and order specified by this schema. diff --git a/python/datafusion/expr.py b/python/datafusion/expr.py index 4fdbdc5d4..1a3c87e96 100644 --- a/python/datafusion/expr.py +++ b/python/datafusion/expr.py @@ -15,21 +15,21 @@ # specific language governing permissions and limitations # under the License. -""":py:class:`Expr` — the logical expression type used to build DataFusion queries. +"""`Expr` — the logical expression type used to build DataFusion queries. -An :py:class:`Expr` represents a computation over columns or literals: a +An [`Expr`][datafusion.expr.Expr] represents a computation over columns or literals: a column reference (``col("a")``), a literal (``lit(5)``), an operator combination (``col("a") + lit(1)``), or the output of a function from -:py:mod:`datafusion.functions`. Expressions are passed to -:py:class:`~datafusion.dataframe.DataFrame` methods such as -:py:meth:`~datafusion.dataframe.DataFrame.select`, -:py:meth:`~datafusion.dataframe.DataFrame.filter`, -:py:meth:`~datafusion.dataframe.DataFrame.aggregate`, and -:py:meth:`~datafusion.dataframe.DataFrame.sort`. +[`functions`][datafusion.functions]. Expressions are passed to +[`DataFrame`][datafusion.dataframe.DataFrame] methods such as +[`select`][datafusion.dataframe.DataFrame.select], +[`filter`][datafusion.dataframe.DataFrame.filter], +[`aggregate`][datafusion.dataframe.DataFrame.aggregate], and +[`sort`][datafusion.dataframe.DataFrame.sort]. Convenience constructors are re-exported at the package level: -:py:func:`datafusion.col` / :py:func:`datafusion.column` for column references -and :py:func:`datafusion.lit` / :py:func:`datafusion.literal` for scalar +[`col`][datafusion.col] / [`column`][datafusion.column] for column references +and [`lit`][datafusion.lit] / [`literal`][datafusion.literal] for scalar literals. Examples: @@ -691,11 +691,11 @@ def __getitem__(self, key: str | int) -> Expr: If ``key`` is a string, returns the subfield of the struct. If ``key`` is an integer, retrieves the element in the array. Note that the element index begins at ``0``, unlike - :py:func:`~datafusion.functions.array_element` which begins at ``1``. + [`array_element`][datafusion.functions.array_element] which begins at ``1``. If ``key`` is a slice, returns an array that contains a slice of the original array. Similar to integer indexing, this follows Python convention where the index begins at ``0`` unlike - :py:func:`~datafusion.functions.array_slice` which begins at ``1``. + [`array_slice`][datafusion.functions.array_slice] which begins at ``1``. """ if isinstance(key, int): return Expr( @@ -848,7 +848,7 @@ def alias(self, name: str, metadata: dict[str, str] | None = None) -> Expr: return Expr(self.expr.alias(name, metadata)) def sort(self, ascending: bool = True, nulls_first: bool = True) -> SortExpr: - """Creates a sort :py:class:`Expr` from an existing :py:class:`Expr`. + """Creates a sort `Expr` from an existing `Expr`. Args: ascending: If true, sort in ascending order. @@ -959,7 +959,7 @@ def column_name(self, plan: LogicalPlan) -> str: def order_by(self, *exprs: Expr | SortExpr) -> ExprFuncBuilder: """Set the ordering for a window or aggregate function. - This function will create an :py:class:`ExprFuncBuilder` that can be used to + This function will create an `ExprFuncBuilder` that can be used to set parameters for either window or aggregate functions. If used on any other type of expression, an error will be generated when ``build()`` is called. """ @@ -968,7 +968,7 @@ def order_by(self, *exprs: Expr | SortExpr) -> ExprFuncBuilder: def filter(self, filter: Expr) -> ExprFuncBuilder: """Filter an aggregate function. - This function will create an :py:class:`ExprFuncBuilder` that can be used to + This function will create an `ExprFuncBuilder` that can be used to set parameters for either window or aggregate functions. If used on any other type of expression, an error will be generated when ``build()`` is called. """ @@ -977,7 +977,7 @@ def filter(self, filter: Expr) -> ExprFuncBuilder: def distinct(self) -> ExprFuncBuilder: """Only evaluate distinct values for an aggregate function. - This function will create an :py:class:`ExprFuncBuilder` that can be used to + This function will create an `ExprFuncBuilder` that can be used to set parameters for either window or aggregate functions. If used on any other type of expression, an error will be generated when ``build()`` is called. """ @@ -986,7 +986,7 @@ def distinct(self) -> ExprFuncBuilder: def null_treatment(self, null_treatment: NullTreatment) -> ExprFuncBuilder: """Set the treatment for ``null`` values for a window or aggregate function. - This function will create an :py:class:`ExprFuncBuilder` that can be used to + This function will create an `ExprFuncBuilder` that can be used to set parameters for either window or aggregate functions. If used on any other type of expression, an error will be generated when ``build()`` is called. """ @@ -995,7 +995,7 @@ def null_treatment(self, null_treatment: NullTreatment) -> ExprFuncBuilder: def partition_by(self, *partition_by: Expr) -> ExprFuncBuilder: """Set the partitioning for a window function. - This function will create an :py:class:`ExprFuncBuilder` that can be used to + This function will create an `ExprFuncBuilder` that can be used to set parameters for either window or aggregate functions. If used on any other type of expression, an error will be generated when ``build()`` is called. """ @@ -1004,7 +1004,7 @@ def partition_by(self, *partition_by: Expr) -> ExprFuncBuilder: def window_frame(self, window_frame: WindowFrame) -> ExprFuncBuilder: """Set the frame fora window function. - This function will create an :py:class:`ExprFuncBuilder` that can be used to + This function will create an `ExprFuncBuilder` that can be used to set parameters for either window or aggregate functions. If used on any other type of expression, an error will be generated when ``build()`` is called. """ @@ -1125,7 +1125,7 @@ def initcap(self) -> Expr: def list_distinct(self) -> Expr: """Returns distinct values from the array after removing duplicates. - This is an alias for :py:func:`array_distinct`. + This is an alias for [`array_distinct`][array_distinct]. """ from . import functions as F @@ -1308,7 +1308,7 @@ def atanh(self) -> Expr: def list_dims(self) -> Expr: """Returns an array of the array's dimensions. - This is an alias for :py:func:`array_dims`. + This is an alias for [`array_dims`][array_dims]. """ from . import functions as F @@ -1347,7 +1347,7 @@ def ceil(self) -> Expr: def list_length(self) -> Expr: """Returns the length of the array. - This is an alias for :py:func:`array_length`. + This is an alias for [`array_length`][array_length]. """ from . import functions as F @@ -1404,7 +1404,7 @@ def char_length(self) -> Expr: def list_ndims(self) -> Expr: """Returns the number of dimensions of the array. - This is an alias for :py:func:`array_ndims`. + This is an alias for [`array_ndims`][array_ndims]. """ from . import functions as F @@ -1429,7 +1429,7 @@ def sinh(self) -> Expr: return F.sinh(self) def empty(self) -> Expr: - """This is an alias for :py:func:`array_empty`.""" + """This is an alias for [`array_empty`][array_empty].""" from . import functions as F return F.empty(self) @@ -1571,7 +1571,7 @@ def get_upper_bound(self) -> WindowFrameBound: class WindowFrameBound: """Defines a single window frame bound. - :py:class:`WindowFrame` typically requires a start and end bound. + `WindowFrame` typically requires a start and end bound. """ def __init__(self, frame_bound: expr_internal.WindowFrameBound) -> None: @@ -1620,7 +1620,7 @@ def __init__(self, case_builder: expr_internal.CaseBuilder) -> None: """Constructs a case builder. This is not typically called by the end user directly. See - :py:func:`datafusion.functions.case` instead. + [`case`][datafusion.functions.case] instead. """ self.case_builder = case_builder @@ -1671,12 +1671,12 @@ class GroupingSet: """Factory for creating grouping set expressions. Grouping sets control how - :py:meth:`~datafusion.dataframe.DataFrame.aggregate` groups rows. + [`aggregate`][datafusion.dataframe.DataFrame.aggregate] groups rows. Instead of a single ``GROUP BY``, they produce multiple grouping levels in one pass — subtotals, cross-tabulations, or arbitrary column subsets. - Use :py:func:`~datafusion.functions.grouping` in the aggregate list + Use [`grouping`][datafusion.functions.grouping] in the aggregate list to tell which columns are aggregated across in each result row. """ @@ -1707,8 +1707,8 @@ def rollup(*exprs: Expr | str) -> Expr: [30, 30, 60] See Also: - :py:meth:`cube`, :py:meth:`grouping_sets`, - :py:func:`~datafusion.functions.grouping` + [`cube`][cube], [`grouping_sets`][grouping_sets], + [`grouping`][datafusion.functions.grouping] """ args = [_to_raw_expr(e) for e in exprs] return Expr(expr_internal.GroupingSet.rollup(*args)) @@ -1729,7 +1729,7 @@ def cube(*exprs: Expr | str) -> Expr: Examples: With a single column, ``cube`` behaves identically to - :py:meth:`rollup`: + [`rollup`][rollup]: >>> from datafusion.expr import GroupingSet >>> ctx = dfn.SessionContext() @@ -1743,8 +1743,8 @@ def cube(*exprs: Expr | str) -> Expr: [30, 30, 60] See Also: - :py:meth:`rollup`, :py:meth:`grouping_sets`, - :py:func:`~datafusion.functions.grouping` + [`rollup`][rollup], [`grouping_sets`][grouping_sets], + [`grouping`][datafusion.functions.grouping] """ args = [_to_raw_expr(e) for e in exprs] return Expr(expr_internal.GroupingSet.cube(*args)) @@ -1786,8 +1786,8 @@ def grouping_sets(*expr_lists: list[Expr | str]) -> Expr: [3, 3, 4, 2] See Also: - :py:meth:`rollup`, :py:meth:`cube`, - :py:func:`~datafusion.functions.grouping` + [`rollup`][rollup], [`cube`][cube], + [`grouping`][datafusion.functions.grouping] """ raw_lists = [[_to_raw_expr(e) for e in lst] for lst in expr_lists] return Expr(expr_internal.GroupingSet.grouping_sets(*raw_lists)) diff --git a/python/datafusion/functions.py b/python/datafusion/functions.py index c8f07497d..c1068cacb 100644 --- a/python/datafusion/functions.py +++ b/python/datafusion/functions.py @@ -14,15 +14,15 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. -"""Scalar, aggregate, and window functions for :py:class:`~datafusion.expr.Expr`. +"""Scalar, aggregate, and window functions for [`Expr`][datafusion.expr.Expr]. -Each function returns an :py:class:`~datafusion.expr.Expr` that can be combined +Each function returns an [`Expr`][datafusion.expr.Expr] that can be combined with other expressions and passed to -:py:class:`~datafusion.dataframe.DataFrame` methods such as -:py:meth:`~datafusion.dataframe.DataFrame.select`, -:py:meth:`~datafusion.dataframe.DataFrame.filter`, -:py:meth:`~datafusion.dataframe.DataFrame.aggregate`, and -:py:meth:`~datafusion.dataframe.DataFrame.window`. The module is conventionally +[`DataFrame`][datafusion.dataframe.DataFrame] methods such as +[`select`][datafusion.dataframe.DataFrame.select], +[`filter`][datafusion.dataframe.DataFrame.filter], +[`aggregate`][datafusion.dataframe.DataFrame.aggregate], and +[`window`][datafusion.dataframe.DataFrame.window]. The module is conventionally imported as ``F`` so calls read like ``F.sum(col("price"))``. Examples: @@ -449,7 +449,7 @@ def array_join(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for :py:func:`array_to_string`. + This is an alias for [`array_to_string`][array_to_string]. """ return array_to_string(expr, delimiter) @@ -458,7 +458,7 @@ def list_to_string(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for :py:func:`array_to_string`. + This is an alias for [`array_to_string`][array_to_string]. """ return array_to_string(expr, delimiter) @@ -467,7 +467,7 @@ def list_join(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for :py:func:`array_to_string`. + This is an alias for [`array_to_string`][array_to_string]. """ return array_to_string(expr, delimiter) @@ -475,9 +475,9 @@ def list_join(expr: Expr, delimiter: Expr | str) -> Expr: def lambda_var(name: str) -> Expr: """Create an unresolved reference to a lambda parameter by ``name``. - Use this inside the body passed to :py:func:`lambda_` to refer to one of the + Use this inside the body passed to [`lambda_`][lambda_] to refer to one of the lambda's parameters. The owning higher-order function (such as - :py:func:`array_transform`) binds the variable to a concrete element type + [`array_transform`][array_transform]) binds the variable to a concrete element type during query planning. Examples: @@ -490,7 +490,7 @@ def lambda_var(name: str) -> Expr: [2, 4, 6] See Also: - :py:func:`lambda_`, :py:func:`array_transform`, :py:func:`array_any_match`. + `lambda_`, `array_transform`, [`array_any_match`][array_any_match]. """ return Expr(f.lambda_var(name)) @@ -500,13 +500,13 @@ def lambda_(params: list[str], body: Expr) -> Expr: This is the explicit form of building a lambda. Most callers can instead pass a Python callable directly to a higher-order function such as - :py:func:`array_transform`, which builds the lambda automatically. Reach for + `array_transform`, which builds the lambda automatically. Reach for ``lambda_`` when you want explicit control over the parameter names. Args: params: Ordered lambda parameter names. body: Body expression that references the parameters via - :py:func:`lambda_var`. + [`lambda_var`][lambda_var]. Examples: >>> ctx = dfn.SessionContext() @@ -518,7 +518,7 @@ def lambda_(params: list[str], body: Expr) -> Expr: [2, 4, 6] See Also: - :py:func:`lambda_var`, :py:func:`array_transform`, :py:func:`array_any_match`. + `lambda_var`, `array_transform`, [`array_any_match`][array_any_match]. """ return Expr(f.lambda_(params, body.expr)) @@ -526,9 +526,9 @@ def lambda_(params: list[str], body: Expr) -> Expr: def _to_lambda(fn: Expr | Callable[..., Any]) -> Expr: """Coerce ``fn`` to a lambda ``Expr``. - Accepts either an ``Expr`` produced by :py:func:`lambda_` (returned + Accepts either an ``Expr`` produced by [`lambda_`][lambda_] (returned unchanged) or a Python callable. A callable is introspected for its - parameter names; those names become :py:func:`lambda_var` references passed + parameter names; those names become [`lambda_var`][lambda_var] references passed positionally into the callable, and its return value (coerced to an ``Expr``) becomes the lambda body. """ @@ -550,7 +550,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: ``transform`` may be a Python callable, which is converted to a lambda automatically (its parameter names become the lambda parameters), or an - explicit lambda built with :py:func:`lambda_`. + explicit lambda built with [`lambda_`][lambda_]. Examples: Using a Python callable: @@ -562,7 +562,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("d")[0].as_py() [2, 4, 6] - Using an explicit lambda built with :py:func:`lambda_`: + Using an explicit lambda built with [`lambda_`][lambda_]: >>> double_fn = F.lambda_(["v"], F.lambda_var("v") * lit(2)) >>> df.select( @@ -571,7 +571,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: [2, 4, 6] See Also: - :py:func:`array_any_match`, :py:func:`lambda_`. + [`array_any_match`][array_any_match], [`lambda_`][lambda_]. """ return Expr(f.array_transform(array.expr, _to_lambda(transform).expr)) @@ -580,7 +580,7 @@ def list_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: """Transform each element of a list with a lambda. See Also: - This is an alias for :py:func:`array_transform`. + This is an alias for [`array_transform`][array_transform]. """ return array_transform(array, transform) @@ -589,7 +589,7 @@ def array_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Return ``True`` if any element of ``array`` satisfies ``predicate``. ``predicate`` may be a Python callable, converted to a lambda - automatically, or an explicit lambda built with :py:func:`lambda_`. It must + automatically, or an explicit lambda built with [`lambda_`][lambda_]. It must return a boolean expression. Examples: @@ -602,7 +602,7 @@ def array_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("m")[0].as_py() True - Using an explicit lambda built with :py:func:`lambda_`: + Using an explicit lambda built with [`lambda_`][lambda_]: >>> predicate = F.lambda_(["v"], F.lambda_var("v") > lit(2)) >>> df.select( @@ -611,7 +611,7 @@ def array_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: True See Also: - :py:func:`array_transform`, :py:func:`lambda_`. + [`array_transform`][array_transform], [`lambda_`][lambda_]. """ return Expr(f.array_any_match(array.expr, _to_lambda(predicate).expr)) @@ -620,7 +620,7 @@ def any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Return ``True`` if any element of an array satisfies a predicate. See Also: - This is an alias for :py:func:`array_any_match`. + This is an alias for [`array_any_match`][array_any_match]. """ return array_any_match(array, predicate) @@ -629,7 +629,7 @@ def list_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Return ``True`` if any element of a list satisfies a predicate. See Also: - This is an alias for :py:func:`array_any_match`. + This is an alias for [`array_any_match`][array_any_match]. """ return array_any_match(array, predicate) @@ -638,7 +638,7 @@ def array_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Keep the elements of ``array`` for which ``predicate`` is ``True``. ``predicate`` may be a Python callable, converted to a lambda - automatically, or an explicit lambda built with :py:func:`lambda_`. It must + automatically, or an explicit lambda built with [`lambda_`][lambda_]. It must return a boolean expression. The result is a new array containing only the matching elements. @@ -652,7 +652,7 @@ def array_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("f")[0].as_py() [3, 4, 5] - Using an explicit lambda built with :py:func:`lambda_`: + Using an explicit lambda built with [`lambda_`][lambda_]: >>> predicate = F.lambda_(["v"], F.lambda_var("v") > lit(2)) >>> df.select( @@ -661,7 +661,7 @@ def array_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: [3, 4, 5] See Also: - :py:func:`array_transform`, :py:func:`array_any_match`, :py:func:`lambda_`. + `array_transform`, [`array_any_match`][array_any_match], [`lambda_`][lambda_]. """ return Expr(f.array_filter(array.expr, _to_lambda(predicate).expr)) @@ -670,7 +670,7 @@ def list_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Keep the elements of a list for which a predicate is ``True``. See Also: - This is an alias for :py:func:`array_filter`. + This is an alias for [`array_filter`][array_filter]. """ return array_filter(array, predicate) @@ -864,8 +864,8 @@ def count_star(filter: Expr | None = None) -> Expr: def case(expr: Expr) -> CaseBuilder: """Create a case expression. - Create a :py:class:`~datafusion.expr.CaseBuilder` to match cases for the - expression ``expr``. See :py:class:`~datafusion.expr.CaseBuilder` for + Create a [`CaseBuilder`][datafusion.expr.CaseBuilder] to match cases for the + expression ``expr``. See [`CaseBuilder`][datafusion.expr.CaseBuilder] for detailed usage. Examples: @@ -883,8 +883,8 @@ def case(expr: Expr) -> CaseBuilder: def when(when: Expr, then: Expr) -> CaseBuilder: """Create a case expression that has no base expression. - Create a :py:class:`~datafusion.expr.CaseBuilder` to match cases for the - expression ``expr``. See :py:class:`~datafusion.expr.CaseBuilder` for + Create a [`CaseBuilder`][datafusion.expr.CaseBuilder] to match cases for the + expression ``expr``. See [`CaseBuilder`][datafusion.expr.CaseBuilder] for detailed usage. Examples: @@ -1315,7 +1315,7 @@ def ifnull(x: Expr, y: Expr) -> Expr: y: Fallback expression to return when ``x`` is NULL. See Also: - This is an alias for :py:func:`nvl`. + This is an alias for [`nvl`][nvl]. """ return nvl(x, y) @@ -1340,7 +1340,7 @@ def instr(string: Expr, substring: Expr | str) -> Expr: """Finds the position from where the ``substring`` matches the ``string``. See Also: - This is an alias for :py:func:`strpos`. + This is an alias for [`strpos`][strpos]. """ return strpos(string, substring) @@ -1669,7 +1669,7 @@ def position(string: Expr, substring: Expr | str) -> Expr: """Finds the position from where the ``substring`` matches the ``string``. See Also: - This is an alias for :py:func:`strpos`. + This is an alias for [`strpos`][strpos]. """ return strpos(string, substring) @@ -1694,7 +1694,7 @@ def pow(base: Expr, exponent: Expr | int | float) -> Expr: # noqa: PYI041 """Returns ``base`` raised to the power of ``exponent``. See Also: - This is an alias of :py:func:`power`. + This is an alias of [`power`][power]. """ return power(base, exponent) @@ -2329,7 +2329,7 @@ def current_timestamp() -> Expr: """Returns the current timestamp in nanoseconds. See Also: - This is an alias for :py:func:`now`. + This is an alias for [`now`][now]. """ return now() @@ -2361,7 +2361,7 @@ def date_format(arg: Expr, formatter: Expr | str) -> Expr: """Returns a string representation of a date, time, timestamp or duration. See Also: - This is an alias for :py:func:`to_char`. + This is an alias for [`to_char`][to_char]. """ return to_char(arg, formatter) @@ -2446,7 +2446,7 @@ def to_timestamp(arg: Expr, *formatters: Expr) -> Expr: def to_timestamp_millis(arg: Expr, *formatters: Expr) -> Expr: """Converts a string and optional formats to a ``Timestamp`` in milliseconds. - See :py:func:`to_timestamp` for a description on how to use formatters. + See [`to_timestamp`][to_timestamp] for a description on how to use formatters. Examples: >>> ctx = dfn.SessionContext() @@ -2465,7 +2465,7 @@ def to_timestamp_millis(arg: Expr, *formatters: Expr) -> Expr: def to_timestamp_micros(arg: Expr, *formatters: Expr) -> Expr: """Converts a string and optional formats to a ``Timestamp`` in microseconds. - See :py:func:`to_timestamp` for a description on how to use formatters. + See [`to_timestamp`][to_timestamp] for a description on how to use formatters. Examples: >>> ctx = dfn.SessionContext() @@ -2484,7 +2484,7 @@ def to_timestamp_micros(arg: Expr, *formatters: Expr) -> Expr: def to_timestamp_nanos(arg: Expr, *formatters: Expr) -> Expr: """Converts a string and optional formats to a ``Timestamp`` in nanoseconds. - See :py:func:`to_timestamp` for a description on how to use formatters. + See [`to_timestamp`][to_timestamp] for a description on how to use formatters. Examples: >>> ctx = dfn.SessionContext() @@ -2503,7 +2503,7 @@ def to_timestamp_nanos(arg: Expr, *formatters: Expr) -> Expr: def to_timestamp_seconds(arg: Expr, *formatters: Expr) -> Expr: """Converts a string and optional formats to a ``Timestamp`` in seconds. - See :py:func:`to_timestamp` for a description on how to use formatters. + See [`to_timestamp`][to_timestamp] for a description on how to use formatters. Examples: >>> ctx = dfn.SessionContext() @@ -2573,7 +2573,7 @@ def datepart(part: Expr | str, date: Expr) -> Expr: """Return a specified part of a date. See Also: - This is an alias for :py:func:`date_part`. + This is an alias for [`date_part`][date_part]. """ return date_part(part, date) @@ -2603,7 +2603,7 @@ def extract(part: Expr | str, date: Expr) -> Expr: """Extracts a subfield from the date. See Also: - This is an alias for :py:func:`date_part`. + This is an alias for [`date_part`][date_part]. """ return date_part(part, date) @@ -2634,7 +2634,7 @@ def datetrunc(part: Expr | str, date: Expr) -> Expr: """Truncates the date to a specified level of precision. See Also: - This is an alias for :py:func:`date_trunc`. + This is an alias for [`date_trunc`][date_trunc]. """ return date_trunc(part, date) @@ -2776,7 +2776,7 @@ def make_list(*args: Expr) -> Expr: """Returns an array using the specified input expressions. See Also: - This is an alias for :py:func:`make_array`. + This is an alias for [`make_array`][make_array]. """ return make_array(*args) @@ -2785,7 +2785,7 @@ def array(*args: Expr) -> Expr: """Returns an array using the specified input expressions. See Also: - This is an alias for :py:func:`make_array`. + This is an alias for [`make_array`][make_array]. """ return make_array(*args) @@ -2899,8 +2899,7 @@ def arrow_cast(expr: Expr, data_type: Expr | str | pa.DataType) -> Expr: """Casts an expression to a specified data type. The ``data_type`` can be a string, a ``pyarrow.DataType``, or an - ``Expr``. For simple types, :py:meth:`Expr.cast() - ` is more concise + ``Expr``. For simple types, `Expr.cast()` is more concise (e.g., ``col("a").cast(pa.float64())``). Use ``arrow_cast`` when you want to specify the target type as a string using DataFusion's type syntax, which can be more readable for complex types like @@ -2970,13 +2969,13 @@ def get_field(expr: Expr, *names: Expr | str) -> Expr: of nested struct/map fields in a single ``get_field`` call. For a single static-string name, ``expr["field"]`` is a convenient shorthand; use ``get_field`` when the field name is a dynamic - :py:class:`~datafusion.expr.Expr` or when traversing multiple levels at + [`Expr`][datafusion.expr.Expr] or when traversing multiple levels at once. Args: expr: The struct or map expression to read from. *names: One or more field names (``str``) or expressions - (:py:class:`~datafusion.expr.Expr`). + ([`Expr`][datafusion.expr.Expr]). Examples: Single-level lookup: @@ -3086,7 +3085,7 @@ def row(*args: Expr) -> Expr: """Returns a struct with the given arguments. See Also: - This is an alias for :py:func:`struct`. + This is an alias for [`struct`][struct]. """ return struct(*args) @@ -3125,7 +3124,7 @@ def array_push_back(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for :py:func:`array_append`. + This is an alias for [`array_append`][array_append]. """ return array_append(array, element) @@ -3134,7 +3133,7 @@ def list_append(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for :py:func:`array_append`. + This is an alias for [`array_append`][array_append]. """ return array_append(array, element) @@ -3143,7 +3142,7 @@ def list_push_back(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for :py:func:`array_append`. + This is an alias for [`array_append`][array_append]. """ return array_append(array, element) @@ -3167,7 +3166,7 @@ def array_cat(*args: Expr) -> Expr: """Concatenates the input arrays. See Also: - This is an alias for :py:func:`array_concat`. + This is an alias for [`array_concat`][array_concat]. """ return array_concat(*args) @@ -3208,7 +3207,7 @@ def list_cat(*args: Expr) -> Expr: """Concatenates the input arrays. See Also: - This is an alias for :py:func:`array_concat`, :py:func:`array_cat`. + This is an alias for [`array_concat`][array_concat], [`array_cat`][array_cat]. """ return array_concat(*args) @@ -3217,7 +3216,7 @@ def list_concat(*args: Expr) -> Expr: """Concatenates the input arrays. See Also: - This is an alias for :py:func:`array_concat`, :py:func:`array_cat`. + This is an alias for [`array_concat`][array_concat], [`array_cat`][array_cat]. """ return array_concat(*args) @@ -3226,7 +3225,7 @@ def list_distinct(array: Expr) -> Expr: """Returns distinct values from the array after removing duplicates. See Also: - This is an alias for :py:func:`array_distinct`. + This is an alias for [`array_distinct`][array_distinct]. """ return array_distinct(array) @@ -3235,7 +3234,7 @@ def list_dims(array: Expr) -> Expr: """Returns an array of the array's dimensions. See Also: - This is an alias for :py:func:`array_dims`. + This is an alias for [`array_dims`][array_dims]. """ return array_dims(array) @@ -3272,7 +3271,7 @@ def list_empty(array: Expr) -> Expr: """Returns a boolean indicating whether the array is empty. See Also: - This is an alias for :py:func:`array_empty`. + This is an alias for [`array_empty`][array_empty]. """ return array_empty(array) @@ -3281,7 +3280,7 @@ def array_extract(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for :py:func:`array_element`. + This is an alias for [`array_element`][array_element]. """ return array_element(array, n) @@ -3290,7 +3289,7 @@ def list_element(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for :py:func:`array_element`. + This is an alias for [`array_element`][array_element]. """ return array_element(array, n) @@ -3299,7 +3298,7 @@ def list_extract(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for :py:func:`array_element`. + This is an alias for [`array_element`][array_element]. """ return array_element(array, n) @@ -3321,7 +3320,7 @@ def list_length(array: Expr) -> Expr: """Returns the length of the array. See Also: - This is an alias for :py:func:`array_length`. + This is an alias for [`array_length`][array_length]. """ return array_length(array) @@ -3378,7 +3377,7 @@ def array_contains(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for :py:func:`array_has`. + This is an alias for [`array_has`][array_has]. """ return array_has(array, element) @@ -3387,7 +3386,7 @@ def list_has(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for :py:func:`array_has`. + This is an alias for [`array_has`][array_has]. """ return array_has(array, element) @@ -3396,7 +3395,7 @@ def list_has_all(first_array: Expr, second_array: Expr) -> Expr: """Determines if there is complete overlap ``second_array`` in ``first_array``. See Also: - This is an alias for :py:func:`array_has_all`. + This is an alias for [`array_has_all`][array_has_all]. """ return array_has_all(first_array, second_array) @@ -3405,7 +3404,7 @@ def list_has_any(first_array: Expr, second_array: Expr) -> Expr: """Determine if there is an overlap between ``first_array`` and ``second_array``. See Also: - This is an alias for :py:func:`array_has_any`. + This is an alias for [`array_has_any`][array_has_any]. """ return array_has_any(first_array, second_array) @@ -3414,7 +3413,7 @@ def arrays_overlap(first_array: Expr, second_array: Expr) -> Expr: """Returns true if any element appears in both arrays. See Also: - This is an alias for :py:func:`array_has_any`. + This is an alias for [`array_has_any`][array_has_any]. """ return array_has_any(first_array, second_array) @@ -3423,7 +3422,7 @@ def list_overlap(first_array: Expr, second_array: Expr) -> Expr: """Returns true if any element appears in both arrays. See Also: - This is an alias for :py:func:`array_has_any`. + This is an alias for [`array_has_any`][array_has_any]. """ return array_has_any(first_array, second_array) @@ -3432,7 +3431,7 @@ def list_contains(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for :py:func:`array_has`. + This is an alias for [`array_has`][array_has]. """ return array_has(array, element) @@ -3467,7 +3466,7 @@ def array_indexof(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for :py:func:`array_position`. + This is an alias for [`array_position`][array_position]. """ return array_position(array, element, index) @@ -3476,7 +3475,7 @@ def list_position(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for :py:func:`array_position`. + This is an alias for [`array_position`][array_position]. """ return array_position(array, element, index) @@ -3485,7 +3484,7 @@ def list_indexof(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for :py:func:`array_position`. + This is an alias for [`array_position`][array_position]. """ return array_position(array, element, index) @@ -3508,7 +3507,7 @@ def list_positions(array: Expr, element: Expr) -> Expr: """Searches for an element in the array and returns all occurrences. See Also: - This is an alias for :py:func:`array_positions`. + This is an alias for [`array_positions`][array_positions]. """ return array_positions(array, element) @@ -3530,7 +3529,7 @@ def list_ndims(array: Expr) -> Expr: """Returns the number of dimensions of the array. See Also: - This is an alias for :py:func:`array_ndims`. + This is an alias for [`array_ndims`][array_ndims]. """ return array_ndims(array) @@ -3553,7 +3552,7 @@ def array_push_front(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for :py:func:`array_prepend`. + This is an alias for [`array_prepend`][array_prepend]. """ return array_prepend(element, array) @@ -3562,7 +3561,7 @@ def list_prepend(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for :py:func:`array_prepend`. + This is an alias for [`array_prepend`][array_prepend]. """ return array_prepend(element, array) @@ -3571,7 +3570,7 @@ def list_push_front(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for :py:func:`array_prepend`. + This is an alias for [`array_prepend`][array_prepend]. """ return array_prepend(element, array) @@ -3608,7 +3607,7 @@ def list_pop_back(array: Expr) -> Expr: """Returns the array without the last element. See Also: - This is an alias for :py:func:`array_pop_back`. + This is an alias for [`array_pop_back`][array_pop_back]. """ return array_pop_back(array) @@ -3617,7 +3616,7 @@ def list_pop_front(array: Expr) -> Expr: """Returns the array without the first element. See Also: - This is an alias for :py:func:`array_pop_front`. + This is an alias for [`array_pop_front`][array_pop_front]. """ return array_pop_front(array) @@ -3640,7 +3639,7 @@ def list_remove(array: Expr, element: Expr) -> Expr: """Removes the first element from the array equal to the given value. See Also: - This is an alias for :py:func:`array_remove`. + This is an alias for [`array_remove`][array_remove]. """ return array_remove(array, element) @@ -3666,7 +3665,7 @@ def list_remove_n(array: Expr, element: Expr, max: Expr | int) -> Expr: """Removes the first ``max`` elements from the array equal to the given value. See Also: - This is an alias for :py:func:`array_remove_n`. + This is an alias for [`array_remove_n`][array_remove_n]. """ return array_remove_n(array, element, max) @@ -3691,7 +3690,7 @@ def list_remove_all(array: Expr, element: Expr) -> Expr: """Removes all elements from the array equal to the given value. See Also: - This is an alias for :py:func:`array_remove_all`. + This is an alias for [`array_remove_all`][array_remove_all]. """ return array_remove_all(array, element) @@ -3715,7 +3714,7 @@ def list_repeat(element: Expr, count: Expr | int) -> Expr: """Returns an array containing ``element`` ``count`` times. See Also: - This is an alias for :py:func:`array_repeat`. + This is an alias for [`array_repeat`][array_repeat]. """ return array_repeat(element, count) @@ -3739,7 +3738,7 @@ def list_replace(array: Expr, from_val: Expr, to_val: Expr) -> Expr: """Replaces the first occurrence of ``from_val`` with ``to_val``. See Also: - This is an alias for :py:func:`array_replace`. + This is an alias for [`array_replace`][array_replace]. """ return array_replace(array, from_val, to_val) @@ -3771,7 +3770,7 @@ def list_replace_n(array: Expr, from_val: Expr, to_val: Expr, max: Expr | int) - specified element. See Also: - This is an alias for :py:func:`array_replace_n`. + This is an alias for [`array_replace_n`][array_replace_n]. """ return array_replace_n(array, from_val, to_val, max) @@ -3795,7 +3794,7 @@ def list_replace_all(array: Expr, from_val: Expr, to_val: Expr) -> Expr: """Replaces all occurrences of ``from_val`` with ``to_val``. See Also: - This is an alias for :py:func:`array_replace_all`. + This is an alias for [`array_replace_all`][array_replace_all]. """ return array_replace_all(array, from_val, to_val) @@ -3841,7 +3840,7 @@ def list_sort(array: Expr, descending: bool = False, null_first: bool = False) - """Sorts the array. See Also: - This is an alias for :py:func:`array_sort`. + This is an alias for [`array_sort`][array_sort]. """ return array_sort(array, descending=descending, null_first=null_first) @@ -3890,7 +3889,7 @@ def list_slice( """Returns a slice of the array. See Also: - This is an alias for :py:func:`array_slice`. + This is an alias for [`array_slice`][array_slice]. """ return array_slice(array, begin, end, stride) @@ -3918,7 +3917,7 @@ def list_intersect(array1: Expr, array2: Expr) -> Expr: """Returns an the intersection of ``array1`` and ``array2``. See Also: - This is an alias for :py:func:`array_intersect`. + This is an alias for [`array_intersect`][array_intersect]. """ return array_intersect(array1, array2) @@ -3950,7 +3949,7 @@ def list_union(array1: Expr, array2: Expr) -> Expr: Duplicate rows will not be returned. See Also: - This is an alias for :py:func:`array_union`. + This is an alias for [`array_union`][array_union]. """ return array_union(array1, array2) @@ -3973,7 +3972,7 @@ def list_except(array1: Expr, array2: Expr) -> Expr: """Returns the elements that appear in ``array1`` but not in the ``array2``. See Also: - This is an alias for :py:func:`array_except`. + This is an alias for [`array_except`][array_except]. """ return array_except(array1, array2) @@ -4003,7 +4002,7 @@ def list_resize(array: Expr, size: Expr | int, value: Expr) -> Expr: filled with the given ``value``. See Also: - This is an alias for :py:func:`array_resize`. + This is an alias for [`array_resize`][array_resize]. """ return array_resize(array, size, value) @@ -4026,7 +4025,7 @@ def list_any_value(array: Expr) -> Expr: """Returns the first non-null element in the array. See Also: - This is an alias for :py:func:`array_any_value`. + This is an alias for [`array_any_value`][array_any_value]. """ return array_any_value(array) @@ -4051,7 +4050,7 @@ def list_distance(array1: Expr, array2: Expr) -> Expr: """Returns the Euclidean distance between two numeric arrays. See Also: - This is an alias for :py:func:`array_distance`. + This is an alias for [`array_distance`][array_distance]. """ return array_distance(array1, array2) @@ -4074,7 +4073,7 @@ def list_max(array: Expr) -> Expr: """Returns the maximum value in the array. See Also: - This is an alias for :py:func:`array_max`. + This is an alias for [`array_max`][array_max]. """ return array_max(array) @@ -4097,7 +4096,7 @@ def list_min(array: Expr) -> Expr: """Returns the minimum value in the array. See Also: - This is an alias for :py:func:`array_min`. + This is an alias for [`array_min`][array_min]. """ return array_min(array) @@ -4120,7 +4119,7 @@ def list_reverse(array: Expr) -> Expr: """Reverses the order of elements in the array. See Also: - This is an alias for :py:func:`array_reverse`. + This is an alias for [`array_reverse`][array_reverse]. """ return array_reverse(array) @@ -4144,7 +4143,7 @@ def list_zip(*arrays: Expr) -> Expr: """Combines multiple arrays into a single array of structs. See Also: - This is an alias for :py:func:`arrays_zip`. + This is an alias for [`arrays_zip`][arrays_zip]. """ return arrays_zip(*arrays) @@ -4190,7 +4189,7 @@ def string_to_list( """Splits a string based on a delimiter and returns an array of parts. See Also: - This is an alias for :py:func:`string_to_array`. + This is an alias for [`string_to_array`][string_to_array]. """ return string_to_array(string, delimiter, null_string) @@ -4198,7 +4197,7 @@ def string_to_list( def gen_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: """Creates a list of values in the range between start and stop. - Unlike :py:func:`range`, this includes the upper bound. + Unlike [`range`][range], this includes the upper bound. Examples: >>> ctx = dfn.SessionContext() @@ -4226,10 +4225,10 @@ def gen_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: def generate_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: """Creates a list of values in the range between start and stop. - Unlike :py:func:`range`, this includes the upper bound. + Unlike [`range`][range], this includes the upper bound. See Also: - This is an alias for :py:func:`gen_series`. + This is an alias for [`gen_series`][gen_series]. """ return gen_series(start, stop, step) @@ -4264,7 +4263,7 @@ def empty(array: Expr) -> Expr: """Returns true if the array is empty. See Also: - This is an alias for :py:func:`array_empty`. + This is an alias for [`array_empty`][array_empty]. """ return array_empty(array) @@ -4283,7 +4282,7 @@ def make_map(*args: Any) -> Expr: - ``make_map(k1, v1, k2, v2, ...)`` — from alternating keys and their associated values. - Keys and values that are not already :py:class:`~datafusion.expr.Expr` + Keys and values that are not already [`Expr`][datafusion.expr.Expr] are automatically converted to literal expressions. Examples: @@ -4416,7 +4415,7 @@ def element_at(map: Expr, key: Expr) -> Expr: Returns ``[None]`` if the key is absent. See Also: - This is an alias for :py:func:`map_extract`. + This is an alias for [`map_extract`][map_extract]. """ return map_extract(map, key) @@ -4428,9 +4427,9 @@ def approx_distinct( ) -> Expr: """Returns the approximate number of distinct values. - This aggregate function is similar to :py:func:`count` with distinct set, but it + This aggregate function is similar to [`count`][count] with distinct set, but it will approximate the number of distinct entries. It may return significantly faster - than :py:func:`count` for some DataFrames. + than [`count`][count] for some DataFrames. If using the builder functions described in ref:`_aggregation` this function ignores the options ``order_by``, ``null_treatment``, and ``distinct``. @@ -4465,7 +4464,7 @@ def approx_distinct( def approx_median(expression: Expr, filter: Expr | None = None) -> Expr: """Returns the approximate median value. - This aggregate function is similar to :py:func:`median`, but it will only + This aggregate function is similar to [`median`][median], but it will only approximate the median. It may return significantly faster for some DataFrames. If using the builder functions described in ref:`_aggregation` this function ignores @@ -4561,7 +4560,7 @@ def approx_percentile_cont_with_weight( ) -> Expr: """Returns the value of the weighted approximate percentile. - This aggregate function is similar to :py:func:`approx_percentile_cont` except that + This aggregate function is similar to `approx_percentile_cont` except that it uses the associated associated weights. If using the builder functions described in ref:`_aggregation` this function ignores @@ -4613,7 +4612,7 @@ def percentile_cont( ) -> Expr: """Computes the exact percentile of input values using continuous interpolation. - Unlike :py:func:`approx_percentile_cont`, this function computes the exact + Unlike `approx_percentile_cont`, this function computes the exact percentile value rather than an approximation. If using the builder functions described in ref:`_aggregation` this function ignores @@ -4655,7 +4654,7 @@ def quantile_cont( """Computes the exact percentile of input values using continuous interpolation. See Also: - This is an alias for :py:func:`percentile_cont`. + This is an alias for [`percentile_cont`][percentile_cont]. """ return percentile_cont(sort_expression, percentile, filter) @@ -4669,7 +4668,7 @@ def array_agg( """Aggregate values into an array. Currently ``distinct`` and ``order_by`` cannot be used together. As a work around, - consider :py:func:`array_sort` after aggregation. + consider [`array_sort`][array_sort] after aggregation. [Issue Tracker](https://github.com/apache/datafusion/issues/12371) If using the builder functions described in ref:`_aggregation` this function ignores @@ -4731,9 +4730,9 @@ def grouping( aggregate spans all values of that column). This function is meaningful with - :py:meth:`GroupingSet.rollup `, - :py:meth:`GroupingSet.cube `, or - :py:meth:`GroupingSet.grouping_sets `, + [`GroupingSet.rollup`][datafusion.expr.GroupingSet.rollup], + [`GroupingSet.cube`][datafusion.expr.GroupingSet.cube], or + [`GroupingSet.grouping_sets`][datafusion.expr.GroupingSet.grouping_sets], where different rows are grouped by different subsets of columns. In a default aggregation without grouping sets every column is always part of the key, so ``grouping()`` always returns 0. @@ -4745,7 +4744,7 @@ def grouping( ``.alias()`` cannot be applied directly to a ``grouping()`` expression. Doing so will raise an error at execution time. To rename the column, use - :py:meth:`~datafusion.dataframe.DataFrame.with_column_renamed` + [`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] on the result DataFrame instead. Args: @@ -4754,7 +4753,7 @@ def grouping( filter: If provided, only compute against rows for which the filter is True Examples: - With :py:meth:`~datafusion.expr.GroupingSet.rollup`, the result + With [`rollup`][datafusion.expr.GroupingSet.rollup], the result includes both per-group rows (``grouping(a) = 0``) and a grand-total row where ``a`` is aggregated across (``grouping(a) = 1``): @@ -4771,7 +4770,7 @@ def grouping( [30, 30, 60] See Also: - :py:class:`~datafusion.expr.GroupingSet` + [`GroupingSet`][datafusion.expr.GroupingSet] """ filter_raw = filter.expr if filter is not None else None return Expr(f.grouping(expression.expr, distinct=distinct, filter=filter_raw)) @@ -4987,7 +4986,7 @@ def covar(value_y: Expr, value_x: Expr, filter: Expr | None = None) -> Expr: """Computes the sample covariance. See Also: - This is an alias for :py:func:`covar_samp`. + This is an alias for [`covar_samp`][covar_samp]. """ return covar_samp(value_y, value_x, filter) @@ -5028,7 +5027,7 @@ def mean(expression: Expr, filter: Expr | None = None) -> Expr: """Returns the average (mean) value of the argument. See Also: - This is an alias for :py:func:`avg`. + This is an alias for [`avg`][avg]. """ return avg(expression, filter) @@ -5222,7 +5221,7 @@ def stddev_samp(arg: Expr, filter: Expr | None = None) -> Expr: """Computes the sample standard deviation of the argument. See Also: - This is an alias for :py:func:`stddev`. + This is an alias for [`stddev`][stddev]. """ return stddev(arg, filter=filter) @@ -5231,7 +5230,7 @@ def var(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the sample variance of the argument. See Also: - This is an alias for :py:func:`var_samp`. + This is an alias for [`var_samp`][var_samp]. """ return var_samp(expression, filter) @@ -5272,7 +5271,7 @@ def var_population(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the population variance of the argument. See Also: - This is an alias for :py:func:`var_pop`. + This is an alias for [`var_pop`][var_pop]. """ return var_pop(expression, filter) @@ -5313,7 +5312,7 @@ def var_sample(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the sample variance of the argument. See Also: - This is an alias for :py:func:`var_samp`. + This is an alias for [`var_samp`][var_samp]. """ return var_samp(expression, filter) @@ -6063,7 +6062,7 @@ def lead( return the 3rd following value in column ``b``. At the end of the partition, where no further values can be returned it will return the default value of 5. - Here is an example of both the ``lead`` and :py:func:`datafusion.functions.lag` + Here is an example of both the ``lead`` and [`lag`][datafusion.functions.lag] functions on a simple DataFrame:: +--------+------+-----+ @@ -6139,7 +6138,7 @@ def lag( will return the 3rd previous value in column ``b``. At the beginning of the partition, where no values can be returned it will return the default value of 5. - Here is an example of both the ``lag`` and :py:func:`datafusion.functions.lead` + Here is an example of both the ``lag`` and [`lead`][datafusion.functions.lead] functions on a simple DataFrame:: +--------+------+-----+ @@ -6322,7 +6321,7 @@ def dense_rank( ) -> Expr: """Create a dense_rank window function. - This window function is similar to :py:func:`rank` except that the returned values + This window function is similar to [`rank`][rank] except that the returned values will be consecutive. Here is an example of a dataframe with a window ordered by descending ``points`` and the associated dense rank:: @@ -6378,7 +6377,7 @@ def percent_rank( ) -> Expr: """Create a percent_rank window function. - This window function is similar to :py:func:`rank` except that the returned values + This window function is similar to [`rank`][rank] except that the returned values are the percentage from 0.0 to 1.0 from first to last. Here is an example of a dataframe with a window ordered by descending ``points`` and the associated percent rank:: @@ -6436,7 +6435,7 @@ def cume_dist( ) -> Expr: """Create a cumulative distribution window function. - This window function is similar to :py:func:`rank` except that the returned values + This window function is similar to [`rank`][rank] except that the returned values are the ratio of the row number to the total number of rows. Here is an example of a dataframe with a window ordered by descending ``points`` and the associated cumulative distribution:: diff --git a/python/datafusion/io.py b/python/datafusion/io.py index 4f9c3c516..9e32b08a3 100644 --- a/python/datafusion/io.py +++ b/python/datafusion/io.py @@ -43,7 +43,7 @@ def read_parquet( schema: pa.Schema | None = None, file_sort_order: list[list[Expr]] | None = None, ) -> DataFrame: - """Read a Parquet source into a :py:class:`~datafusion.dataframe.Dataframe`. + """Read a Parquet source into a [`Dataframe`][datafusion.dataframe.Dataframe]. This function will use the global context. Any functions or tables registered with another context may not be accessible when used with a DataFrame created @@ -175,7 +175,7 @@ def read_avro( file_partition_cols: list[tuple[str, str | pa.DataType]] | None = None, file_extension: str = ".avro", ) -> DataFrame: - """Create a :py:class:`DataFrame` for reading Avro data source. + """Create a `DataFrame` for reading Avro data source. This function will use the global context. Any functions or tables registered with another context may not be accessible when used with a DataFrame created diff --git a/python/datafusion/plan.py b/python/datafusion/plan.py index b2c6eab3e..9cb4b35f0 100644 --- a/python/datafusion/plan.py +++ b/python/datafusion/plan.py @@ -103,7 +103,7 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: When ``ctx`` is supplied, encoding routes through the session's installed `LogicalExtensionCodec` so user FFI codecs (registered - via :py:meth:`SessionContext.with_logical_extension_codec`) see + via `with_logical_extension_codec`) see the encode path. With ``ctx=None`` a default codec is used. Tables created in memory from record batches are currently not supported. @@ -227,15 +227,15 @@ def collect_metrics(self) -> list[tuple[str, MetricsSet]]: DataFusion executes a query as a pipeline of operators — for example a data source scan, followed by a filter, followed by a projection. After the DataFrame has been executed (via - :py:meth:`~datafusion.DataFrame.collect`, - :py:meth:`~datafusion.DataFrame.execute_stream`, etc.), each operator + [`collect`][datafusion.DataFrame.collect], + [`execute_stream`][datafusion.DataFrame.execute_stream], etc.), each operator records statistics such as how many rows it produced and how much CPU time it consumed. Each entry in the returned list corresponds to one operator that recorded metrics. The first element of the tuple is the operator's description string — the same text shown by - :py:meth:`display_indent` — which identifies both the operator type + [`display_indent`][display_indent] — which identifies both the operator type and its key parameters, for example ``"FilterExec: column1@0 > 1"`` or ``"DataSourceExec: partitions=1"``. @@ -263,8 +263,8 @@ class MetricsSet: """A set of metrics for a single execution plan operator. A physical plan operator runs independently across one or more partitions. - :py:meth:`metrics` returns the raw per-partition :py:class:`Metric` objects. - The convenience properties (:py:attr:`output_rows`, :py:attr:`elapsed_compute`, + [`metrics`][metrics] returns the raw per-partition `Metric` objects. + The convenience properties (`output_rows`, [`elapsed_compute`][elapsed_compute], etc.) automatically sum the named metric across *all* partitions, giving a single aggregate value for the operator as a whole. """ @@ -343,7 +343,7 @@ def value(self) -> int | datetime.datetime | None: """The value of this metric. Returns an ``int`` for counters, gauges, and time-based metrics - (nanoseconds), a :py:class:`~datetime.datetime` (UTC) for + (nanoseconds), a [`datetime`][datetime.datetime] (UTC) for ``start_timestamp`` / ``end_timestamp`` metrics, or ``None`` when the value has not been set or is not representable. """ @@ -351,7 +351,7 @@ def value(self) -> int | datetime.datetime | None: @property def value_as_datetime(self) -> datetime.datetime | None: - """The value as a UTC :py:class:`~datetime.datetime` for timestamp metrics. + """The value as a UTC [`datetime`][datetime.datetime] for timestamp metrics. Returns ``None`` for all non-timestamp metrics and for timestamp metrics whose value has not been set (e.g. before execution). diff --git a/python/datafusion/record_batch.py b/python/datafusion/record_batch.py index c24cde0ac..0722f0c63 100644 --- a/python/datafusion/record_batch.py +++ b/python/datafusion/record_batch.py @@ -18,7 +18,7 @@ """This module provides the classes for handling record batches. These are typically the result of dataframe -:py:func:`datafusion.dataframe.execute_stream` operations. +[`execute_stream`][datafusion.dataframe.execute_stream] operations. """ from __future__ import annotations @@ -33,17 +33,17 @@ class RecordBatch: - """This class is essentially a wrapper for :py:class:`pa.RecordBatch`.""" + """This class is essentially a wrapper for [`RecordBatch`][pa.RecordBatch].""" def __init__(self, record_batch: df_internal.RecordBatch) -> None: """This constructor is generally not called by the end user. - See the :py:class:`RecordBatchStream` iterator for generating this class. + See the `RecordBatchStream` iterator for generating this class. """ self.record_batch = record_batch def to_pyarrow(self) -> pa.RecordBatch: - """Convert to :py:class:`pa.RecordBatch`.""" + """Convert to [`RecordBatch`][pa.RecordBatch].""" return self.record_batch.to_pyarrow() def __arrow_c_array__( @@ -71,7 +71,7 @@ class RecordBatchStream: """This class represents a stream of record batches. These are typically the result of a - :py:func:`~datafusion.dataframe.DataFrame.execute_stream` operation. + [`execute_stream`][datafusion.dataframe.DataFrame.execute_stream] operation. """ def __init__(self, record_batch_stream: df_internal.RecordBatchStream) -> None: @@ -79,16 +79,16 @@ def __init__(self, record_batch_stream: df_internal.RecordBatchStream) -> None: self.rbs = record_batch_stream def next(self) -> RecordBatch: - """See :py:func:`__next__` for the iterator function.""" + """See [`__next__`][__next__] for the iterator function.""" return next(self) async def __anext__(self) -> RecordBatch: - """Return the next :py:class:`RecordBatch` in the stream asynchronously.""" + """Return the next `RecordBatch` in the stream asynchronously.""" next_batch = await self.rbs.__anext__() return RecordBatch(next_batch) def __next__(self) -> RecordBatch: - """Return the next :py:class:`RecordBatch` in the stream.""" + """Return the next `RecordBatch` in the stream.""" next_batch = next(self.rbs) return RecordBatch(next_batch) diff --git a/python/datafusion/substrait.py b/python/datafusion/substrait.py index 6353ef8cc..74aa3bd36 100644 --- a/python/datafusion/substrait.py +++ b/python/datafusion/substrait.py @@ -49,7 +49,7 @@ def __init__(self, plan: substrait_internal.Plan) -> None: """Create a substrait plan. The user should not have to call this constructor directly. Rather, it - should be created via :py:class:`Serde` or py:class:`Producer` classes + should be created via [`Serde`][Serde] or py:class:`Producer` classes in this module. """ self.plan_internal = plan diff --git a/python/datafusion/user_defined.py b/python/datafusion/user_defined.py index 81a516af8..18f2bb014 100644 --- a/python/datafusion/user_defined.py +++ b/python/datafusion/user_defined.py @@ -128,7 +128,7 @@ def __datafusion_physical_extension_codec__(self) -> object: ... # noqa: D105 class ScalarUDF: """Class for performing scalar user-defined functions (UDF). - Scalar UDFs operate on a row by row basis. See also :py:class:`AggregateUDF` for + Scalar UDFs operate on a row by row basis. See also `AggregateUDF` for operating on a group of rows. """ @@ -142,7 +142,7 @@ def __init__( ) -> None: """Instantiate a scalar user-defined function (UDF). - See helper method :py:func:`udf` for argument details. + See helper method [`udf`][datafusion.user_defined.udf] for argument details. """ if hasattr(func, "__datafusion_scalar_udf__"): self._udf = df_internal.ScalarUDF.from_pycapsule(func) @@ -157,9 +157,9 @@ def __init__( def _from_internal(cls, internal: df_internal.ScalarUDF) -> ScalarUDF: """Wrap an already-constructed internal ``ScalarUDF`` handle. - Used by :py:meth:`SessionContext.udf` to surface a function looked + Used by [`udf`][SessionContext.udf] to surface a function looked up from the session's function registry without re-running - :py:meth:`__init__`. + [`__init__`][__init__]. """ wrapper = cls.__new__(cls) wrapper._udf = internal @@ -355,7 +355,7 @@ def from_pycapsule(func: ScalarUDFExportable) -> ScalarUDF: class Accumulator(metaclass=ABCMeta): - """Defines how an :py:class:`AggregateUDF` accumulates values.""" + """Defines how an `AggregateUDF` accumulates values.""" @abstractmethod def state(self) -> list[pa.Scalar]: @@ -402,7 +402,7 @@ class AggregateUDF: """Class for performing scalar user-defined functions (UDF). Aggregate UDFs operate on a group of rows and return a single value. See - also :py:class:`ScalarUDF` for operating on a row by row basis. + also `ScalarUDF` for operating on a row by row basis. """ @overload @@ -438,7 +438,7 @@ def __init__( ) -> None: """Instantiate a user-defined aggregate function (UDAF). - See :py:func:`udaf` for a convenience function and argument + See `udaf` for a convenience function and argument descriptions. """ if hasattr(accumulator, "__datafusion_aggregate_udf__"): @@ -469,9 +469,9 @@ def __init__( def _from_internal(cls, internal: df_internal.AggregateUDF) -> AggregateUDF: """Wrap an already-constructed internal ``AggregateUDF`` handle. - Used by :py:meth:`SessionContext.udaf` to surface a function looked + Used by [`udaf`][SessionContext.udaf] to surface a function looked up from the session's function registry without re-running - :py:meth:`__init__`. + [`__init__`][__init__]. """ wrapper = cls.__new__(cls) wrapper._udaf = internal @@ -541,10 +541,10 @@ def udaf(*args: Any, **kwargs: Any): # noqa: D417, C901 - As a decorator: ``@udaf(input_types, return_type, state_type, volatility, name)``. When using ``udaf`` as a decorator, do not pass ``accum`` explicitly. - If your :py:class:`Accumulator` can be instantiated with no arguments, you + If your `Accumulator` can be instantiated with no arguments, you can simply pass its type as ``accum``. If you need to pass additional arguments to its constructor, you can define a lambda or a factory method. - During runtime the :py:class:`Accumulator` will be constructed for every + During runtime the `Accumulator` will be constructed for every instance in which this UDAF is used. Examples: @@ -617,7 +617,7 @@ def udaf(*args: Any, **kwargs: Any): # noqa: D417, C901 input_types: The data types of the arguments to ``accum``. return_type: The data type of the return value. state_type: The data types of the intermediate accumulation. - volatility: See :py:class:`Volatility` for allowed values. + volatility: See [`Volatility`][Volatility] for allowed values. name: A descriptive name for the function. Returns: @@ -762,13 +762,13 @@ def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array: This function is called once per input *partition* for window functions that *do not use* values from the window frame, such as - :py:func:`~datafusion.functions.row_number`, - :py:func:`~datafusion.functions.rank`, - :py:func:`~datafusion.functions.dense_rank`, - :py:func:`~datafusion.functions.percent_rank`, - :py:func:`~datafusion.functions.cume_dist`, - :py:func:`~datafusion.functions.lead`, - and :py:func:`~datafusion.functions.lag`. + [`row_number`][datafusion.functions.row_number], + [`rank`][datafusion.functions.rank], + [`dense_rank`][datafusion.functions.dense_rank], + [`percent_rank`][datafusion.functions.percent_rank], + [`cume_dist`][datafusion.functions.cume_dist], + [`lead`][datafusion.functions.lead], + and [`lag`][datafusion.functions.lag]. It produces the result of all rows in a single pass. It expects to receive the entire partition as the ``value`` and @@ -874,7 +874,7 @@ class WindowUDF: """Class for performing window user-defined functions (UDF). Window UDFs operate on a partition of rows. See - also :py:class:`ScalarUDF` for operating on a row by row basis. + also `ScalarUDF` for operating on a row by row basis. """ def __init__( @@ -887,7 +887,7 @@ def __init__( ) -> None: """Instantiate a user-defined window function (UDWF). - See :py:func:`udwf` for a convenience function and argument + See `udwf` for a convenience function and argument descriptions. """ if hasattr(func, "__datafusion_window_udf__"): @@ -901,9 +901,9 @@ def __init__( def _from_internal(cls, internal: df_internal.WindowUDF) -> WindowUDF: """Wrap an already-constructed internal ``WindowUDF`` handle. - Used by :py:meth:`SessionContext.udwf` to surface a function looked + Used by [`udwf`][SessionContext.udwf] to surface a function looked up from the session's function registry without re-running - :py:meth:`__init__`. + [`__init__`][__init__]. """ wrapper = cls.__new__(cls) wrapper._udwf = internal @@ -1009,7 +1009,7 @@ def udwf(*args: Any, **kwargs: Any): # noqa: D417 the online documentation for more information. input_types: The data types of the arguments. return_type: The data type of the return value. - volatility: See :py:class:`Volatility` for allowed values. + volatility: See [`Volatility`][Volatility] for allowed values. name: A descriptive name for the function. Returns: @@ -1155,7 +1155,7 @@ def __init__( Configuration changes do **not** propagate; the wrapper holds its own clone of the session config. - See :py:func:`udtf` for a convenience function and argument + See `udtf` for a convenience function and argument descriptions. """ if with_session and hasattr(func, "__datafusion_table_function__"): diff --git a/uv.lock b/uv.lock index 89617aed0..22c38c9d0 100644 --- a/uv.lock +++ b/uv.lock @@ -9,24 +9,12 @@ resolution-markers = [ ] [[package]] -name = "accessible-pygments" -version = "0.0.5" +name = "appnope" +version = "0.1.4" source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "pygments" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/bc/c1/bbac6a50d02774f91572938964c582fff4270eee73ab822a4aeea4d8b11b/accessible_pygments-0.0.5.tar.gz", hash = "sha256:40918d3e6a2b619ad424cb91e556bd3bd8865443d9f22f1dcdf79e33c8046872", size = 1377899, upload-time = "2024-05-10T11:23:10.216Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/8d/3f/95338030883d8c8b91223b4e21744b04d11b161a3ef117295d8241f50ab4/accessible_pygments-0.0.5-py3-none-any.whl", hash = "sha256:88ae3211e68a1d0b011504b2ffc1691feafce124b845bd072ab6f9f66f34d4b7", size = 1395903, upload-time = "2024-05-10T11:23:08.421Z" }, -] - -[[package]] -name = "alabaster" -version = "1.0.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/a6/f8/d9c74d0daf3f742840fd818d69cfae176fa332022fd44e3469487d5a9420/alabaster-1.0.0.tar.gz", hash = "sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e", size = 24210, upload-time = "2024-07-26T18:15:03.762Z" } +sdist = { url = "https://files.pythonhosted.org/packages/35/5d/752690df9ef5b76e169e68d6a129fa6d08a7100ca7f754c89495db3c6019/appnope-0.1.4.tar.gz", hash = "sha256:1de3860566df9caf38f01f86f65e0e13e379af54f9e4bee1e66b48f2efffd1ee", size = 4170, upload-time = "2024-02-06T09:43:11.258Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/7e/b3/6b4067be973ae96ba0d615946e314c5ae35f9f993eca561b356540bb0c2b/alabaster-1.0.0-py3-none-any.whl", hash = "sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b", size = 13929, upload-time = "2024-07-26T18:15:02.05Z" }, + { url = "https://files.pythonhosted.org/packages/81/29/5ecc3a15d5a33e31b26c11426c45c501e439cb865d0bff96315d86443b78/appnope-0.1.4-py2.py3-none-any.whl", hash = "sha256:502575ee11cd7a28c0205f379b525beefebab9d161b7c964670864014ed7213c", size = 4321, upload-time = "2024-02-06T09:43:09.663Z" }, ] [[package]] @@ -92,24 +80,21 @@ wheels = [ ] [[package]] -name = "astroid" -version = "3.3.8" +name = "asttokens" +version = "3.0.0" source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "typing-extensions", marker = "python_full_version < '3.11'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/80/c5/5c83c48bbf547f3dd8b587529db7cf5a265a3368b33e85e76af8ff6061d3/astroid-3.3.8.tar.gz", hash = "sha256:a88c7994f914a4ea8572fac479459f4955eeccc877be3f2d959a33273b0cf40b", size = 398196, upload-time = "2024-12-24T01:13:05.59Z" } +sdist = { url = "https://files.pythonhosted.org/packages/4a/e7/82da0a03e7ba5141f05cce0d302e6eed121ae055e0456ca228bf693984bc/asttokens-3.0.0.tar.gz", hash = "sha256:0dcd8baa8d62b0c1d118b399b2ddba3c4aff271d0d7a9e0d4c1681c79035bbc7", size = 61978, upload-time = "2024-11-30T04:30:14.439Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/07/28/0bc8a17d6cd4cc3c79ae41b7105a2b9a327c110e5ddd37a8a27b29a5c8a2/astroid-3.3.8-py3-none-any.whl", hash = "sha256:187ccc0c248bfbba564826c26f070494f7bc964fd286b6d9fff4420e55de828c", size = 275153, upload-time = "2024-12-24T01:13:02.726Z" }, + { url = "https://files.pythonhosted.org/packages/25/8a/c46dcc25341b5bce5472c718902eb3d38600a903b14fa6aeecef3f21a46f/asttokens-3.0.0-py3-none-any.whl", hash = "sha256:e3078351a059199dd5138cb1c706e6430c05eff2ff136af5eb4790f9d28932e2", size = 26918, upload-time = "2024-11-30T04:30:10.946Z" }, ] [[package]] -name = "asttokens" -version = "3.0.0" +name = "attrs" +version = "26.1.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/4a/e7/82da0a03e7ba5141f05cce0d302e6eed121ae055e0456ca228bf693984bc/asttokens-3.0.0.tar.gz", hash = "sha256:0dcd8baa8d62b0c1d118b399b2ddba3c4aff271d0d7a9e0d4c1681c79035bbc7", size = 61978, upload-time = "2024-11-30T04:30:14.439Z" } +sdist = { url = "https://files.pythonhosted.org/packages/9a/8e/82a0fe20a541c03148528be8cac2408564a6c9a0cc7e9171802bc1d26985/attrs-26.1.0.tar.gz", hash = "sha256:d03ceb89cb322a8fd706d4fb91940737b6642aa36998fe130a9bc96c985eff32", size = 952055, upload-time = "2026-03-19T14:22:25.026Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/25/8a/c46dcc25341b5bce5472c718902eb3d38600a903b14fa6aeecef3f21a46f/asttokens-3.0.0-py3-none-any.whl", hash = "sha256:e3078351a059199dd5138cb1c706e6430c05eff2ff136af5eb4790f9d28932e2", size = 26918, upload-time = "2024-11-30T04:30:10.946Z" }, + { url = "https://files.pythonhosted.org/packages/64/b4/17d4b0b2a2dc85a6df63d1157e028ed19f90d4cd97c36717afef2bc2f395/attrs-26.1.0-py3-none-any.whl", hash = "sha256:c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309", size = 67548, upload-time = "2026-03-19T14:22:23.645Z" }, ] [[package]] @@ -121,6 +106,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ed/20/bc79bc575ba2e2a7f70e8a1155618bb1301eaa5132a8271373a6903f73f8/babel-2.16.0-py3-none-any.whl", hash = "sha256:368b5b98b37c06b7daf6696391c3240c938b37767d4584413e8438c5c435fa8b", size = 9587599, upload-time = "2024-08-08T14:25:42.686Z" }, ] +[[package]] +name = "backrefs" +version = "7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5e/a7/a7dd63622beef68cc0d3c3c36d472e143dd95443d5ebf14cd1a5b4dfbf11/backrefs-7.0.tar.gz", hash = "sha256:4989bb9e1e99eb23647c7160ed51fb21d0b41b5d200f2d3017da41e023097e82", size = 7012453, upload-time = "2026-04-28T16:28:04.215Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d4/39/39a31d7eae729ea14ed10c3ccef79371197177b9355a86cb3525709e8502/backrefs-7.0-py310-none-any.whl", hash = "sha256:b57cd227ea556b0aed3dc9b8da4628db4eabc0402c6d7fcfc69283a93955f7e9", size = 380824, upload-time = "2026-04-28T16:27:55.647Z" }, + { url = "https://files.pythonhosted.org/packages/c9/b5/9302644225ba7dfa934a2ff2b9c7bb85701313a90dddb3dfaf693fa5bae2/backrefs-7.0-py311-none-any.whl", hash = "sha256:a0fa7360c63509e9e077e174ef4e6d3c21c8db94189b9d957289ae6d794b9475", size = 392626, upload-time = "2026-04-28T16:27:57.42Z" }, + { url = "https://files.pythonhosted.org/packages/36/da/87912ddec6e06feffbaa3d7aa18fc6352bee2e8f1fee185d7d1690f8f4e8/backrefs-7.0-py312-none-any.whl", hash = "sha256:ca42ce6a49ace3d75684dfa9937f3373902a63284ecb385ce36d15e5dcb41c12", size = 398537, upload-time = "2026-04-28T16:27:58.913Z" }, + { url = "https://files.pythonhosted.org/packages/00/bb/90ba423612b6aa0adccc6b1874bcd4a9b44b660c0c16f346611e00f64ac3/backrefs-7.0-py313-none-any.whl", hash = "sha256:f2c52955d631b9e1ac4cd56209f0a3a946d592b98e7790e77699339ae01c102a", size = 400491, upload-time = "2026-04-28T16:28:00.928Z" }, + { url = "https://files.pythonhosted.org/packages/3e/5c/fb93d3092640a24dfb7bd7727a24016d7c01774ca013e60efd3f683c8002/backrefs-7.0-py314-none-any.whl", hash = "sha256:a6448b28180e3ca01134c9cf09dcebafad8531072e09903c5451748a05f24bc9", size = 412349, upload-time = "2026-04-28T16:28:02.412Z" }, +] + [[package]] name = "beautifulsoup4" version = "4.12.3" @@ -133,6 +131,23 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b1/fe/e8c672695b37eecc5cbf43e1d0638d88d66ba3a44c4d321c796f4e59167f/beautifulsoup4-4.12.3-py3-none-any.whl", hash = "sha256:b80878c9f40111313e55da8ba20bdba06d8fa3969fc68304167741bbf9e082ed", size = 147925, upload-time = "2024-01-17T16:53:12.779Z" }, ] +[[package]] +name = "bleach" +version = "6.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "webencodings" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/48/3c/e12ac860709702bd5ebeb9b56a4fe334f1001246ee1b8f2b7ee28912df7d/bleach-6.4.0.tar.gz", hash = "sha256:4202482733d85cedd04e59fcb2f89f4e4c7c385a78d3c3c23c30446843a37452", size = 204857, upload-time = "2026-06-05T13:01:13.734Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/58/9d/40b6267367182187139a4000b82a3b287d84d745bccd808e75d916920e9d/bleach-6.4.0-py3-none-any.whl", hash = "sha256:4b6b6a54fff2e69a3dde9d21cc6301220bee3c3cb792187d11403fd795031081", size = 165109, upload-time = "2026-06-05T13:01:12.504Z" }, +] + +[package.optional-dependencies] +css = [ + { name = "tinycss2" }, +] + [[package]] name = "certifi" version = "2024.12.14" @@ -269,6 +284,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/0e/f6/65ecc6878a89bb1c23a086ea335ad4bf21a588990c3f535a227b9eea9108/charset_normalizer-3.4.1-py3-none-any.whl", hash = "sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85", size = 49767, upload-time = "2024-12-24T18:12:32.852Z" }, ] +[[package]] +name = "click" +version = "8.4.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9b/98/518d8e5081007684232226f475082b30087d0f585e8457db087298259f49/click-8.4.1.tar.gz", hash = "sha256:918b5633eddf6b41c32d4f454bf0de810065c74e3f7dbf8ee5452f8be88d3e96", size = 353007, upload-time = "2026-05-22T04:08:37.769Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c7/0d/67e5b4109ea4a837e80daa87c2c696711955e40449a97e8926672534def2/click-8.4.1-py3-none-any.whl", hash = "sha256:482be17c6991b8c19c5429a1e995d9b0efdbb63172824c41f99965dc0ade8ec2", size = 116639, upload-time = "2026-05-22T04:08:35.26Z" }, +] + [[package]] name = "cloudpickle" version = "3.1.2" @@ -296,6 +323,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, ] +[[package]] +name = "comm" +version = "0.2.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/4c/13/7d740c5849255756bc17888787313b61fd38a0a8304fc4f073dfc46122aa/comm-0.2.3.tar.gz", hash = "sha256:2dc8048c10962d55d7ad693be1e7045d891b7ce8d999c97963a5e3e99c055971", size = 6319, upload-time = "2025-07-25T14:02:04.452Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/60/97/891a0971e1e4a8c5d2b20bbe0e524dc04548d2307fee33cdeba148fd4fc7/comm-0.2.3-py3-none-any.whl", hash = "sha256:c615d91d75f7f04f095b30d1c1711babd43bdc6419c1be9886a85f2f4e489417", size = 7294, upload-time = "2025-07-25T14:02:02.896Z" }, +] + [[package]] name = "cryptography" version = "44.0.0" @@ -361,16 +397,13 @@ dev = [ ] docs = [ { name = "ipython" }, - { name = "jinja2" }, - { name = "myst-parser" }, + { name = "mkdocs" }, + { name = "mkdocs-jupyter" }, + { name = "mkdocs-material" }, + { name = "mkdocs-redirects" }, + { name = "mkdocstrings", extra = ["python"] }, { name = "pandas" }, { name = "pickleshare" }, - { name = "pydata-sphinx-theme" }, - { name = "setuptools" }, - { name = "sphinx" }, - { name = "sphinx-autoapi" }, - { name = "sphinx-reredirects", version = "0.1.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "sphinx-reredirects", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, ] release = [ { name = "pygithub" }, @@ -403,18 +436,45 @@ dev = [ ] docs = [ { name = "ipython", specifier = ">=8.12.3" }, - { name = "jinja2", specifier = ">=3.1.5" }, - { name = "myst-parser", specifier = ">=3.0.1" }, + { name = "mkdocs", specifier = ">=1.6,<2" }, + { name = "mkdocs-jupyter", specifier = ">=0.25" }, + { name = "mkdocs-material", specifier = ">=9.5,<10" }, + { name = "mkdocs-redirects", specifier = ">=1.2" }, + { name = "mkdocstrings", extras = ["python"], specifier = ">=0.27" }, { name = "pandas", specifier = ">=2.0.3" }, { name = "pickleshare", specifier = ">=0.7.5" }, - { name = "pydata-sphinx-theme", specifier = ">=0.16,<0.17" }, - { name = "setuptools", specifier = ">=75.3.0" }, - { name = "sphinx", specifier = ">=7.1.2" }, - { name = "sphinx-autoapi", specifier = ">=3.4.0" }, - { name = "sphinx-reredirects", specifier = ">=0.1.5" }, ] release = [{ name = "pygithub", specifier = "==2.5.0" }] +[[package]] +name = "debugpy" +version = "1.8.21" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f2/aa/12037145b7a56eaa5b29b41872f7a21b538e807e13f32c4d3c46e59be084/debugpy-1.8.21.tar.gz", hash = "sha256:a3c53278e84c94e11bd87c53970ec391d1a67396c8b22609fcac576520e611a6", size = 1697577, upload-time = "2026-06-01T19:30:35.156Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fc/f3/6b1d4c71f4cbb5360009f928934a03b42906f28fc7b3f7f35f04e58acead/debugpy-1.8.21-cp310-cp310-macosx_15_0_x86_64.whl", hash = "sha256:8eeab7b5462f683452c57c0126aaa5ec4e974ddb705f39ba87dff8818c8e08f9", size = 2113873, upload-time = "2026-06-01T19:30:37.148Z" }, + { url = "https://files.pythonhosted.org/packages/1c/f2/17c3bf91cebc173bfbf5734cd2669723d0a35c0cf9d2fd2124546efeae83/debugpy-1.8.21-cp310-cp310-manylinux_2_34_x86_64.whl", hash = "sha256:0fddfdc130ac6d8bfc0415b0409822fa901c8f310e5c945ac5653a0352532344", size = 3004715, upload-time = "2026-06-01T19:30:38.888Z" }, + { url = "https://files.pythonhosted.org/packages/5a/22/1f8efd80c7b5909e760f9cfd0c9e8681d2d35d532f7c0a40760cd4da4a19/debugpy-1.8.21-cp310-cp310-win32.whl", hash = "sha256:72b5d676c4cbfac3bac5bb01c138a4656e843f93f03ce2a5f4e394ad49fbee73", size = 5303455, upload-time = "2026-06-01T19:30:40.52Z" }, + { url = "https://files.pythonhosted.org/packages/da/ce/54c79abd6cccef92fa7b43d97e3acafedf4d645557267ece05e948b5e4b8/debugpy-1.8.21-cp310-cp310-win_amd64.whl", hash = "sha256:a7fe47fd23da57b9e0bec3f4a8ee65a2dc55782455ed7f2141d75ab5d2eaeef5", size = 5331751, upload-time = "2026-06-01T19:30:42.146Z" }, + { url = "https://files.pythonhosted.org/packages/89/fb/cbf306d6e07a313a91e7171a98669054502840931432c227cfd505ee367f/debugpy-1.8.21-cp311-cp311-macosx_15_0_universal2.whl", hash = "sha256:da456226c7b4c69e35dbe35dcee6623d912000a77816db7856a41af1c72a0264", size = 2203120, upload-time = "2026-06-01T19:30:43.964Z" }, + { url = "https://files.pythonhosted.org/packages/aa/57/aa739bd4ad2cbf96aeb1b20b56918ddd5ae4c28b68709bfcd327f02123ee/debugpy-1.8.21-cp311-cp311-manylinux_2_34_x86_64.whl", hash = "sha256:f68b891688e61bdc08b8d364d919ff0051e0b94657b39dcd027bc3173edb7cdc", size = 3059958, upload-time = "2026-06-01T19:30:45.622Z" }, + { url = "https://files.pythonhosted.org/packages/a8/31/453d2c9a23d133fe2c8ec7ca1d816ded52a913487fe3ffef7c01b4b706af/debugpy-1.8.21-cp311-cp311-win32.whl", hash = "sha256:f843a8b08c2edeaf9b1582eed4f25441af21a297c22ff16bf76a662557aa9c9e", size = 5236515, upload-time = "2026-06-01T19:30:47.461Z" }, + { url = "https://files.pythonhosted.org/packages/60/94/6660de2f2d7bf388f229335ba4637646eebabdbf38564cb439a95a9193c9/debugpy-1.8.21-cp311-cp311-win_amd64.whl", hash = "sha256:84c564d8cc701d41843b29a92814c1f1bef6798724ca9d675c284ad9f6a547d7", size = 5256138, upload-time = "2026-06-01T19:30:49.113Z" }, + { url = "https://files.pythonhosted.org/packages/a2/df/bf625547431a9cadc9f4cbfeda38866e2b17f6aed147b625377e87834449/debugpy-1.8.21-cp312-cp312-macosx_15_0_universal2.whl", hash = "sha256:9f96713896f39c3dff0ee841f47320c3f2983d33c341e009361bb0ebc79adc4e", size = 2483609, upload-time = "2026-06-01T19:30:50.794Z" }, + { url = "https://files.pythonhosted.org/packages/bf/09/59324b903599031ff9faaec1758292409f6561a0ec2492fe4b703327705a/debugpy-1.8.21-cp312-cp312-manylinux_2_34_x86_64.whl", hash = "sha256:c193d474f0a211191f2b4449d2d06157c689013035bd952f3b617e0ef422b176", size = 3968900, upload-time = "2026-06-01T19:30:52.341Z" }, + { url = "https://files.pythonhosted.org/packages/14/cd/27f65b805d7fe005c44e1a36b9183ecdfbcdbf9d3e721a5115d461ecc7ee/debugpy-1.8.21-cp312-cp312-win32.whl", hash = "sha256:4743373c1cac7f9e74a1b9915bf1dbe0e900eca657ffb170ae07ac8363205ae9", size = 5336340, upload-time = "2026-06-01T19:30:54.047Z" }, + { url = "https://files.pythonhosted.org/packages/77/1d/c84e30c0c674184948b66f076ab271c01d940618a2824c23cd035a27bc20/debugpy-1.8.21-cp312-cp312-win_amd64.whl", hash = "sha256:bd7ba9dd3daa7c2f942c6ca8d4695a16bf9ac16b63615261c7982bc74f7ed20c", size = 5374751, upload-time = "2026-06-01T19:30:55.891Z" }, + { url = "https://files.pythonhosted.org/packages/77/6b/d817e1f8cc77aa055d37fba092e0febfdff40fe652d8d53d4cd7a86ad98d/debugpy-1.8.21-cp313-cp313-macosx_15_0_universal2.whl", hash = "sha256:13678151fc401e2d68c9880b91e28714f797d40422994572b24560ef80910a88", size = 2477398, upload-time = "2026-06-01T19:30:57.644Z" }, + { url = "https://files.pythonhosted.org/packages/48/57/412421516afc3055fa577516f00beec3d663f9b0ab330639547ae6c57720/debugpy-1.8.21-cp313-cp313-manylinux_2_34_x86_64.whl", hash = "sha256:ecbd158386c31ffe71d46f72d44d56e66331ab9b16cad649156d514368f23ab2", size = 3962096, upload-time = "2026-06-01T19:30:59.235Z" }, + { url = "https://files.pythonhosted.org/packages/c1/62/2c616337cf6ba7b07ebbc97f02c6c945a8e2f76b365e33ee809c32ee36d1/debugpy-1.8.21-cp313-cp313-win32.whl", hash = "sha256:2c2ae706dec41d99a9ca1f7ebc987a83e65578363be6f6b3ac9067504917fae1", size = 5336288, upload-time = "2026-06-01T19:31:00.79Z" }, + { url = "https://files.pythonhosted.org/packages/f8/99/9175103392f84c4b1bf7622888cdc68da07f0ff7d9e581266428f6776033/debugpy-1.8.21-cp313-cp313-win_amd64.whl", hash = "sha256:aa648733047443eb1d07682c4ef287d36a54507b643ffdf38b09a3ef002c72a0", size = 5376567, upload-time = "2026-06-01T19:31:02.56Z" }, + { url = "https://files.pythonhosted.org/packages/ce/3d/f4bbb323a548bfab2af3d6b4ffd9bf22636e55956a1285d317a1de643aad/debugpy-1.8.21-cp314-cp314-macosx_15_0_universal2.whl", hash = "sha256:9bb2a685287a2ac9b181cde89edcec64845cb51de7faaa75badb9a698bc24782", size = 2477209, upload-time = "2026-06-01T19:31:04.157Z" }, + { url = "https://files.pythonhosted.org/packages/8c/2d/6e7ec524984a1702777868de49a4c53202bddac2a432a76a093469587750/debugpy-1.8.21-cp314-cp314-manylinux_2_34_x86_64.whl", hash = "sha256:3d6922439bf33fd38a3e2c447869ebc7b97da5cd3d329ff1ef9bc06c4903437e", size = 3927115, upload-time = "2026-06-01T19:31:05.863Z" }, + { url = "https://files.pythonhosted.org/packages/97/47/d1aa6d64005a98a9144647d99306b419396f9ad7bf1d73c119e17a81fb4d/debugpy-1.8.21-cp314-cp314-win32.whl", hash = "sha256:15d4963bd5ffa48f0da0947fd06757fa7621945048a14ad7705431566d3c0e7c", size = 5336724, upload-time = "2026-06-01T19:31:07.711Z" }, + { url = "https://files.pythonhosted.org/packages/5f/67/b905b90d163af11878c1af8abafa4a25206335e112e284e413454543a6da/debugpy-1.8.21-cp314-cp314-win_amd64.whl", hash = "sha256:fe0744a12353406de0ae8ccff0d0a4a666f00801a3db8fd04e7a5f761cd520e8", size = 5373803, upload-time = "2026-06-01T19:31:09.469Z" }, + { url = "https://files.pythonhosted.org/packages/95/51/67e7cf11a53e40694f720457d5b3a1cdaaa3d5a9a633e482f225456b93ff/debugpy-1.8.21-py2.py3-none-any.whl", hash = "sha256:b1e37d333663c8851516a47364ef473da127f9caebe4417e6df6f5825a7e9a92", size = 5352888, upload-time = "2026-06-01T19:31:25.186Z" }, +] + [[package]] name = "decorator" version = "5.1.1" @@ -424,6 +484,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d5/50/83c593b07763e1161326b3b8c6686f0f4b0f24d5526546bee538c89837d6/decorator-5.1.1-py3-none-any.whl", hash = "sha256:b8c3f85900b9dc423225913c5aace94729fe1fa9763b38939a95226f02d37186", size = 9073, upload-time = "2022-01-07T08:20:03.734Z" }, ] +[[package]] +name = "defusedxml" +version = "0.7.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/0f/d5/c66da9b79e5bdb124974bfe172b4daf3c984ebd9c2a06e2b8a4dc7331c72/defusedxml-0.7.1.tar.gz", hash = "sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69", size = 75520, upload-time = "2021-03-08T10:59:26.269Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/07/6c/aa3f2f849e01cb6a001cd8554a88d4c77c5c1a31c95bdf1cf9301e6d9ef4/defusedxml-0.7.1-py2.py3-none-any.whl", hash = "sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61", size = 25604, upload-time = "2021-03-08T10:59:24.45Z" }, +] + [[package]] name = "deprecated" version = "1.2.18" @@ -445,15 +514,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/91/a1/cf2472db20f7ce4a6be1253a81cfdf85ad9c7885ffbed7047fb72c24cf87/distlib-0.3.9-py2.py3-none-any.whl", hash = "sha256:47f8c22fd27c27e25a65601af709b38e4f0a45ea4fc2e710f65755fa8caaaf87", size = 468973, upload-time = "2024-10-09T18:35:44.272Z" }, ] -[[package]] -name = "docutils" -version = "0.21.2" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/ae/ed/aefcc8cd0ba62a0560c3c18c33925362d46c6075480bfa4df87b28e169a9/docutils-0.21.2.tar.gz", hash = "sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f", size = 2204444, upload-time = "2024-04-23T18:57:18.24Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/8f/d7/9322c609343d929e75e7e5e6255e614fcc67572cfd083959cdef3b7aad79/docutils-0.21.2-py3-none-any.whl", hash = "sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2", size = 587408, upload-time = "2024-04-23T18:57:14.835Z" }, -] - [[package]] name = "exceptiongroup" version = "1.2.2" @@ -472,6 +532,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b5/fd/afcd0496feca3276f509df3dbd5dae726fcc756f1a08d9e25abe1733f962/executing-2.1.0-py2.py3-none-any.whl", hash = "sha256:8d63781349375b5ebccc3142f4b30350c0cd9c79f921cde38be2be4637e98eaf", size = 25805, upload-time = "2024-09-01T12:37:33.007Z" }, ] +[[package]] +name = "fastjsonschema" +version = "2.21.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/20/b5/23b216d9d985a956623b6bd12d4086b60f0059b27799f23016af04a74ea1/fastjsonschema-2.21.2.tar.gz", hash = "sha256:b1eb43748041c880796cd077f1a07c3d94e93ae84bba5ed36800a33554ae05de", size = 374130, upload-time = "2025-08-14T18:49:36.666Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/a8/20d0723294217e47de6d9e2e40fd4a9d2f7c4b6ef974babd482a59743694/fastjsonschema-2.21.2-py3-none-any.whl", hash = "sha256:1c797122d0a86c5cace2e54bf4e819c36223b552017172f32c5c024a6b77e463", size = 24024, upload-time = "2025-08-14T18:49:34.776Z" }, +] + [[package]] name = "filelock" version = "3.18.0" @@ -481,6 +550,27 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/4d/36/2a115987e2d8c300a974597416d9de88f2444426de9571f4b59b2cca3acc/filelock-3.18.0-py3-none-any.whl", hash = "sha256:c401f4f8377c4464e6db25fff06205fd89bdd83b65eb0488ed1b160f780e21de", size = 16215, upload-time = "2025-03-14T07:11:39.145Z" }, ] +[[package]] +name = "ghp-import" +version = "2.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "python-dateutil" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/d9/29/d40217cbe2f6b1359e00c6c307bb3fc876ba74068cbab3dde77f03ca0dc4/ghp-import-2.1.0.tar.gz", hash = "sha256:9c535c4c61193c2df8871222567d7fd7e5014d835f97dc7b7439069e2413d343", size = 10943, upload-time = "2022-05-02T15:47:16.11Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f7/ec/67fbef5d497f86283db54c22eec6f6140243aae73265799baaaa19cd17fb/ghp_import-2.1.0-py3-none-any.whl", hash = "sha256:8337dd7b50877f163d4c0289bc1f1c7f127550241988d568c1db512c4324a619", size = 11034, upload-time = "2022-05-02T15:47:14.552Z" }, +] + +[[package]] +name = "griffelib" +version = "2.0.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/9d/82/74f4a3310cdabfbb10da554c3a672847f1ed33c6f61dd472681ce7f1fe67/griffelib-2.0.2.tar.gz", hash = "sha256:3cf20b3bc470e83763ffbf236e0076b1211bac1bc67de13daf494640f2de707e", size = 166461, upload-time = "2026-03-27T11:34:51.091Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/11/8c/c9138d881c79aa0ea9ed83cbd58d5ca75624378b38cee225dcf5c42cc91f/griffelib-2.0.2-py3-none-any.whl", hash = "sha256:925c857658fb1ba40c0772c37acbc2ab650bd794d9c1b9726922e36ea4117ea1", size = 142357, upload-time = "2026-03-27T11:34:46.275Z" }, +] + [[package]] name = "identify" version = "2.6.12" @@ -500,21 +590,36 @@ wheels = [ ] [[package]] -name = "imagesize" -version = "1.4.1" +name = "iniconfig" +version = "2.0.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/a7/84/62473fb57d61e31fef6e36d64a179c8781605429fd927b5dd608c997be31/imagesize-1.4.1.tar.gz", hash = "sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a", size = 1280026, upload-time = "2022-07-01T12:21:05.687Z" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/4b/cbd8e699e64a6f16ca3a8220661b5f83792b3017d0f79807cb8708d33913/iniconfig-2.0.0.tar.gz", hash = "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3", size = 4646, upload-time = "2023-01-07T11:08:11.254Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ff/62/85c4c919272577931d407be5ba5d71c20f0b616d31a0befe0ae45bb79abd/imagesize-1.4.1-py2.py3-none-any.whl", hash = "sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b", size = 8769, upload-time = "2022-07-01T12:21:02.467Z" }, + { url = "https://files.pythonhosted.org/packages/ef/a6/62565a6e1cf69e10f5727360368e451d4b7f58beeac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl", hash = "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374", size = 5892, upload-time = "2023-01-07T11:08:09.864Z" }, ] [[package]] -name = "iniconfig" -version = "2.0.0" +name = "ipykernel" +version = "7.2.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d7/4b/cbd8e699e64a6f16ca3a8220661b5f83792b3017d0f79807cb8708d33913/iniconfig-2.0.0.tar.gz", hash = "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3", size = 4646, upload-time = "2023-01-07T11:08:11.254Z" } +dependencies = [ + { name = "appnope", marker = "sys_platform == 'darwin'" }, + { name = "comm" }, + { name = "debugpy" }, + { name = "ipython" }, + { name = "jupyter-client" }, + { name = "jupyter-core" }, + { name = "matplotlib-inline" }, + { name = "nest-asyncio" }, + { name = "packaging" }, + { name = "psutil" }, + { name = "pyzmq" }, + { name = "tornado" }, + { name = "traitlets" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ca/8d/b68b728e2d06b9e0051019640a40a9eb7a88fcd82c2e1b5ce70bef5ff044/ipykernel-7.2.0.tar.gz", hash = "sha256:18ed160b6dee2cbb16e5f3575858bc19d8f1fe6046a9a680c708494ce31d909e", size = 176046, upload-time = "2026-02-06T16:43:27.403Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ef/a6/62565a6e1cf69e10f5727360368e451d4b7f58beeac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl", hash = "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374", size = 5892, upload-time = "2023-01-07T11:08:09.864Z" }, + { url = "https://files.pythonhosted.org/packages/82/b9/e73d5d9f405cba7706c539aa8b311b49d4c2f3d698d9c12f815231169c71/ipykernel-7.2.0-py3-none-any.whl", hash = "sha256:3bbd4420d2b3cc105cbdf3756bfc04500b1e52f090a90716851f3916c62e1661", size = 118788, upload-time = "2026-02-06T16:43:25.149Z" }, ] [[package]] @@ -563,6 +668,98 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/bd/0f/2ba5fbcd631e3e88689309dbe978c5769e883e4b84ebfe7da30b43275c5a/jinja2-3.1.5-py3-none-any.whl", hash = "sha256:aba0f4dc9ed8013c424088f68a5c226f7d6097ed89b246d7749c2ec4175c6adb", size = 134596, upload-time = "2024-12-21T18:30:19.133Z" }, ] +[[package]] +name = "jsonschema" +version = "4.26.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "attrs" }, + { name = "jsonschema-specifications" }, + { name = "referencing" }, + { name = "rpds-py", version = "0.30.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "rpds-py", version = "2026.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b3/fc/e067678238fa451312d4c62bf6e6cf5ec56375422aee02f9cb5f909b3047/jsonschema-4.26.0.tar.gz", hash = "sha256:0c26707e2efad8aa1bfc5b7ce170f3fccc2e4918ff85989ba9ffa9facb2be326", size = 366583, upload-time = "2026-01-07T13:41:07.246Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/69/90/f63fb5873511e014207a475e2bb4e8b2e570d655b00ac19a9a0ca0a385ee/jsonschema-4.26.0-py3-none-any.whl", hash = "sha256:d489f15263b8d200f8387e64b4c3a75f06629559fb73deb8fdfb525f2dab50ce", size = 90630, upload-time = "2026-01-07T13:41:05.306Z" }, +] + +[[package]] +name = "jsonschema-specifications" +version = "2025.9.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "referencing" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/19/74/a633ee74eb36c44aa6d1095e7cc5569bebf04342ee146178e2d36600708b/jsonschema_specifications-2025.9.1.tar.gz", hash = "sha256:b540987f239e745613c7a9176f3edb72b832a4ac465cf02712288397832b5e8d", size = 32855, upload-time = "2025-09-08T01:34:59.186Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/41/45/1a4ed80516f02155c51f51e8cedb3c1902296743db0bbc66608a0db2814f/jsonschema_specifications-2025.9.1-py3-none-any.whl", hash = "sha256:98802fee3a11ee76ecaca44429fda8a41bff98b00a0f2838151b113f210cc6fe", size = 18437, upload-time = "2025-09-08T01:34:57.871Z" }, +] + +[[package]] +name = "jupyter-client" +version = "8.8.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jupyter-core" }, + { name = "python-dateutil" }, + { name = "pyzmq" }, + { name = "tornado" }, + { name = "traitlets" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/05/e4/ba649102a3bc3fbca54e7239fb924fd434c766f855693d86de0b1f2bec81/jupyter_client-8.8.0.tar.gz", hash = "sha256:d556811419a4f2d96c869af34e854e3f059b7cc2d6d01a9cd9c85c267691be3e", size = 348020, upload-time = "2026-01-08T13:55:47.938Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2d/0b/ceb7694d864abc0a047649aec263878acb9f792e1fec3e676f22dc9015e3/jupyter_client-8.8.0-py3-none-any.whl", hash = "sha256:f93a5b99c5e23a507b773d3a1136bd6e16c67883ccdbd9a829b0bbdb98cd7d7a", size = 107371, upload-time = "2026-01-08T13:55:45.562Z" }, +] + +[[package]] +name = "jupyter-core" +version = "5.9.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "platformdirs" }, + { name = "traitlets" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/02/49/9d1284d0dc65e2c757b74c6687b6d319b02f822ad039e5c512df9194d9dd/jupyter_core-5.9.1.tar.gz", hash = "sha256:4d09aaff303b9566c3ce657f580bd089ff5c91f5f89cf7d8846c3cdf465b5508", size = 89814, upload-time = "2025-10-16T19:19:18.444Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e7/e7/80988e32bf6f73919a113473a604f5a8f09094de312b9d52b79c2df7612b/jupyter_core-5.9.1-py3-none-any.whl", hash = "sha256:ebf87fdc6073d142e114c72c9e29a9d7ca03fad818c5d300ce2adc1fb0743407", size = 29032, upload-time = "2025-10-16T19:19:16.783Z" }, +] + +[[package]] +name = "jupyterlab-pygments" +version = "0.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/90/51/9187be60d989df97f5f0aba133fa54e7300f17616e065d1ada7d7646b6d6/jupyterlab_pygments-0.3.0.tar.gz", hash = "sha256:721aca4d9029252b11cfa9d185e5b5af4d54772bb8072f9b7036f4170054d35d", size = 512900, upload-time = "2023-11-23T09:26:37.44Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b1/dd/ead9d8ea85bf202d90cc513b533f9c363121c7792674f78e0d8a854b63b4/jupyterlab_pygments-0.3.0-py3-none-any.whl", hash = "sha256:841a89020971da1d8693f1a99997aefc5dc424bb1b251fd6322462a1b8842780", size = 15884, upload-time = "2023-11-23T09:26:34.325Z" }, +] + +[[package]] +name = "jupytext" +version = "1.19.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py" }, + { name = "mdit-py-plugins" }, + { name = "nbformat" }, + { name = "packaging" }, + { name = "pyyaml" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ef/2d/15624c3d9440d85a280ff13d2d23afd989802f25470ac59932f4fef6f0c6/jupytext-1.19.3.tar.gz", hash = "sha256:713c3ed4441afe0f31474d28ea2e6b61a268c04c40fd78e5ccfd7f7ac9e9f766", size = 4305350, upload-time = "2026-05-17T09:09:29.294Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/aa/ec/d9be3bd1db141e76b2f525c265f70e66edd30a51a3307d8edf0ef1909c54/jupytext-1.19.3-py3-none-any.whl", hash = "sha256:acf75492f80895ad8e664fd8db1708b617008dd0e71c341a1abc3d0d07310ed0", size = 170579, upload-time = "2026-05-17T09:09:27.478Z" }, +] + +[[package]] +name = "markdown" +version = "3.10.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/2b/f4/69fa6ed85ae003c2378ffa8f6d2e3234662abd02c10d216c0ba96081a238/markdown-3.10.2.tar.gz", hash = "sha256:994d51325d25ad8aa7ce4ebaec003febcce822c3f8c911e3b17c52f7f589f950", size = 368805, upload-time = "2026-02-09T14:57:26.942Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/de/1f/77fa3081e4f66ca3576c896ae5d31c3002ac6607f9747d2e3aa49227e464/markdown-3.10.2-py3-none-any.whl", hash = "sha256:e91464b71ae3ee7afd3017d9f358ef0baf158fd9a298db92f1d4761133824c36", size = 108180, upload-time = "2026-02-09T14:57:25.787Z" }, +] + [[package]] name = "markdown-it-py" version = "3.0.0" @@ -691,20 +888,174 @@ wheels = [ ] [[package]] -name = "myst-parser" -version = "4.0.0" +name = "mergedeep" +version = "1.3.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/3a/41/580bb4006e3ed0361b8151a01d324fb03f420815446c7def45d02f74c270/mergedeep-1.3.4.tar.gz", hash = "sha256:0096d52e9dad9939c3d975a774666af186eda617e6ca84df4c94dec30004f2a8", size = 4661, upload-time = "2021-02-05T18:55:30.623Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2c/19/04f9b178c2d8a15b076c8b5140708fa6ffc5601fb6f1e975537072df5b2a/mergedeep-1.3.4-py3-none-any.whl", hash = "sha256:70775750742b25c0d8f36c55aed03d24c3384d17c951b3175d898bd778ef0307", size = 6354, upload-time = "2021-02-05T18:55:29.583Z" }, +] + +[[package]] +name = "mistune" +version = "3.2.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "docutils" }, + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ca/84/620cc3f7e3adf6f5067e10f4dbae71295d8f9e16d5d3f9ef97c40f2f592c/mistune-3.2.1.tar.gz", hash = "sha256:7c8e5501d38bac1582e067e46c8343f17d57ea1aaa735823f3aba1fd59c88a28", size = 98003, upload-time = "2026-05-03T14:33:22.312Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2a/7f/a946aa4f8752b37102b41e64dca18a1976ac705c3a0d1dfe74d820a02552/mistune-3.2.1-py3-none-any.whl", hash = "sha256:78cdb0ba5e938053ccf63651b352508d2efa9411dc8810bfb05f2dc5140c0048", size = 53749, upload-time = "2026-05-03T14:33:20.551Z" }, +] + +[[package]] +name = "mkdocs" +version = "1.6.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "click" }, + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "ghp-import" }, { name = "jinja2" }, - { name = "markdown-it-py" }, - { name = "mdit-py-plugins" }, + { name = "markdown" }, + { name = "markupsafe" }, + { name = "mergedeep" }, + { name = "mkdocs-get-deps" }, + { name = "packaging" }, + { name = "pathspec" }, { name = "pyyaml" }, - { name = "sphinx" }, + { name = "pyyaml-env-tag" }, + { name = "watchdog" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/85/55/6d1741a1780e5e65038b74bce6689da15f620261c490c3511eb4c12bac4b/myst_parser-4.0.0.tar.gz", hash = "sha256:851c9dfb44e36e56d15d05e72f02b80da21a9e0d07cba96baf5e2d476bb91531", size = 93858, upload-time = "2024-08-05T14:02:45.798Z" } +sdist = { url = "https://files.pythonhosted.org/packages/bc/c6/bbd4f061bd16b378247f12953ffcb04786a618ce5e904b8c5a01a0309061/mkdocs-1.6.1.tar.gz", hash = "sha256:7b432f01d928c084353ab39c57282f29f92136665bdd6abf7c1ec8d822ef86f2", size = 3889159, upload-time = "2024-08-30T12:24:06.899Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ca/b4/b036f8fdb667587bb37df29dc6644681dd78b7a2a6321a34684b79412b28/myst_parser-4.0.0-py3-none-any.whl", hash = "sha256:b9317997552424448c6096c2558872fdb6f81d3ecb3a40ce84a7518798f3f28d", size = 84563, upload-time = "2024-08-05T14:02:43.767Z" }, + { url = "https://files.pythonhosted.org/packages/22/5b/dbc6a8cddc9cfa9c4971d59fb12bb8d42e161b7e7f8cc89e49137c5b279c/mkdocs-1.6.1-py3-none-any.whl", hash = "sha256:db91759624d1647f3f34aa0c3f327dd2601beae39a366d6e064c03468d35c20e", size = 3864451, upload-time = "2024-08-30T12:24:05.054Z" }, +] + +[[package]] +name = "mkdocs-autorefs" +version = "1.4.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown" }, + { name = "markupsafe" }, + { name = "mkdocs" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/52/c0/f641843de3f612a6b48253f39244165acff36657a91cc903633d456ae1ac/mkdocs_autorefs-1.4.4.tar.gz", hash = "sha256:d54a284f27a7346b9c38f1f852177940c222da508e66edc816a0fa55fc6da197", size = 56588, upload-time = "2026-02-10T15:23:55.105Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/28/de/a3e710469772c6a89595fc52816da05c1e164b4c866a89e3cb82fb1b67c5/mkdocs_autorefs-1.4.4-py3-none-any.whl", hash = "sha256:834ef5408d827071ad1bc69e0f39704fa34c7fc05bc8e1c72b227dfdc5c76089", size = 25530, upload-time = "2026-02-10T15:23:53.817Z" }, +] + +[[package]] +name = "mkdocs-get-deps" +version = "0.2.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "mergedeep" }, + { name = "platformdirs" }, + { name = "pyyaml" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ce/25/b3cccb187655b9393572bde9b09261d267c3bf2f2cdabe347673be5976a6/mkdocs_get_deps-0.2.2.tar.gz", hash = "sha256:8ee8d5f316cdbbb2834bc1df6e69c08fe769a83e040060de26d3c19fad3599a1", size = 11047, upload-time = "2026-03-10T02:46:33.632Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/88/29/744136411e785c4b0b744d5413e56555265939ab3a104c6a4b719dad33fd/mkdocs_get_deps-0.2.2-py3-none-any.whl", hash = "sha256:e7878cbeac04860b8b5e0ca31d3abad3df9411a75a32cde82f8e44b6c16ff650", size = 9555, upload-time = "2026-03-10T02:46:32.256Z" }, +] + +[[package]] +name = "mkdocs-jupyter" +version = "0.26.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "ipykernel" }, + { name = "jupytext" }, + { name = "mkdocs" }, + { name = "mkdocs-material" }, + { name = "nbconvert" }, + { name = "pygments" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/00/aa/f8d15409a9a3112486994a80d5a975694c7d145c4f8b5b484aeb383420ef/mkdocs_jupyter-0.26.3.tar.gz", hash = "sha256:e1e8bd48a1b96542e84e3028e3066112bac7b94d95ab69f8b91305c84003ca26", size = 1628353, upload-time = "2026-04-17T18:56:31.517Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/13/95/cf3f7fe4910cf0365fa8ea0c731f4b8a624d97cd76ea777913ac8d0868e2/mkdocs_jupyter-0.26.3-py3-none-any.whl", hash = "sha256:cd6644fb578131157194d750fd4d10fc2fd8f1e84e00036ee62df3b5b4b84c82", size = 1459740, upload-time = "2026-04-17T18:56:30.031Z" }, +] + +[[package]] +name = "mkdocs-material" +version = "9.7.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "babel" }, + { name = "backrefs" }, + { name = "colorama" }, + { name = "jinja2" }, + { name = "markdown" }, + { name = "mkdocs" }, + { name = "mkdocs-material-extensions" }, + { name = "paginate" }, + { name = "pygments" }, + { name = "pymdown-extensions" }, + { name = "requests" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/45/29/6d2bcf41ae40802c4beda2432396fff97b8456fb496371d1bc7aad6512ec/mkdocs_material-9.7.6.tar.gz", hash = "sha256:00bdde50574f776d328b1862fe65daeaf581ec309bd150f7bff345a098c64a69", size = 4097959, upload-time = "2026-03-19T15:41:58.161Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2c/01/bc663630c510822c95c47a66af9fa7a443c295b47d5f041e5e6ae62ef659/mkdocs_material-9.7.6-py3-none-any.whl", hash = "sha256:71b84353921b8ea1ba84fe11c50912cc512da8fe0881038fcc9a0761c0e635ba", size = 9305470, upload-time = "2026-03-19T15:41:55.217Z" }, +] + +[[package]] +name = "mkdocs-material-extensions" +version = "1.3.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/79/9b/9b4c96d6593b2a541e1cb8b34899a6d021d208bb357042823d4d2cabdbe7/mkdocs_material_extensions-1.3.1.tar.gz", hash = "sha256:10c9511cea88f568257f960358a467d12b970e1f7b2c0e5fb2bb48cab1928443", size = 11847, upload-time = "2023-11-22T19:09:45.208Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5b/54/662a4743aa81d9582ee9339d4ffa3c8fd40a4965e033d77b9da9774d3960/mkdocs_material_extensions-1.3.1-py3-none-any.whl", hash = "sha256:adff8b62700b25cb77b53358dad940f3ef973dd6db797907c49e3c2ef3ab4e31", size = 8728, upload-time = "2023-11-22T19:09:43.465Z" }, +] + +[[package]] +name = "mkdocs-redirects" +version = "1.2.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "mkdocs" }, + { name = "properdocs" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/73/25/49725f78ca5d3026b09973f7a2b3a8b179cc2e8c15e43d5a13bc79f6b274/mkdocs_redirects-1.2.3.tar.gz", hash = "sha256:5e980330999299729a2d6a125347d1af78023d68a23681a4de3053ce7dfe2e51", size = 7712, upload-time = "2026-03-28T13:57:41.766Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c6/90/871b1cddc01d2ba1637b858eeeabc2e3013dc8df591306b5567b98ef0870/mkdocs_redirects-1.2.3-py3-none-any.whl", hash = "sha256:ec7312fff462d03ec16395d0c001006a418f8d0c21cdf2b47ff11cf839dc3ce0", size = 6245, upload-time = "2026-03-28T13:57:40.466Z" }, +] + +[[package]] +name = "mkdocstrings" +version = "1.0.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jinja2" }, + { name = "markdown" }, + { name = "markupsafe" }, + { name = "mkdocs" }, + { name = "mkdocs-autorefs" }, + { name = "pymdown-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1d/5d/f888d4d3eb31359b327bc9b17a212d6ef03fe0b0682fbb3fc2cb849fb12b/mkdocstrings-1.0.4.tar.gz", hash = "sha256:3969a6515b77db65fd097b53c1b7aa4ae840bd71a2ee62a6a3e89503446d7172", size = 100088, upload-time = "2026-04-15T09:16:53.376Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/6e/94/be70f8ee9c45f2f62b39a1f0e9303bc20e138a8f3b8e50ffd89498e177e1/mkdocstrings-1.0.4-py3-none-any.whl", hash = "sha256:63464b4b29053514f32a1dbbf604e52876d5e638111b0c295ab7ed3cac73ca9b", size = 35560, upload-time = "2026-04-15T09:16:51.436Z" }, +] + +[package.optional-dependencies] +python = [ + { name = "mkdocstrings-python" }, +] + +[[package]] +name = "mkdocstrings-python" +version = "2.0.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "griffelib" }, + { name = "mkdocs-autorefs" }, + { name = "mkdocstrings" }, + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a4/b4/5fed370d8ebd96e4e399460a7146ae989263f16588b05a6facd6dbd51e60/mkdocstrings_python-2.0.4.tar.gz", hash = "sha256:58c73c5d358e64e9b1673447663f4a2f8a8941e392e225fc0a0c893758cc452f", size = 199219, upload-time = "2026-06-05T08:13:01.819Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5e/e3/00ec594aef5f55522e6d373bc2ac53e53a8f5e9ae32f2d6854b0de4270f3/mkdocstrings_python-2.0.4-py3-none-any.whl", hash = "sha256:fd87c173e1e719a85997b6d4f852cdc55f36710e0ed08da3a7bd9abe79c9db00", size = 104790, upload-time = "2026-06-05T08:13:00.393Z" }, ] [[package]] @@ -777,6 +1128,70 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b9/e5/c740ea047b5ada76175327360d0406ae283159cb1745cbcb51443d90d53b/nanoarrow-0.8.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:8c5529abc4e75b7764ffc6d2fbabd0c676f75ca2ece71a8671c4724207cfb697", size = 591889, upload-time = "2026-02-10T03:33:58.891Z" }, ] +[[package]] +name = "nbclient" +version = "0.11.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jupyter-client" }, + { name = "jupyter-core" }, + { name = "nbformat" }, + { name = "traitlets" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/28/a5/b3bae4b590c0cbcada2c63a34f7580024e834a8ba213e949a2f906705787/nbclient-0.11.0.tar.gz", hash = "sha256:04a134a5b087f2c5887f228aca155db50169b8cd9334dee6942c8e927e56081a", size = 62535, upload-time = "2026-06-05T07:52:41.746Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/36/c9/94d73e5a01c5b926c3fa2496e97d7a8dc28ed5a77c0b2ed712f1a62e6694/nbclient-0.11.0-py3-none-any.whl", hash = "sha256:ef7fa0d59d6e1d41103933d8a445a18d5de860ca6b613b87b8574accdb3c2895", size = 25288, upload-time = "2026-06-05T07:52:40.115Z" }, +] + +[[package]] +name = "nbconvert" +version = "7.17.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "beautifulsoup4" }, + { name = "bleach", extra = ["css"] }, + { name = "defusedxml" }, + { name = "jinja2" }, + { name = "jupyter-core" }, + { name = "jupyterlab-pygments" }, + { name = "markupsafe" }, + { name = "mistune" }, + { name = "nbclient" }, + { name = "nbformat" }, + { name = "packaging" }, + { name = "pandocfilters" }, + { name = "pygments" }, + { name = "traitlets" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/01/b1/708e53fe2e429c103c6e6e159106bcf0357ac41aa4c28772bd8402339051/nbconvert-7.17.1.tar.gz", hash = "sha256:34d0d0a7e73ce3cbab6c5aae8f4f468797280b01fd8bd2ca746da8569eddd7d2", size = 865311, upload-time = "2026-04-08T00:44:14.914Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/67/f8/bb0a9d5f46819c821dc1f004aa2cc29b1d91453297dbf5ff20470f00f193/nbconvert-7.17.1-py3-none-any.whl", hash = "sha256:aa85c087b435e7bf1ffd03319f658e285f2b89eccab33bc1ba7025495ab3e7c8", size = 261927, upload-time = "2026-04-08T00:44:12.845Z" }, +] + +[[package]] +name = "nbformat" +version = "5.10.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "fastjsonschema" }, + { name = "jsonschema" }, + { name = "jupyter-core" }, + { name = "traitlets" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/6d/fd/91545e604bc3dad7dca9ed03284086039b294c6b3d75c0d2fa45f9e9caf3/nbformat-5.10.4.tar.gz", hash = "sha256:322168b14f937a5d11362988ecac2a4952d3d8e3a2cbeb2319584631226d5b3a", size = 142749, upload-time = "2024-04-04T11:20:37.371Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a9/82/0340caa499416c78e5d8f5f05947ae4bc3cba53c9f038ab6e9ed964e22f1/nbformat-5.10.4-py3-none-any.whl", hash = "sha256:3b48d6c8fbca4b299bf3982ea7db1af21580e4fec269ad087b9e81588891200b", size = 78454, upload-time = "2024-04-04T11:20:34.895Z" }, +] + +[[package]] +name = "nest-asyncio" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/83/f8/51569ac65d696c8ecbee95938f89d4abf00f47d58d48f6fbabfe8f0baefe/nest_asyncio-1.6.0.tar.gz", hash = "sha256:6f172d5449aca15afd6c646851f4e31e02c598d553a667e38cafa997cfec55fe", size = 7418, upload-time = "2024-01-21T14:25:19.227Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/c4/c2971a3ba4c6103a3d10c4b0f24f461ddc027f0f09763220cf35ca1401b3/nest_asyncio-1.6.0-py3-none-any.whl", hash = "sha256:87af6efd6b5e897c81050477ef65c62e2b2f35d51703cae01aff2905b1852e1c", size = 5195, upload-time = "2024-01-21T14:25:17.223Z" }, +] + [[package]] name = "nodeenv" version = "1.9.1" @@ -946,6 +1361,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl", hash = "sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759", size = 65451, upload-time = "2024-11-08T09:47:44.722Z" }, ] +[[package]] +name = "paginate" +version = "0.5.7" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ec/46/68dde5b6bc00c1296ec6466ab27dddede6aec9af1b99090e1107091b3b84/paginate-0.5.7.tar.gz", hash = "sha256:22bd083ab41e1a8b4f3690544afb2c60c25e5c9a63a30fa2f483f6c60c8e5945", size = 19252, upload-time = "2024-08-25T14:17:24.139Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/90/96/04b8e52da071d28f5e21a805b19cb9390aa17a47462ac87f5e2696b9566d/paginate-0.5.7-py2.py3-none-any.whl", hash = "sha256:b885e2af73abcf01d9559fd5216b57ef722f8c42affbb63942377668e35c7591", size = 13746, upload-time = "2024-08-25T14:17:22.55Z" }, +] + [[package]] name = "pandas" version = "2.2.3" @@ -995,6 +1419,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ab/5f/b38085618b950b79d2d9164a711c52b10aefc0ae6833b96f626b7021b2ed/pandas-2.2.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:ad5b65698ab28ed8d7f18790a0dc58005c7629f227be9ecc1072aa74c0c1d43a", size = 13098436, upload-time = "2024-09-20T13:09:48.112Z" }, ] +[[package]] +name = "pandocfilters" +version = "1.5.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/70/6f/3dd4940bbe001c06a65f88e36bad298bc7a0de5036115639926b0c5c0458/pandocfilters-1.5.1.tar.gz", hash = "sha256:002b4a555ee4ebc03f8b66307e287fa492e4a77b4ea14d3f934328297bb4939e", size = 8454, upload-time = "2024-01-18T20:08:13.726Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ef/af/4fbc8cab944db5d21b7e2a5b8e9211a03a79852b1157e2c102fcc61ac440/pandocfilters-1.5.1-py2.py3-none-any.whl", hash = "sha256:93be382804a9cdb0a7267585f157e5d1731bbe5545a85b268d6f5fe6232de2bc", size = 8663, upload-time = "2024-01-18T20:08:11.28Z" }, +] + [[package]] name = "parso" version = "0.8.4" @@ -1004,6 +1437,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c6/ac/dac4a63f978e4dcb3c6d3a78c4d8e0192a113d288502a1216950c41b1027/parso-0.8.4-py2.py3-none-any.whl", hash = "sha256:a418670a20291dacd2dddc80c377c5c3791378ee1e8d12bffc35420643d43f18", size = 103650, upload-time = "2024-04-05T09:43:53.299Z" }, ] +[[package]] +name = "pathspec" +version = "1.1.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5a/82/42f767fc1c1143d6fd36efb827202a2d997a375e160a71eb2888a925aac1/pathspec-1.1.1.tar.gz", hash = "sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a", size = 135180, upload-time = "2026-04-27T01:46:08.907Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f1/d9/7fb5aa316bc299258e68c73ba3bddbc499654a07f151cba08f6153988714/pathspec-1.1.1-py3-none-any.whl", hash = "sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189", size = 57328, upload-time = "2026-04-27T01:46:07.06Z" }, +] + [[package]] name = "pexpect" version = "4.9.0" @@ -1071,6 +1513,57 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a9/6a/fd08d94654f7e67c52ca30523a178b3f8ccc4237fce4be90d39c938a831a/prompt_toolkit-3.0.48-py3-none-any.whl", hash = "sha256:f49a827f90062e411f1ce1f854f2aedb3c23353244f8108b89283587397ac10e", size = 386595, upload-time = "2024-09-25T10:20:53.932Z" }, ] +[[package]] +name = "properdocs" +version = "1.6.7" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "click" }, + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "ghp-import" }, + { name = "jinja2" }, + { name = "markdown" }, + { name = "markupsafe" }, + { name = "packaging" }, + { name = "pathspec" }, + { name = "platformdirs" }, + { name = "pyyaml" }, + { name = "pyyaml-env-tag" }, + { name = "watchdog" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ec/29/f27a4e1eddf72ed3db6e47818fbafe6debbf09fd7051f9c1a007239b46ef/properdocs-1.6.7.tar.gz", hash = "sha256:adc7b16e562890af0e098a7e5b02e3a81c20894a87d6a28d345c9300de73c26e", size = 276141, upload-time = "2026-03-20T20:07:48.167Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bd/4d/fc923f5c85318ee8cc903566dc4e0ebe41b2dfc1d2ecf5546db232397ed6/properdocs-1.6.7-py3-none-any.whl", hash = "sha256:6fa0cfa2e01bf338f684892c8a506cf70ea88ae7f3479c933b6fa20168101cbd", size = 225406, upload-time = "2026-03-20T20:07:46.875Z" }, +] + +[[package]] +name = "psutil" +version = "7.2.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/aa/c6/d1ddf4abb55e93cebc4f2ed8b5d6dbad109ecb8d63748dd2b20ab5e57ebe/psutil-7.2.2.tar.gz", hash = "sha256:0746f5f8d406af344fd547f1c8daa5f5c33dbc293bb8d6a16d80b4bb88f59372", size = 493740, upload-time = "2026-01-28T18:14:54.428Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/08/510cbdb69c25a96f4ae523f733cdc963ae654904e8db864c07585ef99875/psutil-7.2.2-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:2edccc433cbfa046b980b0df0171cd25bcaeb3a68fe9022db0979e7aa74a826b", size = 130595, upload-time = "2026-01-28T18:14:57.293Z" }, + { url = "https://files.pythonhosted.org/packages/d6/f5/97baea3fe7a5a9af7436301f85490905379b1c6f2dd51fe3ecf24b4c5fbf/psutil-7.2.2-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:e78c8603dcd9a04c7364f1a3e670cea95d51ee865e4efb3556a3a63adef958ea", size = 131082, upload-time = "2026-01-28T18:14:59.732Z" }, + { url = "https://files.pythonhosted.org/packages/37/d6/246513fbf9fa174af531f28412297dd05241d97a75911ac8febefa1a53c6/psutil-7.2.2-cp313-cp313t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1a571f2330c966c62aeda00dd24620425d4b0cc86881c89861fbc04549e5dc63", size = 181476, upload-time = "2026-01-28T18:15:01.884Z" }, + { url = "https://files.pythonhosted.org/packages/b8/b5/9182c9af3836cca61696dabe4fd1304e17bc56cb62f17439e1154f225dd3/psutil-7.2.2-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:917e891983ca3c1887b4ef36447b1e0873e70c933afc831c6b6da078ba474312", size = 184062, upload-time = "2026-01-28T18:15:04.436Z" }, + { url = "https://files.pythonhosted.org/packages/16/ba/0756dca669f5a9300d0cbcbfae9a4c30e446dfc7440ffe43ded5724bfd93/psutil-7.2.2-cp313-cp313t-win_amd64.whl", hash = "sha256:ab486563df44c17f5173621c7b198955bd6b613fb87c71c161f827d3fb149a9b", size = 139893, upload-time = "2026-01-28T18:15:06.378Z" }, + { url = "https://files.pythonhosted.org/packages/1c/61/8fa0e26f33623b49949346de05ec1ddaad02ed8ba64af45f40a147dbfa97/psutil-7.2.2-cp313-cp313t-win_arm64.whl", hash = "sha256:ae0aefdd8796a7737eccea863f80f81e468a1e4cf14d926bd9b6f5f2d5f90ca9", size = 135589, upload-time = "2026-01-28T18:15:08.03Z" }, + { url = "https://files.pythonhosted.org/packages/81/69/ef179ab5ca24f32acc1dac0c247fd6a13b501fd5534dbae0e05a1c48b66d/psutil-7.2.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:eed63d3b4d62449571547b60578c5b2c4bcccc5387148db46e0c2313dad0ee00", size = 130664, upload-time = "2026-01-28T18:15:09.469Z" }, + { url = "https://files.pythonhosted.org/packages/7b/64/665248b557a236d3fa9efc378d60d95ef56dd0a490c2cd37dafc7660d4a9/psutil-7.2.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7b6d09433a10592ce39b13d7be5a54fbac1d1228ed29abc880fb23df7cb694c9", size = 131087, upload-time = "2026-01-28T18:15:11.724Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2e/e6782744700d6759ebce3043dcfa661fb61e2fb752b91cdeae9af12c2178/psutil-7.2.2-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1fa4ecf83bcdf6e6c8f4449aff98eefb5d0604bf88cb883d7da3d8d2d909546a", size = 182383, upload-time = "2026-01-28T18:15:13.445Z" }, + { url = "https://files.pythonhosted.org/packages/57/49/0a41cefd10cb7505cdc04dab3eacf24c0c2cb158a998b8c7b1d27ee2c1f5/psutil-7.2.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e452c464a02e7dc7822a05d25db4cde564444a67e58539a00f929c51eddda0cf", size = 185210, upload-time = "2026-01-28T18:15:16.002Z" }, + { url = "https://files.pythonhosted.org/packages/dd/2c/ff9bfb544f283ba5f83ba725a3c5fec6d6b10b8f27ac1dc641c473dc390d/psutil-7.2.2-cp314-cp314t-win_amd64.whl", hash = "sha256:c7663d4e37f13e884d13994247449e9f8f574bc4655d509c3b95e9ec9e2b9dc1", size = 141228, upload-time = "2026-01-28T18:15:18.385Z" }, + { url = "https://files.pythonhosted.org/packages/f2/fc/f8d9c31db14fcec13748d373e668bc3bed94d9077dbc17fb0eebc073233c/psutil-7.2.2-cp314-cp314t-win_arm64.whl", hash = "sha256:11fe5a4f613759764e79c65cf11ebdf26e33d6dd34336f8a337aa2996d71c841", size = 136284, upload-time = "2026-01-28T18:15:19.912Z" }, + { url = "https://files.pythonhosted.org/packages/e7/36/5ee6e05c9bd427237b11b3937ad82bb8ad2752d72c6969314590dd0c2f6e/psutil-7.2.2-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:ed0cace939114f62738d808fdcecd4c869222507e266e574799e9c0faa17d486", size = 129090, upload-time = "2026-01-28T18:15:22.168Z" }, + { url = "https://files.pythonhosted.org/packages/80/c4/f5af4c1ca8c1eeb2e92ccca14ce8effdeec651d5ab6053c589b074eda6e1/psutil-7.2.2-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:1a7b04c10f32cc88ab39cbf606e117fd74721c831c98a27dc04578deb0c16979", size = 129859, upload-time = "2026-01-28T18:15:23.795Z" }, + { url = "https://files.pythonhosted.org/packages/b5/70/5d8df3b09e25bce090399cf48e452d25c935ab72dad19406c77f4e828045/psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:076a2d2f923fd4821644f5ba89f059523da90dc9014e85f8e45a5774ca5bc6f9", size = 155560, upload-time = "2026-01-28T18:15:25.976Z" }, + { url = "https://files.pythonhosted.org/packages/63/65/37648c0c158dc222aba51c089eb3bdfa238e621674dc42d48706e639204f/psutil-7.2.2-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b0726cecd84f9474419d67252add4ac0cd9811b04d61123054b9fb6f57df6e9e", size = 156997, upload-time = "2026-01-28T18:15:27.794Z" }, + { url = "https://files.pythonhosted.org/packages/8e/13/125093eadae863ce03c6ffdbae9929430d116a246ef69866dad94da3bfbc/psutil-7.2.2-cp36-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:fd04ef36b4a6d599bbdb225dd1d3f51e00105f6d48a28f006da7f9822f2606d8", size = 148972, upload-time = "2026-01-28T18:15:29.342Z" }, + { url = "https://files.pythonhosted.org/packages/04/78/0acd37ca84ce3ddffaa92ef0f571e073faa6d8ff1f0559ab1272188ea2be/psutil-7.2.2-cp36-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b58fabe35e80b264a4e3bb23e6b96f9e45a3df7fb7eed419ac0e5947c61e47cc", size = 148266, upload-time = "2026-01-28T18:15:31.597Z" }, + { url = "https://files.pythonhosted.org/packages/b4/90/e2159492b5426be0c1fef7acba807a03511f97c5f86b3caeda6ad92351a7/psutil-7.2.2-cp37-abi3-win_amd64.whl", hash = "sha256:eb7e81434c8d223ec4a219b5fc1c47d0417b12be7ea866e24fb5ad6e84b3d988", size = 137737, upload-time = "2026-01-28T18:15:33.849Z" }, + { url = "https://files.pythonhosted.org/packages/8c/c7/7bb2e321574b10df20cbde462a94e2b71d05f9bbda251ef27d104668306a/psutil-7.2.2-cp37-abi3-win_arm64.whl", hash = "sha256:8c233660f575a5a89e6d4cb65d9f938126312bca76d8fe087b947b3a1aaac9ee", size = 134617, upload-time = "2026-01-28T18:15:36.514Z" }, +] + [[package]] name = "ptyprocess" version = "0.7.0" @@ -1155,24 +1648,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/13/a3/a812df4e2dd5696d1f351d58b8fe16a405b234ad2886a0dab9183fb78109/pycparser-2.22-py3-none-any.whl", hash = "sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc", size = 117552, upload-time = "2024-03-30T13:22:20.476Z" }, ] -[[package]] -name = "pydata-sphinx-theme" -version = "0.16.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "accessible-pygments" }, - { name = "babel" }, - { name = "beautifulsoup4" }, - { name = "docutils" }, - { name = "pygments" }, - { name = "sphinx" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/00/20/bb50f9de3a6de69e6abd6b087b52fa2418a0418b19597601605f855ad044/pydata_sphinx_theme-0.16.1.tar.gz", hash = "sha256:a08b7f0b7f70387219dc659bff0893a7554d5eb39b59d3b8ef37b8401b7642d7", size = 2412693, upload-time = "2024-12-17T10:53:39.537Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e2/0d/8ba33fa83a7dcde13eb3c1c2a0c1cc29950a048bfed6d9b0d8b6bd710b4c/pydata_sphinx_theme-0.16.1-py3-none-any.whl", hash = "sha256:225331e8ac4b32682c18fcac5a57a6f717c4e632cea5dd0e247b55155faeccde", size = 6723264, upload-time = "2024-12-17T10:53:35.645Z" }, -] - [[package]] name = "pygithub" version = "2.5.0" @@ -1213,6 +1688,19 @@ crypto = [ { name = "cryptography" }, ] +[[package]] +name = "pymdown-extensions" +version = "10.21.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown" }, + { name = "pyyaml" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9e/26/d1015444da4d952a1ca487a236b522eb979766f0295a0bd0c5fc089989a9/pymdown_extensions-10.21.3.tar.gz", hash = "sha256:72cfcf55f07aea0d4af2c4f11dd4e52466ddfb1bb819673146398e0bd3a77354", size = 854140, upload-time = "2026-05-13T12:57:32.267Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7e/85/545a951eecc270fcd688288c600017e2050a1aacb56c711d208586d3e470/pymdown_extensions-10.21.3-py3-none-any.whl", hash = "sha256:d7a5d08014fc571e80ca21dd6f854e31f94c489800350564d55d15b3c41e76b6", size = 269002, upload-time = "2026-05-13T12:57:30.296Z" }, +] + [[package]] name = "pynacl" version = "1.5.0" @@ -1359,6 +1847,106 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, ] +[[package]] +name = "pyyaml-env-tag" +version = "1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pyyaml" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/eb/2e/79c822141bfd05a853236b504869ebc6b70159afc570e1d5a20641782eaa/pyyaml_env_tag-1.1.tar.gz", hash = "sha256:2eb38b75a2d21ee0475d6d97ec19c63287a7e140231e4214969d0eac923cd7ff", size = 5737, upload-time = "2025-05-13T15:24:01.64Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/11/432f32f8097b03e3cd5fe57e88efb685d964e2e5178a48ed61e841f7fdce/pyyaml_env_tag-1.1-py3-none-any.whl", hash = "sha256:17109e1a528561e32f026364712fee1264bc2ea6715120891174ed1b980d2e04", size = 4722, upload-time = "2025-05-13T15:23:59.629Z" }, +] + +[[package]] +name = "pyzmq" +version = "27.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cffi", marker = "implementation_name == 'pypy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/04/0b/3c9baedbdf613ecaa7aa07027780b8867f57b6293b6ee50de316c9f3222b/pyzmq-27.1.0.tar.gz", hash = "sha256:ac0765e3d44455adb6ddbf4417dcce460fc40a05978c08efdf2948072f6db540", size = 281750, upload-time = "2025-09-08T23:10:18.157Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/67/b9/52aa9ec2867528b54f1e60846728d8b4d84726630874fee3a91e66c7df81/pyzmq-27.1.0-cp310-cp310-macosx_10_15_universal2.whl", hash = "sha256:508e23ec9bc44c0005c4946ea013d9317ae00ac67778bd47519fdf5a0e930ff4", size = 1329850, upload-time = "2025-09-08T23:07:26.274Z" }, + { url = "https://files.pythonhosted.org/packages/99/64/5653e7b7425b169f994835a2b2abf9486264401fdef18df91ddae47ce2cc/pyzmq-27.1.0-cp310-cp310-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:507b6f430bdcf0ee48c0d30e734ea89ce5567fd7b8a0f0044a369c176aa44556", size = 906380, upload-time = "2025-09-08T23:07:29.78Z" }, + { url = "https://files.pythonhosted.org/packages/73/78/7d713284dbe022f6440e391bd1f3c48d9185673878034cfb3939cdf333b2/pyzmq-27.1.0-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bf7b38f9fd7b81cb6d9391b2946382c8237fd814075c6aa9c3b746d53076023b", size = 666421, upload-time = "2025-09-08T23:07:31.263Z" }, + { url = "https://files.pythonhosted.org/packages/30/76/8f099f9d6482450428b17c4d6b241281af7ce6a9de8149ca8c1c649f6792/pyzmq-27.1.0-cp310-cp310-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:03ff0b279b40d687691a6217c12242ee71f0fba28bf8626ff50e3ef0f4410e1e", size = 854149, upload-time = "2025-09-08T23:07:33.17Z" }, + { url = "https://files.pythonhosted.org/packages/59/f0/37fbfff06c68016019043897e4c969ceab18bde46cd2aca89821fcf4fb2e/pyzmq-27.1.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:677e744fee605753eac48198b15a2124016c009a11056f93807000ab11ce6526", size = 1655070, upload-time = "2025-09-08T23:07:35.205Z" }, + { url = "https://files.pythonhosted.org/packages/47/14/7254be73f7a8edc3587609554fcaa7bfd30649bf89cd260e4487ca70fdaa/pyzmq-27.1.0-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:dd2fec2b13137416a1c5648b7009499bcc8fea78154cd888855fa32514f3dad1", size = 2033441, upload-time = "2025-09-08T23:07:37.432Z" }, + { url = "https://files.pythonhosted.org/packages/22/dc/49f2be26c6f86f347e796a4d99b19167fc94503f0af3fd010ad262158822/pyzmq-27.1.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:08e90bb4b57603b84eab1d0ca05b3bbb10f60c1839dc471fc1c9e1507bef3386", size = 1891529, upload-time = "2025-09-08T23:07:39.047Z" }, + { url = "https://files.pythonhosted.org/packages/a3/3e/154fb963ae25be70c0064ce97776c937ecc7d8b0259f22858154a9999769/pyzmq-27.1.0-cp310-cp310-win32.whl", hash = "sha256:a5b42d7a0658b515319148875fcb782bbf118dd41c671b62dae33666c2213bda", size = 567276, upload-time = "2025-09-08T23:07:40.695Z" }, + { url = "https://files.pythonhosted.org/packages/62/b2/f4ab56c8c595abcb26b2be5fd9fa9e6899c1e5ad54964e93ae8bb35482be/pyzmq-27.1.0-cp310-cp310-win_amd64.whl", hash = "sha256:c0bb87227430ee3aefcc0ade2088100e528d5d3298a0a715a64f3d04c60ba02f", size = 632208, upload-time = "2025-09-08T23:07:42.298Z" }, + { url = "https://files.pythonhosted.org/packages/3b/e3/be2cc7ab8332bdac0522fdb64c17b1b6241a795bee02e0196636ec5beb79/pyzmq-27.1.0-cp310-cp310-win_arm64.whl", hash = "sha256:9a916f76c2ab8d045b19f2286851a38e9ac94ea91faf65bd64735924522a8b32", size = 559766, upload-time = "2025-09-08T23:07:43.869Z" }, + { url = "https://files.pythonhosted.org/packages/06/5d/305323ba86b284e6fcb0d842d6adaa2999035f70f8c38a9b6d21ad28c3d4/pyzmq-27.1.0-cp311-cp311-macosx_10_15_universal2.whl", hash = "sha256:226b091818d461a3bef763805e75685e478ac17e9008f49fce2d3e52b3d58b86", size = 1333328, upload-time = "2025-09-08T23:07:45.946Z" }, + { url = "https://files.pythonhosted.org/packages/bd/a0/fc7e78a23748ad5443ac3275943457e8452da67fda347e05260261108cbc/pyzmq-27.1.0-cp311-cp311-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:0790a0161c281ca9723f804871b4027f2e8b5a528d357c8952d08cd1a9c15581", size = 908803, upload-time = "2025-09-08T23:07:47.551Z" }, + { url = "https://files.pythonhosted.org/packages/7e/22/37d15eb05f3bdfa4abea6f6d96eb3bb58585fbd3e4e0ded4e743bc650c97/pyzmq-27.1.0-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c895a6f35476b0c3a54e3eb6ccf41bf3018de937016e6e18748317f25d4e925f", size = 668836, upload-time = "2025-09-08T23:07:49.436Z" }, + { url = "https://files.pythonhosted.org/packages/b1/c4/2a6fe5111a01005fc7af3878259ce17684fabb8852815eda6225620f3c59/pyzmq-27.1.0-cp311-cp311-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5bbf8d3630bf96550b3be8e1fc0fea5cbdc8d5466c1192887bd94869da17a63e", size = 857038, upload-time = "2025-09-08T23:07:51.234Z" }, + { url = "https://files.pythonhosted.org/packages/cb/eb/bfdcb41d0db9cd233d6fb22dc131583774135505ada800ebf14dfb0a7c40/pyzmq-27.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:15c8bd0fe0dabf808e2d7a681398c4e5ded70a551ab47482067a572c054c8e2e", size = 1657531, upload-time = "2025-09-08T23:07:52.795Z" }, + { url = "https://files.pythonhosted.org/packages/ab/21/e3180ca269ed4a0de5c34417dfe71a8ae80421198be83ee619a8a485b0c7/pyzmq-27.1.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:bafcb3dd171b4ae9f19ee6380dfc71ce0390fefaf26b504c0e5f628d7c8c54f2", size = 2034786, upload-time = "2025-09-08T23:07:55.047Z" }, + { url = "https://files.pythonhosted.org/packages/3b/b1/5e21d0b517434b7f33588ff76c177c5a167858cc38ef740608898cd329f2/pyzmq-27.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:e829529fcaa09937189178115c49c504e69289abd39967cd8a4c215761373394", size = 1894220, upload-time = "2025-09-08T23:07:57.172Z" }, + { url = "https://files.pythonhosted.org/packages/03/f2/44913a6ff6941905efc24a1acf3d3cb6146b636c546c7406c38c49c403d4/pyzmq-27.1.0-cp311-cp311-win32.whl", hash = "sha256:6df079c47d5902af6db298ec92151db82ecb557af663098b92f2508c398bb54f", size = 567155, upload-time = "2025-09-08T23:07:59.05Z" }, + { url = "https://files.pythonhosted.org/packages/23/6d/d8d92a0eb270a925c9b4dd039c0b4dc10abc2fcbc48331788824ef113935/pyzmq-27.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:190cbf120fbc0fc4957b56866830def56628934a9d112aec0e2507aa6a032b97", size = 633428, upload-time = "2025-09-08T23:08:00.663Z" }, + { url = "https://files.pythonhosted.org/packages/ae/14/01afebc96c5abbbd713ecfc7469cfb1bc801c819a74ed5c9fad9a48801cb/pyzmq-27.1.0-cp311-cp311-win_arm64.whl", hash = "sha256:eca6b47df11a132d1745eb3b5b5e557a7dae2c303277aa0e69c6ba91b8736e07", size = 559497, upload-time = "2025-09-08T23:08:02.15Z" }, + { url = "https://files.pythonhosted.org/packages/92/e7/038aab64a946d535901103da16b953c8c9cc9c961dadcbf3609ed6428d23/pyzmq-27.1.0-cp312-abi3-macosx_10_15_universal2.whl", hash = "sha256:452631b640340c928fa343801b0d07eb0c3789a5ffa843f6e1a9cee0ba4eb4fc", size = 1306279, upload-time = "2025-09-08T23:08:03.807Z" }, + { url = "https://files.pythonhosted.org/packages/e8/5e/c3c49fdd0f535ef45eefcc16934648e9e59dace4a37ee88fc53f6cd8e641/pyzmq-27.1.0-cp312-abi3-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:1c179799b118e554b66da67d88ed66cd37a169f1f23b5d9f0a231b4e8d44a113", size = 895645, upload-time = "2025-09-08T23:08:05.301Z" }, + { url = "https://files.pythonhosted.org/packages/f8/e5/b0b2504cb4e903a74dcf1ebae157f9e20ebb6ea76095f6cfffea28c42ecd/pyzmq-27.1.0-cp312-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3837439b7f99e60312f0c926a6ad437b067356dc2bc2ec96eb395fd0fe804233", size = 652574, upload-time = "2025-09-08T23:08:06.828Z" }, + { url = "https://files.pythonhosted.org/packages/f8/9b/c108cdb55560eaf253f0cbdb61b29971e9fb34d9c3499b0e96e4e60ed8a5/pyzmq-27.1.0-cp312-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:43ad9a73e3da1fab5b0e7e13402f0b2fb934ae1c876c51d0afff0e7c052eca31", size = 840995, upload-time = "2025-09-08T23:08:08.396Z" }, + { url = "https://files.pythonhosted.org/packages/c2/bb/b79798ca177b9eb0825b4c9998c6af8cd2a7f15a6a1a4272c1d1a21d382f/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:0de3028d69d4cdc475bfe47a6128eb38d8bc0e8f4d69646adfbcd840facbac28", size = 1642070, upload-time = "2025-09-08T23:08:09.989Z" }, + { url = "https://files.pythonhosted.org/packages/9c/80/2df2e7977c4ede24c79ae39dcef3899bfc5f34d1ca7a5b24f182c9b7a9ca/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_i686.whl", hash = "sha256:cf44a7763aea9298c0aa7dbf859f87ed7012de8bda0f3977b6fb1d96745df856", size = 2021121, upload-time = "2025-09-08T23:08:11.907Z" }, + { url = "https://files.pythonhosted.org/packages/46/bd/2d45ad24f5f5ae7e8d01525eb76786fa7557136555cac7d929880519e33a/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:f30f395a9e6fbca195400ce833c731e7b64c3919aa481af4d88c3759e0cb7496", size = 1878550, upload-time = "2025-09-08T23:08:13.513Z" }, + { url = "https://files.pythonhosted.org/packages/e6/2f/104c0a3c778d7c2ab8190e9db4f62f0b6957b53c9d87db77c284b69f33ea/pyzmq-27.1.0-cp312-abi3-win32.whl", hash = "sha256:250e5436a4ba13885494412b3da5d518cd0d3a278a1ae640e113c073a5f88edd", size = 559184, upload-time = "2025-09-08T23:08:15.163Z" }, + { url = "https://files.pythonhosted.org/packages/fc/7f/a21b20d577e4100c6a41795842028235998a643b1ad406a6d4163ea8f53e/pyzmq-27.1.0-cp312-abi3-win_amd64.whl", hash = "sha256:9ce490cf1d2ca2ad84733aa1d69ce6855372cb5ce9223802450c9b2a7cba0ccf", size = 619480, upload-time = "2025-09-08T23:08:17.192Z" }, + { url = "https://files.pythonhosted.org/packages/78/c2/c012beae5f76b72f007a9e91ee9401cb88c51d0f83c6257a03e785c81cc2/pyzmq-27.1.0-cp312-abi3-win_arm64.whl", hash = "sha256:75a2f36223f0d535a0c919e23615fc85a1e23b71f40c7eb43d7b1dedb4d8f15f", size = 552993, upload-time = "2025-09-08T23:08:18.926Z" }, + { url = "https://files.pythonhosted.org/packages/60/cb/84a13459c51da6cec1b7b1dc1a47e6db6da50b77ad7fd9c145842750a011/pyzmq-27.1.0-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:93ad4b0855a664229559e45c8d23797ceac03183c7b6f5b4428152a6b06684a5", size = 1122436, upload-time = "2025-09-08T23:08:20.801Z" }, + { url = "https://files.pythonhosted.org/packages/dc/b6/94414759a69a26c3dd674570a81813c46a078767d931a6c70ad29fc585cb/pyzmq-27.1.0-cp313-cp313-android_24_x86_64.whl", hash = "sha256:fbb4f2400bfda24f12f009cba62ad5734148569ff4949b1b6ec3b519444342e6", size = 1156301, upload-time = "2025-09-08T23:08:22.47Z" }, + { url = "https://files.pythonhosted.org/packages/a5/ad/15906493fd40c316377fd8a8f6b1f93104f97a752667763c9b9c1b71d42d/pyzmq-27.1.0-cp313-cp313t-macosx_10_15_universal2.whl", hash = "sha256:e343d067f7b151cfe4eb3bb796a7752c9d369eed007b91231e817071d2c2fec7", size = 1341197, upload-time = "2025-09-08T23:08:24.286Z" }, + { url = "https://files.pythonhosted.org/packages/14/1d/d343f3ce13db53a54cb8946594e567410b2125394dafcc0268d8dda027e0/pyzmq-27.1.0-cp313-cp313t-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:08363b2011dec81c354d694bdecaef4770e0ae96b9afea70b3f47b973655cc05", size = 897275, upload-time = "2025-09-08T23:08:26.063Z" }, + { url = "https://files.pythonhosted.org/packages/69/2d/d83dd6d7ca929a2fc67d2c3005415cdf322af7751d773524809f9e585129/pyzmq-27.1.0-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d54530c8c8b5b8ddb3318f481297441af102517602b569146185fa10b63f4fa9", size = 660469, upload-time = "2025-09-08T23:08:27.623Z" }, + { url = "https://files.pythonhosted.org/packages/3e/cd/9822a7af117f4bc0f1952dbe9ef8358eb50a24928efd5edf54210b850259/pyzmq-27.1.0-cp313-cp313t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6f3afa12c392f0a44a2414056d730eebc33ec0926aae92b5ad5cf26ebb6cc128", size = 847961, upload-time = "2025-09-08T23:08:29.672Z" }, + { url = "https://files.pythonhosted.org/packages/9a/12/f003e824a19ed73be15542f172fd0ec4ad0b60cf37436652c93b9df7c585/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c65047adafe573ff023b3187bb93faa583151627bc9c51fc4fb2c561ed689d39", size = 1650282, upload-time = "2025-09-08T23:08:31.349Z" }, + { url = "https://files.pythonhosted.org/packages/d5/4a/e82d788ed58e9a23995cee70dbc20c9aded3d13a92d30d57ec2291f1e8a3/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:90e6e9441c946a8b0a667356f7078d96411391a3b8f80980315455574177ec97", size = 2024468, upload-time = "2025-09-08T23:08:33.543Z" }, + { url = "https://files.pythonhosted.org/packages/d9/94/2da0a60841f757481e402b34bf4c8bf57fa54a5466b965de791b1e6f747d/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:add071b2d25f84e8189aaf0882d39a285b42fa3853016ebab234a5e78c7a43db", size = 1885394, upload-time = "2025-09-08T23:08:35.51Z" }, + { url = "https://files.pythonhosted.org/packages/4f/6f/55c10e2e49ad52d080dc24e37adb215e5b0d64990b57598abc2e3f01725b/pyzmq-27.1.0-cp313-cp313t-win32.whl", hash = "sha256:7ccc0700cfdf7bd487bea8d850ec38f204478681ea02a582a8da8171b7f90a1c", size = 574964, upload-time = "2025-09-08T23:08:37.178Z" }, + { url = "https://files.pythonhosted.org/packages/87/4d/2534970ba63dd7c522d8ca80fb92777f362c0f321900667c615e2067cb29/pyzmq-27.1.0-cp313-cp313t-win_amd64.whl", hash = "sha256:8085a9fba668216b9b4323be338ee5437a235fe275b9d1610e422ccc279733e2", size = 641029, upload-time = "2025-09-08T23:08:40.595Z" }, + { url = "https://files.pythonhosted.org/packages/f6/fa/f8aea7a28b0641f31d40dea42d7ef003fded31e184ef47db696bc74cd610/pyzmq-27.1.0-cp313-cp313t-win_arm64.whl", hash = "sha256:6bb54ca21bcfe361e445256c15eedf083f153811c37be87e0514934d6913061e", size = 561541, upload-time = "2025-09-08T23:08:42.668Z" }, + { url = "https://files.pythonhosted.org/packages/87/45/19efbb3000956e82d0331bafca5d9ac19ea2857722fa2caacefb6042f39d/pyzmq-27.1.0-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:ce980af330231615756acd5154f29813d553ea555485ae712c491cd483df6b7a", size = 1341197, upload-time = "2025-09-08T23:08:44.973Z" }, + { url = "https://files.pythonhosted.org/packages/48/43/d72ccdbf0d73d1343936296665826350cb1e825f92f2db9db3e61c2162a2/pyzmq-27.1.0-cp314-cp314t-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:1779be8c549e54a1c38f805e56d2a2e5c009d26de10921d7d51cfd1c8d4632ea", size = 897175, upload-time = "2025-09-08T23:08:46.601Z" }, + { url = "https://files.pythonhosted.org/packages/2f/2e/a483f73a10b65a9ef0161e817321d39a770b2acf8bcf3004a28d90d14a94/pyzmq-27.1.0-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7200bb0f03345515df50d99d3db206a0a6bee1955fbb8c453c76f5bf0e08fb96", size = 660427, upload-time = "2025-09-08T23:08:48.187Z" }, + { url = "https://files.pythonhosted.org/packages/f5/d2/5f36552c2d3e5685abe60dfa56f91169f7a2d99bbaf67c5271022ab40863/pyzmq-27.1.0-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01c0e07d558b06a60773744ea6251f769cd79a41a97d11b8bf4ab8f034b0424d", size = 847929, upload-time = "2025-09-08T23:08:49.76Z" }, + { url = "https://files.pythonhosted.org/packages/c4/2a/404b331f2b7bf3198e9945f75c4c521f0c6a3a23b51f7a4a401b94a13833/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:80d834abee71f65253c91540445d37c4c561e293ba6e741b992f20a105d69146", size = 1650193, upload-time = "2025-09-08T23:08:51.7Z" }, + { url = "https://files.pythonhosted.org/packages/1c/0b/f4107e33f62a5acf60e3ded67ed33d79b4ce18de432625ce2fc5093d6388/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:544b4e3b7198dde4a62b8ff6685e9802a9a1ebf47e77478a5eb88eca2a82f2fd", size = 2024388, upload-time = "2025-09-08T23:08:53.393Z" }, + { url = "https://files.pythonhosted.org/packages/0d/01/add31fe76512642fd6e40e3a3bd21f4b47e242c8ba33efb6809e37076d9b/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cedc4c68178e59a4046f97eca31b148ddcf51e88677de1ef4e78cf06c5376c9a", size = 1885316, upload-time = "2025-09-08T23:08:55.702Z" }, + { url = "https://files.pythonhosted.org/packages/c4/59/a5f38970f9bf07cee96128de79590bb354917914a9be11272cfc7ff26af0/pyzmq-27.1.0-cp314-cp314t-win32.whl", hash = "sha256:1f0b2a577fd770aa6f053211a55d1c47901f4d537389a034c690291485e5fe92", size = 587472, upload-time = "2025-09-08T23:08:58.18Z" }, + { url = "https://files.pythonhosted.org/packages/70/d8/78b1bad170f93fcf5e3536e70e8fadac55030002275c9a29e8f5719185de/pyzmq-27.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:19c9468ae0437f8074af379e986c5d3d7d7bfe033506af442e8c879732bedbe0", size = 661401, upload-time = "2025-09-08T23:08:59.802Z" }, + { url = "https://files.pythonhosted.org/packages/81/d6/4bfbb40c9a0b42fc53c7cf442f6385db70b40f74a783130c5d0a5aa62228/pyzmq-27.1.0-cp314-cp314t-win_arm64.whl", hash = "sha256:dc5dbf68a7857b59473f7df42650c621d7e8923fb03fa74a526890f4d33cc4d7", size = 575170, upload-time = "2025-09-08T23:09:01.418Z" }, + { url = "https://files.pythonhosted.org/packages/f3/81/a65e71c1552f74dec9dff91d95bafb6e0d33338a8dfefbc88aa562a20c92/pyzmq-27.1.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:c17e03cbc9312bee223864f1a2b13a99522e0dc9f7c5df0177cd45210ac286e6", size = 836266, upload-time = "2025-09-08T23:09:40.048Z" }, + { url = "https://files.pythonhosted.org/packages/58/ed/0202ca350f4f2b69faa95c6d931e3c05c3a397c184cacb84cb4f8f42f287/pyzmq-27.1.0-pp310-pypy310_pp73-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:f328d01128373cb6763823b2b4e7f73bdf767834268c565151eacb3b7a392f90", size = 800206, upload-time = "2025-09-08T23:09:41.902Z" }, + { url = "https://files.pythonhosted.org/packages/47/42/1ff831fa87fe8f0a840ddb399054ca0009605d820e2b44ea43114f5459f4/pyzmq-27.1.0-pp310-pypy310_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9c1790386614232e1b3a40a958454bdd42c6d1811837b15ddbb052a032a43f62", size = 567747, upload-time = "2025-09-08T23:09:43.741Z" }, + { url = "https://files.pythonhosted.org/packages/d1/db/5c4d6807434751e3f21231bee98109aa57b9b9b55e058e450d0aef59b70f/pyzmq-27.1.0-pp310-pypy310_pp73-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:448f9cb54eb0cee4732b46584f2710c8bc178b0e5371d9e4fc8125201e413a74", size = 747371, upload-time = "2025-09-08T23:09:45.575Z" }, + { url = "https://files.pythonhosted.org/packages/26/af/78ce193dbf03567eb8c0dc30e3df2b9e56f12a670bf7eb20f9fb532c7e8a/pyzmq-27.1.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:05b12f2d32112bf8c95ef2e74ec4f1d4beb01f8b5e703b38537f8849f92cb9ba", size = 544862, upload-time = "2025-09-08T23:09:47.448Z" }, + { url = "https://files.pythonhosted.org/packages/4c/c6/c4dcdecdbaa70969ee1fdced6d7b8f60cfabe64d25361f27ac4665a70620/pyzmq-27.1.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:18770c8d3563715387139060d37859c02ce40718d1faf299abddcdcc6a649066", size = 836265, upload-time = "2025-09-08T23:09:49.376Z" }, + { url = "https://files.pythonhosted.org/packages/3e/79/f38c92eeaeb03a2ccc2ba9866f0439593bb08c5e3b714ac1d553e5c96e25/pyzmq-27.1.0-pp311-pypy311_pp73-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:ac25465d42f92e990f8d8b0546b01c391ad431c3bf447683fdc40565941d0604", size = 800208, upload-time = "2025-09-08T23:09:51.073Z" }, + { url = "https://files.pythonhosted.org/packages/49/0e/3f0d0d335c6b3abb9b7b723776d0b21fa7f3a6c819a0db6097059aada160/pyzmq-27.1.0-pp311-pypy311_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:53b40f8ae006f2734ee7608d59ed661419f087521edbfc2149c3932e9c14808c", size = 567747, upload-time = "2025-09-08T23:09:52.698Z" }, + { url = "https://files.pythonhosted.org/packages/a1/cf/f2b3784d536250ffd4be70e049f3b60981235d70c6e8ce7e3ef21e1adb25/pyzmq-27.1.0-pp311-pypy311_pp73-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f605d884e7c8be8fe1aa94e0a783bf3f591b84c24e4bc4f3e7564c82ac25e271", size = 747371, upload-time = "2025-09-08T23:09:54.563Z" }, + { url = "https://files.pythonhosted.org/packages/01/1b/5dbe84eefc86f48473947e2f41711aded97eecef1231f4558f1f02713c12/pyzmq-27.1.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:c9f7f6e13dff2e44a6afeaf2cf54cee5929ad64afaf4d40b50f93c58fc687355", size = 544862, upload-time = "2025-09-08T23:09:56.509Z" }, +] + +[[package]] +name = "referencing" +version = "0.37.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "attrs" }, + { name = "rpds-py", version = "0.30.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "rpds-py", version = "2026.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/22/f5/df4e9027acead3ecc63e50fe1e36aca1523e1719559c499951bb4b53188f/referencing-0.37.0.tar.gz", hash = "sha256:44aefc3142c5b842538163acb373e24cce6632bd54bdb01b21ad5863489f50d8", size = 78036, upload-time = "2025-10-13T15:30:48.871Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2c/58/ca301544e1fa93ed4f80d724bf5b194f6e4b945841c5bfd555878eea9fcb/referencing-0.37.0-py3-none-any.whl", hash = "sha256:381329a9f99628c9069361716891d34ad94af76e461dcb0335825aecc7692231", size = 26766, upload-time = "2025-10-13T15:30:47.625Z" }, +] + [[package]] name = "requests" version = "2.32.3" @@ -1374,6 +1962,273 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f9/9b/335f9764261e915ed497fcdeb11df5dfd6f7bf257d4a6a2a686d80da4d54/requests-2.32.3-py3-none-any.whl", hash = "sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6", size = 64928, upload-time = "2024-05-29T15:37:47.027Z" }, ] +[[package]] +name = "rpds-py" +version = "0.30.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.11'", +] +sdist = { url = "https://files.pythonhosted.org/packages/20/af/3f2f423103f1113b36230496629986e0ef7e199d2aa8392452b484b38ced/rpds_py-0.30.0.tar.gz", hash = "sha256:dd8ff7cf90014af0c0f787eea34794ebf6415242ee1d6fa91eaba725cc441e84", size = 69469, upload-time = "2025-11-30T20:24:38.837Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/06/0c/0c411a0ec64ccb6d104dcabe0e713e05e153a9a2c3c2bd2b32ce412166fe/rpds_py-0.30.0-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:679ae98e00c0e8d68a7fda324e16b90fd5260945b45d3b824c892cec9eea3288", size = 370490, upload-time = "2025-11-30T20:21:33.256Z" }, + { url = "https://files.pythonhosted.org/packages/19/6a/4ba3d0fb7297ebae71171822554abe48d7cab29c28b8f9f2c04b79988c05/rpds_py-0.30.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:4cc2206b76b4f576934f0ed374b10d7ca5f457858b157ca52064bdfc26b9fc00", size = 359751, upload-time = "2025-11-30T20:21:34.591Z" }, + { url = "https://files.pythonhosted.org/packages/cd/7c/e4933565ef7f7a0818985d87c15d9d273f1a649afa6a52ea35ad011195ea/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:389a2d49eded1896c3d48b0136ead37c48e221b391c052fba3f4055c367f60a6", size = 389696, upload-time = "2025-11-30T20:21:36.122Z" }, + { url = "https://files.pythonhosted.org/packages/5e/01/6271a2511ad0815f00f7ed4390cf2567bec1d4b1da39e2c27a41e6e3b4de/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:32c8528634e1bf7121f3de08fa85b138f4e0dc47657866630611b03967f041d7", size = 403136, upload-time = "2025-11-30T20:21:37.728Z" }, + { url = "https://files.pythonhosted.org/packages/55/64/c857eb7cd7541e9b4eee9d49c196e833128a55b89a9850a9c9ac33ccf897/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f207f69853edd6f6700b86efb84999651baf3789e78a466431df1331608e5324", size = 524699, upload-time = "2025-11-30T20:21:38.92Z" }, + { url = "https://files.pythonhosted.org/packages/9c/ed/94816543404078af9ab26159c44f9e98e20fe47e2126d5d32c9d9948d10a/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:67b02ec25ba7a9e8fa74c63b6ca44cf5707f2fbfadae3ee8e7494297d56aa9df", size = 412022, upload-time = "2025-11-30T20:21:40.407Z" }, + { url = "https://files.pythonhosted.org/packages/61/b5/707f6cf0066a6412aacc11d17920ea2e19e5b2f04081c64526eb35b5c6e7/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0c0e95f6819a19965ff420f65578bacb0b00f251fefe2c8b23347c37174271f3", size = 390522, upload-time = "2025-11-30T20:21:42.17Z" }, + { url = "https://files.pythonhosted.org/packages/13/4e/57a85fda37a229ff4226f8cbcf09f2a455d1ed20e802ce5b2b4a7f5ed053/rpds_py-0.30.0-cp310-cp310-manylinux_2_31_riscv64.whl", hash = "sha256:a452763cc5198f2f98898eb98f7569649fe5da666c2dc6b5ddb10fde5a574221", size = 404579, upload-time = "2025-11-30T20:21:43.769Z" }, + { url = "https://files.pythonhosted.org/packages/f9/da/c9339293513ec680a721e0e16bf2bac3db6e5d7e922488de471308349bba/rpds_py-0.30.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e0b65193a413ccc930671c55153a03ee57cecb49e6227204b04fae512eb657a7", size = 421305, upload-time = "2025-11-30T20:21:44.994Z" }, + { url = "https://files.pythonhosted.org/packages/f9/be/522cb84751114f4ad9d822ff5a1aa3c98006341895d5f084779b99596e5c/rpds_py-0.30.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:858738e9c32147f78b3ac24dc0edb6610000e56dc0f700fd5f651d0a0f0eb9ff", size = 572503, upload-time = "2025-11-30T20:21:46.91Z" }, + { url = "https://files.pythonhosted.org/packages/a2/9b/de879f7e7ceddc973ea6e4629e9b380213a6938a249e94b0cdbcc325bb66/rpds_py-0.30.0-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:da279aa314f00acbb803da1e76fa18666778e8a8f83484fba94526da5de2cba7", size = 598322, upload-time = "2025-11-30T20:21:48.709Z" }, + { url = "https://files.pythonhosted.org/packages/48/ac/f01fc22efec3f37d8a914fc1b2fb9bcafd56a299edbe96406f3053edea5a/rpds_py-0.30.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:7c64d38fb49b6cdeda16ab49e35fe0da2e1e9b34bc38bd78386530f218b37139", size = 560792, upload-time = "2025-11-30T20:21:50.024Z" }, + { url = "https://files.pythonhosted.org/packages/e2/da/4e2b19d0f131f35b6146425f846563d0ce036763e38913d917187307a671/rpds_py-0.30.0-cp310-cp310-win32.whl", hash = "sha256:6de2a32a1665b93233cde140ff8b3467bdb9e2af2b91079f0333a0974d12d464", size = 221901, upload-time = "2025-11-30T20:21:51.32Z" }, + { url = "https://files.pythonhosted.org/packages/96/cb/156d7a5cf4f78a7cc571465d8aec7a3c447c94f6749c5123f08438bcf7bc/rpds_py-0.30.0-cp310-cp310-win_amd64.whl", hash = "sha256:1726859cd0de969f88dc8673bdd954185b9104e05806be64bcd87badbe313169", size = 235823, upload-time = "2025-11-30T20:21:52.505Z" }, + { url = "https://files.pythonhosted.org/packages/4d/6e/f964e88b3d2abee2a82c1ac8366da848fce1c6d834dc2132c3fda3970290/rpds_py-0.30.0-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:a2bffea6a4ca9f01b3f8e548302470306689684e61602aa3d141e34da06cf425", size = 370157, upload-time = "2025-11-30T20:21:53.789Z" }, + { url = "https://files.pythonhosted.org/packages/94/ba/24e5ebb7c1c82e74c4e4f33b2112a5573ddc703915b13a073737b59b86e0/rpds_py-0.30.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:dc4f992dfe1e2bc3ebc7444f6c7051b4bc13cd8e33e43511e8ffd13bf407010d", size = 359676, upload-time = "2025-11-30T20:21:55.475Z" }, + { url = "https://files.pythonhosted.org/packages/84/86/04dbba1b087227747d64d80c3b74df946b986c57af0a9f0c98726d4d7a3b/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:422c3cb9856d80b09d30d2eb255d0754b23e090034e1deb4083f8004bd0761e4", size = 389938, upload-time = "2025-11-30T20:21:57.079Z" }, + { url = "https://files.pythonhosted.org/packages/42/bb/1463f0b1722b7f45431bdd468301991d1328b16cffe0b1c2918eba2c4eee/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:07ae8a593e1c3c6b82ca3292efbe73c30b61332fd612e05abee07c79359f292f", size = 402932, upload-time = "2025-11-30T20:21:58.47Z" }, + { url = "https://files.pythonhosted.org/packages/99/ee/2520700a5c1f2d76631f948b0736cdf9b0acb25abd0ca8e889b5c62ac2e3/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:12f90dd7557b6bd57f40abe7747e81e0c0b119bef015ea7726e69fe550e394a4", size = 525830, upload-time = "2025-11-30T20:21:59.699Z" }, + { url = "https://files.pythonhosted.org/packages/e0/ad/bd0331f740f5705cc555a5e17fdf334671262160270962e69a2bdef3bf76/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:99b47d6ad9a6da00bec6aabe5a6279ecd3c06a329d4aa4771034a21e335c3a97", size = 412033, upload-time = "2025-11-30T20:22:00.991Z" }, + { url = "https://files.pythonhosted.org/packages/f8/1e/372195d326549bb51f0ba0f2ecb9874579906b97e08880e7a65c3bef1a99/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:33f559f3104504506a44bb666b93a33f5d33133765b0c216a5bf2f1e1503af89", size = 390828, upload-time = "2025-11-30T20:22:02.723Z" }, + { url = "https://files.pythonhosted.org/packages/ab/2b/d88bb33294e3e0c76bc8f351a3721212713629ffca1700fa94979cb3eae8/rpds_py-0.30.0-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:946fe926af6e44f3697abbc305ea168c2c31d3e3ef1058cf68f379bf0335a78d", size = 404683, upload-time = "2025-11-30T20:22:04.367Z" }, + { url = "https://files.pythonhosted.org/packages/50/32/c759a8d42bcb5289c1fac697cd92f6fe01a018dd937e62ae77e0e7f15702/rpds_py-0.30.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:495aeca4b93d465efde585977365187149e75383ad2684f81519f504f5c13038", size = 421583, upload-time = "2025-11-30T20:22:05.814Z" }, + { url = "https://files.pythonhosted.org/packages/2b/81/e729761dbd55ddf5d84ec4ff1f47857f4374b0f19bdabfcf929164da3e24/rpds_py-0.30.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d9a0ca5da0386dee0655b4ccdf46119df60e0f10da268d04fe7cc87886872ba7", size = 572496, upload-time = "2025-11-30T20:22:07.713Z" }, + { url = "https://files.pythonhosted.org/packages/14/f6/69066a924c3557c9c30baa6ec3a0aa07526305684c6f86c696b08860726c/rpds_py-0.30.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:8d6d1cc13664ec13c1b84241204ff3b12f9bb82464b8ad6e7a5d3486975c2eed", size = 598669, upload-time = "2025-11-30T20:22:09.312Z" }, + { url = "https://files.pythonhosted.org/packages/5f/48/905896b1eb8a05630d20333d1d8ffd162394127b74ce0b0784ae04498d32/rpds_py-0.30.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:3896fa1be39912cf0757753826bc8bdc8ca331a28a7c4ae46b7a21280b06bb85", size = 561011, upload-time = "2025-11-30T20:22:11.309Z" }, + { url = "https://files.pythonhosted.org/packages/22/16/cd3027c7e279d22e5eb431dd3c0fbc677bed58797fe7581e148f3f68818b/rpds_py-0.30.0-cp311-cp311-win32.whl", hash = "sha256:55f66022632205940f1827effeff17c4fa7ae1953d2b74a8581baaefb7d16f8c", size = 221406, upload-time = "2025-11-30T20:22:13.101Z" }, + { url = "https://files.pythonhosted.org/packages/fa/5b/e7b7aa136f28462b344e652ee010d4de26ee9fd16f1bfd5811f5153ccf89/rpds_py-0.30.0-cp311-cp311-win_amd64.whl", hash = "sha256:a51033ff701fca756439d641c0ad09a41d9242fa69121c7d8769604a0a629825", size = 236024, upload-time = "2025-11-30T20:22:14.853Z" }, + { url = "https://files.pythonhosted.org/packages/14/a6/364bba985e4c13658edb156640608f2c9e1d3ea3c81b27aa9d889fff0e31/rpds_py-0.30.0-cp311-cp311-win_arm64.whl", hash = "sha256:47b0ef6231c58f506ef0b74d44e330405caa8428e770fec25329ed2cb971a229", size = 229069, upload-time = "2025-11-30T20:22:16.577Z" }, + { url = "https://files.pythonhosted.org/packages/03/e7/98a2f4ac921d82f33e03f3835f5bf3a4a40aa1bfdc57975e74a97b2b4bdd/rpds_py-0.30.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:a161f20d9a43006833cd7068375a94d035714d73a172b681d8881820600abfad", size = 375086, upload-time = "2025-11-30T20:22:17.93Z" }, + { url = "https://files.pythonhosted.org/packages/4d/a1/bca7fd3d452b272e13335db8d6b0b3ecde0f90ad6f16f3328c6fb150c889/rpds_py-0.30.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6abc8880d9d036ecaafe709079969f56e876fcf107f7a8e9920ba6d5a3878d05", size = 359053, upload-time = "2025-11-30T20:22:19.297Z" }, + { url = "https://files.pythonhosted.org/packages/65/1c/ae157e83a6357eceff62ba7e52113e3ec4834a84cfe07fa4b0757a7d105f/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ca28829ae5f5d569bb62a79512c842a03a12576375d5ece7d2cadf8abe96ec28", size = 390763, upload-time = "2025-11-30T20:22:21.661Z" }, + { url = "https://files.pythonhosted.org/packages/d4/36/eb2eb8515e2ad24c0bd43c3ee9cd74c33f7ca6430755ccdb240fd3144c44/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:a1010ed9524c73b94d15919ca4d41d8780980e1765babf85f9a2f90d247153dd", size = 408951, upload-time = "2025-11-30T20:22:23.408Z" }, + { url = "https://files.pythonhosted.org/packages/d6/65/ad8dc1784a331fabbd740ef6f71ce2198c7ed0890dab595adb9ea2d775a1/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f8d1736cfb49381ba528cd5baa46f82fdc65c06e843dab24dd70b63d09121b3f", size = 514622, upload-time = "2025-11-30T20:22:25.16Z" }, + { url = "https://files.pythonhosted.org/packages/63/8e/0cfa7ae158e15e143fe03993b5bcd743a59f541f5952e1546b1ac1b5fd45/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d948b135c4693daff7bc2dcfc4ec57237a29bd37e60c2fabf5aff2bbacf3e2f1", size = 414492, upload-time = "2025-11-30T20:22:26.505Z" }, + { url = "https://files.pythonhosted.org/packages/60/1b/6f8f29f3f995c7ffdde46a626ddccd7c63aefc0efae881dc13b6e5d5bb16/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:47f236970bccb2233267d89173d3ad2703cd36a0e2a6e92d0560d333871a3d23", size = 394080, upload-time = "2025-11-30T20:22:27.934Z" }, + { url = "https://files.pythonhosted.org/packages/6d/d5/a266341051a7a3ca2f4b750a3aa4abc986378431fc2da508c5034d081b70/rpds_py-0.30.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:2e6ecb5a5bcacf59c3f912155044479af1d0b6681280048b338b28e364aca1f6", size = 408680, upload-time = "2025-11-30T20:22:29.341Z" }, + { url = "https://files.pythonhosted.org/packages/10/3b/71b725851df9ab7a7a4e33cf36d241933da66040d195a84781f49c50490c/rpds_py-0.30.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a8fa71a2e078c527c3e9dc9fc5a98c9db40bcc8a92b4e8858e36d329f8684b51", size = 423589, upload-time = "2025-11-30T20:22:31.469Z" }, + { url = "https://files.pythonhosted.org/packages/00/2b/e59e58c544dc9bd8bd8384ecdb8ea91f6727f0e37a7131baeff8d6f51661/rpds_py-0.30.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:73c67f2db7bc334e518d097c6d1e6fed021bbc9b7d678d6cc433478365d1d5f5", size = 573289, upload-time = "2025-11-30T20:22:32.997Z" }, + { url = "https://files.pythonhosted.org/packages/da/3e/a18e6f5b460893172a7d6a680e86d3b6bc87a54c1f0b03446a3c8c7b588f/rpds_py-0.30.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:5ba103fb455be00f3b1c2076c9d4264bfcb037c976167a6047ed82f23153f02e", size = 599737, upload-time = "2025-11-30T20:22:34.419Z" }, + { url = "https://files.pythonhosted.org/packages/5c/e2/714694e4b87b85a18e2c243614974413c60aa107fd815b8cbc42b873d1d7/rpds_py-0.30.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:7cee9c752c0364588353e627da8a7e808a66873672bcb5f52890c33fd965b394", size = 563120, upload-time = "2025-11-30T20:22:35.903Z" }, + { url = "https://files.pythonhosted.org/packages/6f/ab/d5d5e3bcedb0a77f4f613706b750e50a5a3ba1c15ccd3665ecc636c968fd/rpds_py-0.30.0-cp312-cp312-win32.whl", hash = "sha256:1ab5b83dbcf55acc8b08fc62b796ef672c457b17dbd7820a11d6c52c06839bdf", size = 223782, upload-time = "2025-11-30T20:22:37.271Z" }, + { url = "https://files.pythonhosted.org/packages/39/3b/f786af9957306fdc38a74cef405b7b93180f481fb48453a114bb6465744a/rpds_py-0.30.0-cp312-cp312-win_amd64.whl", hash = "sha256:a090322ca841abd453d43456ac34db46e8b05fd9b3b4ac0c78bcde8b089f959b", size = 240463, upload-time = "2025-11-30T20:22:39.021Z" }, + { url = "https://files.pythonhosted.org/packages/f3/d2/b91dc748126c1559042cfe41990deb92c4ee3e2b415f6b5234969ffaf0cc/rpds_py-0.30.0-cp312-cp312-win_arm64.whl", hash = "sha256:669b1805bd639dd2989b281be2cfd951c6121b65e729d9b843e9639ef1fd555e", size = 230868, upload-time = "2025-11-30T20:22:40.493Z" }, + { url = "https://files.pythonhosted.org/packages/ed/dc/d61221eb88ff410de3c49143407f6f3147acf2538c86f2ab7ce65ae7d5f9/rpds_py-0.30.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:f83424d738204d9770830d35290ff3273fbb02b41f919870479fab14b9d303b2", size = 374887, upload-time = "2025-11-30T20:22:41.812Z" }, + { url = "https://files.pythonhosted.org/packages/fd/32/55fb50ae104061dbc564ef15cc43c013dc4a9f4527a1f4d99baddf56fe5f/rpds_py-0.30.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:e7536cd91353c5273434b4e003cbda89034d67e7710eab8761fd918ec6c69cf8", size = 358904, upload-time = "2025-11-30T20:22:43.479Z" }, + { url = "https://files.pythonhosted.org/packages/58/70/faed8186300e3b9bdd138d0273109784eea2396c68458ed580f885dfe7ad/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2771c6c15973347f50fece41fc447c054b7ac2ae0502388ce3b6738cd366e3d4", size = 389945, upload-time = "2025-11-30T20:22:44.819Z" }, + { url = "https://files.pythonhosted.org/packages/bd/a8/073cac3ed2c6387df38f71296d002ab43496a96b92c823e76f46b8af0543/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:0a59119fc6e3f460315fe9d08149f8102aa322299deaa5cab5b40092345c2136", size = 407783, upload-time = "2025-11-30T20:22:46.103Z" }, + { url = "https://files.pythonhosted.org/packages/77/57/5999eb8c58671f1c11eba084115e77a8899d6e694d2a18f69f0ba471ec8b/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:76fec018282b4ead0364022e3c54b60bf368b9d926877957a8624b58419169b7", size = 515021, upload-time = "2025-11-30T20:22:47.458Z" }, + { url = "https://files.pythonhosted.org/packages/e0/af/5ab4833eadc36c0a8ed2bc5c0de0493c04f6c06de223170bd0798ff98ced/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:692bef75a5525db97318e8cd061542b5a79812d711ea03dbc1f6f8dbb0c5f0d2", size = 414589, upload-time = "2025-11-30T20:22:48.872Z" }, + { url = "https://files.pythonhosted.org/packages/b7/de/f7192e12b21b9e9a68a6d0f249b4af3fdcdff8418be0767a627564afa1f1/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9027da1ce107104c50c81383cae773ef5c24d296dd11c99e2629dbd7967a20c6", size = 394025, upload-time = "2025-11-30T20:22:50.196Z" }, + { url = "https://files.pythonhosted.org/packages/91/c4/fc70cd0249496493500e7cc2de87504f5aa6509de1e88623431fec76d4b6/rpds_py-0.30.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:9cf69cdda1f5968a30a359aba2f7f9aa648a9ce4b580d6826437f2b291cfc86e", size = 408895, upload-time = "2025-11-30T20:22:51.87Z" }, + { url = "https://files.pythonhosted.org/packages/58/95/d9275b05ab96556fefff73a385813eb66032e4c99f411d0795372d9abcea/rpds_py-0.30.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a4796a717bf12b9da9d3ad002519a86063dcac8988b030e405704ef7d74d2d9d", size = 422799, upload-time = "2025-11-30T20:22:53.341Z" }, + { url = "https://files.pythonhosted.org/packages/06/c1/3088fc04b6624eb12a57eb814f0d4997a44b0d208d6cace713033ff1a6ba/rpds_py-0.30.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:5d4c2aa7c50ad4728a094ebd5eb46c452e9cb7edbfdb18f9e1221f597a73e1e7", size = 572731, upload-time = "2025-11-30T20:22:54.778Z" }, + { url = "https://files.pythonhosted.org/packages/d8/42/c612a833183b39774e8ac8fecae81263a68b9583ee343db33ab571a7ce55/rpds_py-0.30.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:ba81a9203d07805435eb06f536d95a266c21e5b2dfbf6517748ca40c98d19e31", size = 599027, upload-time = "2025-11-30T20:22:56.212Z" }, + { url = "https://files.pythonhosted.org/packages/5f/60/525a50f45b01d70005403ae0e25f43c0384369ad24ffe46e8d9068b50086/rpds_py-0.30.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:945dccface01af02675628334f7cf49c2af4c1c904748efc5cf7bbdf0b579f95", size = 563020, upload-time = "2025-11-30T20:22:58.2Z" }, + { url = "https://files.pythonhosted.org/packages/0b/5d/47c4655e9bcd5ca907148535c10e7d489044243cc9941c16ed7cd53be91d/rpds_py-0.30.0-cp313-cp313-win32.whl", hash = "sha256:b40fb160a2db369a194cb27943582b38f79fc4887291417685f3ad693c5a1d5d", size = 223139, upload-time = "2025-11-30T20:23:00.209Z" }, + { url = "https://files.pythonhosted.org/packages/f2/e1/485132437d20aa4d3e1d8b3fb5a5e65aa8139f1e097080c2a8443201742c/rpds_py-0.30.0-cp313-cp313-win_amd64.whl", hash = "sha256:806f36b1b605e2d6a72716f321f20036b9489d29c51c91f4dd29a3e3afb73b15", size = 240224, upload-time = "2025-11-30T20:23:02.008Z" }, + { url = "https://files.pythonhosted.org/packages/24/95/ffd128ed1146a153d928617b0ef673960130be0009c77d8fbf0abe306713/rpds_py-0.30.0-cp313-cp313-win_arm64.whl", hash = "sha256:d96c2086587c7c30d44f31f42eae4eac89b60dabbac18c7669be3700f13c3ce1", size = 230645, upload-time = "2025-11-30T20:23:03.43Z" }, + { url = "https://files.pythonhosted.org/packages/ff/1b/b10de890a0def2a319a2626334a7f0ae388215eb60914dbac8a3bae54435/rpds_py-0.30.0-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:eb0b93f2e5c2189ee831ee43f156ed34e2a89a78a66b98cadad955972548be5a", size = 364443, upload-time = "2025-11-30T20:23:04.878Z" }, + { url = "https://files.pythonhosted.org/packages/0d/bf/27e39f5971dc4f305a4fb9c672ca06f290f7c4e261c568f3dea16a410d47/rpds_py-0.30.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:922e10f31f303c7c920da8981051ff6d8c1a56207dbdf330d9047f6d30b70e5e", size = 353375, upload-time = "2025-11-30T20:23:06.342Z" }, + { url = "https://files.pythonhosted.org/packages/40/58/442ada3bba6e8e6615fc00483135c14a7538d2ffac30e2d933ccf6852232/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cdc62c8286ba9bf7f47befdcea13ea0e26bf294bda99758fd90535cbaf408000", size = 383850, upload-time = "2025-11-30T20:23:07.825Z" }, + { url = "https://files.pythonhosted.org/packages/14/14/f59b0127409a33c6ef6f5c1ebd5ad8e32d7861c9c7adfa9a624fc3889f6c/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:47f9a91efc418b54fb8190a6b4aa7813a23fb79c51f4bb84e418f5476c38b8db", size = 392812, upload-time = "2025-11-30T20:23:09.228Z" }, + { url = "https://files.pythonhosted.org/packages/b3/66/e0be3e162ac299b3a22527e8913767d869e6cc75c46bd844aa43fb81ab62/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1f3587eb9b17f3789ad50824084fa6f81921bbf9a795826570bda82cb3ed91f2", size = 517841, upload-time = "2025-11-30T20:23:11.186Z" }, + { url = "https://files.pythonhosted.org/packages/3d/55/fa3b9cf31d0c963ecf1ba777f7cf4b2a2c976795ac430d24a1f43d25a6ba/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:39c02563fc592411c2c61d26b6c5fe1e51eaa44a75aa2c8735ca88b0d9599daa", size = 408149, upload-time = "2025-11-30T20:23:12.864Z" }, + { url = "https://files.pythonhosted.org/packages/60/ca/780cf3b1a32b18c0f05c441958d3758f02544f1d613abf9488cd78876378/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:51a1234d8febafdfd33a42d97da7a43f5dcb120c1060e352a3fbc0c6d36e2083", size = 383843, upload-time = "2025-11-30T20:23:14.638Z" }, + { url = "https://files.pythonhosted.org/packages/82/86/d5f2e04f2aa6247c613da0c1dd87fcd08fa17107e858193566048a1e2f0a/rpds_py-0.30.0-cp313-cp313t-manylinux_2_31_riscv64.whl", hash = "sha256:eb2c4071ab598733724c08221091e8d80e89064cd472819285a9ab0f24bcedb9", size = 396507, upload-time = "2025-11-30T20:23:16.105Z" }, + { url = "https://files.pythonhosted.org/packages/4b/9a/453255d2f769fe44e07ea9785c8347edaf867f7026872e76c1ad9f7bed92/rpds_py-0.30.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:6bdfdb946967d816e6adf9a3d8201bfad269c67efe6cefd7093ef959683c8de0", size = 414949, upload-time = "2025-11-30T20:23:17.539Z" }, + { url = "https://files.pythonhosted.org/packages/a3/31/622a86cdc0c45d6df0e9ccb6becdba5074735e7033c20e401a6d9d0e2ca0/rpds_py-0.30.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c77afbd5f5250bf27bf516c7c4a016813eb2d3e116139aed0096940c5982da94", size = 565790, upload-time = "2025-11-30T20:23:19.029Z" }, + { url = "https://files.pythonhosted.org/packages/1c/5d/15bbf0fb4a3f58a3b1c67855ec1efcc4ceaef4e86644665fff03e1b66d8d/rpds_py-0.30.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:61046904275472a76c8c90c9ccee9013d70a6d0f73eecefd38c1ae7c39045a08", size = 590217, upload-time = "2025-11-30T20:23:20.885Z" }, + { url = "https://files.pythonhosted.org/packages/6d/61/21b8c41f68e60c8cc3b2e25644f0e3681926020f11d06ab0b78e3c6bbff1/rpds_py-0.30.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:4c5f36a861bc4b7da6516dbdf302c55313afa09b81931e8280361a4f6c9a2d27", size = 555806, upload-time = "2025-11-30T20:23:22.488Z" }, + { url = "https://files.pythonhosted.org/packages/f9/39/7e067bb06c31de48de3eb200f9fc7c58982a4d3db44b07e73963e10d3be9/rpds_py-0.30.0-cp313-cp313t-win32.whl", hash = "sha256:3d4a69de7a3e50ffc214ae16d79d8fbb0922972da0356dcf4d0fdca2878559c6", size = 211341, upload-time = "2025-11-30T20:23:24.449Z" }, + { url = "https://files.pythonhosted.org/packages/0a/4d/222ef0b46443cf4cf46764d9c630f3fe4abaa7245be9417e56e9f52b8f65/rpds_py-0.30.0-cp313-cp313t-win_amd64.whl", hash = "sha256:f14fc5df50a716f7ece6a80b6c78bb35ea2ca47c499e422aa4463455dd96d56d", size = 225768, upload-time = "2025-11-30T20:23:25.908Z" }, + { url = "https://files.pythonhosted.org/packages/86/81/dad16382ebbd3d0e0328776d8fd7ca94220e4fa0798d1dc5e7da48cb3201/rpds_py-0.30.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:68f19c879420aa08f61203801423f6cd5ac5f0ac4ac82a2368a9fcd6a9a075e0", size = 362099, upload-time = "2025-11-30T20:23:27.316Z" }, + { url = "https://files.pythonhosted.org/packages/2b/60/19f7884db5d5603edf3c6bce35408f45ad3e97e10007df0e17dd57af18f8/rpds_py-0.30.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:ec7c4490c672c1a0389d319b3a9cfcd098dcdc4783991553c332a15acf7249be", size = 353192, upload-time = "2025-11-30T20:23:29.151Z" }, + { url = "https://files.pythonhosted.org/packages/bf/c4/76eb0e1e72d1a9c4703c69607cec123c29028bff28ce41588792417098ac/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f251c812357a3fed308d684a5079ddfb9d933860fc6de89f2b7ab00da481e65f", size = 384080, upload-time = "2025-11-30T20:23:30.785Z" }, + { url = "https://files.pythonhosted.org/packages/72/87/87ea665e92f3298d1b26d78814721dc39ed8d2c74b86e83348d6b48a6f31/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ac98b175585ecf4c0348fd7b29c3864bda53b805c773cbf7bfdaffc8070c976f", size = 394841, upload-time = "2025-11-30T20:23:32.209Z" }, + { url = "https://files.pythonhosted.org/packages/77/ad/7783a89ca0587c15dcbf139b4a8364a872a25f861bdb88ed99f9b0dec985/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3e62880792319dbeb7eb866547f2e35973289e7d5696c6e295476448f5b63c87", size = 516670, upload-time = "2025-11-30T20:23:33.742Z" }, + { url = "https://files.pythonhosted.org/packages/5b/3c/2882bdac942bd2172f3da574eab16f309ae10a3925644e969536553cb4ee/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4e7fc54e0900ab35d041b0601431b0a0eb495f0851a0639b6ef90f7741b39a18", size = 408005, upload-time = "2025-11-30T20:23:35.253Z" }, + { url = "https://files.pythonhosted.org/packages/ce/81/9a91c0111ce1758c92516a3e44776920b579d9a7c09b2b06b642d4de3f0f/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:47e77dc9822d3ad616c3d5759ea5631a75e5809d5a28707744ef79d7a1bcfcad", size = 382112, upload-time = "2025-11-30T20:23:36.842Z" }, + { url = "https://files.pythonhosted.org/packages/cf/8e/1da49d4a107027e5fbc64daeab96a0706361a2918da10cb41769244b805d/rpds_py-0.30.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:b4dc1a6ff022ff85ecafef7979a2c6eb423430e05f1165d6688234e62ba99a07", size = 399049, upload-time = "2025-11-30T20:23:38.343Z" }, + { url = "https://files.pythonhosted.org/packages/df/5a/7ee239b1aa48a127570ec03becbb29c9d5a9eb092febbd1699d567cae859/rpds_py-0.30.0-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:4559c972db3a360808309e06a74628b95eaccbf961c335c8fe0d590cf587456f", size = 415661, upload-time = "2025-11-30T20:23:40.263Z" }, + { url = "https://files.pythonhosted.org/packages/70/ea/caa143cf6b772f823bc7929a45da1fa83569ee49b11d18d0ada7f5ee6fd6/rpds_py-0.30.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:0ed177ed9bded28f8deb6ab40c183cd1192aa0de40c12f38be4d59cd33cb5c65", size = 565606, upload-time = "2025-11-30T20:23:42.186Z" }, + { url = "https://files.pythonhosted.org/packages/64/91/ac20ba2d69303f961ad8cf55bf7dbdb4763f627291ba3d0d7d67333cced9/rpds_py-0.30.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:ad1fa8db769b76ea911cb4e10f049d80bf518c104f15b3edb2371cc65375c46f", size = 591126, upload-time = "2025-11-30T20:23:44.086Z" }, + { url = "https://files.pythonhosted.org/packages/21/20/7ff5f3c8b00c8a95f75985128c26ba44503fb35b8e0259d812766ea966c7/rpds_py-0.30.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:46e83c697b1f1c72b50e5ee5adb4353eef7406fb3f2043d64c33f20ad1c2fc53", size = 553371, upload-time = "2025-11-30T20:23:46.004Z" }, + { url = "https://files.pythonhosted.org/packages/72/c7/81dadd7b27c8ee391c132a6b192111ca58d866577ce2d9b0ca157552cce0/rpds_py-0.30.0-cp314-cp314-win32.whl", hash = "sha256:ee454b2a007d57363c2dfd5b6ca4a5d7e2c518938f8ed3b706e37e5d470801ed", size = 215298, upload-time = "2025-11-30T20:23:47.696Z" }, + { url = "https://files.pythonhosted.org/packages/3e/d2/1aaac33287e8cfb07aab2e6b8ac1deca62f6f65411344f1433c55e6f3eb8/rpds_py-0.30.0-cp314-cp314-win_amd64.whl", hash = "sha256:95f0802447ac2d10bcc69f6dc28fe95fdf17940367b21d34e34c737870758950", size = 228604, upload-time = "2025-11-30T20:23:49.501Z" }, + { url = "https://files.pythonhosted.org/packages/e8/95/ab005315818cc519ad074cb7784dae60d939163108bd2b394e60dc7b5461/rpds_py-0.30.0-cp314-cp314-win_arm64.whl", hash = "sha256:613aa4771c99f03346e54c3f038e4cc574ac09a3ddfb0e8878487335e96dead6", size = 222391, upload-time = "2025-11-30T20:23:50.96Z" }, + { url = "https://files.pythonhosted.org/packages/9e/68/154fe0194d83b973cdedcdcc88947a2752411165930182ae41d983dcefa6/rpds_py-0.30.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:7e6ecfcb62edfd632e56983964e6884851786443739dbfe3582947e87274f7cb", size = 364868, upload-time = "2025-11-30T20:23:52.494Z" }, + { url = "https://files.pythonhosted.org/packages/83/69/8bbc8b07ec854d92a8b75668c24d2abcb1719ebf890f5604c61c9369a16f/rpds_py-0.30.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:a1d0bc22a7cdc173fedebb73ef81e07faef93692b8c1ad3733b67e31e1b6e1b8", size = 353747, upload-time = "2025-11-30T20:23:54.036Z" }, + { url = "https://files.pythonhosted.org/packages/ab/00/ba2e50183dbd9abcce9497fa5149c62b4ff3e22d338a30d690f9af970561/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0d08f00679177226c4cb8c5265012eea897c8ca3b93f429e546600c971bcbae7", size = 383795, upload-time = "2025-11-30T20:23:55.556Z" }, + { url = "https://files.pythonhosted.org/packages/05/6f/86f0272b84926bcb0e4c972262f54223e8ecc556b3224d281e6598fc9268/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5965af57d5848192c13534f90f9dd16464f3c37aaf166cc1da1cae1fd5a34898", size = 393330, upload-time = "2025-11-30T20:23:57.033Z" }, + { url = "https://files.pythonhosted.org/packages/cb/e9/0e02bb2e6dc63d212641da45df2b0bf29699d01715913e0d0f017ee29438/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9a4e86e34e9ab6b667c27f3211ca48f73dba7cd3d90f8d5b11be56e5dbc3fb4e", size = 518194, upload-time = "2025-11-30T20:23:58.637Z" }, + { url = "https://files.pythonhosted.org/packages/ee/ca/be7bca14cf21513bdf9c0606aba17d1f389ea2b6987035eb4f62bd923f25/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e5d3e6b26f2c785d65cc25ef1e5267ccbe1b069c5c21b8cc724efee290554419", size = 408340, upload-time = "2025-11-30T20:24:00.2Z" }, + { url = "https://files.pythonhosted.org/packages/c2/c7/736e00ebf39ed81d75544c0da6ef7b0998f8201b369acf842f9a90dc8fce/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:626a7433c34566535b6e56a1b39a7b17ba961e97ce3b80ec62e6f1312c025551", size = 383765, upload-time = "2025-11-30T20:24:01.759Z" }, + { url = "https://files.pythonhosted.org/packages/4a/3f/da50dfde9956aaf365c4adc9533b100008ed31aea635f2b8d7b627e25b49/rpds_py-0.30.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:acd7eb3f4471577b9b5a41baf02a978e8bdeb08b4b355273994f8b87032000a8", size = 396834, upload-time = "2025-11-30T20:24:03.687Z" }, + { url = "https://files.pythonhosted.org/packages/4e/00/34bcc2565b6020eab2623349efbdec810676ad571995911f1abdae62a3a0/rpds_py-0.30.0-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fe5fa731a1fa8a0a56b0977413f8cacac1768dad38d16b3a296712709476fbd5", size = 415470, upload-time = "2025-11-30T20:24:05.232Z" }, + { url = "https://files.pythonhosted.org/packages/8c/28/882e72b5b3e6f718d5453bd4d0d9cf8df36fddeb4ddbbab17869d5868616/rpds_py-0.30.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:74a3243a411126362712ee1524dfc90c650a503502f135d54d1b352bd01f2404", size = 565630, upload-time = "2025-11-30T20:24:06.878Z" }, + { url = "https://files.pythonhosted.org/packages/3b/97/04a65539c17692de5b85c6e293520fd01317fd878ea1995f0367d4532fb1/rpds_py-0.30.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:3e8eeb0544f2eb0d2581774be4c3410356eba189529a6b3e36bbbf9696175856", size = 591148, upload-time = "2025-11-30T20:24:08.445Z" }, + { url = "https://files.pythonhosted.org/packages/85/70/92482ccffb96f5441aab93e26c4d66489eb599efdcf96fad90c14bbfb976/rpds_py-0.30.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:dbd936cde57abfee19ab3213cf9c26be06d60750e60a8e4dd85d1ab12c8b1f40", size = 556030, upload-time = "2025-11-30T20:24:10.956Z" }, + { url = "https://files.pythonhosted.org/packages/20/53/7c7e784abfa500a2b6b583b147ee4bb5a2b3747a9166bab52fec4b5b5e7d/rpds_py-0.30.0-cp314-cp314t-win32.whl", hash = "sha256:dc824125c72246d924f7f796b4f63c1e9dc810c7d9e2355864b3c3a73d59ade0", size = 211570, upload-time = "2025-11-30T20:24:12.735Z" }, + { url = "https://files.pythonhosted.org/packages/d0/02/fa464cdfbe6b26e0600b62c528b72d8608f5cc49f96b8d6e38c95d60c676/rpds_py-0.30.0-cp314-cp314t-win_amd64.whl", hash = "sha256:27f4b0e92de5bfbc6f86e43959e6edd1425c33b5e69aab0984a72047f2bcf1e3", size = 226532, upload-time = "2025-11-30T20:24:14.634Z" }, + { url = "https://files.pythonhosted.org/packages/69/71/3f34339ee70521864411f8b6992e7ab13ac30d8e4e3309e07c7361767d91/rpds_py-0.30.0-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:c2262bdba0ad4fc6fb5545660673925c2d2a5d9e2e0fb603aad545427be0fc58", size = 372292, upload-time = "2025-11-30T20:24:16.537Z" }, + { url = "https://files.pythonhosted.org/packages/57/09/f183df9b8f2d66720d2ef71075c59f7e1b336bec7ee4c48f0a2b06857653/rpds_py-0.30.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:ee6af14263f25eedc3bb918a3c04245106a42dfd4f5c2285ea6f997b1fc3f89a", size = 362128, upload-time = "2025-11-30T20:24:18.086Z" }, + { url = "https://files.pythonhosted.org/packages/7a/68/5c2594e937253457342e078f0cc1ded3dd7b2ad59afdbf2d354869110a02/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3adbb8179ce342d235c31ab8ec511e66c73faa27a47e076ccc92421add53e2bb", size = 391542, upload-time = "2025-11-30T20:24:20.092Z" }, + { url = "https://files.pythonhosted.org/packages/49/5c/31ef1afd70b4b4fbdb2800249f34c57c64beb687495b10aec0365f53dfc4/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:250fa00e9543ac9b97ac258bd37367ff5256666122c2d0f2bc97577c60a1818c", size = 404004, upload-time = "2025-11-30T20:24:22.231Z" }, + { url = "https://files.pythonhosted.org/packages/e3/63/0cfbea38d05756f3440ce6534d51a491d26176ac045e2707adc99bb6e60a/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9854cf4f488b3d57b9aaeb105f06d78e5529d3145b1e4a41750167e8c213c6d3", size = 527063, upload-time = "2025-11-30T20:24:24.302Z" }, + { url = "https://files.pythonhosted.org/packages/42/e6/01e1f72a2456678b0f618fc9a1a13f882061690893c192fcad9f2926553a/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:993914b8e560023bc0a8bf742c5f303551992dcb85e247b1e5c7f4a7d145bda5", size = 413099, upload-time = "2025-11-30T20:24:25.916Z" }, + { url = "https://files.pythonhosted.org/packages/b8/25/8df56677f209003dcbb180765520c544525e3ef21ea72279c98b9aa7c7fb/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:58edca431fb9b29950807e301826586e5bbf24163677732429770a697ffe6738", size = 392177, upload-time = "2025-11-30T20:24:27.834Z" }, + { url = "https://files.pythonhosted.org/packages/4a/b4/0a771378c5f16f8115f796d1f437950158679bcd2a7c68cf251cfb00ed5b/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_31_riscv64.whl", hash = "sha256:dea5b552272a944763b34394d04577cf0f9bd013207bc32323b5a89a53cf9c2f", size = 406015, upload-time = "2025-11-30T20:24:29.457Z" }, + { url = "https://files.pythonhosted.org/packages/36/d8/456dbba0af75049dc6f63ff295a2f92766b9d521fa00de67a2bd6427d57a/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ba3af48635eb83d03f6c9735dfb21785303e73d22ad03d489e88adae6eab8877", size = 423736, upload-time = "2025-11-30T20:24:31.22Z" }, + { url = "https://files.pythonhosted.org/packages/13/64/b4d76f227d5c45a7e0b796c674fd81b0a6c4fbd48dc29271857d8219571c/rpds_py-0.30.0-pp311-pypy311_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:dff13836529b921e22f15cb099751209a60009731a68519630a24d61f0b1b30a", size = 573981, upload-time = "2025-11-30T20:24:32.934Z" }, + { url = "https://files.pythonhosted.org/packages/20/91/092bacadeda3edf92bf743cc96a7be133e13a39cdbfd7b5082e7ab638406/rpds_py-0.30.0-pp311-pypy311_pp73-musllinux_1_2_i686.whl", hash = "sha256:1b151685b23929ab7beec71080a8889d4d6d9fa9a983d213f07121205d48e2c4", size = 599782, upload-time = "2025-11-30T20:24:35.169Z" }, + { url = "https://files.pythonhosted.org/packages/d1/b7/b95708304cd49b7b6f82fdd039f1748b66ec2b21d6a45180910802f1abf1/rpds_py-0.30.0-pp311-pypy311_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:ac37f9f516c51e5753f27dfdef11a88330f04de2d564be3991384b2f3535d02e", size = 562191, upload-time = "2025-11-30T20:24:36.853Z" }, +] + +[[package]] +name = "rpds-py" +version = "2026.5.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version >= '3.12' and python_full_version < '3.14'", + "python_full_version == '3.11.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/2e/43/25a8dcd3feedd735039a8f0b5b7e3b118232b5eae288c4fd9ab200d41094/rpds_py-2026.5.1.tar.gz", hash = "sha256:07b24fea40541e28570e5b795a4a38fbdcd12550c06bd0748005ecc8116ca256", size = 64459, upload-time = "2026-05-28T12:02:13.232Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4f/a0/acf8b6fc20bfdcd3a45bd3f57680fb198e157b7e997b9123b10763798bd2/rpds_py-2026.5.1-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:3397a5ed7174dc2786bb214030232fc36fe8e5584fec43a9952cc542b1a12036", size = 355609, upload-time = "2026-05-28T11:58:50.78Z" }, + { url = "https://files.pythonhosted.org/packages/b6/95/f8203fd997484b1690a6869cd0e503b6c3c6be55b0ecc36d1a491fe742f0/rpds_py-2026.5.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:99ab6ba7bfa2cb0f96a04e3652355bf04e3f51aceb1e943b8541dab7ba4828cc", size = 348460, upload-time = "2026-05-28T11:58:52.374Z" }, + { url = "https://files.pythonhosted.org/packages/33/8c/b47326ad2f0be545a5e5c1a55937a12afaea7d392ba2837bb9680f57e6c9/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d0efbe45632665e53e3db8fe1e5692db58fc5cb9bab4459d570b83efefe11164", size = 381031, upload-time = "2026-05-28T11:58:53.775Z" }, + { url = "https://files.pythonhosted.org/packages/22/0b/e83bbd97ffac6f6389b605cd4e1c8ac5761dc7e977769c9255d8c5adb7bd/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:01d17b29c0c23d82b1f4751147ec49cf451f1fc2554eb9ef5f957e55d2656ead", size = 387121, upload-time = "2026-05-28T11:58:55.243Z" }, + { url = "https://files.pythonhosted.org/packages/fd/0e/d285d1bc8864245919c61e1ca82263e4a66d337759c3a4cef72766ff9afc/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7559f72b94ae52659086c595dfa017cde03155f7832071d30959049052cb3ece", size = 501026, upload-time = "2026-05-28T11:58:56.788Z" }, + { url = "https://files.pythonhosted.org/packages/86/06/ccb2109a1e543437b5e43816f2b43b9554cc6783145528a4e3711e05c011/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9e25b7088f9ccbfc0dfcaa52bf969300ca229e10ecf758974ebcbb080a4b37bb", size = 391865, upload-time = "2026-05-28T11:58:58.298Z" }, + { url = "https://files.pythonhosted.org/packages/3d/33/237173db1cfef10105b3839a24de00eb8d2a523711add4632447cdf0aedd/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:613fc4ee9eaef26dc5840666214dd6fbcebcf32f46e76f4abc473059f4e13dda", size = 378012, upload-time = "2026-05-28T11:58:59.589Z" }, + { url = "https://files.pythonhosted.org/packages/97/64/1eae54e34d5161f9969295e80bd6b62a55f2b6ac5f2a5b60d02c2140e758/rpds_py-2026.5.1-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:85264a90ff4c05c1568dd65f5921c837614b67c60358fb4c17df3b7f2e90690a", size = 391111, upload-time = "2026-05-28T11:59:01.104Z" }, + { url = "https://files.pythonhosted.org/packages/d8/34/5bb334a5a0f65d77869217c4654f34c78a7d11b93938a3c076a2edeafc52/rpds_py-2026.5.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fe71bca7d547acb17027c7fd1624ff8aae623499c498d3e7011182c4de5c25e0", size = 409225, upload-time = "2026-05-28T11:59:02.433Z" }, + { url = "https://files.pythonhosted.org/packages/16/0f/007ec21283b5b040b4ec3bd95e0402591e22bfa7d5c93dfe01c465c2d2d7/rpds_py-2026.5.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:a05fa4f41f37ec97c9c260441a940450a192f78d774d2b097eee1379f1e1246a", size = 556487, upload-time = "2026-05-28T11:59:04.012Z" }, + { url = "https://files.pythonhosted.org/packages/ff/10/5437c94508169b6b22d8418fef7a66e9ffb5f3b9e9c94460f2eedafe06ff/rpds_py-2026.5.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:df1d2a1996755b24b9ecee92cb4d36c28f86f464a6a173349c26bab41e94b8c2", size = 620798, upload-time = "2026-05-28T11:59:05.485Z" }, + { url = "https://files.pythonhosted.org/packages/e0/d5/9937dce4d6bda74157b954e7d1460db05a22f5929dccfeeba1ed27a93df0/rpds_py-2026.5.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:8895840ac4809e5f60c88fd07617cd71326e73d6e5a8aa783c5c0f7c24985de2", size = 584053, upload-time = "2026-05-28T11:59:06.837Z" }, + { url = "https://files.pythonhosted.org/packages/6c/31/750617dd0ae1752471bf43f9e41d263398fae7cde7849d23b8574a70e617/rpds_py-2026.5.1-cp311-cp311-win32.whl", hash = "sha256:3684a59b158a7683aaeb8e25352e9a9dd2122cec78f2d8530266e4f91b4c7b3f", size = 214390, upload-time = "2026-05-28T11:59:08.402Z" }, + { url = "https://files.pythonhosted.org/packages/3c/bb/3dcab0e1d9516303f2eb672a5d6f62eca5a69e2886301e9c8c54b520c39b/rpds_py-2026.5.1-cp311-cp311-win_amd64.whl", hash = "sha256:7bd530e6a530bb3ea892f194fafa455f3516ac25ecf7143fd33c09be62b0470a", size = 231097, upload-time = "2026-05-28T11:59:09.786Z" }, + { url = "https://files.pythonhosted.org/packages/49/d6/c6bbf5cb1cf12b9732df8074b57f6ef8341ba884c95d40632ae8bddb44e4/rpds_py-2026.5.1-cp311-cp311-win_arm64.whl", hash = "sha256:0a5ae4dbe43c1076983b72616496919872ae7bbe7a1e21cc48336bc3154d130b", size = 226361, upload-time = "2026-05-28T11:59:11.079Z" }, + { url = "https://files.pythonhosted.org/packages/d4/e7/a78582dc57caa592dcc7d4fb69b61390561e908eb3d2f5df5928a8e354c0/rpds_py-2026.5.1-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:3abe24a66e57adcfa645d718063a5fa5103ecc71ddbf26d78af8f9368018ff1d", size = 353040, upload-time = "2026-05-28T11:59:12.531Z" }, + { url = "https://files.pythonhosted.org/packages/a3/43/35e3f136343aef451e545ce8c38d36c2f93c0ed88703db8b64ba2b205c68/rpds_py-2026.5.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:58b1d94308ddf0b1982f61f2eb54bf92997c9ece8a8093ef014250f4a517906c", size = 345775, upload-time = "2026-05-28T11:59:13.827Z" }, + { url = "https://files.pythonhosted.org/packages/20/e1/0f2160c5982d3157734d5cb3ed63d8b2d583a73c9864f77b666449f32cf8/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0fa92420128dadce7f54bd73ba1825a273e9268fe9e35dbf7e6362890efa4e08", size = 376329, upload-time = "2026-05-28T11:59:15.271Z" }, + { url = "https://files.pythonhosted.org/packages/d0/11/ee0ba42aff83bf4effdbc576673c6be64c5e173978c3f6d537e94482f77d/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ca653c6546386227cd9800d1bef6a348099acf8db4250341da6d90f663d6dfcb", size = 383539, upload-time = "2026-05-28T11:59:16.665Z" }, + { url = "https://files.pythonhosted.org/packages/11/df/d94aa6a499d4ac40afe2d7620f2c597fd3c0f182e854ad7cf3f596a81cb6/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:66c93681c4729e4e3ecba31b8179fae083ff3118841672835140338b4b9867c1", size = 494674, upload-time = "2026-05-28T11:59:17.991Z" }, + { url = "https://files.pythonhosted.org/packages/1f/75/33d30f43bb2f458de11979486a591b1bf6e5651765ed1704c6197c2dc773/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:40ff257542e04796880e011e15cd4dc21c2599975df2aaa8f2c8495ca574e1a5", size = 389268, upload-time = "2026-05-28T11:59:19.434Z" }, + { url = "https://files.pythonhosted.org/packages/f4/1e/2c9096fc19d5fd084b0184ca2b651e659aa0a37e6fdbecf6ece47f147fe1/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b6825cc329b290e93c5f6a9be2393118a763f6ccf6abd83704e0c102ca583644", size = 376280, upload-time = "2026-05-28T11:59:21Z" }, + { url = "https://files.pythonhosted.org/packages/b9/e5/61ec9f8be8211ea7f48448195549e4aaf02004083475493b0e137702ecb2/rpds_py-2026.5.1-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:de42116e69cb53b911cc34aee5ab98f36c597b822545045d49e938818b99e5e4", size = 387233, upload-time = "2026-05-28T11:59:22.454Z" }, + { url = "https://files.pythonhosted.org/packages/0d/ca/bcec1005c4f4a234f92a29078631fee49206c7265ccae966f18fd332e80e/rpds_py-2026.5.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c0f920015df2a504bebaba6d4c31ccf3fcf942f92655c086da30b671aad19aa6", size = 405009, upload-time = "2026-05-28T11:59:23.845Z" }, + { url = "https://files.pythonhosted.org/packages/72/e6/4d5718c5cf26c522dc7c9999e238da1e77380b81d0c5d1df11e271ddfeb1/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:0408a24e44feb919423dc6d9da677cb5cddb894d2ca9e763967d156d9c60fab4", size = 553113, upload-time = "2026-05-28T11:59:25.184Z" }, + { url = "https://files.pythonhosted.org/packages/d4/25/2ee807bdb3e1f0b7eddf7782acd5665a8b5205a331a7d7244a52c4812fd9/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:cea68bcd53467561ae2f96a6bdad1544299ba97b5b0ddcd5ac3d376e5c781c24", size = 618838, upload-time = "2026-05-28T11:59:26.749Z" }, + { url = "https://files.pythonhosted.org/packages/6a/c1/7d4c26f167f8c41501cc073d30ee22082b16ce358cf5b00ec97cbc7804ea/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:4be8b1d2a705cc37d08256004e1d07de143fa0075c8e85a3df020b776f62b732", size = 582436, upload-time = "2026-05-28T11:59:28.11Z" }, + { url = "https://files.pythonhosted.org/packages/04/1d/9d12b0a337bab46f4769f8857f4007e3b2d639e14f9a44a0efe157696e64/rpds_py-2026.5.1-cp312-cp312-win32.whl", hash = "sha256:6736718bd4fc49cbcb538ba30516fdbef161522acefb739657d48b97bd864fed", size = 212734, upload-time = "2026-05-28T11:59:29.689Z" }, + { url = "https://files.pythonhosted.org/packages/c5/93/e4116f2de7f56bc7406a76033dc501811ddeb22b7f056b92d632871ebb0c/rpds_py-2026.5.1-cp312-cp312-win_amd64.whl", hash = "sha256:0a7d1eec967df0e9b22614a5e177622e0c89611d03727fa0cb48e45028907870", size = 229045, upload-time = "2026-05-28T11:59:31.033Z" }, + { url = "https://files.pythonhosted.org/packages/cb/53/6c3419d85eb2ec5938a37627c585b42d76a63bb731d6e42ed4b079ebf486/rpds_py-2026.5.1-cp312-cp312-win_arm64.whl", hash = "sha256:1841d067089e117142d79b98aa0df2f08b52f2ecc1819dd2700636c0db74a473", size = 223967, upload-time = "2026-05-28T11:59:32.318Z" }, + { url = "https://files.pythonhosted.org/packages/6c/32/14c961ad295f490eb0849ada8b79683e93a59b9de3afdd983eaf55fa6867/rpds_py-2026.5.1-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:efef4ac29c6ff495531eb17ee705b62841ecaa291b7c7077e848ea03e237164d", size = 352787, upload-time = "2026-05-28T11:59:33.655Z" }, + { url = "https://files.pythonhosted.org/packages/ca/bb/d1b85117967c11191441a7274ae616c65d93901d082c588f89a50a8da5ae/rpds_py-2026.5.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c39f5b67a8a2e67179ada2a954227d670fe65fa9098457f698f56ddf248709b3", size = 345179, upload-time = "2026-05-28T11:59:35Z" }, + { url = "https://files.pythonhosted.org/packages/7c/46/d84105f062e626a1b233f863907288a4708c2d833b8b4c6fb2764bc080c0/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b5c30f3f04eef4fbd362226a6f31d7c8895ca4fbb6e0b790f6890a98d8da8559", size = 376173, upload-time = "2026-05-28T11:59:36.43Z" }, + { url = "https://files.pythonhosted.org/packages/e2/ae/469d7959ce5b1201e1de135dc735b86db3b35dd0d1734f6a44246d5f061c/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:277f6c82f0580848796c7ecc8a7173aa3bfb928e4ff831261c2f60a81dc270db", size = 383162, upload-time = "2026-05-28T11:59:37.995Z" }, + { url = "https://files.pythonhosted.org/packages/dc/a2/57853d31a1116a561aa072794602ad3f6341e18d70a8523f1bd5b9fc1e5a/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:63c2c4c213f1a4e3f3de28ecab029dbdee976324e729c0d7a55211be72576b02", size = 495093, upload-time = "2026-05-28T11:59:39.453Z" }, + { url = "https://files.pythonhosted.org/packages/99/63/3a8eabcad9314b7daf5c65f451d2c33d989235cd8a5762186cf2c3f5a4f8/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3350ec808fb538fe71a1f94dfaa0e29c598dfad805ce49f0caec5ae3183c652b", size = 389829, upload-time = "2026-05-28T11:59:40.896Z" }, + { url = "https://files.pythonhosted.org/packages/4b/25/05678d97fc25e2622df14dc530fb82023174ecfff6733991ed0d78f167bd/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b1b964e3ab599e718dc46c018d104b1ebc007cbc6567d827c94a687fca56d77e", size = 374786, upload-time = "2026-05-28T11:59:42.626Z" }, + { url = "https://files.pythonhosted.org/packages/88/d1/8c90b6431e80a3b91b284a5c7c8c0c4f9c006444d90477a740d6e0f9c694/rpds_py-2026.5.1-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:19cb09fab7b7fc96b2a6e28f2e34b72a3705ff27b37edb77455316e5d3f3dc9b", size = 386920, upload-time = "2026-05-28T11:59:44.124Z" }, + { url = "https://files.pythonhosted.org/packages/ff/99/4638f672ab356682d633ee0da9255f5b67ce6efd0b85eb94ad3e255e65a5/rpds_py-2026.5.1-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:abe76bcdba31e576cb83eeb8797aa0d882b738fef6dc65d0601fc753806a5b46", size = 405059, upload-time = "2026-05-28T11:59:47.177Z" }, + { url = "https://files.pythonhosted.org/packages/66/3f/3546524b6eb4cc2e1f363a3d638fa52f6c24faae3500c25fb488b02f1740/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:8bff7073db3899158fff55ebf57b113a67030af26f80a18978f9f0aa60250ddf", size = 553030, upload-time = "2026-05-28T11:59:48.603Z" }, + { url = "https://files.pythonhosted.org/packages/c6/c3/7b3388c796fcf471bd17194242d4dc1a7608567c0fa422bcc1c5e79f9c1e/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:8ba264fa49be666cd9cc56bf34ec7002fb3d27a4aee5bcb4d43d0d18feb1bb6f", size = 618975, upload-time = "2026-05-28T11:59:50.314Z" }, + { url = "https://files.pythonhosted.org/packages/61/1e/a3cb07f2795075d1d88efddae2f541359fde5f08c81ee114c29c2949c90a/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:4860b603ddda0475a8885499b3729e90229d480105b42651962a5397d995fa89", size = 581178, upload-time = "2026-05-28T11:59:51.673Z" }, + { url = "https://files.pythonhosted.org/packages/a1/74/e758c03a5ef46f04c37f2651a2893db846d569ba8a7bca469d4b58939bcd/rpds_py-2026.5.1-cp313-cp313-win32.whl", hash = "sha256:7944270ae71383f6e2657dd7d5ce4eeb4ac2d0059a6738f0510583d462ab4842", size = 212481, upload-time = "2026-05-28T11:59:53.148Z" }, + { url = "https://files.pythonhosted.org/packages/70/ec/a2aca432db9c7359b40fa393eeeaa0d166c2f70175be956e75fa24197c44/rpds_py-2026.5.1-cp313-cp313-win_amd64.whl", hash = "sha256:88647f43a73c4e01be19b04ceef0c8d3a1958153604d13c773becd8016f2a0cf", size = 228519, upload-time = "2026-05-28T11:59:54.505Z" }, + { url = "https://files.pythonhosted.org/packages/29/60/a73bfdd45b096574556acf303bbd9fa9eed36ca8a818b514e2a5d5fe2b9d/rpds_py-2026.5.1-cp313-cp313-win_arm64.whl", hash = "sha256:453895624ecf7db7063b1004e44037522bbaef9ff6a945e59bc71662d7a03abd", size = 223446, upload-time = "2026-05-28T11:59:56.081Z" }, + { url = "https://files.pythonhosted.org/packages/18/e2/408105fd611823f00882aea810f3989a30d26b1bab8b6beb20f98c724e0e/rpds_py-2026.5.1-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:b4e4bc98639ec915f512fde3aa7a95e0041d95d9c3cc86eea841fa63cb1e8600", size = 355287, upload-time = "2026-05-28T11:59:57.448Z" }, + { url = "https://files.pythonhosted.org/packages/8d/58/5c4a43436843c90d0f6d19f82c200c80e3843ca9fa07b237623327f6d384/rpds_py-2026.5.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:cacedb7a6e167680acba45ad5716e89067d225dc80da0d7040cae8c81d4572fa", size = 347033, upload-time = "2026-05-28T11:59:58.881Z" }, + { url = "https://files.pythonhosted.org/packages/fb/c2/1a71acdacaf4e259b10278fb87b039ded3cf80041bcd89dd8a3ea702ded6/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:68700371c5d7ae1412862ddfa719090925c93ecf351c566d66f09d04b136ea00", size = 376891, upload-time = "2026-05-28T12:00:00.516Z" }, + { url = "https://files.pythonhosted.org/packages/c2/c8/535f3d9b65addd8e28aa87b83c6e526799c3717a88273db8ea795beeef7a/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:296c799becfa849c779c8725494fe9ed94959ed886787df4364b058465bad7f0", size = 385646, upload-time = "2026-05-28T12:00:02.394Z" }, + { url = "https://files.pythonhosted.org/packages/1c/91/dc033f313345c354ade914dbe73cdb90b615a4409ea02430d5356794f3d8/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d3858b908218ee108d0bbfb2095ccc237648053c9bf98affad7cb079acaf1d97", size = 498830, upload-time = "2026-05-28T12:00:04.189Z" }, + { url = "https://files.pythonhosted.org/packages/27/fc/90fcbea459dbb8ddc18a2e0fd1de9412b48bc84ffff2db771cf714bacfd6/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4fb8d2e7cb2f850b169806d61d1b991738acec96500a75c30f49caf064ce7cef", size = 392830, upload-time = "2026-05-28T12:00:05.797Z" }, + { url = "https://files.pythonhosted.org/packages/b2/1d/46cd11a228c9750684a798d98f878be6f614aa762438da7378f035e79e35/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:27b74c10ed6a8f190f4287f53bcfea348b92a84a9c9f70d30183d1e6172d580d", size = 379613, upload-time = "2026-05-28T12:00:07.433Z" }, + { url = "https://files.pythonhosted.org/packages/24/4a/d9b0c6af3a1de03eb93741bbe8be2bdce84d8fda8224f3005451d86df389/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_31_riscv64.whl", hash = "sha256:b9a6528956191c48c52294a592dbd4a8386d7048bdb25c0efcb6b966466c6d83", size = 388183, upload-time = "2026-05-28T12:00:09.227Z" }, + { url = "https://files.pythonhosted.org/packages/c5/b4/db7aaabdda6d020afc87d981bcc2f57a434c7dec60ecfc2ab3dd50b20351/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:af03e34e860047bc7a352b842856fcf78798fbb81132cc98bd2f907ab4eb9cd2", size = 408578, upload-time = "2026-05-28T12:00:10.779Z" }, + { url = "https://files.pythonhosted.org/packages/08/d6/070f6a41cbb343e2ac4171859bf3f3623e0ab002f72619d6d505313ec2de/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:fea6e836d10abbe191d557d33bd58bd5987725fe63aa1eefe557d230209855bd", size = 553573, upload-time = "2026-05-28T12:00:12.443Z" }, + { url = "https://files.pythonhosted.org/packages/75/ab/1a71ea3589c4345dac0a0518f0e6a031cb42689277851b683c46d27463a5/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:fc0c0f878ea770a0a8a462456c5ad36fc9fe6358e6b76fdadc7f17575e0b8bf1", size = 620861, upload-time = "2026-05-28T12:00:14.09Z" }, + { url = "https://files.pythonhosted.org/packages/8a/22/9bf80a56069c0c443fcfefac639a86a744550a2898817a6dfd3e26654924/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:e0b360f316d966b048b085857630b3cc51f3db2f07b06f440eac8f695374d1e3", size = 585633, upload-time = "2026-05-28T12:00:15.66Z" }, + { url = "https://files.pythonhosted.org/packages/da/68/3b2c0a75c9e04125696f84ebdbbf304acf5a40b58ba4481cdb98a922c3ba/rpds_py-2026.5.1-cp313-cp313t-win32.whl", hash = "sha256:a2999883eedf72fdfb7520b92c7d4ec2572a71ff40239377aa604cc529eecafc", size = 210074, upload-time = "2026-05-28T12:00:17.291Z" }, + { url = "https://files.pythonhosted.org/packages/e7/8b/609157d5a25d37d4f29f92840ba531f416907c34ae5c5739dd21fc2bef98/rpds_py-2026.5.1-cp313-cp313t-win_amd64.whl", hash = "sha256:e07be2a9d7122bd6e82dea89814ef8dc893feb1aae97fec1630f3263bbb30e55", size = 228635, upload-time = "2026-05-28T12:00:18.73Z" }, + { url = "https://files.pythonhosted.org/packages/d4/6f/19c1918a4b590d8de87e712e4abe4b3875771eff60216fb6153cf6665c68/rpds_py-2026.5.1-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:1f2c391c3059798093b65df23aca2cac150460ae9c630d99dec83d703d9485b9", size = 349756, upload-time = "2026-05-28T12:00:20.217Z" }, + { url = "https://files.pythonhosted.org/packages/e5/60/a06fe7da34eca79dacbf958a2ba0c6eea85bc2b29de20080bf40f72f66fa/rpds_py-2026.5.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:413b424f7c4ee65ab5e5be91f5731be0f8b41a1ee2b12dfe810d716312e95a78", size = 343831, upload-time = "2026-05-28T12:00:21.711Z" }, + { url = "https://files.pythonhosted.org/packages/bf/ec/b2333b97b90e2a6ef6ca8ad386ee284968e74bcfe113b3f1a8d9036429a9/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2c595a1d9255dce0599e13130d1440ab2506654f2b50294226ee06402f8fef63", size = 375127, upload-time = "2026-05-28T12:00:23.326Z" }, + { url = "https://files.pythonhosted.org/packages/14/7f/e00aae54067f2b488c4637961d5f58204d470795fc791085fa3f15060d2e/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:1c27c5f6102eac8c03e7595a00827a53b271ba40a53b59ff8709170e0855ea4a", size = 379034, upload-time = "2026-05-28T12:00:24.89Z" }, + { url = "https://files.pythonhosted.org/packages/be/cc/423999bbb8ae8dc93c77fc1d5e984ade5eb89d237d3bb884ccfa72ae2890/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6c7fcf61d44cacecaf3aea542b0e053db77972a4573e7ceda16fb2b399161195", size = 490823, upload-time = "2026-05-28T12:00:26.676Z" }, + { url = "https://files.pythonhosted.org/packages/0f/aa/c671bf660f12e68d3c52ff86c7066ed1372df5a0f4f2ff584e419b8207e7/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2c817a189d4ee14290420e5ff051e4dd6baa13f3edf84685071dee07a6d538ee", size = 388144, upload-time = "2026-05-28T12:00:28.577Z" }, + { url = "https://files.pythonhosted.org/packages/19/c8/d63bb75b68afe77b229e3021c6031bcaf01da5db5b0e69d0d10f9ba679a7/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:21846aac0ed2e0589f38c12dc44e77bb64e494b771eadbcf169cba00566ba7ba", size = 371959, upload-time = "2026-05-28T12:00:30.304Z" }, + { url = "https://files.pythonhosted.org/packages/82/35/c51122014d8274ff37dc606d60049c3db7d83da02b5b282511e5a906a9a6/rpds_py-2026.5.1-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:b317c87a13f769a4e787819bd508aaa5d69aa09b0880de9af6d3a8a54571cdec", size = 383558, upload-time = "2026-05-28T12:00:31.764Z" }, + { url = "https://files.pythonhosted.org/packages/e3/f9/2790cb99c136a5363acdeacf5c27c56f3de0d4118a1f48fca83404c99c89/rpds_py-2026.5.1-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ce87129d9f2c14fa6c4a8601fb80eb4488c80d38a20cd13758ef11123e14995d", size = 402789, upload-time = "2026-05-28T12:00:33.247Z" }, + { url = "https://files.pythonhosted.org/packages/e5/1b/e4fb584f8c75d35c38150ff6a332cda949e6f97acba1f4fd123b14ab56fe/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:9cdddb6c1207d284d94fd1530adf57fbd797fe7c4b8704ba85f49414f2557e7d", size = 551405, upload-time = "2026-05-28T12:00:34.819Z" }, + { url = "https://files.pythonhosted.org/packages/d8/f7/a6731b4216cb3793ea1af5391da240f5683dacc0d13e034fe5fc3503f240/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:4e237e139f94d3c036fd28eb9f564c99055476ff4ff05cd42be55ce349b5aa02", size = 616975, upload-time = "2026-05-28T12:00:36.268Z" }, + { url = "https://files.pythonhosted.org/packages/2c/ea/2e051a81d95d8e63f4b35a1c463a87e8766bc3d083c067c5dfb6bf220747/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ed0954b524873214369184a9c82b0eaa45a3fbb9a798cd95b17e0d98499e7ea0", size = 578701, upload-time = "2026-05-28T12:00:37.82Z" }, + { url = "https://files.pythonhosted.org/packages/65/56/b5f6fdb2083e32bca8a8993d89e70db114b4756c9e2c38421328126689d2/rpds_py-2026.5.1-cp314-cp314-win32.whl", hash = "sha256:2d88621d6a7d4dfa633d21abe90f280bb205274e16b1d1e61c6ad4640b2453b7", size = 209806, upload-time = "2026-05-28T12:00:39.492Z" }, + { url = "https://files.pythonhosted.org/packages/fb/80/65a5aa96c155e611d1ed844e4e1f57f3e36b021f396d9f8585d756e6b90d/rpds_py-2026.5.1-cp314-cp314-win_amd64.whl", hash = "sha256:cef8ac28d26f4dda3533060c20fbf80a325458fa9fd23ea72a73cdfa8e978838", size = 225985, upload-time = "2026-05-28T12:00:40.94Z" }, + { url = "https://files.pythonhosted.org/packages/27/7c/ad185212e87b05f196daef92bc5f3caf07298eb47c295b5585c3dd3093ac/rpds_py-2026.5.1-cp314-cp314-win_arm64.whl", hash = "sha256:eaaea962c68cdc68d4a533ba985ab8e9484277910bbfaa2ab3ef7732667bfed8", size = 221219, upload-time = "2026-05-28T12:00:43.15Z" }, + { url = "https://files.pythonhosted.org/packages/23/58/e14ae18759020334646b031e708ab4158d653a938822bfb7b95ef2e93aa3/rpds_py-2026.5.1-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:21942f52dbbd5f8758bf021213d28bd45c39e873e65e2407faf5f1846f5761ad", size = 352148, upload-time = "2026-05-28T12:00:44.638Z" }, + { url = "https://files.pythonhosted.org/packages/31/9b/5f4a1e2f960bca3ac5d052b139dd31eed97b259f9d909173821760d542e8/rpds_py-2026.5.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f414556f6e3958300ff941e40c9f97e3dc9774ddd1b3434c475d73dd354bbed3", size = 345196, upload-time = "2026-05-28T12:00:46.14Z" }, + { url = "https://files.pythonhosted.org/packages/1a/71/1d9574d6a2fa20ab60eaa55c7467f5aa20cbc770f341a05f09c0876f59e2/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ef1013a8625c74043210190b246f5b1551e09757c1f356c6e4160ef96c5bc081", size = 374981, upload-time = "2026-05-28T12:00:47.531Z" }, + { url = "https://files.pythonhosted.org/packages/0c/9a/37e99f4915a80aa71670263c1267f7ae0af95f53a3f61e6c3bdc016d4515/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:cc68e231a77a5f0d774ae278a1f8e55c0456501820847c1e4efb3829f3441df6", size = 379961, upload-time = "2026-05-28T12:00:49.216Z" }, + { url = "https://files.pythonhosted.org/packages/a8/ff/6e73f74b89d2e0715e0fc86b7dde893f9a61ae2f9b256ff3bdfe41ac4e94/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9baffb505aff33acc69b422a19f77806680f3c8632227d79f48de8a810d1c2c5", size = 495965, upload-time = "2026-05-28T12:00:51.111Z" }, + { url = "https://files.pythonhosted.org/packages/ea/e0/425faba25f59d74d4638b267f7c7a80e8649d2ef4db10a19b0c4a71e6e6f/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b8d2f912928d426e8cfa396f7f3f8d29a59e6689c86dcca3c420730c1096322b", size = 389526, upload-time = "2026-05-28T12:00:52.77Z" }, + { url = "https://files.pythonhosted.org/packages/c6/76/7a41960e3fddae47fab43a28684d5da981401dffd88253de0944148654cb/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:90f628283be835db980c941767d41c9a27b5239e54ba0a9c1335247e82406964", size = 376190, upload-time = "2026-05-28T12:00:54.215Z" }, + { url = "https://files.pythonhosted.org/packages/27/60/5f38dc70824fc6951b51d35377e577a3a3a4c81a6769cc5a2de25ebe0ad1/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:1ebb2f0ab7e16132995a72de805170e0203df0c3dd22e1ef1cd1fdd90bd7a131", size = 383921, upload-time = "2026-05-28T12:00:55.673Z" }, + { url = "https://files.pythonhosted.org/packages/60/1a/d60a38caa1505f4b9483c3fbbde12c94e1079154f4f401a6da96f7e77621/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:f3df3d16ded76f1f8c9cdebd0e1ea55fdf4c23b812de189814da7cf229c22a81", size = 404766, upload-time = "2026-05-28T12:00:57.518Z" }, + { url = "https://files.pythonhosted.org/packages/87/ff/602fd3f174d6425f0bce05ad0dfbec0e96b38d0f7d08a79af5aa20083885/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:9af8905b8f854990e40d5206aa5ac58d9b0fe0b7f351ff2bb086c20f6c8c6a47", size = 551343, upload-time = "2026-05-28T12:00:58.978Z" }, + { url = "https://files.pythonhosted.org/packages/b8/c1/1be13327acdbead3eca1fde03b6a34dbb011f1e864e217f0d32cc1779a7f/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:036a36a87fb1cd3b214d11c4b3c4f7d2ddad933625dca1c900b56a057c07740a", size = 618502, upload-time = "2026-05-28T12:01:00.656Z" }, + { url = "https://files.pythonhosted.org/packages/f3/d7/afb49b49d7f2be8b7ba1a9f0977fa5168003437b93086726f066544e8351/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:62ae3853454fe9ef283a03c96c2d835d39e84b14643a9d62c82ef0fb87d702ca", size = 581916, upload-time = "2026-05-28T12:01:02.22Z" }, + { url = "https://files.pythonhosted.org/packages/25/d1/dbef8c1f8a10f07beb62b5f054e20099fd9924b3ec001b8f0b6ac7813a85/rpds_py-2026.5.1-cp314-cp314t-win32.whl", hash = "sha256:6c3d771a46ec18b12af06ce36243a9a80b07a5d0515236332d90863ca8bb326a", size = 207855, upload-time = "2026-05-28T12:01:03.821Z" }, + { url = "https://files.pythonhosted.org/packages/2a/72/bfa4e61ab8e7dc1c8adf397e05e6cbdd4239357bd72b248d3de662f23915/rpds_py-2026.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:c93c629be4636cf54337bd5f06c104d55e42ced54d681f6fe21ae510a65116f6", size = 225422, upload-time = "2026-05-28T12:01:05.194Z" }, + { url = "https://files.pythonhosted.org/packages/27/3a/7b5da92b640f67b6717ccafc83cdd06bfa7ff2395c3685c68922bb54d703/rpds_py-2026.5.1-cp315-cp315-macosx_10_12_x86_64.whl", hash = "sha256:3574b55c604b8f75dacb007136508bbc0db406e626301778096a133327e7f2fb", size = 349576, upload-time = "2026-05-28T12:01:06.722Z" }, + { url = "https://files.pythonhosted.org/packages/d7/8a/2aafd7ad355a1bd48ca76e2262b74b15e6432b5a1efe150efd4d779cd55d/rpds_py-2026.5.1-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:94068eb3ae6d43f5a786b7db96a406a34e6d5c24489feef32fd6e8946ea7b291", size = 343640, upload-time = "2026-05-28T12:01:08.441Z" }, + { url = "https://files.pythonhosted.org/packages/f7/7d/6c9523c1abbe840a1b7fba3c516d48e1d3487cc80fea4366c4071cf56784/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f3a5b10e8ce894825f380a8f1b6444cf73c294dfea62afbb2d13e3a9e630cec1", size = 375322, upload-time = "2026-05-28T12:01:09.934Z" }, + { url = "https://files.pythonhosted.org/packages/5a/5d/0b7b03fb1dc509321f01de3149784ab773e34c8573022029af8076afcb9c/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:fc09f82e63d4bcd58149572f857a431bae851dc747e313c3b5bdf7abb907fda8", size = 379066, upload-time = "2026-05-28T12:01:11.48Z" }, + { url = "https://files.pythonhosted.org/packages/d7/e2/8ef6012999ebf1cb1c22f876d9ce5e63d960fd4631d2af3202d3f480aa25/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e10464d17df3b582745c25cec695cb9558bca2cb6ddb631aee1787fc72c767b2", size = 494586, upload-time = "2026-05-28T12:01:13.051Z" }, + { url = "https://files.pythonhosted.org/packages/80/af/1eeb029bec67582c226b7809172207cd005073af4ebd906e65ff494f4983/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ba05adbf15d994c38ec0b7ab32e858e5110c21e9009a00a86545fd220f84e038", size = 388415, upload-time = "2026-05-28T12:01:14.631Z" }, + { url = "https://files.pythonhosted.org/packages/18/23/ffbe10711c4d766c1cab0557d6906c074f795814863c67b351355d29354a/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:77c004fdc7b891967106f78ddfd7b076bfe6813c6139c6fff6aed3bcaa960b26", size = 372427, upload-time = "2026-05-28T12:01:16.153Z" }, + { url = "https://files.pythonhosted.org/packages/bd/3a/30ba4a6ad457e5b070c18d742a33fb77d8d922b565cc881f8a5313d63bfe/rpds_py-2026.5.1-cp315-cp315-manylinux_2_31_riscv64.whl", hash = "sha256:83bcf894486c9d78dd290d3c0124ff6dd8875d3025e2090a8ec49fcc37c55fdd", size = 383615, upload-time = "2026-05-28T12:01:17.809Z" }, + { url = "https://files.pythonhosted.org/packages/d3/69/62e242b53ce39c0814bd24e1a6e6eba6c92be716277745f317f9540a2e7b/rpds_py-2026.5.1-cp315-cp315-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c3df104083952a0e0c6f10de33e440eabe98fb6317d23e1a58c68f6df08d01b9", size = 402786, upload-time = "2026-05-28T12:01:19.419Z" }, + { url = "https://files.pythonhosted.org/packages/38/c1/a770b9c186928a1ed0f7e6d7ae50e7f3950ed23e3f9e366dbc8e38cb55de/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:980450826cf22e133c57e0835070bdd0dd3f73b9b708c3ce223def2cb9469e14", size = 551583, upload-time = "2026-05-28T12:01:21.013Z" }, + { url = "https://files.pythonhosted.org/packages/21/7c/68e8579b95375b70d2a963103c42e705856cdb98569258bd807f4423891c/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_i686.whl", hash = "sha256:205dde846f24332ab0c1188699a043b8d165b79bb84529ce272c45048ff6be01", size = 616941, upload-time = "2026-05-28T12:01:22.548Z" }, + { url = "https://files.pythonhosted.org/packages/70/a1/a6135aed5730ff03ab957182259987ac11e55fb392a28dc6f0592048a280/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:3966b82dd563176396df030f3dd52a6e54cb69b718e95e78bd555ed3d1e0185d", size = 578349, upload-time = "2026-05-28T12:01:24.118Z" }, + { url = "https://files.pythonhosted.org/packages/09/6e/f24201a76a84e6c49d0bdfdfcb735210e21701e9b21c5bfc0ba497dd62f6/rpds_py-2026.5.1-cp315-cp315-win32.whl", hash = "sha256:7818f8d0a415be74d2be3590b0a1c1f463a642f4d0217e7d10602dceef5b79aa", size = 209922, upload-time = "2026-05-28T12:01:25.522Z" }, + { url = "https://files.pythonhosted.org/packages/9e/e4/966bc240bb0485fc265278f6de44d05834bf0b3618886e0b22e33d54c49a/rpds_py-2026.5.1-cp315-cp315-win_amd64.whl", hash = "sha256:b3cc20c0d800af78fd0fac68086e28c1856cec51ea528bb81ea851aa40d39325", size = 226003, upload-time = "2026-05-28T12:01:27.062Z" }, + { url = "https://files.pythonhosted.org/packages/5c/5c/a15a59269cd5e74472734516c73795c15eccfc841b3d4b0228c3f53f19d0/rpds_py-2026.5.1-cp315-cp315-win_arm64.whl", hash = "sha256:3609e9939a8a76cd904cf98a3f1f13b5dc7e150adeaee89e0ea09652ea213e16", size = 221245, upload-time = "2026-05-28T12:01:28.51Z" }, + { url = "https://files.pythonhosted.org/packages/e0/22/135ce03804e179a71ceb13be095deda4a279bc88f7a6b8fa161c5ad44e12/rpds_py-2026.5.1-cp315-cp315t-macosx_10_12_x86_64.whl", hash = "sha256:5d333a7127d4b307601ac37792bee01bb95c867cbfacf21b6375b804d6bbd723", size = 352015, upload-time = "2026-05-28T12:01:30.214Z" }, + { url = "https://files.pythonhosted.org/packages/3b/5f/f1f6d2652eb9d848f6eb369d8db83a2da6249bb49ad2c2a48f45d54538d3/rpds_py-2026.5.1-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:b5f077b44a4f7808520f66dae234988d867deb9aed9be5da057ce9ba831b2a41", size = 345016, upload-time = "2026-05-28T12:01:31.656Z" }, + { url = "https://files.pythonhosted.org/packages/88/66/b74182775691ea2290c99e52ac8d5db844e56fbec90ce421f107658c8314/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:55d8f9b7b78c9538fc9e04e82ec0e888ff0c3cffcfad152c77e57cd09351a98a", size = 374775, upload-time = "2026-05-28T12:01:33.136Z" }, + { url = "https://files.pythonhosted.org/packages/ff/8f/15e5a61d9f0a43902d36561d4f07cae6ae9f4716be825159fd72717f33af/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e3a8ae58895ac107ed934a6bf51e5846f95c53b9b940c2c6d310838fd5846358", size = 380270, upload-time = "2026-05-28T12:01:34.574Z" }, + { url = "https://files.pythonhosted.org/packages/02/c3/f859b12763a80540cdf2af0f15b19904cf756a71d7bdd3f82ff3e5b1bbf9/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0957cf3c2b8632ec7aaebffebea8005b353cc2a237b6e2ae3c2cac0820704cfb", size = 495285, upload-time = "2026-05-28T12:01:36.127Z" }, + { url = "https://files.pythonhosted.org/packages/1c/c7/ff27c2ac8411d30b03b1829fd88cae8dad1a4d0da48dd25e57c4038042e6/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c396c1304de421050b3681ea70f371874b54d41b0151e96109758144c231e30b", size = 389581, upload-time = "2026-05-28T12:01:37.635Z" }, + { url = "https://files.pythonhosted.org/packages/6e/67/fe92ee32a6cc05c77228a2f8b1762e7124f386ec20ff83d0757b762d58d0/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aad1bff7f666b9598e573815affd666aac6a13a585dde336f843e33350c7fadc", size = 376041, upload-time = "2026-05-28T12:01:39.307Z" }, + { url = "https://files.pythonhosted.org/packages/f8/91/b4d6685c27aba55bd82f25b278be8237038117d05f9659a6213ad3408130/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_31_riscv64.whl", hash = "sha256:656a042550878f12d45752452d47094b7cfe5ad1e9d7b87b5a22ad3ae5ff8015", size = 383946, upload-time = "2026-05-28T12:01:41.043Z" }, + { url = "https://files.pythonhosted.org/packages/bd/79/2c1d832a53c8e0f8e98fc970ec257b950fecd4f62be2ab7182b500a0cbc8/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:73c4bd4f70294737b5206a3e8e30ccadbf8a60301831c8ea23eec5dbeea1ecfa", size = 405526, upload-time = "2026-05-28T12:01:43.032Z" }, + { url = "https://files.pythonhosted.org/packages/78/c4/c98117b03c6a8581ab2c2dfccfe9a5ad82bd8128a3c28b46a6ad2d97c393/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:43bca78665423cabae77146f2fe7ce55272b6c8d55d82cca83effd42c7e13972", size = 551165, upload-time = "2026-05-28T12:01:44.648Z" }, + { url = "https://files.pythonhosted.org/packages/3b/c1/bc479ca069200af730881b1bd525e3114b2b391a351509fcb1b772f28086/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_i686.whl", hash = "sha256:42d0f20e85e549c870749d0e247f0c10d318a45b7e9676d575d2dcb04a1b2e66", size = 618778, upload-time = "2026-05-28T12:01:46.337Z" }, + { url = "https://files.pythonhosted.org/packages/77/65/38ab2f90df44c2febfb63cc10ced40763d9b4bc94d173e734528663fe7f5/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:b1be5c35683684d5331b93600c210e8367c254683d8a6df6bd21bd2da3a334fb", size = 581839, upload-time = "2026-05-28T12:01:48.109Z" }, + { url = "https://files.pythonhosted.org/packages/15/2d/ce1f605fe036aadd460e5822e578c6c7ec3a860936cca37d6e0f299daa77/rpds_py-2026.5.1-cp315-cp315t-win32.whl", hash = "sha256:75808f6c38ce7749bb68cc2770161aae5045e6c6f6781a9782e74b93304399df", size = 207866, upload-time = "2026-05-28T12:01:49.648Z" }, + { url = "https://files.pythonhosted.org/packages/79/cb/966040123eb102371559746908ef2c9471f4d43e17ec9a645a2258dab64b/rpds_py-2026.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:90bd6630002a1c7f09e7843dd79f0d24f3d2897cc25a753480917865d14f15b3", size = 225441, upload-time = "2026-05-28T12:01:51.408Z" }, + { url = "https://files.pythonhosted.org/packages/42/56/3fe0fb34820ff667be791b3a3c22b85e8bcba54e9c832f47438c191fa7be/rpds_py-2026.5.1-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:edf2765d84e42447f112ad877af8fe1db0089aaec5b28e88d6eab45e7fe99cea", size = 357151, upload-time = "2026-05-28T12:01:53.43Z" }, + { url = "https://files.pythonhosted.org/packages/8b/f2/3eb9ccdb9f143b8c9b003978898cb497f942a324c077401e6b8834238e63/rpds_py-2026.5.1-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:ad3773236e95f7f33991eb125224b7da66f206504d032a253a02da7e134519fb", size = 350195, upload-time = "2026-05-28T12:01:54.901Z" }, + { url = "https://files.pythonhosted.org/packages/a7/24/dbda232bc4f3ed732120692ab0d2c8402cb020516556d8bee622dcef2413/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a04df86b3f0fade39ec8fd0e0aab089b1da9fbd2b48df778a57ef96f5e7d38df", size = 381850, upload-time = "2026-05-28T12:01:56.601Z" }, + { url = "https://files.pythonhosted.org/packages/40/30/32e769839a358f78810c234f160f2cc21d1e4e47e1c0e0e0d535be5a0219/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:6142dbd80c4df62a5d899f0d616d417f84e0bc8d32526c8e5589019d75d028a7", size = 387899, upload-time = "2026-05-28T12:01:58.212Z" }, + { url = "https://files.pythonhosted.org/packages/ab/86/ec84d243aadb3b34b71dd26a010d0930b2d284ff5fc9a69fec53810ee6fd/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0b35217adefe87f2fe4db7e9766cabe84744bfe9616d9667be18988928c7f2dc", size = 501618, upload-time = "2026-05-28T12:01:59.888Z" }, + { url = "https://files.pythonhosted.org/packages/74/25/b60e52686bbff777a64f9e4f4d3dd57980dc846913777177a2c92e4937aa/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b95d5e11fc712b752081183a55a244c03cd00570489edd7014d8899f8ceb8162", size = 394003, upload-time = "2026-05-28T12:02:01.482Z" }, + { url = "https://files.pythonhosted.org/packages/9b/c7/b3a6a588cc2219510ef3f42e207483a93950bedd1e3a0fd4015c95cff9e5/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:141c9498daf2ace9eda35d2b0e376f9ea8b058d84f2aef4f96fccfd449a2f251", size = 379778, upload-time = "2026-05-28T12:02:03.197Z" }, + { url = "https://files.pythonhosted.org/packages/31/00/c7dba3fc8a3da8cb3f6db1eb3386be4d79c2e97c6890d20eb9ac66ae8c43/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_31_riscv64.whl", hash = "sha256:6f249f8b860a200ad35193af961183ebe9132710484e6f6ce0cf89fd83c63a9a", size = 392359, upload-time = "2026-05-28T12:02:04.817Z" }, + { url = "https://files.pythonhosted.org/packages/93/dd/472ba494c70753f93745992c99855bee0636daf74e6984e5e003f150316f/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e4abbf391a70be864920858bf360f4fb380577c9a0f732438a1996726e2c195b", size = 412820, upload-time = "2026-05-28T12:02:06.401Z" }, + { url = "https://files.pythonhosted.org/packages/1d/6f/93831a3bfe789542ed0c1d0d74b78b440f055d6dc3ea4640eba2d95e6e23/rpds_py-2026.5.1-pp311-pypy311_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:c74005a7bb87752acf351c93897ec63ad77a07a0da7ecad9c050e32e7286ba34", size = 557243, upload-time = "2026-05-28T12:02:08.013Z" }, + { url = "https://files.pythonhosted.org/packages/1f/ff/0b3d604614ffc77522c6b288fdbce68957eb583da1002aa65ba38ac0ee40/rpds_py-2026.5.1-pp311-pypy311_pp73-musllinux_1_2_i686.whl", hash = "sha256:8213afbe8a3a906fb9acb2014423fe3359ee783d0bf90995f70623a3217bfa6c", size = 623541, upload-time = "2026-05-28T12:02:09.661Z" }, + { url = "https://files.pythonhosted.org/packages/ea/ea/e7b0251441da9adfeaebcf29601d10f2a1455fcf0772fae9e7e19032bd96/rpds_py-2026.5.1-pp311-pypy311_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:8c43a8a973270fd173bf48cdf80bbe66312421cba68d40845034f174f2389049", size = 586326, upload-time = "2026-05-28T12:02:11.47Z" }, +] + [[package]] name = "ruff" version = "0.15.6" @@ -1399,15 +2254,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7f/d0/578c47dd68152ddddddf31cd7fc67dc30b7cdf639a86275fda821b0d9d98/ruff-0.15.6-py3-none-win_arm64.whl", hash = "sha256:c34de3dd0b0ba203be50ae70f5910b17188556630e2178fd7d79fc030eb0d837", size = 11060497, upload-time = "2026-03-12T23:05:25.968Z" }, ] -[[package]] -name = "setuptools" -version = "75.8.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/92/ec/089608b791d210aec4e7f97488e67ab0d33add3efccb83a056cbafe3a2a6/setuptools-75.8.0.tar.gz", hash = "sha256:c5afc8f407c626b8313a86e10311dd3f661c6cd9c09d4bf8c15c0e11f9f2b0e6", size = 1343222, upload-time = "2025-01-08T18:28:23.98Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/69/8a/b9dc7678803429e4a3bc9ba462fa3dd9066824d3c607490235c6a796be5a/setuptools-75.8.0-py3-none-any.whl", hash = "sha256:e3982f444617239225d675215d51f6ba05f845d4eec313da4418fdbb56fb27e3", size = 1228782, upload-time = "2025-01-08T18:28:20.912Z" }, -] - [[package]] name = "six" version = "1.17.0" @@ -1417,15 +2263,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" }, ] -[[package]] -name = "snowballstemmer" -version = "2.2.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/44/7b/af302bebf22c749c56c9c3e8ae13190b5b5db37a33d9068652e8f73b7089/snowballstemmer-2.2.0.tar.gz", hash = "sha256:09b16deb8547d3412ad7b590689584cd0fe25ec8db3be37788be3810cbf19cb1", size = 86699, upload-time = "2021-11-16T18:38:38.009Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/ed/dc/c02e01294f7265e63a7315fe086dd1df7dacb9f840a804da846b96d01b96/snowballstemmer-2.2.0-py2.py3-none-any.whl", hash = "sha256:c8e1716e83cc398ae16824e5572ae04e0d9fc2c6b985fb0f900f5f0c96ecba1a", size = 93002, upload-time = "2021-11-16T18:38:34.792Z" }, -] - [[package]] name = "soupsieve" version = "2.6" @@ -1435,135 +2272,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d1/c2/fe97d779f3ef3b15f05c94a2f1e3d21732574ed441687474db9d342a7315/soupsieve-2.6-py3-none-any.whl", hash = "sha256:e72c4ff06e4fb6e4b5a9f0f55fe6e81514581fca1515028625d0f299c602ccc9", size = 36186, upload-time = "2024-08-13T13:39:10.986Z" }, ] -[[package]] -name = "sphinx" -version = "8.1.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "alabaster" }, - { name = "babel" }, - { name = "colorama", marker = "sys_platform == 'win32'" }, - { name = "docutils" }, - { name = "imagesize" }, - { name = "jinja2" }, - { name = "packaging" }, - { name = "pygments" }, - { name = "requests" }, - { name = "snowballstemmer" }, - { name = "sphinxcontrib-applehelp" }, - { name = "sphinxcontrib-devhelp" }, - { name = "sphinxcontrib-htmlhelp" }, - { name = "sphinxcontrib-jsmath" }, - { name = "sphinxcontrib-qthelp" }, - { name = "sphinxcontrib-serializinghtml" }, - { name = "tomli", marker = "python_full_version < '3.11'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/be0b61178fe2cdcb67e2a92fc9ebb488e3c51c4f74a36a7824c0adf23425/sphinx-8.1.3.tar.gz", hash = "sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927", size = 8184611, upload-time = "2024-10-13T20:27:13.93Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/26/60/1ddff83a56d33aaf6f10ec8ce84b4c007d9368b21008876fceda7e7381ef/sphinx-8.1.3-py3-none-any.whl", hash = "sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2", size = 3487125, upload-time = "2024-10-13T20:27:10.448Z" }, -] - -[[package]] -name = "sphinx-autoapi" -version = "3.4.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "astroid" }, - { name = "jinja2" }, - { name = "pyyaml" }, - { name = "sphinx" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/4a/eb/cc243583bb1d518ca3b10998c203d919a8ed90affd4831f2b61ad09043d2/sphinx_autoapi-3.4.0.tar.gz", hash = "sha256:e6d5371f9411bbb9fca358c00a9e57aef3ac94cbfc5df4bab285946462f69e0c", size = 29292, upload-time = "2024-11-30T01:09:40.956Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/de/d6/f2acdc2567337fd5f5dc091a4e58d8a0fb14927b9779fc1e5ecee96d9824/sphinx_autoapi-3.4.0-py3-none-any.whl", hash = "sha256:4027fef2875a22c5f2a57107c71641d82f6166bf55beb407a47aaf3ef14e7b92", size = 34095, upload-time = "2024-11-30T01:09:17.272Z" }, -] - -[[package]] -name = "sphinx-reredirects" -version = "0.1.6" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.11'", -] -dependencies = [ - { name = "sphinx", marker = "python_full_version < '3.11'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/16/6b/bcca2785de4071f604a722444d4d7ba8a9d40de3c14ad52fce93e6d92694/sphinx_reredirects-0.1.6.tar.gz", hash = "sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64", size = 7080, upload-time = "2025-03-22T10:52:30.271Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/ac/6f/0b3625be30a1a50f9e4c2cb2ec147b08f15ed0e9f8444efcf274b751300b/sphinx_reredirects-0.1.6-py3-none-any.whl", hash = "sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea", size = 5675, upload-time = "2025-03-22T10:52:29.113Z" }, -] - -[[package]] -name = "sphinx-reredirects" -version = "1.1.0" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version >= '3.12' and python_full_version < '3.14'", - "python_full_version == '3.11.*'", -] -dependencies = [ - { name = "sphinx", marker = "python_full_version >= '3.11'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/1b/8d/0e39fe2740d7d71417edf9a6424aa80ca2c27c17fc21282cdc39f90d5a40/sphinx_reredirects-1.1.0.tar.gz", hash = "sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29", size = 614624, upload-time = "2025-12-22T08:28:02.792Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/51/81/b5dd07067f3daac6d23687ec737b2d593740671ebcd145830c8f92d381c5/sphinx_reredirects-1.1.0-py3-none-any.whl", hash = "sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9", size = 6351, upload-time = "2025-12-22T08:27:59.724Z" }, -] - -[[package]] -name = "sphinxcontrib-applehelp" -version = "2.0.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/ba/6e/b837e84a1a704953c62ef8776d45c3e8d759876b4a84fe14eba2859106fe/sphinxcontrib_applehelp-2.0.0.tar.gz", hash = "sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1", size = 20053, upload-time = "2024-07-29T01:09:00.465Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/5d/85/9ebeae2f76e9e77b952f4b274c27238156eae7979c5421fba91a28f4970d/sphinxcontrib_applehelp-2.0.0-py3-none-any.whl", hash = "sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5", size = 119300, upload-time = "2024-07-29T01:08:58.99Z" }, -] - -[[package]] -name = "sphinxcontrib-devhelp" -version = "2.0.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/f6/d2/5beee64d3e4e747f316bae86b55943f51e82bb86ecd325883ef65741e7da/sphinxcontrib_devhelp-2.0.0.tar.gz", hash = "sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad", size = 12967, upload-time = "2024-07-29T01:09:23.417Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/35/7a/987e583882f985fe4d7323774889ec58049171828b58c2217e7f79cdf44e/sphinxcontrib_devhelp-2.0.0-py3-none-any.whl", hash = "sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2", size = 82530, upload-time = "2024-07-29T01:09:21.945Z" }, -] - -[[package]] -name = "sphinxcontrib-htmlhelp" -version = "2.1.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/43/93/983afd9aa001e5201eab16b5a444ed5b9b0a7a010541e0ddfbbfd0b2470c/sphinxcontrib_htmlhelp-2.1.0.tar.gz", hash = "sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9", size = 22617, upload-time = "2024-07-29T01:09:37.889Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/0a/7b/18a8c0bcec9182c05a0b3ec2a776bba4ead82750a55ff798e8d406dae604/sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl", hash = "sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8", size = 98705, upload-time = "2024-07-29T01:09:36.407Z" }, -] - -[[package]] -name = "sphinxcontrib-jsmath" -version = "1.0.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/b2/e8/9ed3830aeed71f17c026a07a5097edcf44b692850ef215b161b8ad875729/sphinxcontrib-jsmath-1.0.1.tar.gz", hash = "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8", size = 5787, upload-time = "2019-01-21T16:10:16.347Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/c2/42/4c8646762ee83602e3fb3fbe774c2fac12f317deb0b5dbeeedd2d3ba4b77/sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", size = 5071, upload-time = "2019-01-21T16:10:14.333Z" }, -] - -[[package]] -name = "sphinxcontrib-qthelp" -version = "2.0.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/68/bc/9104308fc285eb3e0b31b67688235db556cd5b0ef31d96f30e45f2e51cae/sphinxcontrib_qthelp-2.0.0.tar.gz", hash = "sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab", size = 17165, upload-time = "2024-07-29T01:09:56.435Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/27/83/859ecdd180cacc13b1f7e857abf8582a64552ea7a061057a6c716e790fce/sphinxcontrib_qthelp-2.0.0-py3-none-any.whl", hash = "sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb", size = 88743, upload-time = "2024-07-29T01:09:54.885Z" }, -] - -[[package]] -name = "sphinxcontrib-serializinghtml" -version = "2.0.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/3b/44/6716b257b0aa6bfd51a1b31665d1c205fb12cb5ad56de752dfa15657de2f/sphinxcontrib_serializinghtml-2.0.0.tar.gz", hash = "sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d", size = 16080, upload-time = "2024-07-29T01:10:09.332Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/52/a7/d2782e4e3f77c8450f727ba74a8f12756d5ba823d81b941f1b04da9d033a/sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl", hash = "sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331", size = 92072, upload-time = "2024-07-29T01:10:08.203Z" }, -] - [[package]] name = "stack-data" version = "0.6.3" @@ -1578,6 +2286,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f1/7b/ce1eafaf1a76852e2ec9b22edecf1daa58175c090266e9f6c64afcd81d91/stack_data-0.6.3-py3-none-any.whl", hash = "sha256:d5558e0c25a4cb0853cddad3d77da9891a08cb85dd9f9f91b9f8cd66e511e695", size = 24521, upload-time = "2023-09-30T13:58:03.53Z" }, ] +[[package]] +name = "tinycss2" +version = "1.5.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "webencodings" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a3/ae/2ca4913e5c0f09781d75482874c3a95db9105462a92ddd303c7d285d3df2/tinycss2-1.5.1.tar.gz", hash = "sha256:d339d2b616ba90ccce58da8495a78f46e55d4d25f9fd71dfd526f07e7d53f957", size = 88195, upload-time = "2025-11-23T10:29:10.082Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/60/45/c7b5c3168458db837e8ceab06dc77824e18202679d0463f0e8f002143a97/tinycss2-1.5.1-py3-none-any.whl", hash = "sha256:3415ba0f5839c062696996998176c4a3751d18b7edaaeeb658c9ce21ec150661", size = 28404, upload-time = "2025-11-23T10:29:08.676Z" }, +] + [[package]] name = "toml" version = "0.10.2" @@ -1626,6 +2346,23 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/6e/c2/61d3e0f47e2b74ef40a68b9e6ad5984f6241a942f7cd3bbfbdbd03861ea9/tomli-2.2.1-py3-none-any.whl", hash = "sha256:cb55c73c5f4408779d0cf3eef9f762b9c9f147a77de7b258bef0a5628adc85cc", size = 14257, upload-time = "2024-11-27T22:38:35.385Z" }, ] +[[package]] +name = "tornado" +version = "6.5.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/50/57/6d7303a77ae439d9189108f76c0c4fd89ee5e2cc8387bffb55232565c4ed/tornado-6.5.6.tar.gz", hash = "sha256:9a365179fe8ff6b8766f602c0f67c185d778193e9bdd828b19f0b6ed7764177d", size = 518139, upload-time = "2026-05-27T15:35:54.646Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1b/0d/b4f481e18c5a51864e6d12b9a05ecf72919696680b747c958c3fc1f4fbae/tornado-6.5.6-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:65fcfaafb079435c2c19dc9e07c0f1cf0fa9051759ed0a7d0a3ba7ea7f64919c", size = 447737, upload-time = "2026-05-27T15:35:38.122Z" }, + { url = "https://files.pythonhosted.org/packages/9e/9c/5430c39fcab1144d35860f457b15e9c08b4bc7ac86764354204e983d6183/tornado-6.5.6-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:38bc01b4acacded2de63ae78023548e41ebe6fbed3ec05a796d7ae3ad893887e", size = 445899, upload-time = "2026-05-27T15:35:40.519Z" }, + { url = "https://files.pythonhosted.org/packages/8b/79/fa7e14a2f939c807a8d30619b4eb604eab219601b78792516ebe22d40cf9/tornado-6.5.6-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b942e6a137fda31ff54bf8e6e2c8d1c37f1f50583f3ed53fb840b53b9601d104", size = 448964, upload-time = "2026-05-27T15:35:42.106Z" }, + { url = "https://files.pythonhosted.org/packages/a7/71/bd67d5f5199f937dafe03a49a37989f60f600ff6fef34c79412a829d97bd/tornado-6.5.6-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8666946e70171b8c3f1fc9b7876fac492e84822c4c7f3746f4e8f8bc9ac92a79", size = 449935, upload-time = "2026-05-27T15:35:43.906Z" }, + { url = "https://files.pythonhosted.org/packages/cc/a4/c24388c9cf5b3c3a513b56a158af9f23092c9a2810d789e294310797df21/tornado-6.5.6-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:1c34cfab7ad6d104f052f55de06d39bbafc5885cfeb4da688803308dbcfa90b7", size = 449767, upload-time = "2026-05-27T15:35:45.793Z" }, + { url = "https://files.pythonhosted.org/packages/a5/eb/6a07ad550c3f7b37244bd0becdf293ec3d3e961783d8b720a97df50de1b2/tornado-6.5.6-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:385f35e4e22fb52551dfcda4cdc8c30c61c2c001aef5ddad99cdfe116952efd3", size = 449174, upload-time = "2026-05-27T15:35:47.485Z" }, + { url = "https://files.pythonhosted.org/packages/bb/84/3469e098dccdb6763130e06aacd786bb4363fca7b590a55c101ddf34ed30/tornado-6.5.6-cp39-abi3-win32.whl", hash = "sha256:db475f1b67b2809b10bb16264829087724ca8d24fe4ed47f7b8675cae453ef86", size = 450230, upload-time = "2026-05-27T15:35:49.322Z" }, + { url = "https://files.pythonhosted.org/packages/d2/3c/273a04e0b9dd9016f1685cca0c1c8795a71ac88a34a8c889a0b443483226/tornado-6.5.6-cp39-abi3-win_amd64.whl", hash = "sha256:6739bf1e8eb09230f1280ddbd3236f0309db70f2c551a8dbc40f62babdf82f79", size = 450667, upload-time = "2026-05-27T15:35:51.194Z" }, + { url = "https://files.pythonhosted.org/packages/02/98/0cffe22a224f60c5fb1e3aa0b76f9da2e1ca78b0e9545e3d077c68ce60a7/tornado-6.5.6-cp39-abi3-win_arm64.whl", hash = "sha256:2543597b24a695d72338a9a77818362d72387c03ae173f1f169eadc5c91466ac", size = 449690, upload-time = "2026-05-27T15:35:52.902Z" }, +] + [[package]] name = "traitlets" version = "5.14.3" @@ -1676,6 +2413,38 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f3/40/b1c265d4b2b62b58576588510fc4d1fe60a86319c8de99fd8e9fec617d2c/virtualenv-20.31.2-py3-none-any.whl", hash = "sha256:36efd0d9650ee985f0cad72065001e66d49a6f24eb44d98980f630686243cf11", size = 6057982, upload-time = "2025-05-08T17:58:21.15Z" }, ] +[[package]] +name = "watchdog" +version = "6.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/db/7d/7f3d619e951c88ed75c6037b246ddcf2d322812ee8ea189be89511721d54/watchdog-6.0.0.tar.gz", hash = "sha256:9ddf7c82fda3ae8e24decda1338ede66e1c99883db93711d8fb941eaa2d8c282", size = 131220, upload-time = "2024-11-01T14:07:13.037Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0c/56/90994d789c61df619bfc5ce2ecdabd5eeff564e1eb47512bd01b5e019569/watchdog-6.0.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:d1cdb490583ebd691c012b3d6dae011000fe42edb7a82ece80965b42abd61f26", size = 96390, upload-time = "2024-11-01T14:06:24.793Z" }, + { url = "https://files.pythonhosted.org/packages/55/46/9a67ee697342ddf3c6daa97e3a587a56d6c4052f881ed926a849fcf7371c/watchdog-6.0.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:bc64ab3bdb6a04d69d4023b29422170b74681784ffb9463ed4870cf2f3e66112", size = 88389, upload-time = "2024-11-01T14:06:27.112Z" }, + { url = "https://files.pythonhosted.org/packages/44/65/91b0985747c52064d8701e1075eb96f8c40a79df889e59a399453adfb882/watchdog-6.0.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:c897ac1b55c5a1461e16dae288d22bb2e412ba9807df8397a635d88f671d36c3", size = 89020, upload-time = "2024-11-01T14:06:29.876Z" }, + { url = "https://files.pythonhosted.org/packages/e0/24/d9be5cd6642a6aa68352ded4b4b10fb0d7889cb7f45814fb92cecd35f101/watchdog-6.0.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:6eb11feb5a0d452ee41f824e271ca311a09e250441c262ca2fd7ebcf2461a06c", size = 96393, upload-time = "2024-11-01T14:06:31.756Z" }, + { url = "https://files.pythonhosted.org/packages/63/7a/6013b0d8dbc56adca7fdd4f0beed381c59f6752341b12fa0886fa7afc78b/watchdog-6.0.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:ef810fbf7b781a5a593894e4f439773830bdecb885e6880d957d5b9382a960d2", size = 88392, upload-time = "2024-11-01T14:06:32.99Z" }, + { url = "https://files.pythonhosted.org/packages/d1/40/b75381494851556de56281e053700e46bff5b37bf4c7267e858640af5a7f/watchdog-6.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:afd0fe1b2270917c5e23c2a65ce50c2a4abb63daafb0d419fde368e272a76b7c", size = 89019, upload-time = "2024-11-01T14:06:34.963Z" }, + { url = "https://files.pythonhosted.org/packages/39/ea/3930d07dafc9e286ed356a679aa02d777c06e9bfd1164fa7c19c288a5483/watchdog-6.0.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:bdd4e6f14b8b18c334febb9c4425a878a2ac20efd1e0b231978e7b150f92a948", size = 96471, upload-time = "2024-11-01T14:06:37.745Z" }, + { url = "https://files.pythonhosted.org/packages/12/87/48361531f70b1f87928b045df868a9fd4e253d9ae087fa4cf3f7113be363/watchdog-6.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c7c15dda13c4eb00d6fb6fc508b3c0ed88b9d5d374056b239c4ad1611125c860", size = 88449, upload-time = "2024-11-01T14:06:39.748Z" }, + { url = "https://files.pythonhosted.org/packages/5b/7e/8f322f5e600812e6f9a31b75d242631068ca8f4ef0582dd3ae6e72daecc8/watchdog-6.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6f10cb2d5902447c7d0da897e2c6768bca89174d0c6e1e30abec5421af97a5b0", size = 89054, upload-time = "2024-11-01T14:06:41.009Z" }, + { url = "https://files.pythonhosted.org/packages/68/98/b0345cabdce2041a01293ba483333582891a3bd5769b08eceb0d406056ef/watchdog-6.0.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:490ab2ef84f11129844c23fb14ecf30ef3d8a6abafd3754a6f75ca1e6654136c", size = 96480, upload-time = "2024-11-01T14:06:42.952Z" }, + { url = "https://files.pythonhosted.org/packages/85/83/cdf13902c626b28eedef7ec4f10745c52aad8a8fe7eb04ed7b1f111ca20e/watchdog-6.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:76aae96b00ae814b181bb25b1b98076d5fc84e8a53cd8885a318b42b6d3a5134", size = 88451, upload-time = "2024-11-01T14:06:45.084Z" }, + { url = "https://files.pythonhosted.org/packages/fe/c4/225c87bae08c8b9ec99030cd48ae9c4eca050a59bf5c2255853e18c87b50/watchdog-6.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a175f755fc2279e0b7312c0035d52e27211a5bc39719dd529625b1930917345b", size = 89057, upload-time = "2024-11-01T14:06:47.324Z" }, + { url = "https://files.pythonhosted.org/packages/30/ad/d17b5d42e28a8b91f8ed01cb949da092827afb9995d4559fd448d0472763/watchdog-6.0.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:c7ac31a19f4545dd92fc25d200694098f42c9a8e391bc00bdd362c5736dbf881", size = 87902, upload-time = "2024-11-01T14:06:53.119Z" }, + { url = "https://files.pythonhosted.org/packages/5c/ca/c3649991d140ff6ab67bfc85ab42b165ead119c9e12211e08089d763ece5/watchdog-6.0.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:9513f27a1a582d9808cf21a07dae516f0fab1cf2d7683a742c498b93eedabb11", size = 88380, upload-time = "2024-11-01T14:06:55.19Z" }, + { url = "https://files.pythonhosted.org/packages/a9/c7/ca4bf3e518cb57a686b2feb4f55a1892fd9a3dd13f470fca14e00f80ea36/watchdog-6.0.0-py3-none-manylinux2014_aarch64.whl", hash = "sha256:7607498efa04a3542ae3e05e64da8202e58159aa1fa4acddf7678d34a35d4f13", size = 79079, upload-time = "2024-11-01T14:06:59.472Z" }, + { url = "https://files.pythonhosted.org/packages/5c/51/d46dc9332f9a647593c947b4b88e2381c8dfc0942d15b8edc0310fa4abb1/watchdog-6.0.0-py3-none-manylinux2014_armv7l.whl", hash = "sha256:9041567ee8953024c83343288ccc458fd0a2d811d6a0fd68c4c22609e3490379", size = 79078, upload-time = "2024-11-01T14:07:01.431Z" }, + { url = "https://files.pythonhosted.org/packages/d4/57/04edbf5e169cd318d5f07b4766fee38e825d64b6913ca157ca32d1a42267/watchdog-6.0.0-py3-none-manylinux2014_i686.whl", hash = "sha256:82dc3e3143c7e38ec49d61af98d6558288c415eac98486a5c581726e0737c00e", size = 79076, upload-time = "2024-11-01T14:07:02.568Z" }, + { url = "https://files.pythonhosted.org/packages/ab/cc/da8422b300e13cb187d2203f20b9253e91058aaf7db65b74142013478e66/watchdog-6.0.0-py3-none-manylinux2014_ppc64.whl", hash = "sha256:212ac9b8bf1161dc91bd09c048048a95ca3a4c4f5e5d4a7d1b1a7d5752a7f96f", size = 79077, upload-time = "2024-11-01T14:07:03.893Z" }, + { url = "https://files.pythonhosted.org/packages/2c/3b/b8964e04ae1a025c44ba8e4291f86e97fac443bca31de8bd98d3263d2fcf/watchdog-6.0.0-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:e3df4cbb9a450c6d49318f6d14f4bbc80d763fa587ba46ec86f99f9e6876bb26", size = 79078, upload-time = "2024-11-01T14:07:05.189Z" }, + { url = "https://files.pythonhosted.org/packages/62/ae/a696eb424bedff7407801c257d4b1afda455fe40821a2be430e173660e81/watchdog-6.0.0-py3-none-manylinux2014_s390x.whl", hash = "sha256:2cce7cfc2008eb51feb6aab51251fd79b85d9894e98ba847408f662b3395ca3c", size = 79077, upload-time = "2024-11-01T14:07:06.376Z" }, + { url = "https://files.pythonhosted.org/packages/b5/e8/dbf020b4d98251a9860752a094d09a65e1b436ad181faf929983f697048f/watchdog-6.0.0-py3-none-manylinux2014_x86_64.whl", hash = "sha256:20ffe5b202af80ab4266dcd3e91aae72bf2da48c0d33bdb15c66658e685e94e2", size = 79078, upload-time = "2024-11-01T14:07:07.547Z" }, + { url = "https://files.pythonhosted.org/packages/07/f6/d0e5b343768e8bcb4cda79f0f2f55051bf26177ecd5651f84c07567461cf/watchdog-6.0.0-py3-none-win32.whl", hash = "sha256:07df1fdd701c5d4c8e55ef6cf55b8f0120fe1aef7ef39a1c6fc6bc2e606d517a", size = 79065, upload-time = "2024-11-01T14:07:09.525Z" }, + { url = "https://files.pythonhosted.org/packages/db/d9/c495884c6e548fce18a8f40568ff120bc3a4b7b99813081c8ac0c936fa64/watchdog-6.0.0-py3-none-win_amd64.whl", hash = "sha256:cbafb470cf848d93b5d013e2ecb245d4aa1c8fd0504e863ccefa32445359d680", size = 79070, upload-time = "2024-11-01T14:07:10.686Z" }, + { url = "https://files.pythonhosted.org/packages/33/e8/e40370e6d74ddba47f002a32919d91310d6074130fe4e17dabcafc15cbf1/watchdog-6.0.0-py3-none-win_ia64.whl", hash = "sha256:a1914259fa9e1454315171103c6a30961236f508b9b623eae470268bbcc6a22f", size = 79067, upload-time = "2024-11-01T14:07:11.845Z" }, +] + [[package]] name = "wcwidth" version = "0.2.13" @@ -1685,6 +2454,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/fd/84/fd2ba7aafacbad3c4201d395674fc6348826569da3c0937e75505ead3528/wcwidth-0.2.13-py2.py3-none-any.whl", hash = "sha256:3da69048e4540d84af32131829ff948f1e022c1c6bdb8d6102117aac784f6859", size = 34166, upload-time = "2024-01-06T02:10:55.763Z" }, ] +[[package]] +name = "webencodings" +version = "0.5.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/0b/02/ae6ceac1baeda530866a85075641cec12989bd8d31af6d5ab4a3e8c92f47/webencodings-0.5.1.tar.gz", hash = "sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923", size = 9721, upload-time = "2017-04-05T20:21:34.189Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/24/2a3e3df732393fed8b3ebf2ec078f05546de641fe1b667ee316ec1dcf3b7/webencodings-0.5.1-py2.py3-none-any.whl", hash = "sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78", size = 11774, upload-time = "2017-04-05T20:21:32.581Z" }, +] + [[package]] name = "wrapt" version = "1.17.2" From 3e54dad98f2ce07c1cb35a30e10b351c174148e0 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Sun, 7 Jun 2026 17:41:31 +0200 Subject: [PATCH 04/18] docs: hide notebook setup cells from rendered output MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tag the `nb-setup` cell in every notebook with `remove-input` and `remove-output`, configure mkdocs-jupyter's `remove_tag_config` to honor both tags, and hide the residual `
` container via a `.celltag_nb-setup { display: none }` rule in `theme_overrides.css`. The setup cell still executes — only the visible representation is stripped. End users no longer see the `os.chdir(...)` boilerplate or the convenience imports at the top of every user-guide page. Co-Authored-By: Claude Opus 4.7 --- docs/source/_static/theme_overrides.css | 8 ++++++++ docs/source/index.ipynb | 4 +++- docs/source/user-guide/basics.ipynb | 4 +++- .../common-operations/aggregations.ipynb | 18 ++++++++++-------- .../common-operations/basic-info.ipynb | 4 +++- .../common-operations/expressions.ipynb | 8 +++++--- .../common-operations/functions.ipynb | 4 +++- .../user-guide/common-operations/joins.ipynb | 4 +++- .../common-operations/select-and-filter.ipynb | 6 ++++-- .../common-operations/udf-and-udfa.ipynb | 10 ++++++---- .../user-guide/common-operations/windows.ipynb | 8 +++++--- docs/source/user-guide/data-sources.ipynb | 4 +++- docs/source/user-guide/introduction.ipynb | 4 +++- docs/source/user-guide/io/arrow.ipynb | 4 +++- docs/source/user-guide/sql.ipynb | 4 +++- mkdocs.yml | 4 ++++ 16 files changed, 69 insertions(+), 29 deletions(-) diff --git a/docs/source/_static/theme_overrides.css b/docs/source/_static/theme_overrides.css index 596404ad9..16a714b48 100644 --- a/docs/source/_static/theme_overrides.css +++ b/docs/source/_static/theme_overrides.css @@ -44,6 +44,14 @@ color: #FF8A75; } +/* Hide notebook setup cells (chdir + boilerplate imports) so they don't + * appear as empty containers in the rendered output. The cells still + * execute — mkdocs-jupyter's TagRemovePreprocessor only strips input + * and outputs, leaving the wrapping div behind. */ +.celltag_nb-setup { + display: none !important; +} + /* Center the footer copyright/trademark block */ .md-copyright { text-align: center; diff --git a/docs/source/index.ipynb b/docs/source/index.ipynb index deb5a4304..c6cf06041 100644 --- a/docs/source/index.ipynb +++ b/docs/source/index.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/docs/source/user-guide/basics.ipynb b/docs/source/user-guide/basics.ipynb index e2cc9603e..a10188a71 100644 --- a/docs/source/user-guide/basics.ipynb +++ b/docs/source/user-guide/basics.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/docs/source/user-guide/common-operations/aggregations.ipynb b/docs/source/user-guide/common-operations/aggregations.ipynb index c16c4ff6e..be89647fa 100644 --- a/docs/source/user-guide/common-operations/aggregations.ipynb +++ b/docs/source/user-guide/common-operations/aggregations.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], @@ -255,7 +257,7 @@ "cell_type": "markdown", "id": "3ed186c9a28b402fb0bc4494df01f08d", "metadata": {}, - "source": "\n### Comparing subsets within a group\n\nSometimes you need to compare the full membership of a group against a\nsubset that meets some condition — for example, \"which groups have at least\none failure, but not every member failed?\". The `filter` argument on an\naggregate restricts the rows that contribute to *that* aggregate without\ndropping the group, so a single pass can produce both the full set and the\nfiltered subset side by side. Pairing\n[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and\n`filter=` is a compact way to express this: collect the distinct values\nof the group, collect the distinct values that satisfy the condition, then\ncompare the two arrays.\n\nSuppose each row records a line item with the supplier that fulfilled it and\na flag for whether that supplier met the commit date. We want to identify\n*partially failed* orders — orders where at least one supplier failed but\nnot every supplier failed:\n\n" + "source": "\n### Comparing subsets within a group\n\nSometimes you need to compare the full membership of a group against a\nsubset that meets some condition \u2014 for example, \"which groups have at least\none failure, but not every member failed?\". The `filter` argument on an\naggregate restricts the rows that contribute to *that* aggregate without\ndropping the group, so a single pass can produce both the full set and the\nfiltered subset side by side. Pairing\n[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and\n`filter=` is a compact way to express this: collect the distinct values\nof the group, collect the distinct values that satisfy the condition, then\ncompare the two arrays.\n\nSuppose each row records a line item with the supplier that fulfilled it and\na flag for whether that supplier met the commit date. We want to identify\n*partially failed* orders \u2014 orders where at least one supplier failed but\nnot every supplier failed:\n\n" }, { "cell_type": "code", @@ -294,7 +296,7 @@ "cell_type": "markdown", "id": "379cbbc1e968416e875cc15c1202d7eb", "metadata": {}, - "source": "\nOrder 1 is partial (one of three suppliers failed). Order 2 is excluded\nbecause no supplier failed, order 3 because its only supplier failed, and\norder 4 because both of its suppliers failed.\n\n## Grouping Sets\n\nThe default style of aggregation produces one row per group. Sometimes you want a single query to\nproduce rows at multiple levels of detail — for example, totals per type *and* an overall grand\ntotal, or subtotals for every combination of two columns plus the individual column totals. Writing\nseparate queries and concatenating them is tedious and runs the data multiple times. Grouping sets\nsolve this by letting you specify several grouping levels in one pass.\n\nDataFusion supports three grouping set styles through the\n[`GroupingSet`][datafusion.expr.GroupingSet] class:\n\n- [`rollup`][datafusion.expr.GroupingSet.rollup] — hierarchical subtotals, like a drill-down report\n- [`cube`][datafusion.expr.GroupingSet.cube] — every possible subtotal combination, like a pivot table\n- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] — explicitly list exactly which grouping levels you want\n\nBecause result rows come from different grouping levels, a column that is *not* part of a\nparticular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to\ndistinguish a real `null` in the data from one that means \"this column was aggregated across.\"\nIt returns `0` when the column is a grouping key for that row, and `1` when it is not.\n\n### Rollup\n\n[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces\ngrouping sets `(a, b)`, `(a)`, and `()` — like nested subtotals in a report. This is useful\nwhen your columns have a natural hierarchy, such as region → city or type → subtype.\n\nSuppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With\nthe default aggregation style we would need two separate queries. With `rollup` we get it all at\nonce:\n\n" + "source": "\nOrder 1 is partial (one of three suppliers failed). Order 2 is excluded\nbecause no supplier failed, order 3 because its only supplier failed, and\norder 4 because both of its suppliers failed.\n\n## Grouping Sets\n\nThe default style of aggregation produces one row per group. Sometimes you want a single query to\nproduce rows at multiple levels of detail \u2014 for example, totals per type *and* an overall grand\ntotal, or subtotals for every combination of two columns plus the individual column totals. Writing\nseparate queries and concatenating them is tedious and runs the data multiple times. Grouping sets\nsolve this by letting you specify several grouping levels in one pass.\n\nDataFusion supports three grouping set styles through the\n[`GroupingSet`][datafusion.expr.GroupingSet] class:\n\n- [`rollup`][datafusion.expr.GroupingSet.rollup] \u2014 hierarchical subtotals, like a drill-down report\n- [`cube`][datafusion.expr.GroupingSet.cube] \u2014 every possible subtotal combination, like a pivot table\n- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] \u2014 explicitly list exactly which grouping levels you want\n\nBecause result rows come from different grouping levels, a column that is *not* part of a\nparticular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to\ndistinguish a real `null` in the data from one that means \"this column was aggregated across.\"\nIt returns `0` when the column is a grouping key for that row, and `1` when it is not.\n\n### Rollup\n\n[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces\ngrouping sets `(a, b)`, `(a)`, and `()` \u2014 like nested subtotals in a report. This is useful\nwhen your columns have a natural hierarchy, such as region \u2192 city or type \u2192 subtype.\n\nSuppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With\nthe default aggregation style we would need two separate queries. With `rollup` we get it all at\nonce:\n\n" }, { "cell_type": "code", @@ -319,7 +321,7 @@ "cell_type": "markdown", "id": "db7b79bc585a40fcaf58bf750017e135", "metadata": {}, - "source": "\nThe first row — where `Type 1` is `null` — is the grand total across all types. But how do you\ntell a grand-total `null` apart from a Pokemon that genuinely has no type? The\n[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key\nfor that row and `1` when it is aggregated across.\n\n!!! note\n\n Due to an upstream DataFusion limitation\n ([apache/datafusion#21411](https://github.com/apache/datafusion/issues/21411)),\n `.alias()` cannot be applied directly to a `grouping()` expression — it will raise an\n error at execution time. Instead, use\n [`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] on the result DataFrame to\n give the column a readable name. Once the upstream issue is resolved, you will be able to\n use `.alias()` directly and the workaround below will no longer be necessary.\n\nThe raw column name generated by `grouping()` contains internal identifiers, so we use\n[`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] to clean it up:\n\n" + "source": "\nThe first row \u2014 where `Type 1` is `null` \u2014 is the grand total across all types. But how do you\ntell a grand-total `null` apart from a Pokemon that genuinely has no type? The\n[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key\nfor that row and `1` when it is aggregated across.\n\n!!! note\n\n Due to an upstream DataFusion limitation\n ([apache/datafusion#21411](https://github.com/apache/datafusion/issues/21411)),\n `.alias()` cannot be applied directly to a `grouping()` expression \u2014 it will raise an\n error at execution time. Instead, use\n [`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] on the result DataFrame to\n give the column a readable name. Once the upstream issue is resolved, you will be able to\n use `.alias()` directly and the workaround below will no longer be necessary.\n\nThe raw column name generated by `grouping()` contains internal identifiers, so we use\n[`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] to clean it up:\n\n" }, { "cell_type": "code", @@ -346,7 +348,7 @@ "cell_type": "markdown", "id": "1671c31a24314836a5b85d7ef7fbf015", "metadata": {}, - "source": "\nWith two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces:\n\n- one row per `(Type 1, Type 2)` pair — the most detailed level\n- one row per `Type 1` — subtotals\n- one grand total row\n\n" + "source": "\nWith two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces:\n\n- one row per `(Type 1, Type 2)` pair \u2014 the most detailed level\n- one row per `Type 1` \u2014 subtotals\n- one grand total row\n\n" }, { "cell_type": "code", @@ -368,7 +370,7 @@ "cell_type": "markdown", "id": "f6fa52606d8c4a75a9b52967216f8f3f", "metadata": {}, - "source": "\n### Cube\n\n[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)`\nproduces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` — one more than `rollup` because\nit also includes `(b)` alone. This is useful when neither column is \"above\" the other in a\nhierarchy and you want all cross-tabulations.\n\nFor our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair,\nby `Type 1` alone, by `Type 2` alone, and a grand total — all in one query:\n\n" + "source": "\n### Cube\n\n[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)`\nproduces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` \u2014 one more than `rollup` because\nit also includes `(b)` alone. This is useful when neither column is \"above\" the other in a\nhierarchy and you want all cross-tabulations.\n\nFor our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair,\nby `Type 1` alone, by `Type 2` alone, and a grand total \u2014 all in one query:\n\n" }, { "cell_type": "code", @@ -390,7 +392,7 @@ "cell_type": "markdown", "id": "cdf66aed5cc84ca1b48e60bad68798a8", "metadata": {}, - "source": "\nCompared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but\n`Type 2` has a value — those are the per-`Type 2` subtotals that `rollup` does not include.\n\n### Explicit Grouping Sets\n\n[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels\nyou need when `rollup` or `cube` would produce too many or too few. Each argument is a list of\ncolumns forming one grouping set.\n\nFor example, if we want only the per-`Type 1` totals and per-`Type 2` totals — but *not* the\nfull `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that:\n\n" + "source": "\nCompared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but\n`Type 2` has a value \u2014 those are the per-`Type 2` subtotals that `rollup` does not include.\n\n### Explicit Grouping Sets\n\n[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels\nyou need when `rollup` or `cube` would produce too many or too few. Each argument is a list of\ncolumns forming one grouping set.\n\nFor example, if we want only the per-`Type 1` totals and per-`Type 2` totals \u2014 but *not* the\nfull `(Type 1, Type 2)` detail rows or the grand total \u2014 we can ask for exactly that:\n\n" }, { "cell_type": "code", @@ -444,7 +446,7 @@ "cell_type": "markdown", "id": "5b09d5ef5b5e4bb6ab9b829b10b6a29f", "metadata": {}, - "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions —\n the accumulator class is captured by value via {mod}`cloudpickle`,\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the accumulator class is captured by value via {mod}`cloudpickle`,\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" } ], "metadata": { diff --git a/docs/source/user-guide/common-operations/basic-info.ipynb b/docs/source/user-guide/common-operations/basic-info.ipynb index 4b934710b..1e8831db1 100644 --- a/docs/source/user-guide/common-operations/basic-info.ipynb +++ b/docs/source/user-guide/common-operations/basic-info.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/docs/source/user-guide/common-operations/expressions.ipynb b/docs/source/user-guide/common-operations/expressions.ipynb index 988a916b2..c76694e9a 100644 --- a/docs/source/user-guide/common-operations/expressions.ipynb +++ b/docs/source/user-guide/common-operations/expressions.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], @@ -289,7 +291,7 @@ "cell_type": "markdown", "id": "1671c31a24314836a5b85d7ef7fbf015", "metadata": {}, - "source": "\n**Searched CASE** (an independent boolean predicate per branch). Use this\nform whenever a branch tests more than simple equality — for example,\nchecking whether a joined column is `NULL` to gate a computed value:\n\n" + "source": "\n**Searched CASE** (an independent boolean predicate per branch). Use this\nform whenever a branch tests more than simple equality \u2014 for example,\nchecking whether a joined column is `NULL` to gate a computed value:\n\n" }, { "cell_type": "code", @@ -315,7 +317,7 @@ "cell_type": "markdown", "id": "f6fa52606d8c4a75a9b52967216f8f3f", "metadata": {}, - "source": "\nThis searched-CASE pattern is idiomatic for \"attribute the measure to the\nmatching side of a left join, otherwise contribute zero\" — a shape that\nappears in TPC-H Q08 and similar market-share calculations.\n\nIf a switched CASE only groups several equality matches into one bucket,\n`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often\nsimpler than the full `case` builder.\n\n## Structs\n\nColumns that contain struct elements can be accessed using the bracket notation as if they were\nPython dictionary style objects. This expects a string key as the parameter passed.\n\n" + "source": "\nThis searched-CASE pattern is idiomatic for \"attribute the measure to the\nmatching side of a left join, otherwise contribute zero\" \u2014 a shape that\nappears in TPC-H Q08 and similar market-share calculations.\n\nIf a switched CASE only groups several equality matches into one bucket,\n`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often\nsimpler than the full `case` builder.\n\n## Structs\n\nColumns that contain struct elements can be accessed using the bracket notation as if they were\nPython dictionary style objects. This expects a string key as the parameter passed.\n\n" }, { "cell_type": "code", diff --git a/docs/source/user-guide/common-operations/functions.ipynb b/docs/source/user-guide/common-operations/functions.ipynb index bdbda3c7c..aabb6a765 100644 --- a/docs/source/user-guide/common-operations/functions.ipynb +++ b/docs/source/user-guide/common-operations/functions.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/docs/source/user-guide/common-operations/joins.ipynb b/docs/source/user-guide/common-operations/joins.ipynb index 3c0beac92..f31c2a98a 100644 --- a/docs/source/user-guide/common-operations/joins.ipynb +++ b/docs/source/user-guide/common-operations/joins.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/docs/source/user-guide/common-operations/select-and-filter.ipynb b/docs/source/user-guide/common-operations/select-and-filter.ipynb index 8143c83c9..aca9408bc 100644 --- a/docs/source/user-guide/common-operations/select-and-filter.ipynb +++ b/docs/source/user-guide/common-operations/select-and-filter.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], @@ -69,7 +71,7 @@ "cell_type": "markdown", "id": "8edb47106e1a46a883d545849b8ab81b", "metadata": {}, - "source": "\n!!! warning\n\n Please be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters\n (ex: Name) you must put your column name in double quotes or the selection won’t work. As an alternative for simple\n column selection use [`select`][datafusion.dataframe.DataFrame.select] without double quotes\n\nFor selecting columns with capital letters use `'\"VendorID\"'`\n\n" + "source": "\n!!! warning\n\n Please be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters\n (ex: Name) you must put your column name in double quotes or the selection won\u2019t work. As an alternative for simple\n column selection use [`select`][datafusion.dataframe.DataFrame.select] without double quotes\n\nFor selecting columns with capital letters use `'\"VendorID\"'`\n\n" }, { "cell_type": "code", diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb index 8b7a793a2..31c2b4ffb 100644 --- a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb +++ b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], @@ -104,7 +106,7 @@ "cell_type": "markdown", "id": "8edb47106e1a46a883d545849b8ab81b", "metadata": {}, - "source": "\nIn this example we passed the PyArrow `DataType` when we defined the function\nby calling `udf()`. If you need additional control, such as specifying\nmetadata or nullability of the input or output, you can instead specify a\nPyArrow `Field`.\n\nIf you need to write a custom function but do not want to incur the performance\ncost of converting to Python objects and back, a more advanced approach is to\nwrite Rust based UDFs and to expose them to Python. There is an example in the\n[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/)\ndescribing how to do this.\n\n### When not to use a UDF\n\nA UDF is the right tool when the per-row computation genuinely cannot be\nexpressed with DataFusion's built-in expressions. It is often the *wrong*\ntool for a predicate that *can* be written as an `Expr` tree but feels\neasier to write as a Python function — for example, a filter that keeps\na row if it matches any one of several rule sets, where each rule set\nchecks its own combination of columns (the worked example at the end of\nthis section keeps a row when it matches any one of several brand-specific\nrules). Looping over the rules in Python and returning a boolean per row\nreads naturally and is tempting to wrap in a UDF, but a UDF is opaque to\nthe optimizer: filters expressed as UDFs lose several rewrites that the\nengine applies to filters built from native expressions. The most visible\nof these is **predicate pushdown into the table provider**: a native\npredicate can be handed to the source so it skips data before it is read,\nwhile a UDF predicate cannot. The example below uses Parquet, where\npushdown prunes whole row groups using the min/max statistics in the\nfooter, but the same mechanism applies to any table provider that\nadvertises filter support — including custom providers.\n\nThe following example writes a small Parquet file, then filters it two\nways: first with a native expression, then with a UDF that computes the\nsame result. The filter itself is simple on purpose so we can compare\nthe plans side by side.\n\n" + "source": "\nIn this example we passed the PyArrow `DataType` when we defined the function\nby calling `udf()`. If you need additional control, such as specifying\nmetadata or nullability of the input or output, you can instead specify a\nPyArrow `Field`.\n\nIf you need to write a custom function but do not want to incur the performance\ncost of converting to Python objects and back, a more advanced approach is to\nwrite Rust based UDFs and to expose them to Python. There is an example in the\n[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/)\ndescribing how to do this.\n\n### When not to use a UDF\n\nA UDF is the right tool when the per-row computation genuinely cannot be\nexpressed with DataFusion's built-in expressions. It is often the *wrong*\ntool for a predicate that *can* be written as an `Expr` tree but feels\neasier to write as a Python function \u2014 for example, a filter that keeps\na row if it matches any one of several rule sets, where each rule set\nchecks its own combination of columns (the worked example at the end of\nthis section keeps a row when it matches any one of several brand-specific\nrules). Looping over the rules in Python and returning a boolean per row\nreads naturally and is tempting to wrap in a UDF, but a UDF is opaque to\nthe optimizer: filters expressed as UDFs lose several rewrites that the\nengine applies to filters built from native expressions. The most visible\nof these is **predicate pushdown into the table provider**: a native\npredicate can be handed to the source so it skips data before it is read,\nwhile a UDF predicate cannot. The example below uses Parquet, where\npushdown prunes whole row groups using the min/max statistics in the\nfooter, but the same mechanism applies to any table provider that\nadvertises filter support \u2014 including custom providers.\n\nThe following example writes a small Parquet file, then filters it two\nways: first with a native expression, then with a UDF that computes the\nsame result. The filter itself is simple on purpose so we can compare\nthe plans side by side.\n\n" }, { "cell_type": "code", @@ -158,7 +160,7 @@ "cell_type": "markdown", "id": "7cdc8c89c7104fffa095e18ddfef8986", "metadata": {}, - "source": "\nNotice the `DataSourceExec` line. It carries three annotations the\noptimizer computed from the predicate:\n\n- `predicate=brand@1 = A AND qty@2 >= 150` — the filter is pushed\n into the Parquet scan itself, so the scan only reads matching rows.\n- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ...\n qty_max@4 >= 150` — the scan prunes whole row groups by consulting\n the Parquet min/max statistics in the footer *before* reading any\n column data.\n- `required_guarantees=[brand in (A)]` — the scan uses this when a\n bloom filter or dictionary is available to skip pages.\n\n**UDF predicate.** Now wrap the same logic in a Python UDF:\n\n" + "source": "\nNotice the `DataSourceExec` line. It carries three annotations the\noptimizer computed from the predicate:\n\n- `predicate=brand@1 = A AND qty@2 >= 150` \u2014 the filter is pushed\n into the Parquet scan itself, so the scan only reads matching rows.\n- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ...\n qty_max@4 >= 150` \u2014 the scan prunes whole row groups by consulting\n the Parquet min/max statistics in the footer *before* reading any\n column data.\n- `required_guarantees=[brand in (A)]` \u2014 the scan uses this when a\n bloom filter or dictionary is available to skip pages.\n\n**UDF predicate.** Now wrap the same logic in a Python UDF:\n\n" }, { "cell_type": "code", @@ -187,7 +189,7 @@ "cell_type": "markdown", "id": "938c804e27f84196a10c8828c723f798", "metadata": {}, - "source": "\nThe `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`.\nThere is no `pruning_predicate` and no `required_guarantees`: the\nscan has to materialize every row group and hand each row to the\nPython callback just to decide whether to keep it.\n\nAt small scale the cost difference is invisible; on a Parquet file with\nmany row groups, or data whose min/max statistics line up well with\nthe predicate, the native form can skip most of the file. The UDF form\nreads all of it.\n\n**Takeaway.** Reach for a UDF when the per-row computation is genuinely\nnot expressible as a tree of built-in functions (custom numerical work,\nexternal lookups, complex business rules). When it *is* expressible —\neven if the native form is a little more verbose — build the `Expr`\ntree directly so the optimizer can see through it. For disjunctive\npredicates the idiom is to produce one clause per bucket and combine\nthem with `|`:\n\n```python\nfrom functools import reduce\nfrom operator import or_\nfrom datafusion import col, lit, functions as f\n\nbuckets = {\n \"Brand#12\": {\"containers\": [\"SM CASE\", \"SM BOX\"], \"min_qty\": 1, \"max_size\": 5},\n \"Brand#23\": {\"containers\": [\"MED BAG\", \"MED BOX\"], \"min_qty\": 10, \"max_size\": 10},\n}\n\ndef bucket_clause(brand, spec):\n return (\n (col(\"brand\") == lit(brand))\n & f.in_list(col(\"container\"), [lit(c) for c in spec[\"containers\"]])\n & (col(\"quantity\") >= lit(spec[\"min_qty\"]))\n & (col(\"quantity\") <= lit(spec[\"min_qty\"] + 10))\n & (col(\"size\") >= lit(1))\n & (col(\"size\") <= lit(spec[\"max_size\"]))\n )\n\npredicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items()))\ndf = df.filter(predicate)\n```\n\n## Aggregate Functions\n\nThe [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined\nAggregate Functions (UDAFs). To use this you must implement an\n[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed.\n\nWhen defining a UDAF there are four methods you need to implement. The `update` function takes the\narray(s) of input and updates the internal state of the accumulator. You should define this function\nto have as many input arguments as you will pass when calling the UDAF. Since aggregation may be\nsplit into multiple batches, we must have a method to combine multiple batches. For this, we have\ntwo functions, `state` and `merge`. `state` will return an array of scalar values that contain\nthe current state of a single batch accumulation. Then we must `merge` the results of these\ndifferent states. Finally `evaluate` is the call that will return the final result after the\n`merge` is complete.\n\nIn the following example we want to define a custom aggregate function that will return the\ndifference between the sum of two columns. The state can be represented by a single value and we can\nalso see how the inputs to `update` and `merge` differ.\n\n```python\nimport pyarrow as pa\nimport pyarrow.compute\nimport datafusion\nfrom datafusion import col, udaf, Accumulator\nfrom typing import List\n\nclass MyAccumulator(Accumulator):\n \"\"\"\n Interface of a user-defined accumulation.\n \"\"\"\n def __init__(self):\n self._sum = 0.0\n\n def update(self, values_a: pa.Array, values_b: pa.Array) -> None:\n self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py()\n\n def merge(self, states: list[pa.Array]) -> None:\n self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py()\n\n def state(self) -> list[pa.Scalar]:\n return [pyarrow.scalar(self._sum)]\n\n def evaluate(self) -> pa.Scalar:\n return pyarrow.scalar(self._sum)\n\nctx = datafusion.SessionContext()\ndf = ctx.from_pydict(\n {\n \"a\": [4, 5, 6],\n \"b\": [1, 2, 3],\n }\n)\n\nmy_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable')\n\ndf.aggregate([], [my_udaf(col(\"a\"), col(\"b\")).alias(\"col_diff\")])\n```\n\n### FAQ\n\n**How do I return a list from a UDAF?**\n\nBoth the `evaluate` and the `state` functions expect to return scalar values.\nIf you wish to return a list array as a scalar value, the best practice is to\nwrap the values in a `pyarrow.Scalar` object. For example, you can return a\ntimestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp(\"ms\")))` and\nregister the appropriate return or state types as\n`return_type=pa.list_(pa.timestamp(\"ms\"))` and\n`state_type=[pa.list_(pa.timestamp(\"ms\"))]`, respectively.\n\nAs of DataFusion 52.0.0 , you can pass return any Python object, including a\nPyArrow array, as the return value(s) for these functions and DataFusion will\nattempt to create a scalar type from the value. DataFusion has been tested to\nconvert PyArrow, nanoarrow, and arro3 objects as well as primitive data types\nlike integers, strings, and so on.\n\n## Window Functions\n\nTo implement a User-Defined Window Function (UDWF) you must call the\n[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract\nclass [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator].\n\nThere are three methods of evaluation of UDWFs.\n\n- `evaluate` is the simplest case, where you are given an array and are expected to calculate the\n value for a single row of that array. This is the simplest case, but also the least performant.\n- `evaluate_all` computes the values for all rows for an input array at a single time.\n- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank\n information for the rows.\n\nWhich methods you implement are based upon which of these options are set.\n\n| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement |\n|---|---|---|---|\n| False (default) | False (default) | False (default) | `evaluate_all` |\n| False | True | False | `evaluate` |\n| False | True | False | `evaluate_all_with_rank` |\n| True | True/False | True/False | `evaluate` |\n\n### UDWF options\n\nWhen you define your UDWF you can override the functions that return these values. They will\ndetermine which evaluate functions are called.\n\n- `uses_window_frame` is set for functions that compute based on the specified window frame. If\n your function depends upon the specified frame, set this to `True`.\n- `supports_bounded_execution` specifies if your function can be incrementally computed.\n- `include_rank` is set to `True` for window functions that can be computed only using the rank\n information.\n\n```python\nimport pyarrow as pa\nfrom datafusion import udwf, col, SessionContext\nfrom datafusion.user_defined import WindowEvaluator\n\nclass ExponentialSmooth(WindowEvaluator):\n def __init__(self, alpha: float) -> None:\n self.alpha = alpha\n\n def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array:\n results = []\n curr_value = 0.0\n values = values[0]\n for idx in range(num_rows):\n if idx == 0:\n curr_value = values[idx].as_py()\n else:\n curr_value = values[idx].as_py() * self.alpha + curr_value * (\n 1.0 - self.alpha\n )\n results.append(curr_value)\n\n return pa.array(results)\n\nexp_smooth = udwf(\n ExponentialSmooth(0.9),\n pa.float64(),\n pa.float64(),\n volatility=\"immutable\",\n)\n\nctx = SessionContext()\n\ndf = ctx.from_pydict({\n \"a\": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0]\n})\n\ndf.select(\"a\", exp_smooth(col(\"a\")).alias(\"smooth_a\")).show()\n```\n\n## Table Functions\n\nUser Defined Table Functions are slightly different than the other functions\ndescribed here. These functions take any number of `Expr` arguments, but only\nliteral expressions are supported. Table functions must return a Table\nProvider as described in the ref:`_io_custom_table_provider` page.\n\nOnce you have a table function, you can register it with the session context\nby using [`register_udtf`][datafusion.context.SessionContext.register_udtf].\n\nThere are examples of both rust backed and python based table functions in the\nexamples folder of the repository. If you have a rust backed table function\nthat you wish to expose via PyO3, you need to expose it as a `PyCapsule`.\n\n```rust\n#[pymethods]\nimpl MyTableFunction {\n fn __datafusion_table_function__<'py>(\n &self,\n py: Python<'py>,\n ) -> PyResult> {\n let name = cr\"datafusion_table_function\".into();\n\n let func = self.clone();\n let provider = FFI_TableFunction::new(Arc::new(func), None);\n\n PyCapsule::new(py, provider, Some(name))\n }\n}\n```\n\n### Accessing the Calling Session\n\nPure-Python UDTFs can opt into receiving the calling\n[`SessionContext`][datafusion.SessionContext] by registering with\n`with_session=True`. The context is passed as a `session` keyword\nargument on every invocation. Use it to look up registered tables,\nUDFs, or session configuration from inside the callback.\n\n```python\nfrom datafusion import SessionContext, Table, udtf\nfrom datafusion.context import TableProviderExportable\nimport pyarrow as pa\nimport pyarrow.dataset as ds\n\n@udtf(\"list_tables\", with_session=True)\ndef list_tables(*, session: SessionContext) -> TableProviderExportable:\n names = sorted(session.catalog().schema().names())\n batch = pa.RecordBatch.from_pydict({\"name\": names})\n return Table(ds.dataset([batch]))\n\nctx = SessionContext()\nctx.register_batch(\"t1\", pa.RecordBatch.from_pydict({\"x\": [1]}))\nctx.register_udtf(list_tables)\nctx.sql(\"SELECT * FROM list_tables()\").show()\n```\n\nWithout `with_session=True`, the callback receives only the positional\nexpression arguments. The flag is opt-in so existing UDTFs keep working\nunchanged.\n\nThe injected `session` is a fresh [`SessionContext`][datafusion.SessionContext]\nwrapper backed by the same underlying state as the caller, so registries\n(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering\na new table or UDF) propagate to the live session because the registries\nare reference-counted and shared. Configuration changes made through the\nwrapper (e.g. setting session options) do **not** propagate — the wrapper\nholds its own clone of the session config.\n" + "source": "\nThe `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`.\nThere is no `pruning_predicate` and no `required_guarantees`: the\nscan has to materialize every row group and hand each row to the\nPython callback just to decide whether to keep it.\n\nAt small scale the cost difference is invisible; on a Parquet file with\nmany row groups, or data whose min/max statistics line up well with\nthe predicate, the native form can skip most of the file. The UDF form\nreads all of it.\n\n**Takeaway.** Reach for a UDF when the per-row computation is genuinely\nnot expressible as a tree of built-in functions (custom numerical work,\nexternal lookups, complex business rules). When it *is* expressible \u2014\neven if the native form is a little more verbose \u2014 build the `Expr`\ntree directly so the optimizer can see through it. For disjunctive\npredicates the idiom is to produce one clause per bucket and combine\nthem with `|`:\n\n```python\nfrom functools import reduce\nfrom operator import or_\nfrom datafusion import col, lit, functions as f\n\nbuckets = {\n \"Brand#12\": {\"containers\": [\"SM CASE\", \"SM BOX\"], \"min_qty\": 1, \"max_size\": 5},\n \"Brand#23\": {\"containers\": [\"MED BAG\", \"MED BOX\"], \"min_qty\": 10, \"max_size\": 10},\n}\n\ndef bucket_clause(brand, spec):\n return (\n (col(\"brand\") == lit(brand))\n & f.in_list(col(\"container\"), [lit(c) for c in spec[\"containers\"]])\n & (col(\"quantity\") >= lit(spec[\"min_qty\"]))\n & (col(\"quantity\") <= lit(spec[\"min_qty\"] + 10))\n & (col(\"size\") >= lit(1))\n & (col(\"size\") <= lit(spec[\"max_size\"]))\n )\n\npredicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items()))\ndf = df.filter(predicate)\n```\n\n## Aggregate Functions\n\nThe [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined\nAggregate Functions (UDAFs). To use this you must implement an\n[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed.\n\nWhen defining a UDAF there are four methods you need to implement. The `update` function takes the\narray(s) of input and updates the internal state of the accumulator. You should define this function\nto have as many input arguments as you will pass when calling the UDAF. Since aggregation may be\nsplit into multiple batches, we must have a method to combine multiple batches. For this, we have\ntwo functions, `state` and `merge`. `state` will return an array of scalar values that contain\nthe current state of a single batch accumulation. Then we must `merge` the results of these\ndifferent states. Finally `evaluate` is the call that will return the final result after the\n`merge` is complete.\n\nIn the following example we want to define a custom aggregate function that will return the\ndifference between the sum of two columns. The state can be represented by a single value and we can\nalso see how the inputs to `update` and `merge` differ.\n\n```python\nimport pyarrow as pa\nimport pyarrow.compute\nimport datafusion\nfrom datafusion import col, udaf, Accumulator\nfrom typing import List\n\nclass MyAccumulator(Accumulator):\n \"\"\"\n Interface of a user-defined accumulation.\n \"\"\"\n def __init__(self):\n self._sum = 0.0\n\n def update(self, values_a: pa.Array, values_b: pa.Array) -> None:\n self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py()\n\n def merge(self, states: list[pa.Array]) -> None:\n self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py()\n\n def state(self) -> list[pa.Scalar]:\n return [pyarrow.scalar(self._sum)]\n\n def evaluate(self) -> pa.Scalar:\n return pyarrow.scalar(self._sum)\n\nctx = datafusion.SessionContext()\ndf = ctx.from_pydict(\n {\n \"a\": [4, 5, 6],\n \"b\": [1, 2, 3],\n }\n)\n\nmy_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable')\n\ndf.aggregate([], [my_udaf(col(\"a\"), col(\"b\")).alias(\"col_diff\")])\n```\n\n### FAQ\n\n**How do I return a list from a UDAF?**\n\nBoth the `evaluate` and the `state` functions expect to return scalar values.\nIf you wish to return a list array as a scalar value, the best practice is to\nwrap the values in a `pyarrow.Scalar` object. For example, you can return a\ntimestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp(\"ms\")))` and\nregister the appropriate return or state types as\n`return_type=pa.list_(pa.timestamp(\"ms\"))` and\n`state_type=[pa.list_(pa.timestamp(\"ms\"))]`, respectively.\n\nAs of DataFusion 52.0.0 , you can pass return any Python object, including a\nPyArrow array, as the return value(s) for these functions and DataFusion will\nattempt to create a scalar type from the value. DataFusion has been tested to\nconvert PyArrow, nanoarrow, and arro3 objects as well as primitive data types\nlike integers, strings, and so on.\n\n## Window Functions\n\nTo implement a User-Defined Window Function (UDWF) you must call the\n[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract\nclass [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator].\n\nThere are three methods of evaluation of UDWFs.\n\n- `evaluate` is the simplest case, where you are given an array and are expected to calculate the\n value for a single row of that array. This is the simplest case, but also the least performant.\n- `evaluate_all` computes the values for all rows for an input array at a single time.\n- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank\n information for the rows.\n\nWhich methods you implement are based upon which of these options are set.\n\n| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement |\n|---|---|---|---|\n| False (default) | False (default) | False (default) | `evaluate_all` |\n| False | True | False | `evaluate` |\n| False | True | False | `evaluate_all_with_rank` |\n| True | True/False | True/False | `evaluate` |\n\n### UDWF options\n\nWhen you define your UDWF you can override the functions that return these values. They will\ndetermine which evaluate functions are called.\n\n- `uses_window_frame` is set for functions that compute based on the specified window frame. If\n your function depends upon the specified frame, set this to `True`.\n- `supports_bounded_execution` specifies if your function can be incrementally computed.\n- `include_rank` is set to `True` for window functions that can be computed only using the rank\n information.\n\n```python\nimport pyarrow as pa\nfrom datafusion import udwf, col, SessionContext\nfrom datafusion.user_defined import WindowEvaluator\n\nclass ExponentialSmooth(WindowEvaluator):\n def __init__(self, alpha: float) -> None:\n self.alpha = alpha\n\n def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array:\n results = []\n curr_value = 0.0\n values = values[0]\n for idx in range(num_rows):\n if idx == 0:\n curr_value = values[idx].as_py()\n else:\n curr_value = values[idx].as_py() * self.alpha + curr_value * (\n 1.0 - self.alpha\n )\n results.append(curr_value)\n\n return pa.array(results)\n\nexp_smooth = udwf(\n ExponentialSmooth(0.9),\n pa.float64(),\n pa.float64(),\n volatility=\"immutable\",\n)\n\nctx = SessionContext()\n\ndf = ctx.from_pydict({\n \"a\": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0]\n})\n\ndf.select(\"a\", exp_smooth(col(\"a\")).alias(\"smooth_a\")).show()\n```\n\n## Table Functions\n\nUser Defined Table Functions are slightly different than the other functions\ndescribed here. These functions take any number of `Expr` arguments, but only\nliteral expressions are supported. Table functions must return a Table\nProvider as described in the ref:`_io_custom_table_provider` page.\n\nOnce you have a table function, you can register it with the session context\nby using [`register_udtf`][datafusion.context.SessionContext.register_udtf].\n\nThere are examples of both rust backed and python based table functions in the\nexamples folder of the repository. If you have a rust backed table function\nthat you wish to expose via PyO3, you need to expose it as a `PyCapsule`.\n\n```rust\n#[pymethods]\nimpl MyTableFunction {\n fn __datafusion_table_function__<'py>(\n &self,\n py: Python<'py>,\n ) -> PyResult> {\n let name = cr\"datafusion_table_function\".into();\n\n let func = self.clone();\n let provider = FFI_TableFunction::new(Arc::new(func), None);\n\n PyCapsule::new(py, provider, Some(name))\n }\n}\n```\n\n### Accessing the Calling Session\n\nPure-Python UDTFs can opt into receiving the calling\n[`SessionContext`][datafusion.SessionContext] by registering with\n`with_session=True`. The context is passed as a `session` keyword\nargument on every invocation. Use it to look up registered tables,\nUDFs, or session configuration from inside the callback.\n\n```python\nfrom datafusion import SessionContext, Table, udtf\nfrom datafusion.context import TableProviderExportable\nimport pyarrow as pa\nimport pyarrow.dataset as ds\n\n@udtf(\"list_tables\", with_session=True)\ndef list_tables(*, session: SessionContext) -> TableProviderExportable:\n names = sorted(session.catalog().schema().names())\n batch = pa.RecordBatch.from_pydict({\"name\": names})\n return Table(ds.dataset([batch]))\n\nctx = SessionContext()\nctx.register_batch(\"t1\", pa.RecordBatch.from_pydict({\"x\": [1]}))\nctx.register_udtf(list_tables)\nctx.sql(\"SELECT * FROM list_tables()\").show()\n```\n\nWithout `with_session=True`, the callback receives only the positional\nexpression arguments. The flag is opt-in so existing UDTFs keep working\nunchanged.\n\nThe injected `session` is a fresh [`SessionContext`][datafusion.SessionContext]\nwrapper backed by the same underlying state as the caller, so registries\n(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering\na new table or UDF) propagate to the live session because the registries\nare reference-counted and shared. Configuration changes made through the\nwrapper (e.g. setting session options) do **not** propagate \u2014 the wrapper\nholds its own clone of the session config.\n" } ], "metadata": { diff --git a/docs/source/user-guide/common-operations/windows.ipynb b/docs/source/user-guide/common-operations/windows.ipynb index dc3d2d584..007c9a82d 100644 --- a/docs/source/user-guide/common-operations/windows.ipynb +++ b/docs/source/user-guide/common-operations/windows.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], @@ -179,7 +181,7 @@ "cell_type": "markdown", "id": "59bbdb311c014d738909a11f9e486628", "metadata": {}, - "source": "\n## Aggregate Functions\n\nYou can use any [Aggregation Function](aggregation) as a window function. Here\nis an example that shows how to compare each pokemons’s attack power with the average attack\npower in its `\"Type 1\"` using the [`avg`][datafusion.functions.avg] function.\n\n" + "source": "\n## Aggregate Functions\n\nYou can use any [Aggregation Function](aggregation) as a window function. Here\nis an example that shows how to compare each pokemons\u2019s attack power with the average attack\npower in its `\"Type 1\"` using the [`avg`][datafusion.functions.avg] function.\n\n" }, { "cell_type": "code", @@ -207,7 +209,7 @@ "cell_type": "markdown", "id": "8a65eabff63a45729fe45fb5ade58bdc", "metadata": {}, - "source": "\n## Available Functions\n\nThe possible window functions are:\n\n1. Rank Functions\n : - [`rank`][datafusion.functions.rank]\n - [`dense_rank`][datafusion.functions.dense_rank]\n - [`ntile`][datafusion.functions.ntile]\n - [`row_number`][datafusion.functions.row_number]\n2. Analytical Functions\n : - [`cume_dist`][datafusion.functions.cume_dist]\n - [`percent_rank`][datafusion.functions.percent_rank]\n - [`lag`][datafusion.functions.lag]\n - [`lead`][datafusion.functions.lead]\n3. Aggregate Functions\n : - All [Aggregation Functions](aggregation) can be used as window functions.\n\n## User-Defined Window Functions\n\nYou can ship custom window functions to the engine by subclassing\n[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it\nvia [`udwf`][datafusion.udwf]. See [`user_defined`][datafusion.user_defined]\nfor the evaluator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python window UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions —\n the evaluator class is captured by value via {mod}`cloudpickle`, so\n worker processes do not need to pre-register the UDF. Any names the\n evaluator resolves via `import` are captured **by reference** and\n must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + "source": "\n## Available Functions\n\nThe possible window functions are:\n\n1. Rank Functions\n : - [`rank`][datafusion.functions.rank]\n - [`dense_rank`][datafusion.functions.dense_rank]\n - [`ntile`][datafusion.functions.ntile]\n - [`row_number`][datafusion.functions.row_number]\n2. Analytical Functions\n : - [`cume_dist`][datafusion.functions.cume_dist]\n - [`percent_rank`][datafusion.functions.percent_rank]\n - [`lag`][datafusion.functions.lag]\n - [`lead`][datafusion.functions.lead]\n3. Aggregate Functions\n : - All [Aggregation Functions](aggregation) can be used as window functions.\n\n## User-Defined Window Functions\n\nYou can ship custom window functions to the engine by subclassing\n[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it\nvia [`udwf`][datafusion.udwf]. See [`user_defined`][datafusion.user_defined]\nfor the evaluator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python window UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the evaluator class is captured by value via {mod}`cloudpickle`, so\n worker processes do not need to pre-register the UDF. Any names the\n evaluator resolves via `import` are captured **by reference** and\n must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" } ], "metadata": { diff --git a/docs/source/user-guide/data-sources.ipynb b/docs/source/user-guide/data-sources.ipynb index 6374e4cf1..c2591d466 100644 --- a/docs/source/user-guide/data-sources.ipynb +++ b/docs/source/user-guide/data-sources.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/docs/source/user-guide/introduction.ipynb b/docs/source/user-guide/introduction.ipynb index 9861d936d..756d29aea 100644 --- a/docs/source/user-guide/introduction.ipynb +++ b/docs/source/user-guide/introduction.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/docs/source/user-guide/io/arrow.ipynb b/docs/source/user-guide/io/arrow.ipynb index 548183e25..2c3ed5787 100644 --- a/docs/source/user-guide/io/arrow.ipynb +++ b/docs/source/user-guide/io/arrow.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/docs/source/user-guide/sql.ipynb b/docs/source/user-guide/sql.ipynb index 1e2933ee1..f12b45d2a 100644 --- a/docs/source/user-guide/sql.ipynb +++ b/docs/source/user-guide/sql.ipynb @@ -6,7 +6,9 @@ "id": "7fb27b941602401d91542211134fc71a", "metadata": { "tags": [ - "nb-setup" + "nb-setup", + "remove-input", + "remove-output" ] }, "outputs": [], diff --git a/mkdocs.yml b/mkdocs.yml index 1366d92c6..26944e615 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -60,6 +60,10 @@ plugins: allow_errors: false include_source: false ignore_h1_titles: true + remove_tag_config: + remove_cell_tags: ["remove-cell"] + remove_input_tags: ["remove-input"] + remove_all_outputs_tags: ["remove-output"] - mkdocstrings: default_handler: python handlers: From b3dd1997b126cba7731abb73b4a669107d24c1df Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Sun, 7 Jun 2026 17:55:02 +0200 Subject: [PATCH 05/18] docs: scrollable notebook output, live display() repr, polish * `theme_overrides.css`: force `white-space: pre` + `overflow-x: auto` on `jp-OutputArea-output` and `jp-RenderedText` (and their `pre` children). Material's default `pre-wrap` rule was breaking wide `df.show()` ASCII tables mid-row; with these overrides they scroll horizontally inside the output box. * Introduction page: replace the static `jupyter_lab_df_view.png` screenshot with a live `display(df)` code cell so mkdocs-jupyter captures DataFrame's `_repr_html_` output (styled, expandable) at build time. Delete the unreferenced PNG and now-empty `docs/source/images/` directory. * `mkdocs.yml`: rename the nav entry for `basics.ipynb` from "Basics" to "Concepts" so the sidebar label matches the page H1. Co-Authored-By: Claude Opus 4.7 --- docs/source/_static/theme_overrides.css | 19 +++++++++++++++++++ docs/source/images/jupyter_lab_df_view.png | Bin 150303 -> 0 bytes docs/source/user-guide/introduction.ipynb | 9 ++++++++- mkdocs.yml | 2 +- 4 files changed, 28 insertions(+), 2 deletions(-) delete mode 100644 docs/source/images/jupyter_lab_df_view.png diff --git a/docs/source/_static/theme_overrides.css b/docs/source/_static/theme_overrides.css index 16a714b48..a72b28bbb 100644 --- a/docs/source/_static/theme_overrides.css +++ b/docs/source/_static/theme_overrides.css @@ -52,6 +52,25 @@ display: none !important; } +/* Notebook code output (e.g. `df.show()` ASCII tables) often exceeds + * the content width. Force a non-wrapping pre with horizontal scroll + * so wide tables stay legible instead of wrapping mid-row. + * + * Material's `.md-typeset pre` rule sets `white-space: pre-wrap`, which + * would otherwise wrap each row across lines and destroy column + * alignment. */ +.jp-OutputArea-output, +.jp-OutputArea-output pre, +.jp-RenderedText, +.jp-RenderedText pre, +.md-typeset .jp-RenderedText pre, +.md-typeset .jp-OutputArea-output pre { + white-space: pre !important; + overflow-x: auto !important; + max-width: 100%; + word-wrap: normal; +} + /* Center the footer copyright/trademark block */ .md-copyright { text-align: center; diff --git a/docs/source/images/jupyter_lab_df_view.png b/docs/source/images/jupyter_lab_df_view.png deleted file mode 100644 index 9dafb4f61d3a82b6f51e03fa43f3c97a4f8bde0f..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 150303 zcma&M1yozj)&NQkDGtTGP~6=q1b5fs1P{SoT8b5?c(I}_UR;W_xCgfYDeeTfATRyy z{bjwk*1P{%=j6kBs( z6cj8e2LM1*2>_tgbO+lyI0I2o6hEaJpc(2666cy|)77Iq)s_K=zE45Xma%_QMDkl+ z@o5-=HEq=VgxrfcPU0W=<*%cu@Wia_@DGBtWTlNrFuga`UhcG32ZB4mM|W3we2*d? zMi+BE^z3nFk zo>9Q4bjbze;nA4?CRCgnMUlV9;ZEF)vnLYe7n`4zk|c^O8CX)loW1+UQxFCg-dZ?s zcK)wa-t3ZJhz-G$8t162C$#16EuXParWa`;%zuDyu^S|#a%2PZ(NVlmdpcTLl5g<7 z$v8g)_23PkXF5pj*xae{ffSR6M@hS)TLv&iG&7jSR@D(wQzxA&F7ck@kwg_`nV-iE z7qTr^T>HQHR&sy=sPFg;!;!(&@ zvJZWV)jW0B?Xtk>aSi_(-W8}W-s6wQ^=jiqFAGReg|VMCDBo+BKu~4S{EzZRFtCGK zxsK3K`F<|*X}mU#4%<+yMrg*%8o3YLJOow(QB-HO>s_gWg)zJfNgo9X94gp@F{7+s z6eOaAOX??bV#Mn$rf+RwkC8yLSvF)@Pd?m@Tx-e&|FEStpq9$UoMptTAwczd?q^Mp zy4^+c9F=7&-V0L}!%aB{ckS-IVXt8}>)0p##dX`zY&wC^#H_UR94S_>4G3I_|`fu~)wE4OC8-*zxi-_brHdy7JuGV&*i z!7l;bZ2c|%HR0{EV^MFEG)d&Js`@AJ0X z?!8bxH^^_lz$PZ~H}!j>gL{L57-dg}UCxb^`xV#WpK1NwWD+2)E zkqDvSG_##|cin%r6B?*ZbsMhjx>iDf`L0(Tm=BrNU#0NIu4&ZV_!WK0`2tOhW}lPV zQowgw)a+OQNJt^<9utEkox4LCdx)*jYR={z4qkg>m7`%DFwDMhu;>T++IvYPRaZ%a zYfgP@^W%?c9#Hct*pvO#1=Jdfg7U}1-NNF}jt}Pn0}4Xo&!yC9PMu|!M&wHrlKA7I z&re@O4s$-gPScDOQ==h$=3(({d;RIs2ccs$X<97V559?5j9oi?tD@=eOD#|{FqPku z8KUZcys#jgf9V}6@dxEQTK_KxSDbgKpg*Y1Xa*W2zE5s{i5-(k;}OUJcA_M$LJePE zMqzfz1}4!mJpCm1IZ2n{89tr14Jt1#3qYTCKZ!$8pq54?G+UnY_2B^32L9y7HhIZ> z(z0mBLiz7Rq6EMKc7Dn@1*WfLJFn`|a}qeE=L+<8T_g1U#WI5>uH{4yex^y zw89}Jw6@kVe3=%n^ux%IML%B8vcZr(aBS`eP6Y`fiqK7T>=)IampHwIuGEq*^1`|L z=GMhdMBB(ZBc!?z$KdDWzY{Gqj$gZ?TVs%gm4*?>h**lPa<8&4shEcO$iCJ}w^uuV zAHb8DA(yTGPIc++l8U&TGm|#~LW$wCWr-jP`=|D2RxN_=q%8Ns01@C{r6)5xSh;#kFs~mlZpH8`7t$X4Nw-Xa!uP z>`~1K@sp26Z)vFHZ%(C+mpRzk<2ukx?>NRgQVMh|Iyby*5N#-TK#o8mzlQ*c)6rzx_k~Ki$=b=6lWR_+ zbBkqO^5x0#6QjG%q6v}-9dW^FE5&Nljk8J{vEja#gH+I#2w z&D)DxTrAgTVjoFAmZDN%kCC6_%uydnWDB)V+H8#e!##J@q;G319Mb<&b z4|?r-O%1sM{jCG7eYYLhqkRzDXtmT)FOvd}DM6_BM^F}(Y`8`6b`rV=Fiz*6J@Xv|mzGe%S;J4LATFL9Sp&kFmhvA*(~%b}ZMIiO(r8oRi2rAtPdp!okf$t62u%sufjOObi>S~c4tk&Bd2Z^KPT>w>R#9Nxa|_} zvW*akERX09uf`3bv?QVXWKa1q9)qIJedJzoSW(L+=vCC3Y-OxJ9*9cf&8n(>0kAv9xGq>|*CAXhzE4^qvTydT07Rj_~_R6{%lzE0Z*ODyU4szQ4^GVir@$K2p ziR`K_igd*cCC(bVG$i~k7tK{CF6%0Nuxty7eAlktPpE_*jMRBWlApwOEm-LUSiB7O zNL*euMMLxYl*(22*K7Qoe7UY4?)2`cKO|U$|1oOSxQ#wP!pe(HC6mYu&fLswmuYlu zKi1Q?pDn4lTF_b$t3`AF?0&iR*+a)8A*x<>pE&00K*nU223ZgQAP z<2Ici;VJoP!)iic%(VP%%(>PFu(aIo$kIcCB)Iq?~>`R&ZjTtg=gyOQ~2 z>vYg=AFu~l02E3`ASH;5hulQ;SgN)DY^7BNtLcUJMbQk&u<5W+Wqwss)vh5$O`~N_ zZ!2)WqXP0Sr114uD{!&pX2H-E#Wh8h%Q?*O=b*jLLWxFJcorCpv7l`L)YI!oZEmpG zp8|b?jy?9>Rr>$@eOY^G?i1zhc@Vq2@a|B{w0+ohZj!uS-gv4m;O?kV;zIzrBb8&* z!OXHrBkn=+Q9y^3*W>83RkSZSytuQ}wc@&d8NtooBa^8*Ctd=H%&j`8%us(O@EHI13ZohkJ2T_eKjA(k1c0t z8bb~aUE#tH%*R^8>&^XFqJBACxf>EUlLC({hYmM<$#yf#oXh?Gp#h7z2>~k?GIwhp z;K95vA^IKlettV#kDbQ1eV1PCpWDcN+ppsuvZ%nOFu@CjE2+SA#L*dI68DzIJJ=>z z{E=mMst%&lnb=Wr?}%tPF}ZF+JZpLO%&IDk?UCr0h1}HC6W{h{D8{@fp1zWW@jBU$ z8!xo7jW*Hmm@0(nhhoG~8jc?&N=P1-m!4X{1by#9Z}--pM3p?jgJX} zei0xa->*;q3mUcd>$CqVKeb1;p-Afhl$4Nf9cyBFlfPxn9%$1LEl<@!C*LlNJDW2hs|1a&vOOmV8M|ODpbfV=JmHEC28A$S;Z4 z_MV=uqFh`)K0cg2e4Jo+J1!m(5fLtKUM^l<4kQGJho6h5r7wqz2mQYW`LA(gfgaZG z4z8XKU>DlI<62sQy*wpezy3SXe_#LlpFm%S|C!0f$vS|LG(*lIDaa!z{tx{Br{#ZU{9io{|EDL95D)+V?)tx){@-2oJb>;1 zFbFxPr{w>v*S|ae@5X<36zBSj`u`%uzv%o=Et1fZFU7h3d(k9cPI6u|A=i=8K~_T- z`9|)tztfT|r4ZxbN56!z!=s5(Ab0597X#U~hl+n)c~`yseeqMpV=UvJa)8N4@`#Hj;!Re2j`M$*96k08614BMhFRx7qJ;$Dwb`Zm&t!dqPpX0h zjoY-=D*6A%tnbq=sfUwt6>ZBo2*14S6>-+Cv z`r>|dKHkF*GVOA;?Chd&x6({p202@I3TBDs%tP)EQmX@j`%U_?7P`+n9xkfwq#kb| zCh0Ix({e45!MmCZ6SmFg!58J`K%r$%oxW3_PK2r8enp*ab;$jG9Y@nr?H$7*8S|69 zrf+${q5(%9d1B-Hs0c@VLkkGx3U~?azzKJEvlyr|V|66M{LC zLH>e>gSsnkYn}6*7_JUs#I8ojV(U*+?}axez*Ifs&VI|y#|QXhv>oHKA^Y4uZzw2z zIT&%++iQM1nQlvB=DTXoTTrfU^fWo2lKjP8AGQ&4UPJFmB_^UPnK!;W`H{r?AveS3 zwd)B|9fti@_p7)e!U0HL;)@QL6_9)T+}8DJg4BcWC350cIm^O0!689d;Mpv^`E=j> zQ512S-nq1RSDW6>9x&=;Hg#K>>i@jsG_~5s2NvXG{;BQAt9n@9Jm{ihEXSECpH}p0 z$%Ai2A-R{>gdGa*yg%3;PnWu#NT?m=>TIjtA5(-)#0jlrn1telZS~k;$ zcg`z*9jhv-$cx97jqygH}AJ)c77$xyL7d|5m)Z>U?aUZ$ljYgdRD^i);<( zn}8ae7wa7sV$9A9W-r~0{Qi7EtK}OLE*uyW+0IVpsOcdfi^vYTTK0KYD|{T-cGV6C z@zv%A-|kXD>7PL~2VZFZlDgSQ9&G1>h=O+p_X1 z)>qy2YUU@=1~y{wHVI52(w2~x5{A?alV=^iDIOfxljKmux2Mzb%@`D7J z^#vU}3qhUh3WCXg#%_~oXU^Q0j#q!Z|J5{{$Fkv8@|ArX+)w4hzDTgFljW}v_~#kT zfPGVDw*Ou!y-%psb37Qsp)ppWey(9&s345k#5sxM$FHYpAPzLy_opX%)&6=WdvotR z)EcL?RT@B9dr}2vaUBX}L4q?PSbGnBHI(Gj?U$cj)*~v)t2PA{i zX8TpX+rHRVulzkB@_hL~hkrc^4%Xp6z1!?SEW>Sw+3R9q)5e{bi%xv6@)&*Zg4YQ1 ze&iX?GN71{UePn<`mSMF>fYVML-O*E`QIefoL*FbD7H>9&97y=r?nI=yasc;7g@x% zGOTY!eRy-itXiQ#9k(;pobhDg3Y*{aR*~EXjTO&myYA&0yPUViMJx=szr!+OUZ^x3 ziKBAs!C#<&UH44}+|Bn(rUjkEm^&YaY>UA%PwdFk4+9=dmgj)Hp#|bs&(VMFi1BhR$@uhixj378)h6i7TdFwZw zl^xmXDrP>5!7|cgnU%tMb3LIzv#c*)tlfJoz3_3WuNkxTr6;jjS$$nu8ej5YVt8^- zz{(kQ!t0iJ5BDiA?^78G7v0PmvUx%c@gyqWKOYEJu6&8%k9Jas1+4xSGPORn53zhp zCM&0DX(wd|et!&Q5M{}Qx(h9f59}XUfue;8$mJW=hoT?tG?P(D(km%fd&AqW#uU*} z{j1mYaj?&A0~Fib0K|ngk{6{KL`GkEt|tkS|8wx#ld?$&9zW}6mlvko?rHIC@A zM0Nkly)X2MWv&k2*S!RgTTiVy5zB}$k-t?sqGC+A^CS*z0}p^9KBMVr{b0~=wqO4q zM~l7O_v`b`fP&=bpj{R7vJ^?Pl&~VwOZy8gscxFb>;8qRpGS*UVYSIgk!sOcMvzI_ z{1N77D-CXVvC2qxgoRc7VGiw9%sf9G8;4zQFi9;azwAADf<`%hE3ubaYCphR!n{O^ zp~Mi+7_LhDqHMxxJBVHz{?M;~3fan3BjOD_~bne9HrvM_4`J`{?Z zxh+hP{L+56$)&MXnp{W0q|0H)7(<82#FJ~Fx8MPhsO!d+YTH`{h*XgP(t!5zEOqF; z${B46)P6lK?dzX;FSB5Gwx!*6{mD!xV{Y2CeA@MM=#;V13wC)yj3w7>W&&>B?K0EV764w~oq!mE$OC+hhp1I3m{ReG`NCO<=6EQs?=&L@UpG zasSObgo=92HXm@XHg#T+)xJ)$4dG*@i=XUW7xn&uo`n~_y$JyI-Y4G(t&B0h!=Te* zEEKd^t%><&`-{FS>v1;2c`=DqDPcsY2s*bhPxSII^Q)l3;D~n8 zH{#`Hvvv#sVmcsR*RPu&e7Pj_IK;;Gbe#@6`<+Pg@X5a}c@E@7ZQ|YC%qcC4W&Sz) zwMj!CKumv^!M1Jwcxy08>aY|qemXD``Lq7eD=p05_Epqi*2;Ma*0(2v$LN8FPUczT zW@grL0@1gJ;aq{1-VlkaS9C4+%pF=y2CWmncZ&`r*2IDkP?*^l*%^0RUea3aAchW~ z*#5BZK}ca1eW~Ipvm^Mq@zu(EZ&^UvP{reizWk4kc_cDhg<(v(?1u)6YGy09vrV?l zx^@g?Brka7y-VI6Zv%Cv#KPyzoF zK1y#c?*7|CVlLV$x$RiqG$$|e(GQl-KjQl#x@jIX`2$JJ15aW_y81v?E>;cjAKf?+ z#Tg;=5i@l;O)LIReKt*uDx57gXsuScn+UZ+0f;)J&VVn)!y{YFth7qlO?dP`eln*5BbJN4><<1GS|05A^ z1up=7Q+sPz#_XF=AkMSe^1FW5Zf-+JWt67X z+fey*6x=1E;#;nN8OTiFv|C69Jks9S87z+iC~)rSy?I{{cY@A)ReMy%=JvW^YcKsJ-NZWOy&%U%|{{f2>0-B?qN3qXVTP z^E;_Q9wD8L)pPD8S#+4TOHCwLVatm-57({xVRgloRx(g*I~%My#(E7hs;(wb*LOwv z8(JU?55dfKO>EyeQB8NZK)Z#JH3B5gMtxI1%KWnpPkOKfrRH#iWE&u_y{&nCH0E=pN$HU0BLSE?hD%M#zgmRcC56uFXMiH04E{VpliYF zLOIG{O{j&fx~Dhx;&{7qHM~$G&nMXj#q^0N$4bBu!=fSS*2FIDUCFZTW`;)L+`iC_ z1iu!}1o1c6(}2Ys6Irv(yRd{ONa6#-<(-?@D60Tcl5TJY647@}SFr)pG(mXH|<=3fnbs5IDfZL?JfG z(Kki#=HSbNd_h41zTwG&LOA}=TTF$g#%sVJN)+U%B$Km zvNq1=O$SBW&#*3jz!xeA!LwKRN^2tCSRlbgud^|+YLFv`n|_z3rmbOqd!Kz%3U5j3 z0E&T?;rR5=(M^oFgYEC#JN6#H;>Awpr#gow9TI!PugV!+2seMyG`tnV>&B+*SL5E2 zedbiXsmcT(Mr}iyhmCri?xv@*na|QKG2$8O6V$lX7tZk-YpW24X%q~urf}@41kBW+h>hk(dm!M9lL zDZBL(J1}1*GS_Q$(g_L}pIE|eM29Q?T`xFR#GA##ZR?YaRA#W-9z{&Q;JsL}fr{IX z=W_6hf(qB=iL;pcAK`h~Nb+1Xd1|ivcb%p1d>4^0>1h1Mu&axBz{3OcM+ZLtZlAJT z`E4b!o^KzYq;)7#nPx#$boLvtjeLy)ijBF*_KPwO-cV7o{q>Tx<018&hrkrfk$pSt z9+wFASSqZ;tMCpDNc9xTi5XjsR^JY@{8&r0;pD>1)$a$I3tN>y1EYuP}Nb)&X`}rht6X|}Z`kBko!g-?>Ct7Rm z?0}6Wiosvu5ZEK?Q%g~6zw>2s_DF0@?gS~q>W-Of@7dMzr>E;EdW2-Q^X#I=2YTpP zy}6MVWk%O89m#ZtVpMRW0?_m#)HY!&nFFF;6+-^++oe_Vdm)gbby{0&2s9`w}_=teP1hvPGCb&d!i@(P`-osR;FhU1J45yV|J_oWYN0Jr@m3;W{Wb+4jzsDN&LQE1fRws-sekfmk zF5%ETRvd_1EO zz?1iw$0uBC%WNwY9+wNR+{T;Jf#LNzgoK<>9(mng%q3(Y9wf+yD>}f9k%T%!J{@+H zIQ}<{1)tlYR}}k_wM$@AizXYX=t4@EdKm@enjDNMuYtUv!72n|@u2fIrPF8`Z)f(4((9FE(3Qg^0bD@;J(FnE zY^3d7_Z>3q`SMbATtW-i4*?6Thhy zG<)!e$Q&k>8@!U30_e@|(8D`(x$s$;VaB&dWpc=US}oY z;R}DF14`itU#4RO`%#RRqHk5!IPIB7+4y^zQahP`bR0!Oh>I>v@f>A4Sd%!jNwiae zMGwn}Kh#gZCiyKHfNUd=?W(J;8$eU-|Ekm1i6B=rKCp-`?#~58QM*W8M63$Q+S4n) z%$B|*y<_6sY6Y*E$-Vz>lR1-S`N;z3(82Ev-mRMN*OQb7q+oVmAI?z=%~r^|GCU+N z+N)LkxG(}f{+g1vi1%WjyI7q;jyQ2W;5SWi z2|5nQ(AHy7&9%qv#itZU<7@5#XL`6EOYB7IU)`rxp zx8&|EB`btE-CXT1R2pe17=>hjgc&R-1@hn7sSC^NojJGLHPY1dHIRMU>3mi-*fiQ< z|6ZNU;NZ*0ck%KDnR0Z0b6GEKZLl@eQDq%oS!Zbf1$Yq-{0Pn}V+P z?yBz|PZ^TE)NG5CE<=fOiom161oa}EO$inq{ke9S`nH0?GqE(@&!S|EDjW^E3WW;a zzyp&pDl0THUo-1emSFqvL4Df`{}8Wu7*eTW5>`7M(~{TH=ZP-K(h_A|DBoZjo0|rk z{%Y7uIXSOpw$r)r_=X@2r(;V?7XPN3F{OTUiP$?g5W8p@+Uwh4Fxk#jAb#j@MsB?m z(T(b8my&eecBGmqfmwKLo%KOb_>dHaL;E;=R*$HRXF>E08R{q$7knYjlIGCyZzw~a zb#Qu|!fBNxa1-Jj`88LK_^WsE5PkA_y0KDP=q8@uPKlT!Jw&gJ(yjb9(LEl#-F`6{ zQ#G15H}CJ#09HK>we~k_CJyoI9ak3O!ZU`RC6ZMh8sqH86)Id zJ|}#293}}A8~p2RX6TyGi*-F4&`vAtB9_tQJ ziIaH$LBL_d;Wsr)?W={PG3Jh>C11WAyKRvZoP^}-wp593@f204ge&D{v64$-?hh%0 z=krHaWWzrKJq@722(53~qZCIVq`yVfs1{&aGL$sXG6n8 zZZ_lnh;&r_WTF(Rz@C;$seX_=8yD%_+@-%hZ27>JSy=|hV%lDPSezW8+F>kGk^p%L zs|?ZZnI~UAY(#h=ZLg*!_u)pTV~%dBL9ys553at(`wpxaFThGE=N=Q}!$n|`chbuD zhhDohF{2dQ@K(MpDIZ~$cnA?L;pJw|p}z5rebH{<{zIU7i7V~JPZ-?KSjKC}YtddZ zXx3wW#Pc*>^QvDkygFw6=EvD#mG~>|6SdQoRZ zI2^`~Z7c^F6=$zL&$xCvIXF$60zrlQuILvz`NL>Zf?7@+I5Z?76=w%$6An`z-wFzH zDC>wipNQp&#+>G!zW1&LHN83fUBb|jhji-;c)|202jdP1=b?N*-T;k{Qn)4Z5|}b= zj>7op;>6GYoG%$lN#v;%+hZCVH=Z9ZwR%bC@Jh}IeOs{;0}6O)-|4qRBHiWUgI0+Y z)(AS>6MtXAS^t zlL9krV!fgx1nC9ZyL8bAZy^Zdp6UVFNiMBdwfR+meF5cA4gh+~RU<)(APo*Nl2FTH z$ir>?qj&=bRUxeY!{|Cm(-7+r z`zw-`^!~6uUw#{}pXu{>Cpo3_%rVi5qNn=7i@e^hXM?`b_(6%o7BHK86`Q%PbOB#COg7llMJ#J#Jd3 z7_!jQ_wHphpW=9{#rK)orE}K(>Q)95<-Q3~YcS0-kO$ho%bX7KF1(iCIqn6=!(ZX7 zGn`}=<>rH7Zh?}Ji7=C2q^nZ}Q@h0^g<(=$RSG2?*>Mq3rH%t#zhs!wq*j;~mq<}1 zLT0tB;*{5fVhZNNz|d_0y)<`cF1RWYFwC^mnM^Dou$r+RKQPxwNG_TxS>wR8o{9PD zrf^K~Q;up!x*^z_o4TP#{07dp#i3VXef_pzjgE$lN`oubJD2693K#MS_8PvpZW~mQ z{L~ETI=6Q{g+Et*l@~JWkY3ZCB`YT_DFf_cxyi-_?Ix#dp~K$>a;*hH!9)3hR_Be* zeV1Ggo78)t{-ReBrLEr&i>O03%pO4E?NcC0N%F?x zk@~w{K@wOq+{Epjr}vsEPpJ;cG{<@&U4p+zn#RpQPa~L+vyj`S*W26EK9p%_ciN*4 zvB)-vWB1-jzs#f;rTp`#mi2PG_5bW zNVN-a`mZ#ss0-F?bbFJBYb%B(7OfjX)YsS!S+J=1oAwpxq*s*Lx(6& z{ra0hxvZFVDMAso?*6dy@p|vt?Th(yj{ziFZb6HEZfZP zG?#=`CE?RVgn`f#KcGpNP%R9L3BqQEFk)Zd*b{`KC5LavwKZCE#D0clP$=9%V(+$c z;u=BS4-+c((x+(#eIUB)d3i&=En?bO!c}`!_JjBL@qlifSP?<+w5=anm-_JOm|R)_ zfG9sWG%)cEkA;(ukNu z>Xy)PnngE>2;x{YoubX)F zP#zN5Yn(@of=dNCVum}+#e*72_a@^h`gnyVi3uS(eYfM;rA||3medTGM#Rw0mMLZY zGOk<-qg=N>GP$UqzVDMcgjgzGyw zGy>7WJcd2ZF+GsI0u1DBEX7Y=KtlO~G9Xv#~k6hg2aKBar+1)%aMh5H z^DUvw@(MEQTe;z2!s``5vgK-Gm*aI;GD~UnkAR4Rf%c$`>nCLCserKZqQG8Gz~dXC zF_McW-1Z&7__sX=8J)E7@Tx*5rvMb}`|rSkA&H3Sr=Qx!rBAG)1tk$EZ0@oF4OF=ONe6hPAAC@3iOXrQ^Ws1T8w>txP*&4}tObSIXS4I3>rx&QY?( znad&0$6phz*4!R^UD~5rF8xKoW#qMMf!u=dex&n$vE-{wf!F0Aw2?+ zf926>_@(<@!E^!5A2B`8an0E*&0K%W_j!poz>20d4AKuCTpP`PJDmYqFZ)lkN4~6mIE9MQ>$W8A{2Cuk)N6XD z;dONAXA*p-slM=%t-?YordSH~1rIzlWDbfm7Gg2~H!A<~8!41zSQonDU{rFq<2goq z>GoK^d|g<%mP2{q-!{B8j|c4x#B6E$LA7WszsYo_Hm9%zM49x0}p`b9wx0WP6?%HFpxh-X?N39!ALcvB|@}xo7Ox4=f{3^3oq|MQ!XOgfo{D z7GvG4w^I=vD+zRL=oI~}Gjar)9%hAbvqpE^jDm>^kAqfpqFJR{^P{y2T>dfPbvS*^f+a** z&YcDp?F-|g@oAac?6N?DA)oTA)|c}l)%SP5q!b5k3x*~nm$&hGC$SEp4{dQY&RfZt zf*gwW-1ZN8Phma;hj;43^DL@VS&`U7q?NqNpJ1)v{+_^Itu=8#)_cls2CuyI+S|#- z!iwX*Fk}>`b<fg?swg2uLE7cxz zbru2;_iL*6-KHO|Gx+)T%M8L5R&S!jkq+I71*BK-4C?aHvjDAwre@jOH9=~viGK{J zpU@&WjU+~>VDBKo`ag%U8mc`qlZmQW7n*dbKlnz6GNeWk$w;bsD z9@YI3<1i`9rIjTM>3;y}^qmcLPd5Y?(q5@+UOHO^gcDX!JLF_nr(76V=MCTfepoU* z3IOQm*o_HTQYu6||Kn$G5$#wJ}4TzVi&X z9c(G15YxY~U(4R-3Co=L^f%#4GwE&Yrjv)w!fua%t%*@fnt`~tJr7HR6sHjTD-SyE z1fpG9w|D}YPT{@)$-kjdhdLQbSVl1P$qla6Q-rm9F~|-5%=7wi15~S6?P23)-g2b} z4a8PG|KbycRO?r&2=RKZe{Y6K8Jny z;LehZg(4`pylOjVtg%Ah&+Z^>iKqDRON)D#cSD+6E_uhCIK$z&xU4G=89!lZf7{|9 zPoKnvy7KA7Os~y`s$wpMfN^AxNNqC?gvX3aD7t=(tXFv?8Qgz;w?T!#@Pp)7Bintx zh(!6PE*+11V@vr7ILvU&>71eVh)=FWL71tnJN-i36Bc%toFqTnclrHj5L95-xryjk z`#$9!ZoHr1->m9;nVSp-_g_}$aY4TIw?#|FEiv!;T(#qP!1|8ocIwB@hKSyR<|>if zTPAgXVs`bOJXN#gX4?sFJEC!9IDPrf!uBkyq$A%jKH03w8LZOXNps@g{*a42v8%82 zw|#Hr9&XZ7H{X3k2F)-#unD%|CmzE_58vT;Hx*|tv|f9mt-!zvKt?&on-ePXUP`0s zCrOsmMAvQG`R9z3U=Q=T!@o!Kg;9$5JA3Gb2U^)IomZ~NrykM5SirL6v$-u1PU`JdX zo06wPvmZ-)8kKU>{}K!qxR zQuMV0Cg;!Xo`>T8RIGTMF)(jG6f97{7F*9Ioc8&fK?6m+jpE$WwWi7EQ$7CLan!mW zebeGhp(1YJM}sH6%id;FlfTxvgOTi0Thc>mv=n@<90Q5}VLqa=ReV%OaW3>(C@n@a zKlUx}>b(UHbJoZ1#sny@f7sbkZz6VzSFAVhZ^HN=D#1P0lx_q7pS}r!H4*HvDe#14DWn=U>7pGG40L9pT)I819sxl?RI{+4*FB0YM47J1Z%78p!@v7 zV?4IZrHZ%pgMXH5X3cUBC3)XR(To5=B|p#g&5<4KJd`datRwRd^bFQ#ebN9hFF_(p0Iy&?(z8l=N;ib$7!c#OnH1|vI z(nrhRtG~y0V1$C2Or5GG<{vgmS56w{%%C`2U7Ebw!L%qO?}qVA(na7vQZGS}AIc(tTQ|>_g(GCCJyFQ?@4~4&o{z=}e(V(C zG9gcw<3Hi%q~1o5lBq5@0!cmQG^=jGIX~j-*W2+PlI-4+twCQrWK7>8RekLnn(4-Q z>H>~06!wdYpm!8o)1Y%V)u?$(FH1@e8EBm6-&_WxM9XFJ(=Ud1xF)sav=JQ!OwBe%b@0QOWGGgn;u5?zeOS0 zh$t^qpd8kVVYiyN@p6|4{<4(a9C@4K6|F9tsXo5^%0!A7=$=P)NEFOaIg&3^!Dc3i z1~SrJeby0RIn8_sy3JQ5vD!?P>XWyb`O8prxf;{~ERovqyydKAuGTU_b>ZK^#I0)k z*2R0f^Ru9w9sS#hhL$M`lS<$~?YpnX0?&w`NX<3=^+WK{;JhHNl4Gct3P9zb#2Mlb ztP$*|1VsC3Qs4n)FLX`qsjuI#ilo=JG3q@>qb4e~^!pQg8!{69r0A&ERFL324~zNH z+~v8k+Z@nKNc4#G%0fl$Ua{Si*J^uY8$Dp;y)C5k4%5NL7_-DdWki3|^HdqLTG&d; zlERtba(-SGV@RCEwJ2XYa=>Q{A@EiW2q4DxZ%)DJ``t@o-UjkPMl&>F=rE}qjg)ya z&AYFzDj1e9?)dnRx;!?2lT``VrgU=g>T(0}17qbsE?ILHVhPYS`ba+Q3no_S;;=}e zLlu3w*t+Q~E=4ZC$SL`YhI#2s%c+u$pUT)Z8{0^0Hd{L}tPZ8+2-C~ESenp330iIv(ed2$opXNDjtQ_^qt^~WZ zC4}6qldb+q^abf8DxgwkeJT8K*q!+ORbT1*7H#*0jWz1hEO*=u45I~PM#7@#M2%WE zMj#0(;p*tSP-ezJW|2Lsq36lhs)ckh>w78tdZJf3SVr|X%+{**0mlz#9dta|MnVmnr7?RwgSr=2lSg-C5Zp`9tq8joRW9(q_e&Kb9_NG^4cur5p*@TBJniZAB+P!lr$`Q@Y6-}L2G12@?S5xB6tGFQsSS(suF zfO43J*~Nk6EKbM8_auU#u@LXufdaF}-^4>z?&eowgpZ6SnW@d;ADpeIiLmg@d?7p4 zR?}UHDpHw^CX#EvX!OrxDj&%a5|oM*xvqir%jkoVHN7!HHglDj$oNRebvF%Tpvg3J z8o-{(G!g>y#x3NvSw?0B4|Y7c$G9z>=BSs>pkdjgy!Lj(;{ODxOn-1Z@VIasC_{`` z=SX*o0tcc&!7KY_o_s%$BJCR%MhD3E?xf$&mY@57A7J!T?Gj9>}URzBLBd*5*xJ1Crna4EL3oe2tXfXKTQ*1!XU*U#~ve&%o zr-5CTghgQpjrHln37Kjss^GKf@z$CZ=gG z9bu#T%Yr+RZ}kAg4*eqj^IOR6N%uQAj2E)SA_<-vqOKi|1{OfsjN765Eo$uE} z35WdN4(IX4w*of#7WrB<@E8OgszCHF<{CPJt8c*wKk9jiSnu<#nhDuRX4bbNxr1)j z3ZD;`!_+(BiPxe{(3b8c^Ir7NL%3(wf7)qYXe>=%GUhIS?Z6lPT^jG&)^z<-nVvV7 zFAAoYOBZMV%2mrqD~*PH+*jm_bm)49)Ze7-K^@f^!=~)G9AmQD&&_D9Jn{ZA3trc& zkxbQ}cP8|%WRjMK97Q9W~-qFa~!kt3)1JZFa& zK)hYFNE7U-qw15`S*YDS)NtIPa6+konJR+e7U)U3D#|R&^CeUEh0ai--nVzvlPz$Y z0hyL?E8z|?=Wat1qByr#%0u5jSVdg%G%{(j7A~Cv%iPE}J|{JiR{7(f!WGIDifvNs zzw7XXWms$g+}2+J(phTnygPFv`&qI<_d?K8(=wR4oyXN3bx)>i9q`U$JJg$bv zzX_IlkVLIf_uC>fXh(&$&TN)~znh4*4X1zu=@k#qepIIW$uXH^_TFQwxW4*rR6t@W z`Q>jCyO**qN|U#z;fb<8PB~J1u&lRuQ#00ZGyE{qlLrpG4iT2r135wOE$%k6PA>pC z5030;0RlEj)uIsbSu!BfN(e$dH7Ztx+YS^2=LaWb1lJyFu<}q^H^Q)y%D0n2|ORqZ#l51Fn4({lXVY^ zq8ny&YI9E*!hw*7C8{!WG5j?oxE-JB8vAx<;MJnZ*(uvI_glhN5_?5>+R`*4<9Cob z%QLegR3&sVZGWDmLd2%Fc#h8O_rvbLF6Da78iA$;(rTVzDs+kI;ft?oDatAdpolOE z${zPgYwbzw1R3*3pNjZI<1+;Z=mBk&cekc?>m_HfjhI`kqNsdRy!5~Ldw;x^w~8#E z^0+{>g)V_{kL zF<-6jS9b*QutVtSBb}Gz!=GCnUj3AS@YHn3Y%>NxgoJ91P zxgv5gMe64A;7M{Gb!oOtggtzy+6uSAwiBkJ99mht>77Sy_%bhkDs~V0g?$N^C6S`? zXZNaFvnzMbNXAf2b}3~<{Q#Z42dv@fD=9=Lt|Rff)6c`>a(k`0eZ| z@(_rFMhed(^PTR1(&ZX&mtgh7lNn{O@Vr;eQ3*bhWfsFMAxHPWN>clqe|lrzQ{;HX z+#LLMNmJP6P&lO7awLE*yxL>@%j8jtv5?zqLXanqgtD8*3WQcOXZpHM=uvOUxvNuZI6Sq+pLaW zKXchvaMzPe)9LUW;N5QFU6JS~0YrX;tftuW2Wc9F&skU4G8$L1g-yrUtQSI3Al$Pn zu;`6p8yPIY?F!XT)|!MLJ0jocVw?`Cy9|hMu6bS5>-LZYDi-IwC+mI!iAc$9uB-5KG$P}jm7t(YbJGKXVTmfryFu znsD+sC~nECjUne7-x}$=ii~$@uC`0@`ZbGjgDGE;Qw}a>0xrW3I)?oHp(Uf5vBcW( zl1`~{p@*zGS|8t)74qG`%Ndl$*B-v~a_GsFQRBq$!sO>Z&sOh|<7DsSNjt|1RW>GY zE9kGCIGw4}Z*K1iJ8N^B-E25M2(LEbybN)4C`?jaAU_}1FrYfeSdnO_1v81-X-LKnR$|h(kyIqX zARYy?+&Za{{0!6UUwP;|>ift2w7IzD!4j!+X2wC!aRFjeLt;-}OXwoz%a86wajrO@ z4YZMs0(h;LndQSVDu(7JgXd9vVJ`mIK|cYTHsTX$8LIAgUe-SuyF>i@&iPMu?{lB? zVh7V!-4y--wy;O&3|0uAboWhtg|yHXXqu#d-e{?6nRyRA7gBpB4}l+K4?pJ`C;#R! zC1t&XW#_&qF}uv@dIpzv|a|BR+*R9%)1m|taWb53S zm>1t089Yikf3Th)C$hTm%ld^|J@>lsxk+(JrHT=4Tdh%1wew?!mYHH8{g@u}UxDNgmE{rN4oQArUn>jf!H49i;gqjJ~X)&{C|IA2A*EvpyYrvx)Q&+U#GZargRHj;Jg&agGypY=A+2PxH zqnp~jpwmL};dnJCip=gdX`>IR%K9mVH&w~0zo;F{HM052$1vin@yfN$6SIDqR_!lb zbA7r=e+d2j%zp*guY>aB;CB5=diHOBv`}|tn;V2OG8%S}w%ceU*BWmBY4qZoaNp&x zM&W}&Dc{MZps!6vqWok1V~voU9XD641g(=1vaXOmR#`{Rki;dI5@pVT)oL29Fk8S zaLo4ZJ(0PH6AM2nO`$^91 zmUAB+5`CEkNOv=g5O)eC)mz-dSB{Eo!&ij&jDkv-S=sSWNh#rn7-F+U$_P117B_Ma z0xL13ssVM=2d?%sIGqbT!Jpy1wRjq@*svRe=MMkpbw!>o}c*mma(QmUi`h$floGu+1a*t6HMzP3vl= zm~h4>l219d8&|J>4@T1!i!;#B{q;q8meJ=^DI^;LR_LiMlRJVm*_T}bT4kCc{mSiY zo{C$qI`7HRCLED~%4kqYjXPFPw4i(X(=yR`x?+9vxE(j}?PvLQA{EToiOdkxS^L^^ zg*qpp7W^-|u>HB#(xhojjCS-ld;oy*n_>;?AM3!&qN!I;ER}LPoEEgz2jQCvHnQrO z0@BRzdSS6Zq2Vg3t*X&+fN!GSluTNH&K}9xy-Q_~`?2Vz?)(mT2hn(l>4Ohpqp2!? zyVTTw?(N*YnB;rVL83vOvfEC>FP<;Hu{wp}@PyA2&#!2N6h#0NLD+9IFFXdf_R9L@ zj~om&;=W~oeC1FR(NGd+w5`4W z!_BYsBe(x!DO}JsB}^Q@3zm8fTN%0jb~bAH!sm`n$afyX5ZROT{9Y^F8o1)5H45jZ zYCL>>wKr+b=u728cp?*6v19Y1wzq}5%V!h!215z zBB28HzyI?|4UpVZSn>a!;>HU3_7D{;SVby2e}g5@6av`wD+b+v#`-@O`M>$e{Q=Pa zgd4=;fK<(K;ZY458Agn$4l!t+DfAOHIwnD7G>MEE!s zl!^Xtgzb<3i13fhMTh@~pYVVCnJMWW%IPsc;zRzeTLMtp4x5=A#vsY%-&qP?!D=kIOyl^ zf4l+BV0t_2eF6iz9n+fmFb*s6&2s;8RB(RFrIVh9fzn}`z}ZtM=9Bj|xbxF>{!Q81 zJAvQ81v&%XVrZ3|mpJOQ$w9kQ$4mHv&S{HFih&V(zXqH`Hya|f86oj@tg!K>Cq+- z=Y%vmxoI4gG6>(xq(B(CV-*8+vFZ_ zvSt9Ei&2%Mw#O&G;?jJt&jPcfU9A9$$@%^6L=JA<)$w#y#?_{E=aE~to{97L>vlZh ze&DA-9-l5ADLw#NzMCkEj2J=OecJP(V3qhuFI(2>uf69RWf^uglUBd+N3SW-fv3ea z_r+m>)NhXmPdF>&Yq}qJBJ!1fA8pzDieGdIH)}bW-&~)xSl{ES*3^=m&#mfZPC=v> z)5Q6I18GpAH$YkJVJ<+dF_>WLu?GAq+FWnO%X|bcfJ4IQD?pE&uPYdNf*|_7{L%Ka zTbK_)%(vO=cgAO8Gu zzRs{ESo#}svCMnRw(}rQ65Q?5>N53T7is*AoIDbIHM+#dBL!zHU|s`rr`*5t--KU{6vqP!=_UTV8$5hQc+ zkp*#bzNbC?hKUpC_d%{VgijW1m^}5^#weo7#LniW(at$BXhMlQyF^p%+y9}q|Y4XD+8 z6exG-_znF{!&c4fo2!#SV6zaq7@vO9xB>%k?5i5ET+AmL8Iq6X=$sZ~6l;zxLL}F6 zSWtTUx@N>{>nLnf=6e+Z%vZN6ht8kxdKNWLx=$! z)Cp!kfN&=Fx@j{?0ru%r^&?pRR{Q;+p2`(DAxU5+VugAhEQ_o8DR0=*w*qKF@iM@v zi-QU8cw|TU@HQL)R0kS!ggP!1Q%O6%Hy3d^)c`>khK-H?&c$Ye)!O!lTzI0M)JLM( z=w9E!rsF8)8ybx(@o%knF-79^fxijG0B`l(jSyTz?~(TKO#ok~3=IKhUn5gY4ZRC4 zty^se-oaApk!bmf)w?iUoRoZda3(6LZ#!jc-@bdcVS4hljYqbQ9Bu?GT3!@Cq70a` zd`Cj9y4}R*3OrU`1tPO=U-vLXkunf$B3YN45A(PMt@`P!A3_PC{>{sRWfgrkqKs&W zE7o&W@Xn-|lajUYzBTbVTdFCB*eO{2?nCV!o4f4<_6MpKzR7eP?3lGb&BM(F?#)fs zJ;<_BQB)|MR2 zZtXYx>`_|!Fj)aK_={L8>_hFDqOCDb&paVFH1G&9G8jym2B_>S!tHyVS=h*6luOAexCv_wYojv@-?s0eN7p_qx#@d;9eahcka3=>o90t6 zmSVK*Sr48X6AP}FcbR%wpl3r)D(S|qsPO1$1e+yQ(@B<2GoLWynG@7gcqaM?>=x#- zO4Z;6VsBqa8aBX6yAFT!)uuZ6$2b+8G?oNh(JQzh^_@x$FgvNu+lY5}lWYMVC~vm9 zSry#w>X7ZJ>DXtAa9+I8Vm9&G>r|i&Xy}Yy>ZJM#xvK@s&6M)}xdE4DNTv7F#`VY0 z*7F5UGijyK;bVOedB%1YY3~4%yDt(QlB7dJWvlz61L@{7{qfoPRDQBgY2D*2m*!Y# zJP3%fc%&KNe#7%l834?Ej~YtyO1j#QfsYa^Uk0CLmVCixWqA+nr{y>4Rn=%vNm;5ya9 zHr?m-Ynt>eOI-FZMa@mXOHGpBS3Y#?HbD}k$tQuR?HOa8PAQQOFDvfwD%)|QHr9)w+Ea2Z*y20V!l89HWdMaO&tlU`8UKs50 zkGt0{c$ffzrw6z^-s2XPU_luoSt^C-K1iBaLUkr16?vD$^9mR2s8q0_cX(aN9Rr>| zCb%m#=xt|f3_sM+(S=_-3$3|@G1iOmLts=3B%}?>17@PWb`w` zdp9x(x%5DN9!e6Z7w=arIf{c!-L%by3|Ie4smtAVik-draBkDHX9*K8xEm-cIi&(< zYph3)vzzHjxefGO?_mV>J*pIw1vo5g+CdYjuRAiym zoHB>Q#5rrM(r+uue%Po#)rv`*I(MYnp3cV$>&f;q1w}+0IdYoY@Eo=m0g0jpb7s82 zbEyEI>Z~TkzPVm7gLqDxZ7>p;PtiXIuQc>krm>thbiq9M8ZY(M&P*>$hOu!l4;^bM zMW~Kdk1Y0eT*hUtNP4{su>PsQcF!%TAasJQsfs95b@|;=u}6v`<3bHaV!#qfuT=IJ~QKxNp;H+KNq&iso@3g+tbE48bNJn?uLt3esGj zO@CsSU8szx$0;6&hpoL5H9)Ig;gv|HJ8PJ|dh_mq#-VY58q{D)`1=m%ZqsrVs0V%t z>eA7h8M3|kbbUX*tsHB}KEL#IS-7SC?eeCJ3EUL*f9`n?0Shog z54WdsdM)RK7+jjxv6$4SRw;HhBaMfZIV!A2$?rOUIuR=vP9T!#!qo-aJ+;K#39!M1 zgTB;B-mDrO=zbf8y$%z)t4IlS%&$i2G(rQhLsXDOK%2=>|Cz0CixM1}M!W?ole;FQ znr>s0@&l>C`(CJIjFi~PQpXQd2*c~1+KX;ND0=5>6~jRD3y^}!Zr`1$q_pO?^Llw{mtQ; z<_oM(@eEPMTVZEdb(n~6Wc1&+Lp4=cz1)OMS@~_o3btoSy}im1Qd&H2=IBeJA9qWM z2DVIgJoJYjR&DuAYM5SnD){iO>(sctkSE6JFACE1-L5OmQRUHa#MMBsu+3d8cq&m; z@;uOiPI?tt9jZ`N+IW>!bWs8?SJ(BjjR;(XJ?w0M1=lmpa}pSq-OHh2aDrs@u%?2( z4YY+9C(<>^!;sGl4U&(2FJx%hERx!`0YQ;jvM^;QP?RS(1w%ePR_+RaPut|bfa`4s zjb>nzBRh%>oZS}4prza7Ov{&Zi6U>^L*nIFey{5R5ZOdin5eD+#qh?Nn9o~upR0}M zwX#~H_?7~%%Vcl9Pu(m`4XhreGxzN^CDi^(oKlZlT4T>3%p{n6a-%fXmGTYbyE=zw z$afTSVHN_}_WeCRWdzwRN;eiQSD@2oJo-I)yi77c$x}qvordj1CIxX;v>`Sh1rEYr zA}6!HzI_adDY10-aEOVLo}r=%E72mX&h~SCK2JI%^vzw)?z#AZFWD8vpq)2BuVQge z2R8AANyQOvJX{>BB`Nu%iN^j3=^`oz>pK{zvG{a@I*z1RI7JEURcCKQ-IY~=KU#}- z+g{CW*baqt>M%qc&eN|f@1u!kWNi9b*bvunhIRZ&2o0_r1iz3GmHJW6o6kUSsT!@F zY_7x??k_s2b5(0>ESCFy2(@0%1AAArwo|^he`i6jSeh`j6rQ3hXxaK}dGp0KmQ=;d z`#;zMi6c1f)d!3WqDL9QYi2K(h3!TInDAsr;1T6M2W?FbQIPrM=j~L3m!0D7e=ZjC z+s=W}oN50>Ta&y$Uzi8{WTw3&jxJmP&O3-}!H{Ft#PjN2*-Lm9!X2Vvk-QT{J1*vo zgY&v$#`d{yLPU=kFej)4OwA)B@1yZ>Eop>3f#j*#Txl(kU8}$KgN$FPjc;gwHVauk z+x6MQ=bInI)dX=dh*bay5M3;ZK$r)7pjo<0oc2Mx+je>t`uufO$L{33-<(4!yB(S1Lbv1mRllEdB)?H8o0xTX0$2DYQpJGOfOOXc3WTYIam{3K9Fvicc z#QWtn888~Tggk%!J4kx5E0j4<5K%v_q7x^9$e&x{0tXL>sWn!dJMZiF^Nh^CZO03w z3-hP&wm3-IO^LSW_MGf9;DXCQ+BJ2vs+Stu@x@DilCRjDO-(c>pe)ilC2?&bOiAcT zy)(&Ax}dS0^_J`9{Bn8@!Wc}AP(2RkJda6b4R-75Z|{~6C>IK(JsJFTzwD5M-;&7X zd)qg&%K*AT@)auF2#9OUYJ?FB_g<51VXtTB#;6QYr56`v^Dtg)HjZk1Fl`DjZ8fNYMw5Q zux1{63ddsBZWE#0<;;k4?(Gr9UF->EN&mUu12iJ@22}AqJvOOc$c~BbLiaf2!07QX z;VuJ*gDib68(k^L3@%qKaYrFMI`8_pYAx-(7O0FG8mpkTIeL^VW*D&4mbZx3wfFmz z_i{LrS2&j_zB`a9yeFzgl{Sp%u(@7QKk zq=9g(c5OaLBS}sBX@5mLB_uy-CMl4z5PcG#VDI2++oGH zse){y_m7JS_O=YpAFXOQ{Af?c^d#{1K+mDv`U#9SCo4xrE5p=nA79wr392xiJw!W9-zs2tIWJUIlV=A*I7c#h6e?C$vgj`F zGdxtCR9tCamTn^%yV6rBXNtH^6lb4$>vJ(T8qNQwq@VUP5mT z%g}Fz*sX}b$#U-u+K3dl4^UO|><>?Ve2t?n)mDhj?QH+{kEoejM4SAKm9)&{>zziL z6}ds4{Wt!2LaZ)F2X8Jx4Z@bf8S=Z6m76;N#<2K+$j_a;v2v7FxabH77bzX!-(+9D z=TQt1G`&2?j3L`0=;S_$=!kX%372f(R3oWFIwO*+`5rFklcgX_QZ;7h{rN#APNJnq za`+WwfDkcfWpmK2y6lY9p`4YlrTk{K+UeFQuHE@oKQE`N7P|%FH}X@fc^)y;d+|Mz z<~Y_q$v$=Gz1N795Q7^KsqC<)t{obrrF;$UefFbZ8)jUyCpp^n-jkZ**Z1h9J~sF& z8U5&N3V~sd%SiQ^;ibbfT3R$8OQsx&*NbWTw{oGDd-#)=eh>kkgbo)9*xj6KyJo3v z%Ju~4KJN9I5vwMy+894?Zsg!?I&*Ap$_!c@W+eO#OmOgLo8r0C7fBJc>Wa0CVWHM~ z0?b0bRvCN%4%xf+EseRXlYCXDqC$Imu4oZw z2+Kvh|H7^9XgT+-4~uom7ZUL{V0XTKp18By35QxIpJP*>)_g!@%zm{I)OKEG#LGs# z+|mi>9C4BEq|Lv#?o@|sn6_#Ij54GZ(T1a+y6`-(aeKh8{O(YigTu`GnKPcDsp$6T z(V1d>#C&GhDG_6H^zlsfuS9oUUFE7#-euk{-hfh`oglHzM!I6;FYUB_;Ue#VXC3bY z3E3kFpJ{2L^HNx;@ED(4TVi0cTjL_SM4jJE)__V}nF40g6`6X-k!p^P zvj*9kWlx(;2P(g*u5uxD+ z)iLcZ!82LNnwOKDx2HIZKYx8?l#TczpZbva&Xxl58Psx0qIU1>;%j<EK9ih#j{FePk%Mn;aqDKJc2Yi!Wj_!G4sbX|m}R={_p*dbLG6?0 zk)B!?L;|_(SbU^JiyyR1eB=!pBd?gzX)4B)!=>2K8^!EM@6fOv?3T0-;2J{gG~es> zrYt!aC-}%GoOm3cRbiyhT5Y(Q@a?yD;NqwHqh*B=zKI?7mf)BnNDzv}+@U!5_@W`O zpW%1WPEcU_JoSEG4|ITOEUyW3lS&a_Va@LL0sIP4K(Oau?gx6Q`W4uLgeQ?5FZMgT zZmtg;DUHGXdeZOSBZEygeoJ?Mddz)qVhpT2DY-)(ZDHaSAxha4-MFCrIu#HV*Q=xel=vz->H<~$@^Nk{`11lYrS zJ>49=!A+TV)!^5a=?QhUJ` z+Bq)|+VR%ZU2Xix0P^lrn*$uhl{-=`?~twd7R)%(%f<>tV!DRu)vsC121{o5_3R2< z($59Yu$(q%)tbdW@;d$kUY}`P{a5uGO zm%Q8WPNcZ~JzAl;uDh4jXNF!@k#C+GXU|`YGN^E5po|I&;ZifFxwwX2 zRGW%C{Ra$CTP{Y8d$^*(BeQU=-teyzBNAo|G;lzEMIB7k((mrKAEHi|^_nPv35mExF0}^)w-wzzl??QIL^*Za0 z1;lZe9&rF0K==8~Y>&mEKGyPqX!pg!OC3afuMerF6X!QW%oQQTTiBK!^^^WRk{k#_ zE0BKJWTQ46*KIM2Ml7dw+6aC1_!M>Y67Lust{(Sl+9AcE?e^hPRgja4wCHF8ch@$d5E8h{cEZIpufw!Xr3no7o{x+R$dL3f%QNZ#AdMm^)7Z=}2DI`!G z!U-i$a^L`8g|_So<~ko@Q>6VK}<8#&i~MKt@JQ zXPkC@IK(I>gS`LveMmP;eI;9u@TbG8pK(5j=>xB0{0`#+)F!%lbWf2 zp>OaB`Dj}bjG>~7NrH+Y!Qrj6+c88k`&;M$hAhSV{Wy!dduXxjU8j&(X1p zsUK!6~o4-ArgEx3{{C51z zfFbjq6+HkwFS6h&$bm1iM&?Y~=UW>~vw+MgMCIdfa+@jtTOTf=mr ze7Q3ix^evI$kRvF9(wWH$$dIBV@e$)8?&FVQE?$=|JBDC4ImT!F@d826SwEa0=9dd z;>ap%$mPiDHxy{+k3O#|Bxf@)!8w5|`gj!(=jLJsr-|iH3sF>lB^vPohto534_?a` z9hL5)UUeA!mYp%rBrTj+d9XU$Cdoey>->gVKE>k-A?HG>p=%h7$Z2-vTpp;n9Zxor z0bP&7yzkBMu#Bj}N)J^oPeK-ach8C-sJ!&L1EGaD zL~9dXpJ^dtcUX7D=JA_T3%fL}TdXZw#=O1x zp|lZ})6k~FAkLc^z1~EEveC7f#@1pLo7S-}dzmm(bhti5obX0%{QHym6QmfuPHS|a zY`Rq2`EUpqiIk|FFf3zFnWD0h+*x?XS@tB$#_A0*X+REHvxRxc3H$Zqtp19aEQCJ{ z_VC0m%PHA$fONgd)TmqP$+zW|jd;wQmyoo{@KcX&akQh@phz0d+o?vH>L={g7pk|l zj*O@9SsxQlW$W0Pb{0_wd8ghiJFtJ77GZ*qZokCCS0ua)Ktgz7wi&nX9W-_{b!BI`)wj^o1xd9Y^kC zXk_Z#*zD?0-2aFYlP#sW2m{7Bd$ta=S-zN1rjbRy)zW|E@q^u2P1rkR!WUnxG8os; zE=pet>;)g0hr;Rls4P&$V?DRWRE2?Pk%!qsL%Af1-H*uzz?PRAtmsrEeRw9k@G+yW zCu(H>Y@gv1NW?DPO(|eVchXfdJwLZIw@GLPCv-FTnW?xY&

x6C^NjF2J3m(Je<2+;640kKUYK6*|wB&(X9( zc+|DKqnGYGn-Y=k43a*nxCCS)M}E6Kf&?_e75hTGK?9r&iu~$cxNmBx*g?gr?VgG| z?R6I7N2Xw>;&tY^+lpx&@PUhDA&$?Imy&45EjKQ)k$bvWV6|_5WYMKeWP{1%CSCW4 ztSfF1>V+yHl*X7f8S8gnN4<}Kd^4?W_kBVfj>A+vyjeA9+CRKTW56kR<0W|B^$9*( ziQUPL@pAcEUg)Qpe_M{Ab3uTFMF)8{kq4Ws|0MKObDcf6q-j7>c0#+09=`t~9V>TT ziK9)M_!afuRi3&3H6HvHtRj$=d*jBzp-t`$DBUSY8axZ}6f!Vkf_kyyU+=V_<~wvM zJ@V-)KfV?zO+@YeN6xk^*6d%>kQx5R@bvfbqJ`JPtBwWKx=i572gWrhEN9M|5m|6< zfe{a1?>~}USMlVvH1vR-6V68E6Lffe_K!u(2$|aEOXQ4Bo{s5gbb3czH-nnhv$*JC zrRP9fPpS4iG!*#yWxUsyH-BgVGP$_q%+}Bk7=dqfMLaD?8B`(IbAkJQqIA&)A{UN* zP)i5Xq8w>Ozca4>xU?X4IZJQ4h>1HG7~H674O(v>`~?OCl-Be9+{QD!zV?I(gK(4JrnGM~!oqn#>C9HY+jj3SPxGIhRdepA z!@5s#@vVBym4xTgQG+yo_Mu^*;z`+U2XD7%^;P3u8r-JJ7phXh;^T_<|@zai#{t7P!GBPrsG4j3o zkujKVu`{}jS6rFay&ZZwZE@BWmSa>RugwHWE_XdoyKx|ug113(7r=YRI6%hl`+c_2c#nI@UJ!7Wl0+%w~$f1+I z8dOSEm$i`zSN#@30CnVMHQPw@xXwZ0`*GytvsrcxqruN{b@SI2wxiHT+p zbjV9NZ}p1|Ax4I_1qmfo!E)HDHb&nkx6U;zYb%IBucR1K2-gX2x+7wpi~H+nTi!a6 zDAPoVChYX>xQoOrtDR>mVGM+QMm%rNr=(`;mCLJ&T&}juS3=xYuO2nkkQ_27XNK-4 z1dO|=%#U{|%cC|X;@@4k8RP0foMjryYZqS6>DN_2R(Q28$I1D~gJ<7-OW*T-|LxFG zMxuhZSaV`mpqG86IVuXYD)HEE47hV@sGHf3dI3I)Y~Z;=H@!eW|F~^cXGE16PhS;{m`g=(WIDwhppCA3GZ5GVQ6qY$d_5iNO z-${h6oR@xhZP#(#?Jg&cIu!FED=W{Vr2VwHX4Xfmzea9%y@g_uv7jw0OV?q9l>r@x zuqW!OMc3w_tpEs(_BTb4%0Ezru-k3u79@kX!Pt$uJQ&2t;g(<_w84QFhMj+;yU7S_ z0$g8#wi5eHVZ%;ygXX6LwInxGoCamwfx9mJw20!8%_gc0Aar9D+}PV-v2!0?n*rm%1-)iTh>$wOEgI^p_s)%K0v-^3|eLr3i9n@0hAS-&T(lEmk6oz zxfUpw$Z({a?<28r+nFILz|q~|LiEQI`%mFVsjy$c+Y0q+lhR)ANHe`W$8z-ab_Y^q zW(OwT@u1rnX|f<9Zsq25H~JiJ&)-*XYLj^87Ve80Q4#npcgv5(O4o~kc}2yU5dDnc_k89x{bIGKX@m8>EUf(TPTq z&U-F-rE1t*43Sl6CBn-)f8LK6_pd9X;Qh|NR63>3KcB+;WxE~(=d0gqW6dcbIGsYO zSqRr@;Bq3$61(5-C@>OJwSK?;#KFaHc~nf$`eHWUJPS|W^lf$igLXsW;CHq;A4et6 z)}MmrdwfzPIp7pS-wRtl6?Fg`IY-H%DUsX?^#Wc*FG5p^PdluQ_phUrHE1tHV5i3| z;xB9M{WFQP$X62Sv#cLETYwd|A5EF7&QSgU%;@mX0nW{ssDSqFSA|BIO#FsI%O1ab z2lr73Mv2RPs2Y*%BNbG3t+t!M<(YBWFo$R8?;%`=Nl6MZ?pC|>+?t&>-Cx6d4_nEj zQ~|X<1TI8Wbr_*S;xC5LCvv0E@{dE)5=8N-q#KJ?qYE+a0EB=S>Ozb9V1)Vi;{RevZi5ed`s!s@LX^s&fg<`lzt)N6*Ni&DW*;(F$ZNud2cS8VSK z{j3`dYAs~yll^_B`OoE_%Tz}qJ_AX?Q$<=o9h2Hm%!RBvO}i8vplm*tOk&2zM&$9~ zvxR~gl>X0(tIBU-Wbez-{QEPxa6a~w@}##k5WU^l6AZM*pFHZ+K)|L@AlXIT9fxNx zya=W+u`MT;d;JXIl#L>4PaUQr+y5T^%FPyU(@cxtCSLuzmquA;R*=(i1xnQ4J8h^V!!bViOwO%^Vq)U( zNri>js_ND6G+%_pV8NSJLYp*DaTQQeNA|3_p&z{%riEk;(Z>+POtZ z6;vZr<;Lc)UKjr>k#wRzes#YnMX-%i9wf%A;da`6ZzCwd!mazxd7E;sG3&u&!@-$y zt;q~QW6NUlLs^3+ielqs3EM-mQu7Ug?S^s=(dr>VEuk9NdcA_$sO7V!|y6#Wdq- z8|`49$V+&}-tHy{G#|nrpO7BmZa{hA4proL9YC5hv0$?Jc>TyDV&*i2nBf=eEQTZQ z?&Elh6p!8qvs8Pj%4LZ=c7*P0OtW!Y21rvC%Eor$5kd{Lo*m?hsv^*arEhkmZWB5DLa#kQM|L02lTXp$IdLS*= zVT=c5zJDv>m(D@y_el$g|CW9~4CwdOVsyp+XOYqWzp}`yKp(goAD{F$p7ndIF-D%z zL*rNZS)+f4F_O=8K*fHfE_T?zjidKi0x}@>|C5CK*C@z`E^H9{BHB!>e<$|QQCLOh zCxL$(_^54y1;qY>Qhes$SPy9uRP6g@kNjJ31$=@{DvDiUcuOC~#-C*H@;!`a03{=mJd3 z)Z4r~pJ*{PX7SFg!H@k0EUi-ZtSeX2J^z}dc$HedNYxBZ^Ge47(E=N(ci)JgaK_j* zpDj-U=s4p$Xn>LoRN#S6f8#P?;yO7vvSc34l>mnD_~czE)-TkyT-l+7&M+s5(>H`u z1Aj3rz#bh21W_8@n+^kP1*x&WfqY2e44?v!xs9C1pH`tJWg?x;#QY$`11JweRBf^n z)D;9hf!$5`^&lWG3?<}{-_*l+XRhqMH;gD={ycd8rhX~R5v5jt4W@n24x6l1UJ-vf zkbL_cSZYY3CZIlR)IiOf`v)MD{v0*ww9KuOgR)HcqG^y~s}VJ5wE>_M;bLL-G)zQW zedBsYTscj>2W!mi#SKK1TRv07V>zfgUlmwbF3-7p&kYV4O!s3GEx`oPr+;Z~Eig8! zVPecBY07BM)9MF>sVJNBe}Hht#`@ClwWNQ*!_xjKAyC; zR-|^2UyFEy;-G-ItQaQ?5$yufi$j$^U{s>%F-p%Wg8FWKKTJKhOyh?3;!P!mw&iSaREIPR2tC@dmN;#JxoJs#u`mSTN0F)6;(S+9n6!&HsnFt5#% z@+?|Zjp7)|)u*m%Y z3`J`~EhH?vwrR^QaDqR52R-}%~879i}JMua)^^~&l+#=wFoLv&M?5AtdVc6u ze+60!17rqfAO7Ol-hQmGcSdnrJW!~XAx$WJ!k)Ah;AO@fI`FO+x$QSquFP*|ffZr~ zvEehu5!qPntWCxkR`I6EeXLLf8L^6jF1tLD%2BjzF+Kg1lB`m=QVfp!5Iv%DcE@YQ zyKL?2p!Y!krS)>l@4=mOuWip|slH_XS;uA03FkCNz@W5{^5gNU+{;R}M(5Iz3LN>u zMG{mn`x4D$S{-$g@gK(K=j4|8T@za{I+uOlEZ^&*vKHCkH))bb?^6DQ!sL#@-O`Z3 zH6#kxa6vVGd+_+`7Bvir7?QU?EYtN{%aN%xn=;bMK91AK;EfZmv)(UEv3djMz!=WW zfD#2i3Z4o-Fg^fhmh%tyE7a-zp$2_F=(S-Dpf99ssNmZvvS{ws7Jm)+O}?&2b7jt; zHy=p(<fgx&>VlJm`eR1NV#-^mGm{t4qP$97`XhhB&K4&cSLmj*40Z(6F!|PJakzMxM`;D^4UpZSrb((BAH>)! zo&8GUUAY_(-!V=f#A~Y>7=1AKIIQpBtZ}1iU_((D?aTul>*q4{m7aR<&uog5jVlaI z-qo;W^rAajXx`On?<}4sr=NwK8V+VXI)-t5#R$cM$4N_WIT=E=1~U)JkDCrD z7gUqC2Zh9Oyq6_m7hAR5ND*Y^8FLdwrB`Q~XSjmsS>!3$U8$O&n16W=Xrjfv>d7X8 z$A%4X(d6d)`DO44o}9n-c)~qFH0$Lws=A&0V`5UW2z)7FPFCb!1qh%nDi^bXV{# z!3#Zz3shd=(k0wa`?$|zh4qnCsKU(){7=q>bdB#c4Y?xFm7qxVW~(2ZcV9LCP;r-v zw5+h_qWz*}>WK19%g7?}CjsU1$0{{!4C?^V)e||0@jLZz)^hf{U1yD*Ola@@pZuZ%8INVfVCHAvjvgqw~=~(HB8j{;a&tlWhp1C@vFVQFi zf<*tjXxSZl9u+jm=MuOW$mPPXb41&EqXm(O)f>Yf-JQz$p_$w=zA`tpDkZWyQA$-_tL+P4u;HaSUjwa4`cfxsVriqLe^IBs z|LYGQ!YD75a0=V#pN~5NPeLec`m3sn9KQ2o+k(>E7sdP21Ypc|rvv3)fZCm&bY<#% zg-aVw+&j+iK)#c>L%|-ghG=^qY13SHP$JO3(wA{xU~mdlfRN?z-+;p5J@``QZTI8L zwPH%+hYE%aE4u{xj1{6;0F?D$N@-oF;Q{dv{JF@p59G31)-Nq4Dwt*qW~4IIOR?gD zUa0QB#I$x-CwVYS>7HGr^^^G#nd|=jr6BLPYm?JsOx}R5nU~s;Tg?~SFb{LiW4jgv zT_2=Ejp7Q-lO9Z%Gk<$Be?LGVzd-qQ2c^59-oQlDJ*0NuldSo%?Y!$Mj|dOB5rW&) z6eR~Qw}NHlBtcO>{95Pf<8*G>=lQ%yyGRyX4OL~+F+}ee==7NJC^&&HW3Q;qK4P%-39f@x~*hl}` zlf+<=Mps-axmZ#)@KKN ze)Ihj(mLKVaEcn6I)eO_Jdwva?sqT|9kW~8Pr+sxhxt&Jgzl}zS?x2k{@vIze?I@) z9nZ2$6?1ahjeF1_fJnx{!kT7R~Hh| zoPZCy5?RQ|f*RIr*_Uy`yBN4soOApK?$hmxY8^ZIL7cPE;^(V5)}2k*w#{>zk^rJb zyj&d2vN_D5l1;eGmw`}G{1AGSPzG8*Qpd)W1*4Bli2@w(3;8W$n7bGR?HPd+${J1s z7$i{`G!u5y-h#GinT6)&Irb zTSis+w(H)4h;+xKg(*mPHzE^|4(V?fP3EebzgCwzvDO%UAQSNrGsDlKaIxQjxnpN{*79TX?G+- zlr)@MH*K@$C!kf6P{$m0IE0VO9^$`CQWq~xVQS=V@_vJOjZEaXxp4f= z8<<$-Unoe_iq39wm8W9T<*)FaFVNbdb8mK#b$JV910$>Xo?SsUD&>16>H^W$hNjX; zrwSCJ2kVi2L5G{IXBAXme^JBuItWuG#|G_(pgqw-$6bGn_ix=?TG!xgy%0kRaf(9% z@1SDI9=_VeX8qQPn66EuWk8*#u3p3wmV-~`aip%^ zFmLPe$7~VqZ*!x#1?^5JQqv`wlBf!$7t{lzX2!RFtt^~rS;e~F8tq4SlwcdQt3YdE zYHe31+tGYFq;F;#EqtF#*!e2w*g!QM|K^)_&L>=AFJ&K`6p;Q&oFs4j7I7{u^@s6N z&CWcZFm!o%p}2?AdY;$T26V?gjGA7q@~$XbMh9VW+F<~0jn`Y_=eR@=`V1c7s)q^! z9yP+YC&2;XQ3ZAmZ)@-LW0B*)2=Pd-FPeB6xceb&Z~5nNr(z6g-7pUm%d)P8?68z$ z1`4|7!Gejbc(r*$=dbb~V22TqKXqM` z@3xx!@QyJWMyun!;O85Xr4g6rmg0!D-}Vs=2ylIv@5v)uHR*X99!SaqHE+|oAL>;4 zcfe{Tj@Gs{%aI%*^bwgC?7T+P$_F$+jLp@99I1yD`D4g1vzEdjQ)|mhF2_{~Oli2= zue#;T-m&T!>;5h-TQkU_#orH_ z@(pHH`w97|Y=L;X0i`zl+46(|EbPI%vdOXxE{|4zmSxP$$7RSmauNqCgR1(>!y?#K zlR-AAF%-97bLz~0jWjpyglLBFY@hIr+*AEIJq9ZkUn29tRn0WhzBPX5?!QWShJoVr zX3H%e-ITKc?FN-Z`MkmSO(}w|KVH{NRry+797Ts+$gTk*jt@5(L9CAZ=fGdX?Tt!t zz=85XD~)za(i;O(d5S_Ugcd}|x<%)p!$=u*6h`u>LIp%M#r#G?J}Sphqh?}c!x@yW zxbk-ezfI?RE#1@zAB`DWfujR;djaM~0{T8P9ya-z{0Mp@ep}=d(jDseU;M*r7be97 z^Mu~5JTwUrNo~Cg&25>cs@24sYYoiMV+h};85-~Gh|_++Ie4&F)m)Xvlz7>>C6KI{ z-yIYtv&$9Z@8`9+<{_KKrn3F;yrHuxT)2VK-Qx#cc@ItC%|L&dpGQ?jkk@1#vapTO zlug9J`aB_LnrmYSu}0@M_g+;vxa}mQ+K=;1jcq;QC|}|Psl{`#mA1p3E@}Ef(PGW& zh=N-2hx|V6j+xqjGite2%>#81{RqzAg{P&a? zG2|XN>B8iE&c{kX1>5IK<=D0-+jsk$m5(dmVGM8UYEr^VgO%HsGBUoEeZ3O=Y_}cG zDXI4>%<;QVk!upxE;Z^wl|W^Sy9d)X#YU2egug^3D4HGm{*Y#m8&~>Ke1eT@Wqc@B z!Fi2E-KD^&+>d#1HfORY&D|JxpL&ByILqHTGHQ6r@O@ts649wtR|rAT$N$VKb=en%}gR zBDvX(G()IcMlvv&Rk+bGn#<*@<-+GZAl{KE^d)k)&GLxD2_g=WO^h&tSR7MxR>tA*;klFoo)DyG2?4`$VZuI+oO?0 zJ=&2xNq!l~d{O*Px`*(N=tjJ}9XAL_*v?IPzxs=9)z9NbN*ByXQYvJPHLLcf=CWsC zvW=8YVZ;>&Wl)=WtcD%w?Q7!*K_MLUUV*ThGpoZXoW2=OYz|D0VIK;*hcND*P{(am zD(ADF#^s=dvMm2|&v4E)p~|)5BT-OOEU50JcJSu8k|}==CUEu>_Rsr`i#Tzr0@6K` z@2pX9v&%7N6sgmP@G(nzTWaDY_T_l;zNNo*3lCj;)&g3BqdY^j*G}$i1_8Up-iw#W z*xq~0q}kKPFFyCaWotBDw+gQ)^S5=Bj0xwI*_GB~0b`9<(vaH1Ss6PDvpKdjv*ibK zoOUUD^|=uFz}=(=f#O)pmCx%5F8Y@lBJPt83dhc-Yz9)(N&bAegneFWqa4Csa|&~a zgfFd2U1{Z9vgupT?#o|>jXH%3s%MR`8qAl z$&YzQKNJH(!#S;Z9hQI4!yLwa`hHlCA&udl`CC9dXWYUV-LRp#JI4^iDEUs?EdEx) zdSWWsTRzp?yc9&@8bFedSZWD^wv>JO_w$u!n*_h=Q^Yw6*l1y9yv^Z{c1^6E{OiMG zONRVV%pj$BD}4KGrRFS(IZOC4tq;XupE8l?TytF|i41xQNEV_g=}PE*jc+Wu?v1&_ zyb}?m|9qw>S43L3C@B{`@p>>rGI8etFZ`Ow&z-im;(Kos6O}X?tj5TYPF(xEcnT(R zbASIa4|};(%-2OlVrNHMas7+AOW}O4JZf9xzJ7bR?+1nZ8&`zS)Pn;Bn`bCZY26oX z+Ux!>*bZd;@+gbWZlJLW=e)$rCy3me;Nhi-5Np(VgAYts)kC)Bc|RD^DYwaMP+hH4 zd*q>Tv?V7maTzjsij-(J9G`?QrB>pt!7nJ#hz)*WgQez%vrVoaTW^cj z-)dUuR0%y3UB6JjejJkQnRqS0D#8PPk(f^Jx#K(VmnHA(SoPI^w5ls@LKv1}kq`!dF3tf~1qzmLpuY8l|! zjD1pD^Ev9Je1D+-=Y_Wd&3#u|a0s~&<;}R?9eLJxx}y8fgPI|BG^#p3k8{r@e$w|Q zp363y((jGrjLu`}kM`d31hYw}kn~CuwJ=LZIWcKAD;&q=%E1Twtr1!sY2FdhbM;ZL z>3lNmLoxj&s`r0j_}>;M88Qn~hX@`*cdww6mAo9iRygD<6yH`|5>hlpC=S5xoP(wW>k-`-0h{<9+HqQ+yo$S=*g zua7!;9;;W?S8Goe@g(8s^eg1?pzHkv%~r~ZqfqNV8pH9MSD4$R)&yX(re*3cZw3(2+CSMX`}feo+KwGY)mx(17RV@4Vti zIysRcxHEYn|IVJ)GKoY7l0a{U$!9Yk03kkfuHE7fBb^lDRy$7d1djjmsb_*JYVEsI zEVM;d*JW_Qz|I%2e$^-gG4-}`{wF@Y4QDu~IwFXl&@)?w=j=<$t9)uldBeQ@Qo9S& zz6!LTzKC)HFL5fW7q*0!R%t1!l)+UOopIY1YB}sRjaWX2!8>N;;iE#2e$Zj>!Z0%L zQaM^rrJp=Z`8MJcxBk=|rXQfww$gX$VD_ca*56>Enq|CU|4FN}C%4PMoRZ^0Oe&AV zkjfK_Ni2iEis2Q?D(RhT)|;x}iOjx03iQp9zKpG_<(xhP?@{ex$zT86MI@v1z8L5+ zz)htaU8(`E*WGs1*L6mM+MGaygIU02fBY3@tp&N#qOI|^g48|E91IIcR^b#O8vr?Y(*6=l06EGaaok@8$PB% z_BZFe4H{3?L6S})^Wc;6wEasK%=DL6eblTITNgW(?B&czINjg4GIKS zvK?NuI?&nkGeer4Bjas2By#yCdg85nNHqGY4{V3J~X=ai--C zSWys3Xnreeqg6EfLcn(wvFC0Dj3-M6fnJyge82xp5x-IcFDVcK(LPpZtciy{2yLw;vFAACkl3}mlL4QMPx|$h0r?oblyIGK$U}bON3kgWR?TKOAIppTg z0nRjw6%Mc8sU-OW=^j`XN8To%uaXC$+{cjYlO2$FOitP82r5h(gzm@MBzHdfVb4yo z2*}v^EjA^zL|>0L%qGcEsgRg%Qm&dapG!>&)||oLq2btjwJ07Z&2Huc3;d_!ItAH7 zg{AYG_rrsvY4-`$9|)3Ze=_>qbsBV+fb~L0FRTw6A#!|?NnBFeM%rHv+sCK(Tbd9- zO$D<;{s;B0O706$8;|UWe>B|`4NkYMsH`ptDTw{rq160cFpz&E+q%(;_#FP)oRz&? zo+;^MQGz=$u%39wexk4Vd!K{tzSwBtIPPL2n6Q)Nt;MY-!5r}=G4)zYW?fc^nt#0y zcrl3XnQUK+(uI-@U8r|ls)Le}*HfKaO@dxLLm)8CV?^d`hwwhX;E%tO4=^8$|E9?>0ITmr|jy@dX$mGCCZ!e{v4<>+L{ z`9`5i)KvE%4MvWLSsfrLx)rUNP6OIbFJ8wF1pFyCA31KYWq17=LovV3@b}zs_OSJM zi3xxLE zb55>=tj&S%Y;)AB`rk#*^k#`OhEG1Pt7=i|H^@g`OaT)Lu_O)K^p!a+ zzWa0udW>0{qfRM221UjtEn1HhECw(t{wkT1-Xz+HV^_4_UMl>3-}c~(6whaC>r>#g zB0R77#N4ggZ;7Mqpl~cpLYBgp1j)=|ieyAG?_Dx}zUz~!(clzEGuS91KLcJrQ`V`B zK9ZuOh^Yqd+$wl9@dI;-AL5X zF8omP?p*M$c$sPvoGd_qJz(vCHaiFSC2V``|A)2k4rBindQMNAKkoUa&>GdNtk`d=b zB0L+t038~kikVYR27P})`Sg%bF!IiSntcq@V)y_1Nc!lV!j?Pm9L*xl5|Q?kr=a|4 zmSQMv#5q*_KGRlEZsKo+y;M0b4yqdsj?c20sFQ-E2*a~vm)1WshSCMl%U_OCX>pN* z$Bvj$+dAAEA5y$G{CNZ1%om-uzzr6{(r8f7fRb*>G)fUdpt&*B0whxzeL8k`djblI&57x zIvMvx6x}>Wb+bcb<)#04r7a`3wiR`LK7C?A0*;3XZHPagAMxcx=?I~DP>U)TXC=8s zBHDurlUzyHWN%`&yY zA)#$M7>=4&#j^jjP0_J7ruxbI@C)9`If9*aj8%#TTiOJpkzKkiKUo5by5A4bwYug@ z+y%amuKRy(-};YyD0pRToAQjG>{E34cxfkVyvC<~`~h}GxO>I4m{q)nZ1 z+fb2#Ghuj>M_{@S*&Tdqy!Z~%@tVdhy}(rkmnBiwr&E)us7ljMRLS8|5`U)lh6zA?8jZXkl9X3Z-jR>>XYisfq3#09Rovx1Zq5an&I1?{5AY5a@>vF)4 z@!Db-#BBn=hRb?H{K>FX#e@Mf$^L(ZPArC{$fUE9{?b%71g&;H>4 zxnr8>9K_Tman@{t8{8w=$ zT9<)uP%@WMJ*%->FL@nG6sluT*y6((UV*ZTofq56N&>dVGDF<$*T=sA!zzDTq9|2% zzKBY&-)CjqHUxj8@KJ|X`^(j?wN4GNkFK?sT&HSC$Xf-lT_CS@GdM!n;!<#yv?@|Y z20fBp76ajWbvhbSLd_@!9OXXUyMx-P&STOzYhHs&-55LE_*QzKFrovsyIsp{1GVwm zN+9*7@FGLstNdhkn$|DLVr(5FWS6}M6|VOERvsmcWf=fyxiR{yx&Bk)ims9e9cQnR z6N>sKZnJEYTb6`=bEfydFB1kHhWzTnS4S4E|MPFNN_# z{3(?E^wCmtUO_X;&FeH=q=nJ>snOtBm+bi*RWFtMFiPDa`^Au|Vnr!(VIJ$+ueR~;ukpB88gdUkUUAcL55O~(y z+nRj;A7mR?Ar*rSE5&$KpO)ezLokXPS6Gsd90T!D0t>NhC_*QT4%9;CQ4;JMVAjX6 z%U1PfDswqm3i!%vEa}nf8=4kC=0`zkC@I_j_-mNfyVI>9v(ws*I#;kM|K|(|!2W7m zl%repo|67=AF%B=+k5!fC$Nvo-UgtnR$7=ucUB~AFKAZI`4(x@lnPn{jFG@q1 z-HZt1jrs(tJ5@oC_-CoT{I6SLLjbo2Cg|OrUmO##ODKw z(uo1PI=N-|`@w|%7N-EI7x&wiXE5?}nZj*h+zTaDcd)^5pl+;+Cha&3?fmeL7StTV z_(fVy&)4I|2*kJZ4qske-Q)3=d)1*&5oPl?-61$D&hBLUkrHgg%fuB9f^!L%GSEA+>sp(YQgg(mK|MXAw8!4)Hpd==- z=(|^$wB>2_3d^F&cgT1-5sce6Phq=&mevsWNnnr$yLqz0C>^wxZ3Uh|6nXRuQ0ex& zBy_GyKfqY@JIyByZ~#t-0lmbEv%9s6mP?kzQl=LrywG&W{^+}mqZ<%(QGCp`Uo zk^8r_*<{P_K@iC;-GR3vvGx+{_iV4_&k(i^hO6{3f0iq7-ZY`@a!>b=pO81w7D-FU z=(}Nzpj~t2I?X-F+9E47Xo}F%XS2lk0Sd!8;!?-!^_g0}Mpx+*6!5twFgNFFH3ooJ zihHQ)pELzeTUJ!yMlEIgdGDqr+_%1)iaBEPTh}rW#d5@$+TK}kRLzS+E&{%}4s`(V z{)wtB!CP7;pK{oO`A#9<m4H6Dvw8&Fa$aDk0uT06$}$3hQz)+gyNpv-~~ zwU!@v5M;pDl){SEL5E-R{ZSThB^%#)LKXYA6W0fCQ>4&6UiZ<=O5x254A6Srk?#6) z50!}nPY{FmJ?51ipGa1Hgx1CdlPKVoofnC;cxm_h1swf*+$Vu>TGCkS3Snyso$9dV z@i_ktX!OvLgZz;M*IRzv`#cy$cmg!%u3?`?N@dg-yG!a`lJzYvpEf@3RD*bDTMvj> zHin|28H)JuLa^P7R4y6Eg>I@^PZ!`5LRe2l5eBC_ack3dwH*VFo(3{C`m6z@9d}*< zgMZ4-Kmg>9@Y?tsg;yE)QFgahzLO2Kz$p8>>_%%_f4{Je$WCA!%Uz@CB2W}P?O+@a zA8=CpN|RHxcT2WPs#UL*I6UYorgl6TO^Rz59Pr-^ed@zwNHs$emdH#f6kXndl%M}3 zI=%~qs-qU+#CdA(}cHIMFCbiyaK&^7jl!acly|AGl{ac*jxh) z2hIk)u0ai^u!%)@#CsOWjXv|wqZGDXh%7+#dpr!f3-x?9XjtcR2ymJQ_Bvm5*?6~2 zGSO7%b}%(P-(jhWp(We1(jO^V??)tO`ThP(m9vI5<@eHD^&i0k{m(N{!o(RXm!qsL z)*2%7@OI$>o0l3zq%T~Z@Id2{9Y_#LpUPL0f=A(PLg|}Y_DVv}3phnB=}=9_!7 zxyG`RXA`9H~XBuNhY)0dbt;&M|12 zOrj*pH4CPobg5Nr4c1Y$Mcu@iuMdvjo4iuix~1S*yUjOQ0^v2$Q257V+|$|_9H=+Z z!VkV`YrGKQD7C^`9NpHNy^KJ!gg(|R1F4XF(lFMpOSj?IjOmi-6(PO8PPxH28 zv7t{O%(4Clek+;Z=xue!PaE3Fh zspUuUqgh7Dll6XgrLUmvPy6%RtG)SG{VP1n3}QrzthP?b$JnoW#xBoHNKX#e#8~lf zPMXqo8n;zZo7UdeTL!nE^pgD*A>BUkAn&bO;i$98pIL6vo(R#nIWnPq?J)ZZw2-?v zRt6nZoez4DLcpL|+Ncey)!O=~KmOf&fi5h<8D>648NMaDLMkQm4v&384zQt`-`;Mlo+w^41BSLbu3u0Yp;X(1LUzL} zdo&aV#Ezw6F6XW?`H8G4>7Da6>GN%BwYam2X*(lFBh!-6KC7Adw=uPOUoF&YQScyy zeV6vkM2`BDqg|byaA*g|&RP!wYE%;Y(^AI~0|noUh6%z-Ga&)*G-EISylr_KUmMNu zY!vz`@>PD>_tx{ZXk3k#%h-f&G@acCs3uD<7XGO`l(L^loK-k6sP^mb=3j~Ozou9o zYBFTos==uAr!lQ4qZZIhYWnpfgpfP1mSO;GgzSs!vmc=Q+Y2|#&ebWvH?cd=v)s@leq!~ruPtl)$lLWuUz~q`*kW%o9iRu8GYMswH zcB1S}F;yQ&QOz^{k{7Qv5&7Jrum6&KjyS`6bSTb%_z|FLO`xE=v<+t#@@~PEJlycQ|TD=+a(<2ToGnwfiD<+ zw2+%%K-`2u6}La=Qd;NcY8N){c8yUkIW)|N2UU31-X&RXev&bp40T+s zto5z`983HRPIR_u3*zo2hCblMw4mw0GU{Tzh7kduZ!h;KdD3@JUQN z%X~}DzrK`NlUfgS`6_L8szA%?%M;bMo>&AD&w*W8RIGTmucTTxL9@B&g+T6wD(wDv zj|e}NSUm_XBE!rB2d~n!vT^Z5GNiW<4=B_3RsvsLwB&n&uNY3!`pLG^Q+r!1MCH}xzk=KMWn6XDVLZsyo%NC!MOwSu>IV{_CF?YplLyorf zH5I)V0@FJth2k_N&$6yG4 zG%v{{O@1Zk10Kz|k9zM@GEFpmRDA_%g?=iG39XYP7ke*RRT1Y&V+%Bvk5rJgxYHS(2h;b~`G9 zP^9&aOrc-TxWWp9T!ieUZ~t3RBJzfNMqnW|^2${A{yO5+XrL!sPSaooI9vvc{xPOW z49x!Hc{PJl4Ag%cAsDQDr|!fGtqlZamiH%qQx5h9B9E=c9b5~0AjnP6R&WxxAv$y) zoI)ckre8v(*c90ekK0CR4HC@ETHpHe2&a(u``a@K+VmpUEImli9<+5&g4`}jUsyuh z4kD9cbx~$5SOmYTpkA;1Xm$Xjo zje8lzRVMB5y|pvArI)orGtGArj}^{uXGG1jp_BRLZGso7@9OqNZTYZ}e5YbU`J~Ik zrb6M#G9!;_#!<~b@&a)7*nSZBTq~Cz;?aF}vYeK$&15_97|mf%B+Mzv0^VbwlxZ6y zG?TWeyu{3XYj%tYt!_C_fU#9;zq$GFTVQP%ulyMli>l_i4u)_5C0@_zQjIVk)S%Vk zj-VpiZuImCRm>A9K}RRNPW=A+#vG&Vnfp4oQ=gBbAfxlt4W5M<*B&s{%}7hLNy1?6 z9ewh&2Q^Z&d&_3H4Q#zh3l6kw+S-~@%0$8H!N436%-%X1PC4+ z>IxM^AG%D_pOJ)rphLSHA0F(Yu4H2zVN=uxIhBU*J=6{igLfjZ;O}Igr3)iicWSt> z46=!w%<<$=%7aNg5ee4B8t~>tit6@}_>j4=zSMlE1WM5qgy^hfF3IFkZ*+u@PvI5( z^0J}@KY>7&GscXsm-+1KV=MX`P3_}*B4>;cG=oL-qrGDfKAVXnzAr4}VfA5> z#vKKbzO!V@j4!diBZD{7G#6j`&ASIwld(ggJ8>vt{63|75B{tJp7oF7!1~e`pJBG1 zO3#t)<;iU`IS=*fX|&&J&1X#;oDAEAcd$*u?)1{K26)eKf2g!kPq}S5%_O$i-9=SO zdX-)e4y7HozNqP?=pClM^`AwaS~{9z^eKS3fkrhJTO~+{@&#HW%;b|5(c;4 zdQDa4Fbl}pUcNW~&-8=Vg%(B*A(y+G$!yv;;)bR`zk-49W>d)wlc>S`76JnHUeO0K z{C&wFkCnZHa_c>Zs^a-5HH|)F0CfdnRd;JAjL@U>jg`PBdV@Mk)ZR5`ShG&`vCLov%Cx;gmrtT(gJ8+Dtz`b#W3R;ieVN8#G$O4e~ zi$R**t}TnuAI-Tlp0)nzLlV4~(t;R%v8vf#ziRr!AwRF%fLjLt43*q~yegxk<1Zcr zUcYEq4}1Z#z^cd>#i$mO*4rz8+m8%}Ylbkst)qm#Pe(rj4S06=E+S8kD&(w@r+P#p z*0&GQ2Unotw!$A{?@N$OiDHw&A#SEe-_S(7!vBXOmWB8sFxXCAaZ|YV`RR7iM3DO&QcTi2epBRv+Wbh++%jpJjNtZPcf(U_&Z!e z6&=y6+DwiCDe4%|wRIiICqFYUW&rC z2QH+Hm5kv}i6Z&(c220d>MVoC{*;of4e?q^8JhwcCRXNUH4g~8BH+M`Wfc6#_c(9* z6jNf9PyAk(WMA8~uAqqsp1huDV%U~y^tq#?=J{$7Esj%he%4`^pc?C2a0GzUY`jah z_$Uqn3vr9QA&u?*G_|73EUtMEdkvhj`52^6Sd(&jaa0R{pX#$~NXl2;T)Un%BiFpH z^A2i7z9XLwnilYDy08=v44UAlorT!INP3f zVfHl^DO^{Dr?uWPr_R=w8*boVUbBWN_&3(-)^&~9vuqbPwuflGKP`rhCxnHYi0+K{oM%BCA5C z_oKe9$(jk`yc`6l;(1$15_Y3L(FMfLxUCOLk}I%)d!+B>p>9GmETC6$inr4)`;<$ z&ZCDc6%9j5oDZ}3rC+zk_F|u(KD=SvRb4s#5NJ0j2kU3D%@UHOC#ek`xCL0%3 zWIZg2)IROAIvo(jH$3%EK2Y@u&*vs<&mpC^ zt?=_i$xYwq-!TNvOVRh}QRLFAC*^pC%mr5Wp<_U2uJ>!!XW&x81*y_R#V&ywU8taq8<$U=zYAhR@Aztv)7$$+pRZcqRZ<*i>*`R3JeT5T`9ESdz&nU;b(8CdRra4=0P1ZT zh(P|Z0;wQm4~G{B%;qHUjQJ08Rmg^_`3sj%?jL!g_9{#76;#KhtHT;fs4lu1T98Fp zO@kECkw4nki;1NX9Oj;_LvV(3et4R%K=;(=UffGAPS?DD7OVh!=@+a%yz$diE!R>g zlw9#LOyf{TcFnod>v4uMOYWV`_vyJ6-U>1OSbUP-DT)^)E$lSGq)OvjSo;OxbxwOj5$r~Unk9%Cr0VFv4rhkf zGYqIczHHmwFX9IYy|{U6_|bW)1X~AZ#GjsIkTJiF%RB{uRk&iuSnr_V9glM@52$yB z^_|16$YG$u2aml3-bE_eC45d3x2dp?;^BgEoOD9*l2Ggh3w?-#c-zE||Fvy+2K zQ?f9$R-Jh4&@Bq2n$kP(9f!BioTfj=+GLC3n3?8N-h3t@6AOU&8_+;3b*2eDI++@O z)|x58FO+V?>rc?`*nX&H-BO#tcbmCO{hs9lqyFb^ z0zu3AZYv;h%uqc1S*2GK7*R@ zNJnbvi3KA*`56X4+Ey^z_$R=wgsCxu}c$u0tG>I{Ej7pJ??jXp(2{V zT3BC+IHBJ`4aKqq1D+wTmB(UN7%r}f8e&e)aJXeVGFLls1+men6+(;Ko7~CD&zlwM zlzM&%%0HXkv77jBq#F-tgn{>kY51f~;)k+2b4m@kzu;~>`tSG^v2kcX zI{|4s^F9`+i}+vhJu?&F&nhh@UN{qWy3>9mQdLXMR7mN>(UaV!y42(Q8Y>o~Uh7&F zowk@RI4Ambe(q$B`vA%^XEck*Y5iqP%r9$(LW+I;uND~Bg0ZEpUmx4}%E1PhDteni z>!0E(;DZ-&{O!9A2)1sDqcuF0Ug`mu3-l;W8N&nZd=FhlzLXRp8Qb>7OI|eoPG7wM z>x5=1e=i!oEaJRtk@56pY~kEvOs&H(9>fa36{eIs#FssYhWC;Q7^VpNdJw@+E91H{ z9(hpz2A*PF=k)-0qUZ!?l8bOIGeQWYCC}l3nXpj@O|525;WgsATZ!bgdW#6cXVu46 z2I>sKUh5?UKHFNiCcl8S0jKDDCJnK@=RPdpGK*)Q3P@?u@Kh ziDt4p4)NUTmRuIAAxaF8! zXU25m2Zc3reqjuMGloBp-vr< zd*@#2>D$t+I@h=^tt^4wrlufnp^98nM4n`a*WEd9@N$F}NLnR-d^h+U%GXxe(9k7Z z*c<7Z+D1y^9(yt>x*93*yVLv6s8?IkQb{2T^kVxH)3aES-g}qtAO6UeyqMwTuLfD4 z?p6I_DX)u1Aa~xh>&nd{b53OGh=PepOE{)rv~*Hc%&0T;Y;g}Vz>ERXh^-S|(Rdl( z_+?(MrpH;?64F|kuqhRf7LwxFlm0JpjSMVtRV&0Qlh90m0OKFs0skCngkD%vNn`Dob9T#Bp#yV)t(7t0{yl7Xij!2jj~>9FV0w zd>u&T%O^j?+tdcJbkr?3I_GSm8E^9xZP^whN~ zpP*K&`x@4A48O)R-$jQvLddzsRNPMAv|L`V2;HPDM+!Ywo9OfU2#_S^LM1UpY895a zJGOMtmlaus@C6VAQU`kMpLiE^Ty-c`KUk4am-}bNJH2emllq!GRgcTY;jAmRb)#!| zom&40NPn4Zs?QP22(!Wp7MHo=DtRjyL2P+Nc-MK)f)N`KVPj9c=wz{m~=)-;$6BNOT8--24e&?nsc-^dXlROb()i)UR3 z6BQ6JH(-}=YVPV`M@lA&(1n)IQ7mXf;h3-aHNo zE0&6RC9cmW#}>6^_-pwBWMeHE(%4nEo<>=wx#KC~BI0qu_J|Sr8@Ico$#w?>1%Q@; zrJuFtIpSm&<<8;;L^)KQX=_82V1Wv?axj5J>#egrENDua^m;6H*u{Ns?k_E-JU#yut zTb58youbpv+IrXRnUhg%AZg*na1aS@h;Xym^S;qN11YpN=9vU!_|3E_a%3l|x9w}y zPPQ)srA5aQWSfE!6G|1*{Lhum5-6<{7$#xhb~=04>9-0yOjTF_s{0Wb&wGa99qd`2mz?{ zO3pxHEY%TP4UH(4;Kwxp&`i92FKY1!CCSKo<`E${Dcj!b6CV+@^%GX1{jEKe&_Q{I zOHG25w+VN2sJL}yA`63< zmLBsMH@B;ywnsE&V_1Bdx05+yYKw1x;bgDF1Ni40u7e^6j8(*N1`d3~O-pn*41YP71TS zXxO#Nf30?4kS5N8Uie_VUX@0{1PMRI7~dRkR1$mi=^J^*GI=)kr7{nYNrmj%-MDA| zitCNHB`!<|FCvs>b+K*X=U)J#hI;0XNNyaTnP+m0pOBL(pYY3$0=(sT|J=b}OVKl8 z+Z{>4feV?ZEeH8&CGr5UwNxc~-TOVDR@;X(F{^6_*;D9N4P;fddIWqZ#!O6lcNqly zt58YztUuaLaA8l5G3f(UuSidjDn*@nJ|Qo?=`TY31-DPMkJ1ppdC&*sJd@FnY#b%m zIL3PwNOqQ}c@Ocl`>z{E5Wnvu%zcYX#r2dXe#by9eO&Q>u=kfyQMPaVF0Oz`N=i3U z0+NE#jnaseNVkB1N(loZCEeYf(j^St1JaFvfOHHD-T!Nz@7})`|2O(-zu0T9#ahpL zT(f3wt~u|UE6(F{e0~aXnoSES$sKGxlNPQny|SY?DKE>uGWo;ehLbB8R3_;g>Bu^w z25V1qQ6-1p$JCw@U2t2$bioXsyQSm>|e-?kW^4drm86aQu#7;NN|em zA@#qgE69XF9d)On{fo)qn?wk>3ZBQ{!TtANklPv5QK%>;@jp~PzBd`T3X|8L{zW`u zv;*ph!kzG66bW=FJm4z23O4>l1JOE-M7T~Ksr^IcOQ4y6s|Zpb`6r`E1rey`otGK^ zQ2AdlI>9~lFgwfi??X(7VhXGdlhe@>e}A}u`{Ek`cr44w_Wb?6mO$%P0VPzy;{7id z@&8xIN}}FK8NduOrXC4eYTQ2g?A?ZOwWZwxLnd_Z5QMwRfD^4I+wx>5CwsI8PoG=R z|6bX+c)i!uW$B2uk$!t+=^XC<%hd7nYM|Rx>CXB4|KJg#LzSQxf+&ys8xE3>8XI%q z0|QFkf-U2(6qPS*p2Qt&Cy9X;WiP&)nC7ua$m?4bc_Fqq2JS_SE3mQ|^aRl?7TN}G z=FVfvg699l-~)i-)90T^WON2eD(u{xws_To^L=M5o6+}s^AX3OrI_)uTd+A&9}p)q zAiF>58oXwR2KWUmL8!fFEiyxz(zd)Uhlfnp1G$yGgFFR`DMGU46l|9NoyA?zgRM=- z2|!(~%nl}emeJJIpx`sviT|*Yzv!2w$d{*ud~ybTQ)c`jAVz-97VMmdh+74PBj&vj z(diruM&Kl+FYUO*a-ta*w+HpXvy6=aBoQ%WaJ{8HwUCW1(@I2#2ih#%X8KstQ?8MX zB1p8DI3EzT9XmYXZUAf-1x=Yxn)SWXPXeTW0~=)D1HcyS0BS|K7IZ)lEvw9&1H6FO z{nrXjyy#ITUaC`R_%!q=my2_j&ev(HFrQ>-lku&ey6SIaytOk(UNh%Ud+Gj%P1Q}| z1SIn{7aRh7-w!6y488-)MSw_DfnZzKOF*hbeFstJGf1eOAtWC9hv{WUbxp-%)FIVU z&~rG0T++DA^bU=Aih&b`U>BrbGbrr-uI22{7~^orj;YNG8eA153eF&N@MFA?bbHXr zDU#oCoANI78|k=*lnP+`JX3fPvRvk|${wBRZ0-f}pe?~C=2s#AWd^L3wM8tE)&}sA z#kK%r=2c+s3qR?kbIY>RwXi=*g?q4EA^{Y2nkytjgtRCwyXSFa=!a&9c?r);Q6Fw=^ zUN9nC!Ycu0bts!U$fpG~!hqZt=tuI{JD~aucPfM?qwbimGR41*()T=H^?N`l`t|tI z$Y6cA;G&MnbzVRV#b;dGG_-)mK=IBsB6;y@4QD>EFPrALAWPM#WzKdJPRI<|N5WEe zUlXR?KR-1yexCCoZzMGe2pt67cWn%lg+$B3X+I9|_+_QVo86KRxytQcCh=M6AJ#PTWCS3-5k4i%_OKeSLZR`Pz?Q({ej1$ZPu z_n$Tf8r;*MjTgIP7kW2~lw9a;%z7DE)+pXH+VVy=^E)HmvdFnHz&zlAcT&QE`)hVs z%fM|lt7FjU>ux!U%>De-*DT80ISmdMBDWwLvVyzRxL&bOZ}DQ%7HAw^{ztBWAqM1%Hgx&n z40sB7vb99}?g@Ek6ZnrnPG<5&UI-6SeS-Piwv5G!+DSv7*n>_o?u}(Zx44t#5B;2^ zqGdiEEZkOU8?WMc8Z43ENk~DVV8^s~_O}Z^;I5^nHNSWTZb3;*a-xf?4a~H&PL^(! zZ5?bBa64K$4@=(vd6D-@sIYWXn${JdRqRv>ChANOb=Ktqx=pz|_vvOskboU8-30_B zcf6erNASw2KcM#C5Hz8cD^UwWY+H4`UGcwY5?ytD6JSWpo(0`VvA#`P+f z7aLTmgvkGtm8RY`Qk@YBXpu=@i}MD5Kj8{1a$5K*ADsA9i}3Gqpy;B_$?T!qhh z>E|FTUEdid2yT<300$B2cElAxoX4>;WY$bk=_)R?1b_tFgCf_~3ARGePVN$yhT@9I z{uTF4f(Rw#6OsWjAXE8~roy0LhoA}-h0c-ZnmY}10QoC?S6ilezdMnOheuR%K`5wVQzTnt4v3v62W7#=_;d1rZJG;a$Hk^>~8(( zf_uJ7_|+Nmu5Tyro*g1PC+oWct?wCD;-=qfDuW|oYvTStym%omzVyr%Ng9DJz;bd;{MnD({R{A4LS*^&gA8*PT#ca9*1&!r1 zv+0n#8>qwa+*S7(&!9a=jAyp!KL0ScTG4j)2XBgu`K%Kj&M zA@vq9BvQ%o0t9+5-BC7rfXd~<8F`}zkAyrEeC3~1;fiF}V1o_~1`{p2Guwj)_+-`1 z6sjD@K#4!WF24PADx};tpoF(7lgX~0!!$;Fer9E|> zl%b{vmZ^Fvo_}C|Q5vNQ1s=!04y&L7w5wWf-&MlGT9J+KdBLma9-?ux7=IK36In!- zFst@{#=6yY*n7HgS1K>YaEDX2H%Tam7eC>JsKVM=Qu>NV1NAcVP3z1AJi+jPigum*imn!!G_ z9`?LyaD8k)=Gav1dR=MnjBnec^lg&c8eh8Z!tMX;3!XZV6!w6%l;l6Cz(o+^E@twM z`AQ2>_|4MD1TL>j0{cH21eK1_*+t*Tp)~GosOK+PI+rZAL;8{e)e!xn!3~(0pjNDP ze7FUao%fu^jb5-be*I8oujKH8#{*O+A#M@E&_R%X??cc_{aA6qLmNB_^v;D{a1w}grjGhTEzHlV1KZ)J z$So|lw3D(NQSVp$F5VtWJ`u8$sj*B3b zdw0hOG%SwU&PQcrLDRiS^GkdG5)W7u3SsJQ3=~kKPuQcoY8bCU{`_~dR?pak<*)u~ zR%F$eYG`W<%zOae;>v>QH9o1yjUH(F;o^ZHFLa-yS{GG=y#-yXeCOljp+3Lg?DaJf z$P=p4^2A7-_j*U!4@CGO3RRi`2N!Q`pE@oit+nUGBb}Yl{YK2Ad5-|-COWLvF1E|K zM&qs}bGa7sT+Q^DSZP9EHjBw1Mr%blBq(b3AEl^cERrs=xQQL(_AIx#D2sf=yPn?6 zq-JbB-^fe+W#ecyv?2W4<06(9uBPQrO@1pfY6Lt-oMrYVUzj=1ZVn6OJk(aj6xc`O zKhfkU*+dJ@#-*E^$wOB{UqkH`Jd$3ox{Z?LlWl!XsNj9*IpN4oRFp>9*aBP(Jg^4A zlLru*%Vp9Q9Z&6Zey+X@>x?m-A<Re0b0uaI=bY@`zx3Y&R%&H9PPEb3|PwkE{w z#y4&J;YFU2Y}J2$X{38PX-~VLXk7f-5Z=)lh?sZ^L5%2_pH~MdN$mK-P({{hl5=m!qvJ*jVNag9Hb>x^#b30vp5q^#9Z@bSu zZdRl6HAg|*ie2WX)Q{4hzh${Tt2>H@atQTTT$0}-(dd4YQY<&ALxp&%z9y39Zz@*Z z?BUWF_iPorVd%%ggNzbHHfMc!J@>o&e(tQu>3QFyhwq#5t=vLsH@gF@4nVy4a&Jf= zl_09(K!j|{7DYBI9f!vpI6VM=5GGSj*U~(MauSQ|S<fzWqb(nX$%lgIh>eOr0V;wRviZFj!8A{=&|=U;-&1ug$W zcFC+ht~*O+S^MYm_49Pqd@bLXGgQ$Y*U7r8!b=C=BUwz$Bm72y>ETqssZ7*mv~KMf z3$CN9NqXynm21-Z5GJ}+l<-G`Gv*xao6u^A%^ur8OC#ga`X9uGbVsXz8gGEz}{m4Sok9*XXTCE;CLlXj~`rX z54*;tNi1v?6Ec^GSF7}sc?sR%lOD!r2q&vQvrV49U#NpYivT0minaooq|wr_Tg9~w z6&u-(hpi%`um!(qXIyAV$b3MX3G&FR-Cv*8&NaZ;w}mI+&0CW+wYTYIW!eT+VA|v> zIZ(Z{G)B0R-bG)F8+%V5fggD~_Np`O{-wib9}nC_avZvf?On1(+iA73;*@WAi^9gf z-A@#eN%FmE9kmLX3JK8+3#?tj{12b;20iBR`u^58mJJdb4ALnhYn;V?El(3DBtu7P zuW-^u{{vW>>0Lm8z);XTqR9pNw)Zz+CHHF0TDUuo;M2K~q8t{9q|g<8i7 zkBPj!tlQ}%JijDCR&(A3h&arw}BHN9Y z(5zo8m>%_26~QW*PcIR4Pu~>-H;09V1>sn{0EW5HwdhMhl%oTvWT3P6L_E*Dbaa01 zXuwIXU+UH&p_o&>K8y4xqU8e{2M@lmXX8Dc+xP|YO$9f{{MF*PX#VPk${p@QQ_n8E z*|p-%zr!`M*)g3oZr+^y5KC!e&;7rc6terC`JW~r`70oCUI(wK?snP0I$Nhg#?r1r ztyc%&G9A98I&V;exXIeQm#N|E;un5lf>WP)%#%cu-G7A3QOi+i$T+5{vl;B93HWnp zn6J}mO^Pynw1W`+e6gf z=1m_lR;tR#D({b>yUU_^&&zaBkr90Y!0cRefGh*_wc;aSFS;O zFzZ_nJ2BU>{i@C5guDG|iXzbJ)!mL4yOgw8iwXH((xZY-k4EF<6XKv;Lv8zQw2auH z_YT@4%to2;+=jTCC5d;Se2h3(LSL7JjCyFjJjiJuQz58eQyvxPx*AJ<)Y-3B&EV~m z%}7zqGkli3z2)okWOQS+%r4pvS#8Yz7tPOMG_DCB@8F^>Mmz%&xFELWjf6_E9P#y7 znBAd&E-ief`f3N3S$2uUUJB3+nP1W9T(m#TMNMHNEoj-HS`^d}{```>@=4RtC_*eG zJ%@df)jUJ{0gYoPfmnSOPVG7_MLf45?yJ+mpibW#qYHgg>C~4^L=-*GTa4PielR=C zD3hXaGxf6awID_vqT`AXx02CInSMo6C1+57GMRBe)_q*v_N}ff{lm|^!5o_M6V!EG zJ{IDSVGJRUE7a?KiHd`Z_w-YSB3+C~C?h9go6;Gc@T+}ZYYqt^-@C4vr4{oo5>Y=r zYN#yP@|-1jh8;zFo;?_Kq-ykbvp-o!U+zLHbaw0ZJ83w6BSmG=sO3nO>}nl3g|djg zIN`UG41goRUQvz=o=VXU;9PFBUVR$hcsE-NAiH>75q^%o#=oxIvavDE zvfo^M_O{09p)69{@s(nk;C-AphJ9!;`yA(C=5IH%^^WdH?L5C=Z=wfSdX0idwiPQr z;0v*$jN!@LFJ*rq+ts;+=C#S~u_DLQ;(27wLS|dOe2MKb0AZq@gI;%RyMRHO@t-l+Irar~JA~lD zmEi=hA1m)Wy#8j|(i{v)2IdoIwFCctKWx&$AM?UkIuJ^+_{AJQIN1QEmC}7Ob0SL2 zy~QPfaHeV6>%88*%-ky@Y$^CbzE)g1_2ZUNE%(s|Y{835Q{%(uvY+LK8J{RbkrhOm z+)z2N7`_zbOml5rR*>;FLuCJf9SDYTVdO44!Zi&fT$sJIh1?22wOZx~HcWbp6w{I0 zj^^n(?|eCV>YTB{@}f@AZa|REU^!ZVJ-gp!h@2%!nA+=Gf0-k93d~Q1WqAjXb(@s$ zD&M|G%09QeKukwTdq$wyNprF38|y({=*^deYZzWi^O$8qvDpbcO?xGlo+Kt#91H)J z%7z7Nmj{IE9E#BM<#$5kq`T7C#?I7CQfY~U;@)Qx4u#>p>uoQd-jjXa8!#P^1J!6G zqlX?at{vMRJiyg;s0s^6_Z%oarI7?PZ~f5-Ka?q*E@CI&bvNFgT1>{LI|~Wz@!LlB zevjrPA9XdpBoiB?Pe-QZm_s`Vp6a`vRRi)QxQdp1b82-nD&XpkxaM}BE>+U|CYGjK zXN}lsyt+Von>YRDazAyNRhv-FcfnR5(KYs1L21a*Ta8OL)qVlD4liLyTzX;sDFg~~ z8)sV#XAX`Od1z{$zM*1v+fKFL zaqn~p(F&Y)IgXF&2KO-kla7Y4 zhT(PH6)QbxprZgonXkz%-a>@}bV=}2R$+s^qqz2e^#J1<{h`3!{KlZ*U4XV<=ErM- z0vM5|s`ab5^GnFQ>E=+J7#1*iEa8)BUR+GC{Ia-q89f`|8#>CbStRnV1$OAomCH+> z`wF*MlTT#I7_C*md@`PO9@cCCgd+C5hy%w82yt2ntiAP19T8&&nX~oKCD@s=iYJ+xK8VT6>!Z+J`c{p&d#+;D zyPR^0j@9(8<15oshW?)5O(Ik4ejaymc-LxvXAs*t#|@dpj}}jPHb~XlM z^%aK9e=d-{v2B*qwO~yLglK@H~UrWsoL-(2@U-tNSGH`je4d(`!EdU zGAgE;j76T=*nt%!hha^jJLiLF=&^^9f+!ImvD?YgQBmQksgQ9$`ycxB?i(7z``sb4 zG0C-Z#7j?~np{}o*racwLdaD;+ctepNW#b_nIA2L%;NUj9EM@%XFXVP2m7G09NRIr zOHILt_Arj=o)nXY%F{79|HE1LwQBEOVXMZ{U_8~!d5qVqfv@uf?`iHS$Ne zaQacO{C8yWdY&tkrij~@-dE5PCYvU6^xkqWGS;Hu2$oHDvB(h4d zcZC&vBPc{DuKey9YIF^!lzt{WT0>6Yzxwls&-K7F$tq0KGdglA!aq7s@+ZI znsIeIZQgXP7f-(~hE4(Lsg5hxN}H{hR-W&oVl;0QHyf&2O7fC;oC*~vlD+?JGKkp! zTE^97lcTak+DI4{79?a76A9DrL)+NRdl^!YG7{WeegzXl`wx)+bO~DW{!Uy&>7NcX zC|aW$9x*MTl4L_-jp%CaW9@7k^c&L&l$BbvVzsZ{7-V*Y`tfaXuKpmGHd>#3im8K3i*ap}MK zFqefRINC!{7H+I?mR`nC`27x*=lGmNXy1#lAw*J1!9wwoWid5koSdSGC&M}a^~wkL z@KaPmHSSy$b%tj>y4GlC)iNAfxi=XwU8g8)LfBciT}1pQ=CZp(Sp49@g2bx4NC13& zE*`{Ss}Pmj^S%D+D6f6ee5LT*>3~W+No6}((hT}4mZd!bH=Dy$h|@;H6eIpf5o|y! z-cHTZRntN!TNdkHS>cn7M?8ITF|1WrU9uH@c4H5lc~z*X+*%iCGFX1I2Yl-lN~9J& z-0IghkE&BGkVj8rS80c$dy_q2oxCyUC-kXQzNOWcDi1H`hP*T4-k9Xk|GxGX0~}q~ ziC1ig+EaI`kfcjHcZLA&nCeqmpMbEyr(e|W_@h_~FLo!_2lM_+qYNfJIQnQY*%eyI ziwt)**`mUZ|9+ZRcdz>S;P|`D59lkg^T|M;w=be7z7<+1rj$e`UN0cP5WKQr8fP6$ zW^CkmG&YxuqkEI;0mW!kfvOYxI4Ye9d|&Bz*~sOzrU>$|!_4D=}L69Ad`zAMNkzB}t^-$dWPKF553u$j(n zmTosj3tqk3Y#l8m9jD(DHzE>~>1h^no7d2t`?9$*tQS;(k(Cp+wQ(2~70}GJX32cR z+_dVpXfICA3gE84&sY^(T;cHC!dl(@N_Ad1WCgjROZ3{#6F$nGDG*e;iXp79ez+@y z;1`ibZBCsMP-$)c3E(~UOW9zO*iG7x8x>?uY>J#Xc9)E5u`E{_voYqA>7|rUtp`7P zjQLEKiG|IkD;gziZl|wZO0pKVH zy&wd6K{~&~Jz52%3bEyyy_&(%pS#4=nD|zY@QI~DM!Cb3B8iAJWU7}eu~Gyyu#=)5 zJC&V2E(HU8idf*bcw{<@vl8(RNM>GjO|8!Rg4j3hr*uwoyiqlL<~Ll zYO0$%-)_L^vZVqeQQ3=A4eX24lkL12h^Ag1XN5~U>%(oF%<4KDD1E=Ui`s&wBi0a4 zds|vKqDLne05=B|G4j#XGHpf&2mU2b+*m!<_v?_9%>t#CMi7ViYKFGXxZCu{@0uaT zT>^lu+3!j)+w2w8W(mQm+=euKG>qmQg&DzQLKHIp$J4+=|Kn+B*~Lw#Lbr8DSJ@Rr z6j`QoI%6(#0^%icbe(C<>jk$o(CPJ8Mj!oO%%v7dq=jZTahXSGD2FkH=!&e)5^;Z= z*3#seCb_jkLb-@6wJ7E?uDes(%vwyV@b5FT<$P{tYLn%DESTrZk0^0zVWB?7=S=(DD&0hpz+F zM)onCd%4k8Zn5r=6!0K%^#tOu=i)-GAzqp79Ok{to_u?)&ueKg(0PQ)a&psazkFJU zaN0*0{NW7gVV(SGWxS5Ln1gMPoxM3Jv=BG+ZOCmsH6Gf;8P05-2zJ5xq*;d!Tr)H% zwo_c;c5-fX3w-wA6!W7Fw^aXVD5tgEa=@7ZINW!x|8-#vZ4dg1kg>YMYM&NGjVRN+ z**C@-i%9{=mQ0EZb2ty3A)ty7+(S6qLhN3gO&+3C%=Xpb>7vhUJl7Q=d&MdGn_N$O zjH`tDaUuX4U@x21^B~8IQ3sz-8yEH02n?s30)yx0^7d+6l`kfR@2E|>I+M5n>6k4Z zh{aP~(9zHo)Jdq#6|08tfmrry!;?YOt+1@C4piSx2`LO;e8QwQ34F;Qe4MYH=-ue( z_g)9x8~KujPS7SEK}IPjW9w@gg8L8+7Yj>A?Ji}(3-w_P6beO|FkAxcTgckJ!J_a? zlig`8{PO4XMHl$q;NYNGqYG@V+j&atN1Rt}b(c>>LGeU#g!hrYTi+=WR8{PN=2GnG zQ{ODrL!JrqyL+2YZ{ORF3Hb9B8`;wrxZxotUmRwBPmpIjPM64b`8peG-49j2v!LtN zM>&vV+6cXxoucu6FJ}1e>quG6rR=r2`%|qwElk%R(hUnavFdB*C2wkqPHYcxp6YVV zr9l`wD2s!|4l!NZI1hPniVg`)HKKc3sl!pz$8SGVT!@9BB^_&bxTH zfBnvg?|U@uQqIjmq2tJf)9T?FBe!oZPErSjo=}Zq+LIe6krGy{J@`skf=m9)m5F5P zB#5KMA9pDAF4el@VoHBFY-^h{{_5s%NL;)dE_2`j6d_`^xV5S44|TbP5WS2x=7hf^7zp~Sj6STk;{zse`Ow=hUwp&i{UXc!Z>Jf)@RhgGPa zB|jBeggow;iyzx@Z+Jm>+*>74`$1?`T`sb#Mi>&H_Jq=2Vd^g#TjbE-%G=5D@{ewUzx|E{*0W?POg-|b=U%9rwb8H2c=-%ds_;eRY7c`)K3Dd8*6V|-YD-I-b6x%Z@!Z%`*zC#px6V>= zoOFZVVZ7;TZYAE`?mN;OZDdS2s()6fR#Xi`<3EoWHlz*fyUc3}$Mf#LUp^hGlEvKe zdw3N6-IxCOczewS?x@%jPKP5+!Y6YO=lV-8jUbvd?Vx!nS6NCIJ~bTkyX2b1y5*;c zEJ<#x`4HK}g8MA*nkux;_3Crkov_?)(%r?-!Gf=#G_!2=)-EX)u)771+(RC2mT8sf zsJk0J+mklebhLlG;w6{r)^c7sB&Z*q&|FlIZt4~3=#!>liI&7x2@|88eQ!g124NtQ z-2BReyOe25(_8V0W^sl(bfJp-i+}dDF%HZXPZ!6IX+`BiS$Hr(TzSh&gJG6N0A+IF z52#pYuIwMplUl26l~?unKB^&!NO4X+opb9%R4(~K6{)=~~Ri(=tE%qdxU zpu15Sf%%mpF79raTP--u^FNmxYg3vg<;{G%8^_nBl)!7QU?+SXrEJyy^&YmAK;cWg ziCQG35z&S7z_dH0a4sJ6vK39XFYFMTT$_F60?}hq|VMnc69_;zg<2P=c4x5 zZ==`X?UH6m_xPX#v|F5dBp2*vu!33Vh4_*KTx62*u9`(4H8MwS*7<@-1L~{zKj8kVJz@s->f#-NWb zIDl8uDMKWkH&eYAUDa`_M7K(cIc7gVIxhxGH=pR#oM@ceW$#zRN}}2$v$#vA%h6CJ zR@%z_dz*U^xWgS$Op22~*6!Y?!Y|Oa@S|A_4i#K@X6Dm0dv5MJg=M{@IJw~udRdC& z_^Qw`bfw?S4PK67hHIsn=W$>q4s#pu>b0;_edwjH>36w!V6N~Z0uxI*4_Xo%d&;E2 zd*3rVF6mnCO1R(N0pwSG%eNrPb^twglb+aGX%LlU0Zyex|E5^pH1c>FiA(pR%-2&3sfy4$xmIj z=QH-JuR}l_rvEXckVpC|58Ts^=QM@(RN}5n0kL=OlI{R&|y_d6L~H=Ubm}MNTMQseI#Kx z#j#qkLgkOJOmUR6+lkbEaD8_wLsjiv>X&PQbLugd{6ee~(8i_P8?$ab;xv~S;52QS z4%s@f&K5_bXd=?AHvb-G$(yT{AZTykMHlkB*!l2$)-^-Z<47CnvMHX4EyKLq?I{2k z1M`CR5qVmD7rBQ+0SE7yZEace`$FQ{4|-?~UF__t<-at$tioa)#pZUD)?N%u__xaZcMaq|LV(j|YTA-JT&{Lq!$N}SvWVfF0TFbt(z6b`+_2H%DbcSR9V#Iz;5Qf4S~oY2(Tc>^PNpF<`5smuz5P!FprHHUcebYEm$cJI7x#L|CG3DYEgJ3(6#Lb6V!x}ez;*#ESAwdaJYWV;5ny zs!NzRGQH-Pxb=rRYfooI-=Xc^L7mm=A?C+lu1)KiE!qpp3-(?Q;J~Qapb`ToYB}bs zipXAoh4a>v&_6sg6Rzp3bMZcuQ~3J*^!+}EXmr+P%}Jg*W#12-Pir+T9M3{(O7#Wp zu!SLlLe}EJdlm7Qkhd2{O?LS-)m109A)M*e0kU=lh95baNG6k9H*z85E#eo=at3&* z<6T<|8%^)VPeHWiAGkKY}jCgzL z)30{XVL4N4w;pPjKl2~#Xm>QQ%}pg*f?bMUU83#OC6Y436pFu!65JoL1;B@i2E*8 zv&;!|Sa4Vs|0gpSy&*%p-0UE|9$l11By{Lc>5;tkV@5n{ZGmKi#-Lr8Nad^DHzd*G zW|;<3dA~9EEC+F-nUxi)tQt{JT+NsKq-R0jVn=5n5j$&*BZ0`A5*^_gva79Qy26w) z{dbr}^tJ0A_XZ9C{*_pv5@wwD-q%X8vi{ge7mZbgj~)zKr>eM{?kwlwmFW;>Jt-p335$gaIKsO9$BwNGu0 z`m1?fJq{7G5{6Dk9b#{Jvx9cX0ZhW}wzaIVt%#6KxMzcPP6!N`?KNIYU`cie1mB=l zwk2J~KJ_Hz#LwBUwmkJDH`*mT`lM>iowD2XA%;DR@&acMJm9h>JBj46XlY~MZwZ<6v~fV(?4rT=_nD_wE#)yC z9ih(HWi!r%>*fk&U#55u&v(Js$-%)W*B}kWJejUUQpzLS;>ZRIgEN zCQAYhIzx8PG=MjT>;a0aCBZ{FJ|Ud3d{w+E%b^F9cGV*euRSqr0(+=;nIc1LXDpIy zyCGX%0sG#?Fa~ZI2Hg}x11T?kw~wodW+V(IosiETrKzpnQyo&uRym4o4Un4*cQg!_ z1YLw26S@?a6o(xMxpE>MfOfnyzCyIKjhj#U|AisIxpSAT68$%Q0TzSWJ#X{=I6saO zH4ZF1ev>am_7AbMIrwFJr5<;ivNn9XA8MWu^|!aP< z!pzjl<7AVB#w?LpuD{SAK2j1vC6aE0MFAMEW$5(igv_whyN_S$^dsA!vB8dy;xt%8 zb)-E1_&7NCjk=R&$X_msm@tB&7(evEzd%Pv(O-yluXD<5{sry&|K9lj4C(*dheAR* z@)%4~kE&=IYX4Gh$T4YUWhe(|^3x_gw#p=iSLpu$+;|sZzKGO(svi3oumgyTv?-Y;pHBCP2_Q)>JL{bnUh*yS!sq$o*KBF3UOm6B zzcGff?i^A~6vp?I0)Oc~@N+yv#zlIIdE}=$V;#*Qtr{KRM08q>Er9zth`kg8;CbsS zG8;%)%QTp3ntI*-S{D;cTkLlySDspiG;S3YziC+2PHgM4~iAOujENrhhvno4P|fBl_GS@x;ir7spZ={$oAu*^|1($1ZL5u{Uc4 z*iz!R|TaYNkhZT**HJXolG&hcmH<)z<)Wa+(K)dtB~dR43F#9`(hDZ+%v(y-IqH9o!s4SkjZ=<9zb+$@ShJ`&_=s0C%}>gdc8mS*w0CNS z4jKTx1qk6@!rgwGz)SMvEg9~I$>#~Xt*ivzCXA~fnSM52^-kA>q5&f6%Ylme=WV@3 ziZoDHWgqK<)|*r^XX#P&nfHM-C?J@t`5EkT=aniDA9a9vqGj?06)y_1waj(#azVTJ zX?Hs73eO*oXt1vpu>dHr1Ba6t%c)dOSuN@ZcHbm&(GoZgDNUoyFf)Y=bbQI$^^XBK zXHorn{n_Mfe>@jnmBr7>odb=a5+N$cBKVW$72e8TdDirseuS0&Tir`;l4%_N=j##D z53D~Wg$1^%5$&&t`tq~tZ)oqe+}^MboP*nq432kqQ5&dmoML#8Fe0jfbX=I&#YQ<` zlH|)u@E{w~HDcR2(JC-aXX6Tgh91JHMR2y>0kAyM(jRL{8>^i>o*k^%2oaN=U!ERm z4(*r~_G`1gQLUz*fv*G;yp8?QQQPNqU&{wDiS&(ARj`9%b^X(mj3~$iJQPBg*!GDl z50}2SD;2&$!RS+yYNx^L%#zvQ5%Itz+93jaLLddz5rjzWK15oexSh7f9YB6X=Lcm8 z7huNn?Z6ZNm0kpwmT0XD*P53*T;kpJjx8J7!#|s>Ykn9@tR)09_dLWCJ2N_x%k!!_ z0>UQ3aQce(Q{#5u*2^!1F~EZi zq)dH${TWVYFMB!Gv?t{eJ8M6aNzG||SCNu3UG{S%v*y~(m!>|qrucBEu~Y(lfAN^y zWhNXOqAvA0#Jc+Ei>u4mBEOL&-wt09@}H>HI#+Li+{i*CnwBJ0uyCYR)B^Z+@V&z@ z@!O3yM{{cq3HFTd!HK?}^9_mcy#?W!j)G~rv;g6S#F3!~QC~Do8gI_|6YD~xX`u|k zZLP%y$3-`qa@r)`cOQLW@GA$f10ylT0vn&jK*2=J3ajeA!XsqouQ8wj%c$3}wSJN$ z;f9+)%Q&t9mKI`znv{X!End;gnVBP*VaA5L61ci7yl@cau;fSD&Ed*K4e-mjCU`n4 zloMTO#p1%DY~u`Moe|#4_oaG7;Oq#N;)N5J!G>2RM{CCSrTBc;d_PsA38Gn(P~Y{# zR(O)Tg#iVUv(Q7TEH!?_cyHq}K!3>}wot8{6Rmx#nddn8wZF%-v}D}EC|o+!#yq^| zQE%(cJqjARNKZSD8Az9c~eLq(d!%kxbKq4qdPTyQ?M zvboyLWa(}IIDScfbK*9ouSCf(Y>whv!F)w^$VsSpV#M0}%rL0rNeBGH!9cKEsjCSA zPjS&BS*_$p1Qf&1vz0re{1V``^$BPzcI~g739KUTv_EO_M}gsS{Zb6B@DIJ#`SvsA zc~u=_f&Sd8A>>UI6G)7 z?|ND1PWDF?n=v>d>0DK~kCJYQ2Ob-t^}zTZ!M7+KpVFBNVdpflOGV@#*O`-y78jAq zvb|7r40E|>!%*LhNYBU@WUf}aURt$f{f%)anu@If5VP+J?#|y0?x_ZXbYFgpp~E{S zoGfUA770H1v+NCTn!*r%&w2#1-C|xNB|dwH?cXp_9}}uuLA~u~*1xaQp)@`E-Q-M0 z%6lSnAi2mOtHqTrb38Q*Hf~K?THnU<$W)CNc})Ci`nl%GTqooF4X0w_2w0<@f39U% zkzKQ{?AIFNf=wxqT^(0So^%lJgEbm)?AVDX2V;2zo$+xmdFPyj9du+zrJbjM3~*(&hy#hM6IV zHIP^R`jB0#^E3Bz6(TIwl6~#cx4KoM^mxZ|1t&q;&@9u1%^JBmXL8v%%pBJTH$M)- z2!y`LB=Cu_xUK(ehXrJB{+S7@wamHP<+^)8=-y(jcZgnr6M>m1r|^9oQo!7fk+c-u z!akVKwJiA68yyBwM4qkK+0pwF%3>8EsrivPzr&5OwV`x-*`wp=6QtNJj3_{ai7M7q zpGhMkCx|2YYFIE6jYC1~?aUOgCzd0lR?R1F1Tcr|_!i2as|x2IJ|Vn`s#$pjcV+Jh z$dlIT$y!kGQT<<88SNM1*y9pB;VCm#lBe7fHP!o3TnW)8erJQ0n|2mDbhS8AIdtj8w3&?YDg90m z#~OBTtu0k=%8Q_qy()P3L*z6KXx7#rlBD|kz-5|el_LFcg78b8J7=5sArwqN>OB^~xY}^>E4GhMur*oyg}6b1FtWf)Ll8g86}sC#|ZmXm|s? z6{MXgO-;BH@+N_U!d}QXD`uVZOBTvq(-B`xCbF*z=yQcqou#Hqw!+s+4`Zq7J|-Qg z^i$Z)nG@@N+GnkL(fO`UW?hv<)g+?5>bQ@+a_9N^%8mO%)o~B_MbN$PbfZ`Q$N?*+6mv49Bg#YgcT??# zxUDkt#r7xtH(p*!3JG_jM=gIJZo0dpmESj$v>25aLqu+n51EaLG$2(hXp`NKqmd@e zDJjzh*OgE|qJG|~;U5!(8*3i*1)>Ef4_+s+Na!h!WB;KPJHn$8@V(EJg<|K(2u}jD zhO+Qxmw;9ze{Hp4V*J)^57<-fO{%eH5ppm^krqx{E`OE~T|^X~cmhSS{Db&^Dq|tIGFDxBkiPht%`hu zN>F;j$0tN`&^oZ9?6UZd`|^-SmyY%;L(B{Qda=y63=7Y-E_{WVc6^v<1={swhaLDN z(nQ^zf!p;4U-}$B?2B(J7CX11L1dR0`uL~+){_HYr^uFSUwlIIB|pGz^uzu03|E{6 zH>JM{3QqBC($J|+eVl4vx;oAr8X23QRgWhR66B`%rDUnZ*s-nZbj~5dCx_m%SN>KI z4~!4;YcygIu^31QOW(DEXf?pw7+WP0#X3arG3~yTFV=aN1j;5SO*Nkj^V+{XIj@~# z5219Upy#fvVbdAg26lrCE>F4^Mk?8}m-8PP${ zOHD5ZlEa@ZfBQv=fS{`jCa|;DsJ4cDn8j$xPpy48BIf0ZdoB(dpN`krA8G#ct}tB;a#bRZ80&GeXBWQrr-6Et&}D>m z^w}Fk(8FZ9yl6DUg@>D?8RYB{tnIKrz**v(!mkg6C@dU1FIayQh((erLMD$8eJ*qh zHguY1tA5v+5Auu-jUP>x=xQ7)NhC_Hz`tMPqjX^O-+%RZqv`hMvLia}D906t31g^B zZ*!QQ;SYb^B9JVxb?>aB4OI}pb8>L%`hkwo@pl(F;adu6^ zS=@bEHpx|dEOPWk<6?J*A{@d5rhc*qo>R)D7rxCLI2>#YI|-C11G~2qHp}xr(rPT- zDu41KA0}~vdskqYO~kC6^tzE}En{FRaJd3RY~G)JsmS`v>i@^yTSis6_TR#Sh@^BW z(%mK9NOyNhOGzv`rKBW8y1S&MOS%MUK|;EuOVV@Qd%Vv(&WH1C$N6x+{l7W3%f0S( z-BLrdRD{;bqeYkR?Eh&m%Lj18n$P;z`unUQ4h&jc?AcBc9 zR~f7L{e!M2I1VPBFTZDV4Ej|{+B)2VMAJ{q+{**IOqEaYMTj?&b^AYzF=`<_3keWm zz48ghQ@~R)`aamKb_4RE4B3$szG7#DZ?}4$?!G6m|NZ@<-Si@&JU4{9I#D|3WtYJ4 z%+QG~^~Uta+hFEuZyBHq3W&7Q%KY45JTMcMen$N4xbR6HpQof2=abeS<#U#UsL7GI zy6ao5TpvB_ZFJ}H1C6kGL8wF?{ID?&HQrJ#lE0c5e2%V-v<<;m@98YtWnT~RWux5u zE|ewgrJ|`XxWd7n5>9Kg%yxxbwKZOM;n=~03aL8Z(%Ic!YlsUk7gg`y(AKBC`PqbK zCLi5JqD$giFZC1h(>bfJS?~y`eZXTIo5Q=&_W5&V1dX4Y!}X2?$`gIwkuxKyVYiQ` zD@yZ0<6TX&mplE(Zu@*Ln=gAGWyXVsTw{abH=x*aoaJve~j%^f)mq9{3%UU^PXEA$T&DTZ~q!8eO?Z=C|dU?_!im z5Go47I<1KR9Jjvn>be%oKl7}v968ge(c`rluBQnbHXF5I5+F?AU-x)h>qsiHu}CIjjTa$l$angg zX`)4ZVhRov#?6}k&Nv#u^m9=DH$vBHHrh{&0yK2eaO7-c_wU@jC~rw2LArpONvWJ6%*@YDgaI{ z&S3$bWa8zw)2LxVedG7-kzi3|6Vg&K;+xlr)Y;R>hz7pYmu%FPw_FUN#j>|m1*-m; z&vGFzOE?1Soevh8s!2AdUmAD9su!pe7?zrX5_M{^F%%~NT1nV69+Q%Uq1B5drzlIn zY5C`Np7B?AHvu`uI`k*iwj!LQE?x8{!z$rbNQzqRz=MgC z9nLJ#(t%e#VOV2lJCl1#CTJ1x!!yH{<-86Hje40JV=Y?mpuG{=E~J0s88ttb)Drk- zvmh;d^NcWc6)kWjG%N2$-&|PHTaMxy*(|-)6~yfHEm{b^_bwbb5ucx;SacXgSW`16 z!t#v}{ftmfnKz~H&}8vkOj|(PcH{E3rwkbG;d)+ar@5NpX zXml&0lNy_Ig%*IQzLtH^{TKlalyz?9QVfk>-X^L9ex0YN{N4X9@ZuXS=}iCOmH-M@ z0$T9hm*1VN*=d{4>+I1)k;2XPSiouHx#8}GvA~Lhs$Ux<-1{oM$n*0e30xlvVS6m5 zA3nAI80GVu^$n|A^ThjLikEcrL-{O~O#6G!1C2()^zAXc+-LLFDnd*=}>A8@{n8+9goaUWr_}w2`&wziy zbQg1h>BZ@Zt+6m1&aHEu^*!iLRS74NkQ1627Ch9+z7H48LQcMA&Cow@(rflfLns#- zMrD>!%^1=Kn`FWcdgb@}9s|UTCoa;Pn~Cr4dwlAqU%E0!=?lny8u6SNo~iLn?n3@L zVZ&j{w1uO_rW|RNQws_CJ&L3c)G=AZ!rrN_YFKu2#D6e;h+akY7 z^71yB&&6^92CzBAC@z@+8Qo{XtmN)g>m1D_GhYv=&K2S;4#xMjPJpjp1m?Vu@)fY) zoT2{tsajR-cgW)G*XQv3c28N#H;(mAY#o#1OARFx8{X0WLQGtP@WPhX-aHa6$wcNZ z1q)B1wHSxxGe{<8O}Sy~y}3Hs7!Qh)o3Qb4%+~8gZ3m?B1Dd&gp&G9e z=nZ|)|5k|P`muBVv+Y9^D#VOvgaam&7W2V)9Rk*c?wZ8}x#*d_TFf$=#zHN}(Mx4X zQwx3OvLPRCr({9T6MiGhd;Um#W|VSN*=E3V&Fb2YY<;~sxOvw4{+Uo&>*;6KtOvGL zrt9K=BJuoHyPiGU^Nq)l-Z;H+PmrZBi{=un8LK=DlFW{{dQ&+c^o4?l`1A1?K~Q)u z<~CvGoL7l(k3r>i^GY_K$7ge4JM!&q(#fhgEi+Oz;SAa|l@u6r60^Q#(UkARA3YCU z5&Y8O{OCf2)-AKV>b%Q@Q{)=9+RHnMIR7a7GQ{Lw>7+*GFq#-c>sC3UJR-1^7mLsw-Y+|J>q+9CO}VwPKj~9A zc~5-*hN%*YNZ836b?O`{PW1MVkyVr8Y-sTmkt)oVCB|G=CPH|)2KlA9x*io75Zk^2sB{WL%SK zA@8mZR&x7PTc=(hh+#bDJqS~H{-{bn*VMm^5fBZJ2_rzZ(9Yz53w-3%h9w*itn`}$ zR*RPh_1|nUzdYEpiV(c;BxmqtUUAa7gp=}&d(tZDL>a^4FXZbO{lJtPsX^>d)a;Mn z$7?Rib>~hKehVBo%VOhRq>0*DL9(5U{PQI_L~dH+By}!^8}=~)o{wC#D8DrE@E>JT z-198S8qXTpe{3;?Z_lz@Z(7@Oh$LWr6879pXJ2h(hgYc?)RB4G9t&^&w65ew$q^@W zcU;5NRf%y>7+DQ{j(Y>QUY?M=(G)yv-LuI;I*SMDfVTX_Hrp3!<{#rK>+=R`TDs>} zKtpkO1mi_Ybz73`E16E8R{at@$(cV7A3XT;=ORBI@jvlUzcP>)gw(G(F(RnU3H-%% zDQct=3dD=4n9Xg;<>R>JRpT`B6(i&0DL#Iy_(nl-AJ>!hPAySfvOxUq{xrj@!euL? z>9YMW!@cS7&|5g`GNTD--_ox#Q)&grTrxSw(kb#O7X1I*+0o!ad9Y`oi~pk44QyNt zV^CqK;Qgywy6I0atsptbV)PNJJk3X-V&e^jSgIWmnXcsdnH=Vm!+3o@~&8=#1EUH$ZVMsMT({^l^t=w^P_?fsSj)>d}V6FKNg?z#Jr z^tUK*R1?-b@IQ_;R#+#;D9ZBl;=lNAHY@>Azt@25TjaZ2x@_mk0GfE4<6Z(~5l}Pm z_m>IWt@;A4QW<(do^zLgy{!Yj=I+$TfUu&m2VCbyHHQE1?j7ARFRg397{qhWnSVI> zzM1BjDGZwJ-3`IXcbRaGKZ`xEh4{=)+z|eA z(7abs2WAo}b0PUu5+IYZv57sbqBH6t%Z?u@6C@JuPy)0LGmJ3_K58 ze2mXJWrq6>xV!8dNPC_vVnPkJLU6l0fKTq=~fj9MsrLqk?-;fzZQ%?e& z<1F9UF#)hR+c<<;8YlZ)DR2d#dkfSz;oUd5ro18i%S49vu=R8Tg|VD%^8$p5YR&^@ zZg_3Fp3_l#`MkotfS~o~BeRMk#e91zu3jRSxW|YToj^;S)X^GsQ^?as-?oZZNeCR4 z1ov43-OryNdPp~s{@MO;?0dWK=+Af~3Td48y)lm$M-|}hqr$>IVOY|n%*B-fm(dr% zPS0(lTLr@(L<`BscunOslN_u@vK~Z*Z~yQ;O-Oc;rm^_4de&MN}O{zhQy z`xOu$#2P`t(EXVgbw#kg5yQA!cFaKVK{_GeajG2kHSRgE7BrTLDmd-r^@gl8qo$EH zn>wpDGM0yZkrf`p?Gp7V57frm5=njNR0j@%&S*#AX*eJ%=$F?!)Ah1}3ZSky{J3V> zH-;5^Jdv3bP;Duy(g`M6T;-9flGLxB4J+QzDUyevj>NWuI4DKCm#` zD=D-*S+eU<#kCdFy`b@Y4*sS>`-tih5KNc|85Jk_x}hzxbqVYi=ka4mEDJ_`U;SBS zZ1k>^F zJ@%0gOF5Ni{;c8GxFCJ3+&u0{g0s@`SQu={iE?3ur<1T-TjLu&C>r!;DU;{{-KWwV z;~PmIro@aLb?Vts$7{n@fA--v7R`A_OGW8r`c*bj^NZW;K%yh|Joj(5Q9Le4FRys$ zYX)|z&cBR!&d()%04H7LnMPo2tF~xzP(bz5Q2V{oohxw7h#F+e925$jT7<)ph-xcc zvsztpom4DPrGWoZ81Tg$>0#QchYuUPhnK0PS7p^!rgx#l6Fzf=<=W_z1DNrctVs=W z@j$9wm-+`kR>>K*lW7`B{MynZvT@smw_ZoOG3}2fuAMP80<QtKY8GkOvVC?s+cy!Y{Z1|gv(q@o4DXRRVKGQ-LFh5SX(ahNdDI2;d>R57;q8<@x zNabtQT4d|^h>#*<9{jf9HP=EXdrfqOTJ#mlD~#Dtte8Odhef5B?%))s8%B0Vh6uS& z6p>up@BVS~4oT1y<4D9sYA;py2d8A~*M0qk1BbeCDeBPOmZca);XDR+=48~2o3Bq& zWaci0leVbYh!BdslTC-va$oloc0aO=gkq7R8#WE4Jzg5>pc{l?z5@51vdrCEOH0Bb zHp?2Vhxm4No-5e(UVb#Uh3|O*(S@_Gvg9^F!=aya!Ry|n^ObW*&;TiPIPX@_9(gFH z^hbu^`#ytbd$>o(h?}-nYT$?ne8XTK1yWuRKh1!;e(vf6-LC8H^DDzjm zfX*_(mq|K;R)vamT9_@G_4nI?FCBtR;?p#Dxri;|np{ zdvh8O%@tYi(Q+GH)6fefwk0cWEf`u99B7Y|LJLT!^5dfog;i;W@++_0PTRjOADLUE8ExU3|dWA2Z7?2_Z&AO*4w zh$01I`vH2Hm^WxDPv_-ilacGVVZI3!3=7@u(1+;aMHC=m}2Q zjy*vUKlrE*$sd((&j!{8Ze^rTKvfx+n*CqKpD2>^Pn zJ$D|lzs)u{WHd38Og>4%RWH_$|80{0=SgEb^TGUhL9e5z@wX{wq64N}DE0RFU#Hv$ zaZ$;ce3KWde|ywGQ?M|eCT7pd|7~F?pg@i7*kY3{zyUw!f{6{~65xH-pKU1};g0v71@`_kkSHF;kdz_shyA({oc+ zV82d16&QaYHaj@HcRgs@Dtt`p!}bZtS8V_!lA5MEGCg(=PFscyzw4b1@1tCTHtPz@ zm2P>HR)l*XV5Yq%IesS{b*Z){Z=#1BRTT5$6Vdr5c5G)IPE~vI1;ut6+;A~#t}f0B zs7ti3n1%!_72~eSunCd>=@7UE!4Gi|e&z}UuLV%T+dlA@m)N)i;DJ-~e*KS$czKo- zsIxF%+MEsAsEeAamRk_ZY*uRCnNxrnwx0O$U5_{(K~fAJ>gQNFEws0!N2 z^gfW4XR3ZV52P#*YLHaF;5>W=z@nTBOK1cpAxtl`+PckD5cI zkDt4$@PivBsqo6u0G7#-7a(^M;m^f5{!Apb*X6Y?0MYU!NGjGea+}e$Dge*pA4?TQ z8K^7qkBoJdwF^KwC%ETk#KR3pnASzYpad_1z>AF%JZ%GZD7QrkGK|#&fZXY};Z5Gc zV*o|TPc{G%dBv8^^)Y}+UNBd6!hbIE2Tt7ujFqh%h9CxF2b}w|0Orxt1D(s0>Y(zC zUjbbj`ra9j!)V`JZ6pqHhfDtg_o}fvyQ@dESWj*_uk7J!@2i;K5 z;$nQLAQ*86u#*a?%ylYy-us^?fwd#&Ua;Z^BeD-~9$r%z0}M^=j1kn~9R*OJojvIO zumPH9`X7^pKl)xMZml(*RFSdYx_xgSgWi)m48U(oVA5xfg45q1MyFw(VuY~`4CSS1mhF{(9F{; z7wo1%`G6pDk+1utP2L3kwElKf9plR+w8b|@pSKlJ4k`6fG7&?pTXn^L0zA)` z{@37-ObE5G0Z?0+uQ2~HfNk9Azn4V@WXCA+c7-&v4f%fi2jqjw{6({9zR>vBfSWkr ze{X^p^Z_WCYxg|v9|m3_)27Elch2>&Mbh{O@WIe*C}7K>FNx-!r&H~N@FDetiZ$>i zn^`XU>#(y)Y)kV>i9l29-2{-Gs0pRh`3Gaolbsy_K=*|%85E<>*V0J9gd=uqtY<^O#q%>rI=Ia0; zO6Qq}h;%TxCh9)TNTwQ@MVlB|lOg1h6wV@z8dA^oxD0PcATZehZ6+RD4FY+R53a^< z02ZoL=ni^x)$W6xhS|gngx!)Y;mf8CradSWz=Fu+v#btm zT63}iV4O~4PZ(G)jF|%g48xM)Iw8Bkm+%Sg>CQv8;kPMAjmYGknY!*lbR>N@`w2Wp z%TZ*aO^(ewg%(E1e7yqWxb;A`IAP)LcMq^6GjTFJam`j5JJ1|#Q&Ny92U4CSy69-4 z9&Rs_`kswnZ5$|51ac3BHYcA%L$F-U+50cvK{WzHcZ!Z)>U^ktt4$eC;YWI2X>~&# zCnup(Kk6zQExHVU+Z1KY&{HPp9(wc(^qJcw+)$X2ScZat>BCb9#3E;VudCCz)P;l< zgm7K9qPe!C?C&K`(#3&_h^#;xC~~)>Tu{!E)g<8uj(e*TH2*{Gwq4EV7&L_JpDkQ@ z)9`dlzHQgzqkQQmzDFUKu(e5z1+@e_e8>YorIne&Tbe5tRPvSdE@vBsS0Oj4d?BAz zX^q&+Y8XcWL^sc;-AMO~e=EidUX{QPSnwCe{2Y%XKLUwN1*i8}z_E(*gCi$H;0~2a z54yGvJN=#7lfWgUY2 zC#(J1bABDVGJZRN=p}u<0$T9}yyuCYM7M{3?j2`6{T0$h$-n=qD(cWgYHq=1oEc)e zF*n~H2`a|?FOH1$UA zy`GYpkA`t~;|$;}6|%YQCM-e#b;a!+XeHTx8iQwR?wWDgzm7v^>FYie=GJt5r`k`j zB+u`J@@PF-LDE+w!Z(f`gEI)DZv2uyK{KPqr~BE`xAVFLU(?|4d@-#(WY8!O7P}8# zph&z^N)z+7AT)$0ueE5Mv81D+Pk<4^=Il^E(bu5tRVhgtH*%!er~8l~M-c~hDXMOz z4G&FR#Uf0y}G}cz@x2Iwrnc zyWZ{0^@7Xz@ycyGy{o3etF$%oG~`@!|B9X9_EOXoPJJoOb`SUM6^#AePSL66)AyZM7XaBh z!v5^FRwK2S0ifK1F{0D&^7vKdsh&R2 zpsv`xJSqP(H;27#v@dC+2%{NsR-HlZ9df!Z1_LxaWcpLnnU1w(A>rt801f_sloTZ$mMQ zp#eTgJ2O6@|bmU$%rOKC?s$fX}|N67m>tY+YKME z?x!T(|2P(&ZmSsg(R~!N{3$rDcBnDYpGE&5TQ=7T#_2eMTORaTQq7NNocX@ZmD*FV z+S^q%=o}lUw^G}%_q4+7LBP5Cv{(8wf`ejy^cP)sKwT|lh))-y-^1bWDVP~Y#OYa- z&}wQZO?DU}U#Qd^=YFDs7m*0w>9RWjT=@X6U^oji&~KE^m^8N^_PkHU)#bwN)nhZ8 z@abhVQe|K`x!Jn;!coq>Ds1QZWOt<*b0!EWk|=o^csh1@EqqfnA(Yfwp($iCTj~~E zKeVsBOtYnPL3lRJLHqSp&XJy&j2|ligQgLdz@F;9%WYYX%Yx0{k%p_QDqU@_R!+d; z%fDmbe|K&e^J3Tcn{~q|xL{OlJzbY}sCygV!a{}_bMe-#lT8pVICHA|%_3;23{|d@ zF{M~mV&vhDn?VYW59ju#Ah&$*EzA)Gq6zqyIWIXC&;w0}5ev+wx)o7m`Y6Yj&Fn0a z`QbzMMwPB`_dY6Y_vI}Sqy7C8e3--g*iC_#bLKu-z8>HY7XA%jw!5{3dUYEi=43cK~dUH3N zzVBlqu*o1F@TsE(zWLLoaM>(oG09AOvd>kpalAwM=46mI>G08=1pMIxU;YI*?VABu z)5s5>rRwI`{t|}sy;bk>=kR=&;R#>qi_#}bI1hkmq)#E_M(lbow1hIYB}Jk{^5;#j zN_Rdk z((_f{og#T*x#bA<6lf4{d4{JwE`PWtI2MS<62zEUq+{3U@}$@F^9OZJBPZhn z19-NLj9h{WCN+4v2{w~A85o&vz#hr@PIf5$GIRM@m`%MGY(+EyEAWRLhtx2oY8|(f zf>suENySWUG@Mn#Q9O80;tQ9(IhAmq&uKGMk;ZQ-Y56EpQMB=N>NsWYif&bz_rjcv3qT;D>Es&!eSley22C5eveH7*0$HEa5xb#>54|kma*iu2 z#zQz5m{1A7Z;@-Os9JI9if$ZZu4~TX416NRV$+<>$(4ATuHv&nzJPpU$Fu0F?pL~C zb)jaUzyH1OxRinsfjAx1TOGnQU*E`fUm2(jpGv2?>g<+#xvcbAiQLRp)VWA%!JKR; zUajEl@aUeoQ}Yts4dCgv_S4Cw-Iva!28|57_Uf>q%~N>OprxsF!qP7w*Q)Y#L{f4s zZJSn(UF>iOXd!cOy;Oz zIlfiKPRpBU1nS{ES0A{P76I#k3dd&5kDo( zdu=>j(-+ESQ@CAtW)8X^SV`V*xEqSH6DD?OW;N#wuFvq)BPf-&p)K^vbQ&=F0mTdEE$Q`vW9|V6y7qOp73Ta8=&++K{NJ ze@$%3|J@~HeSRbywl7cj6gFX@U6%(&icDo}i$eECvFU3hlPIEP9Xs}SFJ;~_%3LoX zU91wHl$Sbffj3(i>fGW^RCtIZ^*F(*GWHq%%pkUx;T|8d0yhMJDtf%0u!<5j)F>Kw zO7*x-+CW(#G3?tHjsCCy$SCZINT?B!yN9G7H}!k0))fv6J+@VlEkor0v2sc@5CmUV zxaJ6eCXqmcE3RK;bjpk~g{7Nh4G4};bvX4=z6vE?qai}l7nr5FD((9021=<>M5CZl zhW>-X&3JBq7-zP}dqSN}6k-Gwr$tcgvu%MG$g;ag-sw3wCqZrV9P5ZR1vm#e&IHwU zMsXIlJ?eK+k8F0P*2#lAru&#^D|YDdpDW*j3QS&=x4Y4{2HJEkoiE=*JP zk--k-g@cqCzob{)s4JSp2wl+a@$7!uohQA+)g_eaX_(G0j@b&o=GQ)RaY(qyzKYVB zaVbZ=`n3wh8Zx3JdNq})5}NA1|IA)1V-|~QHvbl4h4;Mt6{19!Si14uonYjLq?(4V znEuI*MF)N)X-4ql%U!QOS9F9%-WjB5zWDWW1@GLzr}m5)Gh6L2q$()E+B;071xhKx660uvR zy);+Tl-V4Mxv8K#Ebadwn0bSp`~-fyzw&)#AtmdQ%PK;yBhNbuR0(M`CN{U1QVA`x zy~zw1(B^BBdX3&Id{P3yMaGU+TYsG^YwcIex7LbrAtBiCO+cjcnO)y?DUZDKRWAU= z6`lmPic4U6OSt+*ZGwtU{=>M!jm+ly`rKH=teLr}AS%8SBH1r9`{ZL1TUYbbnIgHbmQmDQ1 zoi;2307)vAhu{1wIQ!mNSv~7I+$Ezno_)BytNDf3p7DMvC?jYweYfYb>@L`dUQ#5e zU4MoT6N|!lTh8Id~V0z*p#5b-;Wj0mDG& zzM$*Hcra;u@qpS2?dR5epdnF21)Q5Heh@U?KX^8e9V#O|QNt)J@1S!d5qcOb>4_Fad@>i>ozm19GOv zSXnQ5=B@KIG^R?bs^mkaZ9SR%HzDwr`SX;TS;d;DDRLYDPuuDMrd-e^*N$Ta@@?ra zBgJeX;CR^gJ6^Oh?I@7!eVZF|m*G6P0-~B+bIp-DxsVAK2*E1RQKpGm`fRKdT-5ct zK~SqPkCEuq4AOKREdtC&jlH{kFZFjdkmMD1VUnZYw7>MKwa`bxn&96nR~J^wK~m@y zb^j@y$3%zI1;mdFO%==D@;y)}G#;nskZtM`p3+JCoAzy{d?*w->QDq(0WYwtksL3m zxaLWL11mzM&64NVezUKsYG%ZHSI?*mD3e?k67x-R3^S>dFk0>vzL+(rU0|JwmVHd( zVKRGu9CRJ-6(PzWk{0ajFZc%_XL?BXYAb#9J_}vgmF7BVb`wBGn1{k1(F7j3w1Ka*Jhi=JXOE{*cOK))EuFkcFnqSGbS`{Pf({>2C4d(y zYhj7H;ZoF*c8RldiC24POG@|W0Ch!rYq2!?*B33MI#q5@QlZ_z=8XoJFwDQ_dcaA~? zs&6q59B#!0-sAB&>|Z}B|Kgbs+sQ*1f%kmDhPoB@m}*A|$I4F86!XPtO&Ys=n*>Ql~>e%J5!MBF6xgo<*pnsVn>A z%13SD2Q_r_t3KTN9oOOY;C`Nj9lCZcuxwxWgj z3J|uz)$UX~gDAxcfpL~@uQf{tiIl(prs-#q2!YMFWo`u)fS}P_CaN}|NeGGo-A=|A zqOCjA<{veD7>bmFb9E4?t=i7V_P1h365babwnF7I=A`gMy=e z@a%rhQZv>`usV8J#iLW*oW}azHH{ZN*X3&|ahi%jM7?!EO;!)CDX~I7#z^to+v~e9 z6OyV{DrvXtP~6|3Lq#0cz$02F{AI>(?5Gxx@oa?YGK(otmsjuj-!%>I z-{FA+V;*=)5qAk8uTCw24Nmqf**{ka(>cDn3zA#S)rF!B$srglw3Jpw)O;ctJH>=G zRy=ZHSOgL!1Spda4ww$?hr6v_4D*SeXY0xSg-9KDAQVC+*w;23CHWvj9#>N$g37@ zpD}dl?C6@o#b4ahvmvt~N=>^QUX2{j4yPL*MLU)zsVXiV)F>H}fTa~L(yy>QHk`eq zA7N#^d-IK3+Ed-3yh#&%fFgJW(PX@p5UC$U;d{;iX@gcdo!)69X&`Ysspg5sZV)qn-my8eS$;t z22`f7cAoZ@t_)p!J-S$(m;CKtOHV9vBgzdhjT=dbT^uc~dxhpksU-1JqR(ZlbsPZl z&VJSJMbeA=Tz|Jiu*XGHw;fnE9SWX;bm{ZkcxO_?-`2E>Oe_>tN3Zf_!>EB8d38N< zW_6YugQ3Rx>DrI13LXpzn~>6y|oCr>tyi>I4s z*w==udsY6etg_u}Bxti{UJOrnfao|@`nk+<{!`~RMF*xAQ9gBEF{b_vg#)E}_!Nzz zzflOw2Tw!c%O;Fi2ts}7Dw6^b?j^`7e{UyKT8jk?T=xG=QR4ro<2pa5^#~aGBx{@V znFTw>Noj;O*@`h(AqeMxz1zb_joOoOk0oLZ>Pg#!ksj5l|ny@V-}j+ecgR$u0Tx5fMEnLRtvxkCWzw4cwrgr4tg8 z_|*K>Y=+=wl7{iFI!f%y6P&&uBQz7bk1LLRo5+6m49%9&WQnpouyeMNzRQ{|nQ}*B zV0-%t|8+%h$xMUb7w~F5FK(}r#iCckt*qcqfI(Wpo_?j?+D3&{2~P2|ZFjSyfNb0( z9d^-G!5|z70ff@GR`ait)eG%h9^wpT&%Ql@@jYGtQPMJMHYrV}y)fyXKbog3^(V#8 zXlbkB2H!>bkwHv6sTqU8rTXPYVw2ApOiFd~@~rk|)ILGcp-ya>I8Oo|Wi&ZMzG;TY zZp5LPb8Y8D_hZ7tHO=eA#@~IP)G7ccWDdh01LcvW6f#<7B=@y&-CzQzf8F>1sddM( zvsyX&B#gz98ZA@8OwQy4x#zpANK?;meJ|s=} zt_Y-j1}T}`N*ptOmrw3_L8D#~!MhQ_*Kk?w8PoBml?`E_2^d*u-Q@``Rq^nBIwMlnujT?iwDgJ~ZS zf*Wt&Umg2oJrY3WaofR)@p$RiXaJH9Czba26aVNq_2Lab$^{-dgST&*9?K*mVu*@t%h<&`iws-W_pMCXfdVvXSX&bA1z){HcJfWi~pcAr6 z(?B}(TCxG|_jErN`vo`I%~qoUmpsxv^7Ennxi2EUW1lJH`DVq2Ud!EpjdX=xlh=jh z<kMeI7$P;1D;4e-No|`e1sx!(_zg z{%VVMzP#%j1B-0P7juRLBcc9`0lc_lEsJiPk1=izOB0}V;bkKO76sU4O*t0Rk`Zk1=D(h{f2$Pna8Hk}7uCu*jeBw<_E_NJ1HSi}T=-zPy&lL#A5f(~6RQWHhlvUl2Y0LRv% zWS}my3+{)IXZ5&;`hFRptfogOK19*hvv7-;6SJUgp?928uS1_6<=@A($L(4+0>?yl zNXxwQk(xb$#}e?G+$q3z1QMawKDS+xrhGr?%KKUp0?#^TK}l5OV3loaoh8bC!~7gJ zrCM;*l*<20)lMNllVl7`y2_5Cqz7-6uODb`aQSlWN?eDUo@pjccA<(g2omML!q=L$ z+3k9TXhm;r3>tu%N3YCYBcbVYk0%uw6(jc16S6UP+@4IU-)D-$_c9Gkczk*@E3jDs z8Z1ubsa+S7`6yzbo1v)&bTf=d(Xn`D^r;e(!McJjl_c0f@ex~=3qfG)GTGjj#$L8m z1aXq^FXU^BujwrRBqBzhp}GVqn|t@_MVB1;B}Js@ZFsQrc!_{A2HV+WqW_2wqnBw? z!{bmcB&n!^i@k8Aje-;;zrXR5weqJp86WLs-mj||i15KXNXCx#XWwTYANK?HK(U)C zR++r}TJYG(i159**F8)Hz+O~Z>JvLI3jb z;Rt`olE4CH5NtTEEKxHzEI1>ZinHdZew+hm7Ew8y1gdQlo|X=I&Oc&fUEiL} z4y-=NQ8j$7Dc|%@3vI=W5(l063~VKW!L(#d;WC`AN^dw8Sy3RctH6zq#PZ?+e#6;$ zACC1+393pW)T^dUSA)hG)LCUAk)k$x`d-CYMO&pWG({qy=1XU2I}dQhqX0b(Q=LS_ z8|?#~7<@q#PbM#O4kdLdX>-Qv)K<_Di3-g=5ovT1TyFM5&i$a67;`GMGGNaunj9b=iiieH z#U9{>@bBYpG7q(qTu`t*{L2SL;bGv*r&W)W|MI&bp&}6Q?n{~={`&xd2irv|9w^OQ zW&hiJ3$y_ec-#5NPPBg&%0u5bg|$sr16htXpZ@(F#ft7BkN}IzOGR}_E#sF`|2|e= zZdj**w_hKgn)$au5M>0T_4)1i%fF7nJPnxk_PK?$e|xFlAjR<~6E3X3j#fRS?oTV; z`RlZ=1G;D}Mnc7xf14)R_Fz8E)4cZj>n!;Msryl+Z~*uA??3&2hVpN-?f+sZJEK(0 zg@AU`2HfEsZ{`$~LF*?wJj6>55s!g3ZeJI_W#7vJ0PJQ*T}D|6QbLfcZ)Hj;()#|{ zq_XnqsDPu_+^|t1zLCdDN%MLBVpz@9{A_z7FrohN4<>su#Hc*qwpkeL2%4x|!O>gz zVn&B@A6(>JCi9dBYoYcXWytx~3p7M&+@pJ|5C3nXchC}uM!A>04D`)^1kgFSQ(KC! z>#&mv15Mkd4|qvDdwZO>HKDoHhwZVQ6E*>xHB5rK3;z)@Lv7DHpx^x9`@1z>XV9}+ z0Q8wc%+)>EHj&Krg#8~UpTYESbv5oGJ>eu|wtfb*!W;mJ@MlDYTiLD!%H%p=*Qm&f zcgO;G31qn4x4{lcIrya@7(+-QSTwnacL;jMQvzNz>JsAKaLx##i~PQcfCdkxAPO<0~# zX5c@}g^(FsF0>P})&@E)Bs%^|(4guJS*-e$P%-)cu9W!cy?kTTydE#l-{HNVy*MT9 z@*gjNZ%aUk%C9=3XZy(+wCrChSNiVT3iQ)9n|UfG7{~#+w&yiuXsrSd-LRo6_3O0> z$NWW5boj*q`}VAlQfMI6Nh!A+>eI`IoaR5zzIgtUZ?J*D6a-~u#XY~m0C($fFOP3W zBUZwk}+`_~a zO#GHeFf387M+lOT!?-H|*$_}YL64+9=P;mxjN*goe)`@VD($`e)nGJP|C{e8_Uj zNvKRx&OSN%mf^rS@N^TkAD@Z}AYlUk$`~|Ug1xHdl=A-=06d$_mQcO}<>@eY-=H#C zGUT|=KV&}ox!`Q%aI#PMEjz-`pwLpRN8*RzZ@4o6g%B;9eGBDlWX=fH#js6Ac~KZUv>;~-l2$&lpZ*5$^^yaFYew3DRicNe zxcFGb^4)aw?|U!INN;qMEvR9~P+ZY?OFfvSS&&kG(#6NM2TM91TKf-Y2@H9T`oAG~ zU_c(;nf4q*yWp~3b72Hom5GnfvDT62|8kE}75oE;i99Z{o;=B5NPAr1WO6&&KzP)8 zJzY790ow;X)}uSXR$mS>7MMN$4RoFE35e;6zHCD~AS}VvzT?Hj8^*vAs!jzX+)Zgx zU)QbrF|V0g?7o@MajGLAy5InY=A^skBpX7APIPNw$P6`z*fVOnmz z{hCrffHM0QB9?}?Qqjc$YM3$WG@?{7pl6w4F#$aMg=v)h0>0pCu_7SoxpFmaf}&T9 zHu(7$2{v7(Pk;ZF>baky_Wha7}*BfK>RmK9;;%R$8XW-V09s zACpXFZ-CmdGF=f^a@+w+G4veFok>59%E}hwOw07Fk_e{j9wc7x5Kw%z1c!!8ilCfC=ZjIuSbd_Dui;t**id-+S+$+~Y;u5)vc=Tt{~em5O-Bu3Kog9Q zI9bGrBt_^dJDSyZ@6S_bTMN_CV@WmRK29b#-4Hsrtvx^7P7`A)Ns$yCK&PeI%h*lJ zY2pG*=@rYG06H4rL>Jybd0Q4-^9=I|p#7yJ9WGK#I#^v<2sVO-J%qo;V@ZNTNbrw| z$1jq3UNjg5DH0u4=d;4Z2erd4U&upkfaH6`9dHIz271ySz$+u`e-ZZ9QB{5I+CL)F z-JMc`q;yJ4cPO1AUD6<_!lqm4l#uR{+;o?8w}5oRJNNTD&-p#$ea|@G{}`}*T;w{;MO@z>;*|g|7kEJZkGGd2Vf~J`NdLxc3!6Xg6o-W`G zLtFjMzZyY<6!wL*HnHt0&L?N|9cWM28)`Sg!N6gv2eu`3IjL)s68s5hxmP>be^)PK z+FVx+H7=0q^VEQ*l|&AXg;)O%_D+{scQr`8aNEQW;0&xM9@G*_=GKduI;XDM_#9TU z6*x$s=Zfe&^KnSW(^3__ffg-E-}6_N5vd(Z<;~*A=@3Hpk`>~l$!Cgb@`^3=P*q)C zI>$C-RFjpH0rusueb+GcYhHO$(BKzoy-^leYFZxGXa2J{IZEFO!%7TCaJON$50xUD zmoVOTL!v`7uvXqz%=vH?5aO$|mWNkiL_ zSLWgzxZn|$viNP)&1n>n>!!6f*`Z>v95h)0NOZEB>B*d4kjO8SD^9`p+<|gP& zWc_U&ZZbOjq*#4MI4^QH%f0*@I*`x2_SqXpA0a=xB{`}g&@?}%>7&hiHCcf;4BEEQ zuw8mY1F8byLIlolE}KrUyBVZ}*TtqD{2Glmzou(@1hzTO)Usxf1d*q;#G^dSm8$hH zjw&zDvn%54TkRE(>?j|=zmpQ3t}`olzuoaun6TR7kSce7ioJ=qGH)9cLGm5$<<_q- z@1#xT2`PHAB#gD4BUwgUEWf+DtWnr2ai!cqM_l;A)q~xG2$nc<;+=I@xr&O2*C|%L5`G1##O5x9je0OtZJz6yZW(`Vf?V|!172oJW0J97Q)pcQ`g zy>bLYR^JE@7fQSgLcB6^Bu739y>l8Z%VXmA=7G%mhp{L>&HP)KkI_$1X>LBi>{O~( zL&*|v>)*(j^s#1q=$)4|rym@CrHJta8V;6qYrRKWO?1AKsU~hRTKr5{Q8NK#?{i;c zlUy{s7ry>cCJIo6qs24cdgx~Rhu;WpSG)-F(2 zv8%dI#H9z}LO&Tl2_dpa=!pHM%#&zvO&zD~k{kCCcS)gY=G~32o86oFa6I@V`2#AGLWC>e@-!rjFXT#|21`@3je9(<3p>f- z1-#l2`2l7?bt0aJ=`9FvCRK;t4R8N*Z19dY(&kF%Ak$!H=?ar!5406#)7}& z37*Vw!hC_u#v49=>QzCOlhKL^N#KxBlb8|@t`Ss?#>S4;DARdr^FBm8;ez(E*eGKE zs2$}GsCzBG50rc>M=dkF{{^_3ynVMwjYyw$C7KM3zn~1HGfD0kqzDWvVfc{aMxKM; z`g#swiR>oQlWgfii07?qK3iQ@Z^t|Idrk=S#1AOV!$n<@D|a>M{uUzkyTQD=u(+SH z#9N%TNSgHBv}JJ=8>(cmFPy3;ZKM6tS-$}uBc5V8f}#CK!GUNwWbo&r%TB@i4eBC9 zP7Bj`nHkVOKIm@Pxm$>eZegsg?Hth#T)0_^W`y+jemAQ)x-Y1qtu4Vly}VA=1{7pc zg1~D^3xT-PxJ2v7o+&0Xz?Ey?q7ll4-+1Jm=73uDR=k3)xEbATMZI?p^O;J8N{Sx4ro^U zywUnd+%);%yRl^1RQ(+yAcdG2+r`+I8_s%;?5+i5C6c>)^uHMJ^^y#|tP6!#^hHO9 z?zgsQrVet-`-z@fl~0UFP;Qd#V{Zk#=DaJ%?aZXsaoq?i2M$`aoR0RW#8jEpbU%9c z@?Sw{!x2TN73;yViV(`g)kH7ePHylTJ?%yq>r#Shb=S4eyu(L%N_U3Cx}UwKGW^;} z!Ga;2J*PL%!g=?IJE%89aPxFLH51Tt2W*KE-gy?pt>X~;aM;8T8_%;nTceC?ut3B| zb(cV%-?i;g{0)O-B-!6)j2Q1&Jbz7rEU8n*er~HRe8RCXw+o3N#<%7Ct9zdO`P0Cmv6@w{*T21z%lA6oA17?3! z>EoF#*u6!4G(hcSiB-|(iEG*L0D1<>y02w-3VOq8_NVn-hQ{HTON;G0D+P}-Ow2=L zV;v{eWb=qxcBH2U1b^kUtck~Ul;F*#X`^)2qtfUvdZhxJo)tto31}a^qlMJh zBS+}{CC^ZUM5Mr6uI9@Kyl-%^K0%IvjZ7fd=}}F;vG%(3fv+M3zvPw z7sjRUU{Gz)G^~$+Z|h7EIFl>_JraJrW8iofDDAgPzK#fKEPfLs-Rj7#P2|YaUV?Vm zUS)ibZIhfiC@DvRf%jDZ@XgcC7HssMh7YnLi26<{CDc1B<*n%9Ql{dn1leXolH|j2 zqNpWu8Lpb<&xRgt8Q;Zp8+b6Pf^#dH5^%Ve=jJHtVTwqLO0;AlD5^V>nfd$`O4&dh z_npyv7^`J^W_12z00D!1)Q7fMUVSKs5F7qZ5L2aGI3}s~Li`L2o~~5>>SkMX*=5JL zNr>v*segsJ5fTFYb{_xGJ_}e}~S$ zScu#VtX0LGF*vFiqi(C~zYUb;Iq-AiB~%%k7UCh5veag~Wv-Mk)=isL^9}o(7+sbn zMS&yFJ^Tyo8GaaQKjgE0ne8FZjKV(9Ie12Cn*bLUri+#&O1W8mvsawdPcZE+v(;_u z55LVcAROHDYz|JCxfzHyURce`lDB$K*R(sg+RGSqx!h(na_Gy?4E3V2bCgHw-McNE zq{LspZN9!-GXEmsbv?*FL;REOPsQ$Y?1NbD%KR#PhK~+6!Ne@M9V-Nv$76_Ubs`rP zQ7=cMG!Fq|%=QVYF~|VCmb65R2!#8lF9G)pcFfs7%JBDT^e2HPiGG zNHV#Y$`D$sM@^v^0ESF|0^iyF_0;eQ2;Ygu-okTy9oWW)#Wdf6fnSB@_>`DN$9#1f zqNh9c$utb{t12IMFB#X`kKVwJPVf_&_vSKE8sapuBcha2oPnb!AQ6qiY$ZTwh{*3YHe zB*=@(B2C#NcQ*b!ws5bio9A}T8dIwnxjmgg zB+_@?f>%0om({T$p$$@l+m{V^5sTxrp^kO*M!y3En=`{gR z^8mZM7~QCY3?bqt;!$oH)K%+`o>qL9FJs7OMk?XV%vT-Qv{=_%DiA(ZQR(;U<`s9S(1PH-w^6-3J2|GV zH845v^ff)(0i_%50hyWCG}oYHYv=|?*dw!sxk-w?!OXKw!u=tR2-4Tg)9Yg?IqfEh zI-&vsW_^nc+++ zIL2&qE{|9V1vh2!Q-!gT;>A0=X=9D8Xj+v@liHs0Xal*ox%v#L-l7%sq3G=_W;StQ z)z0(HI}sc8qVdD=2V7!dGakVYVAP!t2(b}3hkT54rqvnNUPRk+Lu)-Dfs-Q6ZQhG- z*B=>#Cf|1tj$kVX)22AYpA>2EZh4YQOKTfmcEX3CpF-;R#?_D_?AmIBk zm7j?J!g-2f$-QCkLyIXPvnMFHNUTURNi4NXGBYx1CraIGm8^9N41&}6f`utVDVWw% zB5dAA!D)Y~K5bAJR-p)&W~Hb|FY$JFk>*&&a8pDe(0grqR;RmExdqp6jV{L7SmaIR zF>bP4I99+Tq7og&QWVBIN7x?~-SoV4fZ6xH+!FKBxvP`-B?Lp&JTui0G$#k1W2@4b zmzAbX4Q%QywutZog3~o#Db5Mr@MXCKR#4DgBr?e1m1*+`ZloHaoZKe19A~5*Eqg6G zZN7qSJ@|7TlxYwZlBQ0+Y1U%1geD`=&3KJ|3+yjRBP#{A-z4mHd%x+g(quBX_Q1@> zzyT!vL0^*&FrK4!k4Ram2HM}XMh0o^!!*R4FrS8m{96-a(&Hk;948ED0q3m`vFG1- zunk%Y9eoM^_fs;`g4L=vHW6pA$tzdS#zWwLEth!I1U8V5S%)&^ zaR;SM^TI}Zwd53khU|}m>~}?3l=!FN?+$3qGA)Y1Xknv?rr4eoYardNep|QU8G=XX zvHAhu#%YPKWJ9+AQpfyv2Y1W5=J zPGN!BeUzLCR5hL~@T}Pu)DL-M%axd~Jzl)un>mRGQ&NtKjaPbR5-l2t5ttYM+OGIF zKib=*cLvSUmmZ(2qy;~Kr2mZQ$FCg9_R4fWq^!aboS7!OBymwnS_^2R$F}(HAo@@J zCAUG&9!N?;LfP0L-w0`zXXdZZaM+#r9hvZ5!VwJ?zcOf9(0OiOT1C&aT-Q_Rw5{GG z@%;IQ{%2QqjF0gZN!8Fxi#3;FVQ{)=X_%t;Ty|Z}oMtB14}F%QvV$Yg1ks&1DmDs? zF1C^9l%d(Zin>d=g3v8tnj{qyitYdKYZwL`%AxCt9ubJCgBLHMB#92jE<8-Xb^|INdq>=vZ36!# zn#j|N7J%psJi*-MB126oa;xI}q9WNM2bRCMc|D}|)ak#6B(vOa_~`Nyq;j^4hqXJ^${N}5V%-j-8mCnz!-14PKIB$ubo~UV>tM!<2 zh#T{2YP5YzYiyY7vk#-{S^dI+mGv}hPn6F6mrfc1iS?P64(+hw{MK~GvGUX=r+eO4 z2RDS54S|Gy)t`Ovf?xbUp8t!6lsEvBIM92>p zhu9p~!03y-;k*V-eXTqV}rm{{OKedW)nZxGHEuD=5S6-vRGIJR6h{XPZiVJzazBzil0{h@V2{JALIgm}{A0zJ!0`gP-Au?&kXL z^Lca0oVl_b4t4@(uw*E$9pB+!3zLc|#vZaDy>lsi6GK3OH7?P1WGtye+ka101k&1Iu*Cf_8{a~sig zXFXz3Hdm_CZ}@(Dns11nMIHE6P=O1=f?Og=y#IdnrhJ&f#$c)7Mz?j#JZ+rkq_ZhV zn%wSf+UTpbitbOWmhc<BN}}JUr0hK5|P*hYr%o2Ga+I>hUKdE zvMgdTaxIFF7@1wQnl<*sWJd4N{=*>6r!hQV7KyYDE@#0%LIk_@X~Y43VI``n#}t*d z)*8r3RxS93XKK2+X-@En!o{VH9~z!Nb$e2ET!@9t)NY{D{MEIoC&nV=+=D&jSwAZz z3c01(;H=+(_<{+`CVx^TCV}OL*EzFUsJYfE`0^_Kx^!PcG>65=teCB|=Q(gYFii!` z@}j;v?_Y+%*}|B(b5-jFJgTR82+cBj!#r4jy$~%qhL+GFt%K{bk+e6!8w2>bN3r>~GR= z?AQGe*0xat-Y=Lo4?Y}X;G;!d^g)Tny&R+R3*yMUx{mlG*i-z3C=-kjs0;f#rpl*A3`v@s zSKhoF5BqMx2hT`WU{)Qy1^gVaoGCIXiW^vAo#V zMHHzW-1Tl<0-h0iT$x+2-%w1Qe|8FoaMDpC5P~Ctel4s=^ty?ueX7V+LosRXhZfWG zkxO+V4#bXbtEq1H!#DZXEvDmSmZ|DVR8em-m0*gLTmfI}bMOG=F9EOBJlWKRix^`D zd37w#4L%zEk;G$uC^y8qV_wSsdCGw5Z6DvMD`TaYMOX>YRQ}6R+=6n}^j@Fn}s@<8xFj%U|w)va!g1K;UOmS zJz+3|vTHdBF)$5ZrdofCK@SDdk;;=$p2~0fXwFLYtos(iZKmVM*WmO|9zD%}0$42H zpPd?*^ofymay;!ajrDhV262JH^nx&VFajmk+}+K5a>13 z@NxG>f{;(BY=1&#OUdIKxXVDMUX<&=?ae2T5#Pm^mfORq9+aLq5L_iP2IKM>J^i`c z*tibUjtwrDd6Km$k_9U(nf!JyVm62ZX3|8EZ97H%p=1XA1zz7%PyZW9svFgw`6tlk z!G5R+_C66WbH!;Z)xKg^BT)=eU7Ren{*=>aLvDwY3QH{<0*TF{@#iCMtxKf)4N#$k z9w)JcQe87%->H}EKF%{uO7~PZ51eTJ0JGgGu^2e_2`}-NY5TEDwtNL8PI+e0J!lQ7 z(PT~famUw*ES0tnDIwA3SRYhomnQT@S%AZhw(CMKte3X07lqKWjOPGe-OxK>oggxV zD609OQ#?x0t+E5uq8^`M8I^grtQWgoAxB1jc%YB(A20FyFWw{Yd@rwJ81Of}G6D)3 zH~9MgER9G4f+xit`EK4P?Qh+vM?-1R!k<|RZLR_@JZ6?f$Mu<7+I=XQ!yn`-M2gg4 z+G1toLE92w*!+C8={v^~iJpGk%#a(mWf^b)9A6}K0^{JJljcDMlwQTju!HGA#A%L_ z3T?YdHN@ap1QR6qdo{EM=WL~!Ku6=1H$Bm9;=F`ncDSmFp|s(GNMaLejM>#4f>e1J zrK+lM24GpTpuFY_3uD!%>+UE;4&2k%--7Sh6V$%Cm63bB;w%1M6~hN>GNC=j8+!Ta z9l`?J8R{w`@=Wa5u?&-%mD(~(S|NW@lf3yBbt%a=2~m$MjMt+#xr`##imvFx!0E2e zSsR%#MQL;c*L1L(@&|o%*~HhZC-NPQBq5$gj6}g>tvO%*5IDXhUPe6NLu4jKqCw^x z^nr)?rr-$o+Onow{&xScg_klbFgRZ*D6QQZin}aWWpR3roDva4gbaVqS?YIEBjS=Y zhCEcf%z1E$2gtekNy=S&JlEDc#>t2n7c3UM zG!v3bXO@6IY9J}(`l0Om4HP#GB?w2)dqYm5o9@UoV`@MmAS1p8L?N0$d%iI_VA!_< zF6!PfT!HFyv*s;l$&+hvFwH=NXQ}?=GDYqSr(c=PHZ$p@fK)yBMepu?Z?>am91?2e zSiNGn*UV3wRZTItG;)qGB48(Ya_|kKq3GG8CNenFz(ew{xTz&W5chOMn5veaie`+o z*8+UTVTs5Q3HvV6Vasas@U&{94zLwUGz8d)my!01DsIckU7J$@g z0)VCONJBYJLoQR=yrH(O`IkQ-uOuHg)2eC-^n|&p^k61I!xK6QTWNUR!{(rMD3)Oo zx6FRghquglECc#098dCWCh3?k)UPufGhH z@Yfr*=e@{Uwk_$sHvgsJL{(>o0?Zejq}i@yK7GkfIl2dD(sm6|ZJ$*c&xSF2-lL^V zbq`YLN2nWda3c|QAt3ziRY%v5T>*zUEEo~lKXWO%IjkahrkM=XCS)ZYiENx>xzRCp z_oaReSLYoC3PYTz)&)zkx>4kb;te<{A+Hu5K8u5jwj!D_xSbVV4=N3VyO=ZcjPOxA z^GTe2>du-_eBxjVqk#w3H8n`d)zUioW_r^DyULXc{DdOVOTQu#SExGzD}_~Jw(M(x z;24B+B4|S|IURaYFRH@}{hs=0Q6x8}11betspXoz^RKgd#GAOpm(fIts#z4BAuPU& z(KXW~Sn))de;wor^fnkB6u+H(ZVN3uZwOJ>(2oM!ZjeGig?`}$FBqX2zbZ(*hMCjb zF;4#4N<&;vYCC>3pOz&B&@s5Z=A*Q=U*CF7Evu-H# zK7~HW*%P=RvKT}~WYdOQy>9E|?t&D0wE8IcMmdYd%U=uN^XpUf7v1T%Qk8sLz}j*; zy{~6rb#95gu zT$6ajQZdqxpZ!EoJMur3C!8u(jUvSuan1X7V({5EhJeP29jtLecBAdEpb!&s83E(R z*Z0c2%x;p2yE5fSHDf}?p82L2u78{pxS`F>`0TSAdgsv^rDow_(-(!+C10oBa{(a`Q4{s-uK;jNyUq1gDh@t$iZ{u0+h#zCnsy zqn%+~oi`J|o|%(yK<(FySdu3uYQyJD*cA(%YnXM3p%*AVy9!3~XHQIdJr)9Z6ED8k z`oLJ(&ZiEiBwk9sSpj(5Yw@_o3~=|V9a0OsD&W{54_oGsWdl6z(XtLzP;D63kcfg5 zIcq`+=Yj%CH^gfpS?ks{_jLwlP|yyURHJ=CBwWAwGzK74!keHm8=w5h*6DA&m3-f< zRWQ(UmHCkOJI$Fv_^g+`4OA8yL^hVv_ZA+WQQchEV~$wm_XylT>UWGlYSWtP0U_To z%AniLfYd^UZHe(jhxZ!q0{{R5aNy(FLzChMM8am!(GoA zTrw=WNzR2Td~1A`j3k`RaR~jwsDhWmj404c@!c}Z86z&Ph;6(9%QrKcW$Th`3#qUxfw^r-mgv> zHrL1E52xc+&H1BR&y|y+EHi)9W80PVu6w;$U|KA>$jTyu6<-HIe{;%h5c}KcFnbXy$Xc``1qJ zsm88*);B^BUu=);UM@LfV)#~#&e87+HrDe_NyC@Q{-hS?9<83RmfF4o-}~wWU6JC2 zue`E|oC20zT1=w8YgzD$l_9=1dzj5C7< z%7!NvQe+QZjG|)?+nI0?K<2L~_+*ZDk7+v85@XX)hI_paJ(Q<6C7S&ukj*|dlAnjx z^ty2p>jE_|Vr#8Vm!Rhxo`+oK$|Am#y8qVH!=iA=sPw}%iR9A$G;;`9f=nw`hoBk) z?bZhP*Z}S->dr;$IH8=r;ey@&$;&)omll*ijlJGR|E3A@L*Nc?9gn*Mb60PW=y5ex z3g}k)@pXP6u%oC+)o6S9H4r-@iVlHX?PuRf!O$H6Vg*O7uvrW@P z9Tr&OAEJ7?<-R*QQUxFFDX=xoM~M%45HY> zh>@y)|N4IrB!i}~zN6FX&HMl1hyQV%oN+wN7JhS_`{~U0HGh+NG-A(f8H=e%*79 zesDJt&buk@|8!wE40Y-b+0Xe0E+#;Y0Yn>2Gl~89#}5ZL*%x6eMI(6jQdczQA0B&j z+Yi{|+@pX0@Q;rZ78#I8SQ1754^K%3hQg__zdrq^Z~;qZ*yF4=_GSI2r?dxqoL(s3 z{Qv1f1=!=vd$w=(kB<`uZ%rzU-u+J#(SO`vfSx>fCs<$VQYrjT6U%=L7lpy`z`l416u#n1qQGsRA>9nDIf?+({j0g? zHGU{L3w$;NtcOztV6?g%@B--1U9_=RyaKvP_q8K{{&$H-`TyT}2&6mU54wUlsa${{ zv5QPxtq~(-YiJV2n;nu+kfN zI&tIp8-Kb_6RgA12vBayl<@MaAGpa02(~O%78N(D6H>;%kNPg$zo8D#%Nw&X!fKH` zqq{YtU9byQ0M_`hT%&|>02c7$Cr~|J_3Q!>iUL@o%t2Jb&ro@re1nBX+VK9zf5n9T zS7buQzy0_dG(~NJSN-nnh~EXoMBI=aEahBfp#5am2<~g3*$((;Sp&0wmS<3f^@eqZzbeUl(nR$Vb%d>!6t98jn0LcCi*PE zvi$7oS3|Au&u3JCyvD8&{pUJ%SXU6HAQOep-*z{({7qtc{Z@kx3AC5-Y=8%^UI|1~ zRxq4v-UY1EU8Tu95QL9^WlaCgw+r)(+y$``EbCdLfKpjb{TD>pj2SWj1R|=?Spb2I zgCyWN?uP~E*e7EZJ(TZWcp-Ui!T1b{%|j4$kXqWfS9DTYH7rjzcEI=&v$0PJz#xfYkyFhLdvC*?AyQ9zFC)7jb?*j`3<@#$-3(%M| zz`~g+-T^R4mc9<`(*+G&hIp)NrnFgZl~p|ZHToYe0Rcnbs|C3w6T9e(-US)x5(qHpeCbd+8OH?L(X-((klJ?zNL;7R(OV zf^$F=Wzzj&II7Bun*=-Dwujs}NtG6)wYeVTdw?ml0QV2eIu1mjS&Mal}8#e&# zoc$cc7S6^t0Cpx&YLu+%uoBMq44Z8p1bXIyUKfMhBdn)Y;3dkddld7wf_GQrIe(1H zlA^`!!%je@mrU_`1cN06U0N<1-GcRu;e;VXYV7^AIEnh3m-hXkI z+c!dMvwo6){B6<=z@y*M3uKkb!pB|xy>l1(Rq3T@WQeL|9^EXca|C_kbf2;_coBm< zlzA6jC)T1RF;SpHnxu%oA$raopB})znjI#k)Q=4~;Aa8q_KyjiDQQ*HR(hc(L91my zyOiZ&=Lg~7Wv}~isPsbBdqTEAh4Gb|HcnqU?7wg1j@_F9@;Q7y__ShWa4HZW>Wgzl@V91Fs4qyy&?c} z_j_@qa*^X|MKBH08zzi@!)#!{p3S|!lOm8$$Z*lnaw=My66NKW?e=v5R+NnQE)NL% zP(hmYzTjb3Vb<4hUgyxFSZ4=PNDUq%M^NkiunJz{k z7k=4I9`W1qNQ?s(h1s7EB-Of49OlmjmJymIQRKZ5htKfh#V~1u<;97~(!sPW-ORNE z<3V)oA{_6)hlK|<57z89Ta1?|`D@y({XIeNA$7HD^Oqiv`LRA2B|M2k8;m9lgKDfF z)lHj0tH|o}`uhUppH0Jp`}n|7cqC9q!0w@gnjK-Ca7cU}#PrHB0HTmuBwJKe8l#pPdKe8NLmN-6nPr`0kjjZDV2`7y-B zveoq`H_QqJT27~uk2^V#q3)p^Zv)PE8&MWleOzuH&uMll1gH;J%OwhWI8dMO|+-jJyyz zEB0P{P?R^*s!4me0r59M2dEi&RUP6Pe!zATgY=pbt4ID4a@HnJE_Q>?#Pp&E)-BzQQEHh)^&nslUyj3 zi0KHXcSg%2QY^Tw7JjvP2_Q*BCL?D)j!#FXz1jK#vT1D%=qRdFdXd&%Z3;)fpW(~o z)s50h8%)&vNrzP{jk&YFd7lmI?fbD~M409RoCgqReA1o!nbpGDGp7H>BvxLSZXe6H zb5)N?kZ`xUh>&kCC2Kk>#c#&NI&Bek!X3!&_SLnySaOxngs;O`b)C&&clK5h*3MK` z^ofV7)0f>xb`T!lMZSBS1GIBN!o8=M)Ya#}1gb37dd&=a--Jfy@I`!eeo`jxl6T~i zV}$P9XOcJNBHk*Y)jPR0$@>;CP5AxJOBqehJZe+W4Sc>yf*XCPI&4ixD$3`p?~!3s zkV~ucW3=yKjqvTj3&Pwjw1Eix(OtIEyYal*n62~vR7X^H$E<^NZ7}5y3Hs>MpR7)= zzh%UsPr_KQp5zE`@~%I+yUJmAe=jfXsiYmN(kWyAWNC~uYuHG^avc3=&9JWCIAJ0V zf&`WEym~0>{0#5C`H^31*>u9e4_^W^?8!8?-eoArRFXF0US3v;z@zQQVd{w#MlTu2Cwk$hV#iSgZJ$hylx z6Q*R=#g81m_HpTC1+uQHsdeF_NY3Pf&aUlo@rLMnOfDs-%X@(FFKksw2Q7>$kXDb3!QvQN+i;Fe(jUr+R2ybw3TFYYGlriI-$({c-UyjgEsNdDiaPm9mh1dcZHZO3Nf zkFZev8(wo9Bv*vp=3L#0O2C=XcSWY@ND^M!USbOwb5fT3HhgSG&(O%rqLd#;*!jH6 zUKq!~`%#Xh^2?5~ThbeSWnv?BnF^YLC_C$?E+?QM;3ucLq;EKz6BU35^0)I(qHXcd zmgF+H&E4LJH3(%qvbenrSG#?&)iAAnDnfte#=8{0%~^OhH%;|KY*^eV9xt0)SHM*9 zcrAj(%{Nd?X1R=U^xTvD+*AFr_#2eM zMPmdrFEjapjV;kEXFXE(-)blA!<-UtPV;Mv2I3m*cx~5}%_j5@IwI#zcy=SJ1Z`q^uibKsV8*SWUHR}nD?*V9-Dgxni+_yF+sWhw9ImMR^gEOGI`@^5RW-}7Y-CGVr5LCu&)oZ z7eCuO35DJPgMLdUk#{M6=SP+H%dU=h!0&Vy zLr?qY-tRH(ej>Ki_O2+pzNO~Y)~ofiFQv&V`R{|rb~o)OB3aKVuw5+MI%isMxvGy& z-y65-K~Ih%;vh-7$8WCY8e2W!G>5RXKg_y0`?7uJ_Q=PKA?zU|+n%jBeczicQhryk z_r)qM{Nu_zxsftYZ_*~)p0==4&PcF-fAkb0?rM9h1~o!Z<`Q)F-U69oV|#;WIY!r9ry($gdrDMW(DV4ZZhiT@)cslK6m0ieF+@_yM27#hQ0&ps zk=>&(*_ib>>|=k@SqUjDy&m!6I>2-OE|y9|*vW|Ln<}EIlu7hj+0eu99kZie+GTPp zThI+)pM8n$#QS5B>wRZ`V%i^c`ZHwH!yk@Iu?Lbch8vcEs=?~}=bN)jsCztXFxB&oT%G!I z_Vg7jZ-@MyTATHIEG+!I2LmGkNy4lYb+-(=_^7m`lVh&oyAxYIzXs7GdGVBx9+6P1 zIk*9$j13FjW4mzwB@RdV6NE-mo4b@b;z}l+IkpbnF~4Ig6WjZ$E_um)7Gm+={gRPU z`z0ft{J-2Sf=-5KL^A4XM=s03CdsbSXjd^Kcw5+)8qY*qW7ehrzB^0TcV0nooI6y! zApmEV+j`%8Wmf4_=qS3}`!fzEzuz;a+j=nwoRior!}3|D-!r-ci^Y}SWT7p5)tgR~ z7>aM9*X_vEs1!q|{5R0`KvtrAackr=_#Mih8K;6N z{B!Bz?6mx&$N8@02PjR#-Cn`u2O=Y)b?-F;{YeZDGvlw_Mo9@D?7B(i(kIlq`@>bV z4WMf+xjr64YKKF7MZ`YNaY7n9oBd+L#oHw8s}$~jde`q+Q7ATAT>BX1)2S5MXM_0L&7zL`(A`K-35 z_g2iW!tSE4-tp~CxgS%+BdgMXH5rbRvPwDp;0>T;>mw1R9IQ|F+0~DFNu^=)Ace_J&-TvT);~7ycSlQK5$-1+;7RkMnriQ4?p*Y< zAR-w^WOsiGy&no=99aHA`DZD*2Bb&QS4~b&(24SId+uULx)n1ul2$g;yuf58-hO#L z-$M+YSje^WI|yili-~LG@Ts8L2W^9&{Q}x^#&;Z@aPrn&l=W9PhA0OZF``k^$bUJ! zi>nNthR7&$%<1_;B3v5GY@0C3Jul{vFYRVJjJb+_H!wBD8~?6(HCe_M+r(G41I12` zV+&=SVLI>I+Hkm?Mhj&FV1hzRwnnvcGuMQ)4v@si0 zgw>j_c)J`f6)H6-({j%bJcg|UTBlB8)njjLBG;R&CZ?k;4tTCFBFXJCj`Be<3L7xQ z7!llf%c;LVP{)%S4n`AP;w3`sMvgy?jSub>7)4k@9Pf<(qVkcG_{?@t8z$f`ziY-# zI8`rYigd|b zjLE{PD~{jkVk2*a)iykAxaHn4E>#2^mz&q3faO!Ef9^`TDL*P|=14f8)VAqD%W}cq zx%xDcbMVRYIQmcPp9HLLKQ6R#UqbtM=>d|zBTfYg!ZM?Dr5$WnEbMI{o`I0k&?kJ( ztNgk9|J7qnnjiX4TBF$3BQ}*j)E+HuIWS9Ve^1>$qnh6C@;Y>HUZ((~=GQQjyGUx$ zPbvt7%mHy3H_6nJ>Gq)Sg=w!TPuL=UMj-(yap~Xu8rLq=^lFdwi63~c{Yxjq$PO{e ze4C6~3lP?k`cazWj~Qgz`(isLot8K>w`&dvGB>J}@0wiZO4aG4$mBI%h!iLO6|cFw zA&{~=**6l84CH1sd_qO$tH>!Y8F_`o#AfAB`HtBN_O z>^|&N=P-%0~$K&|7(1bF$z0BW2)HefIF0 ztZu>TPPm%Wyqm>1;=7SRze~mkf1mm9rg-C&jm(d~rC@i*TI&9sM zp|Sj^&~{bBY0k-(f2iI=qA~B$)lYw++=VmTUq9SFQZ}dn)O`;2=XQk1|23HW8vk=J zZPm}7c?H1|c|H(*+d95(8cR1p=73i5rl|Cz7D&2cRJ5}29M)E__T;CsrBKZajmk}E z2k;NJ4Bm47dkXR_qv;nzXoT=d*?t83BJ-Llf5*D~+wk*WONQb`!6sBfQXeAiCYu*O z=_Ka5YS^y(+-04T6r7p%o@D*qNp+F9lWMA9kd`?l&#}E^AIw*k#c}A(rp@j8+!cb#64C6zW^G8w<4y9NGGSS)1xVn@5|UFqq9s69N1THZ9BDZSlTm2DnB1xFw?1FSc> zIF+Z}m%%a5j;yOn-!)!Uff7+raClWG>Se!fI7VWWU|we4v0I=9D3^EzG8Ol`@MV=o z>UDgkTg-j+uk#{Fau_yiME(zZZyi-twDt>&pwb;%LPBW-Hb^%}ODiE=0@4lA-Jx`Y zfQTS1<)&fNA>Aq6-EikV?>+b4@s9J|bH4xY7<(}IW3yOmt~sAMpXd37Yc8Y$hBR8z zTQD}m|E&)lrYv`ekyJ8@Dxw_Z8Y2k%7 zuuWBW&x<;P#*_*4ddOvd(KMLlaTs6uSzY1FE((#KyWWyKrtj&w=R+k9$0^!`x598V z$jAs82s~Jq-+gwmnbDsx)0)=0JehB8mmduCQF$Ctlf^`6P79b8dII|K}8n19L_2A0JU>vo52oiJ9J}KR=Cw zRYX9>=@=!5qkf+4FnUJ&D`L(-1e)vV*UGsZJ+0~dM|C((+?&vQD-q_j9oqV; zmP)2Op;YIha^>{*GXszcMNL0Ta?~Pb3+ZeeTEDED4EME^u^_KBKN4fb8}umdqE5&4 zKWE*zh$C0+O|Zf%&%TJNirJd;1v}WKbsQ# ztOTb*D2+HnB3P_MHWf&Se7<}DAbP9_mmdM02l*rt#6HMQR{Md(_;W_w zlPu$uzQ64r4=+8AGHU`?-Y_bLbf@9G8(*7~5iAYt&)(}?6935pjy+O!HQeDgnj^_y zL`e7I4Th=){9Vsgd>lFUZc^>b6_GZh9<>#BeI@tOoZuYbeLh>FG0+=9uYpYLe-0Zv zuP%z-u>x_zTzxODzDmSv&Z-R_BL0_lQ}2HrSOdkYD-M`{0_piZ{h_@KiNp%( z2%;ePx4n$m1L(fi@W^@EVoM~wQ=Hfw8M0?6H?wxH^vocCj79HEm9SS{Kz;cS9yczb z0Po^VvKqy0Z?nsK?mGF?xZ>i1y=AL+^z(J$LjiQcs;~FF!t#FEvpWB_=`&g%TX#Ym z9ZTUg1LqEv{|W-4YFd%g8kb$Dg=LP3oF3=u6JxzDFyWBri`Tj$QnDHSaEfw=*m6oQ zZZC<9$y#>d#=Yd#>e7E6HfBGL4#O6}8YU9ALldKnf2}zd+-$zb>Gjck|6>=ceg!Kcd#Y@gNBX%^(|p zI`z8@f)#d?41kS)_pGd#I}Gdl)2a?O<#kQU>cS#WN8U9o%5 zLp{AUzeg=N&nlXt>^T#f@*95^mBWuA;MD!wa23uJrxnj?<=JPl$J|%|7C*z_3IoWA zWrK~S6`@I$PE8=u}~0y{2Q7)E=YJ#&CA0@yF}amLh90>qVF8mP(W; zuu6rw9u4wOKgo?H;)SlArDUd^1Tqu0{(Ce)vUF4(P(OC2?-i)w9#DE(A@#=x91XP4 z*h$X$#XV{&YK+Mp`=E)QAG5&w?rM07ypilBL-Cu_= znV|C1@z~WTBE?@qgGh>}92}1(YQWP5RZ|G2P)9uJD>b3nXJ~79GQu9rqVK9ni{E>> zo~4g%@$G;%%Xm*rFF>CzPgct>(*G9 z^+oKVZX{QLG;82`+r_mPEUB-$@+&Q^TZblT=2-3SDunP^mC(%}eTDba6^A)2bp;5= z3KjL4f2Ol0#BlXz<#^Gz#ZFbXhbFDjy5Yerdvi6$nAK{_87g+TN6Q-MS_}qFS^I)G zjW16Ik#C9MS@j~!Wb@RfoAvzP+ucbgY;B#a-*2u^Xq&e=-jZ53M1>e2Dai^vbe4O# z5OQQQUW>Di9x3TnYc5h^CbF9|Hu&}}#{R*whNFG43HLw3MCX4G6Mxn<=N3~6*pB>- z=9)ilp;M zVe{y7Pi~iZ^LZo^eV#nZ;G{N4_RVE58(xdjv{o07I{wz+7Y;KTGA#2`jUM7=SvZCh z4Y$MV3_*!6iE)~bjagT@wRzt$>w32NLDH>v>kQpn%JuX8#UnYaNb~W@oT28pB-61% z5q-mkhv*UylZLaqnU+-rir ztXP#?{kafGUSk2FJ=~bjyP9h}Z!!{y6dqrq zK{KJn-}3cM7w!IjmR2|I3c+Rv%>1H{$rt@W7I*z z8I$PKiEm2FbvZF#V>)tLGC6x_^kjdJVri8hMOC~O>ugRQ1c-85U!0stm>Uio@9;YN z<81HGiDP5liK=d?Iwi1}%!yc2tev$X4i z{7Oh0C3P!u&^oSSykOPipM{q8>X^px@&=*(NGw(W9*U(Nh#JY;mG5Uh=9|bAMqS)l zzm{ul;*|cTw|L`+-7Ju*tbXYBc>NW#gz!_0l?}Nx#ikuB`p6Lblf*nkO^=)l(WB?- z*VA4P5luRT-^`93^%kBA6%uLbJ^AyFC!N#8U5Im&1a=NYJm__b1f4cTpL zi8puMkg&@%YqUB2{A$GouH~0bB-D){rG14>O;^qVwMDjh@Ph>58f0B>zt2MCFKN;I z5!Uz8#y;Y!XN~INcWI%IRJ)QSx^Lt4)C@s9(h@w$!J(KOj8Vl?i%)oI1&o7NA{2P$ zUU}Yw{;~27z-lddZ0YkcvI3bjB^LgYSI>y@Iq0p}7T&#!Wu-NJ#N2~qH`(DezpZzU zzQAX;8I{g&@~8TUnkf@d5JGlM=_yOzkqQv!8wU*T0|n8ZuR2=z!@v63!g|ZpBnzEz zI`i)mdT%3C3&~s(vN z`PBPowoM}PfS*@>>n{c(oDLv&%TYFWzgQo?kB|#*;`21S7JEYR*3D(KaF|mh-8DCg z8-4ReC;=X|%s1$+r~-ec>mL37C|yI`~Gx9l2P zE0>mzgYimyQ>dK^G)?(S#cnq%lv({`rwL}U4$pNjpT%!wZZ!_ECWH^g=A}mx<>S3d zfxQ(ES?}H=+mvCe%Y0zn3<$8~-UqXZ@b+YdD;vqiW>+XtA9tZ|$Wt^Noa^6uFDuN}ap!Eo9{0B9)O zT8X7;(4SDB(CWDb%de=H1IQDhsT)sI(N!Zb^CZqk5;(!gr!JTlX48idAV8%3)yA_= ztX&I}*$~)C$(nbhC=tWZZ)a}UMv>i|wXNvPEd0`QkGEsM60KjM|7DBNRFqp!u;YU} zAauT0@-65g8poJiXuH^64zmS36gBDRGnL(reGm>}I1$1mzKxfAe%*&N#z43Q@;s6I znEuKS+!l-Q+a|MEQMHUiEmn%Y@DRbj)P`zx@3|uHZ|^St*E$VGbi#K}?}7gO_`uwr zU5Ef-zJDV?3UVE#jAfSNG9=g=&dc*npgHB z{<`vO>C;2&%ueA<`qQQ#_Q!9sd)8SQvSyuq)k6HQsQ)-7WBw%+B21OHbZJn{K0Kf-87 zDA!-66!(-@(Pz4S6`5;xwf#k#MW-;o;bJ3vNOk0|@dkDk?gZ0;dDi~$yey-)KDM0T z10vfbL9=ZD;XtbsO>L6le!qIIO0=`l{9rNeVXJ*&LH*#x4d_(5zxZpmb?MJm^qdjr zXqKCtVb$_n=67pPF#qAwXI^Y22LKM+sEjn#NlJsLc)JuU$!nX<7~3jlMen=WS8y=D z^Fe#p89r|jW@dc8B2ws)f5;SQXwhRfv+gmu{)dzFn?m)9mOb(L+$`rVHZxS!Xwz}O zeG#JlLu9bX^kBKwv3COLI{Jy08b$V;AKocEx>89!sp6xTX8G~xsa9L&@6r`Hv!JcU zm4gWD60uujE8G~ei+)W*LT~Gmrne)LmEE4y^i_@!%NfuEpZ0g7{rT%@pKj{B;jAaU z926aCO}7|R35ey-M2-Lg+Tm%aUL1esU9!FE$l>8-;>v#YpfM!^61b|_mIxPeeQL`R zC7_q;YWV3{E9y5B&D$r&=x`E7A)e0yLjj$~Qfo9Pj?OAns1cTUW>z;w6Dgw}$sW3& zSNlKaR|pkTF$ioebJx@<9B9_f?b@uz_4T=`zb;$$6I?j?#)IYFaGbHPGrJ+hL+bIY zw?$P2TQdUTRZzUKxn(MNhABnD@C2j9lH7J)U~W^f!Q(U$$3}&rR8r%zqq>vT~!vzP3&@YSicjFe7>f3J@GzMhXP!<%WGHs#%q4L+*7?6=iR6P^!a^&QiVd}`6VNia&;8Gm&tgg6h> zWH}6Jy%t4*OJ^6|!_#JBDa2t%_=Mr*_gt6EKcdG*4hhX-2)ln>}VH`YMB)p9%-|4^e%3H5# zs2=`{54-(C4ig3PI@Vfp)J6Bzg*a8#DV9ATXd3j6u()hlh?)*k$L=Bv?htX45~9{e zZnay*|KdX6>g;HUYx!4{l(V44AC|X89F){9hwt3(1N5}yMK|5jb;ZN4X++?PevJ6a zX@Zg$^;M{$zW?HbDJvRWOx?1{j`-*Q=HqvWA4_cs^K^Fkci%+(V`{XnSgsem6mUyemF=_18X$0qoSf?TcHKd=XSZZ^PhZK0`anC?D*dxruV=37{E}NB*p>mLtvhH^FO*ejUBQd zFk)GYVgb1dhA;-{@*Q!{=ZTFKPJro&xw(YCNYj+of&~1)z0-i+@+EAQV3w%!}afu zk~)A%2Z9V&4bjdF<~u~#lP@EPP~4LS0Y-q!^>mGp)3pzgvfdnoh_KkWRHhNdc+9SR zVD&l9%t4SmmWXXUSIBtgdl0PmWx#R4+`L=&xhtpzr<@75zC8Tcni%hs+>R(4r`%<& z!B6V;z1s5*s!7jgoEg#(ezYB}t>X3z*Yh>CGjn^~)s~y>w|JW2yLLWKQX(VW^jSJL zfUm{EJ!{u?z_a!Aon9_JxnN_j+s6Z$TYSGuy6k$9h1G)Dh~_+0_L6(x=A>4Y;-SE# zR3DAn$LXhXy})`E>%Nu;i||G5BlZKCntn$IiM|f7XQAytxVS8rxHecG2GR=}=DiuH zX!yDZ{|4~Whax~VG@L+@H|GN3t8jB#>Vp;Jdvy9a#?$1mEJUc7b&z1;9O3Z9i^$na zJy)4K09US(sR_re896+PXP)=(<#vcW zFV@{L>%Lp3KzMG9aD8pZOB)qWz&|6e%>eAUZx@$?K6=GKh^E6a^Eh33{W1RN4KMJ& z3P;#xwaEC2IgCes;nJeCAQ@yj@2|WHpuEYhMfCM9pIr@w5CevxY&np{2yPmcl(`=* z5a!z92SB#)Cx91GXread0$SRHh)Qbh83H2@TYSWpEGK*g+dyQw%VId6lzCTyV2+v< z;P*ro+ylx#UQa?GVv0n78Sv~^XLfsyixsdOf0+r)7skKovkIi(BH$}xe}>jF-C5a- zspQKkY}&HrNw0)8{ZkUkA`L*b1w0t|eEwSnBSd0l9^B zBH*fa9RbW3JUS4+{o~jS@fq^|gU!Xb4dEGP^8)jZZ9wGyX7NRfW@Lr;jxi+}xz?S> zuQl67-RGzG*FN{Qbk`R^@3=NQ!shDc+H8@gJk#d6?04`M ziQ(u6#xK&1lF6umBb0nOoYc|;2SJ3SfIl16UK1Ymf~!<%Dcss_19>a?#x_;yo?1R+!2dWRr8 z#&04Do&Nh@x^d0b))M2Yy&jrItXRj*9!9&e09H=HHjD1xzXu>S2u$?eY_>e^dvo!V^nATRmpc4e;cJ+*$2NIb;mYoC`SQW7S z*l{Jj1L;2#q(!C*&OdeC1iPYZ0Xa#`L1^uuEC-zJWxTsJ-6@edf0g0 zz&1RJPtTnn!O z-?szQvf{ZOr#lVHb^yL$hJYXgt@*+}sB@T%Tu?A%KyZ@PaIUX7K3#2}9A+f+Gc?c0 z?d16lUn43a=;ROw|3gb)cgLo#WtyOaZ{6hgKngj22!R`9rYlqEeP?t}m@{4q zqp?A)2EcZ#n9j?MKxpOZAgbXxDCJZ<%#U@(WjKdWDXJ~^h)WuFOalIsy1|@Oj%6_3 zQCP(;h`}eKiDq?1?4Q5-X#*|R2zszGkS{YQD8Y;pY`6{F*Iu|%|FedaosNiJ8ZwCi z)Da09#nt6RnzMp~OqtS8Xj)6&o+H<4ahqtFw;-%rmKqcNhixDd#E&kT zfVoKr-~>Z@@M#@c`xxLz^( zgl)Twf@J5yDPS56Ik~nHi9MFd_^><3cGhJOJXNN7#!X@_v_n~mV6*x&eM1pHMT!XvX)v* zBMMtVaWsw0()!|E(8JTf0Jo7rKOy@Mojn00+Z>|EmK zBP8YTdFdfJn*ICNOg)4J&=EHFees&+kAhdc@MZOC1_qU8`78RNUwyyN&|=*Cj9V)e zE;(%a(QX6#8yqYWXUEp%Tz zEo}?m3I){cYKOqD`$9)u&nw|&OvqO>4~^-XVW_v*cnYwt*j0BgO@`AOM$#UCCu za~Kl9UM=S*yOFGTwRk1tD*DR)=!U6d_TIz1QolxWgi2t*LDPm5^Gzy>X)F$2IXR{l zo5xhIcrE@Ud)K#2!|yGaN_UU?R+w zB}eDyY!z_Ap7z`|-ckrPU(1C+QM-NO(q>=Gq6GgyxXLXFJl+B^#w%Oj#1wS}ms8RB z!6E$1B@J|L7N0-fY#!wa`W4#?(EaCoo3ugc;vE@XEGWh9vIsL=9pEPSfhtZXQ|N|G z2*gTpkovKj3ctw`-LtX^cNJ0`{^EjCA0L(RXWQOr1soarZ2Ng5za*bed$ec<<OGdnNJhb zZY1m(VqW(&SI~E@wI!^nGIgt--eMkRhgJ@um4EIXXt@}U)CUF>S7sOqAK~sE(ycJk ziW|2_^c3mNZ(v8Ar8{M}o_B!x^!+P(^W)VRlh_B2gz;UdQuK6RAQSz@!lqqEx0W`- zet(w0UiFc8AE?VNEdyyn{fo@2kP*ri)_DBl9a>mUe}YMu9A4U>wVNtu$ZOQ|O%1}y z*)Lw;Ze73ZYaX3Ozk4#?Xdjs;gv80q-f4JB=t#0{<^Zq)PZh}Iw5QdRMUL*Q1KGPF zEMeq$5&Jl4Yk7y|S95Q&sAvbr3d8S#%u2djq2`WspGhJNr;~%Ul{iV6uIYf zpmiMgC_Q{SO8R_%b!Q)XnsHu>o7#;r(XtT`@{5E}&xxNqx!Mb~Y&PA?1p=%AcI5a5 zc<7cgaRaf9>>JY=kO2`8wtCt=*DI9&bgzTk+M>>?s>m>|PJt z+zw@FWSx5=+KW)C44gN;F_9UGSL3SNpP zl}t4sJa`~4q2U85lP+|~+8w1e-5}NJLo=bR2>q%(>d2fvW_!C7bYQ)xd1@~VpZ$p3?KkMY4(S(tMmd-~xK`4Kw0xjq|LJy5>c18Y zdr8$L)gaqoc^(Cb-)&Df9skTw@dY$atf;U?wkZq=Xw}a1uR`k;#NT)Fa|y%=u6G>s?QTG2%BHA*_>cz5hnEWndl?Bo*4Me11-?k5&W#C_)d@XTFIrBCk3{jfT} z`aI*>!uH>oC4pd0!qfTW*ZDaPRBvt6CWEk6ML11V%9VSlNpzi5F8*enr!5_fex9fUAUU$gwJ926FKV*tp= zTkPwnSivGBmtrlSf2>r8_Z6HkK(AEcsPpb1m9qIY!(8$sY7b+=bn(M8ZySxHA`RXz zzV#DT07?62yeucX{9Q2Mi2&HglYE=4uYd7qVLmo8Jbeu+--s2%&w25}J0bunPCouR zR73FcT=QZ@bpfiW^_}I6^Qpy=L1$SPr}Z#*Q{H-p%^NGnY{id4E31BjwZYtK&nl|=*rNLS3?t;>G<%+c z`(a=55;HmBZA292&Ie3C8i9!r_Dh3~mjNw}66mj|wr>kGhK;*&*`zez$^Dj~ENgj8 z#zLy;+K+n6Z_JuEfEM6F5?ltSMhzjls&nTyaeq=AkJnS#9cAg2He&K-^F6mbtx(ZC zLK295yvpMvpo!XaK>fTEE+rNFMk-1ggh6ra2hlTLBxtnZ4XcED^p>X$eLLrBL|&wF z@5`SDjm!ll~ikk#nUz-@*tC$T{Zrp=?vHbf!$=~G%0p$|) z`}&T->?LjKQ|c|b1*Gr9diiK+*fEjTYWu8V_F}=shB5-&*urW>=;RnE2Lq(~LszaI= zR*@Rh4<4}55x#*&68hR(9|!md&Zg*KGpj>)++cjDue(Q}Hi6~F8e^t&>#!k(KaPC{rd;po8?ox~cq2Xzyk48v7{hMc* z2dKfCECccHth$W+#r67*Gb;?<;p%`K#rAEcP&{&^jdPLq6s;W&)Ib?MO}mJ#Rx7-hW?2Y z%hI}C33i_cw7q6wa^A3>_`$)7LSYW{|-+lz(b7Mw1^tImSbKW9k8Bi zIgDiYGShly{mHQPF&T5+VYOkbxSQ%gOwP!F(tHTi#6JBe_mThwNdT3g;^*27G_q4R zTu2RdM1=CX>TYe8V0kNE0LJbEWJ;JByDtsjs`Dh@%Ka5UpBbA%WlJQ7@$7G z-J8V=8}gxvEL2`L9I6nG+dY{op7u{+M4FnHGB?vgGGm5pDw#Vv1{gQd40=eU8QDDp zMrxutFGatb@rJm!K%dZ~JIXZ|K3rwVv3S~Mu<R?l- z!PrIcyjXMa=X(=Re+|0%8-^J-B@)9<*w>et((MyN`&0DImEi<5yU%aEVWbfP2CU+w zSM1X+b#{3F)B;dk8#%f6q_5)h5<%Aq>_XzTlDVRO;IZ~s=A&eyev3VjRzi6c+dsh< z%+V@~AHa2N&lX>>Np6nQS2$jq~X(4cu-61?mUy`MvZOh{uIuh)o}lm+N&MA2S!+{&w@yT1$V-8>4?2 zWD{Wj{WZ#mAm1p7Uo@8La(Q0D5Q636(Bw8|4Cot5rI`<(tY}~jM1Dm8_gKl3VPjv}YWgN~R@ZDn^0rc1e&z;0ax%TG>T(4y>un~`XOl94U^T|#7ZeW`eg`39gu5J^*b{_^h;Kl4ljHO4-L zCB5opoA2NAGzWTaMYr|Tw{Bu&C!BVb3B5X{*6z^HToOE;f27MRlDN6|vTm5{t{AU8 z*zEZR(x@YBFJ>M0)4Z>OxVdfcZ@hMp7D1ZC}Ssl9#H3w!Vb+mJ9e@77;Xnj`M79G+~KXHSaGdOPL*b3;d0hQ2pJ;y(MI7!n|9F_A8fWkx17&hzHHH z7G7WWI*%*H4lZBVh3pnUK!bm{o zewWc%B5;e7-(-Gtxxq0uqCOwI)V7%Vu#7sfFvw&DL{Z2*a7sjkqo1|ZhPochz$LN^ z`&Nps%y{<*WITpt>4P?SJWGq7Z`iTRlvYoFe39@1^;H5EwlA-L3M;*Rcv-h}^pOey zEP%c~V0Lo2G_{$%+fAS5VWoKUS`~*$zg!q*63z%&!5Tr|VQ3fphh3x;uejS`sk@Pt zIrg_asE4Fg*Kz`s_b(O8z(E9b$dX;EWAo##Lr&CYy=+8F6!+)5m-!1)gMOUPIKr2F zz#8C04q+rsqNLW?$z zG&0#QQ0m)&HfLidPj=axx#AQ(q;MV*Jzd6b#=oi-?hMK{$q6J9Z6H5Ae>=QWt#V{J zFQ1<7YOqvCfuXlsE%eT1KDL92fwRO2VmxGY!^@WQaQ9Z4o2uL)T*G84@433$)y>o| zuP4h6ADcA{rSau|)-3>?m0_ZP$d(*>k(=FNA4XIg`LeGP z0~>N~m)sq8TIemZB1O)wS%}+tPe&p3Xj=90oKBKu74*}UW#6)^2=xJT-!g2uBP%?3 zabyYR51kiayu7_?O!lShY?qVYB8C3Nqx|Z0sP$N){e3N?{CdL5YqRVq=x6F)nZA4a z%$iY>$dM|azF(@=PhG>!n2pu;&6m&?eFM)~9Fu1GJU@BaN@vU3@2#t(VmGJcjYAGE*8%cP359E$0NSXVt3N$UyT!07w6B%c?QesR#NH~3fMK#iO4FgmC_ zlfgBz8qqxjC=15nl?KHO>Z1OHmq*Ad8GWK1_8&?^Pa}o-u@s4{_1*;p6y`@zNWF~V zy_noORypYh#h`K+R(=}Vf$v7pfleUNi`SSO;XwmgHQTPtZ=xgU65@tQW_uBm6W)?|UUjKI1xhpqWF8i zv%~2;?2KP{rKe(tviaLLU*ARAwu78JRgfhBah3;y?GL%`24vW zSX*iu^w>lv)i0h0ei-1JQ z(+Hoo&Dy0Ivt+Vbc$tl_P4X&~7+pAF8mNeOCWOCovo`KJkP`JN_8J6&3U5|nsgCG; z0tH?icGW%V1v2!vwMUjkUk=z`vC;Ns-JfLc9)9HKbXYa?LPr2E$jaX6FfPd9;JkI1cKI8_EH72>y%4XQN*0c!2vav4Ig8fu_BC1<3 z|9t2=di=vJU((Z-n>q`STJ-<)2)QF0c*d*=X`78k#4PcUtE}ZXiQa6Pd0!uR9R83w zd+zMKdPP`yzTknOogue695Vc(nm?S__9B?l7%da+#aa@7LA9<@wycE(!Qi*<)F9FD zwxDa5=ccpZgZG?M9K-q=$Z82B}S8nd=iNo z6Rbtic-^6LAN=ClI;G>>N^=@gy%pk*G|K!MOP~`egI0aPEUkp z3Gw{%MvX|BJX6(=ta@km-(6}8*aFqdReE;6^q7? ze;|k{yx&%nQ}8CpV5ol%H>5@Qi(3tD2AO7Sg*mh&%cIyQmic*S|H7H;wk@xn&=%iz zOk&ACWqIl9ZAH0VS+znB4JCJ)k z{V;;O^^{#xal2vJdfzp3#QEhhkfPEi{;-ptC|qnUT#!;yXa)0QI#2)Xqm9=;OR;W1 zV4K}qo_@zg>n)k&)2Yt%6Evpt&?W@|Z&fyK|NIl^5|bMwx0OF=Me*XTml?sDF^UDD za-c)}`PZHUMfHez!-?#Eld&HS4qe+n70#_KZt3e8zotM3*G?tcy4pcIRf=OUC;mLy zbRHbg5|L)0UOrc}_ReerXA`ElvdLuPJQpN_sbBGEY2?b|K~Yt>@x|juMvS?!#{`;` z!JW+K({4v`G3nw1M>5}~)nBv($jQ<7o5{g5^hIzB)k-FR6jcYaBx+52{Ca~Fm6PBR zEz}BjdE1N4E$He(TL(Hc-i#4yOLOKfk;o`L8U6L#5XWP(`C|^Z;WaNe@x8DB;d=!( zug;=AQiZMW?o?OZE*cW!W#{7s)EJTv1JnaA{sIckZR`1{h`xvlOPRx;kh?><4M1|# z?_^J0ZPlnC9Tq3G|l=-*LDb`dg8Kwsz+=;C&gEW^1Sa$qsBmPYM?g<;?bDQ8ipzO3k!!OpA*#I zGrnjGNU4q$v$P=>7jL@iC8Oz=iLL)I$KAuS4%Ky?eQ|av^TgSUYoE&al+S1-UFYsb zQ)t3=JzBS9CM`aZc-8MyuO{QE=XLpapO|kxF7*c#V|azVzv5k-9#&cN(m;cMYDaY> zz>xLSx}Z%)1OZ)uTO|1DL(=8bRKoS*;5G~2g!8~pePK9kou2)rvHcV)H8X-1gti}i zguMm{%$bfay1oe`dEpIwd5x>-Joav-t!RPRrCu@QEv?=+{nQ?K(j&clK{iI{txECr z;pdNA^8zA0g0u;g)&zdS0H!-e-@lnrG!VNQwfCC1ZQiJqy^MTa&ukQ~XSi-496c9s zSRncMK=F&-{cN7dFGuWaJ}3xd)3!jM`_cR$;~{|!-F!7ZZGwx=j52?&m7%!btA^KO z9?qS5oYqWIJ+YFH%1!^)MUN2L{k10tv*l5RAnuYQf?Ktqb`NF}-8}+4Q~PQkuk8zY zga%3#mx(?0akxacub6)eX6p~ZG<|Ar_3G%n z{}PnH_qommFNqwiHB9WWnj)Xc^I^SXO@FrO)kSkZ{~6>fKt4e~m<3j-{GCRD&73}J z3~6gf%Piv;!^u;rU<#K>zq}0ar;$N~MVxyhiaCMOi0%R1neBV@t#M&5`^ZUrtI(54WGjGXcp9FT>KK&{n@j zY_s@D(Jrq`;m_5h9v#zePiWbm8P=UQT+}bO!CAS0qJyQ?(ig*Id|Q;$A}>`7EoihK zdDsF)s$8jcKXU~{LaR$2o;E+#e{p$7*2$%<`%)qi@BMUEBaT5nmRv#l1@u#i@KpI) za_7EbB(2Lcue&7CxaHZkffHv0_Nx*uK{j!H$>c>_RI!N>_SguU>b~LXbm3-lFr+zV zWYbjyaIgg%!+A0<>_hW8^Bt^SmEU?^oiiT^Yivt+l6tYV)($&y72^I7KC!X6X`5V7 zUqT2Vm&}`D=x0if_$-aS&db!K7uPCxAoYsI?ER5YDbqYn{uB6o3|nLs@JkRoK52bG zv%zf@6rV9C0C&~>Y`T7s6{+DeYv0LUZSwo&tHVz^>4oNMowOh~u}6k2W*+Fs^zgR- z4%7VUtAfG@jI08ZoGzeCKEjR91Nwj-DC1^=ydfTXXX{PJv*I?;Zly}#UB`UepyRd_ zgv{&)s*#554|-!@(BVXLTrSo*oqvZ0Y=dZ zN5pnHilDL7(h(tNK$nS$iAPt#D@Ro~L>nVtK#ixPN#%6&mqEpmWi47c&3Cfixn4g`+&BZ*pA(V>;{$KlBjV36H zyhiAoXOoCp1YH%1%DY~AEpldoWo)0W!7iv)YIL~b$f#k=-**4ar%)~ZdT5rnJ-?Oj znj7E=ayPNMa&T;I`MxzSZdfjX^}_N~_?jA#jJ)%_1TRH6TzYHl;l?XSO&X`Y?)N_z z)4jW{YEwfYM!41MLGRXy`5vo7en5DpuT5^`yp6llQypHbI@(dTbd-LtUIBQD5jsEw zsFKgl+Mak?0V@gIc(D-#yNLAxWNQQ>7Q>1nlKPJ%{Maypx5XJ&86VVPswknie2Kd6 zYvPB_Z$@llmtokxcH!JvqU`b=<3%kX)Nc=S?H$K);Ul9+a_uvT;`JKLMgqRL|0_Ma zKoloW1`bqc2)nGPy)S+&Z=6%8(a} zFLG5*UgpDZdfgAU=s@1(E54v~9?`PaNV!ykc=ESfvVW5E*R-ghC@) zpz~#JbHi)e0$mT>Cv;tDEi5h6G~Q=fMWQu?*v|==c@eucF%B5gL4G|oo?m5kWd!2x zErinK+s3LZF9g)H0#H)p6Z6i(*$L4hnaS5W00Z|w+B@s0s=jvLOGty#Al)E{gl5ll<1C+WFGB-^H{jW~&hnxeEH$qcpDs$D zvtU6&PGd-66- z*#DW&wfd((I-5lZTUsO4jBxt!l$p&Vg;s73)Ae>%S`E^NfjW{zkDU(lqr>2G@C=Mp1feUMs34a; zA=$!+R@Ir>zW>y&%CP&;&vYI55fG2?0t*2;c03`lk069b%r0#U*7j>&*S@ZtxU>55 z5+>9^aAzV!@CqrQ!`4}!wmOfzJ^>08if0#Ox+HC6<@_|K0&COelgrbkv2Hv@|UKb36cEZ-dbw{gh=PGVaSDL{ZmHg8=`i!E$M5-9#u3bcsN^OEZ@qG6eOL) z>V_arIeQl}yIJ3=iQUJ9l1&mnIr~A<v5o|%bu_<%m{naJs*3Yab)XW8foGy*nUe~{c%-i+1%lb$m{gbWy%1; ztH&ejpK1xiY$oYeSD_CN>;;Y z4o>HP_+Z5y=aYbVK>OX?;}gi0Y-cfoNU)Fl>5L!$2;fJ#{!NqEU^WhHDg%`prn@8| z+;)Mj5n~e(pGWUI#MPDIM2?UJfAD%LLJ+*T*PNF*czG+o$YRq?FA=Bh3_S>WEx-Bh zvh`Pkd}pFfarmt&mhkffdpR~mR^cCa`Z;oJ^3IE}^M_K|d-D)iysR2UH@{z>+h{vA zXmMa<{H}H-Y)lRILlj6Drv9Yw-ZZd?V$nxW3C&mKiI5lRf8ZuJ^{Pedlx#3vZpAqD zCrz6@1*@Q7VwwPs<%S|2gU{H}J##{N{Slv|a3qyExK^p4UZ=R`Um4`gO4YlhIK1)T z8V1k1j+0%CxxX&Ipl<@_zt zdX@bk&su>^?~u0bS{P6K*RdjbnP3vn(F*+t7ni9dy?&j4B*e)${`kx-WM4zYFd^4+ z_WkJt8B zs_xHQ5kR_5*ylBs|Lj`10)0DlX{}{2dub#W(B>P8dm{qdPZOKvsb;U#Ru4dnDh4nPoAWOl6T`P|7^$p1MUW#tXVO)bDx+`TYh3l>pP&-yySb+qugQ3VdUl zAN6$p2{-3nqkPE`0dr^qu4k*%Hc=}X3f)ZVH0~eMhmtgsd7#>W!oJ-*ppq2a$2V+| zrII06Bc&SCM>00{VLKBk3pXf zFkfkv4HuknadYVj9~s0$VzwB=#mWB8hQo7J>|NndeY@7~(Xl~EepNrS%sX6dSo4x= zIUCL7#0}@>A(0+g*9Me~tRgCbJNDGu39V?XQF-VosGIq4Gh*lw_ zX5m=)Sj(0j>5$-3Lc({M#!wBaZc-1M>Hz~%I`}Ns z5N}tOLq>?1?J&AU@i;tv1b~zwN%!hr&ah^Bo#wMhH;1QU-YS!Npu#A+GWk07^>Sj> zx9*F-(IrT81=#k*%d02IeWCYyUwe3yl*#Tknsv5%%n&3-#9|Ws#TxtGVO(&D6rF|Y zM$9b6j+h!a;p~UJJg5>p`6+?U+##0?H8#&Ma9HXaF`ZQZ*K~5cH(-nhyveul*5mK( zUoWQse!CnpZ4|AS-zgM)pu*fv!{nL#>Mzt)?-_E@rtxR?r;L?Kx@3{>kVSFn+8<}A z;|r%?bOgn*P|JOSe^+bz6uhi09sLeh2p6v|_g_P)i2bYB%uH)~E;?k6+tn1TcJ)C_ z*+tB!X8JkyzrSn$^>yh~z6gA4=@tc^81bSZyj(UAUkIp{{y@jCf3)GB{BR}l8SF{V zkhb*fzs8vLCJ@_XmQ0y99J0u)+z4}<1L`US`PU(DV;}uc8V0=bFwlyiFqgS<4vu*M zgSX5Pw&*#Y3`+-`I~*zanF_CvFQqt)HgHo?xVByc6pwNtLxR#`dH+}e_X@f3wR>Vt zmL-7o{)vVKzsf}e1bH;O(B>Y&eN=>7&r%q5SY^P`)JpZtx3BLIx8P7><;&#n9a zv5Uo6B-GD1{#GB;BCF7r{&>lzQd7*y_BXnj7P%6T-~}C7DSziK8l^`pjJvn%{?5Vw zbK?KgOaIdq|5v-BLg5#}?z9XDOcC+b4TqXQ3K6(T@B^=dj~$`5o-2Q;dHqFnbGY4h z@-@8$4y4Vj2o;)T#Qwy);*98^f0$^#vmAtb-((@=WoBv8WBUAkwmPjyPlfw=O_S2? zS^j#=e!lvp`{zpI5G-JuYk;$S@FIY5J3zppZ3SXU`GD~~-QK!ZkQ7GnlfjB!N70qE zw+bM+hd^4uFHGBnTJPF8fd@|8Il= z=*=&5QH;($Al)fMhM{sP2z+pY5s~E8~zKpgtLPpzXwx8dcV?08N6xSM6Wipf)Rw_(jB?WWMv#NBaA$u1<`lQ z=^ybPD=9I!G;V7L17Fqrvj$;y-RqgjAZ*T>^qPYo*uemvoEWb-itiFOvBKl3Oc)uA z#6W=P9AUyo-CXLopms`Q?yRc%3H5;wIuT;@gMp>1h>)`Yz>I@O6@y>Lub6ed>x~DGFkt{e}w4VX3FNkwaq5ZWl`dq^wA6?4LH^ zwd)9i+5qf?Y&(V`Y*c>@tpZru#t!US&3kQCm?CR;i8KQppC2~~O{6KzSjN4Hm%o6+ zZ1-}QOM~H%z}W9sdXC6h{ozv0b%Q@ykk+vgc%y`n(MHZKz|%zx0lKX9Z#NFBJmTt| zk#7Jw$yk)V=@>x(0uGM%$OwbmGVmWU!f5cxM-)8%bk~{DRP`j2C`5k9g#?kGZeucM z`K?-q^>GJCzHw6eBfF*mKHD)UXAXLq^a9lP1p)4y-%9XwOeGi>GS#wQ+@yleg}n zAPEL#KA+U$b*tuo;t5c9>`qcypO8Dk`uALcuMf$i z69yK^=Fg{1)AW3X);^5N^F1N!n*;ZdZcKZ)9m#G8MDK!f^6{Tdt-j>`A5uk@BtrBZ0*h1=@%7<}9C zggE$Hpf4R^{F^ItJV6}B^Lk*%TUZ?-zQ|{fDBj1~?l$hWXwN~2z3@G3QCaFy^s@Jz z(S0YUG|!rK>|Mh=@6Dll-1;s4`{^r z7Y8jUg{ufHjkaEiPTc;4cV)f@luPXrU-5xE&Vj#NZ)h3$3bOh_IwOlC&W{V?jRgK@ z5Ox;m=Qr2gSQE(~C>8h-I$QQK-y2Plv*5VYCntu?{1gao9>)oVA%SgF_q0HRxPGs$ zkZE+|6Go{;Z%L{#n|t^V z6Xy;#WIKJ#k#sRocF*RB(fPczKVIT>+Fo3U4c2`9qP`lRC^WOOvuNOKIEPF&QWem&t)7a3Qw-pA5QE?57ZtoEU zXKd$H3SuOJN8p7w1&+p3U6R)U(|VKaZ5wzFHDUO^wajAmve$hM2cX`;eF^L+Q`XR& z#i1>1)v}}SpkElr+Pg1P+a^b-aXxUD#@)*1XF6 zF-Xg+!}1!VL&ERD2%S=|)crFw)1R(;6f1IS?5R4Xn64vm#ms&xr?Izcm$3>Q!Dp6j zR`{$b&kBz+Mq2#+v3*Q#$x!xkf`oozM^|gzhBPH=|oM?ViBw|8Lru21_ zWp~+a)p*HY>i$~M&JeIlV5#qRAtYaR^mTqmKZ=?0T@&IU@wysK%z#hCH-HuQ2Ovef7VDJ65C~!XKuwsOr)8(O7 z68|AL8|S$400_bej}gZx$s0^sT6X_nm4zxXvl+f0Jykx<`AxP<>T8G3x=5$|LWPa# z$>c4#Z`5+W#EO#@?~p%Uv?WfpRpA8g8^hGX+t4Abjda+r%hKb4I=aH}hS9<4Tll`I z7OTXvYO`_oBMmS^HyC3q1v%A@e8|AjAo2oW{mHSqx^et~)jC&D3$;cNv-xC9Ki$o25D`IYlihVQs^jWNFa1t#udL4C{UEQu$k6 z7oHR_)!P4HSki8RlcxZ$u&;9YJJ=<5%ekLw_E-92^JR+%_XOJtS^VYTn-@|gHV27+5w@Ir&b_D2p;@XH`!>jgz!9# zX_ALQ9U~xtEuS#cle=7h=nitpQrHYh7m4X*`bWhb$H&uWgaYAlxzS>J=WkFxqtL{2 zo||F6GCw*z3LYcqlSlg-AShiaNS)r&p&(pkjxaVW*osePxL?cJhU@QmVi`5fjwnwW$**lUVXa7rf6mj?P7b7hxBvAlk8^4HkJu zN{9Bu@l|iqyt<8^(ve?AZ-orZ6xrxU&^Ogbvm_oaJoidr`HH=dAyZZPPvJD6B9Mbn zF9$w;T{FMrBJ2uTk&6ZeLZxg`+%IP>`i)XT!md?9A4YLm8@4 zhxqcOtM-DMBdRj&vFAOG7$KUFrYD{GoGk17EFErPN`Gb(XlABb1Dqc()_xZY6sIIu z;dayJY>0k&N!=cy>R%*ikhM!OIfag{VHv*SHfJUJZd2!t#sd-m_A&WQ@@W<(pNrhl?|hOmXfP`TQsiGd1dsSRIPq#_&vhivxu}RsWRH zAZ9MU(c1pSrJu{S>>1h7R40v&8FQ`h2s9y&ZbPr*!5iWb@uGWN4^+6>dqkbQsx;xRVtQZdmnU&DpMnTwyI(4I)3nuN6cvEA=~xyzc!ztGuU)A?$Jpu(3T z@>f(Kww*X;)US_$BCHtnfq_WR#l@L6C6wM8>V&l7O5$q}c@hz;9zA@E`nLaL5WRuc z!}j~HJZp4sue1CGqZ4X3=6E;eYI{nS77}=66^BYpbL$HW3QvUJ|2$qMkIS(X{gL5C z3@!dXag86EdjtDjPH#C?F5TAz922U)Ua|`B1~}Q#v%?4jd4Dfk&G!&N>slthy$?PY zAM#0ke`=|!VUgFEGRjM7>Ugfrk4H&?)baGxHgv|Jwl0dcrP`cLzn1983HIca_{Oz1 zkKTP;k5;nKHlv8l?6c4l@5`PH(%Gy6^5jfVViN)h9=@5)C^-8cE^;pcf-vDj6_9Ey$EfRU7YvT*hx{f;AVA=6%6SG*AWb1_!Io1+E z<97mzUeywMQ+0>7jvGgm&)6dv+^+{AK+6qh8Yf21Y?)t8fC)6{dU!(Js<2YF$!FesAA5F_;IiM9L9 zVV8&Ecip#P=iLn9Cp#t4xKa#)=k!Y~y_>4MN=}XC*MIVRONI9#aSR$plB9~tW|6e_ zkT7s0#NZ#wKNLQCXO=|7ZET6Et;o54ll=92pz#-9T_-C!CtPu1DH_idKH^C~Aj0;( z-=gGHp9o5@aMk$de9_xu?HlewdOlrfh0o&$d)=L8w>uwU&(FL{hVSRRFKmalVrB1j z;qj=@X|C1vMFh4V7bzR{7$@#O#h!f;enHvk5g1g|lS(=i%!Hbo6W47HAVftLTYOXyYP^BF#|iRe4eo*b+l^5EoMSEL}Tp;H>S)J_^h)rVH%C$?A@}Z=&)@Dl$iEfTA#5**k}eU6ff zTV1K5O7U$IBaw!(7TF@nePK?$C|Yq6&o?@{Yf_^7IDfoACW_;=vP22FX8Fvqr7E=Z zqUo_D92MH+JzHfR>y&YIXgKEY9`x71hz9&MFsQihe@<}WN)zbX?w#X@>NtKqQE4iq z43=GCR2@QDE=9>Vu&qqQ)F|7sj6w?=?ZuQ-Pk8OooTYaSgw1BySHK|Iyvj6HJ!LJF zeUV4t2fya4A(_Lr%Y(Rzwl(uzt*bGeQryIs&%!sQG#3X8h!%hE&Q@zs{iwo!NLFQ| z1^s3xrQ&`(z!$v1&^c}`7yxU-XJs!xp$aRWec>{YedJQz`|)#?xwo6i_d)5%HKJK) z)%^J|nd0IHSoCwR=xxgq%kjKae8emWMhDD-vTg;K7yQK*QtFpDt>p`&bXUsHG{%TDD5U+66NaAnGtH?p9NEl*zM5sn|2YdDVChVyD zjuzzi6ZdQsZ0;vT<&L;hAJ)|Q(^9VlMcOkEPz3X{h6Z73w71_ZU#Sv9m1kU6HqW0= zE8|Yf$#8B)t2IQ`pD1;4GTWVBs~52kOdTu|V8xE=df&!A$usmM5EbD>S2L2V{=}VD zcS^S75wVfuDSG2Crh#EbqE6I4!Io#>mV{=(6LoZ+XS3eCzn@577+P-G8L+4ui~rK- zMR)&bciL$q=2zUq>7_*6O*^IK$Y{3m04)!C)0qB(4rIAiwS6~8IZaHYXRGA!rb*jI3!OQb#fJ288hFt9j5ReCKU zstlDoHOEBNBWQ7}fM1ifJXuw!&+Y`4Icqwz=)4uGHN*7tnTW;tn^njgWX{4u6{8-H zw&NkuG+Z&5oKhvnIcCyo+;9ZXI#X5>sg5gMz19n3ClPl}(fzJMc- zd&ui1j}yk7MI?65Ka=XzOinW&$U)j5)lW%y+4eQwX9_9>o!XYEDr=`q;ml<|aoeli zh+05bKF9c~o4%izHx+PV-4tSIM3|W;;IMvC+yr+&JHdG@zjo8a`9oZVUP*%wBJnh# zOl`0tf3Tm6z45SIjI?<$&Rs_j_^yQmD4<$);j(t#IsKd(Rfu(tjBB_o`25|{$DQUh-?0+NR zT{wf5#lqH?lvoZn_&z;{91HOF{D`Ap`wahWsP zi_jGmB-r)9fYsE6*f5caJJn zc{x<)c)eTALJnb@#wmK{vNzXY!)aK~wF0&u3Z6R-`))PA&d+DeC}x*FM!O6B)Kcn{ zb*0|O-t)EPI&$qtoj5#ZrMQTLrfvTWv;AzU0V@ZQe?Oup*s)${5ZG4{l`?F)*waW= ztr!kHuyC%Z^%tXnrGYty;kWJC&NP%(%ORi0IC0B1fz-E7Vj9o4jL_^6O=u&R)`wnH zPhncafO2|SG&{)X;|&>%%yTAi-HNB8aFKa!}MdPh?U2H~~L*Dd;;8Q(U zLz=yP{{=?(<{PwUBpSTu)%{G4;gKP6uXx+XA1I}%IMJBS&jIB=nF?0p?@SCyV5u~w zxDC|-69DvQ`U)^@3QmRHkm363Z%dsTalUd`D!59W$q>oA3VU;OMEd|a`|#m`U25?} zwpjy^<&Cbz@BdtW!DxPYw8mQAsUq2-Hlt)@uC=ow>ShtCT=uwUG-yGR$!+?es%Gn5 zmAKvkI zffCe8$wgHd)Q#~Gh%KB_GOD5C?-jX8zT8Yk{S}tWiut++Y#O47#=#|WS0+so@5 zTSYY+Tim=UxKgm>%DnzSozs) zvcq`AuN@b!LYkYFq2t`=y9@Bm_j|Krcy32EiS8oS{>8R?(5-nTN+xfA@aEo4y}j8a zK}yQe7#jzRYKi(@5H(o+8T*a-}r_k0!(4Np{V ze;Q=H^;f#Cte)c^WzFAP@=|S5A z(p`mE9G+g*KAvvD>GahC6D3}K8+(UqHO_4EVWP

m^m=cWXDZ7b!E{*~tMYDl7vA zp)oGU6v$w9Z$V{jIP|TD6C7+Ca(<2(_>rh2rv7~Skjy$<28jsPEV4j=7B4!5S@AU~ zu+`%>-qnV(rC?NV51GZvzyA{&owCV=fF_(G5<&z8v$cmE?)@P37Rmc8VWGj^9!|F0 zY^;sY2;z-Jr%7!>hyxoo@4!n(dWia3%L;c|aj~Ts+2tfAs@fgv!!ZFh^w6$yeIO4_c#AMjQR%c~H1iEKRxYS*Rs{?WZCX2%f|$tK3|u=GUoFoon&EjekxZA- z-g(7!TLm1|F^!@;dc0(NOeUUVc9DsAcF!%|zFawJdpgI08wM$=k!B`d#Jc8xPi_H%E(1y{o* zuVT}@+lXLzFlQZW!U5}kQ1N~FzzEnxZ;H6u$n)?QXH(g>dTyGw6x`*G0~9tj`X%L} zhwX&s^L0+OujNMr{(ueB>>Ad+=c=qITEZ`_vF)17+c7ipz~=!}NcqD?)r9 z69Y9Ix5VR9)<4S7Z(w7@n^2%OzY&A)zxZNr}VZQ%)?3-c-;thvZzx(n5WhE%6C#m@ZwpVvOIeIvm zRKeC#Z!WLdNIyHt+^VJ6{s^y=m=Zxoq|Q2dwy+;pDQl0ozJ$FBWqZBClpZg0MJ?@; zrpCm@P{nwj`jN5LnUBs$BxkVWmiE%liWf1;S5kdy+2j$=Omew%m)~#I%le7MVc-af zY>hPT3>ll-KVhBp9Vr~El#McFQ?Q6djI0@&vcR6Yil~acI83_lP&kji?UCesRdIJv zSy|$zFIhF@N1woJ8-@&#U_uv&N&lX#g`Yz^B~9g|>gICyO+3hR0FEBHg~3{rd6R0p z*z|&if*3lZHxVz;QWf?E8SB2zSc@Zw?t4sJa_oK9H;)&ztaFIj)mLwb`Q1TG>!xjd zgru;bbU37S$CM%1>F<$a<=jj(v%UR|+fNUiqPP(rpCL)oEWe=xX5jwJaPLikU;a0j28*UFhKP(EKY5YRd%{W zp_YA#g?(nG(sC5MsN?QSyPKlP7vs#-*cnFgSTg76_Z6o%S`fzeUx`wA9<=iQJ~MTM z633|=3T)39H47Z;MD{(L9bTiTILjFZ{(;VCbW=$I}ov)ai&)3{X) zy*Bff*3eiLc0z6o3(xg39<++~=_}hZQVGH=DNC*`+2i(%wmVI6SxD%Y)XTvsmw}k^ zwjT4(3#*>4<~u74#Tsb^g{s98&ALA%S68EMv$52Ud~W5-PXk^AVxp)pcua9HvXw^Yx`P49gRA5ty%$GF>DmVp&og zl@nihK#WYdkWP{MojydBD=&&V+>U&0dAwNgi8S#(y@zQ{;B?Sn)8+(zX(cxk8! zIy6U&YVsJV=Z=F&5>gz8qipyY64h|mt*4`tfW~c+ra(>-R0Od|6{fxC3-cFi4l07J z``6-Qh=Ig_&plK`K=mJgq+$kn-L_bXVekisgkq7;z1Fqx|BGY6fKVY3L7j}I@Q^Te z;!u^0cH_nea;z8J@-dqG7C>e!#bQs6&d+&}(-SY!w9f4cgAJOFXVpEAg-<$};BL|Ff2 z7ySG8NRkZw`;Y(Sv0^Nk>D^mKl}fDt&6WP!l@LLNQU19#|EaktjG#lK$6sdr_m>LJ o2s#hve>(3!+?fB*citbA9?S~|bmwJBx3rQj2ZvX%Q diff --git a/docs/source/user-guide/introduction.ipynb b/docs/source/user-guide/introduction.ipynb index 756d29aea..76eb77eea 100644 --- a/docs/source/user-guide/introduction.ipynb +++ b/docs/source/user-guide/introduction.ipynb @@ -73,7 +73,14 @@ "cell_type": "markdown", "id": "8edb47106e1a46a883d545849b8ab81b", "metadata": {}, - "source": "\nIf you are working in a Jupyter notebook, you can also use the following to give you a table\ndisplay that may be easier to read.\n\n```shell\ndisplay(df)\n```\n\n```{image} ../images/jupyter_lab_df_view.png\n:alt: Rendered table showing Pokemon DataFrame\n:width: 800\n```\n" + "source": "\nIf you are working in a Jupyter notebook, you can also use the following to give you a table\ndisplay that may be easier to read.\n" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": "display(df)" } ], "metadata": { diff --git a/mkdocs.yml b/mkdocs.yml index 26944e615..1f149ee8b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -117,7 +117,7 @@ nav: - User Guide: - user-guide/index.md - Introduction: user-guide/introduction.ipynb - - Basics: user-guide/basics.ipynb + - Concepts: user-guide/basics.ipynb - Data Sources: user-guide/data-sources.ipynb - DataFrame: - user-guide/dataframe/index.md From 3bde9f707efd85e8f272e8cd2c483aa2784c5897 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Sun, 7 Jun 2026 18:48:28 +0200 Subject: [PATCH 06/18] docs: sweep leftover Sphinx/MyST roles, expand short autoref targets MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * `dev/rewrite_doc_roles.py`: extend the role regex to handle bare `:class:`, `:func:`, `:meth:`, `:mod:`, `:attr:`, `:obj:`, `:data:`, `:exc:` (no `py:` prefix) and the matching `{class}` / `{func}` / `{meth}` / ... MyST variants. Also tolerate the `~.foo` leading-dot syntax. Re-run the script: 9 files updated, 0 sphinx-role artifacts left in `docs/source/` or `python/datafusion/`. * Convert all `:ref:` invocations (e.g. `:ref:`user_guide_concepts``) to direct Markdown links pointing at the user guide URLs. * Rename `docs/source/user-guide/basics.ipynb` -> `concepts.ipynb` so the URL slug matches the page H1. Update `mkdocs.yml` nav. * Expand short autoref targets: `[X][datafusion.X.method]` -> `[X][datafusion.module.X.method]` across user-guide pages and `python/datafusion/*.py` docstrings (DataFrame, SessionContext, ExecutionPlan, Expr, RecordBatch, Catalog, ScalarUDF/AggregateUDF /WindowUDF/TableFunction, etc.). * Bare leaf-name autorefs (`[X][SessionContext]` etc.) also expanded. * Bare function-name autorefs (`[X][col]`, `[X][rank]`, ...) mapped to `datafusion.functions.X` based on `datafusion.functions` exports. * Auto-link plain-code mentions in `concepts.ipynb` and `user-guide/dataframe/index.md` so prose references to `DataFrame`, `read_csv`, `SessionContext`, `col`, `lit`, etc. become real links. * Add `CaseBuilder` and `GroupingSet` to `reference/expr.md` so their cross-refs resolve. * `docs/hooks.py`: type the MkDocs hook signature, hoist the parts-count magic literal into a named constant. Add `INP001` to the `docs/*` ruff per-file-ignore so the hook file doesn't need an `__init__.py`. * Shorten the few overlong `[user_guide_concepts](/python/...)` style links in `python/datafusion/*.py` module docstrings — drop the link, keep the prose — so the lines come back under the 88-char limit. Build is green and cross-reference warnings are down from 253 to 101 (remaining are bare lowercase method names like `union`/`sort_by` whose target class is ambiguous, plus a handful of `pa.RecordBatch` PyArrow inventory alias misses). Co-Authored-By: Claude Opus 4.7 --- dev/rewrite_doc_roles.py | 17 +- docs/source/reference/expr.md | 8 + .../common-operations/aggregations.ipynb | 4 +- .../common-operations/udf-and-udfa.ipynb | 2 +- .../common-operations/windows.ipynb | 2 +- .../{basics.ipynb => concepts.ipynb} | 2 +- .../user-guide/dataframe/execution-metrics.md | 62 +++--- docs/source/user-guide/dataframe/index.md | 84 ++++---- docs/source/user-guide/distributing-work.md | 16 +- docs/source/user-guide/io/table_provider.md | 2 +- mkdocs.yml | 5 +- pyproject.toml | 2 +- python/datafusion/catalog.py | 4 +- python/datafusion/context.py | 60 +++--- python/datafusion/dataframe.py | 42 ++-- python/datafusion/dataframe_formatter.py | 4 +- python/datafusion/expr.py | 74 +++---- python/datafusion/functions.py | 202 +++++++++--------- python/datafusion/ipc.py | 50 ++--- python/datafusion/plan.py | 12 +- python/datafusion/substrait.py | 2 +- python/datafusion/user_defined.py | 16 +- 22 files changed, 344 insertions(+), 328 deletions(-) rename docs/source/user-guide/{basics.ipynb => concepts.ipynb} (62%) diff --git a/dev/rewrite_doc_roles.py b/dev/rewrite_doc_roles.py index dc5346933..de26f74cf 100644 --- a/dev/rewrite_doc_roles.py +++ b/dev/rewrite_doc_roles.py @@ -48,25 +48,30 @@ REPO = Path(__file__).resolve().parents[1] ROLE_PATTERNS = [ - # Sphinx RST roles: :py:class:`~mod.Name` or :py:class:`Name ` + # Sphinx RST roles: :py:class:`~mod.Name`, :class:`~mod.Name`, plus + # the `Name ` long form. Both `py:` and bare role names. ( - re.compile(r":py:(?:class|func|meth|mod|attr|obj|data):`~?([\w.]+)`"), + re.compile( + r":(?:py:)?(?:class|func|meth|mod|attr|obj|data|exc):`~?\.?([\w.]+)`" + ), lambda m: f"[`{m.group(1).split('.')[-1]}`][{m.group(1)}]", ), ( re.compile( - r":py:(?:class|func|meth|mod|attr|obj|data):`([^<`]+)\s*<([\w.]+)>`" + r":(?:py:)?(?:class|func|meth|mod|attr|obj|data|exc):`([^<`]+)\s*<\.?([\w.]+)>`" ), lambda m: f"[`{m.group(1).strip()}`][{m.group(2)}]", ), - # MyST roles: {py:class}`~mod.Name` + # MyST roles: {py:class}`~mod.Name` and the bare {class}`~mod.Name` aliases. ( - re.compile(r"\{py:(?:class|func|meth|mod|attr|obj|data)\}`~?([\w.]+)`"), + re.compile( + r"\{(?:py:)?(?:class|func|meth|mod|attr|obj|data|exc)\}`~?\.?([\w.]+)`" + ), lambda m: f"[`{m.group(1).split('.')[-1]}`][{m.group(1)}]", ), ( re.compile( - r"\{py:(?:class|func|meth|mod|attr|obj|data)\}`([^<`]+)\s*<([\w.]+)>`" + r"\{(?:py:)?(?:class|func|meth|mod|attr|obj|data|exc)\}`([^<`]+)\s*<\.?([\w.]+)>`" ), lambda m: f"[`{m.group(1).strip()}`][{m.group(2)}]", ), diff --git a/docs/source/reference/expr.md b/docs/source/reference/expr.md index acea40c80..6a984ed72 100644 --- a/docs/source/reference/expr.md +++ b/docs/source/reference/expr.md @@ -8,6 +8,14 @@ ::: datafusion.expr.WindowFrame +## CaseBuilder + +::: datafusion.expr.CaseBuilder + +## GroupingSet + +::: datafusion.expr.GroupingSet + ## col ::: datafusion.col.col diff --git a/docs/source/user-guide/common-operations/aggregations.ipynb b/docs/source/user-guide/common-operations/aggregations.ipynb index be89647fa..3b53f31d8 100644 --- a/docs/source/user-guide/common-operations/aggregations.ipynb +++ b/docs/source/user-guide/common-operations/aggregations.ipynb @@ -68,7 +68,7 @@ "cell_type": "markdown", "id": "8dd0d8092fe74a7c96281538738b07e2", "metadata": {}, - "source": "\nWhen `group_by` is `None` or an empty list, the aggregation is done over the whole\n{class}`.DataFrame`. For grouping the `group_by` list must contain at least one column.\n\n" + "source": "\nWhen `group_by` is `None` or an empty list, the aggregation is done over the whole\n[`DataFrame`][datafusion.dataframe.DataFrame]. For grouping the `group_by` list must contain at least one column.\n\n" }, { "cell_type": "code", @@ -446,7 +446,7 @@ "cell_type": "markdown", "id": "5b09d5ef5b5e4bb6ab9b829b10b6a29f", "metadata": {}, - "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the accumulator class is captured by value via {mod}`cloudpickle`,\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the accumulator class is captured by value via [`cloudpickle`][cloudpickle],\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" } ], "metadata": { diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb index 31c2b4ffb..e83a6f94e 100644 --- a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb +++ b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb @@ -189,7 +189,7 @@ "cell_type": "markdown", "id": "938c804e27f84196a10c8828c723f798", "metadata": {}, - "source": "\nThe `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`.\nThere is no `pruning_predicate` and no `required_guarantees`: the\nscan has to materialize every row group and hand each row to the\nPython callback just to decide whether to keep it.\n\nAt small scale the cost difference is invisible; on a Parquet file with\nmany row groups, or data whose min/max statistics line up well with\nthe predicate, the native form can skip most of the file. The UDF form\nreads all of it.\n\n**Takeaway.** Reach for a UDF when the per-row computation is genuinely\nnot expressible as a tree of built-in functions (custom numerical work,\nexternal lookups, complex business rules). When it *is* expressible \u2014\neven if the native form is a little more verbose \u2014 build the `Expr`\ntree directly so the optimizer can see through it. For disjunctive\npredicates the idiom is to produce one clause per bucket and combine\nthem with `|`:\n\n```python\nfrom functools import reduce\nfrom operator import or_\nfrom datafusion import col, lit, functions as f\n\nbuckets = {\n \"Brand#12\": {\"containers\": [\"SM CASE\", \"SM BOX\"], \"min_qty\": 1, \"max_size\": 5},\n \"Brand#23\": {\"containers\": [\"MED BAG\", \"MED BOX\"], \"min_qty\": 10, \"max_size\": 10},\n}\n\ndef bucket_clause(brand, spec):\n return (\n (col(\"brand\") == lit(brand))\n & f.in_list(col(\"container\"), [lit(c) for c in spec[\"containers\"]])\n & (col(\"quantity\") >= lit(spec[\"min_qty\"]))\n & (col(\"quantity\") <= lit(spec[\"min_qty\"] + 10))\n & (col(\"size\") >= lit(1))\n & (col(\"size\") <= lit(spec[\"max_size\"]))\n )\n\npredicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items()))\ndf = df.filter(predicate)\n```\n\n## Aggregate Functions\n\nThe [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined\nAggregate Functions (UDAFs). To use this you must implement an\n[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed.\n\nWhen defining a UDAF there are four methods you need to implement. The `update` function takes the\narray(s) of input and updates the internal state of the accumulator. You should define this function\nto have as many input arguments as you will pass when calling the UDAF. Since aggregation may be\nsplit into multiple batches, we must have a method to combine multiple batches. For this, we have\ntwo functions, `state` and `merge`. `state` will return an array of scalar values that contain\nthe current state of a single batch accumulation. Then we must `merge` the results of these\ndifferent states. Finally `evaluate` is the call that will return the final result after the\n`merge` is complete.\n\nIn the following example we want to define a custom aggregate function that will return the\ndifference between the sum of two columns. The state can be represented by a single value and we can\nalso see how the inputs to `update` and `merge` differ.\n\n```python\nimport pyarrow as pa\nimport pyarrow.compute\nimport datafusion\nfrom datafusion import col, udaf, Accumulator\nfrom typing import List\n\nclass MyAccumulator(Accumulator):\n \"\"\"\n Interface of a user-defined accumulation.\n \"\"\"\n def __init__(self):\n self._sum = 0.0\n\n def update(self, values_a: pa.Array, values_b: pa.Array) -> None:\n self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py()\n\n def merge(self, states: list[pa.Array]) -> None:\n self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py()\n\n def state(self) -> list[pa.Scalar]:\n return [pyarrow.scalar(self._sum)]\n\n def evaluate(self) -> pa.Scalar:\n return pyarrow.scalar(self._sum)\n\nctx = datafusion.SessionContext()\ndf = ctx.from_pydict(\n {\n \"a\": [4, 5, 6],\n \"b\": [1, 2, 3],\n }\n)\n\nmy_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable')\n\ndf.aggregate([], [my_udaf(col(\"a\"), col(\"b\")).alias(\"col_diff\")])\n```\n\n### FAQ\n\n**How do I return a list from a UDAF?**\n\nBoth the `evaluate` and the `state` functions expect to return scalar values.\nIf you wish to return a list array as a scalar value, the best practice is to\nwrap the values in a `pyarrow.Scalar` object. For example, you can return a\ntimestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp(\"ms\")))` and\nregister the appropriate return or state types as\n`return_type=pa.list_(pa.timestamp(\"ms\"))` and\n`state_type=[pa.list_(pa.timestamp(\"ms\"))]`, respectively.\n\nAs of DataFusion 52.0.0 , you can pass return any Python object, including a\nPyArrow array, as the return value(s) for these functions and DataFusion will\nattempt to create a scalar type from the value. DataFusion has been tested to\nconvert PyArrow, nanoarrow, and arro3 objects as well as primitive data types\nlike integers, strings, and so on.\n\n## Window Functions\n\nTo implement a User-Defined Window Function (UDWF) you must call the\n[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract\nclass [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator].\n\nThere are three methods of evaluation of UDWFs.\n\n- `evaluate` is the simplest case, where you are given an array and are expected to calculate the\n value for a single row of that array. This is the simplest case, but also the least performant.\n- `evaluate_all` computes the values for all rows for an input array at a single time.\n- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank\n information for the rows.\n\nWhich methods you implement are based upon which of these options are set.\n\n| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement |\n|---|---|---|---|\n| False (default) | False (default) | False (default) | `evaluate_all` |\n| False | True | False | `evaluate` |\n| False | True | False | `evaluate_all_with_rank` |\n| True | True/False | True/False | `evaluate` |\n\n### UDWF options\n\nWhen you define your UDWF you can override the functions that return these values. They will\ndetermine which evaluate functions are called.\n\n- `uses_window_frame` is set for functions that compute based on the specified window frame. If\n your function depends upon the specified frame, set this to `True`.\n- `supports_bounded_execution` specifies if your function can be incrementally computed.\n- `include_rank` is set to `True` for window functions that can be computed only using the rank\n information.\n\n```python\nimport pyarrow as pa\nfrom datafusion import udwf, col, SessionContext\nfrom datafusion.user_defined import WindowEvaluator\n\nclass ExponentialSmooth(WindowEvaluator):\n def __init__(self, alpha: float) -> None:\n self.alpha = alpha\n\n def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array:\n results = []\n curr_value = 0.0\n values = values[0]\n for idx in range(num_rows):\n if idx == 0:\n curr_value = values[idx].as_py()\n else:\n curr_value = values[idx].as_py() * self.alpha + curr_value * (\n 1.0 - self.alpha\n )\n results.append(curr_value)\n\n return pa.array(results)\n\nexp_smooth = udwf(\n ExponentialSmooth(0.9),\n pa.float64(),\n pa.float64(),\n volatility=\"immutable\",\n)\n\nctx = SessionContext()\n\ndf = ctx.from_pydict({\n \"a\": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0]\n})\n\ndf.select(\"a\", exp_smooth(col(\"a\")).alias(\"smooth_a\")).show()\n```\n\n## Table Functions\n\nUser Defined Table Functions are slightly different than the other functions\ndescribed here. These functions take any number of `Expr` arguments, but only\nliteral expressions are supported. Table functions must return a Table\nProvider as described in the ref:`_io_custom_table_provider` page.\n\nOnce you have a table function, you can register it with the session context\nby using [`register_udtf`][datafusion.context.SessionContext.register_udtf].\n\nThere are examples of both rust backed and python based table functions in the\nexamples folder of the repository. If you have a rust backed table function\nthat you wish to expose via PyO3, you need to expose it as a `PyCapsule`.\n\n```rust\n#[pymethods]\nimpl MyTableFunction {\n fn __datafusion_table_function__<'py>(\n &self,\n py: Python<'py>,\n ) -> PyResult> {\n let name = cr\"datafusion_table_function\".into();\n\n let func = self.clone();\n let provider = FFI_TableFunction::new(Arc::new(func), None);\n\n PyCapsule::new(py, provider, Some(name))\n }\n}\n```\n\n### Accessing the Calling Session\n\nPure-Python UDTFs can opt into receiving the calling\n[`SessionContext`][datafusion.SessionContext] by registering with\n`with_session=True`. The context is passed as a `session` keyword\nargument on every invocation. Use it to look up registered tables,\nUDFs, or session configuration from inside the callback.\n\n```python\nfrom datafusion import SessionContext, Table, udtf\nfrom datafusion.context import TableProviderExportable\nimport pyarrow as pa\nimport pyarrow.dataset as ds\n\n@udtf(\"list_tables\", with_session=True)\ndef list_tables(*, session: SessionContext) -> TableProviderExportable:\n names = sorted(session.catalog().schema().names())\n batch = pa.RecordBatch.from_pydict({\"name\": names})\n return Table(ds.dataset([batch]))\n\nctx = SessionContext()\nctx.register_batch(\"t1\", pa.RecordBatch.from_pydict({\"x\": [1]}))\nctx.register_udtf(list_tables)\nctx.sql(\"SELECT * FROM list_tables()\").show()\n```\n\nWithout `with_session=True`, the callback receives only the positional\nexpression arguments. The flag is opt-in so existing UDTFs keep working\nunchanged.\n\nThe injected `session` is a fresh [`SessionContext`][datafusion.SessionContext]\nwrapper backed by the same underlying state as the caller, so registries\n(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering\na new table or UDF) propagate to the live session because the registries\nare reference-counted and shared. Configuration changes made through the\nwrapper (e.g. setting session options) do **not** propagate \u2014 the wrapper\nholds its own clone of the session config.\n" + "source": "\nThe `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`.\nThere is no `pruning_predicate` and no `required_guarantees`: the\nscan has to materialize every row group and hand each row to the\nPython callback just to decide whether to keep it.\n\nAt small scale the cost difference is invisible; on a Parquet file with\nmany row groups, or data whose min/max statistics line up well with\nthe predicate, the native form can skip most of the file. The UDF form\nreads all of it.\n\n**Takeaway.** Reach for a UDF when the per-row computation is genuinely\nnot expressible as a tree of built-in functions (custom numerical work,\nexternal lookups, complex business rules). When it *is* expressible \u2014\neven if the native form is a little more verbose \u2014 build the `Expr`\ntree directly so the optimizer can see through it. For disjunctive\npredicates the idiom is to produce one clause per bucket and combine\nthem with `|`:\n\n```python\nfrom functools import reduce\nfrom operator import or_\nfrom datafusion import col, lit, functions as f\n\nbuckets = {\n \"Brand#12\": {\"containers\": [\"SM CASE\", \"SM BOX\"], \"min_qty\": 1, \"max_size\": 5},\n \"Brand#23\": {\"containers\": [\"MED BAG\", \"MED BOX\"], \"min_qty\": 10, \"max_size\": 10},\n}\n\ndef bucket_clause(brand, spec):\n return (\n (col(\"brand\") == lit(brand))\n & f.in_list(col(\"container\"), [lit(c) for c in spec[\"containers\"]])\n & (col(\"quantity\") >= lit(spec[\"min_qty\"]))\n & (col(\"quantity\") <= lit(spec[\"min_qty\"] + 10))\n & (col(\"size\") >= lit(1))\n & (col(\"size\") <= lit(spec[\"max_size\"]))\n )\n\npredicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items()))\ndf = df.filter(predicate)\n```\n\n## Aggregate Functions\n\nThe [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined\nAggregate Functions (UDAFs). To use this you must implement an\n[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed.\n\nWhen defining a UDAF there are four methods you need to implement. The `update` function takes the\narray(s) of input and updates the internal state of the accumulator. You should define this function\nto have as many input arguments as you will pass when calling the UDAF. Since aggregation may be\nsplit into multiple batches, we must have a method to combine multiple batches. For this, we have\ntwo functions, `state` and `merge`. `state` will return an array of scalar values that contain\nthe current state of a single batch accumulation. Then we must `merge` the results of these\ndifferent states. Finally `evaluate` is the call that will return the final result after the\n`merge` is complete.\n\nIn the following example we want to define a custom aggregate function that will return the\ndifference between the sum of two columns. The state can be represented by a single value and we can\nalso see how the inputs to `update` and `merge` differ.\n\n```python\nimport pyarrow as pa\nimport pyarrow.compute\nimport datafusion\nfrom datafusion import col, udaf, Accumulator\nfrom typing import List\n\nclass MyAccumulator(Accumulator):\n \"\"\"\n Interface of a user-defined accumulation.\n \"\"\"\n def __init__(self):\n self._sum = 0.0\n\n def update(self, values_a: pa.Array, values_b: pa.Array) -> None:\n self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py()\n\n def merge(self, states: list[pa.Array]) -> None:\n self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py()\n\n def state(self) -> list[pa.Scalar]:\n return [pyarrow.scalar(self._sum)]\n\n def evaluate(self) -> pa.Scalar:\n return pyarrow.scalar(self._sum)\n\nctx = datafusion.SessionContext()\ndf = ctx.from_pydict(\n {\n \"a\": [4, 5, 6],\n \"b\": [1, 2, 3],\n }\n)\n\nmy_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable')\n\ndf.aggregate([], [my_udaf(col(\"a\"), col(\"b\")).alias(\"col_diff\")])\n```\n\n### FAQ\n\n**How do I return a list from a UDAF?**\n\nBoth the `evaluate` and the `state` functions expect to return scalar values.\nIf you wish to return a list array as a scalar value, the best practice is to\nwrap the values in a `pyarrow.Scalar` object. For example, you can return a\ntimestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp(\"ms\")))` and\nregister the appropriate return or state types as\n`return_type=pa.list_(pa.timestamp(\"ms\"))` and\n`state_type=[pa.list_(pa.timestamp(\"ms\"))]`, respectively.\n\nAs of DataFusion 52.0.0 , you can pass return any Python object, including a\nPyArrow array, as the return value(s) for these functions and DataFusion will\nattempt to create a scalar type from the value. DataFusion has been tested to\nconvert PyArrow, nanoarrow, and arro3 objects as well as primitive data types\nlike integers, strings, and so on.\n\n## Window Functions\n\nTo implement a User-Defined Window Function (UDWF) you must call the\n[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract\nclass [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator].\n\nThere are three methods of evaluation of UDWFs.\n\n- `evaluate` is the simplest case, where you are given an array and are expected to calculate the\n value for a single row of that array. This is the simplest case, but also the least performant.\n- `evaluate_all` computes the values for all rows for an input array at a single time.\n- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank\n information for the rows.\n\nWhich methods you implement are based upon which of these options are set.\n\n| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement |\n|---|---|---|---|\n| False (default) | False (default) | False (default) | `evaluate_all` |\n| False | True | False | `evaluate` |\n| False | True | False | `evaluate_all_with_rank` |\n| True | True/False | True/False | `evaluate` |\n\n### UDWF options\n\nWhen you define your UDWF you can override the functions that return these values. They will\ndetermine which evaluate functions are called.\n\n- `uses_window_frame` is set for functions that compute based on the specified window frame. If\n your function depends upon the specified frame, set this to `True`.\n- `supports_bounded_execution` specifies if your function can be incrementally computed.\n- `include_rank` is set to `True` for window functions that can be computed only using the rank\n information.\n\n```python\nimport pyarrow as pa\nfrom datafusion import udwf, col, SessionContext\nfrom datafusion.user_defined import WindowEvaluator\n\nclass ExponentialSmooth(WindowEvaluator):\n def __init__(self, alpha: float) -> None:\n self.alpha = alpha\n\n def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array:\n results = []\n curr_value = 0.0\n values = values[0]\n for idx in range(num_rows):\n if idx == 0:\n curr_value = values[idx].as_py()\n else:\n curr_value = values[idx].as_py() * self.alpha + curr_value * (\n 1.0 - self.alpha\n )\n results.append(curr_value)\n\n return pa.array(results)\n\nexp_smooth = udwf(\n ExponentialSmooth(0.9),\n pa.float64(),\n pa.float64(),\n volatility=\"immutable\",\n)\n\nctx = SessionContext()\n\ndf = ctx.from_pydict({\n \"a\": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0]\n})\n\ndf.select(\"a\", exp_smooth(col(\"a\")).alias(\"smooth_a\")).show()\n```\n\n## Table Functions\n\nUser Defined Table Functions are slightly different than the other functions\ndescribed here. These functions take any number of `Expr` arguments, but only\nliteral expressions are supported. Table functions must return a Table\nProvider as described in the ref:`_io_custom_table_provider` page.\n\nOnce you have a table function, you can register it with the session context\nby using [`register_udtf`][datafusion.context.SessionContext.register_udtf].\n\nThere are examples of both rust backed and python based table functions in the\nexamples folder of the repository. If you have a rust backed table function\nthat you wish to expose via PyO3, you need to expose it as a `PyCapsule`.\n\n```rust\n#[pymethods]\nimpl MyTableFunction {\n fn __datafusion_table_function__<'py>(\n &self,\n py: Python<'py>,\n ) -> PyResult> {\n let name = cr\"datafusion_table_function\".into();\n\n let func = self.clone();\n let provider = FFI_TableFunction::new(Arc::new(func), None);\n\n PyCapsule::new(py, provider, Some(name))\n }\n}\n```\n\n### Accessing the Calling Session\n\nPure-Python UDTFs can opt into receiving the calling\n[`SessionContext`][datafusion.context.SessionContext] by registering with\n`with_session=True`. The context is passed as a `session` keyword\nargument on every invocation. Use it to look up registered tables,\nUDFs, or session configuration from inside the callback.\n\n```python\nfrom datafusion import SessionContext, Table, udtf\nfrom datafusion.context import TableProviderExportable\nimport pyarrow as pa\nimport pyarrow.dataset as ds\n\n@udtf(\"list_tables\", with_session=True)\ndef list_tables(*, session: SessionContext) -> TableProviderExportable:\n names = sorted(session.catalog().schema().names())\n batch = pa.RecordBatch.from_pydict({\"name\": names})\n return Table(ds.dataset([batch]))\n\nctx = SessionContext()\nctx.register_batch(\"t1\", pa.RecordBatch.from_pydict({\"x\": [1]}))\nctx.register_udtf(list_tables)\nctx.sql(\"SELECT * FROM list_tables()\").show()\n```\n\nWithout `with_session=True`, the callback receives only the positional\nexpression arguments. The flag is opt-in so existing UDTFs keep working\nunchanged.\n\nThe injected `session` is a fresh [`SessionContext`][datafusion.context.SessionContext]\nwrapper backed by the same underlying state as the caller, so registries\n(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering\na new table or UDF) propagate to the live session because the registries\nare reference-counted and shared. Configuration changes made through the\nwrapper (e.g. setting session options) do **not** propagate \u2014 the wrapper\nholds its own clone of the session config.\n" } ], "metadata": { diff --git a/docs/source/user-guide/common-operations/windows.ipynb b/docs/source/user-guide/common-operations/windows.ipynb index 007c9a82d..3995b3506 100644 --- a/docs/source/user-guide/common-operations/windows.ipynb +++ b/docs/source/user-guide/common-operations/windows.ipynb @@ -209,7 +209,7 @@ "cell_type": "markdown", "id": "8a65eabff63a45729fe45fb5ade58bdc", "metadata": {}, - "source": "\n## Available Functions\n\nThe possible window functions are:\n\n1. Rank Functions\n : - [`rank`][datafusion.functions.rank]\n - [`dense_rank`][datafusion.functions.dense_rank]\n - [`ntile`][datafusion.functions.ntile]\n - [`row_number`][datafusion.functions.row_number]\n2. Analytical Functions\n : - [`cume_dist`][datafusion.functions.cume_dist]\n - [`percent_rank`][datafusion.functions.percent_rank]\n - [`lag`][datafusion.functions.lag]\n - [`lead`][datafusion.functions.lead]\n3. Aggregate Functions\n : - All [Aggregation Functions](aggregation) can be used as window functions.\n\n## User-Defined Window Functions\n\nYou can ship custom window functions to the engine by subclassing\n[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it\nvia [`udwf`][datafusion.udwf]. See [`user_defined`][datafusion.user_defined]\nfor the evaluator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python window UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the evaluator class is captured by value via {mod}`cloudpickle`, so\n worker processes do not need to pre-register the UDF. Any names the\n evaluator resolves via `import` are captured **by reference** and\n must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + "source": "\n## Available Functions\n\nThe possible window functions are:\n\n1. Rank Functions\n : - [`rank`][datafusion.functions.rank]\n - [`dense_rank`][datafusion.functions.dense_rank]\n - [`ntile`][datafusion.functions.ntile]\n - [`row_number`][datafusion.functions.row_number]\n2. Analytical Functions\n : - [`cume_dist`][datafusion.functions.cume_dist]\n - [`percent_rank`][datafusion.functions.percent_rank]\n - [`lag`][datafusion.functions.lag]\n - [`lead`][datafusion.functions.lead]\n3. Aggregate Functions\n : - All [Aggregation Functions](aggregation) can be used as window functions.\n\n## User-Defined Window Functions\n\nYou can ship custom window functions to the engine by subclassing\n[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it\nvia [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`][datafusion.user_defined]\nfor the evaluator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python window UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the evaluator class is captured by value via [`cloudpickle`][cloudpickle], so\n worker processes do not need to pre-register the UDF. Any names the\n evaluator resolves via `import` are captured **by reference** and\n must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" } ], "metadata": { diff --git a/docs/source/user-guide/basics.ipynb b/docs/source/user-guide/concepts.ipynb similarity index 62% rename from docs/source/user-guide/basics.ipynb rename to docs/source/user-guide/concepts.ipynb index a10188a71..63f157faa 100644 --- a/docs/source/user-guide/basics.ipynb +++ b/docs/source/user-guide/concepts.ipynb @@ -65,7 +65,7 @@ "cell_type": "markdown", "id": "8dd0d8092fe74a7c96281538738b07e2", "metadata": {}, - "source": "\n## Session Context\n\nThe first statement group creates a [`SessionContext`][datafusion.context.SessionContext].\n\n```python\n# create a context\nctx = datafusion.SessionContext()\n```\n\nA Session Context is the main interface for executing queries with DataFusion. It maintains the state\nof the connection between a user and an instance of the DataFusion engine. Additionally it provides\nthe following functionality:\n\n- Create a DataFrame from a data source.\n- Register a data source as a table that can be referenced from a SQL query.\n- Execute a SQL query\n\n## DataFrame\n\nThe second statement group creates a `DataFrame`,\n\n```python\n# Create a DataFrame from a file\ndf = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n```\n\nA DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html).\nDataFrames are typically created by calling a method on [`SessionContext`][datafusion.context.SessionContext], such as `read_csv`, and can then be modified by\ncalling the transformation methods, such as [`filter`][datafusion.dataframe.DataFrame.filter], [`select`][datafusion.dataframe.DataFrame.select], [`aggregate`][datafusion.dataframe.DataFrame.aggregate],\nand [`limit`][datafusion.dataframe.DataFrame.limit] to build up a query definition.\n\nFor more details on working with DataFrames, including visualization options and conversion to other formats, see [dataframe/index](dataframe/index.md).\n\n## Expressions\n\nThe third statement uses `Expressions` to build up a query definition. You can find\nexplanations for what the functions below do in the user documentation for\n[`col`][datafusion.col], [`lit`][datafusion.lit], [`round`][datafusion.functions.round],\nand [`alias`][datafusion.expr.Expr.alias].\n\n```python\ndf = df.select(\n \"trip_distance\",\n col(\"total_amount\").alias(\"total\"),\n (f.round(lit(100.0) * col(\"tip_amount\") / col(\"total_amount\"), lit(1))).alias(\"tip_percent\"),\n)\n```\n\nFinally the [`show`][datafusion.dataframe.DataFrame.show] method converts the logical plan\nrepresented by the DataFrame into a physical plan and execute it, collecting all results and\ndisplaying them to the user. It is important to note that DataFusion performs lazy evaluation\nof the DataFrame. Until you call a method such as [`show`][datafusion.dataframe.DataFrame.show]\nor [`collect`][datafusion.dataframe.DataFrame.collect], DataFusion will not perform the query.\n" + "source": "\n## Session Context\n\nThe first statement group creates a [`SessionContext`][datafusion.context.SessionContext].\n\n```python\n# create a context\nctx = datafusion.SessionContext()\n```\n\nA Session Context is the main interface for executing queries with DataFusion. It maintains the state\nof the connection between a user and an instance of the DataFusion engine. Additionally it provides\nthe following functionality:\n\n- Create a DataFrame from a data source.\n- Register a data source as a table that can be referenced from a SQL query.\n- Execute a SQL query\n\n## DataFrame\n\nThe second statement group creates a [`DataFrame`][datafusion.dataframe.DataFrame],\n\n```python\n# Create a DataFrame from a file\ndf = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n```\n\nA DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html).\nDataFrames are typically created by calling a method on [`SessionContext`][datafusion.context.SessionContext], such as [`read_csv`][datafusion.io.read_csv], and can then be modified by\ncalling the transformation methods, such as [`filter`][datafusion.dataframe.DataFrame.filter], [`select`][datafusion.dataframe.DataFrame.select], [`aggregate`][datafusion.dataframe.DataFrame.aggregate],\nand [`limit`][datafusion.dataframe.DataFrame.limit] to build up a query definition.\n\nFor more details on working with DataFrames, including visualization options and conversion to other formats, see [dataframe/index](dataframe/index.md).\n\n## Expressions\n\nThe third statement uses [Expressions](../common-operations/expressions/) to build up a query definition. You can find\nexplanations for what the functions below do in the user documentation for\n[`col`][datafusion.col], [`lit`][datafusion.lit], [`round`][datafusion.functions.round],\nand [`alias`][datafusion.expr.Expr.alias].\n\n```python\ndf = df.select(\n \"trip_distance\",\n col(\"total_amount\").alias(\"total\"),\n (f.round(lit(100.0) * col(\"tip_amount\") / col(\"total_amount\"), lit(1))).alias(\"tip_percent\"),\n)\n```\n\nFinally the [`show`][datafusion.dataframe.DataFrame.show] method converts the logical plan\nrepresented by the DataFrame into a physical plan and execute it, collecting all results and\ndisplaying them to the user. It is important to note that DataFusion performs lazy evaluation\nof the DataFrame. Until you call a method such as [`show`][datafusion.dataframe.DataFrame.show]\nor [`collect`][datafusion.dataframe.DataFrame.collect], DataFusion will not perform the query.\n" } ], "metadata": { diff --git a/docs/source/user-guide/dataframe/execution-metrics.md b/docs/source/user-guide/dataframe/execution-metrics.md index d44a46bc6..1e700e5be 100644 --- a/docs/source/user-guide/dataframe/execution-metrics.md +++ b/docs/source/user-guide/dataframe/execution-metrics.md @@ -37,26 +37,26 @@ Typical metrics include: Metrics are collected *per-partition*: DataFusion may execute each operator in parallel across several partitions. The convenience properties on -[`MetricsSet`][datafusion.MetricsSet] (e.g. `output_rows`, `elapsed_compute`) +[`MetricsSet`][datafusion.plan.MetricsSet] (e.g. `output_rows`, `elapsed_compute`) automatically sum the named metric across **all** partitions, giving a single aggregate value for the operator as a whole. You can also access the raw -per-partition [`Metric`][datafusion.Metric] objects via -[`metrics`][datafusion.MetricsSet.metrics]. +per-partition [`Metric`][datafusion.plan.Metric] objects via +[`metrics`][datafusion.plan.MetricsSet.metrics]. ## When Are Metrics Available? Some operators (for example `DataSourceExec`) eagerly create a -[`MetricsSet`][datafusion.MetricsSet] when the physical plan is built, so -[`metrics`][datafusion.ExecutionPlan.metrics] may return a set even before any +[`MetricsSet`][datafusion.plan.MetricsSet] when the physical plan is built, so +[`metrics`][datafusion.plan.ExecutionPlan.metrics] may return a set even before any rows have been processed. However, metric **values** such as `output_rows` are only meaningful **after** the DataFrame has been executed via one of the terminal operations: -- [`collect`][datafusion.DataFrame.collect] -- [`collect_partitioned`][datafusion.DataFrame.collect_partitioned] -- [`execute_stream`][datafusion.DataFrame.execute_stream] +- [`collect`][datafusion.dataframe.DataFrame.collect] +- [`collect_partitioned`][datafusion.dataframe.DataFrame.collect_partitioned] +- [`execute_stream`][datafusion.dataframe.DataFrame.execute_stream] (metrics are available once the stream has been fully consumed) -- [`execute_stream_partitioned`][datafusion.DataFrame.execute_stream_partitioned] +- [`execute_stream_partitioned`][datafusion.dataframe.DataFrame.execute_stream_partitioned] (metrics are available once all partition streams have been fully consumed) Before execution, metric values will be `0` or `None`. @@ -67,37 +67,37 @@ Before execution, metric values will be `0` or `None`. When a DataFrame is displayed in a notebook (e.g. via `display(df)` or automatic `repr` output), DataFusion runs a *limited* internal execution to fetch preview rows. This internal execution does **not** cache the - physical plan used, so [`collect_metrics`][datafusion.ExecutionPlan.collect_metrics] + physical plan used, so [`collect_metrics`][datafusion.plan.ExecutionPlan.collect_metrics] will not reflect the display execution. To access metrics you must call one of the terminal operations listed above. -If you call [`collect`][datafusion.DataFrame.collect] (or another terminal +If you call [`collect`][datafusion.dataframe.DataFrame.collect] (or another terminal operation) multiple times on the same DataFrame, each call creates a fresh -physical plan. Metrics from [`execution_plan`][datafusion.DataFrame.execution_plan] +physical plan. Metrics from [`execution_plan`][datafusion.dataframe.DataFrame.execution_plan] always reflect the **most recent** execution. ## Reading the Physical Plan Tree -[`execution_plan`][datafusion.DataFrame.execution_plan] returns the root -[`ExecutionPlan`][datafusion.ExecutionPlan] node of the physical plan tree. The tree +[`execution_plan`][datafusion.dataframe.DataFrame.execution_plan] returns the root +[`ExecutionPlan`][datafusion.plan.ExecutionPlan] node of the physical plan tree. The tree mirrors the operator pipeline: the root is typically a projection or coalescing node; its children are filters, aggregates, scans, etc. The `operator_name` string returned by -[`collect_metrics`][datafusion.ExecutionPlan.collect_metrics] is the *display* name of +[`collect_metrics`][datafusion.plan.ExecutionPlan.collect_metrics] is the *display* name of the node, for example `"FilterExec: column1@0 > 1"`. This is the same string you would see when calling `plan.display()`. ## Aggregated vs Per-Partition Metrics DataFusion executes each operator across one or more **partitions** in -parallel. The [`MetricsSet`][datafusion.MetricsSet] convenience properties +parallel. The [`MetricsSet`][datafusion.plan.MetricsSet] convenience properties (`output_rows`, `elapsed_compute`, etc.) automatically **sum** the named metric across all partitions, giving a single aggregate value. To inspect individual partitions — for example to detect data skew where one partition processes far more rows than others — iterate over the raw -[`Metric`][datafusion.Metric] objects: +[`Metric`][datafusion.plan.Metric] objects: ```python for metric in metrics_set.metrics(): @@ -111,7 +111,7 @@ apply globally (not tied to a specific partition). ## Available Metrics The following metrics are directly accessible as properties on -[`MetricsSet`][datafusion.MetricsSet]: +[`MetricsSet`][datafusion.plan.MetricsSet]: | Property | Description | |----------|-------------| @@ -122,13 +122,13 @@ The following metrics are directly accessible as properties on | `spilled_rows` | Total rows written to disk during spill events (summed across partitions). | Any metric not listed above can be accessed via -[`sum_by_name`][datafusion.MetricsSet.sum_by_name], or by iterating over the raw -[`Metric`][datafusion.Metric] objects returned by -[`metrics`][datafusion.MetricsSet.metrics]. +[`sum_by_name`][datafusion.plan.MetricsSet.sum_by_name], or by iterating over the raw +[`Metric`][datafusion.plan.Metric] objects returned by +[`metrics`][datafusion.plan.MetricsSet.metrics]. ## Labels -A [`Metric`][datafusion.Metric] may carry *labels*: key/value pairs that +A [`Metric`][datafusion.plan.Metric] may carry *labels*: key/value pairs that provide additional context. Labels are operator-specific; most metrics have an empty label dict. @@ -143,10 +143,10 @@ for metric in metrics_set.metrics(): # output_rows {'output_type': 'intermediate'} ``` -When summing by name (via [`output_rows`][datafusion.MetricsSet.output_rows] or -[`sum_by_name`][datafusion.MetricsSet.sum_by_name]), **all** metrics with that +When summing by name (via [`output_rows`][datafusion.plan.MetricsSet.output_rows] or +[`sum_by_name`][datafusion.plan.MetricsSet.sum_by_name]), **all** metrics with that name are summed regardless of labels. To filter by label, iterate over the -raw [`Metric`][datafusion.Metric] objects directly. +raw [`Metric`][datafusion.plan.Metric] objects directly. ## End-to-End Example @@ -183,10 +183,10 @@ for operator_name, ms in plan.collect_metrics(): ## API Reference -- [`ExecutionPlan`][datafusion.ExecutionPlan] — physical plan node -- [`collect_metrics`][datafusion.ExecutionPlan.collect_metrics] — walk the tree and +- [`ExecutionPlan`][datafusion.plan.ExecutionPlan] — physical plan node +- [`collect_metrics`][datafusion.plan.ExecutionPlan.collect_metrics] — walk the tree and return `(operator_name, MetricsSet)` pairs -- [`metrics`][datafusion.ExecutionPlan.metrics] — return the - [`MetricsSet`][datafusion.MetricsSet] for a single node -- [`MetricsSet`][datafusion.MetricsSet] — aggregated metrics for one operator -- [`Metric`][datafusion.Metric] — a single per-partition metric value +- [`metrics`][datafusion.plan.ExecutionPlan.metrics] — return the + [`MetricsSet`][datafusion.plan.MetricsSet] for a single node +- [`MetricsSet`][datafusion.plan.MetricsSet] — aggregated metrics for one operator +- [`Metric`][datafusion.plan.Metric] — a single per-partition metric value diff --git a/docs/source/user-guide/dataframe/index.md b/docs/source/user-guide/dataframe/index.md index 76d46b38f..bdfe4c3f9 100644 --- a/docs/source/user-guide/dataframe/index.md +++ b/docs/source/user-guide/dataframe/index.md @@ -21,18 +21,18 @@ ## Overview -The `DataFrame` class is the core abstraction in DataFusion that represents tabular data and operations +The [`DataFrame`][datafusion.dataframe.DataFrame] class is the core abstraction in DataFusion that represents tabular data and operations on that data. DataFrames provide a flexible API for transforming data through various operations such as filtering, projection, aggregation, joining, and more. A DataFrame represents a logical plan that is lazily evaluated. The actual execution occurs only when -terminal operations like `collect()`, `show()`, or `to_pandas()` are called. +terminal operations like [`collect()`][datafusion.dataframe.DataFrame.collect], [`show()`][datafusion.dataframe.DataFrame.show], or [`to_pandas()`][datafusion.dataframe.DataFrame.to_pandas] are called. ## Creating DataFrames DataFrames can be created in several ways: -- From SQL queries via a `SessionContext`: +- From SQL queries via a [`SessionContext`][datafusion.context.SessionContext]: ```python from datafusion import SessionContext @@ -50,16 +50,16 @@ DataFrames can be created in several ways: - From various data sources: ```python - # From CSV files (see :ref:`io_csv` for detailed options) + # From CSV files (see [io_csv](/python/user-guide/io/csv/) for detailed options) df = ctx.read_csv("path/to/data.csv") - # From Parquet files (see :ref:`io_parquet` for detailed options) + # From Parquet files (see [io_parquet](/python/user-guide/io/parquet/) for detailed options) df = ctx.read_parquet("path/to/data.parquet") - # From JSON files (see :ref:`io_json` for detailed options) + # From JSON files (see [io_json](/python/user-guide/io/json/) for detailed options) df = ctx.read_json("path/to/data.json") - # From Avro files (see :ref:`io_avro` for detailed options) + # From Avro files (see [io_avro](/python/user-guide/io/avro/) for detailed options) df = ctx.read_avro("path/to/data.avro") # From Pandas DataFrame @@ -127,18 +127,18 @@ df = df.drop("temporary_column") ## Column Names as Function Arguments -Some `DataFrame` methods accept column names when an argument refers to an +Some [`DataFrame`][datafusion.dataframe.DataFrame] methods accept column names when an argument refers to an existing column. These include: -- [`select`][datafusion.DataFrame.select] -- [`sort`][datafusion.DataFrame.sort] -- [`drop`][datafusion.DataFrame.drop] -- [`join`][datafusion.DataFrame.join] (`on` argument) -- [`aggregate`][datafusion.DataFrame.aggregate] (grouping columns) +- [`select`][datafusion.dataframe.DataFrame.select] +- [`sort`][datafusion.dataframe.DataFrame.sort] +- [`drop`][datafusion.dataframe.DataFrame.drop] +- [`join`][datafusion.dataframe.DataFrame.join] (`on` argument) +- [`aggregate`][datafusion.dataframe.DataFrame.aggregate] (grouping columns) See the full function documentation for details on any specific function. -Note that [`join_on`][datafusion.DataFrame.join_on] expects `col()`/`column()` expressions rather than plain strings. +Note that [`join_on`][datafusion.dataframe.DataFrame.join_on] expects [`col()`][datafusion.col.col]/[`column()`][datafusion.col.column] expressions rather than plain strings. For such methods, you can pass column names directly: @@ -149,7 +149,7 @@ df.sort('id') df.aggregate('id', [f.count(col('value'))]) ``` -The same operation can also be written with explicit column expressions, using either `col()` or `column()`: +The same operation can also be written with explicit column expressions, using either [`col()`][datafusion.col.col] or [`column()`][datafusion.col.column]: ```python from datafusion import col, column, functions as f @@ -158,21 +158,21 @@ df.sort(col('id')) df.aggregate(column('id'), [f.count(col('value'))]) ``` -Note that `column()` is an alias of `col()`, so you can use either name; the example above shows both in action. +Note that [`column()`][datafusion.col.column] is an alias of [`col()`][datafusion.col.col], so you can use either name; the example above shows both in action. Whenever an argument represents an expression—such as in -[`filter`][datafusion.DataFrame.filter] or -[`with_column`][datafusion.DataFrame.with_column]—use `col()` to reference -columns. The comparison and arithmetic operators on `Expr` will automatically -convert any non-`Expr` value into a literal expression, so writing +[`filter`][datafusion.dataframe.DataFrame.filter] or +[`with_column`][datafusion.dataframe.DataFrame.with_column]—use [`col()`][datafusion.col.col] to reference +columns. The comparison and arithmetic operators on [`Expr`][datafusion.expr.Expr] will automatically +convert any non-[`Expr`][datafusion.expr.Expr] value into a literal expression, so writing ```python from datafusion import col df.filter(col("age") > 21) ``` -is equivalent to using `lit(21)` explicitly. Use `lit()` (also available -as `literal()`) when you need to construct a literal expression directly. +is equivalent to using `lit(21)` explicitly. Use [`lit()`][datafusion.lit] (also available +as [`literal()`][datafusion.literal]) when you need to construct a literal expression directly. ## Terminal Operations @@ -224,7 +224,7 @@ for batch in reader: ... # process each batch as it is produced ``` -DataFrames are also iterable, yielding {class}`datafusion.RecordBatch` +DataFrames are also iterable, yielding [`RecordBatch`][datafusion.RecordBatch] objects lazily so you can loop over results directly without importing PyArrow: @@ -233,7 +233,7 @@ for batch in df: ... # each batch is a ``datafusion.RecordBatch`` ``` -Each batch exposes `to_pyarrow()`, allowing conversion to a PyArrow +Each batch exposes [`to_pyarrow()`][datafusion.record_batch.RecordBatch.to_pyarrow], allowing conversion to a PyArrow table. `pa.table(df)` collects the entire DataFrame eagerly into a PyArrow table: @@ -250,8 +250,8 @@ async for batch in df: ... # process each batch as it is produced ``` -To work with the stream directly, use `execute_stream()`, which returns a -{class}`~datafusion.RecordBatchStream`. +To work with the stream directly, use [`execute_stream()`][datafusion.dataframe.DataFrame.execute_stream], which returns a +[`RecordBatchStream`][datafusion.RecordBatchStream]. ```python stream = df.execute_stream() @@ -262,8 +262,8 @@ for batch in stream: ### Execute as Stream For finer control over streaming execution, use -[`execute_stream`][datafusion.DataFrame.execute_stream] to obtain a -[`RecordBatchStream`][datafusion.RecordBatchStream]: +[`execute_stream`][datafusion.dataframe.DataFrame.execute_stream] to obtain a +[`RecordBatchStream`][datafusion.record_batch.RecordBatchStream]: ```python stream = df.execute_stream() @@ -278,8 +278,8 @@ for batch in stream: `pa.RecordBatchReader.from_stream(df)`. When partition boundaries are important, -[`execute_stream_partitioned`][datafusion.DataFrame.execute_stream_partitioned] -returns an iterable of [`RecordBatchStream`][datafusion.RecordBatchStream] objects, one per +[`execute_stream_partitioned`][datafusion.dataframe.DataFrame.execute_stream_partitioned] +returns an iterable of [`RecordBatchStream`][datafusion.record_batch.RecordBatchStream] objects, one per partition: ```python @@ -316,7 +316,7 @@ rendering, formatting options, and advanced styling, see [rendering](rendering.m : The main DataFrame class for building and executing queries. - See: [`DataFrame`][datafusion.DataFrame] + See: [`DataFrame`][datafusion.dataframe.DataFrame] **SessionContext** @@ -324,16 +324,16 @@ rendering, formatting options, and advanced styling, see [rendering](rendering.m Key methods for DataFrame creation: - - [`read_csv`][datafusion.SessionContext.read_csv] - Read CSV files - - [`read_parquet`][datafusion.SessionContext.read_parquet] - Read Parquet files - - [`read_json`][datafusion.SessionContext.read_json] - Read JSON files - - [`read_avro`][datafusion.SessionContext.read_avro] - Read Avro files - - [`table`][datafusion.SessionContext.table] - Access registered tables - - [`sql`][datafusion.SessionContext.sql] - Execute SQL queries - - [`from_pandas`][datafusion.SessionContext.from_pandas] - Create from Pandas DataFrame - - [`from_arrow`][datafusion.SessionContext.from_arrow] - Create from Arrow data + - [`read_csv`][datafusion.context.SessionContext.read_csv] - Read CSV files + - [`read_parquet`][datafusion.context.SessionContext.read_parquet] - Read Parquet files + - [`read_json`][datafusion.context.SessionContext.read_json] - Read JSON files + - [`read_avro`][datafusion.context.SessionContext.read_avro] - Read Avro files + - [`table`][datafusion.context.SessionContext.table] - Access registered tables + - [`sql`][datafusion.context.SessionContext.sql] - Execute SQL queries + - [`from_pandas`][datafusion.context.SessionContext.from_pandas] - Create from Pandas DataFrame + - [`from_arrow`][datafusion.context.SessionContext.from_arrow] - Create from Arrow data - See: [`SessionContext`][datafusion.SessionContext] + See: [`SessionContext`][datafusion.context.SessionContext] ## Expression Classes @@ -341,7 +341,7 @@ rendering, formatting options, and advanced styling, see [rendering](rendering.m : Represents expressions that can be used in DataFrame operations. - See: [`Expr`][datafusion.Expr] + See: [`Expr`][datafusion.expr.Expr] **Functions for creating expressions:** @@ -358,7 +358,7 @@ For a complete list of available functions, see the [`functions`][datafusion.fun ## Execution Metrics -After executing a DataFrame (via `collect()`, `execute_stream()`, etc.), +After executing a DataFrame (via [`collect()`][datafusion.dataframe.DataFrame.collect], [`execute_stream()`][datafusion.dataframe.DataFrame.execute_stream], etc.), DataFusion populates per-operator runtime statistics such as row counts and compute time. See [execution-metrics](execution-metrics.md) for a full explanation and worked example. diff --git a/docs/source/user-guide/distributing-work.md b/docs/source/user-guide/distributing-work.md index 6728de925..d9f03d311 100644 --- a/docs/source/user-guide/distributing-work.md +++ b/docs/source/user-guide/distributing-work.md @@ -21,7 +21,7 @@ DataFusion supports splitting work across processes by shipping serialized expressions to workers: the driver builds an -[`Expr`][datafusion.Expr], each worker evaluates it against its +[`Expr`][datafusion.expr.Expr], each worker evaluates it against its own slice of data. This pattern suits embarrassingly-parallel workloads where the driver decides partitioning up front. @@ -102,9 +102,9 @@ print(results) # [[2, 4, 6], [20, 40, 60]] captured in closures travel inside the serialized expression and are reconstructed on the worker automatically. Applies equally to: - - **scalar UDFs** ([`udf`][datafusion.udf]) - - **aggregate UDFs** ([`udaf`][datafusion.udaf]) - - **window UDFs** ([`udwf`][datafusion.udwf]) + - **scalar UDFs** ([`udf`][datafusion.user_defined.udf]) + - **aggregate UDFs** ([`udaf`][datafusion.user_defined.udaf]) + - **window UDFs** ([`udwf`][datafusion.user_defined.udwf]) - **UDFs imported via the FFI capsule protocol** — travel **by name only**. The worker must already have a matching registration on its @@ -186,12 +186,12 @@ every start method — prefer it over relying on inherited state. state is captured at serialization time. Surprises are possible if the captured state is large, mutable, or not portable to the worker's environment. See [Portability requirements for inline - Python UDFs][portability requirements for inline python udfs] for the Python-version and imported-module rules. + Python UDFs](#portability-requirements-for-inline-python-udfs) for the Python-version and imported-module rules. ### Disabling Python UDF inlining For a stricter wire format, call -[`SessionContext.with_python_udf_inlining(enabled=False)`][datafusion.SessionContext.with_python_udf_inlining] on the session +[`SessionContext.with_python_udf_inlining(enabled=False)`][datafusion.context.SessionContext.with_python_udf_inlining] on the session producing or consuming the bytes. With inlining disabled, Python UDFs travel by name only — the same way FFI-capsule UDFs do — and the receiver must have a matching registration. @@ -227,8 +227,8 @@ set_sender_ctx(SessionContext().with_python_udf_inlining(enabled=False)) Pair with a matching strict worker context ([`set_worker_ctx`][datafusion.ipc.set_worker_ctx]) so the `pickle.loads` side also refuses inline payloads. Explicit -[`Expr.to_bytes(ctx)`][Expr.to_bytes] and -[`Expr.from_bytes(blob, ctx=ctx)`][Expr.from_bytes] calls +[`Expr.to_bytes(ctx)`][datafusion.expr.Expr.to_bytes] and +[`Expr.from_bytes(blob, ctx=ctx)`][datafusion.expr.Expr.from_bytes] calls honor the supplied `ctx` directly and ignore the sender / worker contexts. diff --git a/docs/source/user-guide/io/table_provider.md b/docs/source/user-guide/io/table_provider.md index 0462ca28d..375f0b8ed 100644 --- a/docs/source/user-guide/io/table_provider.md +++ b/docs/source/user-guide/io/table_provider.md @@ -47,7 +47,7 @@ impl MyTableProvider { ``` Once you have this library available, you can construct a -[`Table`][datafusion.Table] in Python and register it with the +[`Table`][datafusion.catalog.Table] in Python and register it with the `SessionContext`. ```python diff --git a/mkdocs.yml b/mkdocs.yml index 1f149ee8b..bc3e6ab84 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -112,12 +112,15 @@ markdown_extensions: watch: - python/datafusion +hooks: + - docs/hooks.py + nav: - Home: index.ipynb - User Guide: - user-guide/index.md - Introduction: user-guide/introduction.ipynb - - Concepts: user-guide/basics.ipynb + - Concepts: user-guide/concepts.ipynb - Data Sources: user-guide/data-sources.ipynb - DataFrame: - user-guide/dataframe/index.md diff --git a/pyproject.toml b/pyproject.toml index c2f7e40b3..274ed5890 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -178,7 +178,7 @@ extend-allowed-calls = ["datafusion.lit", "lit"] "TRY", "UP", ] -"docs/*" = ["D"] +"docs/*" = ["D", "INP001"] # Notebook content cells originate from prose-driven user-guide pages # where bare `print()` calls, magic comparison values, and per-cell # re-imports are part of the explanation rather than production code. diff --git a/python/datafusion/catalog.py b/python/datafusion/catalog.py index 20da5e671..b7c5dda25 100644 --- a/python/datafusion/catalog.py +++ b/python/datafusion/catalog.py @@ -220,7 +220,7 @@ def __repr__(self) -> str: @staticmethod @deprecated("Use Table() constructor instead.") def from_dataset(dataset: pa.dataset.Dataset) -> Table: - """Turn a :mod:`pyarrow.dataset` ``Dataset`` into a :class:`Table`.""" + """Turn a `dataset` ``Dataset`` into a [`Table`][datafusion.catalog.Table].""" return Table(dataset) @property @@ -239,7 +239,7 @@ class TableProviderFactory(ABC): @abstractmethod def create(self, cmd: CreateExternalTable) -> Table: - """Create a table using the :class:`CreateExternalTable`.""" + """Create a table using the [`CreateExternalTable`][CreateExternalTable].""" ... diff --git a/python/datafusion/context.py b/python/datafusion/context.py index 6bdb68c60..c5634fe3f 100644 --- a/python/datafusion/context.py +++ b/python/datafusion/context.py @@ -20,12 +20,12 @@ A `SessionContext` holds registered tables, catalogs, and configuration for the current session. It is the first object most programs create: from it you register data, run SQL strings -([`sql`][SessionContext.sql]), read files -([`read_csv`][SessionContext.read_csv], -[`read_parquet`][SessionContext.read_parquet], ...), and construct +([`sql`][datafusion.context.SessionContext.sql]), read files +([`read_csv`][datafusion.context.SessionContext.read_csv], +[`read_parquet`][datafusion.context.SessionContext.read_parquet], ...), and construct [`DataFrame`][datafusion.dataframe.DataFrame] objects in memory -([`from_pydict`][SessionContext.from_pydict], -[`from_arrow`][SessionContext.from_arrow]). +([`from_pydict`][datafusion.context.SessionContext.from_pydict], +[`from_arrow`][datafusion.context.SessionContext.from_arrow]). Session behavior (memory limits, batch size, configured optimizer passes, ...) is controlled by [`SessionConfig`][datafusion.context.SessionConfig] and @@ -38,7 +38,7 @@ >>> ctx.sql("SELECT 1 AS n").to_pydict() {'n': [1]} -See :ref:`user_guide_concepts` in the online documentation for the broader +See user_guide_concepts in the online documentation for the broader execution model. """ @@ -530,7 +530,7 @@ def with_allow_statements(self, allow: bool = True) -> SQLOptions: class SessionContext: """This is the main interface for executing queries and creating DataFrames. - See :ref:`user_guide_concepts` in the online documentation for more information. + See user_guide_concepts in the online documentation for more information. """ def __init__( @@ -655,7 +655,7 @@ def sql( param_values: dict[str, Any] | None = None, **named_params: Any, ) -> DataFrame: - """Create a [`DataFrame`][datafusion.DataFrame] from SQL query text. + """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from SQL query text. See the online documentation for a description of how to perform parameterized substitution via either the ``param_values`` option @@ -859,14 +859,14 @@ def register_table( name: str, table: Table | TableProviderExportable | DataFrame | pa.dataset.Dataset, ) -> None: - """Register a [`Table`][datafusion.Table] with this context. + """Register a [`Table`][datafusion.catalog.Table] with this context. The registered table can be referenced from SQL statements executed against this context. Args: name: Name of the resultant table. - table: Any object that can be converted into a :class:`Table`. + table: Any object that can be converted into a `Table`. """ self.ctx.register_table(name, table) @@ -886,7 +886,7 @@ def register_table_factory( Args: format: The value to be used in `STORED AS ${format}` clause. - factory: A PyCapsule that implements :class:`TableProviderFactoryExportable` + factory: A PyCapsule that implements `TableProviderFactoryExportable` """ self.ctx.register_table_factory(format, factory) @@ -921,7 +921,7 @@ def register_table_provider( ) -> None: """Register a table provider. - Deprecated: use :meth:`register_table` instead. + Deprecated: use [`register_table`][register_table] instead. """ self.register_table(name, provider) @@ -973,12 +973,12 @@ def register_record_batches( self.ctx.register_record_batches(name, partitions) def read_batch(self, batch: pa.RecordBatch) -> DataFrame: - """Return a [`DataFrame`][datafusion.DataFrame] reading a single batch. + """Return a `DataFrame` reading a single batch. Convenience wrapper around [`read_batches`][read_batches] for the single-batch case. Unlike [`register_batch`][register_batch], this does not register the batch as a named table; it returns an anonymous - [`DataFrame`][datafusion.DataFrame] directly. + [`DataFrame`][datafusion.dataframe.DataFrame] directly. Args: batch: Record batch to wrap as a DataFrame. @@ -992,14 +992,14 @@ def read_batch(self, batch: pa.RecordBatch) -> DataFrame: return self.read_batches([batch]) def read_batches(self, batches: Iterable[pa.RecordBatch]) -> DataFrame: - """Return a [`DataFrame`][datafusion.DataFrame] reading the given batches. + """Return a `DataFrame` reading the given batches. All batches must share the same schema. Any iterable of [`RecordBatch`][pa.RecordBatch] is accepted (list, tuple, generator); it is materialized into a list before being handed to the underlying Rust binding. Unlike `register_record_batches`, this does not register the batches as a named table; it returns - an anonymous [`DataFrame`][datafusion.DataFrame] directly. + an anonymous [`DataFrame`][datafusion.dataframe.DataFrame] directly. Args: batches: Record batches to wrap as a DataFrame. @@ -1385,7 +1385,7 @@ def udaf(self, name: str) -> AggregateUDF: Examples: Look up a built-in aggregate by name and use it in - [`aggregate`][datafusion.DataFrame.aggregate]: + [`aggregate`][datafusion.dataframe.DataFrame.aggregate]: >>> ctx = dfn.SessionContext() >>> sum_fn = ctx.udaf("sum") @@ -1618,13 +1618,13 @@ def add_physical_optimizer_rule( The rule is imported via its ``__datafusion_physical_optimizer_rule__`` PyCapsule, typically produced by a separate compiled extension. The - underlying :class:`SessionState` is rebuilt from its current state + underlying [`SessionState`][SessionState] is rebuilt from its current state with the new rule appended, so previously registered tables, UDFs, and catalogs are preserved. Args: rule: Object exposing ``__datafusion_physical_optimizer_rule__``, - a :class:`PhysicalOptimizerRuleExportable`. + a [`PhysicalOptimizerRuleExportable`][PhysicalOptimizerRuleExportable]. Examples: >>> from datafusion import SessionContext @@ -1921,7 +1921,7 @@ def read_empty(self) -> DataFrame: """Create an empty `DataFrame` with no columns or rows. See Also: - This is an alias for :meth:`empty_table`. + This is an alias for [`empty_table`][empty_table]. """ return self.empty_table() @@ -1943,7 +1943,7 @@ def _convert_file_sort_order( Each ``SortKey`` can be a column name string, an ``Expr``, or a ``SortExpr`` and will be converted using - :func:`datafusion.expr.sort_list_to_raw_sort_list`. + [`sort_list_to_raw_sort_list`][datafusion.expr.sort_list_to_raw_sort_list]. """ # Convert each ``SortKey`` in the provided sort order to the low-level # representation expected by the Rust bindings. @@ -2049,24 +2049,24 @@ def with_python_udf_inlining(self, *, enabled: bool) -> SessionContext: * **Cross-language portability.** The bytes can be decoded by a non-Python receiver, which must already have UDFs registered under matching names. - * **Safer deserialization.** :meth:`Expr.from_bytes` will refuse + * **Safer deserialization.** `from_bytes` will refuse to rebuild Python UDFs rather than call ``cloudpickle.loads`` on untrusted input. - The setting affects :meth:`Expr.to_bytes` and - :meth:`Expr.from_bytes` whenever this session is passed as the - ``ctx`` argument. :func:`pickle.dumps` and :func:`pickle.loads` + The setting affects [`to_bytes`][datafusion.expr.Expr.to_bytes] and + `from_bytes` whenever this session is passed as the + ``ctx`` argument. [`dumps`][pickle.dumps] and [`loads`][pickle.loads] do not pass a context, so to apply the setting through pickle, register this session with - :func:`datafusion.ipc.set_sender_ctx` on the sender and - :func:`datafusion.ipc.set_worker_ctx` on the receiver. + [`set_sender_ctx`][datafusion.ipc.set_sender_ctx] on the sender and + [`set_worker_ctx`][datafusion.ipc.set_worker_ctx] on the receiver. .. warning:: Security - This setting narrows only :meth:`Expr.from_bytes`. Calling - :func:`pickle.loads` on untrusted bytes remains unsafe + This setting narrows only `from_bytes`. Calling + [`loads`][pickle.loads] on untrusted bytes remains unsafe regardless of the toggle. - Returns a new :class:`SessionContext` with the toggle applied; + Returns a new `SessionContext` with the toggle applied; the original session is unchanged. Examples: diff --git a/python/datafusion/dataframe.py b/python/datafusion/dataframe.py index 9baaed6b9..692d64cde 100644 --- a/python/datafusion/dataframe.py +++ b/python/datafusion/dataframe.py @@ -19,11 +19,11 @@ A `DataFrame` is a logical plan over one or more data sources. Methods that reshape the plan ([`select`][datafusion.dataframe.DataFrame.select], `filter`, `aggregate`, -[`sort`][DataFrame.sort], [`join`][DataFrame.join], +`sort`, [`join`][datafusion.dataframe.DataFrame.join], `limit`, the set-operation methods, ...) return a new `DataFrame` and do no work until a terminal method such as -[`collect`][datafusion.dataframe.DataFrame.collect], [`to_pydict`][DataFrame.to_pydict], -[`show`][DataFrame.show], or one of the ``write_*`` methods is called. +`collect`, [`to_pydict`][datafusion.dataframe.DataFrame.to_pydict], +`show`, or one of the ``write_*`` methods is called. DataFrames are produced from a [`SessionContext`][datafusion.context.SessionContext], typically via @@ -38,7 +38,7 @@ >>> df.filter(col("a") > 1).select("b").to_pydict() {'b': [20, 30]} -See :ref:`user_guide_concepts` in the online documentation for a high-level +See user_guide_concepts in the online documentation for a high-level overview of the execution model. """ @@ -92,7 +92,7 @@ class ExplainFormat(Enum): """Output format for explain plans. - Controls how the query plan is rendered in [`explain`][DataFrame.explain]. + Controls how the query plan is rendered in `explain`. """ INDENT = "indent" @@ -348,9 +348,9 @@ class DataFrame: """Two dimensional table representation of data. DataFrame objects are iterable; iterating over a DataFrame yields - :class:`datafusion.RecordBatch` instances lazily. + [`RecordBatch`][datafusion.RecordBatch] instances lazily. - See :ref:`user_guide_concepts` in the online documentation for more information. + See user_guide_concepts in the online documentation for more information. """ def __init__(self, df: DataFrameInternal) -> None: @@ -362,7 +362,7 @@ def __init__(self, df: DataFrameInternal) -> None: self.df = df def into_view(self, temporary: bool = False) -> Table: - """Convert ``DataFrame`` into a :class:`~datafusion.Table`. + """Convert ``DataFrame`` into a [`Table`][datafusion.Table]. Examples: >>> from datafusion import SessionContext @@ -644,8 +644,8 @@ def filter(self, *predicates: Expr | str) -> DataFrame: Rows for which ``predicate`` evaluates to ``False`` or ``None`` are filtered out. If more than one predicate is provided, these predicates will be combined as a logical AND. Each ``predicate`` can be an - :class:`~datafusion.expr.Expr` created using helper functions such as - :func:`datafusion.col` or :func:`datafusion.lit`, or a SQL expression string + [`Expr`][datafusion.expr.Expr] created using helper functions such as + [`col`][datafusion.col] or [`lit`][datafusion.lit], or a SQL expression string that will be parsed against the DataFrame schema. If more complex logic is required, see the logical operations in [`functions`][datafusion.functions]. @@ -696,8 +696,8 @@ def parse_sql_expr(self, expr: str) -> Expr: def with_column(self, name: str, expr: Expr | str) -> DataFrame: """Add an additional column to the DataFrame. - The ``expr`` must be an :class:`~datafusion.expr.Expr` constructed with - :func:`datafusion.col` or :func:`datafusion.lit`, or a SQL expression + The ``expr`` must be an [`Expr`][datafusion.expr.Expr] constructed with + [`col`][datafusion.col] or [`lit`][datafusion.lit], or a SQL expression string that will be parsed against the DataFrame schema. Examples: @@ -724,8 +724,8 @@ def with_columns( By passing expressions, iterables of expressions, string SQL expressions, or named expressions. - All expressions must be :class:`~datafusion.expr.Expr` objects created via - :func:`datafusion.col` or :func:`datafusion.lit`, or SQL expression strings. + All expressions must be [`Expr`][datafusion.expr.Expr] objects created via + [`col`][datafusion.col] or [`lit`][datafusion.lit], or SQL expression strings. To pass named expressions use the form ``name=Expr``. Example usage: The following will add 4 columns labeled ``a``, ``b``, ``c``, @@ -812,7 +812,7 @@ def aggregate( [`cube`][datafusion.expr.GroupingSet.cube], or [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]) as the ``group_by`` argument. See the - :ref:`aggregation` user guide for detailed examples. + aggregation user guide for detailed examples. Args: group_by: Sequence of expressions or column names to group @@ -883,7 +883,7 @@ def sort(self, *exprs: SortKey) -> DataFrame: >>> df.sort("a").to_pydict() {'a': [1, 2, 3], 'b': [20, 30, 10]} - Sort descending using [`sort`][Expr.sort]: + Sort descending using [`sort`][datafusion.expr.Expr.sort]: >>> df.sort(col("a").sort(ascending=False)).to_pydict() {'a': [3, 2, 1], 'b': [10, 30, 20]} @@ -1064,7 +1064,7 @@ def join( conjunction. When non-key columns share the same name in both DataFrames, use - [`col`][DataFrame.col] on each DataFrame **before** the join to + `col` on each DataFrame **before** the join to obtain fully qualified column references that can disambiguate them. See [`join_on`][join_on] for an example. @@ -1158,11 +1158,11 @@ def join_on( ) -> DataFrame: """Join two `DataFrame` using the specified expressions. - Join predicates must be :class:`~datafusion.expr.Expr` objects, typically - built with :func:`datafusion.col`. On expressions are used to support + Join predicates must be [`Expr`][datafusion.expr.Expr] objects, typically + built with [`col`][datafusion.col]. On expressions are used to support in-equality predicates. Equality predicates are correctly optimized. - Use [`col`][DataFrame.col] on each DataFrame **before** the join to + Use `col` on each DataFrame **before** the join to obtain fully qualified column references. These qualified references can then be used in the join predicate and to disambiguate columns with the same name when selecting from the result. @@ -1216,7 +1216,7 @@ def explain( verbose: If ``True``, more details will be included. analyze: If ``True``, the plan will run and metrics reported. format: Output format for the plan. Defaults to - [`INDENT`][ExplainFormat.INDENT]. + [`INDENT`][datafusion.dataframe.ExplainFormat.INDENT]. Examples: Show the plan in tree format: diff --git a/python/datafusion/dataframe_formatter.py b/python/datafusion/dataframe_formatter.py index fd2da99f0..8f5d21695 100644 --- a/python/datafusion/dataframe_formatter.py +++ b/python/datafusion/dataframe_formatter.py @@ -343,7 +343,7 @@ def repr_rows(self) -> int: """Get the maximum number of rows (deprecated name). .. deprecated:: - Use :attr:`max_rows` instead. This property is provided for + Use [`max_rows`][max_rows] instead. This property is provided for backward compatibility. Returns: @@ -356,7 +356,7 @@ def repr_rows(self, value: int) -> None: """Set the maximum number of rows using deprecated name. .. deprecated:: - Use :attr:`max_rows` setter instead. This property is provided for + Use [`max_rows`][max_rows] setter instead. This property is provided for backward compatibility. Args: diff --git a/python/datafusion/expr.py b/python/datafusion/expr.py index 1a3c87e96..1cd52f694 100644 --- a/python/datafusion/expr.py +++ b/python/datafusion/expr.py @@ -38,7 +38,7 @@ >>> df.select((col("a") * lit(10)).alias("ten_a")).to_pydict() {'ten_a': [10, 20, 30]} -See :ref:`expressions` in the online documentation for details on available +See expressions in the online documentation for details on available operators and helpers. """ @@ -261,12 +261,12 @@ def ensure_expr(value: Expr | Any) -> expr_internal.Expr: """Return the internal expression from ``Expr`` or raise ``TypeError``. - This helper rejects plain strings and other non-:class:`Expr` values so - higher level APIs consistently require explicit :func:`~datafusion.col` or - :func:`~datafusion.lit` expressions. + This helper rejects plain strings and other non-`Expr` values so + higher level APIs consistently require explicit [`col`][datafusion.col] or + [`lit`][datafusion.lit] expressions. See Also: - :func:`coerce_to_expr` — the opposite behavior: *wraps* non-``Expr`` + `coerce_to_expr` — the opposite behavior: *wraps* non-``Expr`` values as literals instead of rejecting them. Args: @@ -276,7 +276,7 @@ def ensure_expr(value: Expr | Any) -> expr_internal.Expr: The internal expression representation. Raises: - TypeError: If ``value`` is not an instance of :class:`Expr`. + TypeError: If ``value`` is not an instance of [`Expr`][datafusion.expr.Expr]. """ if not isinstance(value, Expr): raise TypeError(EXPR_TYPE_ERROR) @@ -295,7 +295,7 @@ def ensure_expr_list( A flat list of raw expressions. Raises: - TypeError: If any item is not an instance of :class:`Expr`. + TypeError: If any item is not an instance of [`Expr`][datafusion.expr.Expr]. """ def _iter( @@ -316,9 +316,9 @@ def _iter( def coerce_to_expr(value: Any) -> Expr: """Coerce a native Python value to an ``Expr`` literal, passing ``Expr`` through. - This is the complement of :func:`ensure_expr`: where ``ensure_expr`` + This is the complement of [`ensure_expr`][ensure_expr]: where ``ensure_expr`` *rejects* non-``Expr`` values, ``coerce_to_expr`` *wraps* them via - :meth:`Expr.literal` so that functions can accept native Python types + `literal` so that functions can accept native Python types (``int``, ``float``, ``str``, ``bool``, etc.) alongside ``Expr``. Args: @@ -335,7 +335,7 @@ def coerce_to_expr(value: Any) -> Expr: def coerce_to_expr_or_none(value: Any | None) -> Expr | None: """Coerce a value to ``Expr`` or pass ``None`` through unchanged. - Same as :func:`coerce_to_expr` but accepts ``None`` for optional parameters. + Same as `coerce_to_expr` but accepts ``None`` for optional parameters. Args: value: An ``Expr`` instance, a Python literal to wrap, or ``None``. @@ -355,10 +355,10 @@ def _to_raw_expr(value: Expr | str) -> expr_internal.Expr: value: Candidate expression or column name. Returns: - The internal :class:`~datafusion._internal.expr.Expr` representation. + The internal [`Expr`][datafusion._internal.expr.Expr] representation. Raises: - TypeError: If ``value`` is neither an :class:`Expr` nor ``str``. + TypeError: If ``value`` is neither an `Expr` nor ``str``. """ if isinstance(value, str): return Expr.column(value).expr @@ -411,7 +411,7 @@ class Expr: # noqa: PLW1641 """Expression object. Expressions are one of the core concepts in DataFusion. See - :ref:`Expressions` in the online documentation for more information. + Expressions in the online documentation for more information. """ def __init__(self, expr: expr_internal.RawExpr) -> None: @@ -443,19 +443,19 @@ def variant_name(self) -> str: def to_bytes(self, ctx: SessionContext | None = None) -> bytes: """Serialize this expression to bytes for shipping to another process. - Use this — or :func:`pickle.dumps` — to send an expression to a + Use this — or [`dumps`][pickle.dumps] — to send an expression to a worker process for distributed evaluation. When ``ctx`` is supplied, encoding routes through that session's - installed :class:`LogicalExtensionCodec` (so settings like - :meth:`SessionContext.with_python_udf_inlining` take effect). + installed [`LogicalExtensionCodec`][LogicalExtensionCodec] (so settings like + `with_python_udf_inlining` take effect). When ``ctx`` is ``None``, the default codec is used (Python UDF inlining on, no user-installed extension codec). Built-in functions travel inside the returned bytes. Python UDFs (scalar, aggregate, window) also inline by default, so the worker does not need to pre-register them; when the encoding session has - :meth:`SessionContext.with_python_udf_inlining` set to ``False``, + `with_python_udf_inlining` set to ``False``, Python UDFs travel by name only and must be registered on the worker. UDFs imported via the FFI capsule protocol always travel by name only and must be registered on the worker. @@ -463,8 +463,8 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: .. warning:: Security Bytes returned here may embed a cloudpickled Python callable (when the expression carries a Python UDF). - Reconstructing them via :meth:`from_bytes` or - :func:`pickle.loads` executes arbitrary Python on the + Reconstructing them via [`from_bytes`][datafusion.expr.Expr.from_bytes] or + [`loads`][pickle.loads] executes arbitrary Python on the receiver. Only accept payloads from trusted sources. .. warning:: Portability @@ -472,7 +472,7 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: stable across Python minor versions**. A payload produced on Python 3.11 will fail to load on Python 3.12. The wire format stamps the sender's ``(major, minor)``; - :meth:`from_bytes` raises a :class:`ValueError` naming + `from_bytes` raises a [`ValueError`][ValueError] naming both versions on mismatch. cloudpickle captures the UDF callable **by value** — @@ -526,14 +526,14 @@ def double(x): def from_bytes(cls, buf: bytes, ctx: SessionContext | None = None) -> Expr: """Reconstruct an expression from serialized bytes. - Accepts output of :meth:`to_bytes` or :func:`pickle.dumps`. - ``ctx`` is the :class:`SessionContext` used to resolve any + Accepts output of `to_bytes` or [`dumps`][pickle.dumps]. + ``ctx`` is the `SessionContext` used to resolve any function references that travel by name (e.g. FFI UDFs, or Python UDFs sent with inlining disabled via - :meth:`SessionContext.with_python_udf_inlining`). When + `with_python_udf_inlining`). When ``ctx`` is ``None`` the worker context installed via - :func:`datafusion.ipc.set_worker_ctx` is consulted; if no worker - context is installed, the global :class:`SessionContext` is used + [`set_worker_ctx`][datafusion.ipc.set_worker_ctx] is consulted; if no worker + context is installed, the global `SessionContext` is used (sufficient for built-ins and Python UDFs, plus any UDFs registered on the global context). @@ -547,9 +547,9 @@ def from_bytes(cls, buf: bytes, ctx: SessionContext | None = None) -> Expr: cloudpickle payloads are **not portable across Python minor versions**. The wire format stamps the sender's ``(major, minor)``; if it does not match the current - interpreter, this method raises :class:`ValueError` + interpreter, this method raises [`ValueError`][ValueError] naming both versions. Modules the UDF imports must also - be importable on the receiver — see :meth:`to_bytes` for + be importable on the receiver — see `to_bytes` for by-value vs. by-reference details. Examples: @@ -567,17 +567,17 @@ def __reduce__(self) -> tuple[Callable[[bytes], Expr], tuple[bytes]]: """Pickle protocol hook. Lets expressions be shipped to worker processes via - :func:`pickle.dumps` / :func:`pickle.loads`. Built-in functions + [`dumps`][pickle.dumps] / [`loads`][pickle.loads]. Built-in functions and Python UDFs (scalar, aggregate, window) travel inside the pickle bytes; only FFI-capsule UDFs require pre-registration on - the worker. The worker's :class:`SessionContext` for resolving + the worker. The worker's `SessionContext` for resolving those references is looked up via - :func:`datafusion.ipc.set_worker_ctx`, falling back to the - global :class:`SessionContext` if none has been installed on + [`set_worker_ctx`][datafusion.ipc.set_worker_ctx], falling back to the + global `SessionContext` if none has been installed on the worker. .. warning:: Security - :func:`pickle.loads` on the returned tuple executes + [`loads`][pickle.loads] on the returned tuple executes arbitrary Python on the receiver, including any cloudpickled UDF callable embedded in the payload. Only unpickle expressions from trusted sources. @@ -585,7 +585,7 @@ def __reduce__(self) -> tuple[Callable[[bytes], Expr], tuple[bytes]]: .. warning:: Portability Sender and receiver must run the same Python ``(major, minor)`` version; cloudpickle bytecode is not - portable across minor versions. See :meth:`to_bytes` for + portable across minor versions. See `to_bytes` for details on what travels by value vs. by reference. Examples: @@ -596,17 +596,17 @@ def __reduce__(self) -> tuple[Callable[[bytes], Expr], tuple[bytes]]: 'a * Int64(2)' The encoding side honors a driver-side sender context installed - via :func:`datafusion.ipc.set_sender_ctx` — that is how - :meth:`SessionContext.with_python_udf_inlining` propagates + via [`set_sender_ctx`][datafusion.ipc.set_sender_ctx] — that is how + `with_python_udf_inlining` propagates through ``pickle.dumps``. The sender context is read by - ``__reduce__``, so :func:`copy.copy` and :func:`copy.deepcopy` + ``__reduce__``, so [`copy`][copy.copy] and [`deepcopy`][copy.deepcopy] — which also go through ``__reduce__`` — pick it up too. """ return (Expr._reconstruct, (self.to_bytes(get_sender_ctx()),)) @classmethod def _reconstruct(cls, proto_bytes: bytes) -> Expr: - """Internal entry point used by :meth:`__reduce__` on unpickle. + """Internal entry point used by [`__reduce__`][__reduce__] on unpickle. Examples: >>> from datafusion import Expr, col, lit diff --git a/python/datafusion/functions.py b/python/datafusion/functions.py index c1068cacb..7891af255 100644 --- a/python/datafusion/functions.py +++ b/python/datafusion/functions.py @@ -32,7 +32,7 @@ >>> df.aggregate([], [F.sum(col("a")).alias("total")]).to_pydict() {'total': [10]} -See :ref:`aggregation` and :ref:`window_functions` in the online documentation +See aggregation and window_functions in the online documentation for categorized catalogs of aggregate and window functions. """ @@ -449,7 +449,7 @@ def array_join(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for [`array_to_string`][array_to_string]. + This is an alias for [`array_to_string`][datafusion.functions.array_to_string]. """ return array_to_string(expr, delimiter) @@ -458,7 +458,7 @@ def list_to_string(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for [`array_to_string`][array_to_string]. + This is an alias for [`array_to_string`][datafusion.functions.array_to_string]. """ return array_to_string(expr, delimiter) @@ -467,7 +467,7 @@ def list_join(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for [`array_to_string`][array_to_string]. + This is an alias for [`array_to_string`][datafusion.functions.array_to_string]. """ return array_to_string(expr, delimiter) @@ -475,9 +475,9 @@ def list_join(expr: Expr, delimiter: Expr | str) -> Expr: def lambda_var(name: str) -> Expr: """Create an unresolved reference to a lambda parameter by ``name``. - Use this inside the body passed to [`lambda_`][lambda_] to refer to one of the + Use this inside the body passed to `lambda_` to refer to one of the lambda's parameters. The owning higher-order function (such as - [`array_transform`][array_transform]) binds the variable to a concrete element type + `array_transform`) binds the variable to a concrete element type during query planning. Examples: @@ -490,7 +490,7 @@ def lambda_var(name: str) -> Expr: [2, 4, 6] See Also: - `lambda_`, `array_transform`, [`array_any_match`][array_any_match]. + `lambda_`, `array_transform`, `array_any_match`. """ return Expr(f.lambda_var(name)) @@ -506,7 +506,7 @@ def lambda_(params: list[str], body: Expr) -> Expr: Args: params: Ordered lambda parameter names. body: Body expression that references the parameters via - [`lambda_var`][lambda_var]. + [`lambda_var`][datafusion.functions.lambda_var]. Examples: >>> ctx = dfn.SessionContext() @@ -518,7 +518,7 @@ def lambda_(params: list[str], body: Expr) -> Expr: [2, 4, 6] See Also: - `lambda_var`, `array_transform`, [`array_any_match`][array_any_match]. + `lambda_var`, `array_transform`, `array_any_match`. """ return Expr(f.lambda_(params, body.expr)) @@ -526,9 +526,9 @@ def lambda_(params: list[str], body: Expr) -> Expr: def _to_lambda(fn: Expr | Callable[..., Any]) -> Expr: """Coerce ``fn`` to a lambda ``Expr``. - Accepts either an ``Expr`` produced by [`lambda_`][lambda_] (returned + Accepts either an ``Expr`` produced by `lambda_` (returned unchanged) or a Python callable. A callable is introspected for its - parameter names; those names become [`lambda_var`][lambda_var] references passed + parameter names; those names become `lambda_var` references passed positionally into the callable, and its return value (coerced to an ``Expr``) becomes the lambda body. """ @@ -550,7 +550,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: ``transform`` may be a Python callable, which is converted to a lambda automatically (its parameter names become the lambda parameters), or an - explicit lambda built with [`lambda_`][lambda_]. + explicit lambda built with [`lambda_`][datafusion.functions.lambda_]. Examples: Using a Python callable: @@ -562,7 +562,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("d")[0].as_py() [2, 4, 6] - Using an explicit lambda built with [`lambda_`][lambda_]: + Using an explicit lambda built with [`lambda_`][datafusion.functions.lambda_]: >>> double_fn = F.lambda_(["v"], F.lambda_var("v") * lit(2)) >>> df.select( @@ -571,7 +571,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: [2, 4, 6] See Also: - [`array_any_match`][array_any_match], [`lambda_`][lambda_]. + `array_any_match`, [`lambda_`][datafusion.functions.lambda_]. """ return Expr(f.array_transform(array.expr, _to_lambda(transform).expr)) @@ -580,7 +580,7 @@ def list_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: """Transform each element of a list with a lambda. See Also: - This is an alias for [`array_transform`][array_transform]. + This is an alias for [`array_transform`][datafusion.functions.array_transform]. """ return array_transform(array, transform) @@ -589,7 +589,7 @@ def array_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Return ``True`` if any element of ``array`` satisfies ``predicate``. ``predicate`` may be a Python callable, converted to a lambda - automatically, or an explicit lambda built with [`lambda_`][lambda_]. It must + automatically, or an explicit lambda built with `lambda_`. It must return a boolean expression. Examples: @@ -602,7 +602,7 @@ def array_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("m")[0].as_py() True - Using an explicit lambda built with [`lambda_`][lambda_]: + Using an explicit lambda built with [`lambda_`][datafusion.functions.lambda_]: >>> predicate = F.lambda_(["v"], F.lambda_var("v") > lit(2)) >>> df.select( @@ -611,7 +611,7 @@ def array_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: True See Also: - [`array_transform`][array_transform], [`lambda_`][lambda_]. + `array_transform`, [`lambda_`][datafusion.functions.lambda_]. """ return Expr(f.array_any_match(array.expr, _to_lambda(predicate).expr)) @@ -620,7 +620,7 @@ def any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Return ``True`` if any element of an array satisfies a predicate. See Also: - This is an alias for [`array_any_match`][array_any_match]. + This is an alias for [`array_any_match`][datafusion.functions.array_any_match]. """ return array_any_match(array, predicate) @@ -629,7 +629,7 @@ def list_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Return ``True`` if any element of a list satisfies a predicate. See Also: - This is an alias for [`array_any_match`][array_any_match]. + This is an alias for [`array_any_match`][datafusion.functions.array_any_match]. """ return array_any_match(array, predicate) @@ -638,7 +638,7 @@ def array_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Keep the elements of ``array`` for which ``predicate`` is ``True``. ``predicate`` may be a Python callable, converted to a lambda - automatically, or an explicit lambda built with [`lambda_`][lambda_]. It must + automatically, or an explicit lambda built with `lambda_`. It must return a boolean expression. The result is a new array containing only the matching elements. @@ -652,7 +652,7 @@ def array_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("f")[0].as_py() [3, 4, 5] - Using an explicit lambda built with [`lambda_`][lambda_]: + Using an explicit lambda built with [`lambda_`][datafusion.functions.lambda_]: >>> predicate = F.lambda_(["v"], F.lambda_var("v") > lit(2)) >>> df.select( @@ -661,7 +661,7 @@ def array_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: [3, 4, 5] See Also: - `array_transform`, [`array_any_match`][array_any_match], [`lambda_`][lambda_]. + `array_transform`, `array_any_match`, [`lambda_`][datafusion.functions.lambda_]. """ return Expr(f.array_filter(array.expr, _to_lambda(predicate).expr)) @@ -670,7 +670,7 @@ def list_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Keep the elements of a list for which a predicate is ``True``. See Also: - This is an alias for [`array_filter`][array_filter]. + This is an alias for [`array_filter`][datafusion.functions.array_filter]. """ return array_filter(array, predicate) @@ -1315,7 +1315,7 @@ def ifnull(x: Expr, y: Expr) -> Expr: y: Fallback expression to return when ``x`` is NULL. See Also: - This is an alias for [`nvl`][nvl]. + This is an alias for [`nvl`][datafusion.functions.nvl]. """ return nvl(x, y) @@ -1340,7 +1340,7 @@ def instr(string: Expr, substring: Expr | str) -> Expr: """Finds the position from where the ``substring`` matches the ``string``. See Also: - This is an alias for [`strpos`][strpos]. + This is an alias for [`strpos`][datafusion.functions.strpos]. """ return strpos(string, substring) @@ -1669,7 +1669,7 @@ def position(string: Expr, substring: Expr | str) -> Expr: """Finds the position from where the ``substring`` matches the ``string``. See Also: - This is an alias for [`strpos`][strpos]. + This is an alias for [`strpos`][datafusion.functions.strpos]. """ return strpos(string, substring) @@ -1694,7 +1694,7 @@ def pow(base: Expr, exponent: Expr | int | float) -> Expr: # noqa: PYI041 """Returns ``base`` raised to the power of ``exponent``. See Also: - This is an alias of [`power`][power]. + This is an alias of [`power`][datafusion.functions.power]. """ return power(base, exponent) @@ -2329,7 +2329,7 @@ def current_timestamp() -> Expr: """Returns the current timestamp in nanoseconds. See Also: - This is an alias for [`now`][now]. + This is an alias for [`now`][datafusion.functions.now]. """ return now() @@ -2361,7 +2361,7 @@ def date_format(arg: Expr, formatter: Expr | str) -> Expr: """Returns a string representation of a date, time, timestamp or duration. See Also: - This is an alias for [`to_char`][to_char]. + This is an alias for [`to_char`][datafusion.functions.to_char]. """ return to_char(arg, formatter) @@ -2446,7 +2446,7 @@ def to_timestamp(arg: Expr, *formatters: Expr) -> Expr: def to_timestamp_millis(arg: Expr, *formatters: Expr) -> Expr: """Converts a string and optional formats to a ``Timestamp`` in milliseconds. - See [`to_timestamp`][to_timestamp] for a description on how to use formatters. + See `to_timestamp` for a description on how to use formatters. Examples: >>> ctx = dfn.SessionContext() @@ -2465,7 +2465,7 @@ def to_timestamp_millis(arg: Expr, *formatters: Expr) -> Expr: def to_timestamp_micros(arg: Expr, *formatters: Expr) -> Expr: """Converts a string and optional formats to a ``Timestamp`` in microseconds. - See [`to_timestamp`][to_timestamp] for a description on how to use formatters. + See `to_timestamp` for a description on how to use formatters. Examples: >>> ctx = dfn.SessionContext() @@ -2484,7 +2484,7 @@ def to_timestamp_micros(arg: Expr, *formatters: Expr) -> Expr: def to_timestamp_nanos(arg: Expr, *formatters: Expr) -> Expr: """Converts a string and optional formats to a ``Timestamp`` in nanoseconds. - See [`to_timestamp`][to_timestamp] for a description on how to use formatters. + See `to_timestamp` for a description on how to use formatters. Examples: >>> ctx = dfn.SessionContext() @@ -2503,7 +2503,7 @@ def to_timestamp_nanos(arg: Expr, *formatters: Expr) -> Expr: def to_timestamp_seconds(arg: Expr, *formatters: Expr) -> Expr: """Converts a string and optional formats to a ``Timestamp`` in seconds. - See [`to_timestamp`][to_timestamp] for a description on how to use formatters. + See `to_timestamp` for a description on how to use formatters. Examples: >>> ctx = dfn.SessionContext() @@ -2573,7 +2573,7 @@ def datepart(part: Expr | str, date: Expr) -> Expr: """Return a specified part of a date. See Also: - This is an alias for [`date_part`][date_part]. + This is an alias for [`date_part`][datafusion.functions.date_part]. """ return date_part(part, date) @@ -2603,7 +2603,7 @@ def extract(part: Expr | str, date: Expr) -> Expr: """Extracts a subfield from the date. See Also: - This is an alias for [`date_part`][date_part]. + This is an alias for [`date_part`][datafusion.functions.date_part]. """ return date_part(part, date) @@ -2634,7 +2634,7 @@ def datetrunc(part: Expr | str, date: Expr) -> Expr: """Truncates the date to a specified level of precision. See Also: - This is an alias for [`date_trunc`][date_trunc]. + This is an alias for [`date_trunc`][datafusion.functions.date_trunc]. """ return date_trunc(part, date) @@ -2776,7 +2776,7 @@ def make_list(*args: Expr) -> Expr: """Returns an array using the specified input expressions. See Also: - This is an alias for [`make_array`][make_array]. + This is an alias for [`make_array`][datafusion.functions.make_array]. """ return make_array(*args) @@ -2785,7 +2785,7 @@ def array(*args: Expr) -> Expr: """Returns an array using the specified input expressions. See Also: - This is an alias for [`make_array`][make_array]. + This is an alias for [`make_array`][datafusion.functions.make_array]. """ return make_array(*args) @@ -3085,7 +3085,7 @@ def row(*args: Expr) -> Expr: """Returns a struct with the given arguments. See Also: - This is an alias for [`struct`][struct]. + This is an alias for [`struct`][datafusion.functions.struct]. """ return struct(*args) @@ -3124,7 +3124,7 @@ def array_push_back(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for [`array_append`][array_append]. + This is an alias for [`array_append`][datafusion.functions.array_append]. """ return array_append(array, element) @@ -3133,7 +3133,7 @@ def list_append(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for [`array_append`][array_append]. + This is an alias for [`array_append`][datafusion.functions.array_append]. """ return array_append(array, element) @@ -3142,7 +3142,7 @@ def list_push_back(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for [`array_append`][array_append]. + This is an alias for [`array_append`][datafusion.functions.array_append]. """ return array_append(array, element) @@ -3166,7 +3166,7 @@ def array_cat(*args: Expr) -> Expr: """Concatenates the input arrays. See Also: - This is an alias for [`array_concat`][array_concat]. + This is an alias for [`array_concat`][datafusion.functions.array_concat]. """ return array_concat(*args) @@ -3207,7 +3207,7 @@ def list_cat(*args: Expr) -> Expr: """Concatenates the input arrays. See Also: - This is an alias for [`array_concat`][array_concat], [`array_cat`][array_cat]. + This is an alias for `array_concat`, `array_cat`. """ return array_concat(*args) @@ -3216,7 +3216,7 @@ def list_concat(*args: Expr) -> Expr: """Concatenates the input arrays. See Also: - This is an alias for [`array_concat`][array_concat], [`array_cat`][array_cat]. + This is an alias for `array_concat`, `array_cat`. """ return array_concat(*args) @@ -3280,7 +3280,7 @@ def array_extract(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for [`array_element`][array_element]. + This is an alias for [`array_element`][datafusion.functions.array_element]. """ return array_element(array, n) @@ -3289,7 +3289,7 @@ def list_element(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for [`array_element`][array_element]. + This is an alias for [`array_element`][datafusion.functions.array_element]. """ return array_element(array, n) @@ -3298,7 +3298,7 @@ def list_extract(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for [`array_element`][array_element]. + This is an alias for [`array_element`][datafusion.functions.array_element]. """ return array_element(array, n) @@ -3377,7 +3377,7 @@ def array_contains(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for [`array_has`][array_has]. + This is an alias for [`array_has`][datafusion.functions.array_has]. """ return array_has(array, element) @@ -3386,7 +3386,7 @@ def list_has(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for [`array_has`][array_has]. + This is an alias for [`array_has`][datafusion.functions.array_has]. """ return array_has(array, element) @@ -3395,7 +3395,7 @@ def list_has_all(first_array: Expr, second_array: Expr) -> Expr: """Determines if there is complete overlap ``second_array`` in ``first_array``. See Also: - This is an alias for [`array_has_all`][array_has_all]. + This is an alias for [`array_has_all`][datafusion.functions.array_has_all]. """ return array_has_all(first_array, second_array) @@ -3404,7 +3404,7 @@ def list_has_any(first_array: Expr, second_array: Expr) -> Expr: """Determine if there is an overlap between ``first_array`` and ``second_array``. See Also: - This is an alias for [`array_has_any`][array_has_any]. + This is an alias for [`array_has_any`][datafusion.functions.array_has_any]. """ return array_has_any(first_array, second_array) @@ -3413,7 +3413,7 @@ def arrays_overlap(first_array: Expr, second_array: Expr) -> Expr: """Returns true if any element appears in both arrays. See Also: - This is an alias for [`array_has_any`][array_has_any]. + This is an alias for [`array_has_any`][datafusion.functions.array_has_any]. """ return array_has_any(first_array, second_array) @@ -3422,7 +3422,7 @@ def list_overlap(first_array: Expr, second_array: Expr) -> Expr: """Returns true if any element appears in both arrays. See Also: - This is an alias for [`array_has_any`][array_has_any]. + This is an alias for [`array_has_any`][datafusion.functions.array_has_any]. """ return array_has_any(first_array, second_array) @@ -3431,7 +3431,7 @@ def list_contains(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for [`array_has`][array_has]. + This is an alias for [`array_has`][datafusion.functions.array_has]. """ return array_has(array, element) @@ -3466,7 +3466,7 @@ def array_indexof(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for [`array_position`][array_position]. + This is an alias for [`array_position`][datafusion.functions.array_position]. """ return array_position(array, element, index) @@ -3475,7 +3475,7 @@ def list_position(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for [`array_position`][array_position]. + This is an alias for [`array_position`][datafusion.functions.array_position]. """ return array_position(array, element, index) @@ -3484,7 +3484,7 @@ def list_indexof(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for [`array_position`][array_position]. + This is an alias for [`array_position`][datafusion.functions.array_position]. """ return array_position(array, element, index) @@ -3507,7 +3507,7 @@ def list_positions(array: Expr, element: Expr) -> Expr: """Searches for an element in the array and returns all occurrences. See Also: - This is an alias for [`array_positions`][array_positions]. + This is an alias for [`array_positions`][datafusion.functions.array_positions]. """ return array_positions(array, element) @@ -3552,7 +3552,7 @@ def array_push_front(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for [`array_prepend`][array_prepend]. + This is an alias for [`array_prepend`][datafusion.functions.array_prepend]. """ return array_prepend(element, array) @@ -3561,7 +3561,7 @@ def list_prepend(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for [`array_prepend`][array_prepend]. + This is an alias for [`array_prepend`][datafusion.functions.array_prepend]. """ return array_prepend(element, array) @@ -3570,7 +3570,7 @@ def list_push_front(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for [`array_prepend`][array_prepend]. + This is an alias for [`array_prepend`][datafusion.functions.array_prepend]. """ return array_prepend(element, array) @@ -3639,7 +3639,7 @@ def list_remove(array: Expr, element: Expr) -> Expr: """Removes the first element from the array equal to the given value. See Also: - This is an alias for [`array_remove`][array_remove]. + This is an alias for [`array_remove`][datafusion.functions.array_remove]. """ return array_remove(array, element) @@ -3665,7 +3665,7 @@ def list_remove_n(array: Expr, element: Expr, max: Expr | int) -> Expr: """Removes the first ``max`` elements from the array equal to the given value. See Also: - This is an alias for [`array_remove_n`][array_remove_n]. + This is an alias for [`array_remove_n`][datafusion.functions.array_remove_n]. """ return array_remove_n(array, element, max) @@ -3690,7 +3690,7 @@ def list_remove_all(array: Expr, element: Expr) -> Expr: """Removes all elements from the array equal to the given value. See Also: - This is an alias for [`array_remove_all`][array_remove_all]. + This is an alias for `array_remove_all`. """ return array_remove_all(array, element) @@ -3714,7 +3714,7 @@ def list_repeat(element: Expr, count: Expr | int) -> Expr: """Returns an array containing ``element`` ``count`` times. See Also: - This is an alias for [`array_repeat`][array_repeat]. + This is an alias for [`array_repeat`][datafusion.functions.array_repeat]. """ return array_repeat(element, count) @@ -3738,7 +3738,7 @@ def list_replace(array: Expr, from_val: Expr, to_val: Expr) -> Expr: """Replaces the first occurrence of ``from_val`` with ``to_val``. See Also: - This is an alias for [`array_replace`][array_replace]. + This is an alias for [`array_replace`][datafusion.functions.array_replace]. """ return array_replace(array, from_val, to_val) @@ -3770,7 +3770,7 @@ def list_replace_n(array: Expr, from_val: Expr, to_val: Expr, max: Expr | int) - specified element. See Also: - This is an alias for [`array_replace_n`][array_replace_n]. + This is an alias for [`array_replace_n`][datafusion.functions.array_replace_n]. """ return array_replace_n(array, from_val, to_val, max) @@ -3794,7 +3794,7 @@ def list_replace_all(array: Expr, from_val: Expr, to_val: Expr) -> Expr: """Replaces all occurrences of ``from_val`` with ``to_val``. See Also: - This is an alias for [`array_replace_all`][array_replace_all]. + This is an alias for `array_replace_all`. """ return array_replace_all(array, from_val, to_val) @@ -3840,7 +3840,7 @@ def list_sort(array: Expr, descending: bool = False, null_first: bool = False) - """Sorts the array. See Also: - This is an alias for [`array_sort`][array_sort]. + This is an alias for [`array_sort`][datafusion.functions.array_sort]. """ return array_sort(array, descending=descending, null_first=null_first) @@ -3889,7 +3889,7 @@ def list_slice( """Returns a slice of the array. See Also: - This is an alias for [`array_slice`][array_slice]. + This is an alias for [`array_slice`][datafusion.functions.array_slice]. """ return array_slice(array, begin, end, stride) @@ -3917,7 +3917,7 @@ def list_intersect(array1: Expr, array2: Expr) -> Expr: """Returns an the intersection of ``array1`` and ``array2``. See Also: - This is an alias for [`array_intersect`][array_intersect]. + This is an alias for [`array_intersect`][datafusion.functions.array_intersect]. """ return array_intersect(array1, array2) @@ -3949,7 +3949,7 @@ def list_union(array1: Expr, array2: Expr) -> Expr: Duplicate rows will not be returned. See Also: - This is an alias for [`array_union`][array_union]. + This is an alias for [`array_union`][datafusion.functions.array_union]. """ return array_union(array1, array2) @@ -3972,7 +3972,7 @@ def list_except(array1: Expr, array2: Expr) -> Expr: """Returns the elements that appear in ``array1`` but not in the ``array2``. See Also: - This is an alias for [`array_except`][array_except]. + This is an alias for [`array_except`][datafusion.functions.array_except]. """ return array_except(array1, array2) @@ -4002,7 +4002,7 @@ def list_resize(array: Expr, size: Expr | int, value: Expr) -> Expr: filled with the given ``value``. See Also: - This is an alias for [`array_resize`][array_resize]. + This is an alias for [`array_resize`][datafusion.functions.array_resize]. """ return array_resize(array, size, value) @@ -4025,7 +4025,7 @@ def list_any_value(array: Expr) -> Expr: """Returns the first non-null element in the array. See Also: - This is an alias for [`array_any_value`][array_any_value]. + This is an alias for [`array_any_value`][datafusion.functions.array_any_value]. """ return array_any_value(array) @@ -4050,7 +4050,7 @@ def list_distance(array1: Expr, array2: Expr) -> Expr: """Returns the Euclidean distance between two numeric arrays. See Also: - This is an alias for [`array_distance`][array_distance]. + This is an alias for [`array_distance`][datafusion.functions.array_distance]. """ return array_distance(array1, array2) @@ -4073,7 +4073,7 @@ def list_max(array: Expr) -> Expr: """Returns the maximum value in the array. See Also: - This is an alias for [`array_max`][array_max]. + This is an alias for [`array_max`][datafusion.functions.array_max]. """ return array_max(array) @@ -4096,7 +4096,7 @@ def list_min(array: Expr) -> Expr: """Returns the minimum value in the array. See Also: - This is an alias for [`array_min`][array_min]. + This is an alias for [`array_min`][datafusion.functions.array_min]. """ return array_min(array) @@ -4119,7 +4119,7 @@ def list_reverse(array: Expr) -> Expr: """Reverses the order of elements in the array. See Also: - This is an alias for [`array_reverse`][array_reverse]. + This is an alias for [`array_reverse`][datafusion.functions.array_reverse]. """ return array_reverse(array) @@ -4143,7 +4143,7 @@ def list_zip(*arrays: Expr) -> Expr: """Combines multiple arrays into a single array of structs. See Also: - This is an alias for [`arrays_zip`][arrays_zip]. + This is an alias for [`arrays_zip`][datafusion.functions.arrays_zip]. """ return arrays_zip(*arrays) @@ -4189,7 +4189,7 @@ def string_to_list( """Splits a string based on a delimiter and returns an array of parts. See Also: - This is an alias for [`string_to_array`][string_to_array]. + This is an alias for [`string_to_array`][datafusion.functions.string_to_array]. """ return string_to_array(string, delimiter, null_string) @@ -4197,7 +4197,7 @@ def string_to_list( def gen_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: """Creates a list of values in the range between start and stop. - Unlike [`range`][range], this includes the upper bound. + Unlike [`range`][datafusion.functions.range], this includes the upper bound. Examples: >>> ctx = dfn.SessionContext() @@ -4225,10 +4225,10 @@ def gen_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: def generate_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: """Creates a list of values in the range between start and stop. - Unlike [`range`][range], this includes the upper bound. + Unlike [`range`][datafusion.functions.range], this includes the upper bound. See Also: - This is an alias for [`gen_series`][gen_series]. + This is an alias for [`gen_series`][datafusion.functions.gen_series]. """ return gen_series(start, stop, step) @@ -4415,7 +4415,7 @@ def element_at(map: Expr, key: Expr) -> Expr: Returns ``[None]`` if the key is absent. See Also: - This is an alias for [`map_extract`][map_extract]. + This is an alias for [`map_extract`][datafusion.functions.map_extract]. """ return map_extract(map, key) @@ -4427,9 +4427,9 @@ def approx_distinct( ) -> Expr: """Returns the approximate number of distinct values. - This aggregate function is similar to [`count`][count] with distinct set, but it + This aggregate function is similar to `count` with distinct set, but it will approximate the number of distinct entries. It may return significantly faster - than [`count`][count] for some DataFrames. + than [`count`][datafusion.functions.count] for some DataFrames. If using the builder functions described in ref:`_aggregation` this function ignores the options ``order_by``, ``null_treatment``, and ``distinct``. @@ -4464,7 +4464,7 @@ def approx_distinct( def approx_median(expression: Expr, filter: Expr | None = None) -> Expr: """Returns the approximate median value. - This aggregate function is similar to [`median`][median], but it will only + This aggregate function is similar to `median`, but it will only approximate the median. It may return significantly faster for some DataFrames. If using the builder functions described in ref:`_aggregation` this function ignores @@ -4654,7 +4654,7 @@ def quantile_cont( """Computes the exact percentile of input values using continuous interpolation. See Also: - This is an alias for [`percentile_cont`][percentile_cont]. + This is an alias for [`percentile_cont`][datafusion.functions.percentile_cont]. """ return percentile_cont(sort_expression, percentile, filter) @@ -4668,7 +4668,7 @@ def array_agg( """Aggregate values into an array. Currently ``distinct`` and ``order_by`` cannot be used together. As a work around, - consider [`array_sort`][array_sort] after aggregation. + consider [`array_sort`][datafusion.functions.array_sort] after aggregation. [Issue Tracker](https://github.com/apache/datafusion/issues/12371) If using the builder functions described in ref:`_aggregation` this function ignores @@ -4986,7 +4986,7 @@ def covar(value_y: Expr, value_x: Expr, filter: Expr | None = None) -> Expr: """Computes the sample covariance. See Also: - This is an alias for [`covar_samp`][covar_samp]. + This is an alias for [`covar_samp`][datafusion.functions.covar_samp]. """ return covar_samp(value_y, value_x, filter) @@ -5027,7 +5027,7 @@ def mean(expression: Expr, filter: Expr | None = None) -> Expr: """Returns the average (mean) value of the argument. See Also: - This is an alias for [`avg`][avg]. + This is an alias for [`avg`][datafusion.functions.avg]. """ return avg(expression, filter) @@ -5221,7 +5221,7 @@ def stddev_samp(arg: Expr, filter: Expr | None = None) -> Expr: """Computes the sample standard deviation of the argument. See Also: - This is an alias for [`stddev`][stddev]. + This is an alias for [`stddev`][datafusion.functions.stddev]. """ return stddev(arg, filter=filter) @@ -5230,7 +5230,7 @@ def var(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the sample variance of the argument. See Also: - This is an alias for [`var_samp`][var_samp]. + This is an alias for [`var_samp`][datafusion.functions.var_samp]. """ return var_samp(expression, filter) @@ -5271,7 +5271,7 @@ def var_population(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the population variance of the argument. See Also: - This is an alias for [`var_pop`][var_pop]. + This is an alias for [`var_pop`][datafusion.functions.var_pop]. """ return var_pop(expression, filter) @@ -5312,7 +5312,7 @@ def var_sample(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the sample variance of the argument. See Also: - This is an alias for [`var_samp`][var_samp]. + This is an alias for [`var_samp`][datafusion.functions.var_samp]. """ return var_samp(expression, filter) @@ -6321,7 +6321,7 @@ def dense_rank( ) -> Expr: """Create a dense_rank window function. - This window function is similar to [`rank`][rank] except that the returned values + This window function is similar to `rank` except that the returned values will be consecutive. Here is an example of a dataframe with a window ordered by descending ``points`` and the associated dense rank:: @@ -6377,7 +6377,7 @@ def percent_rank( ) -> Expr: """Create a percent_rank window function. - This window function is similar to [`rank`][rank] except that the returned values + This window function is similar to `rank` except that the returned values are the percentage from 0.0 to 1.0 from first to last. Here is an example of a dataframe with a window ordered by descending ``points`` and the associated percent rank:: @@ -6435,7 +6435,7 @@ def cume_dist( ) -> Expr: """Create a cumulative distribution window function. - This window function is similar to [`rank`][rank] except that the returned values + This window function is similar to `rank` except that the returned values are the ratio of the row number to the total number of rows. Here is an example of a dataframe with a window ordered by descending ``points`` and the associated cumulative distribution:: diff --git a/python/datafusion/ipc.py b/python/datafusion/ipc.py index 487abd4c3..ed6389fca 100644 --- a/python/datafusion/ipc.py +++ b/python/datafusion/ipc.py @@ -17,12 +17,12 @@ """Driver- and worker-side setup for distributing DataFusion expressions. -When a :class:`Expr` is shipped to a worker process (e.g. through -:func:`multiprocessing.Pool` or a Ray actor), the worker reconstructs the -expression against a :class:`SessionContext`. If the expression references +When a [`Expr`][datafusion.expr.Expr] is shipped to a worker process (e.g. through +[`Pool`][multiprocessing.Pool] or a Ray actor), the worker reconstructs the +expression against a `SessionContext`. If the expression references UDFs imported via the FFI capsule protocol — or any UDF the worker would otherwise resolve from its registered functions rather than from inside -the shipped expression — install a configured :class:`SessionContext` +the shipped expression — install a configured `SessionContext` once per worker: .. code-block:: python @@ -42,21 +42,21 @@ def init_worker(): .. note:: Serialization model Expressions containing Python UDFs (scalar, aggregate, window) are - serialized using :mod:`cloudpickle`. The callable itself travels + serialized using [`cloudpickle`][cloudpickle]. The callable itself travels **by value** (bytecode and closure cells inlined), but any names the callable resolves via ``import`` are captured **by reference** and must be importable on the receiving worker. The serialized payload is stamped with the sender's Python ``(major, minor)`` version. Loading on a different minor version - raises :class:`ValueError` with an actionable message — cloudpickle + raises [`ValueError`][ValueError] with an actionable message — cloudpickle payloads are not portable across Python minor versions. See - :meth:`datafusion.Expr.to_bytes` for examples of what travels by + [`to_bytes`][datafusion.Expr.to_bytes] for examples of what travels by value vs. by reference. -On the driver side, call :func:`set_sender_ctx` to control how -:func:`pickle.dumps` encodes expressions — for example, to apply -:meth:`SessionContext.with_python_udf_inlining` to every pickled +On the driver side, call [`set_sender_ctx`][set_sender_ctx] to control how +[`dumps`][pickle.dumps] encodes expressions — for example, to apply +`with_python_udf_inlining` to every pickled expression on this thread: >>> import pickle @@ -77,11 +77,11 @@ def init_worker(): ``ctx``. The thread-local sender context holds a strong reference to the -installed :class:`SessionContext` until :func:`clear_sender_ctx` is +installed `SessionContext` until [`clear_sender_ctx`][clear_sender_ctx] is called or the thread exits. Long-running driver threads that install a sender context once and never clear it will retain that session for the -lifetime of the thread; pair :func:`set_sender_ctx` with -:func:`clear_sender_ctx` (e.g. in a ``try``/``finally``) when the +lifetime of the thread; pair [`set_sender_ctx`][set_sender_ctx] with +[`clear_sender_ctx`][clear_sender_ctx] (e.g. in a ``try``/``finally``) when the sender context is only needed for a bounded scope. """ @@ -108,7 +108,7 @@ def init_worker(): def set_worker_ctx(ctx: SessionContext) -> None: - """Install this worker's :class:`SessionContext` for shipped expressions. + """Install this worker's `SessionContext` for shipped expressions. Call once per worker — typically from a ``multiprocessing.Pool`` initializer or a Ray actor ``__init__``. Idempotent: overwrites any @@ -127,10 +127,10 @@ def set_worker_ctx(ctx: SessionContext) -> None: def clear_worker_ctx() -> None: - """Remove this worker's installed :class:`SessionContext`. + """Remove this worker's installed `SessionContext`. After clearing, expressions reconstructed in this worker fall back to - the global :class:`SessionContext` — adequate for built-ins and Python + the global `SessionContext` — adequate for built-ins and Python UDFs (scalar, aggregate, window), but anything imported via the FFI capsule protocol must be registered on the global context to resolve. @@ -147,7 +147,7 @@ def clear_worker_ctx() -> None: def get_worker_ctx() -> SessionContext | None: - """Return this worker's installed :class:`SessionContext`, or ``None``. + """Return this worker's installed `SessionContext`, or ``None``. Examples: >>> from datafusion.ipc import get_worker_ctx, clear_worker_ctx @@ -159,18 +159,18 @@ def get_worker_ctx() -> SessionContext | None: def set_sender_ctx(ctx: SessionContext) -> None: - """Install this driver's :class:`SessionContext` for outbound pickles. + """Install this driver's `SessionContext` for outbound pickles. - Controls how :func:`pickle.dumps` encodes :class:`Expr` instances on + Controls how `dumps` encodes [`Expr`][datafusion.expr.Expr] instances on this thread. The most useful application is propagating a session configured with - :meth:`SessionContext.with_python_udf_inlining` so the toggle takes + `with_python_udf_inlining` so the toggle takes effect through pickle (which otherwise calls - :meth:`Expr.to_bytes` with no context and uses the default codec). + `to_bytes` with no context and uses the default codec). Idempotent: overwrites any previous value. Stored in a thread-local slot, so worker threads on the driver may install their own contexts. - Does not affect :meth:`Expr.to_bytes` calls that pass an explicit + Does not affect `to_bytes` calls that pass an explicit ``ctx`` — those continue to use the supplied context. Examples: @@ -185,7 +185,7 @@ def set_sender_ctx(ctx: SessionContext) -> None: def clear_sender_ctx() -> None: - """Remove this driver's installed sender :class:`SessionContext`. + """Remove this driver's installed sender `SessionContext`. After clearing, pickled expressions fall back to the default codec (Python UDF inlining on). @@ -205,7 +205,7 @@ def clear_sender_ctx() -> None: def get_sender_ctx() -> SessionContext | None: - """Return this driver's installed sender :class:`SessionContext`, or ``None``. + """Return this driver's installed sender `SessionContext`, or ``None``. Examples: >>> from datafusion.ipc import get_sender_ctx, clear_sender_ctx @@ -222,7 +222,7 @@ def _resolve_ctx( """Resolve a context for Expr reconstruction. Priority: explicit argument > worker context > global context. - Falling back to the global :class:`SessionContext` (instead of a + Falling back to the global `SessionContext` (instead of a freshly constructed one) preserves any registrations the user has installed on it. diff --git a/python/datafusion/plan.py b/python/datafusion/plan.py index 9cb4b35f0..592bdbc97 100644 --- a/python/datafusion/plan.py +++ b/python/datafusion/plan.py @@ -113,7 +113,7 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: @staticmethod def from_proto(ctx: SessionContext, data: bytes) -> LogicalPlan: - """Deprecated alias for :meth:`from_bytes`.""" + """Deprecated alias for [`from_bytes`][datafusion.expr.Expr.from_bytes].""" warnings.warn( "LogicalPlan.from_proto is deprecated; use from_bytes instead", DeprecationWarning, @@ -122,7 +122,7 @@ def from_proto(ctx: SessionContext, data: bytes) -> LogicalPlan: return LogicalPlan.from_bytes(ctx, data) def to_proto(self) -> bytes: - """Deprecated alias for :meth:`to_bytes`.""" + """Deprecated alias for [`to_bytes`][datafusion.expr.Expr.to_bytes].""" warnings.warn( "LogicalPlan.to_proto is deprecated; use to_bytes instead", DeprecationWarning, @@ -191,7 +191,7 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: @staticmethod def from_proto(ctx: SessionContext, data: bytes) -> ExecutionPlan: - """Deprecated alias for :meth:`from_bytes`.""" + """Deprecated alias for [`from_bytes`][datafusion.expr.Expr.from_bytes].""" warnings.warn( "ExecutionPlan.from_proto is deprecated; use from_bytes instead", DeprecationWarning, @@ -200,7 +200,7 @@ def from_proto(ctx: SessionContext, data: bytes) -> ExecutionPlan: return ExecutionPlan.from_bytes(ctx, data) def to_proto(self) -> bytes: - """Deprecated alias for :meth:`to_bytes`.""" + """Deprecated alias for [`to_bytes`][datafusion.expr.Expr.to_bytes].""" warnings.warn( "ExecutionPlan.to_proto is deprecated; use to_bytes instead", DeprecationWarning, @@ -227,8 +227,8 @@ def collect_metrics(self) -> list[tuple[str, MetricsSet]]: DataFusion executes a query as a pipeline of operators — for example a data source scan, followed by a filter, followed by a projection. After the DataFrame has been executed (via - [`collect`][datafusion.DataFrame.collect], - [`execute_stream`][datafusion.DataFrame.execute_stream], etc.), each operator + [`collect`][datafusion.dataframe.DataFrame.collect], + `execute_stream`, etc.), each operator records statistics such as how many rows it produced and how much CPU time it consumed. diff --git a/python/datafusion/substrait.py b/python/datafusion/substrait.py index 74aa3bd36..ffc6d10bb 100644 --- a/python/datafusion/substrait.py +++ b/python/datafusion/substrait.py @@ -49,7 +49,7 @@ def __init__(self, plan: substrait_internal.Plan) -> None: """Create a substrait plan. The user should not have to call this constructor directly. Rather, it - should be created via [`Serde`][Serde] or py:class:`Producer` classes + should be created via [`Serde`][Serde] or py[`Producer`][Producer] classes in this module. """ self.plan_internal = plan diff --git a/python/datafusion/user_defined.py b/python/datafusion/user_defined.py index 18f2bb014..07cb29349 100644 --- a/python/datafusion/user_defined.py +++ b/python/datafusion/user_defined.py @@ -157,7 +157,7 @@ def __init__( def _from_internal(cls, internal: df_internal.ScalarUDF) -> ScalarUDF: """Wrap an already-constructed internal ``ScalarUDF`` handle. - Used by [`udf`][SessionContext.udf] to surface a function looked + Used by `udf` to surface a function looked up from the session's function registry without re-running [`__init__`][__init__]. """ @@ -469,7 +469,7 @@ def __init__( def _from_internal(cls, internal: df_internal.AggregateUDF) -> AggregateUDF: """Wrap an already-constructed internal ``AggregateUDF`` handle. - Used by [`udaf`][SessionContext.udaf] to surface a function looked + Used by `udaf` to surface a function looked up from the session's function registry without re-running [`__init__`][__init__]. """ @@ -901,7 +901,7 @@ def __init__( def _from_internal(cls, internal: df_internal.WindowUDF) -> WindowUDF: """Wrap an already-constructed internal ``WindowUDF`` handle. - Used by [`udwf`][SessionContext.udwf] to surface a function looked + Used by `udwf` to surface a function looked up from the session's function registry without re-running [`__init__`][__init__]. """ @@ -1107,7 +1107,7 @@ def _wrap_session_kwarg_for_udtf(func: Callable[..., Any]) -> Callable[..., Any] The Rust call site forwards a ``datafusion._internal.SessionContext``, but UDTF authors expect to interact with the public - :class:`datafusion.SessionContext` wrapper. This closure wraps the + [`SessionContext`][datafusion.SessionContext] wrapper. This closure wraps the internal object once per call before delegating to ``func``. """ @@ -1138,7 +1138,7 @@ def __init__( """Instantiate a user-defined table function (UDTF). Set ``with_session=True`` to have the calling - :class:`SessionContext` passed as a ``session`` keyword argument + `SessionContext` passed as a ``session`` keyword argument on each invocation. Use it inside the callback to look up registered tables, UDFs, or session configuration. When ``with_session`` is ``False`` (the default), ``func`` is invoked @@ -1147,11 +1147,11 @@ def __init__( ``with_session=True`` is only supported for pure-Python callables. Passing it together with an FFI-exported table function (one exposing ``__datafusion_table_function__``) raises - :class:`TypeError`. + [`TypeError`][TypeError]. Registry mutations performed through the injected session (such as registering tables or UDFs) propagate to the caller's - :class:`SessionContext` because the registries are shared. + `SessionContext` because the registries are shared. Configuration changes do **not** propagate; the wrapper holds its own clone of the session config. @@ -1194,7 +1194,7 @@ def udtf(*args: Any, with_session: bool = False, **kwargs: Any): """Create a new User-Defined Table Function (UDTF). Pass ``with_session=True`` to have the calling - :class:`SessionContext` injected as a ``session`` keyword + `SessionContext` injected as a ``session`` keyword argument on each invocation. """ if args and callable(args[0]): From e2d0d917e3c3bb4a1f55daf9aaa4ee62a9148cb8 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Sun, 7 Jun 2026 20:27:06 +0200 Subject: [PATCH 07/18] docs: polish links, navigation, formatter coverage, admonitions * Replace stale Sphinx `{toctree}` blocks in every section index with proper Markdown lists or tables, and drop the hidden root toctree from `index.ipynb` (the sidebar nav covers it). * `user-guide/io/index.md`: rewrite as a per-format table linking to each reader page and the corresponding `SessionContext.read_*` method. * Rename `basics.ipynb` -> `concepts.ipynb` to match the page H1. * Fix on-page `read_csv` mention that pointed at `datafusion.io.read_csv`; it now targets `SessionContext.read_csv` since the page calls it as `ctx.read_csv`. * Auto-link plain-code mentions of `SessionContext` methods (`from_pydict`, `from_pylist`, `from_arrow`, `from_polars`, `create_dataframe`, `register_*`, ...) across user-guide pages and notebooks. * Resolve bare-anchor links (`[Parquet](io_parquet)` and `:ref:` artifacts) to real relative paths from each source file's location. * Rewrite `[X][datafusion.col]` / `[X][datafusion.column]` to canonical anchors `datafusion.col.col` / `datafusion.col.column`. * Link the second `deltalake` mention on the data-sources page. * `reference/catalog.md`: add the 8 catalog/schema base classes that were referenced by user-guide but missing from the reference tree. * New `reference/formatter.md` covering the full `datafusion.dataframe_formatter` module. Update `dataframe.md` to cross-link instead of duplicating `configure_formatter`. * `user-guide/dataframe/rendering.md`: link formatter symbols inline; remove the unrelated "Additional Resources" section. * Notebook admonitions: rewrite `!!! note` / `!!! warning` / `!!! tip` in notebook markdown cells to `

` HTML so mistune passes them through and Material's CSS styles them. * Reorder Common Operations nav so Basic Info comes before Views. * Notebook setup cell now calls `configure_formatter(max_rows=10, show_truncation_message=False)` so rendered DataFrame output stays compact and free of "Data truncated" banners. * Collapse remaining long markdown links inside docstrings to inline code so the 88-char ruff limit holds. Co-Authored-By: Claude Opus 4.7 --- docs/source/contributor-guide/ffi.md | 6 +-- docs/source/contributor-guide/index.md | 11 ++--- docs/source/index.ipynb | 7 +++- docs/source/links.md | 12 ++---- docs/source/reference/catalog.md | 32 +++++++++++++++ docs/source/reference/dataframe.md | 8 +++- docs/source/reference/formatter.md | 41 +++++++++++++++++++ .../common-operations/aggregations.ipynb | 19 +++++---- .../common-operations/basic-info.ipynb | 5 ++- .../common-operations/expressions.ipynb | 15 ++++--- .../common-operations/functions.ipynb | 5 ++- .../user-guide/common-operations/index.md | 28 +++++++------ .../user-guide/common-operations/joins.ipynb | 5 ++- .../common-operations/select-and-filter.ipynb | 9 ++-- .../common-operations/udf-and-udfa.ipynb | 11 +++-- .../user-guide/common-operations/views.md | 2 +- .../common-operations/windows.ipynb | 11 +++-- docs/source/user-guide/concepts.ipynb | 7 +++- docs/source/user-guide/data-sources.ipynb | 11 +++-- docs/source/user-guide/dataframe/index.md | 13 +++--- docs/source/user-guide/dataframe/rendering.md | 22 +++------- docs/source/user-guide/index.md | 37 ++++++++++------- docs/source/user-guide/introduction.ipynb | 10 ++++- docs/source/user-guide/io/arrow.ipynb | 5 ++- docs/source/user-guide/io/index.md | 34 ++++++++++----- docs/source/user-guide/sql.ipynb | 7 +++- mkdocs.yml | 4 +- python/datafusion/context.py | 8 ++-- python/datafusion/dataframe.py | 8 ++-- python/datafusion/expr.py | 4 +- python/datafusion/record_batch.py | 4 +- 31 files changed, 269 insertions(+), 132 deletions(-) create mode 100644 docs/source/reference/formatter.md diff --git a/docs/source/contributor-guide/ffi.md b/docs/source/contributor-guide/ffi.md index def0b906f..a8e9c6575 100644 --- a/docs/source/contributor-guide/ffi.md +++ b/docs/source/contributor-guide/ffi.md @@ -28,9 +28,9 @@ when doing these integrations and the approach our project uses. ## The Primary Issue Suppose you wish to use DataFusion and you have a custom data source that can produce tables that -can then be queried against, similar to how you can register a [CSV](io_csv) or -[Parquet](io_parquet) file. In DataFusion terminology, you likely want to implement a -[Custom Table Provider](io_custom_table_provider). In an effort to make your data source +can then be queried against, similar to how you can register a [CSV](../../user-guide/io/csv/) or +[Parquet](../../user-guide/io/parquet/) file. In DataFusion terminology, you likely want to implement a +[Custom Table Provider](../../user-guide/io/table_provider/). In an effort to make your data source as performant as possible and to utilize the features of DataFusion, you may decide to write your source in Rust and then expose it through [PyO3](https://pyo3.rs) as a Python library. diff --git a/docs/source/contributor-guide/index.md b/docs/source/contributor-guide/index.md index a989a068d..6fcd86017 100644 --- a/docs/source/contributor-guide/index.md +++ b/docs/source/contributor-guide/index.md @@ -21,9 +21,10 @@ Guides for contributors to the DataFusion in Python project. -```{toctree} -:maxdepth: 2 +## Contents -introduction -ffi -``` +- [Introduction](introduction.md) — workflow, code layout, how to run + the test suite, how PRs are reviewed. +- [FFI](ffi.md) — exposing Rust-backed extensions through the Foreign + Function Interface so they appear as first-class DataFusion symbols + in Python. diff --git a/docs/source/index.ipynb b/docs/source/index.ipynb index c6cf06041..0d43d6a30 100644 --- a/docs/source/index.ipynb +++ b/docs/source/index.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { @@ -57,7 +60,7 @@ "cell_type": "markdown", "id": "8dd0d8092fe74a7c96281538738b07e2", "metadata": {}, - "source": "\n```{toctree}\n:hidden: true\n:maxdepth: 1\n\nuser-guide/index\ncontributor-guide/index\nAPI Reference \nlinks\n```\n" + "source": "\n" } ], "metadata": { diff --git a/docs/source/links.md b/docs/source/links.md index 9ad3ff305..c7dc360c1 100644 --- a/docs/source/links.md +++ b/docs/source/links.md @@ -21,11 +21,7 @@ External resources for the DataFusion in Python project. -```{toctree} -:maxdepth: 1 - -GitHub and Issue Tracker -Rust API Docs -Code of Conduct -Examples -``` +- [GitHub and Issue Tracker](https://github.com/apache/datafusion-python) +- [Rust API Docs](https://docs.rs/datafusion/latest/datafusion/) +- [Code of Conduct](https://github.com/apache/datafusion/blob/main/CODE_OF_CONDUCT.md) +- [Examples](https://github.com/apache/datafusion-python/tree/main/examples) diff --git a/docs/source/reference/catalog.md b/docs/source/reference/catalog.md index 04b6e6680..9590cf50f 100644 --- a/docs/source/reference/catalog.md +++ b/docs/source/reference/catalog.md @@ -4,6 +4,38 @@ ::: datafusion.catalog.Catalog +## CatalogProvider + +::: datafusion.catalog.CatalogProvider + +## CatalogProviderExportable + +::: datafusion.catalog.CatalogProviderExportable + +## CatalogList + +::: datafusion.catalog.CatalogList + +## CatalogProviderList + +::: datafusion.catalog.CatalogProviderList + +## CatalogProviderListExportable + +::: datafusion.catalog.CatalogProviderListExportable + +## Schema + +::: datafusion.catalog.Schema + +## SchemaProvider + +::: datafusion.catalog.SchemaProvider + +## SchemaProviderExportable + +::: datafusion.catalog.SchemaProviderExportable + ## Table ::: datafusion.catalog.Table diff --git a/docs/source/reference/dataframe.md b/docs/source/reference/dataframe.md index c278ec538..dcd093f33 100644 --- a/docs/source/reference/dataframe.md +++ b/docs/source/reference/dataframe.md @@ -28,6 +28,10 @@ ::: datafusion.dataframe.ExplainFormat -## configure_formatter +## DataFrame Formatter -::: datafusion.dataframe_formatter.configure_formatter +See [DataFrame Formatter](formatter.md) for the full formatter API +([`configure_formatter`][datafusion.dataframe_formatter.configure_formatter], +[`DataFrameHtmlFormatter`][datafusion.dataframe_formatter.DataFrameHtmlFormatter], +[`CellFormatter`][datafusion.dataframe_formatter.CellFormatter], +[`StyleProvider`][datafusion.dataframe_formatter.StyleProvider], etc.). diff --git a/docs/source/reference/formatter.md b/docs/source/reference/formatter.md new file mode 100644 index 000000000..24a0f1ba0 --- /dev/null +++ b/docs/source/reference/formatter.md @@ -0,0 +1,41 @@ +# DataFrame Formatter + +The `datafusion.dataframe_formatter` module controls how DataFrames render +in notebooks and HTML contexts. See the user-guide +[Rendering](../user-guide/dataframe/rendering.md) page for worked examples. + +## configure_formatter + +::: datafusion.dataframe_formatter.configure_formatter + +## get_formatter + +::: datafusion.dataframe_formatter.get_formatter + +## set_formatter + +::: datafusion.dataframe_formatter.set_formatter + +## reset_formatter + +::: datafusion.dataframe_formatter.reset_formatter + +## DataFrameHtmlFormatter + +::: datafusion.dataframe_formatter.DataFrameHtmlFormatter + +## CellFormatter + +::: datafusion.dataframe_formatter.CellFormatter + +## StyleProvider + +::: datafusion.dataframe_formatter.StyleProvider + +## DefaultStyleProvider + +::: datafusion.dataframe_formatter.DefaultStyleProvider + +## FormatterManager + +::: datafusion.dataframe_formatter.FormatterManager diff --git a/docs/source/user-guide/common-operations/aggregations.ipynb b/docs/source/user-guide/common-operations/aggregations.ipynb index 3b53f31d8..039284b7f 100644 --- a/docs/source/user-guide/common-operations/aggregations.ipynb +++ b/docs/source/user-guide/common-operations/aggregations.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { @@ -257,7 +260,7 @@ "cell_type": "markdown", "id": "3ed186c9a28b402fb0bc4494df01f08d", "metadata": {}, - "source": "\n### Comparing subsets within a group\n\nSometimes you need to compare the full membership of a group against a\nsubset that meets some condition \u2014 for example, \"which groups have at least\none failure, but not every member failed?\". The `filter` argument on an\naggregate restricts the rows that contribute to *that* aggregate without\ndropping the group, so a single pass can produce both the full set and the\nfiltered subset side by side. Pairing\n[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and\n`filter=` is a compact way to express this: collect the distinct values\nof the group, collect the distinct values that satisfy the condition, then\ncompare the two arrays.\n\nSuppose each row records a line item with the supplier that fulfilled it and\na flag for whether that supplier met the commit date. We want to identify\n*partially failed* orders \u2014 orders where at least one supplier failed but\nnot every supplier failed:\n\n" + "source": "\n### Comparing subsets within a group\n\nSometimes you need to compare the full membership of a group against a\nsubset that meets some condition — for example, \"which groups have at least\none failure, but not every member failed?\". The `filter` argument on an\naggregate restricts the rows that contribute to *that* aggregate without\ndropping the group, so a single pass can produce both the full set and the\nfiltered subset side by side. Pairing\n[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and\n`filter=` is a compact way to express this: collect the distinct values\nof the group, collect the distinct values that satisfy the condition, then\ncompare the two arrays.\n\nSuppose each row records a line item with the supplier that fulfilled it and\na flag for whether that supplier met the commit date. We want to identify\n*partially failed* orders — orders where at least one supplier failed but\nnot every supplier failed:\n\n" }, { "cell_type": "code", @@ -296,7 +299,7 @@ "cell_type": "markdown", "id": "379cbbc1e968416e875cc15c1202d7eb", "metadata": {}, - "source": "\nOrder 1 is partial (one of three suppliers failed). Order 2 is excluded\nbecause no supplier failed, order 3 because its only supplier failed, and\norder 4 because both of its suppliers failed.\n\n## Grouping Sets\n\nThe default style of aggregation produces one row per group. Sometimes you want a single query to\nproduce rows at multiple levels of detail \u2014 for example, totals per type *and* an overall grand\ntotal, or subtotals for every combination of two columns plus the individual column totals. Writing\nseparate queries and concatenating them is tedious and runs the data multiple times. Grouping sets\nsolve this by letting you specify several grouping levels in one pass.\n\nDataFusion supports three grouping set styles through the\n[`GroupingSet`][datafusion.expr.GroupingSet] class:\n\n- [`rollup`][datafusion.expr.GroupingSet.rollup] \u2014 hierarchical subtotals, like a drill-down report\n- [`cube`][datafusion.expr.GroupingSet.cube] \u2014 every possible subtotal combination, like a pivot table\n- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] \u2014 explicitly list exactly which grouping levels you want\n\nBecause result rows come from different grouping levels, a column that is *not* part of a\nparticular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to\ndistinguish a real `null` in the data from one that means \"this column was aggregated across.\"\nIt returns `0` when the column is a grouping key for that row, and `1` when it is not.\n\n### Rollup\n\n[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces\ngrouping sets `(a, b)`, `(a)`, and `()` \u2014 like nested subtotals in a report. This is useful\nwhen your columns have a natural hierarchy, such as region \u2192 city or type \u2192 subtype.\n\nSuppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With\nthe default aggregation style we would need two separate queries. With `rollup` we get it all at\nonce:\n\n" + "source": "\nOrder 1 is partial (one of three suppliers failed). Order 2 is excluded\nbecause no supplier failed, order 3 because its only supplier failed, and\norder 4 because both of its suppliers failed.\n\n## Grouping Sets\n\nThe default style of aggregation produces one row per group. Sometimes you want a single query to\nproduce rows at multiple levels of detail — for example, totals per type *and* an overall grand\ntotal, or subtotals for every combination of two columns plus the individual column totals. Writing\nseparate queries and concatenating them is tedious and runs the data multiple times. Grouping sets\nsolve this by letting you specify several grouping levels in one pass.\n\nDataFusion supports three grouping set styles through the\n[`GroupingSet`][datafusion.expr.GroupingSet] class:\n\n- [`rollup`][datafusion.expr.GroupingSet.rollup] — hierarchical subtotals, like a drill-down report\n- [`cube`][datafusion.expr.GroupingSet.cube] — every possible subtotal combination, like a pivot table\n- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] — explicitly list exactly which grouping levels you want\n\nBecause result rows come from different grouping levels, a column that is *not* part of a\nparticular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to\ndistinguish a real `null` in the data from one that means \"this column was aggregated across.\"\nIt returns `0` when the column is a grouping key for that row, and `1` when it is not.\n\n### Rollup\n\n[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces\ngrouping sets `(a, b)`, `(a)`, and `()` — like nested subtotals in a report. This is useful\nwhen your columns have a natural hierarchy, such as region → city or type → subtype.\n\nSuppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With\nthe default aggregation style we would need two separate queries. With `rollup` we get it all at\nonce:\n\n" }, { "cell_type": "code", @@ -321,7 +324,7 @@ "cell_type": "markdown", "id": "db7b79bc585a40fcaf58bf750017e135", "metadata": {}, - "source": "\nThe first row \u2014 where `Type 1` is `null` \u2014 is the grand total across all types. But how do you\ntell a grand-total `null` apart from a Pokemon that genuinely has no type? The\n[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key\nfor that row and `1` when it is aggregated across.\n\n!!! note\n\n Due to an upstream DataFusion limitation\n ([apache/datafusion#21411](https://github.com/apache/datafusion/issues/21411)),\n `.alias()` cannot be applied directly to a `grouping()` expression \u2014 it will raise an\n error at execution time. Instead, use\n [`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] on the result DataFrame to\n give the column a readable name. Once the upstream issue is resolved, you will be able to\n use `.alias()` directly and the workaround below will no longer be necessary.\n\nThe raw column name generated by `grouping()` contains internal identifiers, so we use\n[`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] to clean it up:\n\n" + "source": "\nThe first row — where `Type 1` is `null` — is the grand total across all types. But how do you\ntell a grand-total `null` apart from a Pokemon that genuinely has no type? The\n[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key\nfor that row and `1` when it is aggregated across.\n\n
\n

Note

\n\nDue to an upstream DataFusion limitation\n([apache/datafusion#21411](https://github.com/apache/datafusion/issues/21411)),\n`.alias()` cannot be applied directly to a `grouping()` expression — it will raise an\nerror at execution time. Instead, use\n[`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] on the result DataFrame to\ngive the column a readable name. Once the upstream issue is resolved, you will be able to\nuse `.alias()` directly and the workaround below will no longer be necessary.\n\n
\n\nThe raw column name generated by `grouping()` contains internal identifiers, so we use\n[`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] to clean it up:\n\n" }, { "cell_type": "code", @@ -348,7 +351,7 @@ "cell_type": "markdown", "id": "1671c31a24314836a5b85d7ef7fbf015", "metadata": {}, - "source": "\nWith two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces:\n\n- one row per `(Type 1, Type 2)` pair \u2014 the most detailed level\n- one row per `Type 1` \u2014 subtotals\n- one grand total row\n\n" + "source": "\nWith two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces:\n\n- one row per `(Type 1, Type 2)` pair — the most detailed level\n- one row per `Type 1` — subtotals\n- one grand total row\n\n" }, { "cell_type": "code", @@ -370,7 +373,7 @@ "cell_type": "markdown", "id": "f6fa52606d8c4a75a9b52967216f8f3f", "metadata": {}, - "source": "\n### Cube\n\n[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)`\nproduces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` \u2014 one more than `rollup` because\nit also includes `(b)` alone. This is useful when neither column is \"above\" the other in a\nhierarchy and you want all cross-tabulations.\n\nFor our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair,\nby `Type 1` alone, by `Type 2` alone, and a grand total \u2014 all in one query:\n\n" + "source": "\n### Cube\n\n[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)`\nproduces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` — one more than `rollup` because\nit also includes `(b)` alone. This is useful when neither column is \"above\" the other in a\nhierarchy and you want all cross-tabulations.\n\nFor our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair,\nby `Type 1` alone, by `Type 2` alone, and a grand total — all in one query:\n\n" }, { "cell_type": "code", @@ -392,7 +395,7 @@ "cell_type": "markdown", "id": "cdf66aed5cc84ca1b48e60bad68798a8", "metadata": {}, - "source": "\nCompared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but\n`Type 2` has a value \u2014 those are the per-`Type 2` subtotals that `rollup` does not include.\n\n### Explicit Grouping Sets\n\n[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels\nyou need when `rollup` or `cube` would produce too many or too few. Each argument is a list of\ncolumns forming one grouping set.\n\nFor example, if we want only the per-`Type 1` totals and per-`Type 2` totals \u2014 but *not* the\nfull `(Type 1, Type 2)` detail rows or the grand total \u2014 we can ask for exactly that:\n\n" + "source": "\nCompared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but\n`Type 2` has a value — those are the per-`Type 2` subtotals that `rollup` does not include.\n\n### Explicit Grouping Sets\n\n[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels\nyou need when `rollup` or `cube` would produce too many or too few. Each argument is a list of\ncolumns forming one grouping set.\n\nFor example, if we want only the per-`Type 1` totals and per-`Type 2` totals — but *not* the\nfull `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that:\n\n" }, { "cell_type": "code", @@ -446,7 +449,7 @@ "cell_type": "markdown", "id": "5b09d5ef5b5e4bb6ab9b829b10b6a29f", "metadata": {}, - "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the accumulator class is captured by value via [`cloudpickle`][cloudpickle],\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n
\n

Note

\n\nSerialization\n\n
\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions —\n the accumulator class is captured by value via [`cloudpickle`][cloudpickle],\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" } ], "metadata": { diff --git a/docs/source/user-guide/common-operations/basic-info.ipynb b/docs/source/user-guide/common-operations/basic-info.ipynb index 1e8831db1..5d77c2ce6 100644 --- a/docs/source/user-guide/common-operations/basic-info.ipynb +++ b/docs/source/user-guide/common-operations/basic-info.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { diff --git a/docs/source/user-guide/common-operations/expressions.ipynb b/docs/source/user-guide/common-operations/expressions.ipynb index c76694e9a..a3eebb71e 100644 --- a/docs/source/user-guide/common-operations/expressions.ipynb +++ b/docs/source/user-guide/common-operations/expressions.ipynb @@ -25,19 +25,22 @@ " literal,\n", ")\n", "from datafusion import functions as f\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { "cell_type": "markdown", "id": "acae54e37e7d407bbb7b55eff062a284", "metadata": {}, - "source": "\n\n\n# Expressions\n\nIn DataFusion an expression is an abstraction that represents a computation.\nExpressions are used as the primary inputs and outputs for most functions within\nDataFusion. As such, expressions can be combined to create expression trees, a\nconcept shared across most compilers and databases.\n\n## Column\n\nThe first expression most new users will interact with is the Column, which is created by calling [`col`][datafusion.col].\nThis expression represents a column within a DataFrame. The function [`col`][datafusion.col] takes as in input a string\nand returns an expression as it's output.\n\n## Literal\n\nLiteral expressions represent a single value. These are helpful in a wide range of operations where\na specific, known value is of interest. You can create a literal expression using the function [`lit`][datafusion.lit].\nThe type of the object passed to the [`lit`][datafusion.lit] function will be used to convert it to a known data type.\n\nIn the following example we create expressions for the column named `color` and the literal scalar string `red`.\nThe resultant variable `red_units` is itself also an expression.\n\n" + "source": "\n\n\n# Expressions\n\nIn DataFusion an expression is an abstraction that represents a computation.\nExpressions are used as the primary inputs and outputs for most functions within\nDataFusion. As such, expressions can be combined to create expression trees, a\nconcept shared across most compilers and databases.\n\n## Column\n\nThe first expression most new users will interact with is the Column, which is created by calling [`col`][datafusion.col.col].\nThis expression represents a column within a DataFrame. The function [`col`][datafusion.col.col] takes as in input a string\nand returns an expression as it's output.\n\n## Literal\n\nLiteral expressions represent a single value. These are helpful in a wide range of operations where\na specific, known value is of interest. You can create a literal expression using the function [`lit`][datafusion.lit].\nThe type of the object passed to the [`lit`][datafusion.lit] function will be used to convert it to a known data type.\n\nIn the following example we create expressions for the column named `color` and the literal scalar string `red`.\nThe resultant variable `red_units` is itself also an expression.\n\n" }, { "cell_type": "code", @@ -91,7 +94,7 @@ "cell_type": "markdown", "id": "8763a12b2bbd4a93a75aff182afb95dc", "metadata": {}, - "source": "\n!!! warning\n\n Indexing an element of an array via `[]` starts at index 0 whereas\n [`array_element`][datafusion.functions.array_element] starts at index 1.\n\nStarting in DataFusion 49.0.0 you can also create slices of array elements using\nslice syntax from Python.\n\n" + "source": "\n
\n

Warning

\n\nIndexing an element of an array via `[]` starts at index 0 whereas\n[`array_element`][datafusion.functions.array_element] starts at index 1.\n\n
\n\nStarting in DataFusion 49.0.0 you can also create slices of array elements using\nslice syntax from Python.\n\n" }, { "cell_type": "code", @@ -232,7 +235,7 @@ "cell_type": "markdown", "id": "379cbbc1e968416e875cc15c1202d7eb", "metadata": {}, - "source": "\n!!! note\n\n Lambda expressions cannot yet be serialized: calling\n [`to_bytes`][datafusion.expr.Expr.to_bytes] or pickling an expression that\n contains a lambda raises `Lambda not implemented`. SQL lambda syntax is\n only parsed by dialects that support lambdas; set\n `datafusion.sql_parser.dialect` to one of `DuckDB`, `ClickHouse`,\n `Snowflake`, or `Databricks`. Both arrow syntax (`x -> x * 2`) and\n keyword syntax (`lambda x: x * 2`) parse. DuckDB will drop the arrow\n form in v2.1, so prefer `lambda x: x * 2` for forward compatibility.\n The Python expression builder shown above works regardless of dialect.\n\n## Testing membership in a list\n\nA common need is filtering rows where a column equals *any* of a small set of\nvalues. DataFusion offers three forms; they differ in readability and in how\nthey scale:\n\n1. A compound boolean using `|` across explicit equalities.\n2. [`in_list`][datafusion.functions.in_list], which accepts a list of\n expressions and tests equality against all of them in one call.\n3. A trick with [`array_position`][datafusion.functions.array_position] and\n [`make_array`][datafusion.functions.make_array], which returns the 1-based\n index of the value in a constructed array, or null if it is not present.\n\n" + "source": "\n
\n

Note

\n\nLambda expressions cannot yet be serialized: calling\n[`to_bytes`][datafusion.expr.Expr.to_bytes] or pickling an expression that\ncontains a lambda raises `Lambda not implemented`. SQL lambda syntax is\nonly parsed by dialects that support lambdas; set\n`datafusion.sql_parser.dialect` to one of `DuckDB`, `ClickHouse`,\n`Snowflake`, or `Databricks`. Both arrow syntax (`x -> x * 2`) and\nkeyword syntax (`lambda x: x * 2`) parse. DuckDB will drop the arrow\nform in v2.1, so prefer `lambda x: x * 2` for forward compatibility.\nThe Python expression builder shown above works regardless of dialect.\n\n
\n\n## Testing membership in a list\n\nA common need is filtering rows where a column equals *any* of a small set of\nvalues. DataFusion offers three forms; they differ in readability and in how\nthey scale:\n\n1. A compound boolean using `|` across explicit equalities.\n2. [`in_list`][datafusion.functions.in_list], which accepts a list of\n expressions and tests equality against all of them in one call.\n3. A trick with [`array_position`][datafusion.functions.array_position] and\n [`make_array`][datafusion.functions.make_array], which returns the 1-based\n index of the value in a constructed array, or null if it is not present.\n\n" }, { "cell_type": "code", @@ -291,7 +294,7 @@ "cell_type": "markdown", "id": "1671c31a24314836a5b85d7ef7fbf015", "metadata": {}, - "source": "\n**Searched CASE** (an independent boolean predicate per branch). Use this\nform whenever a branch tests more than simple equality \u2014 for example,\nchecking whether a joined column is `NULL` to gate a computed value:\n\n" + "source": "\n**Searched CASE** (an independent boolean predicate per branch). Use this\nform whenever a branch tests more than simple equality — for example,\nchecking whether a joined column is `NULL` to gate a computed value:\n\n" }, { "cell_type": "code", @@ -317,7 +320,7 @@ "cell_type": "markdown", "id": "f6fa52606d8c4a75a9b52967216f8f3f", "metadata": {}, - "source": "\nThis searched-CASE pattern is idiomatic for \"attribute the measure to the\nmatching side of a left join, otherwise contribute zero\" \u2014 a shape that\nappears in TPC-H Q08 and similar market-share calculations.\n\nIf a switched CASE only groups several equality matches into one bucket,\n`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often\nsimpler than the full `case` builder.\n\n## Structs\n\nColumns that contain struct elements can be accessed using the bracket notation as if they were\nPython dictionary style objects. This expects a string key as the parameter passed.\n\n" + "source": "\nThis searched-CASE pattern is idiomatic for \"attribute the measure to the\nmatching side of a left join, otherwise contribute zero\" — a shape that\nappears in TPC-H Q08 and similar market-share calculations.\n\nIf a switched CASE only groups several equality matches into one bucket,\n`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often\nsimpler than the full `case` builder.\n\n## Structs\n\nColumns that contain struct elements can be accessed using the bracket notation as if they were\nPython dictionary style objects. This expects a string key as the parameter passed.\n\n" }, { "cell_type": "code", diff --git a/docs/source/user-guide/common-operations/functions.ipynb b/docs/source/user-guide/common-operations/functions.ipynb index aabb6a765..fe9c803a6 100644 --- a/docs/source/user-guide/common-operations/functions.ipynb +++ b/docs/source/user-guide/common-operations/functions.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { diff --git a/docs/source/user-guide/common-operations/index.md b/docs/source/user-guide/common-operations/index.md index cf1559e8c..1f3478b72 100644 --- a/docs/source/user-guide/common-operations/index.md +++ b/docs/source/user-guide/common-operations/index.md @@ -21,16 +21,20 @@ The contents of this section are designed to guide a new user through how to use DataFusion. -```{toctree} -:maxdepth: 2 +## Contents -views -basic-info -select-and-filter -expressions -joins -functions -aggregations -windows -udf-and-udfa -``` +- [Basic Info](basic-info.ipynb) — inspecting schema, row counts, and + summary statistics. +- [Views](views.md) — saving and reusing query fragments as views. +- [Select and Filter](select-and-filter.ipynb) — projecting columns and + applying predicates. +- [Expressions](expressions.ipynb) — `col`, `lit`, boolean operators, + array indexing, and chaining. +- [Joins](joins.ipynb) — inner / outer / semi / anti joins. +- [Functions](functions.ipynb) — scalar functions across math, string, + date/time, and array families. +- [Aggregations](aggregations.ipynb) — `group_by`, rollup, cube, + grouping sets. +- [Windows](windows.ipynb) — partitioned and ranking window functions. +- [UDFs and UDAFs](udf-and-udfa.ipynb) — scalar, aggregate, window, and + table user-defined functions. diff --git a/docs/source/user-guide/common-operations/joins.ipynb b/docs/source/user-guide/common-operations/joins.ipynb index f31c2a98a..71ec27f9e 100644 --- a/docs/source/user-guide/common-operations/joins.ipynb +++ b/docs/source/user-guide/common-operations/joins.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { diff --git a/docs/source/user-guide/common-operations/select-and-filter.ipynb b/docs/source/user-guide/common-operations/select-and-filter.ipynb index aca9408bc..dc0daaa5a 100644 --- a/docs/source/user-guide/common-operations/select-and-filter.ipynb +++ b/docs/source/user-guide/common-operations/select-and-filter.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { @@ -55,7 +58,7 @@ "cell_type": "markdown", "id": "8dd0d8092fe74a7c96281538738b07e2", "metadata": {}, - "source": "\nFor mathematical or logical operations use [`col`][datafusion.col] to select columns, and give meaningful names to the resulting\noperations using [`alias`][datafusion.expr.Expr.alias]\n\n" + "source": "\nFor mathematical or logical operations use [`col`][datafusion.col.col] to select columns, and give meaningful names to the resulting\noperations using [`alias`][datafusion.expr.Expr.alias]\n\n" }, { "cell_type": "code", @@ -71,7 +74,7 @@ "cell_type": "markdown", "id": "8edb47106e1a46a883d545849b8ab81b", "metadata": {}, - "source": "\n!!! warning\n\n Please be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters\n (ex: Name) you must put your column name in double quotes or the selection won\u2019t work. As an alternative for simple\n column selection use [`select`][datafusion.dataframe.DataFrame.select] without double quotes\n\nFor selecting columns with capital letters use `'\"VendorID\"'`\n\n" + "source": "\n
\n

Warning

\n\nPlease be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters\n(ex: Name) you must put your column name in double quotes or the selection won’t work. As an alternative for simple\ncolumn selection use [`select`][datafusion.dataframe.DataFrame.select] without double quotes\n\n
\n\nFor selecting columns with capital letters use `'\"VendorID\"'`\n\n" }, { "cell_type": "code", diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb index e83a6f94e..676f5a6b3 100644 --- a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb +++ b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { @@ -106,7 +109,7 @@ "cell_type": "markdown", "id": "8edb47106e1a46a883d545849b8ab81b", "metadata": {}, - "source": "\nIn this example we passed the PyArrow `DataType` when we defined the function\nby calling `udf()`. If you need additional control, such as specifying\nmetadata or nullability of the input or output, you can instead specify a\nPyArrow `Field`.\n\nIf you need to write a custom function but do not want to incur the performance\ncost of converting to Python objects and back, a more advanced approach is to\nwrite Rust based UDFs and to expose them to Python. There is an example in the\n[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/)\ndescribing how to do this.\n\n### When not to use a UDF\n\nA UDF is the right tool when the per-row computation genuinely cannot be\nexpressed with DataFusion's built-in expressions. It is often the *wrong*\ntool for a predicate that *can* be written as an `Expr` tree but feels\neasier to write as a Python function \u2014 for example, a filter that keeps\na row if it matches any one of several rule sets, where each rule set\nchecks its own combination of columns (the worked example at the end of\nthis section keeps a row when it matches any one of several brand-specific\nrules). Looping over the rules in Python and returning a boolean per row\nreads naturally and is tempting to wrap in a UDF, but a UDF is opaque to\nthe optimizer: filters expressed as UDFs lose several rewrites that the\nengine applies to filters built from native expressions. The most visible\nof these is **predicate pushdown into the table provider**: a native\npredicate can be handed to the source so it skips data before it is read,\nwhile a UDF predicate cannot. The example below uses Parquet, where\npushdown prunes whole row groups using the min/max statistics in the\nfooter, but the same mechanism applies to any table provider that\nadvertises filter support \u2014 including custom providers.\n\nThe following example writes a small Parquet file, then filters it two\nways: first with a native expression, then with a UDF that computes the\nsame result. The filter itself is simple on purpose so we can compare\nthe plans side by side.\n\n" + "source": "\nIn this example we passed the PyArrow `DataType` when we defined the function\nby calling `udf()`. If you need additional control, such as specifying\nmetadata or nullability of the input or output, you can instead specify a\nPyArrow `Field`.\n\nIf you need to write a custom function but do not want to incur the performance\ncost of converting to Python objects and back, a more advanced approach is to\nwrite Rust based UDFs and to expose them to Python. There is an example in the\n[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/)\ndescribing how to do this.\n\n### When not to use a UDF\n\nA UDF is the right tool when the per-row computation genuinely cannot be\nexpressed with DataFusion's built-in expressions. It is often the *wrong*\ntool for a predicate that *can* be written as an `Expr` tree but feels\neasier to write as a Python function — for example, a filter that keeps\na row if it matches any one of several rule sets, where each rule set\nchecks its own combination of columns (the worked example at the end of\nthis section keeps a row when it matches any one of several brand-specific\nrules). Looping over the rules in Python and returning a boolean per row\nreads naturally and is tempting to wrap in a UDF, but a UDF is opaque to\nthe optimizer: filters expressed as UDFs lose several rewrites that the\nengine applies to filters built from native expressions. The most visible\nof these is **predicate pushdown into the table provider**: a native\npredicate can be handed to the source so it skips data before it is read,\nwhile a UDF predicate cannot. The example below uses Parquet, where\npushdown prunes whole row groups using the min/max statistics in the\nfooter, but the same mechanism applies to any table provider that\nadvertises filter support — including custom providers.\n\nThe following example writes a small Parquet file, then filters it two\nways: first with a native expression, then with a UDF that computes the\nsame result. The filter itself is simple on purpose so we can compare\nthe plans side by side.\n\n" }, { "cell_type": "code", @@ -160,7 +163,7 @@ "cell_type": "markdown", "id": "7cdc8c89c7104fffa095e18ddfef8986", "metadata": {}, - "source": "\nNotice the `DataSourceExec` line. It carries three annotations the\noptimizer computed from the predicate:\n\n- `predicate=brand@1 = A AND qty@2 >= 150` \u2014 the filter is pushed\n into the Parquet scan itself, so the scan only reads matching rows.\n- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ...\n qty_max@4 >= 150` \u2014 the scan prunes whole row groups by consulting\n the Parquet min/max statistics in the footer *before* reading any\n column data.\n- `required_guarantees=[brand in (A)]` \u2014 the scan uses this when a\n bloom filter or dictionary is available to skip pages.\n\n**UDF predicate.** Now wrap the same logic in a Python UDF:\n\n" + "source": "\nNotice the `DataSourceExec` line. It carries three annotations the\noptimizer computed from the predicate:\n\n- `predicate=brand@1 = A AND qty@2 >= 150` — the filter is pushed\n into the Parquet scan itself, so the scan only reads matching rows.\n- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ...\n qty_max@4 >= 150` — the scan prunes whole row groups by consulting\n the Parquet min/max statistics in the footer *before* reading any\n column data.\n- `required_guarantees=[brand in (A)]` — the scan uses this when a\n bloom filter or dictionary is available to skip pages.\n\n**UDF predicate.** Now wrap the same logic in a Python UDF:\n\n" }, { "cell_type": "code", @@ -189,7 +192,7 @@ "cell_type": "markdown", "id": "938c804e27f84196a10c8828c723f798", "metadata": {}, - "source": "\nThe `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`.\nThere is no `pruning_predicate` and no `required_guarantees`: the\nscan has to materialize every row group and hand each row to the\nPython callback just to decide whether to keep it.\n\nAt small scale the cost difference is invisible; on a Parquet file with\nmany row groups, or data whose min/max statistics line up well with\nthe predicate, the native form can skip most of the file. The UDF form\nreads all of it.\n\n**Takeaway.** Reach for a UDF when the per-row computation is genuinely\nnot expressible as a tree of built-in functions (custom numerical work,\nexternal lookups, complex business rules). When it *is* expressible \u2014\neven if the native form is a little more verbose \u2014 build the `Expr`\ntree directly so the optimizer can see through it. For disjunctive\npredicates the idiom is to produce one clause per bucket and combine\nthem with `|`:\n\n```python\nfrom functools import reduce\nfrom operator import or_\nfrom datafusion import col, lit, functions as f\n\nbuckets = {\n \"Brand#12\": {\"containers\": [\"SM CASE\", \"SM BOX\"], \"min_qty\": 1, \"max_size\": 5},\n \"Brand#23\": {\"containers\": [\"MED BAG\", \"MED BOX\"], \"min_qty\": 10, \"max_size\": 10},\n}\n\ndef bucket_clause(brand, spec):\n return (\n (col(\"brand\") == lit(brand))\n & f.in_list(col(\"container\"), [lit(c) for c in spec[\"containers\"]])\n & (col(\"quantity\") >= lit(spec[\"min_qty\"]))\n & (col(\"quantity\") <= lit(spec[\"min_qty\"] + 10))\n & (col(\"size\") >= lit(1))\n & (col(\"size\") <= lit(spec[\"max_size\"]))\n )\n\npredicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items()))\ndf = df.filter(predicate)\n```\n\n## Aggregate Functions\n\nThe [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined\nAggregate Functions (UDAFs). To use this you must implement an\n[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed.\n\nWhen defining a UDAF there are four methods you need to implement. The `update` function takes the\narray(s) of input and updates the internal state of the accumulator. You should define this function\nto have as many input arguments as you will pass when calling the UDAF. Since aggregation may be\nsplit into multiple batches, we must have a method to combine multiple batches. For this, we have\ntwo functions, `state` and `merge`. `state` will return an array of scalar values that contain\nthe current state of a single batch accumulation. Then we must `merge` the results of these\ndifferent states. Finally `evaluate` is the call that will return the final result after the\n`merge` is complete.\n\nIn the following example we want to define a custom aggregate function that will return the\ndifference between the sum of two columns. The state can be represented by a single value and we can\nalso see how the inputs to `update` and `merge` differ.\n\n```python\nimport pyarrow as pa\nimport pyarrow.compute\nimport datafusion\nfrom datafusion import col, udaf, Accumulator\nfrom typing import List\n\nclass MyAccumulator(Accumulator):\n \"\"\"\n Interface of a user-defined accumulation.\n \"\"\"\n def __init__(self):\n self._sum = 0.0\n\n def update(self, values_a: pa.Array, values_b: pa.Array) -> None:\n self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py()\n\n def merge(self, states: list[pa.Array]) -> None:\n self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py()\n\n def state(self) -> list[pa.Scalar]:\n return [pyarrow.scalar(self._sum)]\n\n def evaluate(self) -> pa.Scalar:\n return pyarrow.scalar(self._sum)\n\nctx = datafusion.SessionContext()\ndf = ctx.from_pydict(\n {\n \"a\": [4, 5, 6],\n \"b\": [1, 2, 3],\n }\n)\n\nmy_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable')\n\ndf.aggregate([], [my_udaf(col(\"a\"), col(\"b\")).alias(\"col_diff\")])\n```\n\n### FAQ\n\n**How do I return a list from a UDAF?**\n\nBoth the `evaluate` and the `state` functions expect to return scalar values.\nIf you wish to return a list array as a scalar value, the best practice is to\nwrap the values in a `pyarrow.Scalar` object. For example, you can return a\ntimestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp(\"ms\")))` and\nregister the appropriate return or state types as\n`return_type=pa.list_(pa.timestamp(\"ms\"))` and\n`state_type=[pa.list_(pa.timestamp(\"ms\"))]`, respectively.\n\nAs of DataFusion 52.0.0 , you can pass return any Python object, including a\nPyArrow array, as the return value(s) for these functions and DataFusion will\nattempt to create a scalar type from the value. DataFusion has been tested to\nconvert PyArrow, nanoarrow, and arro3 objects as well as primitive data types\nlike integers, strings, and so on.\n\n## Window Functions\n\nTo implement a User-Defined Window Function (UDWF) you must call the\n[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract\nclass [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator].\n\nThere are three methods of evaluation of UDWFs.\n\n- `evaluate` is the simplest case, where you are given an array and are expected to calculate the\n value for a single row of that array. This is the simplest case, but also the least performant.\n- `evaluate_all` computes the values for all rows for an input array at a single time.\n- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank\n information for the rows.\n\nWhich methods you implement are based upon which of these options are set.\n\n| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement |\n|---|---|---|---|\n| False (default) | False (default) | False (default) | `evaluate_all` |\n| False | True | False | `evaluate` |\n| False | True | False | `evaluate_all_with_rank` |\n| True | True/False | True/False | `evaluate` |\n\n### UDWF options\n\nWhen you define your UDWF you can override the functions that return these values. They will\ndetermine which evaluate functions are called.\n\n- `uses_window_frame` is set for functions that compute based on the specified window frame. If\n your function depends upon the specified frame, set this to `True`.\n- `supports_bounded_execution` specifies if your function can be incrementally computed.\n- `include_rank` is set to `True` for window functions that can be computed only using the rank\n information.\n\n```python\nimport pyarrow as pa\nfrom datafusion import udwf, col, SessionContext\nfrom datafusion.user_defined import WindowEvaluator\n\nclass ExponentialSmooth(WindowEvaluator):\n def __init__(self, alpha: float) -> None:\n self.alpha = alpha\n\n def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array:\n results = []\n curr_value = 0.0\n values = values[0]\n for idx in range(num_rows):\n if idx == 0:\n curr_value = values[idx].as_py()\n else:\n curr_value = values[idx].as_py() * self.alpha + curr_value * (\n 1.0 - self.alpha\n )\n results.append(curr_value)\n\n return pa.array(results)\n\nexp_smooth = udwf(\n ExponentialSmooth(0.9),\n pa.float64(),\n pa.float64(),\n volatility=\"immutable\",\n)\n\nctx = SessionContext()\n\ndf = ctx.from_pydict({\n \"a\": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0]\n})\n\ndf.select(\"a\", exp_smooth(col(\"a\")).alias(\"smooth_a\")).show()\n```\n\n## Table Functions\n\nUser Defined Table Functions are slightly different than the other functions\ndescribed here. These functions take any number of `Expr` arguments, but only\nliteral expressions are supported. Table functions must return a Table\nProvider as described in the ref:`_io_custom_table_provider` page.\n\nOnce you have a table function, you can register it with the session context\nby using [`register_udtf`][datafusion.context.SessionContext.register_udtf].\n\nThere are examples of both rust backed and python based table functions in the\nexamples folder of the repository. If you have a rust backed table function\nthat you wish to expose via PyO3, you need to expose it as a `PyCapsule`.\n\n```rust\n#[pymethods]\nimpl MyTableFunction {\n fn __datafusion_table_function__<'py>(\n &self,\n py: Python<'py>,\n ) -> PyResult> {\n let name = cr\"datafusion_table_function\".into();\n\n let func = self.clone();\n let provider = FFI_TableFunction::new(Arc::new(func), None);\n\n PyCapsule::new(py, provider, Some(name))\n }\n}\n```\n\n### Accessing the Calling Session\n\nPure-Python UDTFs can opt into receiving the calling\n[`SessionContext`][datafusion.context.SessionContext] by registering with\n`with_session=True`. The context is passed as a `session` keyword\nargument on every invocation. Use it to look up registered tables,\nUDFs, or session configuration from inside the callback.\n\n```python\nfrom datafusion import SessionContext, Table, udtf\nfrom datafusion.context import TableProviderExportable\nimport pyarrow as pa\nimport pyarrow.dataset as ds\n\n@udtf(\"list_tables\", with_session=True)\ndef list_tables(*, session: SessionContext) -> TableProviderExportable:\n names = sorted(session.catalog().schema().names())\n batch = pa.RecordBatch.from_pydict({\"name\": names})\n return Table(ds.dataset([batch]))\n\nctx = SessionContext()\nctx.register_batch(\"t1\", pa.RecordBatch.from_pydict({\"x\": [1]}))\nctx.register_udtf(list_tables)\nctx.sql(\"SELECT * FROM list_tables()\").show()\n```\n\nWithout `with_session=True`, the callback receives only the positional\nexpression arguments. The flag is opt-in so existing UDTFs keep working\nunchanged.\n\nThe injected `session` is a fresh [`SessionContext`][datafusion.context.SessionContext]\nwrapper backed by the same underlying state as the caller, so registries\n(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering\na new table or UDF) propagate to the live session because the registries\nare reference-counted and shared. Configuration changes made through the\nwrapper (e.g. setting session options) do **not** propagate \u2014 the wrapper\nholds its own clone of the session config.\n" + "source": "\nThe `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`.\nThere is no `pruning_predicate` and no `required_guarantees`: the\nscan has to materialize every row group and hand each row to the\nPython callback just to decide whether to keep it.\n\nAt small scale the cost difference is invisible; on a Parquet file with\nmany row groups, or data whose min/max statistics line up well with\nthe predicate, the native form can skip most of the file. The UDF form\nreads all of it.\n\n**Takeaway.** Reach for a UDF when the per-row computation is genuinely\nnot expressible as a tree of built-in functions (custom numerical work,\nexternal lookups, complex business rules). When it *is* expressible —\neven if the native form is a little more verbose — build the `Expr`\ntree directly so the optimizer can see through it. For disjunctive\npredicates the idiom is to produce one clause per bucket and combine\nthem with `|`:\n\n```python\nfrom functools import reduce\nfrom operator import or_\nfrom datafusion import col, lit, functions as f\n\nbuckets = {\n \"Brand#12\": {\"containers\": [\"SM CASE\", \"SM BOX\"], \"min_qty\": 1, \"max_size\": 5},\n \"Brand#23\": {\"containers\": [\"MED BAG\", \"MED BOX\"], \"min_qty\": 10, \"max_size\": 10},\n}\n\ndef bucket_clause(brand, spec):\n return (\n (col(\"brand\") == lit(brand))\n & f.in_list(col(\"container\"), [lit(c) for c in spec[\"containers\"]])\n & (col(\"quantity\") >= lit(spec[\"min_qty\"]))\n & (col(\"quantity\") <= lit(spec[\"min_qty\"] + 10))\n & (col(\"size\") >= lit(1))\n & (col(\"size\") <= lit(spec[\"max_size\"]))\n )\n\npredicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items()))\ndf = df.filter(predicate)\n```\n\n## Aggregate Functions\n\nThe [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined\nAggregate Functions (UDAFs). To use this you must implement an\n[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed.\n\nWhen defining a UDAF there are four methods you need to implement. The `update` function takes the\narray(s) of input and updates the internal state of the accumulator. You should define this function\nto have as many input arguments as you will pass when calling the UDAF. Since aggregation may be\nsplit into multiple batches, we must have a method to combine multiple batches. For this, we have\ntwo functions, `state` and `merge`. `state` will return an array of scalar values that contain\nthe current state of a single batch accumulation. Then we must `merge` the results of these\ndifferent states. Finally `evaluate` is the call that will return the final result after the\n`merge` is complete.\n\nIn the following example we want to define a custom aggregate function that will return the\ndifference between the sum of two columns. The state can be represented by a single value and we can\nalso see how the inputs to `update` and `merge` differ.\n\n```python\nimport pyarrow as pa\nimport pyarrow.compute\nimport datafusion\nfrom datafusion import col, udaf, Accumulator\nfrom typing import List\n\nclass MyAccumulator(Accumulator):\n \"\"\"\n Interface of a user-defined accumulation.\n \"\"\"\n def __init__(self):\n self._sum = 0.0\n\n def update(self, values_a: pa.Array, values_b: pa.Array) -> None:\n self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py()\n\n def merge(self, states: list[pa.Array]) -> None:\n self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py()\n\n def state(self) -> list[pa.Scalar]:\n return [pyarrow.scalar(self._sum)]\n\n def evaluate(self) -> pa.Scalar:\n return pyarrow.scalar(self._sum)\n\nctx = datafusion.SessionContext()\ndf = ctx.from_pydict(\n {\n \"a\": [4, 5, 6],\n \"b\": [1, 2, 3],\n }\n)\n\nmy_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable')\n\ndf.aggregate([], [my_udaf(col(\"a\"), col(\"b\")).alias(\"col_diff\")])\n```\n\n### FAQ\n\n**How do I return a list from a UDAF?**\n\nBoth the `evaluate` and the `state` functions expect to return scalar values.\nIf you wish to return a list array as a scalar value, the best practice is to\nwrap the values in a `pyarrow.Scalar` object. For example, you can return a\ntimestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp(\"ms\")))` and\nregister the appropriate return or state types as\n`return_type=pa.list_(pa.timestamp(\"ms\"))` and\n`state_type=[pa.list_(pa.timestamp(\"ms\"))]`, respectively.\n\nAs of DataFusion 52.0.0 , you can pass return any Python object, including a\nPyArrow array, as the return value(s) for these functions and DataFusion will\nattempt to create a scalar type from the value. DataFusion has been tested to\nconvert PyArrow, nanoarrow, and arro3 objects as well as primitive data types\nlike integers, strings, and so on.\n\n## Window Functions\n\nTo implement a User-Defined Window Function (UDWF) you must call the\n[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract\nclass [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator].\n\nThere are three methods of evaluation of UDWFs.\n\n- `evaluate` is the simplest case, where you are given an array and are expected to calculate the\n value for a single row of that array. This is the simplest case, but also the least performant.\n- `evaluate_all` computes the values for all rows for an input array at a single time.\n- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank\n information for the rows.\n\nWhich methods you implement are based upon which of these options are set.\n\n| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement |\n|---|---|---|---|\n| False (default) | False (default) | False (default) | `evaluate_all` |\n| False | True | False | `evaluate` |\n| False | True | False | `evaluate_all_with_rank` |\n| True | True/False | True/False | `evaluate` |\n\n### UDWF options\n\nWhen you define your UDWF you can override the functions that return these values. They will\ndetermine which evaluate functions are called.\n\n- `uses_window_frame` is set for functions that compute based on the specified window frame. If\n your function depends upon the specified frame, set this to `True`.\n- `supports_bounded_execution` specifies if your function can be incrementally computed.\n- `include_rank` is set to `True` for window functions that can be computed only using the rank\n information.\n\n```python\nimport pyarrow as pa\nfrom datafusion import udwf, col, SessionContext\nfrom datafusion.user_defined import WindowEvaluator\n\nclass ExponentialSmooth(WindowEvaluator):\n def __init__(self, alpha: float) -> None:\n self.alpha = alpha\n\n def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array:\n results = []\n curr_value = 0.0\n values = values[0]\n for idx in range(num_rows):\n if idx == 0:\n curr_value = values[idx].as_py()\n else:\n curr_value = values[idx].as_py() * self.alpha + curr_value * (\n 1.0 - self.alpha\n )\n results.append(curr_value)\n\n return pa.array(results)\n\nexp_smooth = udwf(\n ExponentialSmooth(0.9),\n pa.float64(),\n pa.float64(),\n volatility=\"immutable\",\n)\n\nctx = SessionContext()\n\ndf = ctx.from_pydict({\n \"a\": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0]\n})\n\ndf.select(\"a\", exp_smooth(col(\"a\")).alias(\"smooth_a\")).show()\n```\n\n## Table Functions\n\nUser Defined Table Functions are slightly different than the other functions\ndescribed here. These functions take any number of `Expr` arguments, but only\nliteral expressions are supported. Table functions must return a Table\nProvider as described in the ref:`_io_custom_table_provider` page.\n\nOnce you have a table function, you can register it with the session context\nby using [`register_udtf`][datafusion.context.SessionContext.register_udtf].\n\nThere are examples of both rust backed and python based table functions in the\nexamples folder of the repository. If you have a rust backed table function\nthat you wish to expose via PyO3, you need to expose it as a `PyCapsule`.\n\n```rust\n#[pymethods]\nimpl MyTableFunction {\n fn __datafusion_table_function__<'py>(\n &self,\n py: Python<'py>,\n ) -> PyResult> {\n let name = cr\"datafusion_table_function\".into();\n\n let func = self.clone();\n let provider = FFI_TableFunction::new(Arc::new(func), None);\n\n PyCapsule::new(py, provider, Some(name))\n }\n}\n```\n\n### Accessing the Calling Session\n\nPure-Python UDTFs can opt into receiving the calling\n[`SessionContext`][datafusion.context.SessionContext] by registering with\n`with_session=True`. The context is passed as a `session` keyword\nargument on every invocation. Use it to look up registered tables,\nUDFs, or session configuration from inside the callback.\n\n```python\nfrom datafusion import SessionContext, Table, udtf\nfrom datafusion.context import TableProviderExportable\nimport pyarrow as pa\nimport pyarrow.dataset as ds\n\n@udtf(\"list_tables\", with_session=True)\ndef list_tables(*, session: SessionContext) -> TableProviderExportable:\n names = sorted(session.catalog().schema().names())\n batch = pa.RecordBatch.from_pydict({\"name\": names})\n return Table(ds.dataset([batch]))\n\nctx = SessionContext()\nctx.register_batch(\"t1\", pa.RecordBatch.from_pydict({\"x\": [1]}))\nctx.register_udtf(list_tables)\nctx.sql(\"SELECT * FROM list_tables()\").show()\n```\n\nWithout `with_session=True`, the callback receives only the positional\nexpression arguments. The flag is opt-in so existing UDTFs keep working\nunchanged.\n\nThe injected `session` is a fresh [`SessionContext`][datafusion.context.SessionContext]\nwrapper backed by the same underlying state as the caller, so registries\n(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering\na new table or UDF) propagate to the live session because the registries\nare reference-counted and shared. Configuration changes made through the\nwrapper (e.g. setting session options) do **not** propagate — the wrapper\nholds its own clone of the session config.\n" } ], "metadata": { diff --git a/docs/source/user-guide/common-operations/views.md b/docs/source/user-guide/common-operations/views.md index 4feaac028..0522f8f1f 100644 --- a/docs/source/user-guide/common-operations/views.md +++ b/docs/source/user-guide/common-operations/views.md @@ -19,7 +19,7 @@ # Registering Views -You can use the context's `register_view` method to register a DataFrame as a view +You can use the context's [`register_view`][datafusion.context.SessionContext.register_view] method to register a DataFrame as a view ```python from datafusion import SessionContext, col, literal diff --git a/docs/source/user-guide/common-operations/windows.ipynb b/docs/source/user-guide/common-operations/windows.ipynb index 3995b3506..6b227a210 100644 --- a/docs/source/user-guide/common-operations/windows.ipynb +++ b/docs/source/user-guide/common-operations/windows.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { @@ -94,7 +97,7 @@ "cell_type": "markdown", "id": "8763a12b2bbd4a93a75aff182afb95dc", "metadata": {}, - "source": "\n### Partitions\n\nA window function can take a list of `partition_by` columns similar to an\n[Aggregation Function](aggregation). This will cause the window values to be evaluated\nindependently for each of the partitions. In the example above, we found the rank of each\nPokemon per `Type 1` partitions. We can see the first couple of each partition if we do\nthe following:\n\n" + "source": "\n### Partitions\n\nA window function can take a list of `partition_by` columns similar to an\n[Aggregation Function](../aggregations/). This will cause the window values to be evaluated\nindependently for each of the partitions. In the example above, we found the rank of each\nPokemon per `Type 1` partitions. We can see the first couple of each partition if we do\nthe following:\n\n" }, { "cell_type": "code", @@ -181,7 +184,7 @@ "cell_type": "markdown", "id": "59bbdb311c014d738909a11f9e486628", "metadata": {}, - "source": "\n## Aggregate Functions\n\nYou can use any [Aggregation Function](aggregation) as a window function. Here\nis an example that shows how to compare each pokemons\u2019s attack power with the average attack\npower in its `\"Type 1\"` using the [`avg`][datafusion.functions.avg] function.\n\n" + "source": "\n## Aggregate Functions\n\nYou can use any [Aggregation Function](../aggregations/) as a window function. Here\nis an example that shows how to compare each pokemons’s attack power with the average attack\npower in its `\"Type 1\"` using the [`avg`][datafusion.functions.avg] function.\n\n" }, { "cell_type": "code", @@ -209,7 +212,7 @@ "cell_type": "markdown", "id": "8a65eabff63a45729fe45fb5ade58bdc", "metadata": {}, - "source": "\n## Available Functions\n\nThe possible window functions are:\n\n1. Rank Functions\n : - [`rank`][datafusion.functions.rank]\n - [`dense_rank`][datafusion.functions.dense_rank]\n - [`ntile`][datafusion.functions.ntile]\n - [`row_number`][datafusion.functions.row_number]\n2. Analytical Functions\n : - [`cume_dist`][datafusion.functions.cume_dist]\n - [`percent_rank`][datafusion.functions.percent_rank]\n - [`lag`][datafusion.functions.lag]\n - [`lead`][datafusion.functions.lead]\n3. Aggregate Functions\n : - All [Aggregation Functions](aggregation) can be used as window functions.\n\n## User-Defined Window Functions\n\nYou can ship custom window functions to the engine by subclassing\n[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it\nvia [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`][datafusion.user_defined]\nfor the evaluator interface and worked examples.\n\n!!! note\n\n Serialization\n\n Python window UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the evaluator class is captured by value via [`cloudpickle`][cloudpickle], so\n worker processes do not need to pre-register the UDF. Any names the\n evaluator resolves via `import` are captured **by reference** and\n must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + "source": "\n## Available Functions\n\nThe possible window functions are:\n\n1. Rank Functions\n : - [`rank`][datafusion.functions.rank]\n - [`dense_rank`][datafusion.functions.dense_rank]\n - [`ntile`][datafusion.functions.ntile]\n - [`row_number`][datafusion.functions.row_number]\n2. Analytical Functions\n : - [`cume_dist`][datafusion.functions.cume_dist]\n - [`percent_rank`][datafusion.functions.percent_rank]\n - [`lag`][datafusion.functions.lag]\n - [`lead`][datafusion.functions.lead]\n3. Aggregate Functions\n : - All [Aggregation Functions](../aggregations/) can be used as window functions.\n\n## User-Defined Window Functions\n\nYou can ship custom window functions to the engine by subclassing\n[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it\nvia [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`][datafusion.user_defined]\nfor the evaluator interface and worked examples.\n\n
\n

Note

\n\nSerialization\n\n
\n\n Python window UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions —\n the evaluator class is captured by value via [`cloudpickle`][cloudpickle], so\n worker processes do not need to pre-register the UDF. Any names the\n evaluator resolves via `import` are captured **by reference** and\n must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" } ], "metadata": { diff --git a/docs/source/user-guide/concepts.ipynb b/docs/source/user-guide/concepts.ipynb index 63f157faa..83bf49550 100644 --- a/docs/source/user-guide/concepts.ipynb +++ b/docs/source/user-guide/concepts.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { @@ -65,7 +68,7 @@ "cell_type": "markdown", "id": "8dd0d8092fe74a7c96281538738b07e2", "metadata": {}, - "source": "\n## Session Context\n\nThe first statement group creates a [`SessionContext`][datafusion.context.SessionContext].\n\n```python\n# create a context\nctx = datafusion.SessionContext()\n```\n\nA Session Context is the main interface for executing queries with DataFusion. It maintains the state\nof the connection between a user and an instance of the DataFusion engine. Additionally it provides\nthe following functionality:\n\n- Create a DataFrame from a data source.\n- Register a data source as a table that can be referenced from a SQL query.\n- Execute a SQL query\n\n## DataFrame\n\nThe second statement group creates a [`DataFrame`][datafusion.dataframe.DataFrame],\n\n```python\n# Create a DataFrame from a file\ndf = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n```\n\nA DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html).\nDataFrames are typically created by calling a method on [`SessionContext`][datafusion.context.SessionContext], such as [`read_csv`][datafusion.io.read_csv], and can then be modified by\ncalling the transformation methods, such as [`filter`][datafusion.dataframe.DataFrame.filter], [`select`][datafusion.dataframe.DataFrame.select], [`aggregate`][datafusion.dataframe.DataFrame.aggregate],\nand [`limit`][datafusion.dataframe.DataFrame.limit] to build up a query definition.\n\nFor more details on working with DataFrames, including visualization options and conversion to other formats, see [dataframe/index](dataframe/index.md).\n\n## Expressions\n\nThe third statement uses [Expressions](../common-operations/expressions/) to build up a query definition. You can find\nexplanations for what the functions below do in the user documentation for\n[`col`][datafusion.col], [`lit`][datafusion.lit], [`round`][datafusion.functions.round],\nand [`alias`][datafusion.expr.Expr.alias].\n\n```python\ndf = df.select(\n \"trip_distance\",\n col(\"total_amount\").alias(\"total\"),\n (f.round(lit(100.0) * col(\"tip_amount\") / col(\"total_amount\"), lit(1))).alias(\"tip_percent\"),\n)\n```\n\nFinally the [`show`][datafusion.dataframe.DataFrame.show] method converts the logical plan\nrepresented by the DataFrame into a physical plan and execute it, collecting all results and\ndisplaying them to the user. It is important to note that DataFusion performs lazy evaluation\nof the DataFrame. Until you call a method such as [`show`][datafusion.dataframe.DataFrame.show]\nor [`collect`][datafusion.dataframe.DataFrame.collect], DataFusion will not perform the query.\n" + "source": "\n## Session Context\n\nThe first statement group creates a [`SessionContext`][datafusion.context.SessionContext].\n\n```python\n# create a context\nctx = datafusion.SessionContext()\n```\n\nA Session Context is the main interface for executing queries with DataFusion. It maintains the state\nof the connection between a user and an instance of the DataFusion engine. Additionally it provides\nthe following functionality:\n\n- Create a DataFrame from a data source.\n- Register a data source as a table that can be referenced from a SQL query.\n- Execute a SQL query\n\n## DataFrame\n\nThe second statement group creates a [`DataFrame`][datafusion.dataframe.DataFrame],\n\n```python\n# Create a DataFrame from a file\ndf = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n```\n\nA DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html).\nDataFrames are typically created by calling a method on [`SessionContext`][datafusion.context.SessionContext], such as [`read_csv`][datafusion.context.SessionContext.read_csv], and can then be modified by\ncalling the transformation methods, such as [`filter`][datafusion.dataframe.DataFrame.filter], [`select`][datafusion.dataframe.DataFrame.select], [`aggregate`][datafusion.dataframe.DataFrame.aggregate],\nand [`limit`][datafusion.dataframe.DataFrame.limit] to build up a query definition.\n\nFor more details on working with DataFrames, including visualization options and conversion to other formats, see [dataframe/index](dataframe/index.md).\n\n## Expressions\n\nThe third statement uses [Expressions](../common-operations/expressions/) to build up a query definition. You can find\nexplanations for what the functions below do in the user documentation for\n[`col`][datafusion.col.col], [`lit`][datafusion.lit], [`round`][datafusion.functions.round],\nand [`alias`][datafusion.expr.Expr.alias].\n\n```python\ndf = df.select(\n \"trip_distance\",\n col(\"total_amount\").alias(\"total\"),\n (f.round(lit(100.0) * col(\"tip_amount\") / col(\"total_amount\"), lit(1))).alias(\"tip_percent\"),\n)\n```\n\nFinally the [`show`][datafusion.dataframe.DataFrame.show] method converts the logical plan\nrepresented by the DataFrame into a physical plan and execute it, collecting all results and\ndisplaying them to the user. It is important to note that DataFusion performs lazy evaluation\nof the DataFrame. Until you call a method such as [`show`][datafusion.dataframe.DataFrame.show]\nor [`collect`][datafusion.dataframe.DataFrame.collect], DataFusion will not perform the query.\n" } ], "metadata": { diff --git a/docs/source/user-guide/data-sources.ipynb b/docs/source/user-guide/data-sources.ipynb index c2591d466..30f7bf199 100644 --- a/docs/source/user-guide/data-sources.ipynb +++ b/docs/source/user-guide/data-sources.ipynb @@ -25,19 +25,22 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { "cell_type": "markdown", "id": "acae54e37e7d407bbb7b55eff062a284", "metadata": {}, - "source": "\n\n\n# Data Sources\n\nDataFusion provides a wide variety of ways to get data into a DataFrame to perform operations.\n\n## Local file\n\nDataFusion has the ability to read from a variety of popular file formats, such as [Parquet](io_parquet),\n[CSV](io_csv), [JSON](io_json), and [AVRO](io_avro).\n\n" + "source": "\n\n\n# Data Sources\n\nDataFusion provides a wide variety of ways to get data into a DataFrame to perform operations.\n\n## Local file\n\nDataFusion has the ability to read from a variety of popular file formats, such as [Parquet](../io/parquet/),\n[CSV](../io/csv/), [JSON](../io/json/), and [AVRO](../io/avro/).\n\n" }, { "cell_type": "code", @@ -55,7 +58,7 @@ "cell_type": "markdown", "id": "8dd0d8092fe74a7c96281538738b07e2", "metadata": {}, - "source": "\n## Create in-memory\n\nSometimes it can be convenient to create a small DataFrame from a Python list or dictionary object.\nTo do this in DataFusion, you can use one of the three functions\n[`from_pydict`][datafusion.context.SessionContext.from_pydict],\n[`from_pylist`][datafusion.context.SessionContext.from_pylist], or\n[`create_dataframe`][datafusion.context.SessionContext.create_dataframe].\n\nAs their names suggest, `from_pydict` and `from_pylist` will create DataFrames from Python\ndictionary and list objects, respectively. `create_dataframe` assumes you will pass in a list\nof list of [PyArrow Record Batches](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html).\n\nThe following three examples all will create identical DataFrames:\n\n" + "source": "\n## Create in-memory\n\nSometimes it can be convenient to create a small DataFrame from a Python list or dictionary object.\nTo do this in DataFusion, you can use one of the three functions\n[`from_pydict`][datafusion.context.SessionContext.from_pydict],\n[`from_pylist`][datafusion.context.SessionContext.from_pylist], or\n[`create_dataframe`][datafusion.context.SessionContext.create_dataframe].\n\nAs their names suggest, [`from_pydict`][datafusion.context.SessionContext.from_pydict] and [`from_pylist`][datafusion.context.SessionContext.from_pylist] will create DataFrames from Python\ndictionary and list objects, respectively. [`create_dataframe`][datafusion.context.SessionContext.create_dataframe] assumes you will pass in a list\nof list of [PyArrow Record Batches](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html).\n\nThe following three examples all will create identical DataFrames:\n\n" }, { "cell_type": "code", @@ -98,7 +101,7 @@ "cell_type": "markdown", "id": "8edb47106e1a46a883d545849b8ab81b", "metadata": {}, - "source": "\n## Object Store\n\nDataFusion has support for multiple storage options in addition to local files.\nThe example below requires an appropriate S3 account with access credentials.\n\nSupported Object Stores are\n\n- [`AmazonS3`][datafusion.object_store.AmazonS3]\n- [`GoogleCloud`][datafusion.object_store.GoogleCloud]\n- [`Http`][datafusion.object_store.Http]\n- [`LocalFileSystem`][datafusion.object_store.LocalFileSystem]\n- [`MicrosoftAzure`][datafusion.object_store.MicrosoftAzure]\n\n```python\nfrom datafusion.object_store import AmazonS3\n\nregion = \"us-east-1\"\nbucket_name = \"yellow-trips\"\n\ns3 = AmazonS3(\n bucket_name=bucket_name,\n region=region,\n access_key_id=os.getenv(\"AWS_ACCESS_KEY_ID\"),\n secret_access_key=os.getenv(\"AWS_SECRET_ACCESS_KEY\"),\n)\n\npath = f\"s3://{bucket_name}/\"\nctx.register_object_store(\"s3://\", s3, None)\n\nctx.register_parquet(\"trips\", path)\n\nctx.table(\"trips\").show()\n```\n\n## Other DataFrame Libraries\n\nDataFusion can import DataFrames directly from other libraries, such as\n[Polars](https://pola.rs/) and [Pandas](https://pandas.pydata.org/).\nSince DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule\ninterface can be imported to DataFusion using the\n[`from_arrow`][datafusion.context.SessionContext.from_arrow] function. Older versions of Polars may\nnot support the arrow interface. In those cases, you can still import via the\n[`from_polars`][datafusion.context.SessionContext.from_polars] function.\n\n```python\nimport pandas as pd\n\ndata = { \"a\": [1, 2, 3], \"b\": [10.0, 20.0, 30.0], \"c\": [\"alpha\", \"beta\", \"gamma\"] }\npandas_df = pd.DataFrame(data)\n\ndatafusion_df = ctx.from_arrow(pandas_df)\ndatafusion_df.show()\n```\n\n```python\nimport polars as pl\npolars_df = pl.DataFrame(data)\n\ndatafusion_df = ctx.from_arrow(polars_df)\ndatafusion_df.show()\n```\n\n## Delta Lake\n\nDataFusion 43.0.0 and later support the ability to register table providers from sources such\nas Delta Lake. This will require a recent version of\n[deltalake](https://delta-io.github.io/delta-rs/) to provide the required interfaces.\n\n```python\nfrom deltalake import DeltaTable\n\ndelta_table = DeltaTable(\"path_to_table\")\nctx.register_table(\"my_delta_table\", delta_table)\ndf = ctx.table(\"my_delta_table\")\ndf.show()\n```\n\nOn older versions of `deltalake` (prior to 0.22) you can use the\n[Arrow DataSet](https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Dataset.html)\ninterface to import to DataFusion, but this does not support features such as filter push down\nwhich can lead to a significant performance difference.\n\n```python\nfrom deltalake import DeltaTable\n\ndelta_table = DeltaTable(\"path_to_table\")\nctx.register_dataset(\"my_delta_table\", delta_table.to_pyarrow_dataset())\ndf = ctx.table(\"my_delta_table\")\ndf.show()\n```\n\n## Apache Iceberg\n\nDataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface.\n\nThis requires either the [pyiceberg](https://pypi.org/project/pyiceberg/) library (>=0.10.0) or the [pyiceberg-core](https://pypi.org/project/pyiceberg-core/) library (>=0.5.0).\n\n- The `pyiceberg-core` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings.\n- The `pyiceberg` library utilizes the `pyiceberg-core` python bindings under the hood and provides a native way for Python users to interact with the DataFusion.\n\n```python\nfrom datafusion import SessionContext\nfrom pyiceberg.catalog import load_catalog\nimport pyarrow as pa\n\n# Load catalog and create/load a table\ncatalog = load_catalog(\"catalog\", type=\"in-memory\")\ncatalog.create_namespace_if_not_exists(\"default\")\n\n# Create some sample data\ndata = pa.table({\"x\": [1, 2, 3], \"y\": [4, 5, 6]})\niceberg_table = catalog.create_table(\"default.test\", schema=data.schema)\niceberg_table.append(data)\n\n# Register the table with DataFusion\nctx = SessionContext()\nctx.register_table_provider(\"test\", iceberg_table)\n\n# Query the table using DataFusion\nctx.table(\"test\").show()\n```\n\nNote that the Datafusion integration rely on features from the [Iceberg Rust](https://github.com/apache/iceberg-rust/) implementation instead of the [PyIceberg](https://github.com/apache/iceberg-python/) implementation.\nFeatures that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion.\n\n## Custom Table Provider\n\nYou can implement a custom Data Provider in Rust and expose it to DataFusion through the\nthe interface as describe in the [Custom Table Provider](io_custom_table_provider)\nsection. This is an advanced topic, but a\n[user example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example)\nis provided in the DataFusion repository.\n\n# Catalog\n\nA common technique for organizing tables is using a three level hierarchical approach. DataFusion\nsupports this form of organizing using the [`Catalog`][datafusion.catalog.Catalog],\n[`Schema`][datafusion.catalog.Schema], and [`Table`][datafusion.catalog.Table]. By default,\na [`SessionContext`][datafusion.context.SessionContext] comes with a single Catalog and a single Schema\nwith the names `datafusion` and `public`, respectively.\n\nThe default implementation uses an in-memory approach to the catalog and schema. We have support\nfor adding additional in-memory catalogs and schemas. You can access tables registered in a schema\neither through the Dataframe API or via sql commands. This can be done like in the following\nexample:\n\n```python\nimport pyarrow as pa\nfrom datafusion.catalog import Catalog, Schema\nfrom datafusion import SessionContext\n\nctx = SessionContext()\n\nmy_catalog = Catalog.memory_catalog()\nmy_schema = Schema.memory_schema()\nmy_catalog.register_schema('my_schema_name', my_schema)\nctx.register_catalog_provider('my_catalog_name', my_catalog)\n\n# Create an in-memory table\ntable = pa.table({\n 'name': ['Bulbasaur', 'Charmander', 'Squirtle'],\n 'type': ['Grass', 'Fire', 'Water'],\n 'hp': [45, 39, 44],\n})\ndf = ctx.create_dataframe([table.to_batches()], name='pokemon')\n\nmy_schema.register_table('pokemon', df)\n\nctx.sql('SELECT * FROM my_catalog_name.my_schema_name.pokemon').show()\n```\n\n## User Defined Catalog and Schema\n\nIf the in-memory catalogs are insufficient for your uses, there are two approaches you can take\nto implementing a custom catalog and/or schema. In the below discussion, we describe how to\nimplement these for a Catalog, but the approach to implementing for a Schema is nearly\nidentical.\n\nDataFusion supports Catalogs written in either Rust or Python. If you write a Catalog in Rust,\nyou will need to export it as a Python library via PyO3. There is a complete example of a\ncatalog implemented this way in the\n[examples folder](https://github.com/apache/datafusion-python/tree/main/examples/)\nof our repository. Writing catalog providers in Rust provides typically can lead to significant\nperformance improvements over the Python based approach.\n\nTo implement a Catalog in Python, you will need to inherit from the abstract base class\n[`CatalogProvider`][datafusion.catalog.CatalogProvider]. There are examples in the\n[unit tests](https://github.com/apache/datafusion-python/tree/main/python/tests) of\nimplementing a basic Catalog in Python where we simply keep a dictionary of the\nregistered Schemas.\n\nOne important note for developers is that when we have a Catalog defined in Python, we have\ntwo different ways of accessing this Catalog. First, we register the catalog with a Rust\nwrapper. This allows for any rust based code to call the Python functions as necessary.\nSecond, if the user access the Catalog via the Python API, we identify this and return back\nthe original Python object that implements the Catalog. This is an important distinction\nfor developers because we do *not* return a Python wrapper around the Rust wrapper of the\noriginal Python object.\n" + "source": "\n## Object Store\n\nDataFusion has support for multiple storage options in addition to local files.\nThe example below requires an appropriate S3 account with access credentials.\n\nSupported Object Stores are\n\n- [`AmazonS3`][datafusion.object_store.AmazonS3]\n- [`GoogleCloud`][datafusion.object_store.GoogleCloud]\n- [`Http`][datafusion.object_store.Http]\n- [`LocalFileSystem`][datafusion.object_store.LocalFileSystem]\n- [`MicrosoftAzure`][datafusion.object_store.MicrosoftAzure]\n\n```python\nfrom datafusion.object_store import AmazonS3\n\nregion = \"us-east-1\"\nbucket_name = \"yellow-trips\"\n\ns3 = AmazonS3(\n bucket_name=bucket_name,\n region=region,\n access_key_id=os.getenv(\"AWS_ACCESS_KEY_ID\"),\n secret_access_key=os.getenv(\"AWS_SECRET_ACCESS_KEY\"),\n)\n\npath = f\"s3://{bucket_name}/\"\nctx.register_object_store(\"s3://\", s3, None)\n\nctx.register_parquet(\"trips\", path)\n\nctx.table(\"trips\").show()\n```\n\n## Other DataFrame Libraries\n\nDataFusion can import DataFrames directly from other libraries, such as\n[Polars](https://pola.rs/) and [Pandas](https://pandas.pydata.org/).\nSince DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule\ninterface can be imported to DataFusion using the\n[`from_arrow`][datafusion.context.SessionContext.from_arrow] function. Older versions of Polars may\nnot support the arrow interface. In those cases, you can still import via the\n[`from_polars`][datafusion.context.SessionContext.from_polars] function.\n\n```python\nimport pandas as pd\n\ndata = { \"a\": [1, 2, 3], \"b\": [10.0, 20.0, 30.0], \"c\": [\"alpha\", \"beta\", \"gamma\"] }\npandas_df = pd.DataFrame(data)\n\ndatafusion_df = ctx.from_arrow(pandas_df)\ndatafusion_df.show()\n```\n\n```python\nimport polars as pl\npolars_df = pl.DataFrame(data)\n\ndatafusion_df = ctx.from_arrow(polars_df)\ndatafusion_df.show()\n```\n\n## Delta Lake\n\nDataFusion 43.0.0 and later support the ability to register table providers from sources such\nas Delta Lake. This will require a recent version of\n[deltalake](https://delta-io.github.io/delta-rs/) to provide the required interfaces.\n\n```python\nfrom deltalake import DeltaTable\n\ndelta_table = DeltaTable(\"path_to_table\")\nctx.register_table(\"my_delta_table\", delta_table)\ndf = ctx.table(\"my_delta_table\")\ndf.show()\n```\n\nOn older versions of [`deltalake`](https://delta-io.github.io/delta-rs/) (prior to 0.22) you can use the\n[Arrow DataSet](https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Dataset.html)\ninterface to import to DataFusion, but this does not support features such as filter push down\nwhich can lead to a significant performance difference.\n\n```python\nfrom deltalake import DeltaTable\n\ndelta_table = DeltaTable(\"path_to_table\")\nctx.register_dataset(\"my_delta_table\", delta_table.to_pyarrow_dataset())\ndf = ctx.table(\"my_delta_table\")\ndf.show()\n```\n\n## Apache Iceberg\n\nDataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface.\n\nThis requires either the [pyiceberg](https://pypi.org/project/pyiceberg/) library (>=0.10.0) or the [pyiceberg-core](https://pypi.org/project/pyiceberg-core/) library (>=0.5.0).\n\n- The `pyiceberg-core` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings.\n- The `pyiceberg` library utilizes the `pyiceberg-core` python bindings under the hood and provides a native way for Python users to interact with the DataFusion.\n\n```python\nfrom datafusion import SessionContext\nfrom pyiceberg.catalog import load_catalog\nimport pyarrow as pa\n\n# Load catalog and create/load a table\ncatalog = load_catalog(\"catalog\", type=\"in-memory\")\ncatalog.create_namespace_if_not_exists(\"default\")\n\n# Create some sample data\ndata = pa.table({\"x\": [1, 2, 3], \"y\": [4, 5, 6]})\niceberg_table = catalog.create_table(\"default.test\", schema=data.schema)\niceberg_table.append(data)\n\n# Register the table with DataFusion\nctx = SessionContext()\nctx.register_table_provider(\"test\", iceberg_table)\n\n# Query the table using DataFusion\nctx.table(\"test\").show()\n```\n\nNote that the Datafusion integration rely on features from the [Iceberg Rust](https://github.com/apache/iceberg-rust/) implementation instead of the [PyIceberg](https://github.com/apache/iceberg-python/) implementation.\nFeatures that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion.\n\n## Custom Table Provider\n\nYou can implement a custom Data Provider in Rust and expose it to DataFusion through the\nthe interface as describe in the [Custom Table Provider](../io/table_provider/)\nsection. This is an advanced topic, but a\n[user example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example)\nis provided in the DataFusion repository.\n\n# Catalog\n\nA common technique for organizing tables is using a three level hierarchical approach. DataFusion\nsupports this form of organizing using the [`Catalog`][datafusion.catalog.Catalog],\n[`Schema`][datafusion.catalog.Schema], and [`Table`][datafusion.catalog.Table]. By default,\na [`SessionContext`][datafusion.context.SessionContext] comes with a single Catalog and a single Schema\nwith the names `datafusion` and `public`, respectively.\n\nThe default implementation uses an in-memory approach to the catalog and schema. We have support\nfor adding additional in-memory catalogs and schemas. You can access tables registered in a schema\neither through the Dataframe API or via sql commands. This can be done like in the following\nexample:\n\n```python\nimport pyarrow as pa\nfrom datafusion.catalog import Catalog, Schema\nfrom datafusion import SessionContext\n\nctx = SessionContext()\n\nmy_catalog = Catalog.memory_catalog()\nmy_schema = Schema.memory_schema()\nmy_catalog.register_schema('my_schema_name', my_schema)\nctx.register_catalog_provider('my_catalog_name', my_catalog)\n\n# Create an in-memory table\ntable = pa.table({\n 'name': ['Bulbasaur', 'Charmander', 'Squirtle'],\n 'type': ['Grass', 'Fire', 'Water'],\n 'hp': [45, 39, 44],\n})\ndf = ctx.create_dataframe([table.to_batches()], name='pokemon')\n\nmy_schema.register_table('pokemon', df)\n\nctx.sql('SELECT * FROM my_catalog_name.my_schema_name.pokemon').show()\n```\n\n## User Defined Catalog and Schema\n\nIf the in-memory catalogs are insufficient for your uses, there are two approaches you can take\nto implementing a custom catalog and/or schema. In the below discussion, we describe how to\nimplement these for a Catalog, but the approach to implementing for a Schema is nearly\nidentical.\n\nDataFusion supports Catalogs written in either Rust or Python. If you write a Catalog in Rust,\nyou will need to export it as a Python library via PyO3. There is a complete example of a\ncatalog implemented this way in the\n[examples folder](https://github.com/apache/datafusion-python/tree/main/examples/)\nof our repository. Writing catalog providers in Rust provides typically can lead to significant\nperformance improvements over the Python based approach.\n\nTo implement a Catalog in Python, you will need to inherit from the abstract base class\n[`CatalogProvider`][datafusion.catalog.CatalogProvider]. There are examples in the\n[unit tests](https://github.com/apache/datafusion-python/tree/main/python/tests) of\nimplementing a basic Catalog in Python where we simply keep a dictionary of the\nregistered Schemas.\n\nOne important note for developers is that when we have a Catalog defined in Python, we have\ntwo different ways of accessing this Catalog. First, we register the catalog with a Rust\nwrapper. This allows for any rust based code to call the Python functions as necessary.\nSecond, if the user access the Catalog via the Python API, we identify this and return back\nthe original Python object that implements the Catalog. This is an important distinction\nfor developers because we do *not* return a Python wrapper around the Rust wrapper of the\noriginal Python object.\n" } ], "metadata": { diff --git a/docs/source/user-guide/dataframe/index.md b/docs/source/user-guide/dataframe/index.md index bdfe4c3f9..787f76ace 100644 --- a/docs/source/user-guide/dataframe/index.md +++ b/docs/source/user-guide/dataframe/index.md @@ -77,7 +77,7 @@ DataFrames can be created in several ways: ``` For detailed information about reading from different data sources, see the [I/O Guide](../io/index.md). -For custom data sources, see [io_custom_table_provider](io_custom_table_provider). +For custom data sources, see [io_custom_table_provider](../../io/table_provider/). ## Common DataFrame Operations @@ -345,7 +345,7 @@ rendering, formatting options, and advanced styling, see [rendering](rendering.m **Functions for creating expressions:** -- [`column`][datafusion.column] - Reference a column by name +- [`column`][datafusion.col.column] - Reference a column by name - [`literal`][datafusion.literal] - Create a literal value expression ## Built-in Functions @@ -363,9 +363,8 @@ DataFusion populates per-operator runtime statistics such as row counts and compute time. See [execution-metrics](execution-metrics.md) for a full explanation and worked example. -```{toctree} -:maxdepth: 1 +## Further reading -rendering -execution-metrics -``` +- [Rendering](rendering.md) — Jupyter HTML repr customization. +- [Execution Metrics](execution-metrics.md) — per-operator row counts, + compute time, spill events. diff --git a/docs/source/user-guide/dataframe/rendering.md b/docs/source/user-guide/dataframe/rendering.md index 4ae9854e5..d6c6083a5 100644 --- a/docs/source/user-guide/dataframe/rendering.md +++ b/docs/source/user-guide/dataframe/rendering.md @@ -20,10 +20,11 @@ # DataFrame Rendering DataFusion provides configurable rendering for DataFrames in both plain text and HTML -formats. The `datafusion.dataframe_formatter` module controls how DataFrames are +formats. The [`datafusion.dataframe_formatter`](../../reference/formatter.md) module controls how DataFrames are displayed in Jupyter notebooks (via `_repr_html_`), in the terminal (via `__repr__`), and anywhere else a string or HTML representation is needed. + ## Basic Rendering In a Jupyter environment, displaying a DataFrame triggers HTML rendering: @@ -70,7 +71,7 @@ The formatter settings affect all DataFrames displayed after configuration. ## Custom Style Providers For HTML styling, you can create a custom style provider that implements the -`StyleProvider` protocol: +[`StyleProvider`][datafusion.dataframe_formatter.StyleProvider] protocol: ```python from datafusion.dataframe_formatter import configure_formatter @@ -157,7 +158,7 @@ When `use_shared_styles=True`: ## Working with the Formatter Directly -You can use `get_formatter()` and `set_formatter()` for direct access to the global +You can use [`get_formatter()`][datafusion.dataframe_formatter.get_formatter] and [`set_formatter()`][datafusion.dataframe_formatter.set_formatter] for direct access to the global formatter instance: ```python @@ -208,20 +209,9 @@ These parameters help balance comprehensive data display against performance con ## Best Practices -1. **Global Configuration**: Use `configure_formatter()` at the beginning of your notebook to set up consistent formatting for all DataFrames. +1. **Global Configuration**: Use [`configure_formatter()`][datafusion.dataframe_formatter.configure_formatter] at the beginning of your notebook to set up consistent formatting for all DataFrames. 2. **Memory Management**: Set appropriate `max_memory_bytes` limits to prevent performance issues with large datasets. 3. **Shared Styles**: Keep `use_shared_styles=True` (default) for better performance in notebooks with multiple DataFrames. -4. **Reset When Needed**: Call `reset_formatter()` when you want to start fresh with default settings. +4. **Reset When Needed**: Call [`reset_formatter()`][datafusion.dataframe_formatter.reset_formatter] when you want to start fresh with default settings. 5. **Cell Expansion**: Use `enable_cell_expansion=True` when cells might contain longer content that users may want to see in full. -## Additional Resources - -- [../dataframe/index](../dataframe/index.md) - Complete guide to using DataFrames -- [../io/index](../io/index.md) - I/O Guide for reading data from various sources -- [../data-sources](../data-sources.ipynb) - Comprehensive data sources guide -- [io_csv](io_csv) - CSV file reading -- [io_parquet](io_parquet) - Parquet file reading -- [io_json](io_json) - JSON file reading -- [io_avro](io_avro) - Avro file reading -- [io_custom_table_provider](io_custom_table_provider) - Custom table providers -- [API Reference](https://arrow.apache.org/datafusion-python/api/index.html) - Full API reference diff --git a/docs/source/user-guide/index.md b/docs/source/user-guide/index.md index 0155c4e91..c689e1861 100644 --- a/docs/source/user-guide/index.md +++ b/docs/source/user-guide/index.md @@ -22,18 +22,27 @@ The user guide walks through installing DataFusion in Python, building queries with the DataFrame API or SQL, reading and writing data, and tuning execution. -```{toctree} -:maxdepth: 2 +## Contents -introduction -basics -data-sources -dataframe/index -common-operations/index -io/index -configuration -distributing-work -sql -upgrade-guides -ai-coding-assistants -``` +- [Introduction](introduction.ipynb) — what DataFusion in Python is and + when to reach for it. +- [Concepts](concepts.ipynb) — `SessionContext`, `DataFrame`, and + `Expr` at a glance. +- [Data Sources](data-sources.ipynb) — reading Parquet / CSV / JSON / + Avro, in-memory DataFrames, object stores, Delta Lake, Iceberg, + custom table providers, and catalogs. +- [DataFrame](dataframe/index.md) — building queries with the DataFrame + API, rendering, and execution metrics. +- [Common Operations](common-operations/index.md) — select, filter, + joins, aggregations, windows, expressions, UDFs/UDAFs. +- [I/O](io/index.md) — per-format reading and writing details. +- [Configuration](configuration.md) — `SessionConfig` / + `RuntimeEnvBuilder` tuning options. +- [Distributing Work](distributing-work.md) — shipping expressions to + worker processes via pickle / cloudpickle, FFI-capsule UDFs, and + the sender/worker context model. +- [SQL](sql.ipynb) — registering tables and running SQL queries. +- [Upgrade Guides](upgrade-guides.md) — notes on cross-version + migrations. +- [AI Coding Assistants](ai-coding-assistants.md) — agent-facing + reference material and skill files. diff --git a/docs/source/user-guide/introduction.ipynb b/docs/source/user-guide/introduction.ipynb index 76eb77eea..3a0666529 100644 --- a/docs/source/user-guide/introduction.ipynb +++ b/docs/source/user-guide/introduction.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { @@ -78,9 +81,12 @@ { "cell_type": "code", "execution_count": null, + "id": "10185d26023b46108eb7d9f57d49d2b3", "metadata": {}, "outputs": [], - "source": "display(df)" + "source": [ + "display(df)" + ] } ], "metadata": { diff --git a/docs/source/user-guide/io/arrow.ipynb b/docs/source/user-guide/io/arrow.ipynb index 2c3ed5787..243266b73 100644 --- a/docs/source/user-guide/io/arrow.ipynb +++ b/docs/source/user-guide/io/arrow.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { diff --git a/docs/source/user-guide/io/index.md b/docs/source/user-guide/io/index.md index 764e05cb9..6b5478186 100644 --- a/docs/source/user-guide/io/index.md +++ b/docs/source/user-guide/io/index.md @@ -19,13 +19,27 @@ # IO -```{toctree} -:maxdepth: 2 - -arrow -avro -csv -json -parquet -table_provider -``` +DataFusion can read and write a range of file formats and stream data in +through Arrow-compatible Python objects. + +## File formats + +| Format | Reader | Notes | +|---|---|---| +| [Apache Arrow](arrow.ipynb) | [`SessionContext.read_arrow`][datafusion.context.SessionContext.read_arrow] | Single Arrow IPC file. | +| [Avro](avro.md) | [`SessionContext.read_avro`][datafusion.context.SessionContext.read_avro] | Schema-on-read; requires the Avro feature in the wheel. | +| [CSV](csv.md) | [`SessionContext.read_csv`][datafusion.context.SessionContext.read_csv] | Header inference, custom delimiters, gzip/bz2 compression. | +| [JSON](json.md) | [`SessionContext.read_json`][datafusion.context.SessionContext.read_json] | Newline-delimited JSON; one record per line. | +| [Parquet](parquet.md) | [`SessionContext.read_parquet`][datafusion.context.SessionContext.read_parquet] | Predicate / projection push-down, partitioned datasets. | + +## Custom sources + +- [Table Provider](table_provider.md) — register an arbitrary data source + (Delta Lake, Iceberg, your own Rust crate, etc.) by implementing the + table-provider FFI interface. + +## See also + +- [Data Sources](../data-sources.ipynb) — concept overview, including + in-memory DataFrame creation from `pyarrow` / `pandas` / `polars` and + object-store integration. diff --git a/docs/source/user-guide/sql.ipynb b/docs/source/user-guide/sql.ipynb index f12b45d2a..ce87e4335 100644 --- a/docs/source/user-guide/sql.ipynb +++ b/docs/source/user-guide/sql.ipynb @@ -25,12 +25,15 @@ " literal,\n", ")\n", "from datafusion import functions as f # noqa: F401\n", + "from datafusion.dataframe_formatter import configure_formatter\n", "\n", "_p = pathlib.Path.cwd()\n", "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", " _p = _p.parent\n", "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)" + " os.chdir(_p)\n", + "\n", + "configure_formatter(max_rows=10, show_truncation_message=False)" ] }, { @@ -87,7 +90,7 @@ "cell_type": "markdown", "id": "8edb47106e1a46a883d545849b8ab81b", "metadata": {}, - "source": "\nWhen passing parameters like the example above we convert the Python objects\ninto their string representation. We also have special case handling\nfor [`DataFrame`][datafusion.dataframe.DataFrame] objects, since they cannot simply\nbe turned into string representations for an SQL query. In these cases we\nwill register a temporary view in the [`SessionContext`][datafusion.context.SessionContext]\nusing a generated table name.\n\nThe formatting for passing string replacement objects is to precede the\nvariable name with a single `$`. This works for all dialects in\nthe SQL parser except `hive` and `mysql`. Since these dialects do not\nsupport named placeholders, we are unable to do this type of replacement.\nWe recommend either switching to another dialect or using Python\nf-string style replacement.\n\n!!! warning\n\n To support DataFrame parameterized queries, your session must support\n registration of temporary views. The default\n [`CatalogProvider`][datafusion.catalog.CatalogProvider] and\n [`SchemaProvider`][datafusion.catalog.SchemaProvider] do have this capability.\n If you have implemented custom providers, it is important that temporary\n views do not persist across [`SessionContext`][datafusion.context.SessionContext]\n or you may get unintended consequences.\n\nThe following example shows passing in both a [`DataFrame`][datafusion.dataframe.DataFrame]\nobject as well as a Python object to be used in parameterized replacement.\n\n" + "source": "\nWhen passing parameters like the example above we convert the Python objects\ninto their string representation. We also have special case handling\nfor [`DataFrame`][datafusion.dataframe.DataFrame] objects, since they cannot simply\nbe turned into string representations for an SQL query. In these cases we\nwill register a temporary view in the [`SessionContext`][datafusion.context.SessionContext]\nusing a generated table name.\n\nThe formatting for passing string replacement objects is to precede the\nvariable name with a single `$`. This works for all dialects in\nthe SQL parser except `hive` and `mysql`. Since these dialects do not\nsupport named placeholders, we are unable to do this type of replacement.\nWe recommend either switching to another dialect or using Python\nf-string style replacement.\n\n
\n

Warning

\n\nTo support DataFrame parameterized queries, your session must support\nregistration of temporary views. The default\n[`CatalogProvider`][datafusion.catalog.CatalogProvider] and\n[`SchemaProvider`][datafusion.catalog.SchemaProvider] do have this capability.\nIf you have implemented custom providers, it is important that temporary\nviews do not persist across [`SessionContext`][datafusion.context.SessionContext]\nor you may get unintended consequences.\n\n
\n\nThe following example shows passing in both a [`DataFrame`][datafusion.dataframe.DataFrame]\nobject as well as a Python object to be used in parameterized replacement.\n\n" }, { "cell_type": "code", diff --git a/mkdocs.yml b/mkdocs.yml index bc3e6ab84..7ac398923 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -72,6 +72,7 @@ plugins: inventories: - https://docs.python.org/3/objects.inv - https://arrow.apache.org/docs/objects.inv + - https://docs.pola.rs/api/python/stable/objects.inv options: docstring_style: google show_source: false @@ -128,8 +129,8 @@ nav: - Execution Metrics: user-guide/dataframe/execution-metrics.md - Common Operations: - user-guide/common-operations/index.md - - Views: user-guide/common-operations/views.md - Basic Info: user-guide/common-operations/basic-info.ipynb + - Views: user-guide/common-operations/views.md - Select and Filter: user-guide/common-operations/select-and-filter.ipynb - Expressions: user-guide/common-operations/expressions.ipynb - Joins: user-guide/common-operations/joins.ipynb @@ -158,6 +159,7 @@ nav: - reference/index.md - SessionContext: reference/context.md - DataFrame: reference/dataframe.md + - DataFrame Formatter: reference/formatter.md - Expr: reference/expr.md - Functions: reference/functions.md - User-Defined Functions: reference/user_defined.md diff --git a/python/datafusion/context.py b/python/datafusion/context.py index c5634fe3f..f0149f033 100644 --- a/python/datafusion/context.py +++ b/python/datafusion/context.py @@ -748,7 +748,7 @@ def create_dataframe( """Create and return a dataframe using the provided partitions. Args: - partitions: [`RecordBatch`][pa.RecordBatch] partitions to register. + partitions: [`RecordBatch`][pyarrow.RecordBatch] partitions to register. name: Resultant dataframe name. schema: Schema for the partitions. @@ -930,7 +930,7 @@ def register_udtf(self, func: TableFunction) -> None: self.ctx.register_udtf(func._udtf) def register_batch(self, name: str, batch: pa.RecordBatch) -> None: - """Register a single [`RecordBatch`][pa.RecordBatch] as a table. + """Register a single [`RecordBatch`][pyarrow.RecordBatch] as a table. Args: name: Name of the resultant table. @@ -995,7 +995,7 @@ def read_batches(self, batches: Iterable[pa.RecordBatch]) -> DataFrame: """Return a `DataFrame` reading the given batches. All batches must share the same schema. Any iterable of - [`RecordBatch`][pa.RecordBatch] is accepted (list, tuple, generator); + [`RecordBatch`][pyarrow.RecordBatch] is accepted (list, tuple, generator); it is materialized into a list before being handed to the underlying Rust binding. Unlike `register_record_batches`, this does not register the batches as a named table; it returns @@ -1279,7 +1279,7 @@ def register_arrow( ) def register_dataset(self, name: str, dataset: pa.dataset.Dataset) -> None: - """Register a [`Dataset`][pa.dataset.Dataset] as a table. + """Register a [`Dataset`][pyarrow.dataset.Dataset] as a table. Args: name: Name of the table to register. diff --git a/python/datafusion/dataframe.py b/python/datafusion/dataframe.py index 692d64cde..5a74a1b40 100644 --- a/python/datafusion/dataframe.py +++ b/python/datafusion/dataframe.py @@ -645,7 +645,7 @@ def filter(self, *predicates: Expr | str) -> DataFrame: out. If more than one predicate is provided, these predicates will be combined as a logical AND. Each ``predicate`` can be an [`Expr`][datafusion.expr.Expr] created using helper functions such as - [`col`][datafusion.col] or [`lit`][datafusion.lit], or a SQL expression string + `col` or [`lit`][datafusion.lit], or a SQL expression string that will be parsed against the DataFrame schema. If more complex logic is required, see the logical operations in [`functions`][datafusion.functions]. @@ -697,7 +697,7 @@ def with_column(self, name: str, expr: Expr | str) -> DataFrame: """Add an additional column to the DataFrame. The ``expr`` must be an [`Expr`][datafusion.expr.Expr] constructed with - [`col`][datafusion.col] or [`lit`][datafusion.lit], or a SQL expression + [`col`][datafusion.col.col] or [`lit`][datafusion.lit], or a SQL expression string that will be parsed against the DataFrame schema. Examples: @@ -725,7 +725,7 @@ def with_columns( By passing expressions, iterables of expressions, string SQL expressions, or named expressions. All expressions must be [`Expr`][datafusion.expr.Expr] objects created via - [`col`][datafusion.col] or [`lit`][datafusion.lit], or SQL expression strings. + `col` or [`lit`][datafusion.lit], or SQL expression strings. To pass named expressions use the form ``name=Expr``. Example usage: The following will add 4 columns labeled ``a``, ``b``, ``c``, @@ -1159,7 +1159,7 @@ def join_on( """Join two `DataFrame` using the specified expressions. Join predicates must be [`Expr`][datafusion.expr.Expr] objects, typically - built with [`col`][datafusion.col]. On expressions are used to support + built with [`col`][datafusion.col.col]. On expressions are used to support in-equality predicates. Equality predicates are correctly optimized. Use `col` on each DataFrame **before** the join to diff --git a/python/datafusion/expr.py b/python/datafusion/expr.py index 1cd52f694..5e9b0cc5d 100644 --- a/python/datafusion/expr.py +++ b/python/datafusion/expr.py @@ -28,7 +28,7 @@ [`sort`][datafusion.dataframe.DataFrame.sort]. Convenience constructors are re-exported at the package level: -[`col`][datafusion.col] / [`column`][datafusion.column] for column references +[`col`][datafusion.col.col] / [`column`][datafusion.col.column] for column references and [`lit`][datafusion.lit] / [`literal`][datafusion.literal] for scalar literals. @@ -262,7 +262,7 @@ def ensure_expr(value: Expr | Any) -> expr_internal.Expr: """Return the internal expression from ``Expr`` or raise ``TypeError``. This helper rejects plain strings and other non-`Expr` values so - higher level APIs consistently require explicit [`col`][datafusion.col] or + higher level APIs consistently require explicit [`col`][datafusion.col.col] or [`lit`][datafusion.lit] expressions. See Also: diff --git a/python/datafusion/record_batch.py b/python/datafusion/record_batch.py index 0722f0c63..afdeb3918 100644 --- a/python/datafusion/record_batch.py +++ b/python/datafusion/record_batch.py @@ -33,7 +33,7 @@ class RecordBatch: - """This class is essentially a wrapper for [`RecordBatch`][pa.RecordBatch].""" + """This class is essentially a wrapper for [`RecordBatch`][pyarrow.RecordBatch].""" def __init__(self, record_batch: df_internal.RecordBatch) -> None: """This constructor is generally not called by the end user. @@ -43,7 +43,7 @@ def __init__(self, record_batch: df_internal.RecordBatch) -> None: self.record_batch = record_batch def to_pyarrow(self) -> pa.RecordBatch: - """Convert to [`RecordBatch`][pa.RecordBatch].""" + """Convert to [`RecordBatch`][pyarrow.RecordBatch].""" return self.record_batch.to_pyarrow() def __arrow_c_array__( From d5579c39f85d8af368fe8b7b31c4405c8d1696f9 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Mon, 8 Jun 2026 08:16:49 +0200 Subject: [PATCH 08/18] docs: content cleanup, dead-link fixes, FFI page refresh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit User-guide * Drop the obsolete `grouping().alias()` workaround in `common-operations/aggregations.ipynb` — verified that `apache/datafusion#21411` no longer reproduces against the current pinned DataFusion. Both the admonition note and the `with_column_renamed` post-hoc cleanup loops in cells 24 and 32 are gone; the examples now alias `grouping()` directly. Same `.. warning::` block stripped from the `grouping()` docstring in `python/datafusion/functions.py`. * `distributing-work.md`: - Point `datafusion-distributed` at the active `datafusion-contrib/datafusion-distributed` repo (the previous `apache/datafusion-distributed` 404s). - Point Ballista at `https://datafusion.apache.org/ballista/` instead of the GitHub source. - Shorten the "Implicit; access via" link label from `SessionContext.global_ctx` to just `global_ctx` so the session-slot reference table is easier to scan. - Replace the two file-level example links (`multiprocessing_pickle_expr.py` / `ray_pickle_expr.py`) with a single link to the `examples/` folder so the docs don't drift when specific scripts get renamed. * `sql.ipynb`: fix the broken `[configuration options](configuration)` bare anchor — now `../configuration/`. * `upgrade-guides.md`: split the `## DataFusion 54.0.0` section into two `###` subsections so the `Config` -> `SessionConfig` and the `distinct` argument on `sum`/`avg` changes are independently scannable. * Rename the Common Operations nav entry from "UDF and UDAF" to "User-Defined Functions" (the page also covers UDWF and UDTF), and expand the index list bullet to enumerate all four flavors. Contributor guide * `introduction.md`: fix the `[PyO3 class mutability guidelines]` bare anchor — now `ffi.md#pyo3-class-mutability-guidelines`. Also correct `maturin develop -uv` -> `maturin develop --uv`. * `ffi.md`: - Strip `{file}` MyST roles that mistune passed through as literal text. `dev/rewrite_doc_roles.py` extended to handle `{file}`, `{samp}`, and `{kbd}` for future passes. - Strip the stray double-backtick `` `` `datafusion-python` `` `` wrapping that rendered as literal extra backticks. - Fix the `[Data Sources](user_guide_data_sources)` bare anchor — now `../user-guide/data-sources.ipynb`. - Update the abi_stable mention: the FFI implementation switched to `stabby` in DataFusion 54.0.0; the page now points at `stabby` with a parenthetical note about `abi_stable`. - Refresh the `FFI_TableProvider` usage example to match `examples/datafusion-ffi-example/src/table_provider.rs`: `FFI_TableProvider::new_with_ffi_codec(...)` with a logical codec pulled out of the calling session via `ffi_logical_codec_from_pycapsule`. Bump the docs.rs link from `datafusion/45.0.0` to `datafusion/latest`. - Update the receiver-side snippet: `ffi_provider.into()` now produces an `Arc` directly, mirroring `crates/util/src/lib.rs::table_provider_from_pycapsule`. The `ForeignTableProvider` wrapper is no longer needed and is mentioned only as a historical parenthetical. PyCapsule construction modernized to the `cr"..."` literal + `PyCapsule::new` form used by the current example file. Co-Authored-By: Claude Opus 4.7 --- dev/rewrite_doc_roles.py | 7 +- docs/source/contributor-guide/ffi.md | 72 +++++++++++-------- docs/source/contributor-guide/introduction.md | 4 +- .../common-operations/aggregations.ipynb | 49 +++---------- .../user-guide/common-operations/index.md | 5 +- docs/source/user-guide/distributing-work.md | 17 +++-- docs/source/user-guide/sql.ipynb | 2 +- docs/source/user-guide/upgrade-guides.md | 4 ++ mkdocs.yml | 2 +- python/datafusion/functions.py | 10 --- 10 files changed, 76 insertions(+), 96 deletions(-) diff --git a/dev/rewrite_doc_roles.py b/dev/rewrite_doc_roles.py index de26f74cf..25a40ee79 100644 --- a/dev/rewrite_doc_roles.py +++ b/dev/rewrite_doc_roles.py @@ -75,8 +75,11 @@ ), lambda m: f"[`{m.group(1).strip()}`][{m.group(2)}]", ), - # {code}`text` -> `text` - (re.compile(r"\{code\}`([^`]+)`"), lambda m: f"`{m.group(1)}`"), + # {code}`text`, {file}`path`, {samp}`text`, {kbd}`keys` -> `text` + ( + re.compile(r"\{(?:code|file|samp|kbd)\}`([^`]+)`"), + lambda m: f"`{m.group(1)}`", + ), # {doc}`Label ` -> [Label](path.md) ( re.compile(r"\{doc\}`([^<`]+)\s*<([^>]+)>`"), diff --git a/docs/source/contributor-guide/ffi.md b/docs/source/contributor-guide/ffi.md index a8e9c6575..a269c5e80 100644 --- a/docs/source/contributor-guide/ffi.md +++ b/docs/source/contributor-guide/ffi.md @@ -74,12 +74,12 @@ code that does **not** require the `datafusion-python` crate as a dependency, ex code in Python via PyO3, and have it interact with the DataFusion Python package. Early adopters of this approach include [delta-rs](https://delta-io.github.io/delta-rs/) -who has adapted their Table Provider for use in `` `datafusion-python` `` with only a few lines +who has adapted their Table Provider for use in `datafusion-python` with only a few lines of code. Also, the DataFusion Python project uses the existing definitions from [Apache Arrow CStream Interface](https://arrow.apache.org/docs/format/CStreamInterface.html) to support importing **and** exporting tables. Any Python package that supports reading the Arrow C Stream interface can work with DataFusion Python out of the box! You can read -more about working with Arrow sources in the [Data Sources](user_guide_data_sources) +more about working with Arrow sources in the [Data Sources](../user-guide/data-sources.ipynb) page. To learn more about the Foreign Function Interface in Rust, the @@ -118,23 +118,35 @@ The bulk of the code necessary to perform our FFI operations is in the upstream documentation in the [datafusion-ffi] crate. Our FFI implementation is narrowly focused at sharing data and functions with Rust backed -libraries. This allows us to use the [abi_stable crate](https://crates.io/crates/abi_stable). -This is an excellent crate that allows for easy conversion between Rust native types -and FFI-safe alternatives. For example, if you needed to pass a `Vec` via FFI, -you can simply convert it to a `RVec` in an intuitive manner. It also supports -features like `RResult` and `ROption` that do not have an obvious translation to a +libraries. Starting in DataFusion 54.0.0 we use the +[stabby crate](https://crates.io/crates/stabby) (previously the +[abi_stable crate](https://crates.io/crates/abi_stable)). `stabby` provides +FFI-safe equivalents of common Rust types with a thinner runtime cost +and stricter ABI stability guarantees. For example, passing a +`Vec` across the FFI boundary is done via stabby's `Vec` / +`String` wrappers, and the crate also supplies FFI-safe analogues of +`Result` and `Option` that do not have an obvious translation to a C equivalent. The [datafusion-ffi] crate has been designed to make it easy to convert from DataFusion traits into their FFI counterparts. For example, if you have defined a custom -[TableProvider](https://docs.rs/datafusion/45.0.0/datafusion/catalog/trait.TableProvider.html) -and you want to create a sharable FFI counterpart, you could write: +[TableProvider](https://docs.rs/datafusion/latest/datafusion/catalog/trait.TableProvider.html) +and you want to expose it through a PyCapsule, you can pull the logical +codec out of the calling session and hand both to `FFI_TableProvider`: ```rust +use datafusion_ffi::table_provider::FFI_TableProvider; +use datafusion_python_util::ffi_logical_codec_from_pycapsule; + let my_provider = MyTableProvider::default(); -let ffi_provider = FFI_TableProvider::new(Arc::new(my_provider), false, None); +let codec = ffi_logical_codec_from_pycapsule(session)?; +let ffi_provider = + FFI_TableProvider::new_with_ffi_codec(Arc::new(my_provider), false, None, codec); ``` +See [`examples/datafusion-ffi-example/src/table_provider.rs`](https://github.com/apache/datafusion-python/blob/main/examples/datafusion-ffi-example/src/table_provider.rs) +for a complete runnable example. + ## PyO3 class mutability guidelines @@ -144,7 +156,7 @@ interior-mutable state. In practice this means that any `#[pyclass]` containing unless there is a compelling reason not to. The execution context illustrates the preferred pattern. `PySessionContext` in -{file}`src/context.rs` stays frozen even though it shares mutable state internally via +`src/context.rs` stays frozen even though it shares mutable state internally via `SessionContext`. This ensures PyO3 tracks borrows correctly while Python-facing APIs clone the inner `SessionContext` or return new wrappers instead of mutating the existing instance in place: @@ -160,7 +172,7 @@ pub struct PySessionContext { Occasionally a type must remain mutable—for example when PyO3 attribute setters need to update fields directly. In these rare cases add an inline justification so reviewers and future contributors understand why `frozen` is unsafe to enable. `DataTypeMap` in -{file}`src/common/data_type.rs` includes such a comment because PyO3 still needs to track +`src/common/data_type.rs` includes such a comment because PyO3 still needs to track field updates: ```rust @@ -181,22 +193,23 @@ When reviewers encounter a mutable `#[pyclass]` without a comment, they should r an explanation or ask that `frozen` be added. Keeping these wrappers frozen by default helps avoid subtle bugs stemming from PyO3's interior mutability tracking. -If you were interfacing with a library that provided the above `FFI_TableProvider` and -you needed to turn it back into an `TableProvider`, you can turn it into a -`ForeignTableProvider` with implements the `TableProvider` trait. +If you are interfacing with a library that provided the above `FFI_TableProvider` and +need a usable `TableProvider`, the `.into()` conversion now yields an +`Arc` directly: ```rust -let foreign_provider: ForeignTableProvider = ffi_provider.into(); +let provider: Arc = ffi_provider.into(); ``` +(Older revisions of `datafusion-ffi` produced a `ForeignTableProvider` wrapper as an +intermediate; that step is no longer needed.) + If you review the code in [datafusion-ffi] you will find that each of the traits we share -across the boundary has two portions, one with a `FFI_` prefix and one with a `Foreign` -prefix. This is used to distinguish which side of the FFI boundary that struct is -designed to be used on. The structures with the `FFI_` prefix are to be used on the -**provider** of the structure. In the example we're showing, this means the code that has -written the underlying `TableProvider` implementation to access your custom data source. -The structures with the `Foreign` prefix are to be used by the receiver. In this case, -it is the `datafusion-python` library. +across the boundary has a struct prefixed with `FFI_`. This is the struct that lives on +the **provider** side of the FFI boundary — the code that has written the underlying +`TableProvider` implementation to access your custom data source. The receiver +(`datafusion-python`, in our case) consumes the `FFI_` struct through the FFI trait +implementations supplied by `datafusion-ffi`. In order to share these FFI structures, we need to wrap them in some kind of Python object that can be used to interface from one package to another. As described in the above @@ -204,19 +217,20 @@ section on our inspiration from Arrow, we use `PyCapsule`. We can create a `PyCa for our provider thusly: ```rust -let name = CString::new("datafusion_table_provider")?; -let my_capsule = PyCapsule::new_bound(py, provider, Some(name))?; +let name = cr"datafusion_table_provider".into(); +let my_capsule = PyCapsule::new(py, provider, Some(name))?; ``` -On the receiving side, turn this pycapsule object into the `FFI_TableProvider`, which -can then be turned into a `ForeignTableProvider` the associated code is: +On the receiving side, turn this pycapsule object into the `FFI_TableProvider`, then +convert directly to an `Arc`: ```rust let capsule = capsule.cast::()?; let data: NonNull = capsule - .pointer_checked(Some(name))? + .pointer_checked(Some(c"datafusion_table_provider"))? .cast(); -let codec = unsafe { data.as_ref() }; +let ffi_provider = unsafe { data.as_ref() }; +let provider: Arc = ffi_provider.into(); ``` By convention the `datafusion-python` library expects a Python object that has a diff --git a/docs/source/contributor-guide/introduction.md b/docs/source/contributor-guide/introduction.md index 691a0d2d2..44bae72ed 100644 --- a/docs/source/contributor-guide/introduction.md +++ b/docs/source/contributor-guide/introduction.md @@ -29,7 +29,7 @@ In addition to submitting new PRs, we have a healthy tradition of community memb Doing so is a great way to help the community as well as get more familiar with Rust and the relevant codebases. Before opening a pull request that touches PyO3 bindings, please review the -[PyO3 class mutability guidelines](ffi_pyclass_mutability) so you can flag missing +[PyO3 class mutability guidelines](ffi.md#pyo3-class-mutability-guidelines) so you can flag missing `#[pyclass(frozen)]` annotations during development and review. ## How to develop @@ -64,7 +64,7 @@ Whenever rust code changes (your changes or via `git pull`): ```shell # make sure you activate the venv using "source .venv/bin/activate" first -maturin develop -uv +maturin develop --uv python -m pytest ``` diff --git a/docs/source/user-guide/common-operations/aggregations.ipynb b/docs/source/user-guide/common-operations/aggregations.ipynb index 039284b7f..0e7947b33 100644 --- a/docs/source/user-guide/common-operations/aggregations.ipynb +++ b/docs/source/user-guide/common-operations/aggregations.ipynb @@ -260,7 +260,7 @@ "cell_type": "markdown", "id": "3ed186c9a28b402fb0bc4494df01f08d", "metadata": {}, - "source": "\n### Comparing subsets within a group\n\nSometimes you need to compare the full membership of a group against a\nsubset that meets some condition — for example, \"which groups have at least\none failure, but not every member failed?\". The `filter` argument on an\naggregate restricts the rows that contribute to *that* aggregate without\ndropping the group, so a single pass can produce both the full set and the\nfiltered subset side by side. Pairing\n[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and\n`filter=` is a compact way to express this: collect the distinct values\nof the group, collect the distinct values that satisfy the condition, then\ncompare the two arrays.\n\nSuppose each row records a line item with the supplier that fulfilled it and\na flag for whether that supplier met the commit date. We want to identify\n*partially failed* orders — orders where at least one supplier failed but\nnot every supplier failed:\n\n" + "source": "\n### Comparing subsets within a group\n\nSometimes you need to compare the full membership of a group against a\nsubset that meets some condition \u2014 for example, \"which groups have at least\none failure, but not every member failed?\". The `filter` argument on an\naggregate restricts the rows that contribute to *that* aggregate without\ndropping the group, so a single pass can produce both the full set and the\nfiltered subset side by side. Pairing\n[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and\n`filter=` is a compact way to express this: collect the distinct values\nof the group, collect the distinct values that satisfy the condition, then\ncompare the two arrays.\n\nSuppose each row records a line item with the supplier that fulfilled it and\na flag for whether that supplier met the commit date. We want to identify\n*partially failed* orders \u2014 orders where at least one supplier failed but\nnot every supplier failed:\n\n" }, { "cell_type": "code", @@ -299,7 +299,7 @@ "cell_type": "markdown", "id": "379cbbc1e968416e875cc15c1202d7eb", "metadata": {}, - "source": "\nOrder 1 is partial (one of three suppliers failed). Order 2 is excluded\nbecause no supplier failed, order 3 because its only supplier failed, and\norder 4 because both of its suppliers failed.\n\n## Grouping Sets\n\nThe default style of aggregation produces one row per group. Sometimes you want a single query to\nproduce rows at multiple levels of detail — for example, totals per type *and* an overall grand\ntotal, or subtotals for every combination of two columns plus the individual column totals. Writing\nseparate queries and concatenating them is tedious and runs the data multiple times. Grouping sets\nsolve this by letting you specify several grouping levels in one pass.\n\nDataFusion supports three grouping set styles through the\n[`GroupingSet`][datafusion.expr.GroupingSet] class:\n\n- [`rollup`][datafusion.expr.GroupingSet.rollup] — hierarchical subtotals, like a drill-down report\n- [`cube`][datafusion.expr.GroupingSet.cube] — every possible subtotal combination, like a pivot table\n- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] — explicitly list exactly which grouping levels you want\n\nBecause result rows come from different grouping levels, a column that is *not* part of a\nparticular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to\ndistinguish a real `null` in the data from one that means \"this column was aggregated across.\"\nIt returns `0` when the column is a grouping key for that row, and `1` when it is not.\n\n### Rollup\n\n[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces\ngrouping sets `(a, b)`, `(a)`, and `()` — like nested subtotals in a report. This is useful\nwhen your columns have a natural hierarchy, such as region → city or type → subtype.\n\nSuppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With\nthe default aggregation style we would need two separate queries. With `rollup` we get it all at\nonce:\n\n" + "source": "\nOrder 1 is partial (one of three suppliers failed). Order 2 is excluded\nbecause no supplier failed, order 3 because its only supplier failed, and\norder 4 because both of its suppliers failed.\n\n## Grouping Sets\n\nThe default style of aggregation produces one row per group. Sometimes you want a single query to\nproduce rows at multiple levels of detail \u2014 for example, totals per type *and* an overall grand\ntotal, or subtotals for every combination of two columns plus the individual column totals. Writing\nseparate queries and concatenating them is tedious and runs the data multiple times. Grouping sets\nsolve this by letting you specify several grouping levels in one pass.\n\nDataFusion supports three grouping set styles through the\n[`GroupingSet`][datafusion.expr.GroupingSet] class:\n\n- [`rollup`][datafusion.expr.GroupingSet.rollup] \u2014 hierarchical subtotals, like a drill-down report\n- [`cube`][datafusion.expr.GroupingSet.cube] \u2014 every possible subtotal combination, like a pivot table\n- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] \u2014 explicitly list exactly which grouping levels you want\n\nBecause result rows come from different grouping levels, a column that is *not* part of a\nparticular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to\ndistinguish a real `null` in the data from one that means \"this column was aggregated across.\"\nIt returns `0` when the column is a grouping key for that row, and `1` when it is not.\n\n### Rollup\n\n[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces\ngrouping sets `(a, b)`, `(a)`, and `()` \u2014 like nested subtotals in a report. This is useful\nwhen your columns have a natural hierarchy, such as region \u2192 city or type \u2192 subtype.\n\nSuppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With\nthe default aggregation style we would need two separate queries. With `rollup` we get it all at\nonce:\n\n" }, { "cell_type": "code", @@ -324,7 +324,7 @@ "cell_type": "markdown", "id": "db7b79bc585a40fcaf58bf750017e135", "metadata": {}, - "source": "\nThe first row — where `Type 1` is `null` — is the grand total across all types. But how do you\ntell a grand-total `null` apart from a Pokemon that genuinely has no type? The\n[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key\nfor that row and `1` when it is aggregated across.\n\n
\n

Note

\n\nDue to an upstream DataFusion limitation\n([apache/datafusion#21411](https://github.com/apache/datafusion/issues/21411)),\n`.alias()` cannot be applied directly to a `grouping()` expression — it will raise an\nerror at execution time. Instead, use\n[`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] on the result DataFrame to\ngive the column a readable name. Once the upstream issue is resolved, you will be able to\nuse `.alias()` directly and the workaround below will no longer be necessary.\n\n
\n\nThe raw column name generated by `grouping()` contains internal identifiers, so we use\n[`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] to clean it up:\n\n" + "source": "\nThe first row \u2014 where `Type 1` is `null` \u2014 is the grand total across all types. But how do you\ntell a grand-total `null` apart from a Pokemon that genuinely has no type? The\n[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key\nfor that row and `1` when it is aggregated across.\n\nUse `.alias()` to give the column a readable name:\n\n" }, { "cell_type": "code", @@ -332,26 +332,13 @@ "id": "916684f9a58a4a2aa5f864670399430d", "metadata": {}, "outputs": [], - "source": [ - "result = df.aggregate(\n", - " [GroupingSet.rollup(col_type_1)],\n", - " [\n", - " f.count(col_speed).alias(\"Count\"),\n", - " f.avg(col_speed).alias(\"Avg Speed\"),\n", - " f.grouping(col_type_1),\n", - " ],\n", - ")\n", - "for field in result.schema():\n", - " if field.name.startswith(\"grouping(\"):\n", - " result = result.with_column_renamed(field.name, \"Is Total\")\n", - "result.sort(col_type_1.sort(ascending=True, nulls_first=True))" - ] + "source": "df.aggregate(\n [GroupingSet.rollup(col_type_1)],\n [\n f.count(col_speed).alias(\"Count\"),\n f.avg(col_speed).alias(\"Avg Speed\"),\n f.grouping(col_type_1).alias(\"Is Total\"),\n ],\n).sort(col_type_1.sort(ascending=True, nulls_first=True))" }, { "cell_type": "markdown", "id": "1671c31a24314836a5b85d7ef7fbf015", "metadata": {}, - "source": "\nWith two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces:\n\n- one row per `(Type 1, Type 2)` pair — the most detailed level\n- one row per `Type 1` — subtotals\n- one grand total row\n\n" + "source": "\nWith two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces:\n\n- one row per `(Type 1, Type 2)` pair \u2014 the most detailed level\n- one row per `Type 1` \u2014 subtotals\n- one grand total row\n\n" }, { "cell_type": "code", @@ -373,7 +360,7 @@ "cell_type": "markdown", "id": "f6fa52606d8c4a75a9b52967216f8f3f", "metadata": {}, - "source": "\n### Cube\n\n[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)`\nproduces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` — one more than `rollup` because\nit also includes `(b)` alone. This is useful when neither column is \"above\" the other in a\nhierarchy and you want all cross-tabulations.\n\nFor our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair,\nby `Type 1` alone, by `Type 2` alone, and a grand total — all in one query:\n\n" + "source": "\n### Cube\n\n[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)`\nproduces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` \u2014 one more than `rollup` because\nit also includes `(b)` alone. This is useful when neither column is \"above\" the other in a\nhierarchy and you want all cross-tabulations.\n\nFor our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair,\nby `Type 1` alone, by `Type 2` alone, and a grand total \u2014 all in one query:\n\n" }, { "cell_type": "code", @@ -395,7 +382,7 @@ "cell_type": "markdown", "id": "cdf66aed5cc84ca1b48e60bad68798a8", "metadata": {}, - "source": "\nCompared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but\n`Type 2` has a value — those are the per-`Type 2` subtotals that `rollup` does not include.\n\n### Explicit Grouping Sets\n\n[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels\nyou need when `rollup` or `cube` would produce too many or too few. Each argument is a list of\ncolumns forming one grouping set.\n\nFor example, if we want only the per-`Type 1` totals and per-`Type 2` totals — but *not* the\nfull `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that:\n\n" + "source": "\nCompared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but\n`Type 2` has a value \u2014 those are the per-`Type 2` subtotals that `rollup` does not include.\n\n### Explicit Grouping Sets\n\n[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels\nyou need when `rollup` or `cube` would produce too many or too few. Each argument is a list of\ncolumns forming one grouping set.\n\nFor example, if we want only the per-`Type 1` totals and per-`Type 2` totals \u2014 but *not* the\nfull `(Type 1, Type 2)` detail rows or the grand total \u2014 we can ask for exactly that:\n\n" }, { "cell_type": "code", @@ -425,31 +412,13 @@ "id": "0e382214b5f147d187d36a2058b9c724", "metadata": {}, "outputs": [], - "source": [ - "result = df.aggregate(\n", - " [GroupingSet.grouping_sets([col_type_1], [col_type_2])],\n", - " [\n", - " f.count(col_speed).alias(\"Count\"),\n", - " f.avg(col_speed).alias(\"Avg Speed\"),\n", - " f.grouping(col_type_1),\n", - " f.grouping(col_type_2),\n", - " ],\n", - ")\n", - "for field in result.schema():\n", - " if field.name.startswith(\"grouping(\"):\n", - " clean = field.name.split(\".\")[-1].rstrip(\")\")\n", - " result = result.with_column_renamed(field.name, f\"grouping({clean})\")\n", - "result.sort(\n", - " col_type_1.sort(ascending=True, nulls_first=True),\n", - " col_type_2.sort(ascending=True, nulls_first=True),\n", - ")" - ] + "source": "df.aggregate(\n [GroupingSet.grouping_sets([col_type_1], [col_type_2])],\n [\n f.count(col_speed).alias(\"Count\"),\n f.avg(col_speed).alias(\"Avg Speed\"),\n f.grouping(col_type_1).alias(\"grouping(Type 1)\"),\n f.grouping(col_type_2).alias(\"grouping(Type 2)\"),\n ],\n).sort(\n col_type_1.sort(ascending=True, nulls_first=True),\n col_type_2.sort(ascending=True, nulls_first=True),\n)" }, { "cell_type": "markdown", "id": "5b09d5ef5b5e4bb6ab9b829b10b6a29f", "metadata": {}, - "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n
\n

Note

\n\nSerialization\n\n
\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions —\n the accumulator class is captured by value via [`cloudpickle`][cloudpickle],\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" + "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n
\n

Note

\n\nSerialization\n\n
\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the accumulator class is captured by value via [`cloudpickle`][cloudpickle],\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" } ], "metadata": { diff --git a/docs/source/user-guide/common-operations/index.md b/docs/source/user-guide/common-operations/index.md index 1f3478b72..384bce3e0 100644 --- a/docs/source/user-guide/common-operations/index.md +++ b/docs/source/user-guide/common-operations/index.md @@ -36,5 +36,6 @@ The contents of this section are designed to guide a new user through how to use - [Aggregations](aggregations.ipynb) — `group_by`, rollup, cube, grouping sets. - [Windows](windows.ipynb) — partitioned and ranking window functions. -- [UDFs and UDAFs](udf-and-udfa.ipynb) — scalar, aggregate, window, and - table user-defined functions. +- [User-Defined Functions](udf-and-udfa.ipynb) — scalar (UDF), + aggregate (UDAF), window (UDWF), and table (UDTF) user-defined + functions. diff --git a/docs/source/user-guide/distributing-work.md b/docs/source/user-guide/distributing-work.md index d9f03d311..ea9d56e56 100644 --- a/docs/source/user-guide/distributing-work.md +++ b/docs/source/user-guide/distributing-work.md @@ -27,8 +27,8 @@ workloads where the driver decides partitioning up front. Query-level distribution — where the runtime partitions a single logical or physical plan across worker nodes — is in progress -upstream via [datafusion-distributed](https://github.com/apache/datafusion-distributed) and [Apache -Ballista](https://github.com/apache/datafusion-ballista). Both +upstream via [datafusion-distributed](https://github.com/datafusion-contrib/datafusion-distributed) and [Apache +Ballista](https://datafusion.apache.org/ballista/). Both have short sections at the end of this page; integration details will land as those projects become usable from datafusion-python. @@ -259,7 +259,7 @@ up to four *slots* in a running program: | Slot | Lifetime | Purpose | Set how | |------|----------|---------|---------| | User-held | Local variable / attribute | Build and run queries | `ctx = SessionContext(...)` | -| Global | Process singleton (lazy-init) | Backs module-level [`read_parquet`][datafusion.io.read_parquet], [`read_csv`][datafusion.io.read_csv], [`read_json`][datafusion.io.read_json], [`read_avro`][datafusion.io.read_avro]; final fallback for [`Expr.from_bytes`][datafusion.expr.Expr.from_bytes] | Implicit; access via [`SessionContext.global_ctx`][datafusion.context.SessionContext.global_ctx] | +| Global | Process singleton (lazy-init) | Backs module-level [`read_parquet`][datafusion.io.read_parquet], [`read_csv`][datafusion.io.read_csv], [`read_json`][datafusion.io.read_json], [`read_avro`][datafusion.io.read_avro]; final fallback for [`Expr.from_bytes`][datafusion.expr.Expr.from_bytes] | Implicit; access via [`global_ctx`][datafusion.context.SessionContext.global_ctx] | | Sender | Thread-local on the driver | Codec settings for outbound `pickle.dumps` / [`Expr.to_bytes`][datafusion.expr.Expr.to_bytes] without `ctx` | [`set_sender_ctx`][datafusion.ipc.set_sender_ctx] | | Worker | Thread-local on the worker | Function registry for inbound `pickle.loads` / [`Expr.from_bytes`][datafusion.expr.Expr.from_bytes] without `ctx` | [`set_worker_ctx`][datafusion.ipc.set_worker_ctx] | @@ -297,7 +297,7 @@ Sharp edges: 🚧 *Work in progress upstream — not yet usable from datafusion-python.* -[datafusion-distributed](https://github.com/apache/datafusion-distributed) +[datafusion-distributed](https://github.com/datafusion-contrib/datafusion-distributed) splits a single physical plan into stages and runs each stage on a different worker node. The driver writes a SQL or DataFrame query once; the runtime handles partitioning, shuffles, and reassembly. @@ -311,7 +311,7 @@ require automatic plan partitioning. 🚧 *Work in progress upstream — not yet usable from datafusion-python.* -[Apache Ballista](https://github.com/apache/datafusion-ballista) +[Apache Ballista](https://datafusion.apache.org/ballista/) provides distributed query execution on top of DataFusion with a scheduler / executor model better suited to long-lived cluster deployments. A datafusion-python integration is on the roadmap; this @@ -320,7 +320,6 @@ section will fill in once the integration is usable. ## See also - [`ipc`][datafusion.ipc] — worker context API. -- `examples/multiprocessing_pickle_expr.py` — runnable - `multiprocessing.Pool` example that ships a different parametric - expression to each worker and collects results back. -- `examples/ray_pickle_expr.py` — runnable Ray actor example. +- [`examples/`](https://github.com/apache/datafusion-python/tree/main/examples) — + runnable scripts for `multiprocessing.Pool` and Ray actor patterns, + plus other end-to-end demos. diff --git a/docs/source/user-guide/sql.ipynb b/docs/source/user-guide/sql.ipynb index ce87e4335..25c673b78 100644 --- a/docs/source/user-guide/sql.ipynb +++ b/docs/source/user-guide/sql.ipynb @@ -118,7 +118,7 @@ "cell_type": "markdown", "id": "8763a12b2bbd4a93a75aff182afb95dc", "metadata": {}, - "source": "\nThe approach implemented for conversion of variables into a SQL query\nrelies on string conversion. This has the potential for data loss,\nspecifically for cases like floating point numbers. If you need to pass\nvariables into a parameterized query and it is important to maintain the\noriginal value without conversion to a string, then you can use the\noptional parameter `param_values` to specify these. This parameter\nexpects a dictionary mapping from the parameter name to a Python\nobject. Those objects will be cast into a\n[PyArrow Scalar Value](https://arrow.apache.org/docs/python/generated/pyarrow.Scalar.html).\n\nUsing `param_values` will rely on the SQL dialect you have configured\nfor your session. This can be set using the [configuration options](configuration)\nof your [`SessionContext`][datafusion.context.SessionContext]. Similar to how\n[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html)\nwork, these parameters are limited to places where you would pass in a\nscalar value, such as a comparison.\n\n" + "source": "\nThe approach implemented for conversion of variables into a SQL query\nrelies on string conversion. This has the potential for data loss,\nspecifically for cases like floating point numbers. If you need to pass\nvariables into a parameterized query and it is important to maintain the\noriginal value without conversion to a string, then you can use the\noptional parameter `param_values` to specify these. This parameter\nexpects a dictionary mapping from the parameter name to a Python\nobject. Those objects will be cast into a\n[PyArrow Scalar Value](https://arrow.apache.org/docs/python/generated/pyarrow.Scalar.html).\n\nUsing `param_values` will rely on the SQL dialect you have configured\nfor your session. This can be set using the [configuration options](../configuration/)\nof your [`SessionContext`][datafusion.context.SessionContext]. Similar to how\n[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html)\nwork, these parameters are limited to places where you would pass in a\nscalar value, such as a comparison.\n\n" }, { "cell_type": "code", diff --git a/docs/source/user-guide/upgrade-guides.md b/docs/source/user-guide/upgrade-guides.md index 09e5f3ba4..c8ea98493 100644 --- a/docs/source/user-guide/upgrade-guides.md +++ b/docs/source/user-guide/upgrade-guides.md @@ -21,6 +21,8 @@ ## DataFusion 54.0.0 +### `Config` removed in favor of `SessionConfig` + The `Config` class has been removed. It was a standalone wrapper around `ConfigOptions` that could not be connected to a `SessionContext`, making it effectively unusable. Use [`SessionConfig`][datafusion.context.SessionConfig] instead, @@ -45,6 +47,8 @@ config = SessionConfig().set("datafusion.execution.batch_size", "4096") ctx = SessionContext(config) ``` +### `distinct` argument added to `sum` and `avg` + The aggregate functions [`sum`][datafusion.functions.sum] and [`avg`][datafusion.functions.avg] now accept a `distinct` argument, matching the other aggregate functions. `distinct` is inserted *before* `filter` in the diff --git a/mkdocs.yml b/mkdocs.yml index 7ac398923..2b4d0d490 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -137,7 +137,7 @@ nav: - Functions: user-guide/common-operations/functions.ipynb - Aggregations: user-guide/common-operations/aggregations.ipynb - Windows: user-guide/common-operations/windows.ipynb - - UDF and UDAF: user-guide/common-operations/udf-and-udfa.ipynb + - User-Defined Functions: user-guide/common-operations/udf-and-udfa.ipynb - I/O: - user-guide/io/index.md - Arrow: user-guide/io/arrow.ipynb diff --git a/python/datafusion/functions.py b/python/datafusion/functions.py index 7891af255..1d372098f 100644 --- a/python/datafusion/functions.py +++ b/python/datafusion/functions.py @@ -4737,16 +4737,6 @@ def grouping( default aggregation without grouping sets every column is always part of the key, so ``grouping()`` always returns 0. - .. warning:: - - Due to an upstream DataFusion limitation - (`#21411 `_), - ``.alias()`` cannot be applied directly to a ``grouping()`` - expression. Doing so will raise an error at execution time. To - rename the column, use - [`with_column_renamed`][datafusion.dataframe.DataFrame.with_column_renamed] - on the result DataFrame instead. - Args: expression: The column to check grouping status for distinct: If True, compute on distinct values only From 168ac0019072160d1933145e876dfdf870ea2191 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Mon, 8 Jun 2026 08:32:00 +0200 Subject: [PATCH 09/18] docs: switch executable code blocks from mkdocs-jupyter to markdown-exec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Notebook (.ipynb) files were JSON-encoded and hostile to manual edits and diffs. Convert every notebook page to plain Markdown that uses markdown-exec fences, so authors edit straight `.md` files and reviewers see real diffs. Each former code cell becomes one of: ```python exec="1" source="material-block" result="text" session="" ...code... ``` Per-page sessions (`session=""`) keep kernel state across blocks within a single page, mirroring how Jupyter cells shared globals. Setup blocks (previously the `nb-setup`-tagged invisible cell) become ```python exec="1" session="" ...imports + chdir + configure_formatter(...)... ``` with no `source=` attribute so the source is not rendered. They still execute, populating the kernel state for the rest of the page. The chdir logic now probes both `docs/source` (mkdocs builds from the repo root) and `..` (covers `mkdocs serve` runs from the docs directory). Other changes * `pyproject.toml` [dependency-groups] docs: swap `mkdocs-jupyter` and the `ipython` / `pickleshare` ipykernel deps for `markdown-exec[ansi]`. * `mkdocs.yml`: drop the `mkdocs-jupyter` plugin block (with its `remove_tag_config`) in favor of a bare `- markdown-exec`. Update every `.ipynb` nav entry to `.md`. Remove the now-obsolete `hooks: docs/hooks.py` line — `mkdocs-autorefs` resolves `[X][datafusion.Y.Z]` cross-references directly in `.md` markdown, so the custom HTML-pass hook the notebooks needed is no longer required. * `docs/hooks.py`: deleted. * `docs/source/images/jupyter_lab_df_view.png`: restored. The previous attempt to replace the static screenshot with a live `display(df)` cell relied on Jupyter's rich `_repr_html_` rendering, which markdown-exec doesn't drive — and the Sphinx-era docs always used text `__repr__` output for the rest of the page anyway, so a single static screenshot for the `display(df)` demonstration is the right fit. `introduction.md` references it again. * Fix the bulk extension rewrite's collateral: every cross-page link that used to point at a `.ipynb` URL (in user-guide index pages, the FFI guide, and others) now points at the corresponding `.md` source. Build is green: 0 execution errors across all converted pages, ~4 s wall-clock, every DataFrame `df.show()` materializes as text output in the rendered HTML. Co-Authored-By: Claude Opus 4.7 --- docs/source/contributor-guide/ffi.md | 2 +- docs/source/images/jupyter_lab_df_view.png | Bin 0 -> 267052 bytes docs/source/index.ipynb | 78 -- docs/source/index.md | 73 ++ .../common-operations/aggregations.ipynb | 436 -------- .../common-operations/aggregations.md | 497 +++++++++ .../common-operations/basic-info.ipynb | 143 --- .../common-operations/basic-info.md | 91 ++ .../common-operations/expressions.ipynb | 387 ------- .../common-operations/expressions.md | 365 +++++++ .../common-operations/functions.ipynb | 242 ----- .../user-guide/common-operations/functions.md | 173 ++++ .../user-guide/common-operations/index.md | 16 +- .../user-guide/common-operations/joins.ipynb | 242 ----- .../user-guide/common-operations/joins.md | 196 ++++ .../common-operations/select-and-filter.ipynb | 120 --- .../common-operations/select-and-filter.md | 90 ++ .../common-operations/udf-and-udfa.ipynb | 210 ---- .../common-operations/udf-and-udfa.md | 485 +++++++++ .../common-operations/windows.ipynb | 230 ----- .../user-guide/common-operations/windows.md | 252 +++++ docs/source/user-guide/concepts.ipynb | 86 -- docs/source/user-guide/concepts.md | 121 +++ docs/source/user-guide/data-sources.ipynb | 119 --- docs/source/user-guide/data-sources.md | 305 ++++++ docs/source/user-guide/dataframe/index.md | 2 +- docs/source/user-guide/index.md | 8 +- docs/source/user-guide/introduction.ipynb | 104 -- docs/source/user-guide/introduction.md | 98 ++ docs/source/user-guide/io/arrow.ipynb | 92 -- docs/source/user-guide/io/arrow.md | 95 ++ docs/source/user-guide/io/index.md | 4 +- docs/source/user-guide/sql.ipynb | 153 --- docs/source/user-guide/sql.md | 158 +++ mkdocs.yml | 40 +- pyproject.toml | 4 +- uv.lock | 963 +----------------- 37 files changed, 3055 insertions(+), 3625 deletions(-) create mode 100644 docs/source/images/jupyter_lab_df_view.png delete mode 100644 docs/source/index.ipynb create mode 100644 docs/source/index.md delete mode 100644 docs/source/user-guide/common-operations/aggregations.ipynb create mode 100644 docs/source/user-guide/common-operations/aggregations.md delete mode 100644 docs/source/user-guide/common-operations/basic-info.ipynb create mode 100644 docs/source/user-guide/common-operations/basic-info.md delete mode 100644 docs/source/user-guide/common-operations/expressions.ipynb create mode 100644 docs/source/user-guide/common-operations/expressions.md delete mode 100644 docs/source/user-guide/common-operations/functions.ipynb create mode 100644 docs/source/user-guide/common-operations/functions.md delete mode 100644 docs/source/user-guide/common-operations/joins.ipynb create mode 100644 docs/source/user-guide/common-operations/joins.md delete mode 100644 docs/source/user-guide/common-operations/select-and-filter.ipynb create mode 100644 docs/source/user-guide/common-operations/select-and-filter.md delete mode 100644 docs/source/user-guide/common-operations/udf-and-udfa.ipynb create mode 100644 docs/source/user-guide/common-operations/udf-and-udfa.md delete mode 100644 docs/source/user-guide/common-operations/windows.ipynb create mode 100644 docs/source/user-guide/common-operations/windows.md delete mode 100644 docs/source/user-guide/concepts.ipynb create mode 100644 docs/source/user-guide/concepts.md delete mode 100644 docs/source/user-guide/data-sources.ipynb create mode 100644 docs/source/user-guide/data-sources.md delete mode 100644 docs/source/user-guide/introduction.ipynb create mode 100644 docs/source/user-guide/introduction.md delete mode 100644 docs/source/user-guide/io/arrow.ipynb create mode 100644 docs/source/user-guide/io/arrow.md delete mode 100644 docs/source/user-guide/sql.ipynb create mode 100644 docs/source/user-guide/sql.md diff --git a/docs/source/contributor-guide/ffi.md b/docs/source/contributor-guide/ffi.md index a269c5e80..cccb6a957 100644 --- a/docs/source/contributor-guide/ffi.md +++ b/docs/source/contributor-guide/ffi.md @@ -79,7 +79,7 @@ of code. Also, the DataFusion Python project uses the existing definitions from [Apache Arrow CStream Interface](https://arrow.apache.org/docs/format/CStreamInterface.html) to support importing **and** exporting tables. Any Python package that supports reading the Arrow C Stream interface can work with DataFusion Python out of the box! You can read -more about working with Arrow sources in the [Data Sources](../user-guide/data-sources.ipynb) +more about working with Arrow sources in the [Data Sources](../user-guide/data-sources.md) page. To learn more about the Foreign Function Interface in Rust, the diff --git a/docs/source/images/jupyter_lab_df_view.png b/docs/source/images/jupyter_lab_df_view.png new file mode 100644 index 0000000000000000000000000000000000000000..31245ce15e29f06cb76f161d2cd4e676cf1a1773 GIT binary patch literal 267052 zcmeFa^_N}OnI>E%m5P~}nVD}fGc#A+0u?h?rDG>{Y}r=gfE`oYaS}ruQ;c!qwQLIw zl4X=-Nvk{5i8I|Z-~0pL^PG3rxg{l?7_e@c=D1IhZY#LjgWNX(kG|JJ00@ zG7oUp&L|`KDRH^E+*qo#b#CL_y^Th(-XTg)y2$)Mvnr}np!|WvcW&cKahoI61!iSF z^931$@oztOWY)&s8QnKbeM19-nV=aEU{2F4iEgiU&aP;XvB&|uKnBZpAg#?-E#LXI zbO`5KB{N57A*c$bXqYijecG&Y8XDej-66wvIZaTaq|pGu%YoWCu$7Cq(WHmbb@{!M4k3gl6Yy+N${M55P-7E%XmyW4rzFNyszb%sC3-oG0DRekK0#=f8yp zH^3QyAx^p=y6~Nbd?;JaN$wDgmzdei+DORQU9eS@&}srw3^9rFiLdVv>~vHBwZ9;y~}*2?h**c*h2WF_^|E4jwY& zic>?3MfT_z;8lQ)WnZwlPm!6wJ_03t6U=%5tcVHf7HSUe=j+7~WtecL{eTgVPAe#h zpahV>vM~e-tr*O~T2L8d#1-TQvIYTmaZaZnV36E|b+mc2bwpTz3^fHohFiLrE*?Th z0)tk^hJ~&|Nn4cGCUYk$KLXq>&~45+*LppY-i#Xw*8XyJ5s)#CAODD~#;*mSp zY{+OZGz_}$%4i8ho-bIGw0SlkQSCh;L#@6284`;(Sloj~1Ik5gxrITQ3kFNL{@Ug~ zN0*m|xIti@HVi^g zWsp3ieIyt_XXJ_sKE&hN)fD-3S*qk3VkFwGfpH7min8t0f@7a0t154?>TLAdVB`ZV zWj@8ut|g^V3P!-XLcst#kXEIG!7kMIpW84m{!mjof9_MH`*i6Y<2z}iO3SCn3LkDYr0Dlp& z5N(NI`J7nCanU*gAY)h8{2j zvIWsLz;QY-Ms(U=paK{Tt52SluH#KSGs(9PtlQVaw=#Wh3MwlXCnu#ZXLzxF=?$>U z&y-(!0g}rB><7MhYL#Jq6K1#{M!$gN_}Y)_KYm%7lM`Hs1kj_0v4}=BDCXlQ!E-<# zJqiZU?n^7nE7A(QsBTm9Fyu%8&r2)y2Y?OCgXv(jwW%IH`5Qs@Am%e|c^FRi3r88u zDuXTkc!Mv9NfQV%fLO28j-ahOG?((0E)dY`(96knl!&Gb#(6m%$!~&5lv==~9WCLc z87^S#%TWg$N?G!r_IdwAK`Y~>iQx`-f|%~WILSui+fcc~*7xrR7rgg?$oM)&M%R1K z=_Av$_gK$34GiSpV6j;A(3tItAb^ZBub+KgRB#$bz}1=Oz-lj80w@!-2ATy0a4KFj z)j7Nl!A(+nfw=5^P%+JtQ^L^XwFy@3Ol}@orAb(2KN_r-BcQD_KL|^xfCBUthyoM{G8j#pk*E+jXy3~B@Q1(o}GjAT&pViQ>-(-fB@LTnpaxVhHL)O{E2NI^Cqa9ljeCmh8|CK^+#Zpb`|q}kie*jzsi-F z^|1+E6NNEgI>O;Wqcmtfy25_J7Tm($al_O<29>=>N{`eYr6UEdBCHJdnRZa=1RIdu zB3pq=vEa_WJZwJ5^UUqwKpPDXu#*e2v}^VqJt z4Dq!9*m9l?FhoF^v`>koTlwr|o~V_e(Lq49lG38BlSBm{0tMGk*0eQpVU@E5G09Wq zEoguK?NO6S>7e+!xhigxkEoeOupqDW8>$x@wV>+{4jCX8K&QTC>9!oH+|TgL0}aK7 zWW=m?Li3ta=eqI;T1>+_5U@GDVcI8CPffus1Kl9*K!?4q`Ecny(C_a573&Rm0kn5a ze)E|rLiMI zB_tYLX}-8zx`nqoc^>zKYOCU^zL&|~-OQcN%O%3=)9NPPdkFeu!WdTa$ zAttXrX4*upvB8Kw;~@9zIS3~B>?KIY`0Bw+uS40DPVGvl#w=;$MY&|T{KGpw_}*y8 zr_V6kV*9~ZfdpKWJ?eVBA4(Tf0O0fB)a%Bte$Auz?SnDWN4P@{33x3@1)~b&$=1t1 zF~!LIJQKC0)7Z{v(xKvsrTH(hbyL^iIy%-?^xi&b&Hd6;VMYs9^nCQ9Nfn(14j?qM<#KFS#fJ=bv0z`=#tSm zw>&w&QfI!=g$|pAs_tboGtUT;d36i(9IJs(wRNp@)z{^Y&d(+ekIj!56E|iTIG0)e zMlRG@+y<5~cKwBMuRT7T8$hf_d}S!%i={vnppFnU zZ6mIZFk01RNS4%SKDysStcs4gqA z`1=^b^{g=ArJvqsa4}Z?1GnJn;NVj50Sgyr^S=jN$LX~jx(YmvHUGd0MYtm#bTl;h z$lt|iH7^(J#Gn5zPXqi@g?;yy)!(;ek;UonU}gJm21pku=Ib5yq z#K8j4qg(&3HE}^IV>saNd)C~=yH+m7^`m@P{B0AR{uVAN1PS;w1ja#ApqrmSjOWGk z4}*suQBOttZl$0T&{sQsmvkKFe06A^Am_uocMi;yNPGwn^1=4!?>)Q2?CmHUYO{dU zF=5)LdYJPy_Ua2Bo$AG}m)o~cPtZZ_wR}bGPNipm=2t&^k*8Bn&;+Poz5vE2jZd`s zpI-SXl)YTO%&pt3$ z8hWW`lowfyFK`WBy8!|kIZL@ZM<#UA%`bLbA{X>m5zo1&Rl$`jwp4_S(ZRdxyj`H) zEjtExrTxJ}Gf~ij)#S_EYNZT$_Ohq;fIX*abz72YQS|c?Q_eEKfVd6QbF6TN zF~a`>)pM=7WH22()?s}9#^Dy{&5@3dUd}S}#V@8V-+6;K7Q59_Rv703Dy*U9{NcfS znQefpkH}pS>&#wWctb4ADqE=Y1zS@+* zXGR9MW%M=&7xe2I6jd9b8H!Y1=)+5VJpmfkK5C$=WKZIxwK7b5lzaMU-w|immUIh` zb$|O8^^DBUu{`i)K8b*yX_4SoDaF+VkonSc;rU)DJP}~p9lbkSe&rjSoNI5>_Pd~& zEHV(Vr;H2Lr$xGiUwrm(>)l8Ava={%2jL&EPR$k63fiiqLyiW4sT#vhP#Cu~V&M$V zve{z~)&c4Q0o=D|FSn!oO`FapSd~Et+99^uc}Adbxn7hO8jgkp=v*q~M>3NcA8z^pF9|p6^@a#z+2IxT!)@S#eO*iECJ$rQPg-`2m=gNDnI>30 z_Yn0M?w$dZyRK^a2pYhyh)NPAq;k1I`j{ulyM90T#1fcb!YW`zv_HT4xG1I&DwK{f+GXLb zTX)tOY=RH|b&O}_`|k)Njd2Zx4K{d|dfI9wi$(w#qyicqSnJSxCfZoZSS;+j+ZpDC zj>^cDL1=Z*^DQ8t`0_2_r(d-Lw_Wj6Atz(tX`pi_^`0jVWB(RzR#%NJTweO}$(m}-_LAHqbF-TLaS+zH#9o{%I z`I{&13g*6VLM1o83#uKIk%9sJ3U6qPyJqg4t`}VXxvr|@(c9dgJuLvJdKG>2+DnV9 z5;P~Ue*IQaqD9b(e-%+&wMIQZ%&q+8BEz4)_ayg}3gQgjHrltqBcbx7^9&iko@#K@ zIjWM;>|16i_k#o-*2;DR+p8?70GD7u@^nlY`I+7ZH|Kz=B`9-xQOmGV! zc2V?wCnsb`89;y&k0L0|f%GZ`Gfq};A#0E_O=aLF8tS!!2h^cLW#Y`e%z9w4B{;1` zRBL4!v?f5i+=rWY|EEvOa8~97bIE4t^B+QlgE~QsHUW#p`A0?H<3{{MFL6yu5G{;0 z-2*YZVnOOOP9D%LKrDg>j{>(Uo_!4DXb;xv?gP$jAKy=lHLZ${T;>)}&)^W0&gv=@ zW%}qon~-K6Ds$}Lp1%`v++D@uE*e~fCSN=b2B;-?PQZ%+%3J|*tdo0qr65}SaUY%v z@Q$E~7%&szrJf;1(~i)U2J#AX^z51B+*2SONof^4nuRgd5`rH4-9t~xC}Z=}e+6so zDD_n79Eawh4ATeRQ(xZAb%FP86A8X5+Sa%`lyeX0{XD3-w5)d^r`1dY1EXkV$Ow4S z&M*yY0J$q;a4$>_E_xWeCT~~o;rYuzOXI1`{FYrBaUFE#RFk>zyHy#QEl;1xpX}j06u+XJi2Wn+{f!!Y6Z8dzSLo`g05m3 zS`=Lej52xaK=f6L_d74p(FLGsR?YNQ+Dj`r1}KXvb)f>|Eyv$EDXI!tG#K8)(z*Hj z`=5d4w3o+RosmxPVVd5RU+3^<-0`_I4N`M33mB)TPM&%&2(pp?mH6yUu#B)ePX>Zu zfR77?2VmoK<`J7Sk22Q)-qrQak1AatUCd5B%>1DdJd=XK%RvmK=~dtN|7F+$nhVPU z!26ogcf1Daf7TITc4ST&3Jt!m3-%dupS-BIw|IatP-HD%aH!n`n%8N=g_Q|d0R_j7ccPvnNuaX5p%R098J=_@Z;R|3%7w)62bT9yP3>QEx z=B9V(6hYwZ^1j}HvLuZVyYVky@nVVqUpyod{Ht44nvXF49VW(j2}7jOC*ZADX5V_E zssfE*`oU*!fyyVR4N)AGuGIv?<_ zWGn#c6wr=$OHV=eXFq|~5HPU|q2hU)x^dMq*8wu577XxaOaWT*8*^umL62c|>>iH^ zkfA(A1CZrq3=-`oo{Khb8EK(abf_@Y+k4P>vYL4sZfLw=WsN(zc<}buh{ig}tlvDB z<%P!!2X!%V1oV_(9D;WUIIkhS*%5*c1deQf@erSC22N=Tr+&0rI@FE4)GvMzT;R!* z@-)B(D3V*w232A>(+qdADl%E}^$%&*L|v~pzUG-yU2 z5T)HC_1a0Oj9WSaV6@Ut00VQx!Zl>3ZpfX<0?f{D_x39NI88`n2`Y5WrAt&^$7Cqt z@-ubnf>S5U(PS+EGW%KCnvy%gJI85->))zAeLv*Y$HA7?Dl*3Sj1h2eLCc?>;61>+ z2ulXzFo^3l7KTc%@XP@`of!5^4aQS1Xf%=@q49yLqgU<`4ZIMY8&)j(DJNh9(NnI zs@#x<2G9p$HRzWn_<97Gseqx8&Id;T$n~hZOEqN9n*A&{sme$m)Y;<+iye?nP2 zXiS0t)9%eWM>zWNFJ-{AMQKkiGjG0lGg|vT5shk)0mgac9CL+dD9UpNcc4k@lflZ4 zG75pZs8a1!bQb#6m)jGyAGKk09|a9_H6|cXpRAIZ69Eww+&gnYMbM=uRTu=w1=&`n zc#)nG#)m2!J>#Feh88{bSCL0=A9xmOkyy-FYWMzzlZM1wb&A1F<27b>ujE|SaSti>#|P?T^J zd(?P6zyRZ2owe*_ZgH*3+9YsbCXX)62GPO3AghW%jxnMIb2*U3AQ!z$6vi_^LrtZz z{HrQOBUR_U^HT_O5KtA+FG_7Iq(k>c_cmltlFwI<-v=ckARvz4*M{n>eF*@{4#*A! zOgMnC-o=9gZZK2hg=f*n7mR^8ux1bt%^fgKn?O7pkYi0N(3+n;%7avbva6I8KK~k8 zT!Ba0W;kBGlNWG~DBsj*s}T-V!~`0Q7w(0oZc|r4w1EOZ00TM^M2~Z((U~a+`&=9P zYKn)-rIqL$mDbPzZjPlF*cSn=LNK0F0Uy7}+qD0wFq(CQ#jYizK{p9n{Y5GV>HSbH ze4BftpBYmCv*a)a-O-A+m>HuDfdG~Z0Z%g%A4a1FErGFeEFA?4;O?XCxlLNz00JV_ z_S2x#Y4LfE;9RyYdf(fSrt-X(!&Uj}L{X8Iivgu9Pk=i2wfmvxvPx!c^)ZS)Kmga- z-`E0=pOPMMF3S73YPx#d3`0)=0bD*mqFox%Ymil-u0~!$@94-h873*mN$<9R_Q^@G zdC;7y+;+Aye&pKSqJVR#zY{EkY5(r`!2kx~y>1bl^FuLBvHoid*Il6Pz1W(GN{fmIFBl@>|xv4J*Cs5qs_VHUr~5F1Lv> zJW`gseC>x&nCA53I4A=I&?rZE4DWDdkCL5ZXwJv7)9KTD%A)zSum(}kNJXHx?vjqM zk57n#K#CoKs;md?%7f>g<0kF2Z(n)(8P~LoR2N+SP7NQQ8z2?E-B4+{iDTMR8k)ex zBQ3f<26|-z7(L3(jDZAKZZnMZwxzoA>=g~CXfVNz`H#Mt%QdzNI~spl8wn`6~oEsKK+A_ z-q0_AqV}|&p55xj_l9U0G(fE0uu*gzd`k!FCdd!OK;#lF?J;!_lTKhR{`pq)0l+G< z%&aX=shqg{%CtHFy}%#~wFh$m@o~aO`Dn|9Uo=1pzPz^*%Cz3T{7iRqWj9OLYgwkPSYafBxYueHBo-$xu-Cp>H&(LK)Z4 zx*20y5f|}%j64)vf! zM{{oM<)D`Vp{ePl7G-h=9Kc&T1{HI=3{yRlU7*C-nCpk?t*?Yn}-(y2Ui~ z+-H6OHCai{J))|R>~WDFmS}dC?lUh#hM)i0m$z*549+x{j^MVjBqy{ky_^KjJTfxb zfJ_mcd~h-3H-Gc^ONG)WJOYAenw?g`D&03+v#q_38d%;0Ho7(et-}{A8Kk;ZSfgVK zbm28Hfc7wmDt!9NBkLnPG*e9n3$oV^gN4Cy7_5yzzKVe^=((|-G$t^*Kc_sCWr^k= ze)KQocYRGp8LA9&?qTUDQw`Vw^@|FA;Wk|rMP-ZL8>fz?{KK#ij2j?n^HLs$OYzw3 zq@(zfprd}6hq(N0FjYUHZKQ$GjyWI|jnD*~h6m#@4GaF@)haij7IyO86d3V&_qm-F zzH;z{&C&_(2U2yZGPfIz(tfb>-#$y9FxrnXUVsn22e!xeHz2RAZvT^~8J~QX0Gt$v z^3MWy0GXnrK!&(=L71*VQDs58?4u43i5>tE392^Cu4Xd@na_hD9dv{CO$x}RcVC`{ zU;*Tf3=&{D=`?c#um`P{UVAbTGU@^&fIj9St~~(;(9C%>gmNZLU$~wf0|9tAxrqYa zL1#`%ld}Lj+}Oem6=$=jgc~GH7VLu4{lE&SNmP}A(zuD|yE7hCB|bdH;9jHWJV9rE zd3cDXo-N!2FTL~PQOytD|$@|9SkJ(aZ^ju(0K+s8ddi2N@fKEUZ zs41b+i|gQqv!%r_NMykX0_{p)LSw;&pNT4*d7=y^9gJ@eka-?huJnP^ zgPfbSN)WDV1LGOAemm&gU0}+CX;34K^kK3A3}Ez)fztRa&Ijq}D&!@MzDc_t-Xi$& z_0?i%sSI%jJ}{AI%vhz%I;0KG1mbKXAU~ES?K?~vP-tLo>#t(4Ub-jOwJ)_5_XnWO zQ(u2Z23l8H$LP!QZobFGt}m%5MT4g+0;PiJr}Zn}2Nz6O`At_sjR(1aXpjqcak6mk zylXB$W~Geb1IRbpM4u{k;A#U7o~BLn@n+QrMqjMa2uxN6jSnbz6^uEt%{=ix|B$M6 zG>L((^xoU=*8ZnLcR7K8D$qQgsoWb2Ro&+dl~wu}2nY^u03S+nWC)e;A_dR(*>*jN=b)rH|>> zj~)X9l+_C9G=|Mz;<(;b!qp2wlz_LTVIQKIv0npXfB>H#d^IA2k+d|!sp~vgU;05a zK6%?akc=c3D&1M--~pgU5HE>Ru7V0=ieS_MUGt;Kfuhn>0DY6dm6(0Z3LqrNC!mq* z(mpY^+Lu1SJl!ahcT%F!lK3<-8`ELG`Xm(yttr*=TMpGH2cSUf6#1=j-nkd zwhNj+|L6|w0nCU<;vC%a?3u^i4brFcKrS-CL3Nth0$Or*kXy?DZGn~|RTzEvO=iwv zGz{?6ejp|hvr@36*|MUb6-^cYW##kwu&xA%w#Y{&ogv_ElUY~*V}Y+I@BKn#rq}gX zq4h5ItjUq+JnzQ2lbJMYYb%owDte$TgJw5BfBuMC>$oym z!S&ZMGC9N)8EN^${r57y(>CV7eaQg7cW;xQ$mH^Z>El6AY?HiM=NzT4afD|H^8{1J z;4XJ%(BZg26!2OPQE7L;MMU8z57+TDe~CRO^f6Ag1#TLzc=vI=hgBgLG$B0<`*rS_ zVUyv)9Sim*nLe%+DGc;PRL?N{gXSW*PGB3P9MK&(jFb?USpZp!zYM%D+ljP}A7$BVmp2m!OvO4L8x^#+tt z8WgyGpI;@!g%e=Ef9Ni7?p#ni$#T}oJ_ZO-(Zy$&Cts%cfSS}kGzAL4KxWwHWHMgI zH@R8^NWfaLT}o%|fh<6^9v|Ea0Yn5Ez}@Kn5^g*1zn*vnB9JMPT?GcLs3UJd4=JD% z&i#^){^%00T_TWgdk?nsoTJS0p63vv+?yt(AA;pG*357XZgzjuhpS~ z+;cp>)7fiz2L1JJpMH~bv!*m2eA(xi+%Ut-FMgdvhhd{PlutDCcxMboxh5vk!a$-Q zSj^xcIPDJ5*vX=@s5n%fQURKmIA!I`GzOwYfdBrUZ-IAx{#`yP1cAE7kXD+_s5ZrCrFn!Jz5Wo%) zgA=d>g~l(i@T<6WgPO4FupN~>4sG2OE@&|-nHMULc{5PBtNj+))Mt0ol|1LAyJ@!6 zEHF>b*9rnIJ_t4q8W;`?x9j%2@}~6HFfx)re%)Y;ARopiu+ditPIDx4oQ0x{Qw#b< zFpmnx^xiL@f|`~Brry7^WhkL>hYntrmhfs3+i$(1_j1z*udb8Qo38!Z`dF-fjjlLA zJN1S%*9-clW9d;Ht)rj6$oc_P{O5PSulGZtrJ5&PY25}MBX)n(08u{L%|~1(NWjcA z@&IRkWSv0M)=B?OJas*lE@UEH`#IPkXg3jz`w5}_n8cPBzL6QllwRhvX`pm-u*T}b zUCT>Pa$wyy0eq{nb!!1qu)$mXYQe&|f#xqBDF~O5JP6~*PcQPGKw4*Qr;~dH*0~io z#wi=1T~VicQyHCJpoQKEaJB^g+Dip@zAYWJ5x_DJ_=sRh6EI#A*^HpeI3vIa0ZK3a z6nv^g1)g+*Pfu@~IY>)Y1B0}V=$A0fM?jeaiQEgoQ`p*zuDv`%t=C?knIWM;iv@v@ zF8gu?c({5#+Swh|`>x(ypP-`6*VWHROA2^O)9SqB?AJ@IPTdVf9hpJGtkFSNU;j}T z_;FAus5xW)&nPoyZp5 z^yHRy6Hj9UVt6#2xet6pT?&;^X?j%Hc%!+@;i{1dZ7MG`G)EZN)kOH2a4}CiGFX4|a zKA3elUwL!KPHzQ3>!IVKK-E0jr~k4G&JCP5-GtWc!Wwug!=w(>16oNN8|D*^P63Pz zD|II6u~Q*|7IT>aMDojJjd9j~4JA%RG<$lqpO=Lqs55rxI+RT=#U0WVj2b}|L_fO) zmCyufRjMo#;m-?`_$!~L8)le?u2fn3nrcZfbJ;xKoqKlh@veX5dE z>vM`%wO^xOPuDNv1Ub@LAMD_o3?RVoU6#Jd9Fwon98L9+QSe4@8dBg3F~XW7h?<$0 z>FieLL)Vl(!U`7)qAwOF_3lV$+IJcPpwz^Hb?aj->h6wrPol5$=zywnoeWJCI_fL) zdp0ZD`Ns8|MWOF_$cwkSaV22cw6=8Ecdzgehrzp)%b=;0;^sg4CfK!mLgIWkz0233 zhk)EcISldie3lVF70fRB8`uO@f-*p?we&#*NbLw3qhJTbSyVI_pjv|ZCwN*x*Qc+F zQbCV%+KNW7e9%_PIgWLL06wAoKyk57;F%yDyn?Q8l^#Ak03I%xOvk*ZsJjrnHrAA% zOrg1hH{|%%#EpSHgZ6WL({hW+!Y;wl=we3AX70!|#Ig>^JNZ?KB#L zTe1cL>LD;v(m$)>;^w-w&fEf2_Quo&NSoGR#HQAOU4LEGMy|<|Xg^R6Ni&D5f(sS6 z^s-3s)3?s)I+U#%8#RmczRv(%^j-pYmLB0nwTD~K2ctu}q-*!45gG|dys2B7+$Rv^ zp^vgEGN!_7c-nT53R7rC*8uRD?ZR3{20dH^((!@zqP8%TrTzY+_EgcwpoDL3fR{)Y zmk@BO8_sOvH=M` zcpw!@L%&KX=tjDaPjhQqdb6&A&+E6I0DJouoB#3c2e^$G&@w1zR>e1T;NzG001XB; zaK4!ln$Hass6q*r*U5fsEs zS7w}l@7KH{4fGx>iC~nGSH6|v+FY>y#jVk$<7c1Z_5`KeCHKr~yshG37kZl^;L7VM zQ0T40Xix=&nLj^VV)7u`ENm;+@Bi>y{M;4JssdwrnNL?i#(1wP!JYYQlPR=lVPaOi z`*KJ!R7PO%BA6==m<`(Idm_+FQnj1d82vI?|3wm2v~ zjNl!II%%wnpqH=x+|5~9;M@RR;1o$;15@0|vgqKfh++FZdYg+**MsGN?0H@mMeV0T zc33M+W<~>WIP)57vM1*Olo_CH9UlPCX8hBgeGE-tr$S`JfBy6h(f9z`7*dyqQlQ#wyDoD}1?OEU{TV4dpNq-6lc#Q*r= zYc%_s^w3D^5tkn>UHN4TDL(0ptut!pyv+b=V3z4%Sk%!|ei_QFG$p{$P8mpI-U@sm zYTmo|h7u8gKAX)84~?pO=~v#r1=6wRP`H`F*hv7jf-FJ#pli2-MQiU`oBKSDj_CrZ zlb}uM%B+m%jRN``;0;`sYiYl*1qsaGu)(AYKh@69`G9DUiakeDRLPjmJ&I;Od!_}= z1yv*U5h%VO#+iW6!*Ng}h&~4ThQ|N*13&q-sbc`@%zW+JZ!(`uhS6VPS#~x+<)Fjp zLxX$WXJK$~)rYw6ZoPF_nG0V1)7M`&G743VN?X zydZgbeji-`EL({TS%u6IJy!%5DmePtQ?yrWCfLN=uT||-<|IO8Sdw^YDW~8`09wj| zQ}=+rd>$;p2`p@3`ucOU1}( zg=tx)QiK5m2#9NMY&7%cCV^>gh7Ye#Z;xj%dNW@E&I$}WIBlryqE#)Oex8twVs>%M z9iaz5Ib!aDW@f02xlZsx0gXw|dl;n6hfAEOI=2Ft2ZQ9LSnDML4 z@7xS>rm0w`Opg;ZY(Sh&poQ8VbQYQmcY|^LS`ff06Sp}^ZOi%i`T z34ZD%UBi%A!*1Z1}9K!4ffSHp3}aF5*)@rie`Ep#;G?dhNsp2j-n$yq%q#GPkdNA3}SbVEPm(Lotnby7`ix{J|tBz#uPR^a+w$Tf>e)?Ubgo2*y|A zvT`Z(_n!u90J%)@smo3DPlFwXx89(A9JyRmY74ksAXy^V(6rlyuR{VY+Rx) zW)|u2_F+mKH+nBas58#%qoOD~OcZeifF{zReQ}s43)22{0FOsc_)W?cvwd6|pqUm` z`VFrR`zMrKxr?6N-^-H=4s*K2WK#VbgRQ&h@6#_#Y!sEr<=KdRX)p1Irkkf))GL~4{n}$yWGt^=U#Xa99Xq9mfQ2Fi_*UMGy|3yA*fKbq8Ux4 zrs;#~&}h@H`_ixdy(C^!$?JDnkbt+J1=}75<2h(1picV~i9%=U|Z zp07RtmIMOa8hGqICT2Zo6m%GB)(-T2_>v6x^+FoYzM#yRanwV*XWPcoc4;T{K#3DH8sc7wPGlO|*htkr?dVMohI>j)@EYV5$XM+%4i zW?Z2O%6LtF%!0JkUJvSQQN6L$(_u@-c%i_H!7{t<20q6$u?EpSDg+1E2Q1Xu(36Zm zL0igL|Guca7ia}Ji^?(y)&2C7 z($TV{53qUwOWcPIol9c|fO0>4o^frnS)gu_2B2p zqetr|X?HF)rd1#v zWRS6Uzg-#F?ah1e{l6Iq7MB805Gb!w#hOaIWb_4q*7Ep_iCgPCPO~=FE92&)u?y;+ z00T-y2(miY_MfB~$2%amuIk`7vwH{=@6-Z88EHL#Kngv6BIsEa33~S}lbqpV{HLwT~`a4P-6btIpeopg$Nz%G|WsVU#*)KNY2SSwc%v z*Y3^#`CBJAhuWm}E177zgBWyz{o)|Me!A6lHuMIs>sR~@g;|-U_QYITas<=<=i$r) zdYYDr3*h4eeMRl1o>P!HD&J7An6cbrDhNo~;Q0kk(`BBWQ2BgQIW7Zhts>9g1DU?Y z8GAt5_s&=`o7K`Gmb5oOBZeGU2Kr{8^yfjSnHm4lu|+gFusB*N%GF!d(+bU8(3RJu ze-J_y2L>7ABIp<$Dq!YbAujcRq!BRv^KaG1{+Yw zkMD$RWs&QWKH)t+ddmtL9kIoduqxe51t5jNXTsBOaOH(`KFn!v+Q$^I?6e0P10@cF z0m=$~P_<83V&)u1>cJ<|1?{c0)|nm}Ex73XOJKC8e(+SYT`)*MuJL~ z84hQ^4|dd!+s$itp;eo!%BnNWaw0R!&;W{?!3?0?gFk_rLle0~?QORk$7`VKD1heB z&0RD612(``JHWP7yjG~JK$mIjXaMu9id*=M6vkq)^j)P_7<&T7%*S`Qax2`qkM#D7 z01tWhcJOf@?W+TJjLNT`eRGe6iBGuJYxjc#$FA^kndZ+U$5tEzcWIreeD5R_!P~c~ z3e6$Cm%-qmeLO3>esQlpP!5f8(O3b7pJk5HN3^d&_%HtP$3GZH-@f!GTz)iuyWJKI z#>~9-HIc0DH8&X*ZgOM-I$BLO)_%nnU74W7uwK?L3y=<)zTA@&XSOG{PI-O!bEw+K z0hY)$))4BQMa_EkR#)`77-(Gq>w=(G1m8UXdW8qQ_=|BCU?!`vrW#zwE;N7b=cS;0 z@f|SQBo?IoxTW$Knm|2hJqhg6BNtzmE|m@iT*PsiYp{^$2w`-Nf+`r-d=nDzav-Te znB96)Q9uIZmJv?FVVsJsfDPq-awmY2dlGL5s{0))0m<;Bv&u58{L8 zxQ%z7SD}3HmtHTO)c)jeE5U2{Vwasd_Lzw@l?p}=s0IFD-1Xf2*{7I(CXb`hl*Jr9 zsFRm&RUx?b$xG-K^l4uRg03PtZGao#Du6e=tSluxo)8R~D1H=1DHl^Q*jRLip}{RQ zRr8iIS^}_Q__{W+?!NE!cH{t8THQHb8VK8Km(&En>nWr$8EcT1j>}P-rj}u4q#ob59&%+ z&(4+S&@u-x)d1Fyxe3Q4%DzpogZzb>Tm@Ej6pZUOs5PJhrs{yk9;hW|s}&-7amOi| zZs7iPFere%Pk(fz0J8YXt`LeHg@Ol(%ZUX&^4MsIVcsX`;v@Mth2gw zO8deCBEc8ehSF)lK+wfoz<6Gu3FO)WR=~j45#7qes%cX_j`93J>f3jMPk~rrdOsd~wd=b9G1~X!ftH1k% ziUgkM^kZOxL3)E#P!Pl|0n?Xm6NQz(RUL^Ax(R`3muBmo6T=L!Bv2NpU+=H!4SMSp z709aw8u{AOVEV6|E3>1EMqlCYl?*{~qfdCuIq%%_--Q%Zs$_PTX81m99}Q>CoUvL~ zCza9aVMxyU`R&@@c7%4cT86++Jzc<1zjUAU(4#y-sP+q#(x-EVEc0QbnlU#6;`XjG zd;o6|;5$5j60c@)ssSnc@#Ym4I~{Jy)tsykG@$(wKNKHkzF<`4VEJ$%I_OZI=HQd% zq4EGeAmBfKbdTq(IqkCN&V>P=T|ZI4b7rH!&;J^1uUmS#Dll0~d(+u1AQkFuPywC- z82R$xSLUQG{0|R5cJ&tK(Ps6e^i-ce$+QM0OGTQB0@@4=;LA6ZQpmLa-8Wwc@6BO; z03W}hthuqy!z%-Lp1vK8=#u4lxk~3n(3mCzpf1qaUxFp9gwpc>Q)U^|xGimP1`M#- z2-Z%A>!^g51Ih!n#_#}!W&8Jjag@ol1LDzC_~a_v`JxL55uuKS1|w3(W`1*(_GzKg zv)cjb0OHvgulbVVP;8)>P$hb17Xr5Bv zs12_zr*Uq%uFD@BNO=1paP8YF(!otn(D2&RdLwKbs)~q8#=Ww4Im=ao`h#Oo=|mue z+u?@&*LcCf-)Nv=^DT0HARvzXn$XxcEi4Tx!7 z6wps!G353@+QXR>Ss%D9C8~+B>B?dQeE5(^aPQPFZtESFCL^U^RP9e{XMU)C2%PkQ zHU?-~LFSd8={n=g{J>dj7ESs6jK2)f4rsi09bEep=#Z(KHqI>E z0Jn4YzCk+(Ius~%5~&t$3|^2n!*6h$mIJIjMS{;K(7;1o7L&DTaWNgLIF6!&p>7i= zBMb<@Hg3$1rDgccYwFC>&nZ+rDgN@6L$7S<%9G;H|ArrDUhpWHB@q(3f14*;BPecm z8_ULj4@_RqY!z7lpA*5%InH0x`VagCL|m7t{o%gjVQs1+sd)HrEPxy2k1_^Bi>?1# zM^@Irb%4c~_T8Z-<}vWkHGsWL`VY7r7iNPF{?_IV7W%(C3dDpVUp@I#(S*DHjgU4C z;o4X}HFT^Q_`KG^cgcUQ$1x$Z$%2n2KE?$6gC~CvlX9sRD?@Fee>>V`wEmzxOyS;| z#v6i$Pl>+-u5&?&U?t^KS@6%bK>DF_g85?&fB)!wtEoVFfUvHJ2>l209ap9w{9R8z zlYqZ++P^rzZ%A-yhxR+6TuScy=ZXL4c(JJP|LYy^9d|^x$GooTN?qFf-<_4#8U$Z( zfDRGD=hgrB&{A)W+6Pmisa}+!*{e4{xq~l8d=1H$W*US6Q~`?MtIsZoldkhk8HgLt z`}1?Rhz_oD0a^%ynOAB*sFDt)y~2v`L;#;^S6&qzT)hGDHSm8wa^lTy^(HIhtgzvH zoddWV)TERE5RuIs} zHsC0OXn^GEvEgcA?eb2AO{6>RS)l_Wq}Uvm7&q;e)<`R+F@FgugPv8fdBE( zQ2>pWJroOQ<AJKoVW0 zL;E9ZbRZDVY4}>t2&(}B2GE!Hg57Wz+Uq>1{Kd^(yKG(=k%sUI?K>?^%?1DdIL{b)cq~9~@SOdiUA<@C z$S4EWFMNMsOYdd}=#yvn81t$Haa{WtXz42NWyG1LVqDRjdrmTb6Q5;zcG48UC(nX0 z_<#snVWR?!uPAKnXrIeb+eo7Tzy0;A;A`*QIVf%BD8Py(*oENgfcA$jXflSK)(Ofz8sE6=GGTGR;?pXy0YUo&<9D_fD^rRQ!>Ex{{MGIX21eZ9X|*J|kgMfuyK zFGg0ParR_8|2t#9=v>1IQrGcl>ziRP(+jXiVohmm#XW9&@#$-^`P{5<;IE< z>8b)3O-^e7H6?=aF~klFT28}78(u+5TSF{sx)cwv;cSi_BX)_P1p9gGu7z^`l>C#s)!-53(5sW*YWJ` zjK04_61rO2bJvVhlQL*6!b$r=`EiCT&_sU-cmpt7%euhjfKCxs5P6~s4?5h)-9zis z+se!!JV0k(04vR8WPNzgIQTATItuJRe)7Pa@=XAmrvLn-Var8;#97~9^!j&;l`8x3__AE+sy zV<7wPU2yGtagZvWF5b(hD{s%Ow>W*Gs12Gbmwd4KUplGk;4!1U3Io-F(W(Y8hE0n( z7XSuCfG3xLd4YQIciD#_558M5w&dnthj!{Z7?8I}3|gP5 zA6(&ViV{^7(LI?Hu1W-N;`{*b8F6Qpr!4f-wZD7$Y46!`M)$jqfZYVAd34l`(lH@% zvC4@fXpN-XfnR<#5PZD8As<3VJG5PCMQguE&gyN$E#jy9XpYSDceuznxbZD5RVq`x zxm&IW)b?@&6c6eG`7c$03;e?&t;4Ks5=si8ajLUV56T!g&oWipSr}=h4Cu`ll|w%@ zgx2K~oP#8gKQ71O+OEPx`fJm9*z(crI7ZN=pGa@12yl-2GNsh=y^Jmu`8)zGTmt&5 zU{FPXc)f}}>!!#wTI(j{N5y}{zH1MFdDAXA-e4*inxMAKX@AwCxzkvZPCGKZ##wq> zt~~_~#BYOTZg5aw2@%#0W^+WvFUtlkDo|QIt)zXaJ*=YSxHB4dryjLAf17@tDB3qt zI6;d=WR^MI7;2&3$K?W(gQQ(Jv--uWP*cnM3d6^wPdKX|P)2v~MXY1e%e`PDwz51? z25eYunpR3ecyMm0XQg60OZ&+?G`P-4i78sPR82y6mo=DxQ%&kED1ISD){J89&!5Y{ zQjLCrdWNIDxly3^LYb3Bbos`yR%TUwE03=Ic*R_EWIrDSK)J8TQzu3GL3kf)rXvCE z`(uzL%=A!D3nR+1rDGpr^A8U_1EIsajQ)r!+68wzMmH~l)EfswO$+ppfB)B89AmXl zeca}Jxk%r@QCwZw_`y*&jm2IKj1l|JG3b8trM+KvTkr& z6YT{g%unTv^fSZ(H_*G>Bf!0}-ISVd3VpO|n04;m+~PUU>u0j#;p0o(BTSg{Ok)eyc^JdiL?t!8i+msw)(-2Q<1h^Qcivp zo)Kt`aiO+A=?9lz2dPYfnx}Wp`qkJ}98FpGT1Hp9!AbjjP&8diW*s=qTyCBP4)FOS z;F;A1l*TQw80i=acZMDU3N2Ova_Q$h^@e9Y{RMO;`?^)^a6i`CUz*_A|NP5G)RU}G zE&^v^6eJBm_J}^51j(UzUdfPRz*Mtv9sGw;#RPgbGrSL-;=P*}*P7zir#L=pI`{C3SS#=!t{hkAkZ1;>H*cEio7FHb4TQgWP_61>N6Q+QO&g2ne7j7=QxSyuLoL zKWca#HSLIcj?vrC&OQtFT$?x$;D8 z$!}h`3%$0W55C9E9na}fNt%+6CIe5$+)`j88Z<$D1x=kLL;E6vxa|hf5A*-;`wu>e zcDnSjn9-7^dhUVNThCd8D<44v)a2@HYdpfh!Af?!7ayd5ySFylwa-3*?rfRR#L(6L z;5+>?3%aBN5^4>d1}uqKaV_uu?7Pr;#0G}Nc608glMUSpu&AwQKOaMLtA|I{jtri8 z(cP(4#YhNE0x$?_&OZgtI!vPjF(TCMX!O!$HPYMf-o*L>lzs4^C~E$WICs~Tu~zh1 zWm@jo)ib=EE4Y!Fnr?8lG>b*NeTt`=Py1wDbODnh-o9HTc#V}4c&YZKv5eykE`ce$ z(WYqv)+P-QvBPWNv;si0I*95bf|V~iA8Z|@0vtaVGiNAeU|K8RLFJ}bK!V~hw%fVk z&8*2_2rn5(U#hcaF7RYcWYO&o-Gz`J-U9|OJgh|ZRJ&@biQd`B9EeVv{P?Fb;=Zd4 zSlFDrF~X%dC!luS4|S^(j4K1&Ya?sncKIG1LyK<=m$}-$M?W_0|L3dEK-MKUF5A<~ zQOccAx2t@>lLK2>Ex0HHu7-V+Y70>5_6QJd1lWTPeHGW9k4h4E0oZ{8!n9wffyQ8+ z1I2TJ6k9OPvelisbHp2p4X+jqLHMg|;s$igs*Gd4Gqu}6)u-p-B~UW7IW}R zKWGSea}aZb;S7{A9+bDBF&xI;fA#F+kUQERsL5&TGA*CGw_u^pHHa%pn!y~$(pe8n zKz{>~82ewp`u;0l{s4-pn#7{T^2aEy1yzAgKLBO`0d$>U$%h9X<^JcTha2I20B;)= zRp))s-aq-tvjvb<+u3cK0rYdRJ}ycw#KHeyRDs`TIZ~MpEyT{&m4Vj7z@zUOvuZKr zA9N&x=)t*zA4L7&CA4(V;y`H4s9me^vzH=jmeK0_@M-Yof`U(f6)<1x{@b5EeDJlB z4_?0EG=(#TQSmuW+8>rNuh^`vli6s8T~W;$2|i3{K>IybZm7C!ujxo{-vHFCZtc5{ zj6d2rz`oSm5i*=^1iW}KCaAZ)ANYj!Io^5DSjATw62KclzkTlO;K0P6ethsI(!x5x zZJc7d(wM;JAAD%X*@5YxnS1q?C|R=~-5TLga_NLL>G`*` z*K?YjN8qLfd7HYbD}}TG@Rs&aPKCCmkG=gfsB{k_i`Bw@l)h*BCCeuWLlbOzH!8OT zp(A(I)|P<}Ob_mh!fyNxP-)~P_U=OaHOv^%7*RC=X9c59n;#f@yikUz64G0#MLPk<{_7QNv z62!?^3Ct>j{ULyNZu2QyFDjG<_@?R)@?a%ZDWvL(eo>uh@pL?3Ji}NAw9;|@gnf4m zD_NvkxJW(*uhc!476!5`KYs{`{ z%)wQV2c5|b2N*{VAZ2==T@|DAN!xNk7tLGDm}zR09^sC8BE2A#dp`R$^ngV$u3un3 z?HvJ0BLmm&uLWmrpL%T!Jgu~YR{}wsZ#^KI-$=@}JA=3br=z^JB%D#tfMO^u9r#cEdX zy;n=JdhczkciHN_+mdBjy==)RKp>$fB(x+1k~owAp_9OYF$^}w7z`LNwuyama?U07 zmUI8W_dN5RT3e8ExZ~rF@r`esF`l=~xz?I%&bO_#rM=hE&P&V~^H@fQK&LC`#-#;0 zgZXo8pf@fBoYMm6iy~3M>2dCQo?(=JWdA-0+VX%mSfTby{A59GP6avybf80fu1<0^ zn6n0SI!T*}a_SW@9U{>D@cKfo#g}QKeuK!OG|9{IL6Ac%t41!(r7c3^0C}9|*>A;3 z%i#1s(51Zrl@YWlz2I*>X31Wo(;Zq=;(U1|WWr9k8e7Md?NZy1qlXbYrr!Li9)%+N znCS3BfpJ52;+XY2SLZE--`|vGjIgCI>H z4rmM8Ti{d)We>;z9TFq0adpGg&>*b~wx&HAYkNrF_Ae8&*+v? z$W%)Q0Zw5@f-EB(MNe}g^NPtGzRG9^tX-x}oKhKsaTb&d;+Ra$DK-QQaGgb$ADD4! zDmnF-W6IJTO_(>t5dki5qO=Tr1{4C~`gs&j?vkzENvvBCbK6G(mi(ar@hP16cobMD zSF^?{A_&V7$$FL)NS^@Z(_pDI#m=wpIA=PL;{b2orQQ^u#IkecRc{^^4Prv=b55#< z81rsaZ@+*F#3b>w`mCIkyYx>mS^!#}rJ$WrPxOT16e3xn1w z-3EX6ju6r3AV>g1bRs)y6+e20#-$3n3{8}tPks(%yiZz;{WB*vf*D}yF}aY8XFz?5 zy#AnDH}vHXp9iNl-xC&Qy*0*=E2NJ)>3a2$C|4)~fAtU$=*t1)d!>tUQ348l{twvPE;Vhh*dqNn8C^ zXPFsYBOTxYqhO4sNzfXogZ`vn=hzY~LogYdf`Y6Wa zKe+4Qa%$+*YG_97arPu84w-wFT)GGH&@w=?0ss2rZ-aYqrGjhEe|GO$M@Ox+(jayS zDl;x0q%SSd7@hz#$qedw^8~m%!_2*>wZ%3qV4^OSF*P}6XTN_^`!{dt=;asDX`qTO z0@Sq=atq`lU9^Cpb#}>LwO1IrDDg*B(CiB11Ka zGh(a?4(ny1RS63Bv;nUI^{DDl2<@-YQlqVpOkY9xiZC6xi=xeaRmJ zP~U8dObQSE^Nl}{o>&oF0uYip{l+8VnbH7X-~zls+eX)y6hE{JSqVIJupGSlSl~iP z)_|?Cqqcb?I!fhn(#9qznXcIb(oWNLf}tHzyn7$fBO!**-+bwQ$ocoaxAQ*WIrVEL zljCZII};-3+RNZPJOl(3;N|M9g{H`BkEwUeI(4CS;aRYVeL4mH@Y3^W{`RHYz?n|^ z0+hG_oi&lZ2I4>^ra5VVtLIjGXmdHFdwZa-wSTLUClJKi>RGT_l&CL>=g)fEs6PW4 zJ2`jz&ociGz5O8gd>_~rBLP?!0d6PS1jK58y3y@SF9X&h+!@h;xsX=p>KKNs9rW-p zND%d#$N$GSZxTJ&=w;f(eDpJL9-&^f#V}ptiZd9Iz<=EE%Wu#uS)E{xAZ;sXmf_J5 zEClv#h<5oK`wq}92^nK+5Rp8C+0)JeqJ%-)-lo1Y#aKAQ8Tf)+IiV&HkD!HUV4vpn zaxU!sZzg~D>H$ZP8*BCne|qdr>Ew{W z$k}ICoKxn#N1-z?1o}XlG!<0C>CqGdu0}x9ICs;ZC4-D=;Eh+nHjZ4lRhraoMkI|Cf(Jj3*aYH*VWj!>DE*3+8S?R#@hp{%$4rvRY-5RkEO7twN zay^KaA1LDrfb@R#BU;+k@59$<6NYAXqvOIF7e2rhwU*b;iY%o*1V$~NESd#Tjr^YyO zfljdF&gvZ<4AAabC-ts!dim5NU^L%b_Xb$KyK`kj8;EDrM3z@G-&35x)@UhKBtZiN zsz=W6Z~MNZi#-}lm?+)^3RZXi_P&-Assqs2#&rSQnTcU82jc<&^)MDlb^*2T-g5dG z$kM%&(!t;Za#q0tKs8aUQw$z|E3m1^2LJ0f)&AzyE^Voe@ep)#3y5psT!Ng+OwkMz z`byy3y~*5EKNf{#tTK53yQp4u{Vu4JHn_#UoPN1~S{zMJiv;D56iu#;M{-g_Z$7uZ zGFws~2HkVCli35TuULx$?mTgnwZqfs=xTPtA1dfH59!lE)5H*)VP9|}(;nc|_Xj0Q z<9CmLn|p8t{n8^maa-g(L^DVSR>}eBg`jp2;Mm3e5|Pav+K?e@SFUTOsquniR#3hQ9U)Fn}NX80=P(^QhjL$ z%0P>BfU5_lNoz~btJ;x<0p+T>(k5}sh-3BDsli93;Bu@te`cFRx z79u-DeQ?+57YCrbfXd1mbL#{3(Yu~FmBLxHE?v8;YLx{@u+Pv^X`(wiB$V<4c=+aud=po4=IriF4pAe}o*bb!UB&_TKm#6#74*rh=qb_N?C*=%Ef zFsq+|dfNTeed;d;&|(VJV>jlHm9C=YUu^6L#jadimbn;2bLN4r-v$P(1(`+p=o!dQ zeh9V=;xIg|V3j6MGchf+@u4N`%?^d&qyw0Aku3H=qG|sKBj~rcy<+CV7C=#Uv`)M& z7-=`=)1}27N_+Q8!x4hhHF<(~7? z7r9&-bDRsXxOP=< z(3v`Q{ZU6+g4vKi2e!4uLg(Cl{%(kiH-cG#KB59}R~>17J6DP-9&4aVv?%&NY0wP* z@pJNEGtL^&%gfeoW;xnYH{?CG>uF=dZ$7)0M-$p*jGex_r+9QRK2;xxPKFUctC_|Q zx-fJ6$Qo^%yC5oB&in)dP(QK1{Hia0az2kHV?@Mo)P^h z#ON$o>mEjCYhT*iCn0H3#)!L?UK)=fcv^a9wp`UKVKU%>e^%GP=j z;Q5iXnxGt_MAQ#%N%ktqo89^FT&um}x0HkJOWYBdnmFp|Eh?Qqs)Kqz^diuCy%^bJ zd^{Wq*Ti610P)a)Ua%6&bkO3(X4)U@@~vRZp8|%Sfe`hnw^zNEiKhY@tXjeQK(6sc z?pGdZpB*^;crcn~kYHZ#n#%zE@TXw#`FTcG46D^g55zN-mIJ|P03SP!MElcOo$LWcg47pj z)hq0|1-Sk)n6_^}kEg+pwRRq+efBRk=IE7fyXK<@2~R5v|zwZ)I@{QV;%? z9S*G)an`^Wto!~?y=a&sj*puc*`t~+RnPLoco`;#Ff)Noe5PQZ&S2yZV=jXvHEPkvi?&?9$Ms`i zxd!*^cY?ctEU|$GkUek&yighjZ*PWbHkzVvse+n8?UTU9L8H@wp$BwUj)Zq{wsho| zz+5}lk-IVK9o67Z?*e%&2wgM|4>FCZN0UJ-WmE&n=NW6j{)#a(_XTY1>{1|T>DAU_ z3!H5ZuSbPErt!D8J`15^9$MKx#z;VVYZM(1Fr;-vrb*Av=fjQh92A?vl)yRk_c}O* z1}-JY9^m^#bJGr1cf&ddI}SR#PGIv$O^)ouwS4lkRB%rQAJaGSc1ahwo>nJQ>ErJY)j+f|POJvijG~>O(>EWz$~^-? z$3e6dlLg?_q)8`YlO6M9)kd~}kJ0Vtehjt(DgpsPl$?#N#s9KJ`1xlxqkMgR8HzN$ zV4Rg>$#&;Ef32KNWJFtm<5Kvz9+#5{F0TbYMt26~&J0<~5sTN_mG6M(`-9PQQJ`|} zDF}nGjo$|Uk9%Lg2ZD#t=OKsa2?3V@`15)I+n`9Qk}n_MnaXCJgmHW@MfH6h@@G0-w&EM7gM+Z~#E zsw&Gdpw9#C@bY2lN)~ZAJ7wbV0nxo6qUag}%XNlYTM3o(nevoAJsqwx3Dm3)+#F>5 z^0=K#dPP#wfpuDwb+=O|trc`2nyWa*N=+{T>Yx@VdH7{HgyU)yPt*ufA5lK}Rpzd@ z-=dc}*Eo`!12LBwt)5E1nwGxeigqk$)|b&Pppo{1I`bE#!*?tM3f*(+UND_z%q9Ut z|C5)%bfL>(gb0{qg2i0ck2IaNP&xSO-lUKwlYMx6e5y7sI^!)Wmw(J;?;FQ8)|P_V zfdF3j`g63A;?Rfy&9_zIxw}w~f^=w@s-SZi+fn+BMsfsoq}tIxAG{8h&B1};Zjn5V zuiF&(%3EAVDi0QF~H5hVb^qppH0t;y8kQZJjpjZYVQ zOXxNOM+}Ww+)v^_baE(YCK?P#$Ohvyf$Di(F);qw&Cuy8TDElaxHrV+#z#Ln#6*AZ!7^~c@l@NX-WGZs*dyYtS1pO_ z(_(5gs?uvFXN$uUm4rh47sPTczi7O<;n`sS+rRqU1Mzz-<6M-y76X z2T`W4Qq!`9b^;2+XbF}$VA%DzC>0`7MbvQf`J1bYr0Gxh=)t5XiSd2uPOu0w7|*5; zg%$%kFk(uAU`@>hMbMTs7E^(PH#2SlUr?*42B$MVzCoG+X^{IxZ(^{q4bX#AheiMB zHAvbsX;C`j+#y3i6}h!~+BQ|uoAFaNYfntIxYwdB6E=gyB1xww_~_fJBM_kszWMC4^M^%$j$M_1__JNu1&(0t^v zxzLt-Fj2VE|i7veQ-R5HT zLb*&G?CKvi9@ysN940%qp1n~N@P5$mUU-`0Fh`PT6$Ab}9Vga6cO-IZ4bd%}C1CO9 zg~#&Kq;Kj7kFNzagIbP*mE^NVn?bCfQ)AADIG}-OdW{qVVksEy0^^yEHHE7uosYL} zI;fY6Z;hfiHYwlk0OXKEhy3z9Q2mM`zt9?;0L$|`-va9Z;s4k>U&KI*t{AKwwA3H8 zf%A7Atfpbw`=uQV5zo{C0umXZz(@Cc4W%b|tV?UOYRIH(`wG*V?gyqXp7go)Qz+>= z8KE8v*|{gP(=$wM3O39BG-NR-_^t1{%yEp~qY3}*{$Fl>dW-r_md6`EK%aW6=eu_d zSK5WBx1iAo^lWGd%?vcMsC@9NVQ>z3f<=fo;Fqi?6Cb{T$Pd=}HdRE1UN-C4d3-R?dT+%vpe0vDwVU z=prKBj3JhSau5U4ro0{;$PfXZHX~}=xb~bh>06hNAL7!+8Mr(_Lmwk3Bw{RpE&$q2 zKRTifm@GZu+FH^MitQ663BnSSu|v$90?_Q%K*>&C$)3WsX+6-gn&W<;WK$MBPy}M> z;S`xmbQr+Ib`=%ey9ca#TiPaw1kfZ`0@#Z1KYcsHp$XJ4Jq$LF-t)Kes-d$$k3P0& z>ySL#>brShN_OSCK6~kdTgG_^PnMnQ%HX3Xx158S{w(0NCgv)~6R>imiOO9T+2$fg zN3PxsVPhW(&f#XbVxUL_xVB`>lF`E~0ZcR|J*B7CK{I%if&tnjc>&zM2<8u>)2u)~ z=3bK<zl&xUXuw7)(^rD9y=g-YGycd`uu$ibLM3M6VG}yb{Jv;#K;QpI(r`k zEn=@Fiv-Km!A_2N9JCuWAkZA%(c%S3tLgbxaCf$|!xox7Az9D_o6Ye~MgvMS6SCdt z(T?>)$EGUSJFo8zJpCB@@OS6HScb7{zq@PNL9Ugne_#EA4GOT}$o%{LBe!Nl9Fabn zF9LLqW>1r+M*d@t{WxzP)C#I=_*;*W{L+IJv0nn8Dp}R`hg(iCl!32FWY+22HO=o& z`)W^;sr@lt$j*v7*p}broOj^6_78|G#t@`Tx4x zZPY)!5GexQlJVXK$N zo(tgz(RqFUrM{nM>`@Y+hfPP<&ZZFPyj=u%d#yhgPUugJ96r27HiPpc$7bFGU3V1L7-mk&%GPqP5u65@@JWwt7kRdCd z-CcwpU=-9=&pFOY$6s^aCx`Om4WL&b4c;Iwf_kQ<4Q`V*CVF+rxu=X?C_%i1kB$?`qxkU$=o10XkFH$8zkP@Irv|ok)t8Px zKJlC=C0vl80HQjSuBq7xnRidpEnmXr@An?<_Lt7uor(X;<4>UH+#Py43=N=P#8DCa z6l8$w8dJZRe&<=~9DtY|oa?L&llCR`4KUHSSjTnu>jQo2eJ^P051+Du#{-ihHn@f) z(RcG8G;lu;z5rs3OqCo5_f&sf`MZ9nalGDp#sy;I=rVY}ULsUr1Vp17(DB8Op1*tP zb?{jp#Ml^LWN;p3MF;d}3LWMH8~nHdWH9~#nk+$7;m06c__^0WQA#hfo0k%^baG}( zL-Yi_^xlIse*zbf0aE`A%i4ejyhS@)cz~1WF4UhLo6z8=EntoV_6F#P9@K86-2g7x zk^%Htzii;$FHp>>{^{x0i>2AT-2+803tEbF?>u}PEr!yE_tI^vAoWex94LYSLxZxV zqYP>yh^@Jh7!L@+q24d}0-dosjXVO;pl>`s`bmlGe1Vz3rKvwj?;Grb8pT)|>t!_1 zu7ID2Q9H=-{8(w#TGR4P&0M0+gmsr%<%pXe;URV-pdKs9tHzuKf8;A z0F#r|!Rp2K+Lz0hpFwXZ$VqkpXP^Qctq!C^g%$y{M{=dLn-6eDy%)4SAcmjSpHA?S z;_6iP#~=Ud^B=te-FbKcIu{a%jts}-Nz^^(Qz5d#ka0=V*yx*84PIK0%JaG+tZY8+hA|p#6N^sDpOBl0UpMSF2J{E z3S37JM-z0?kUHj|F$M+tLS(en7l_Q-IqDsS&72N1TRr$NSp8ASjwLdYiv~D!D~N$4 zSW!PD=p>-EDTI+X3_3D8Bz=<>OG}5YeC^He-k~=gk`3UT2#{w-2j}B=F;(jw z(3mRwO{2-wzDP+H)$nOS?v4Z3`tddrv9F&*^g1N8JD1CkZ0wcEeBp$v)v&RjkH z)#wo=YFuX?W7lK9EyG4ISj++5WmOkgjKPf3S+!>~bk{EgTsJ=J7t z?GBNCZt@m2WTh{$T~y0T(O!w=7#-?SoSxl1jB&<<&TTg13%~@d=zyS8g_17LAVE4em!Hmb`O6gKYuCO2-9K2fQ;}($v4;pikN+ zw=+0Cd_jHfRCbuQ?uXK}EiG^k^f%!-8m%p7Yz~^t3SnCz0XXMCx4?@g?LU@kqjkVvf zLy#w;6vDPxbY`J|;{iNx;^qW+mUN75xu5-cc|00Cq#*@tA%>NniO!*p@Nxh+Uw;Ce zOX>hI@ZY=~M-QUC8{HQkrj5sw!R$Z|vayt_TI@EpUih(e2oHp? zBtUk76UQy)c?9%+?zv~d8EpZe&C@^JfT+^31yv9PVX{}R`fH&@4xfF*!`KKVIb?SW zdb*Ccne}>}g*$I9g6Zkhvx;)^jQxZH^{iNEK)wir_t06V9&M_D`0T+gPP8d@J)tQW z3P(Q0=?jA1d}y+N0eY5A5%{#R0SXtedi@^oOrx7dWZ0K5KRB_$rQoV|Nc#4O4%I>E zBe7xN*-R4>J#gH$3VaEq9&Zy01z^QUA6u(pAh}2vi&S!l4zSb-uGyPcqt#8Y!6dL| zPzVUfHGTfgJDQ+yMoD{laRKVqO2EfJbIF`Y>i`!CFs1-6O|)eZ3x7fs7%#4X`Yy>9 z6!iw9$YdQ++yj^tkpUo~ ziZ(<@Eu#eJ1Tl7PB2(+sGeD(V2New@;688Pm9_IK=Wb(zzNVm^QnszDJc=-8WgKcr}O-?^R4`1}=O=z)W z>ilz}%|m23a=Q+&Y9HusjAk3qJeS&NbM{BI3<&1PRFbpJ&QF0a(Ov;u6>#bVw~7v~ z^YWdCe6+?Km_7GP*#VD<=9R=e`^gIs)zT@Pp*+FKL`vJ^`mNd;_9d<8YIMQx${`1H zd%*q5TPLSSFFg};{pVbdz$SlCV*nV>3|#m=nEIh7BnQ`vubqJ&xTe0da!Gv$nPb<3zJztO z*DJu>tb#c%z-TF}D3%7D0<<420#7L)1m}bgfyS6@QJl!dXWqLD!oQLyX-=V?>^YcT zPsvf}86X2r!%*j?somL(Rxu|6a3KaBZ*RVtK3Bh2$PSKy3OMwOgLG(Y_w7_{^l^+- zF6=iqdql+YItXNe%4p9xX&am#7y|vz?>v12PiX}jBnJAZGdX$+JGbVK(}&8>vUmu{C8kre6Mom_2@9oReHtQ^uV zKx+;pXzi_FLIe3-7k+4TLy6@KTDn7`vh&{Jss#QY-OCuGy_WSF%we+ubhv;qKc)Rb zH#ma)TflljKwTM_x=3^K+)8ovMEY2xp4F@=kr^Y{_rdo>N!X@Y`}B!;&2W+p74JVL ziov;F13G(X2;i8A?2d5yIK~G|S7Kl!Fr z{rPvQ3#yJ6y^;j<08FlZMPT|Q=8jzEQ7Z^w0Og4azR%D^dF!S3?qbZvsK3%e`PbWD zpLXB9lRlR&k4#)o`bnUZp%g6_>1oOrseUrQmnR1n^nFeC92fDwe)RK)w$?v5Q3c&B zI|$cTK9{RcF*yye9Bm|wI;7FbbmzWK=Vfhkv}#7cRv9+B2CLVCOrxE;1qRUCnOv0h z{yyocr`+TO(zZtRB4kUv2=GKhOJn(=Nh?5%!Y!HsxOkt}#YZ^Gx#z*ooRD7OIyM5; z->TfuS1A3H5K(X5nk6l89m4ao(nF{Ecl+l1Lb|R>9}lH-ufEbL9R_FQ6mT}xJl~YF zvusJ~7b(2A2$J?4;F!YHM>99HX{ID5n zO9x}Ok^#IM+fFX)4F;=wF2SOV$5D13$%oC^zgz8^}ZzO+t~g8r;+P-sy|<0lX~R4 zB^A&!0}2AAspnsC5&}y$qOHpbwEj{0A(@LRuwztla@QUNyYMs^9S9_k59qw+)LNy} z;HiU4hE5MGxr5ajU2oqDRcB9v^j`brA6|b#)WxHPyf^`FtPrdvX(3>p`nu@KNdbv8 zas_7xoV!KmsWicL@>APXZ#sgep2#eF#EJ0u=mqAWdQnDY7>~)54J^Q=VYA%rzdil{ z_~kpfdD3xS8volh!*K#!ds$`25^%NZrf#bEINn6M(WqynIr%Y z>K8Sg4g+eOySMT~2d=%!3%nl)j{(FtiFji5SotDe8i5rMs{lYhe8H{K?f!+nc)C`= zrO$8IgPTEgP!NM2*zWRD;ldGH{j=!xcG=Ry-2jGeN<&meZcEX)+@QnrkT7KuSWzCl5mj?MWYpsH7d4bpXw3ATJZD4)*9i z==#()QN}K;jV;cfs(=3;3X^VT!)~;Nvlsz?+IdN`wx{%RhV;)Hyc*GD$)L>*+8tPA zq$JRW42RvwM#FD`-V2d#^>iSpZ#}-jUct(T&l!3|e?ra_ZEb+3dgmlq$o!}GHgH72 zM$$GE8i!GpCY@eGhx4w#COUXAZJ7(w5h)w;qtn-*F$lD;p&e)#+-iq>>m)6aoUv9J zn_)6THNaIKas;E(c+M}>m^=^vz3RVarZNtG;Zb~tnTi4RtNBSZ)srqDP1+a69-=Ua ze#3P8_;Kl=hk3P2D&(b+v*x&SAlA!3rUYaZrw5$7za{V8d)zH{svbZui*2#K-Kvot z9rum&Ldyej#SVpqR7sgq; zgQoj+6{X!uK>1{`pf0+41SH^k+3nfR%Fg5rr%wUSor_sL)YaTkv>=_FTI2xjclx%O z;_KgRzsQJiq0{%%V}d$}CA&8sKgL`-U@e{2Z+#0&i(gl$8zi%*AH>x+RDca}-!YIj zb6AZ&B^S)7QQw$ph63<>iy2_iaT&}0DNtB(OO|N8y`axdHqjojGriyjESXJ*#EtW~ zP}lq|2zrhd>H+;t&^kLG${5j0Oz^)(ZNJgiDVm>^(R1OKgV0x7jSf*NW-6IL%bnvYUp~HZMlY}j;0XYlF*D1Fo}|?UI*4d!AjvV&ju*2v-;ik2fj4iQhTae`1ODcV z*G@nP7(h#)BoL!Zz!g{q4xC!0Yv>Pvr7&qf7|8(7+K|ENefX_WKD3TE2rLj@N5BNb zq2ubYIs32OMY{GDi5R;0vCbh6TaA-!ocaYQj66i{vSFbIL^JGefZ{<5&d-5LH5U|WG9JZUsYXU$kGbUm193c7(JXgcx|I!BG}F?at%OaNMhk+#Ki8eqH%bUb8N zc%!{O6F3AxcW>ETx>4Os%WlHedpPNb*8l#^=jI_<3$H!wR=O?Qzr<&=&_P!=v1lDD zzkCv+3*-qhfq?X4y2wDh`i!L4)2cn!Et}Qr%AoeDF9BuHTG^&QKmAw)lpOz#FpfHJ zu>e}z&h6k6AoX&!7zc*m-e$a!T>ZW+T)lq}T|k_!C>!+DCr^Tfx~dzZI;h7Tt8y>h zbR>{-4+R0<>+BCP`8rRjC$*So#?Ssb(VkbQwCsdt*-1DJ=FA&{^4+~bN8_u}zWiKk z%ZI--HdLEMO&UqnUJh06UXyOUI?h#_jVl7!Lpum~GJt^9lZXG zbTJ{a%Y)ZA6j1A)`{^s9dX}*K=)r=Zp1K=s*`=2=U*#!d{&4e&&cmPl3~Eks2e^QT z0p}jj2^yX%9qthmemOFr*vbU0r#mEH2NYT+-nZI=%2SwDMIb@7$INtazAp0U z;wpIG8MyPS{7DA>Znzz6LUW6j0{r@e7a&+`1!3c7AAq)KbU+DWfnlk123aaWN-EdS z%S8LO3wMd+$Up^`)*ZqkP|?il0|Z(|tVi?6N`8s=&hr)%)Hh3Hzma^AM}4h47{IJ3 z9Ab~2;*7Rv!9jHL^8NWYXz7+A=1$hZTyTIBDHl~oUPionwl@0mxF;|TQVT1zZy!1H zyl%-h@NSTL4`9n0v`_Epi|{keKq~@Sr8GLb4tPLIj{>8=fP@K=7oLEmX?j4DAcnfG zT+lW^jITzJitP(8QRP_xKN0lk(bcLT`#A=O`Wv=-0;ByKx5y5{J9NmSBWrSPA}O_lm4bbMKQGlbM z5yp3dlXQSjz4b%zbr6GP+gg`F=rqX|2%sl{Y7d(F;zMAzAjYx4X^+SR%LLo3*6D%- zQdBZ{N5z%fj`czD@m&gq=sV)0o(uuiUgcz3Y(R;+VxeaN1e7fofp_aDedjjm70&YN z+QH;d57ksp@zUjejCnxXVpRu6oLwf$zWcFJlh$Gp#yg6*AZra`j3YI9G(Ye z9bk|yRr+@FFd(y@9fH_kNQ)UVX|AJ8%mz?bp7wTn<=W{N&_ScO=YRpU&3M&e*40{2 znjVuitOz7nuj2T?&O?;V0RczH-G=oU8+4!;`$HMxKpYit(01E1;3FMbagZU7VCu!Q zHUZP47w<E+9xya+G(O}pKBIuqxkh+5O7emt%;6ZP^ z24=vMAH)55af$`0ds9f$Xe*uGO4F{6P%Q_YyU7gRkOoF$EjSM}^!n`Yp{+JX84EU` z`9ZK_!#d%)kRlMEe&nQsZ;(N-8||%VanQ}QG4+J4{j4hVbaP!37XZY7Xio#NcJ}R| zXfcqc)kh;Jo!J{ZZ${=Ex+k??p<50PVjal^Ik4snmQ{SSrg6+8v|)3}c>kv-b}p1- z=wZU}J`9*J*W0~&=OQ%5>;ayPgE@l$4^Vc6uZ|Vz18)I!2Y@-`gSmhXf<8C_mRCWS z0Slm1hA%@j8Z_C*(gXbErbjOQTwO98Oe`7&(!Ju#2x9%=)EF*_pc;Ckq|auMCG_AH z^X!<;Z>}C->e3+}eiLk>p=B8&3nXAP(h?+&*#n|Gf&A1~ThZ)AG(zxnsT>lBrAZ1w z+8X9J?vV`~#6V-Ss8zajrdg~-&oiL8%F3u3*{{HF3rf)S`0JoGpeYmkA`IjBjW{Hk<~9x4JAe)3PXvYF9ffZ@bz3@}m(wi4VPB~4ywzJ#{8 zo#!LS2K9f>BSM2cWR1{nT=c!-@KG()e?JIoN*2@GvZ zJ!(1)n!w|DBMkuHaoxe?O&p+AI=Cu^NN2&BC#C1wD`R^_n}R?hOQTXm&ym#0H*bJW zZ#dI-v^uW}>T{(F6v28dT`^!|7rTi%Y4&a|UH=+rA=OAvZ$ zta>dwp9r}aGYEL|cJ1H;ENlj@{$h$*!n19v_eS93%)8$(Hv2&H8(3wq?3YfS(JqOu z1On_>KRToVzK~V{&e?c_xU0a?kCW|GzknHoHeCbe5du~QqKA_ki?qi$z2<2!0pq@S z|G=7wsUk=eVHtiB9qEH_-3C$23h&kwp6T5Nsmn*>dP`T50Jp?ttU7U zmr&L|z_O3Dv*?4lnuROhhK!rA9EhfRI-7P}*3S1+{a7N;J`6^q{o6mE1P7J})!%O` zbhx6Y-~bp`$uYU4Gd~9drkJLA@BIXP-_;lUA!wtD(fpAQUx1=x3>wf{0#;fR$UV-e zmw4tv>jMF~Az<_j!176fFHx-2v8c4Tf~oJL_n-Z-B`Z=HsT>WcFb}G~6Yc!-LV6n@ z=tgZ=XiN`5Qps`ViC}Vna@kNf1DrMO@*TR`bAkYi70?Y@Wu5{IBnMF`Nc0WkB`Ln? z^Pk)WC9Vjp7DT(NHw1HV!TWb{qd@(vb@{k7l5%wQSwOp7`i_WD|7L?44r26o7q5e> zZ_+|`c7h2CAxO>ARG(5rJqE!=S*m~EuxtT8y29{Xh+z=`)UQM#*AK690Sr2x;*k0k zFBb|#gLqJq<>Y9cdqK4)xdj-R-F)Z4*wwh4Jri@h2ms7YUUzn5E?oPgcLLqPv`5K7 zu+M+^qof@5rOuqAdfr^dhc|!$oST6lRC1~|mP#_Sps0Z2M`Vl#STjaKO9$R9tqxyb4vI0C`qkge|3q3|LBB=Ka)AXe4x z;t>WT-3(MS#xg;)48S2mMP&vUph>Gi2WW6c+(K#&_&5WW1+H*>tT<9VpNu&Z0y1!@ zYj|%#CNp?*j74lEoB0EDfYcYai=eDs_(gj0W*RhRCaV<7Q63232`5Xyv~BP+6+KWnX62KTm=cVO*a==J$&-Bj;dd>*v!ve`O(|A_o%-z9;L~E{uz-4rL^e! zrB}dpavWqLf1$25wEEE!TsojW3PL8WNEqGO zc57S}vyj0A)C_uwGL}zU7^pHdW@J$c*yyPAU3x7WN_FMnJj4cwegiK2prF)u-4j|& zY(F@QnVun48^$-YmJ7bdNY=Vp(Jai2(I-s0;d7hr8 zqsD4>< z>SDl%Yrc99xXu)=E`%N^1gS5wr1=l9QnQQzPEieY5G4$F^({U<;GS;A3wPuOP-?p9 z;6WSHc@TEPt zZ+CG|(mVR{I-XgE9>HW0WU7-cKs+8O6_E+iS(p>Xbut$P-8@;8=+iuUH>T~R)qY^4 zquEh?#n><3W{4rb^R=5nWB*)3+fQE&>oq~;qTan#Ht&4WOhr>0)5=EImP}U=bH>_ruTI2+A}6N z$$3*q9TVe^_uK@|cn9XpOf*239%xtY0Hay^{DT%a>2BsDP-dBU2?xvrRp`0ye4463 z->Lguz^g$iJxTO1D~W-*O~z-34r-`Jde7V}TMx^R`=p`1f4(BR!K>S$)>RI9s{gnc z-J-;?W&`b6R*QkTw2VNfk%<4{OGFIwMUU z>j}l&!Cmwcp#gM`fz5Y+3jKI5Pk&GyDx1{TY2%@5FG{a))+2%XkyWBDt^SGXO;Xrm zUnHB`gSf&*X{^zwI}Zo7GwW2)YjKl(z(4=?ql`+x?K~a|0{Pz4U^{={peL27AOT)2 zfWe0cOxf4)+8&aB_u50|U5)VF`qO@+dgn0YvBt_80db zY(+a+x~oLg=tAe{-4o)lOFsT!f@!fVFTb}_i=qZ3(7MPOb1nF>mn zwBz>eO`I)&9u3kV6+57wrXxFWFgp)0^}Ol83ok%r(-9OpU9GP(Z8n_r5p{P6h12s} zQRC9Td$b3#pmQEmw0;t|uhy?%s!)2u0G*DpM;ectw2Nyfk_t4*Bdn z%s~3|P$LYgFIX;DwwEz>fud3?{)qMO0eiTr$7Dg-V+l}>K4{$l4otER0_&_d!60U5 zQ@Q$o=-+)jR%##~Yli&{VE5ElrK6U+uwP;p!`i zjCwt{uT@mCghU)sr4pLw-*fz~ejZT2_N510{>|o4Bu$I`-+rV99Qa!2F3+$m4F5+~ z*8#^1yzQ9!7wK3l^sn;3Ifi3f=-*9+FIuDq;oHa~;@kdijPIIg`dET4ykD z+ouAZB^xOHFK7R43&1x2DZEkN$2(#`Ue2D}12w*v_nfSqdggZ@%^OfJKC9k%tgmXuHYjdesl0szdX9|m^VOHH zYF14kjx~$7tAh6CqAbzyWJNl`fck3)T6RG*Ef7VZZV+9>yKa80(ARakL-4N;z6K@8 zXhS*dyW(yArzc<1_xb24jP2$8vUGSv@#EF{$4`J~JE)(KK?CRv1H6em^=P=Xd6NxP zclOaE3DO#NqgDFtrRBCIQuw|UNZk{*^MkeK-Ft9j`|b|t&0EK%$G11y<>JqwAFl)Z z%`4Kon;(`24H2=t<_**AbE5n zm#Wjm40R#n${pIzTucZD;01IX)DIHOZ8N92-}c1^uiaB5y>4{$q0r85?&vRl4Ea7$ zx#+iV;$Q{_3~kQPM0w{mql2gZ5^6fpp$$FAL88moXn=spF*7}HLea+5A!5qvm^XkTzrR<2brtC)dFXzJ&^ zW+?jxjeaO|oOJ33H-K|&-vBVyB*0>jPhQd|c?5VV+5#)}}x&rqSH z?rvOV1eBLa*QM8adg~7#1aAW6NOvOyOP4DrZnYTc#t;Q&MAY|-EMlbq)W2-ZP;^$i zKy}+3l8w#V`AnXQu%3-s$bBHL&#P%_%yudent<{DB?Pbi+ub*%>nR!AT=PZMReEpA zY8lhP%3e7V8Ygl@r`FPk&vRKkseq%cj+!QIAXn;mAmfAU3R$k%KgQy#7YPlx`kY9E z(_R2$Yz4#vy!tQR39<9;!A|Ic?IL>9KmoM%KRx+fh^3L^4z$lYsQ21;z=7uNI!k?U zYtCnMhZV~&d-_)n-defX#gylm%^-RmtH+7X&9Vm6Zk;XXiL%Ac_5nAG8!5J&YsD0>R~X3 z+48Z}GKk$ciUw(qYxL%v`ra!VdU#{MUE>d+lU2undX=8EI0biO!v^cYPN@w!B>fpK z@7=}>*bV1yZTJhX8{Gj;b}ae7``(Ry5KgRz29lt$n0J+fF$$O}X=H=8Qsx#&T-+3@MhMza-~Sz3b!-g&B1)PN=R-YGQHi-zcD12i=~1_Z zY@+vrIms$6oEzJU)4}%*>yXPvBS{bWHzt6s`G5MwBM?Ba)0I27MFp?x@zfQbp$0H} zQhXbBex!ufCMY$3A!L@>)DQAzeM=1l6O0-fls5IEckW@JHv=jU2HyKFq_8&D zi(Y#5dr){COfYq(L8#xcB=Jx{y~q@j2c|I7Q#DHF+cI1lFx7AXcT z@a|(JjS+7=3hkJ_6iPohOKhV*vxq0jVO|PK3@wezb>v#<4=(Tv*M}=yUY0{$E<_i| z02d>eUh3j-o$+R_X3ufy^@GsS5(YJOYz@3?!|x4!ROy!bACc@lA3P4lUK;Kb8BiS5 ztH+^ik@i&1nn_mn;T^1O`-8zaB9Jg= z?iLMvMYK6O8Hi<}p^E_Z+gqI@+2dDkkzVk{XfT_orGpS0i8I3w!^0=rp>!lLbGc%x z`Vert78Y5D2K7+C$@ow-YDWKZ^F5HPX8|Co8hy#$y3+3CU)W%M7wH1b|t zon`b1sv%f6o4ESsIL3kc%FV@x8;hU=j41Vq5b{rt-YA_({p)}D*{zw3+XYaemJpoy z2R{X)e(E(H_Y+((6G$)u!O77LVc=nxt{>h2SMGy6*UbH)Pg7ADmnFt+C znP_n`fcn`pBVSi-X!*8abkzU;;ZJYg?^LL3@3q?w(QE*(k>S0GUW~-DLT1;Qo4~4> zVhn;67R`PT@WxNM#LOZv+J2pv17JEZM--l#^9L^cXdw}Tqj=WjCSStc!2LYVu{j=` zf!P5PghTWluce24Q;c4oiVNmZ+LX}tfdo!bt2!t-P4z!y1|Ww9oGim{F;@VdxQn9! zUb+o24_a0;x1=Y-g1fiYfdP(DmHsi1o1l&B4}yoPH`VX_O^0&@8t7PBSl@^-8E8J< zZCO&RNatyMPPC==Fz{JAc4M)$zz%J*(N_db+ZS3T*Bv4on*#JqqoYH3?Maeu@Jm0N zNBD<1%X2xS%*h(`zGh!Ofw0zId#hSU;Yug)nl$9SMrzhqi8&@ zsdm{rjBd(x)&|v?SrO%`ZZ$hPr_8dOr9f9m#{`eh1L-YXA*Yn!b?O!f&dCPUVM!gd zdTlF*@*Pt@{08)PclB_l#8Au3SuNxDsXtjUx<{EyN0Xcx_v(w*Emk^Y(ybV+>UYMF zbF3r5JC9G(>qggcXYShCvU^H4(z-};4Y zUK7DJGL8*!FP6bv>G7s%=wb$y70AX({i=)!$wUIUNBC+nZwDPf>T7y$-31NEU?*cS zCM}|b<_!S_)C__%2mbJ5^&EC+3!!d(DVz-8kaqgn638uSmWkNS_2eC+T0W*51T3(E zh8f!&?RG_)&sIVSj2mb4S?2Y?Arv7 z1aa>%Qh=XgRvg>r#!z+_vlju5RrsqWqw`=f{7+{l-28qV+p0HkNopw}cZK zthKEmo*mHpO2h2{{l<8w6&N;xaO=%Uo~jgNDMVbyDy8rz`^XNK%A}nDy+Qibg#<;L#<$)kZ#>nAXYmRgNKH5=zTM4q77Bb)oZQ0f? zSvrcU^g7UFhV`{e|3vP7QG~09qaF&Qk&Z6wP&tshj3$Q@ILATehe^ZSFzQ;R%PaGs zf(&5l0((H!jDA6!H-FjXX}VIDib&DjXTRB1P<@KT2V3TXYxlgI_jh(v}FF4u<&|K)cxoI%> ziK*~8^;g$Uv}zl;dTpEGxT_DL1(@@@z!{V*WsM-<^!G*59zazKBd?V9VrF{lyM^}4 z51_@l^3XJQumKQ|kuWk&x7U8*y2p8bdXMa3@RqCv=>4GC1?z*;TxKTd%v0>?2BV)B zcvGlTo^7tj#JUGC7I=L~0%gawna8#7yg$GH`=?(Wt{AWItH1^WIyh(0 z;gSueM<2-#@2yVl^Ft5)z%4l-z+v3Q%lptOv|&)GbP7)$^WjMaULl6cc+MSbH`=wI zeFu6=Qn;=>Vf|=&xA@xGHF7Zmnp0Vam;N-%!AFce=n5YfYwtEaOsB5ML5%Te^@G7x zPMq1#&<9q1!Ge4n_nR5;tz2~=NZ>ig*wo{nxefY6e`c3gv9Uu}7qcu&+UiGsU7~Z= z{w2fo{J^$!G4OT7I@ozRqosZf;Rz+t#P|nR>H zR}2)C>Y>1YnvKsF|ms4}u)bQxE_>(Mz*zNt;IlJ|OkfRcX4O zdlE|1k$4s%y(ctIW@@r4r+kFU92np)VrLp?jk$v{)B!6NJFZlZvJWc56kwHY1-VL- zD_&1wEO0WLlaZNYr%yr~Ui1PF2MJE!1?ji0UG#^O5Bf?!t(D_)>JbQDyEroAbT`V; z_~T7?m&I>;u;OJ$%0@Fipd%d`(*RI^)i{0UF{sI);;EgRa@ou%;4utjXExF&dIDKN z+d&3aoq16{{c0 zFi1GXP9Gca>fp>7h)m|s0*LX#ENTKJfr>CcYIN<&9CFBR9G+P&kLnNbIHWEtJF&=~u= zEQDBWCcDrY3G{bDb!R`#(hXWyIpYu-aS1>6L1 zd2~^32bkxY_k8@ZmgpLMSI^hYI^LTT@&RqPb2J0#6{WdGqW~JMJivv<7~N2;K`b5R zdvm}e{H=H*b_y1E-dD>5^PQRLt*yKy0!&U|jFFxIqB~P|9*hA!zjAvD;y@o5kjjO# zCIY<-ZGj$48k_?=a_wnifL^Y{V`L#5CNmEP?PcOze3s@E1VvdKu!1g@vwoIlCQbzB z8w?WIL9j*u+bddeOB+U$u?n;Mbi;J%W!WJS;^UK|A5ve7a%Do(Pk1^k zi&iu!3{>A8bpA#iHJe!;v{lrRT29It#Eht-*-Jt3$^Msr4GB!Fg3XJb;am*mxRXmc zSIfw!(e}|1z@iAIQtSj+ILJGnYo$FF7z#Anmrz}N=BbaLL8rrFt!!eJBW2QY|!SQC=iNysNV+q*}k0EL4)1gtb4ED}_U_3WKPXWx; zO;N#zrw45zs-&YKsMFX2#j%4~D4=l-&`E$ExBx2M#3_$>&~dRK0i%RQDz#+=1@xhQ z>i>NuJoj!irMKw6D)ky0gcVq3yZde#S@`XumsfUJg zF@f|4z_GY|Kv$Wmxk!f6D*d0JZsjLtpaw^%AJ?~6UwsuVfE6sUd{f7g?w})=F8vT3 zIH3K#tv<;_UVDwHZeTAomaQeD2gD4T_$gdWHv@*8l7AC!XTv>yIaYTxN+=4h}FT*Uy|h zb6Zuw0re&^9e(J(^bAid@N>(eZO%SF9Sy+@X#&Z+&!O~z^d%Kq4~X&H4?5~2XFuv$Rupy?t#t)%qZMlVnR41nbR$3wJGc18V^3kkSRC6FtrD z3&+QL7%g^;IUkTE(bS()uinOb;qm{l_ulVyRcF4qEz9b?%j%NVEXnG<_ulJK@5PcV z$&ytp9VrkxCKzHcp%_AFNf;n8m{9XP?zmvE!NxY&H<^LSkj%`z-}wi=pU-}m_7TqH z!aO(kr#sK{dCPj&-fOS*mbK53&e7y|%!$7V0bF%?aN@{mvORa>@0NeI_N ze{y1UbTJ5Eak+Q=)6;KF)^(QS_Ks%mX1h>3NFFg~%Ml~o0>A<77qtF#_^uu(gAM4?1;}``DC5?5WFd=Gu5t8`~5ex z01gj~WpTEEuZ;?aG{KXqnF`!;qCr0YbpJsXNCrq?3$bYO2J0vd6gCCMTx;Wi@d4Zp zIS>$M!>cY9w$SbB%qFPZR)Kb&sbiZ~E=y>0fVhQc& zF_aFr+Z&)V>pEvu$uV;{j=;(lvPt`{EEFCH4DDljw5i&s0TfmO284nvwLd+?Y0vA& zPw4LEd0ff$pLE1Q)W?!@uMPpl>`nt$Lt8WOiL1;u0h0;XlMChmve$Nh<~^_q;ZD86UxEGx(XcX~}&wuB57`K~B zyDXXslaMnn$AVjnMspBIWh=B*CQCje5=_S~aC(cNxhC`Vzi?LhOseB|(&t9{i96dg z2#E4+VyzX+3-K_cSWSBe+KpZ(pN2@_dI6TWipv}aix=}YIx=z=tklQP(haive?I;E zBhm>z$QL4lWC*iA{rXji3Q)gT)3h8_$yUWw2X=BGz}26*x4c(c|9psB@g~F4*N;88 zl>zQyhU_T0I@Rp7k@QiN#C@9=3qRo`YmKlciUy9_`&kb@l0Sw3%0UosgMxRGJ(0^RLdgxa2 zpbE6MGS~~9z<@r^PWvFYvp=&e*#KQRSgc8#=AhKi zg)rs=+#-)1+%%^>(H*)Ch}(r=l`CC;x@E3zPF?GQUYDg(4?07xcCB=%biO?+$MM*E zDNT^Kfq?;>O)0WG$)$|7?7GA)Bspx+F@wJGx%e_Tz#=$@C`#J#<&e#pYE{}}Hem>7dx8nS5zW#_O-P2)9=D>+RczG*`hoGf z#=Vc(31l!ic7d4CDw@z7s=PU9HVEhkv5J7ol$0T=L)ybiw0AhwXco4C9e*1PP=4oC zfb>`}wKwf#Ak;kQTwa{X| zvC|B4Q?FX^l^4)rJp=P2yhwKHa7HkX3GE%?dT+bymbk#id;{!7*c?AP%(yZdz)lh1 z|M9DzJfuCdHcJ%>H#8mnIi-#5XvI|c+2`?Rr}Z9Dx;u0OLk~E+G8XG!v)~Hy-Ds() zaNUGA&(JtVydXTTJ_E&(21hr!;@1h6LK4ozH zC(!KIRD69>&|=`bKmgN9M~^4(9V&(S&1W%d(RkK3Wz_U=-f8mk^{5{#ocYDW(#2`HbJK2a?E3_bC>2~yD;sK+9-6`#mJ|Ir-Hk7%{*TBS3Es=(DLngeepZ8e;90+_Jht@Ob8ZVu8(^@#AkP_ z41xZ)kco~qj-xI`kJy2VK>!PC4wS{J3oB=yGFI_q0MCwd5FOl>Nu}J&I5B)#AnxFR z(>^W^P`89rcm8M3rEV$25%l37v=2+Ct1YMp7LJE6#y!ZS%-B?{eYpcFXAuF?(I%P& z#Ob48OoNE*^;N2scC(b0Ll|0ha?FY+zM0Jiy;lNq0r51z|I43)?*T1UIrFL&S8lj00Gfg)e9r~)v+jX?tznRNhN?jox2@8rx1Yz0n384%ktGWne| z<`Co0^MEoC9+=H)1(*nBR$!(~qnBN6MqHU?ylZmm+HpjDtC}(GD;mN}Xe` z{hGye5lnP3U&>fD)-Yy*A$nS%wiX^HsJr+ZQ9=SlPC&S~stH$>kFWpGl>_W_GGOwp z9V2EYRxkG$D3$api&96jXgBXm*8Z=E)`D2t$61tR>80B$!=P*$@(Y=k9-XtyCqa8b zT_Dy?<5VM)x?kJdm|4KMIx+jBgO}g5jf|RT;wAw!nW`jQ`$enZLFoX$`BJy&VUu5Q zQz+~&;UfQ2v_~>&RDS-aN5JT^C271$I+?vX5?MKg17B-D=h-gikrvxL=zW$k&J#d(CAQeai!M zPPCi`?x|~{Ec%C|a%EE4pxK=7#*N7`mN7!-XCdopNT9 zi0vtyLXWx~bZ3?_G3%pLD$6m#8D=0vo!^s)hEdh7zvbR8=~f1MCWH=6x%cdb09~6d1KnJ(w?z{Bb`1QlOkr%B7rA%hJ zjP`4fP($m+gDFMLklZ_YSspc-1*T&lSfwf~s7!7L%i32>W}3%YU#qTi==mSHFmfrN zblx=tZQ0Db&?}GVmM&d8`Qb;ep|MXlK#8zQ;$5Z0*)bc!2c*3{>C?W7tUal zHuOgCX|h;Um4~JuNXJg~I+r`R!+4Zgu=Ft0cD)(j*?zcUwGL>TOm()1W&+3g=Dnl; z`05cTK#7lgt|LonPkvkG#mC32t2aBLWg5}5r*ly)%g!;&jjuzly#!_h0yyZWZ-Dtq zn~^)s4((sQg^auQ>j2Wb1;T|TF6iZGp8jAQ=JfkuGh7*ylGi-brcsbOLXHDh)h+G3J{B)GIeOib#{++S`N@GUn^Y9P4|*Gqk_@efmD-8&36;VB_V!yabEr$N)$p#wX$2I5@sxC5tK+|f1g06dxP+_?k3NIJyJIcg zjzJ?bG>b#j4(6sz)#++@+nq9<`BRqN6XSV2f*DV zIK^`9$KX_w(H*nWfAYYPvuMH{^CBi0_=JS(4?r+?L#1b@_FIL`>))b*u@Yo2oyNud zIB1*LY-#5fb>T6lNiy^6=8u-^A(#w+3y5a|tWbekcAbT8*#XTD)RtGplM}UHr0i77 z4M<$k59Br%AuZdIA}ALAt1Mhr4xlatjVU|5$ejVu{ssuC%9|&!KL)hntJ7m>a%x#1 zpqdlrGI({Si*lVKRKV6dra~248)F5O?n^fA-Efx?D|lh1u1<0V8^K_Q>yIr!gM|gm zaG6ZIV}}=Ny&)~FTftLUU`#1tU6!FjMSet!)$E;<;H(+8*R10er+;N9B}r2{j{R0+-Kot`vJPEBhu@C zpMf`wQe`gB(GB)?v;M&kF~#|Ih9FiI{!C$d!s*}8c{M&XxF{jQ<%a|>NSjdsvnb|= zJn`6h{dvcA+rKw%IG{kD3Cl?1yV&0s=%U_WJbgLY`J?B`Z~~X|_ZHfNbGZrs-xwTY z7CC{*gRi+keQ&sMfEAun>&~K^`u^y0bU+^7UBns<4*CI6b8_**HEzDj<2O7m`fuuS zHD>A-k$_WF5dP-T8%J~skp~~+G#!8UAj-&ghJjkMAF_QE1+G6K-OJt2^>5V8#z=W( zYDIDCu(@36@4f)F)7Tl}ci?r)_d7OjUdkxj+U*BSpGo4St)RbYuIh(W9Zt{+W81Pz z#1EKGpjkZTbMh6O_IG!Ucw9Z;eS6&xn0=mkKlY>t_<|_@2UieZ2nF%RTMqiau?UWd zy-dC09Qghi2bv9pYtQ8W;-`5(U>Vw9ta(!iiqf8O_5FLR_TTRs(@l=AiGRqng8z3l zU6qYdbmT&Fsf-5aYet|Ev~!>6%FBGIa`q=YQouDV^8TP>LbSbmr&xdLMhxw*o`nqC z$c);|sn3Hgg9c`J$)&5Wfdi&@=4>$XHK4LovmNJdf5is7pb8>Y)CTQ$g~xa5eL1?e zmZ+txg?c0s`vIlt;&$a+MKAE(90<|9_X29}T{F<#ue}J~Egc)Vzcflm{`~syP1o@m@lg># zcf6NjrH1eR{_?9|gY)&S06#~zEpkL}PVxN_-%0VcHKz|)W{(a+XuoA*zWy&6_C>jY zMR=Ei=Whcm4KU8pBlf9-Ne}`WIFID3`*TNiD-^CW8q^D_PWk9C1TgaXhxYc6TO2rB zLAD6K6H%*Y^4>rUM;GTN zQ14}DrRAXBRiLFJquvX_pxz%aCk3TB2`cyukh%gV%UAxu7w?OVWswDf_I~@zmp#O$ z{V}n=77eXh_R@>>r9*UUIR}lbU8;Z(m{y3N#1oTgdqCS1Ui}oVGD2Flrj!xG`j5`> zwWN;F^5btxW5*Uq&P=ciLjFF<80c8df%=yt4=Uri2hg+xaV{4Qa+XHW04s;PLoMHR zx?a26v%3w7$^AS?;G=6xSMShuC|mVLtS!{E3g#og5u-$%A2N;jpEzUvNXDhBHH$}3 zy>Jwa%Tr~vHK8KxqleK30-0=+hT$n`g0F;&$}80~;J}EAOouv*kz>ktNw@GGGe`zg z9SrT-A?Xrc%8vSa-6YR!MpOHk9^^IoJ*I4^eUAEJZteM^tK4PC1^p2Ga1Q?e7Uk25J&jeL+^tFv8yx*N_c|+n!>C9{KC83ebrQQ#m=?% zfhs{AVgjc(yXMQk^F(KQ@5Yfra~5HQxb|{ZStg|K+}R(C_3~rc zlicBVssiOKfK+uWfy(@e*>HOZk5f0X26V)OO#2D_El7p<;tstDC9(j_FMtKofE!o7 zwYRI@ZLMqELU{jZvo)hbm$X+x`IdTrgIW^|g#j!dL6mPre|@O2o*Q}~sAQ96;Pw3+ zY_1`J1}@%rkJ=X*FWxxoK|Uv+2Y0PfCB@CIrf?5q4~ zXJEPuV%O++CbC+T_83Rbuaj5){O(qp3Y9a)sUefY(6`}};oLap^1zxvEVi0NmCev2 zxC}t~D!ODFGfgnbdph8d87;~n1k-IXSNY3-nTezv_MIf5n*lTSefc<8o-0#-&Iktd zbgw^fxh7a!f-j8O(#IWQi1&#&yYT8izVo{-sP+cC5eA>PNFOgS!IG;TPpt1sXNm*L z8`3CHF3oE+)j4`peaJE392e-LdqO<6y7+&xUs-23s`gwrN+H+Ibpzbga+b88Z{~>0m!AS>s&4ZG11xhzf-jiz z5-n~a$WWH)4)=ql^erL~a{s1fN_fMr0}F}=86w$)nXT?Xx|ZAJ)= z`OVXuEa#zPIj4VD7+Uo0NN+!Qslh{fGa)!IV1Mk0DH^x#g5~02W;-KU9LPH69vlP$ zYC&D13eQR9=ws4g8b@)>{!2lrCk~0xGRnLmrIGB}v!a`sjeu`fMqHn{g*3vVMb>QPKH}U+57C`KQL%Sf z4}(&!Ue2tXebmea?SDP^z8nF*lc7ENR)}seLgO8i(EFc%GdLoHxU;4DtnKv+g@*!IIVa>=m>_eDr~HsY`7@JXG%QXYK%-(!K-&#S~*S zfHAeC4DC52d$ljRNKbnXFAOkVI!1LZ_R{0f^FgsUUs@dO2-RMll%t(f1a|3YR&*Y> zd|*4M-U`eU6u!jdQDK3$P@OgF2r2MW4*b(gKZ9X>MqQ67q^85Htz#ur7&uOk`?+Yry}aop*eqORpVf(*E#W_!BTL5H^r$64o+KzM-m)jViwEZ&!N zaG4p~hv`4Q0ut0PLu)}--|<|4;EV*Rys_oy*nnqoH4M2R1!CMm?STf}+@%uF+eXVF z*xf>qAx>?a9%)7Lc19R_XR`Fcp=oAzXW4M20?TZ%O+qF49HcI zP6s$|RZ-9yY1vD6EtpXk@Iq}`t|lAB(q`HO?6kkM`Q~R8XpbAvPg-9U=K&N>aWa0| zrxx{NL1LjB8Vn}mp9%t6M9{J7Q-)m(O!Ev_=GCXP=jA|EsmYrr zU`{-kSr>7ILiO*nRag*-)Z0 z&iw|$d#eZM%Yp|=KyAKY$@O4aAYlDAwW2PT2AeJD;yYjtAc3h|;&yTB%Mtt&ER^l; zoQBgVYl2HVbuYbCpkjDtKN`NNTPn{VD24{G+9Lv^S8lvyXJBUTz4EkltYnA;p9&Y1 zBl(D$z9imzan7># zjBYeT9SyyUMks=nS-xseL4kJh;P9!3q^mNT$sE6%AyGbxqJv1}}&( z2AqR=(d7c|)5F{`n7-v8PEE~>hpHjN`%SeKragxsg~^@0aOrIbfE!Djq1h8+g*tav zcTo1Xzj^|qQF+CDD;l0t+)@DL%dLjSykoaTC|xk+!;cPSDz{pIElnYzp1H zMk#q%@FjZ~^3NJeC3qB10NIBDR-<7f!Jx)+jO0~TVB-P08@xd>HVh3;$bWOrRBn4pyb z0{mIF`RCu17PyOOK@j$!tzBSDRYAjv!)JXTCdYBP#i&-Y9>1? z86f)m%;DolD|9Uyp`5ZBUJF6i+J&Fdym@Qp+ zlrxkdmIF|F@qt~c*0?fm__XG*q{cE&J?_jqP)0sj90;iPIeE|@^5#*d=F0lR-7Y?h zmJ!5pnXo`3XRm$4U)n0&czHp<=lh{tR+S+(=I^Yi;1TO@XQ4-f0%|H4wX1i5aX8ip z@Y&0eG&2I&L%{P!&%Bl}xOeE^A372lqCLI=#>SY-zdQRg-o9Eg*1wU!6#9t8diJ+r z;DW&9T~Nwd1jEC_E|(9bSZQDW$u3VdlOyO5X8Y%dcwGigaES?b8(ea@t9FGK&M8Na zPp>4x!r^r=lelypye;M0bBn0&sRm~{0_vow7HJA^gov`5n7|LBK38vYzuOnjZ9N;L zCS2lIvaPX*xl*sax#nQeoT)45UtWH;JHond8mhX%L$lQ40R@OkS?YnXv&goq6^o|` z;Z_g*P0ql9P zxE_0MUmrhkg4+Z=)iuBpEeE;s04|DVU=RIMux)8zjkz%F#wi8T=eeUr9S}f+KUhh( zG_E`cX{rD*f}ebL&xMyheu0Mq9L10r?lud_O^}&V4+5C>I=Y};d}N{5!w1UekBL2a zXHwn1Z$8|Q79F7@KrJ;(jih%#Sp_+3f5BDd@bROt)b@6QpL?pUiz)n_Edy;jp4*No zJ|mxMk1+DyR-M(A>w+#F+EZH_I7>kRFQa@e7H%p4Qw~`F*a6hC(M)i2vA2p4&Rpyh z^~AG~TNuy*F>K^qT>x2dt@}2{}M{a+JfyizxzvR{+#h z{F&yRX-tc8=>YdMg7t%}Kmm&^xa~>W&!usBoAST_n{c=CRh#Y705_ym^OA|6DDB4s z4eg7pjnpDuJ)X0226ZokHG?YI014E5cx5|oNpx5B#e4Q>PtBD>c(%ZFb9Cc00S>wV zD)j7}&iB@y62R06Q)6Xlq_}~p{J8l-H5#KGOq|5iA53z_^tXmWzhQFjZd=cfIqOWDHlt4tlv^*z5!^+r zN2@HohFpD2YnMJph~}xbCTR&b6{Pn!zkcm1#|&X=net&Rwz+W+LvZ1Z9`Lzs+2BA{ zy{9O`Z#@h-&$Tf^0&WypAP1HbXSom(2M*YCX{KvW7}4ZSTx}=LQ2WpPim3&#bD1d! z>;lCweDgM95N@tF);@lNjj1TzQZnKJy-qE=2>S9i>p8s}3NJ7&h8o>ooIWD@7msRx zD>0uejE(H->oAseeDx}N8tIL~^(QmiMmeu?O)RSK++{LhC;**(uoYau6wL}^g`IdG ztnlRFNa@giI3|Ee2k>RH%f+8Ka>>VjEwGQ%>k~>wHv-lY=nhreerSa55g=wz{yL+RGQ$twJqqaL|qfbbz|GK>BAat z$%)&5cCs)-#<=!5#%dl!=k5UkhDLXA?GdMIg~3p>L3A;o79ovv2(^pY8MtHLc$~3O zSuoXh^-cX5uv+HiqmywD0y&4Pgl23uwSx)f0+K{o$-eo9Xmm;>cL2asg!YTXv-6H< zEk*AM5aXCNsykV9#+FMH3_{TF86fRX8%%x9Bn`wnwt(FN(~ibGHmVir-J-aLKzhxa z_FzZtH7IvL;C~%_{w0WQhCkf*Cgf&N%_x|_f^pb&{LWDA@qnivM9X!6BgHSP%7=d9 z0q#}NATO28XAdM>P1(Y*d%V#HqBB4pJ!n%L+qEcPUazZK$%xx^{sr_0Y2SQ>!Zhj3 zNMS{?Am*1_CLkB=z18aVz8m@zllwwuXn!A9z8 z zc|O-E(5k0y*GSnl=J5{6}J86eh7OIkFx~l|EeX#2-F* z5Q3%1=IN(7xDMv5VELOj?|_zRN+(CwGGNsNMq-2#*63Dl>a~qvK&&0u7SVOSs(l2a zg5EJLRqtK@T5ATK=sL<;+y&yO0DUx<*1YP)>H^qZ!NO*eS%YeOheHM&HvDW=NykSY z@~F$NF(&|SZ>Qc7U3-Kqa};2r0Wl(k)0_B!h&nSh!zH(RSDIQ+OhK0%)L4=t+s=i1 zZa=r*85Zr~YQD@C<06o|z_Fh5Wr zh!L0rE%)E}Q2U#s_Oby6fH?+Sy~EPKZ826F6KgG^fLDTe?YPJ-T%&;LSi|CFcC!@Z zKe<^nqVg4<~b7GqwqNEZl8QKN<+*{v344>hTxzh`QX@v1z$qNNn3y|EXc z<@6i3feE(LC0kgRMo=#o$xBU6fOUZY=W6T=Xtf`;4OpBPFn+ z&y$h{xzrgkafyOBh{8+z!HuB!(0mTe7iQB1U;=Ie9Gzv8m&^4D<{`{QEsAFF0aGut z5m?>U03KcrX0B4&i|%1fa)T8SQN;SwLSg$?^W5O0XVZfb)gO=BIvr^r24ed+6?0-Z=JSm`7T)pQB|`0dzebySHB2vp48*x*>@- zxdyc~=>%sk4f!@oyT&Vv=dU*)RgnEAq4RtJHB#-x^+kR3M4R60xy1>3B5OVgluDYn z?|{%z$(GtXjZn=wtu`M8Vr%OZT)l>yp_oEUA|Fn#iZ_8ARmOgA7& z4cE9y+9w`{>Z=>bbE>_{%!+af}GX+bt^sXC+pr(Ur zT*0Oio4{vETfqet>3gC0YMU37d~Soaf=?vh#7Aj;BtUUl{hRJK_Enc(Fgq2 z+YO^mUY{LERj#FF>rHcpF>dlSubx`}xuUGf z+;1QHWlj~0UJOpk`r9<<1E4W>^sRs3;KqyE(`{i}0Wn%Npn8_U42NLy3l<<^rUFX7 z`cOBCk}ATf!qZ>HCaR~A^~HG=f>;P#B}cHfZ%AQS)-t`gL!=!0C71ZwV_>(zdFs@g zFtJ<5wF6s~GZ+EDCl*YVE*IAk%PFWtVo-ogQy|>5*Xd#gw_g)AHIj}3W|(NE%_U5% z<6pc8g)3SAU&3bS^kW8%U8f%hSB106GSG4AMOM5Htyp_U_u7v2!RG7BL|c|#&ea+; z3L0MjQwwzF(|14hJDol}RzC!DEr_la;GZT?xFBV__DU)P2=InA##?{_&O9xp`lI zcD>Be5adTd9#a{k3fsx+!~s~tjM3n6?Wv&<9B4NP*aIs6>UHqM#8vPP5HKSGylfgw z`#g}T(ACh_Uy-A_A=G&#E{TIneu*%gcmgbTA7@$z8UX<+#u>&kRfKVuj(Dy=9IP1! zkbr%loVrjB1UR`Bh{E{L&91JED&0nG&`eqXZ2peDa@-Dx?aHsX2xr9tAtHF^QQIb$-YeFfOA$;&KIXG**1T07jXDGt{0t7XYO`Rr?7NR;7aLWg^f_?~ zj%tv^s~-$iaTnVN?E!Q!T)8VKMtjVwbTR-Z4;DraFu3i$1w#-<^OZrmLj?!g^$&vn ztUbz&^{bK{S_v1w{=u+S=!}*FXaQ8sc$GM{X>S^p_x|n0pK2dBTsEQ)@XB3W)t*zA zo|_?-H4)LA2e6tbG9AL-$|7DJB}Uw_dQ-{wgN6jGK>N zybtQqKRZZ037 zM+6-3*aM;$jc$riF7She#RT-(Ts{b)N=o~gUNr68=Zz4m%DJ&}1IUP4<21D|WN1GW zrQNH$!nkdm)ZVCTuZ_JsT49I<`7uYow83+94T|>jP#hN6^6gKaSDJ{t0xKxmqn04^ zobuUH+ZEumE`=&gc4LW9Qzl-vrID1m<)8 z1y$u}2zbfeIX$}|K7CzG#SlZw$t@5q)=b|*(9IXX=qdqgRlph9B^zRI95#kPRhuhU zqn*pVi&xQc8DL5Y{2;dIt$o{|@*Hz9wL&b?0Rr{BnTC7`QVe&Q9|9I{Y_;wK87#F>*ZASY8=gKOY zC7_{P4{#m6(5$Dt`yf2k7R;@Ts=asGVyNlhXqafd_{?UWG;{0e^}nN|MQ?3HE>jmM z0`bn+nloW97I1e%oZ8Vs#?_8D5-hzI0kQgF+L7IngJQV)99Q6GXtRM>ium#kF@sxq zHqax2#y)Q8$pMVZ+25#0B=cTsK2v&eKN2D<_9XP=lU4-B2{v9 zuIU6c=0Rh7xrD6j_OTFS4>TQ~(C9LM(2Q=ax%m)NhRe5OW^CahH;=Y+TrO$K0xT26 z3K5(;boDS_1_*dkncfQKY&pd42;Bx{WP~prwT}va_Tt74=%3wp;pEA&XD z@3}K{fSf^epP;%xmD4A@I|1QiBD0EB4Gf{}B{oi<^H9^~h@39>D*?y7_0Y$!Krp{J zj|pBAU}kc&P6crs&b|fIR=?bn23a`NJTt{yU1L6)T!uz!7uanWVA`n2H;%9n)K;1k z0Dbp3xiQ!^A0nP5IN^WhH`>>Rp!fk@ML$_NJ&h822W$o@zod|9&ZnZ5)fm-hP`BBG=KRtH< zx}C$osr`^Q4)Hq2eh%ipErB5dxbX-tESe9)msw~oSWBphvKu{wmQOJr~ z+0zC--51lrM2Z?b%dD8cc?cX(+v1)I=yk(jejqj3d8nc4?ohcCjqRY1-;V|dmO)*h zJ)lVtP-R8$Giv9b&CTV`(lWE)u^LBBknLJ>^~1&yy$vO@66!$~ zwKG>=WqA~YfeBRg+R*F@8X}8Z0Q>S?u)6K)njGuWLsb#XLtv{2uKM_+p_NzOfwI}s zmub?<)%)Li5S)(Jei#x`FvJ?O83SWE|LGo@um2j1F}AQW6t+X^U=ZmEmTWX{7(iTi zLI*^YFZ~rHHcr=Jh^_~$ZA`nNSinOY6SR*P>jvf-!!tDvW}hPVVJ?0ePvn7E)!q;p z8HAJ=W?x3K$^DEOz%`sY$|V7*Q|&!bQ&4gfxArnsM(Zn^r3<{$_2LT#A1KxP`fnVt z_S8Fg22e|{d4sr?ubu-Fc=|$eZ_qKBX&M#A^*?h&LJyRKx`Jk*Of$zfmoNc<%zuCN z83^sWPZLl)6OP|^{X;0{j)Lvx%&bi}9<(kq{6M$6$lLf#K}zrL>xXZ%Ub_>97%}?O zNzP1nAkIDAY}2M4mgPnHO_$WjfBxBT@Q28`5Ok};Ym1Y1JR#=%AT%ygZ3+j%)q@jent-BNJbFAj$HF? z;I4EtGY=SH2{98>r@Ggam*=GmoC%D$-uf$T1sHS%OG z5EzLAW7Y$W!+FefAXt&KZsXdjVT%aQ@9vjAW|Ql+SG%Nj@l_6!G^BldduX7=X;kds zecX(iYBz!N2CcpKX>nnOHGHC>HQy(B#niKOA=VSC%F*|!^HMd@Ql1nmrgQ?Ol={ChkKgCq&Fa>}( z1lPC5CHsN2Pa8Adb&m^zYXgf5J zC&$p<05bo}+duPU=#}5Hoh{Lb-*oX|h>g(dqo(Qe$Kzog(0dx8(KV}L*~0tG=PyVj zb!ztrG|p-yY8IT;v=MY#8i*Ak;Qqww+jaG~+9!63#~0NKN|`~x77^NOL#WoE4v;Di zXjUMB3zMzKp6=#;xR=MKf@+<>!eWDZxPz!|C8&d!=4v*gbZke6<5(+q!S0QAmMOj0 z&Pvi==W_DZw7c% z6AX)Btl8}ex^M)H`;j0zx+9BMt>!#|Gw*UydmK7O0~J>$m`yS;FXP7FaY2&kD7sq=YV+o-7A(M`)H zx58EnVtfEjgjK-wa8hM7zdH5RD}0%>%z_5S*b6Mq_=>WTm(uwTD`3UE-|!T={qj$^ zJ$Z3@%xfLp+M|S$)#b%}{GoYU(dL;C>n{qS#v2_6ri=4=G`I4lbH4?jO>Ly#bNG{AJzsugb^ zfFH9Epi@6y>}UN6SOoyHv$?hY^qbO*6k$7tr*%VJ zHn(Ev44{%wi-OpKXf-X5y`Jaahjo|9nLKsq$~{aQ;Qv(EZQ9HMEkHpao8Y>w5JTr4 z2ag8TMcX(gF~F`618$rS*Uhb)FB5>j)6m};3=KDConZVseSfFx{r6J-%Z~i-sQg!w z{l63J|K$YBtJb%#_5Ta-#T)Qn_R0@9A%c4{P~wve-l1;e!3`Ea;8F5&rga%)v-WYC z@b9@l@u~53*4nL5-DC=W;N$1R0*hO48SUree!!~!Ylq~8@`c`+Uy<85|9~G#z;OaM zL$ho1(hqqUK91*|kKX2;a7WP(_<;f(x67-&H7Ut1DR{okI@wEj9l`zn>w ze+T<-M;)#d{J$OS!V29He4jFP{)zY)-Lg^P+leUM@V{Pv@u$)l-|&l~d=shtWur73 zTD|GLK$jeABLcjOZzJ4uqD=I#x|@n#88wA)v8S4C_DOn$9=W zJ5mZ_w?l?VA9xYp__>k++9U7z7HU=6oZ9ubOcUhGa@EC`=6{Xh3r~vio2Ty89?SmG ziyIG8e`G1pZM8dXp>Sq#Yir%Di}HT5Z^1)Cj!1_0J1^a(|@Vh ziEWwdy?odMe1-2J1j;F2y{?vV{QhvgkMFqll}VFtf2_5qwpeM;W>$a8a8#!;lNzi{ z^H?Y?o{ma#$o$J2YW;ltn~YSu>cT$=1b_xW+3k@9OGUk-*3t`}%QwmR{;+P~-<`A% zTe*p;U+>usZL8jL%t_<@ihq0f@^z+^4#%}LrjWp1*Pw6;I&xBUHFk0gdf8ejcq#w1 z0OW#r=dJzD*i4-W5U}m@UxJ^0OPIu-VM+H%oY6qJNWWl;W<$L{PZvUAPPYOW-8Fj1hX4amrYZdwSy(jtqAz$ zm0y5!*9foOy*Mx=R-N}onnA}ekC-tk!wi6q$=PmRn`v0!4HoAgJgIlaL*wNAJQ+Z> zk6h-TJPOU!+Ir9seEaLy9oSQzGT+}Vrc4$^r|3JQJ1v!$&0ApV_EQT{F_xniy2jG1 zMUAFswa>!(+L7Xs9BLbYS-+&>$!~iltrp|O@>)@yNC&HGP@1GhAxft8C zxobR0{gq(hqenwk8KF<0yq}HMN6$=@LwtT8*Fp_on#`a4y)>rk)70igXY!Pd$j#>F zxt`tis+w2)JYnr))-XI9!AM^Gk#1R|uTv)5cl+xmro8qiX-MUYt!f4*?>_r9uQ$RJ z1DGd?9MLBB(#u=K25#@T{^TMwvbZ^M_g8;A$b$;_g7WM4f!lUF)rl=3Jz|GSQ3i?= z0W7)hwYX~!LeMiO?%%u2&2VZJ3}_Vrt_(G{u}a%%3~-?XKEC(dQA>_IG^!sgN@Hyd zl-vp~Vv8M@)o|@YW=|DJppt`$1$2u@6=4s-z$JL>?DVo{X{SpEc!V=pe+XCwXzBVP z?Z0MFE7ce^+VVSK*#^>~e2%tCvLLgrFuj)>ExG7?m>-c&@sG^1HK7nd`t=} zTtG)9pL&)>!Rm_^>qed{4V9iFSD%R~Zt>F|&%vnirp=Rl2EYaxp^WY}a3)s&=< zLnK9Xft%97RQ--?oTKCB?{qUK=?6jVmTr}LIl2wXGH)h;qY;b;L1`Mz^xL>jZSZ1eyJ-~=Ox zAp<;4y`1C3SwO&^>bwOA?ZNkuj=5hxD;w-#Pu zh@Ns}EP6paNzDWr8%96vJFZ4)>YO8S-aH#%*a7XQc|`^HICpP8gg59D&M~>~nDRLq z3}sOdm#c1diRYfZTBbnhKh5#_2)ClIq$@z%^oF#DM9yfzgdqg@bK}WZ)HJ7w< zLYudPsYBFaL)^B(7MVDF{*})ULW`&ctKe}e)~du{0C-vL!#f;}8!vMkp!)!xGQLH0 zGl*PKRdzDqgONN-kSSss!V?s|80t3*882pVZXPif%*0_3kJHfu&4nHVe78H#Gg;RE zzbIB7p=Q@-S)fA3eak%6e9S|8u$gTI?=k?(?>6|NU6&-rGfWuz%bT^q9>DdR``&_3 zD;Ha+2)Zmptqy9KSXUq>mgB(Hu=C7JfEC$iDp2}D@IN1a`+W`>Vc|WsPy4BZ-Dp+y zw6;Q|fC@kY#zQa;G1>9yU7W9=GJ>9O2B~u=ZnAMA;4E;Q+8nZ zp_i|o5hM5#-MLws+6T(pX#{mu`2<4@nMohlODDy+RD$<1ifN1np!~fD>Ur(w?HBu< zRlK)re*)X?%yi^kruxLA;Kb42ft!O^vQ6{OqY#@wYA{~aF&2zh+lV$aEOh=<$L}~=z5GC42FPL2i!z#aa0PL ztI;Z0f!d7_7UoT_97vU=lozy_WA0KLD3|BnutUYKwZPTI#xiQU(vEh9>5M6d(W>2+ z3=S+B!8r2zf5f#rw)BB(KiLqVy>SbiwDJ)9NB1|Jeif>bsWHvHlY21dWdU*k0amqO zYan;+Tb#fG?->~Q})g~d6(;}D)oWL9-wms#xB-)9cYjF1bCwP(xM8pO6xdS&;Qh#uc4S) z(Nkb~AVC!EIX+zQpPqkk*SNXfvr+l@ICW%8Ptt%XZd#f zeaECKL2OJ4ph|=t1ox{n2Q_JxSTJ2x<63VvxT9GU%WyNCTtgEG;OT;kA2N5H62QEf zl>%<8476p&Osw|I2I#KFZ167a<2Ytod7ue!Xp>m~BZS)N(XlA~e?D?p6yXV*m)kGA zEjnLi0ELAjc>>EYpP>`jW-+66SO=Tcc5qn$>r}^hkV+Ybr+q!^p}W{GZwqr6Uk#|u z%$B2rMvYK-TwNxZk-L8Mwkg)14u%>q+dS~;UDlyqP%A8M{IcN0YXemiJgJHfX421s zfEv&?u4c&1z?jpu4D>D#<5wGA%PmTKB>>4p1h`hg^35Z<-lhFs60?RiD&S?ZQ+b;P zy1P{l4Wn`5{&}&27bN`Qt-D!Lp;um{OPfJC?Ky57i&hcVSN0!&bS3kDd~g))+=*&B z3z+L_Ut6nUWvRnWd(HAPbKP<00`hwfSRT>gcmSN7yt30yXkhO z+$ZmVsgS8W%7z7SV5WxWbyPxdJJ%lQ>gb;W)sibPo16*x_a}dAgsNjf*VN=jVd(GE z-qlH$_OnGC4k!U>PZW=tU@CDBG=Lm=bJzwFT>WsxEQP)qpxqy&9L?BHg;l8(8Al5U z24$Oh)dluIF_i!sJaac_Bh*Qph#*+|T9-Qw9w2VVz?pkQZsc@isTlgYMRH49TJ7m^ zGn}}GyREOkn-xTPm{;p4Z-Rm@MkhpczHm5q594&}*PrvgGR_+z5ExW6&^KENje|ye zFhj$+-JHD(EQE39xiy>tb6zmJ(^V}+{+Tokvwb;OydjXM&^cvEV2n;DTj?!eKqey^ z8o=Oo6oXACa<9tPUbjKv$xh5siS`38+D;eNxjPk`9Y4rB>bCA!T0{c!t(dN=8Bf;808?ie|S{1n)tY8PKp~mNUT#dV&f-TS3bp!KZgvr{7>D zJA%fTVx2$! zRBo(`O0cY{H9sRgvC4$trF!J39?wE+X!b7SQT-9epIja0FdH$qqbH_8d*n7#J&;M~ zY}zK*UypMq+Nyl9;ugiLdjI2rCl(U?`{^uTQ$lSbM5%S@iTi7q%=^S%P*&BcJtPjz zYB8O8U=d2K`Ht{jNCxwh17J)62GGttv26_!C}e4#{E5NJyVf-tDg)`IoMj1}yOFQU z0yww6w01Dw{+L<-iv(C%ni(`gnnHQ*-g%}qL%qkvESw{kaRBZ21UxHwu#N=|HXhhK zs9PMMapHa3ue@q#gi^)9Oxu~08Jbog%LbDp(LDZBFu|f-C=UQSLeC~NqP=o?kt$b;W5A$zu-D_^z-t& zq{$J+w&-0<$&o!OBcLpk8J+@weN zk}I>5=X~;f9C!;S03?Q&umi~1eDMG$tUMbfR-Y1)og%P*<)yC2z*(`RVnEt&P-vS-Ez=m-1affQ-Mduu#dDl3x3=POwTVC%#3-Mj#i1NL za_M!$X_in_iZ+`P zB2ebW?s4wB>p#=MwJn#q8+cJ*q(vln59rFh()8s#xm~0$6}Kc(x$!F4Fvu1JtbdN1 zNy-_H8NfV1-RnOQfBY9r;~v*g_nXgKtu^GcWf&_R^MqXf{8)%MBL@$mljWAEq&QFuW5EEprR zRcyi+dRQx*;iVVnIp){*9RSDwkHG~a5LSUjMb6dIW;P9h2Gt(r*V{Q2w2vFlWHlEs z0NMzi@8y^Ue(`xQJCJG7fvadTdb5lH1CzH5Ho}Vl%FpTs!Q(b!C6tAcs(oNo?B*g? zLyB9XTGhVk5h=zgS3gl0mYx*R_jy6`-Uo0A8?pZA%dxMIybOCZ2WNf(%cE_@)p0wy zRXBh8vQ-mgA0ng_Y(GU7F$M9@YA?3C8=*Xh1=PTsUl%&*s|J3X}#Ssg5PK^_EG z?-jm1%3}};>tZsjjP@e!4TPa`_^Z3aohQfZwI|SCd8>k5AT{$nP`_l-`-6ZY4$jy8 zKqzR(`ZrnI&~S0AzVu_q90p$7pCKJvz^p2&%RLjh!3d&s%~Sj1->!mGSc2BisdJu@ z1NL$vtRUvqTtgNer~Mge?2jM31U)n!Ob|4t0?7#jURNG4E+(o)SiqpV9l77_0BKKp zw}euts%pFTVrn}r8DKz3*;sC4aZf6Tbol0(Cm^f~YmObMg%St44O}+R%#;(1i%_nk z(*fpLzo-o&Hi-ad3aG>SSD>o|)cb3CnN{3u1$1fJ?k|7GGE2Dl9(^sy3s}D&m%`xQ z4dz$eqV62peVDVH_EB@@sDHTUS#Y3ZFsFLCA{d&ra?PA{2p{5p!<|G>AcBW;4>boh z40<~oA-NK^LqI?W`-lAUTQgggV|FbyXz$}IUPfDRZ#8w#fuUBQqn z(U(A__DU8CYmM9DYFLl<9KtH-&7gSgfAPDQwd6UO)e!-S$lLXHAZ3NdS@PnAN%j4$7 z$?eou$7L<-;|3Y)YTE$;us)Q#*$wz_2D4Ea;{F=w`NEHH+#chbbImn(T5G1g&pG?-&9P4!z33WF^!0nv!1eUG>c|`4 z9$N6+-MV}?Iw#6L{{R{V*p-GstxFh_Z`>^iobK%fHGz(Dwyeg$_RM%9 znq!~@klx`5Psr-^k!XQj`J*tTm{X!z8SoABq@Mv6at;k(fvW=*PSCgq$LfwcqXlZR z!ryrjZR|S_fiH^^yrL=?`~)49L$?ZAmuPuLxnsk~^2pREq>-+VEk|+GRpRq|p=Wj# zoqppNXx`JChcFsKL0KT+^k}AS_EAR6Xnt<#OOGuI+Aug zEsK(3Mss&Mc{B!D{Ulgm6U|De(X8PNU7%1sE}I?;VBF<2fHe$+l%+>)(I^~~uBecx zX$DP!1ik<3!G|7&=mjcm1FNlY0FV2ZM}FPW3E?op?26`tIUE5;GsnpZOtpr191;fu z@Qx#qUb#D5{Y#l{Y@Slj?>X|pw?YWA(5&NA*0U$=7b|4@gJvr6)eUiTfAO9J3M6F>j!zBg7~IQS!NL;8QPIdWImkcN72J)EG%zq{h=vd%QV54D;PGv^CSy>im)`-M zAHtQ*AWxl%5SiAE(G%e63&yt}gJiWDO{P9yMXP^=paZlJ>55mwzmYWg1l8I&ntY=1g zA|Rb(4%_4$bR{fkg)F#mcOR|D*#rN2&-EvG2FR1{Mps9uk85(eX%GEZz5zx9g)jHe zIKV83a~D{yc0T>xTyE#2p%&rQCk`vD$&`Fp4Y~0oB0xKg zvWln={%H4i8R7uLr26n5(a`5e3qb}reI9F;(Fg>nyV~Eqq3HZWdhTXfFv?s+9{hh= zqG^jIQ4p_lN_w6gqhJ8dZm-Cr(&VA}QoJugp`&5TM6Z|1is1erZDgmAdDe5m0S&-R z)K{3CdSQI6sV`iXbJ~EXo+QL1JtpqE5ya3q3Zr{w`28&C;H!`1~a#%1_9Kq{3b;OgH4((_y3 z^tEZZPY&z7TQ)!XnYBE1^~t(i#B=>&a99kTwbnsxa2>$LN;-RtKHYm5tSdmFGy#Vf zLFOBAM7196Tg@%rn$e5uq@WIpBOL}6^aUS~E zj(UVGP%XsffCtfR(CereK`%M6fK2`YI$s&WNK+>!CiPC>)-dLJ30#Uoba`zTAoG|byvqV4*qc{Y_ zGhF!YDR6GdInc@nAfQ+p;hxL`9^&#%=YMW2&{snyQDka!h3_Vm8mllC-uzJ*kLWha z;~6O6F-@DS7=Q~4@D(t-G#{~rE(p~2>XYDemb=@!j8#|NMWju#M5IR;$E*jhJ}Nqc zPmAO&&3k$>I)9!Opmxdk76MWOpnu!q%6uW(KWYxDGtIV=#<3mC>NoPW6(B;6&0O1a9m{J3m?7#LIfm8 z<5^WDf`hNLbbU5RpLsce9bGeIh1M!<{mVB;YfZtwe)9)ap3(tl;0b~_Hyd46UVem= z)Ey6q>fxpzSv_vj;m?Sse+1<;7EmaS{o6OcbHhXsItR3<<>HZ9F*r{%ov;IovH=4; zSfFb~2IrpF^n|lY9bBAOxEnr7cM^P$(Z-;xWBN8SL;)Re%-eVG$6CeQ#&PKS%oP8-k-q`_j0eIivo?F8%z~$@pq7$ZMbr)_jnqZt4L-^rHGJWL~TK2NAjlx-iio zKuWT-wAt>iL%jRh2N{3+_6zS_&!l6V2zbUMnQ1VyrbzsQ7vg(2(q(!skd{rK{d!fW zcAnpW`|c?RhW(#!eE#E`(K>*DQ>?F+R6P5RBjRm$U&5Z7h zNb6`?KpoL`#v%xl&i2yX-pjDt-W{K&*#&3NR39yzZ2tU7L$^WmG>(BWs9(K}vCrh# zPiOhkLy5Qiz6BKnVD6KvD`{XA9m8d_5P_p3)D6{gbCwK_Gukc62@MU0PpCUO+Uj^b zAT2|^-z+2P&F8&1YHmgQ>N5o-1_*s4P>FQnSd#P@7&<5VA&wVvDu8D)7$Zq1EhG~t zBZlK-#WgULV)^4SRQfP|BLl8o-Y04+SUdq|X?i(tf6fI^|F5>#AAO1|3q2mqXgxqN zAkWL+(uFBgJH(4t2-~EQT((?}quE;p;}mj02Ou4#J(XGLs2h>dc5%rH$iR?Di_+BT zIem6ROBpmtTTCT4Sq9GJZIIpt24-AF5Z%CFVBvQModyMh99i-8UN3zgGGMCTt}*p{ z3`6uf3HFzpACd*&)6?84gIo35`!jSH{;X=d`|;p{0%?5cuXYeG+T+y|{2>(3DE**p zP{}Z@ec^733oftUfcE1%tfv>+)6D(S!NWANfiv&@7&6TmKi*B_F+XjmL%`=iKti~* zIs`v?=zz&1@SN6xj&AUR-vpXZUZ^ecop6GrgK^&VrKdNz3G)ees~^#!hu+kwYVf_P zQX1*y>q5D+$@pysRO6Yq9-j53odNaF$(g*~4cgpRUQ$3~ICX=4`fa+cge;Cb(Blko zg$yKLeBh05qh-~wxo}haLB}Zf+U`NC0vt_|hz4KkNqgg2^j3;RwTGs$9IQ;e2@--^ zI)|)l>RhXJ2RfCokmjdn0lLXR0W&OtMS}ia=vMa_sPA**E1}o`g{dEKZ2AV7c%7gQ z&|+^BI1nohxON0%hfxV|AU(z23@nDP8|ba)wG}^j2t5HE#pOVBGguIax@qPZ{eAu> zFksh1+BaU9ZAr6*yt%3upwX7*d;mRjFX?TgiSh+wVA3t>*Yjgs*RL;>Gk^c?{+f%= zjjThYG3M6#7VVGcK=PhFvRr6+w$OziBs1avTZ}+!tFDijIe-_-Bdw7SDg$}%f&pw| zA_{YuxEIn|}gy|Il(orzmQPqc@ zi5fGPRe=Cq?fYqHAiGi8lJb2!k%YIQY0`mbhTJP^Zj}rgzwkoeM?V)G&I~D#Ub|7c zHPzD6_6FJxAa)sDPmvOvK77i%dlDVSTuyAHtxH3Noi?MxKkp9GAxs-uAwJ=2Y)Rue z;bL;3J;&p#A1{_?5Wq#6(_xfPeAUr%F4&;5qP920x25_k&QVzan{C+foHHa!NH`cFG4!h7-XJ0Gruvt|_?WKFh zay!1xgt6JUE-%3{c<`ow9!Ota+U#n4NXDi$abKsPVgt@KdafUQB>n3Y)Qm(Ve2Z(Z zzL@!nE}~ZrjJxpc_!YM_w)!9Ee#M}M#s2^7+-&ih3#gZ_ z>uXVT4|sVJ4yp$MouH<%L7m>{szy%xzz^gi5Zxv7L z3-8EdfiC*Ft+K|UA2gugY%SmH_6{b>D&YDu4N^I1#n3}|??&~KJ5lkXgSy+)KSHF# zYfSAENP1wL&8XEzvL^V9gZgk0rmrs#4yvGB1r>tywvYgtwXunls{!c?$lWgK6mkIx3vC{x3BT^>7Rm%3nPw%t+#XRf`~Fh^yZ0%73loU z9H#mJ|DHaqu@d77=;<@{jPQe>dKH`KMi0>P!ZL<>t7*}rMfAB#Pu3TjiO>trIIFh_ z=|Q}GKeUJ8*EgJ5YA9Oq$k~cPegrgNGNdDh)O7I~Ll(gj*wp9p<0)YAg!CfN@R}aT zGMPmH=jV+^qc=FX1)0~{%&5Z{qSrWrD4^^TZn6{c3Qij?fa)q_ z0s9AbUHa-X6TMJbz+83%F(P!>4K0~MWFh#BNF3qNrE7Rzi4slm2p4hG@t}DoAuy-i z;lm@%|2JdXi#2`8(mZ~%8;#dK$k__C<2WE@;s`$=EZc)cTZ5WM)V76Wpl_s*69=5= zAIeP3i}!&6x!j*1l4jWa!|%bZdu2*>scD zeH)}YQ3&_8e7LemmbUGVK2t2qRJqMf6 zAq!bro4Z6dI9;CxI{(x$@bEcfK@g-RqsNpA?JNaPH~`bmi12}2020hGNI$wqS8CCP zagXy}VA*MNuIW1vn%;XsmI?Q%>tYDCS*Mn6Eam~r-+B=|m1F6lS64?T!CM)6OdYc| zg?V6mr-Tj0IpdC1pObC?(RuP!`5Ql_iMv4&JAeMs9fzK`2^F;*C#Al;AQbb>+5A+sZ3u9{o|%9+;;jWJ)a z6Cfa{yM_M9tve1bH+Vo`hz^(>z=1Mp!09am`BOnWRAWC&c+a788%K0Bi31+h^Dkxy z)2xjkom}p$4fYuqu0{j6L`ge9vs=DUhL{if{Q5{*f*}S>(FEKHBY-mj=m219OAAbb ziK^S!DnY254E@ z?J9ID{Nr$nnQ&>a9-B7AZl9Uyuld>Tzepmf1G z#TfN`je&kc>FfXIU-lcSf({8lkQg)IkV%wz!-^+jHVd?CT z7jzivD}+Spuc2Z65=5ZUgEe$Izqab3@f;L2hb!2lB?U$OY8LY&hUN;vFAR#Epu<|m z`i{DPv4XSwkH_x^?{St@jxnJDrXzQ{=zrqFN6~I*ze2RUYCfhtAx(!Sv|FH3kj@%N zfHF$qE4R6rvmFB5HlO65K3G>$BT=*4II>p;PbGtLtgS!oS|~qU<45uizwITZSa9$x(FmgL&-E z9uIeikPhN7X{K17I)G*ntEZp^P4-w_L!F_~dqqEUCeqfSbUd{3{B!7?k~Pw~MT&c> z^irmIm>ubWw;ktVJu}fBF{5=G?Jqmvd}iKw65Ja2T>T1!p|8~}EP{TPt|8D-5bIXl z0vKaaz*9I2%4km)_^Z!!S$Ev@FTea=8k9ahv*0|nRFs0f|4i~U8f(;>mv$qzxuRg| zrDtOy>B!Y_aJ&pEMk=5Ox8^O~gKlYGxeaX?=#}SnmL*+!m~~-|RK$6C=wxQbPBN@~K!6TxrhA8Ibk##A2lVp2qCNXK+EtMU z|MkgdAhG&P*ocaLVoHQC{);?vMQiX?T7Z6J$e2JI!`v-T55Od~dbhpg(|1DmW!^9qhh*dH+z7ypgDoGMOs`N z(1`(2H_zf<#Do{ny>ts>`rO@M+PT#nLoEm}gY=<90iJ02tI75UzaH+QJJr{X)=_5g zL;^TRfiWKaYU*atR5BbYb;_qO$m>7=eWTO8wVfHLUEVbbd4+uw7|;kRTADp){2#PzU9$07R+NajdbEe0y;PyAoca1 zgVXZ^W5wV!s7|FG_2yh|&65MC2kg&RivXVNKNR7^7>fxNoeoTNI8k6Qj<(+qztXbm zFArG4K}5TO(F;0M<_mRVK?P_FL8AMrGQr-tD*^n!LcIumT-}g&AWJ#u&()uE@(-`x za3sT={o;%Fd7uc((fVnH>JXQT3W zvrkskF^JCIXTKKJ`3I~8=?qk1c7?J)c%n4#J-}o$U>Ob+UwrEUooMJV80b3zLs8Je zII@OAz-xC-Far<6j7~f#EA-UoccWd(8VhqAfk3mUkLzfs)~!21Hx;xO>b;?dgPwZ+ zo(Bh1!cb#I1DAgVcF;Sc9)Oo3I^hdnzODPM@AcAAhtWud<&J-du5N3xIL94rd|bHn z+@rW(b&3_mF*M16(@`EitLkh?`$evUc~dw=gP8ZgdO$58^`z=hzr?B{Z5euRjb8Nu zaV>y_1gHj8>Va-ucoDtAZdsx&p`$UB$1|Yaam05<=btAQ1+y~5dF~tuU?E92gEeu2 zrscogdkb1YbPJ2SAQMf`j=t{h5S{a`H|u-J%lCGNqSeDZ0@0r8x4+?+hBg@l^v{E> z9R*|A2D*Ai)CIH)HxT%7G|4$7&0}cQhtXMM8HdfIF_@XBjj#U<+(21I7i2v?Mif9> zW`W{17#028T9D>ZOXK&s)5s=}pWO~;KobBo8RJ<`lf92`0|)4teNZ6xZ6I~p)e9Yt z0O&q{7x-rE=>NXv>4!xHo@NqJk8Md8CsC$z!p)!kz8=kZGrc<6t@n258dSa7C<6;i zK`i@ynIGTCkOJ!2T4%UFAZqXtj@dEH=!>A!{md4+l}Bg|`mDB)Ri2(I_~Un;K*Jh4 z0s?q?%K_$$UQ8dKbeLm;rUFOIGQFM<+Xk8lB58mT|PpjiMRW7G^}Q_k-=O$pTeI)b|l< zwY?iFv(YD^r#JH%a$9i)0kWRuyQBo`WYsUcvh?&W+F6ED^DOe2qL{0| zl#-!68BhczP2IW#77YS)agk0Onp1P!pP&QH$w+QOGwX3d;yf9(baNfZ0F?(u&k(rv zmwz5<(?;bIAXS3GMm&!7AQh5&?P*=`3|mi;yJEVO~r_0O}(|#?=45@s{6; zg0>R4qIu)4(?*{u#S8z6J-R5M8jKDTFb04k7Uu+zHfnxp9}6n*+AZ8Rqz{Z541?cl32( zq!q<5y88SE#&g@a8S9PT#3HSg9u5XRUfL>aQ1YV19b%3$li5 zUCWS2CGOIyB;r853I+S%DlovBV$x4Pt4q1|9mER4OdCl>p4RPXaIUD0i?5w#^2I|@Tp^fY72(=Lk5oRn!pQP zn)BqQv6gK*oGVqBy1{|qSZ-H8Y<4oVb!ex-)UT=;5C9!%kP1X$yHe6O3GtM4n{5|Qa3>w0 zs^865(i_zbXK_<&d)1+V|C69TIpump0P316-3-Y+>|=P($ck=(F}MWM|a zIp}q@FeFMrbhwTqNIVtz4g@y5{LBaUKr~R)2XT$X=`d)AHISR3^l4*ms-MJpZUj04 z#|K1W=rypKxJ`U?H5&PQ)UN!1fIh9~zqPVK14Q(g*1Z@UdQ$tWIwMw>GtjO8ec!`E zQ3GOf5AT5isZ7|;|Gf5bG;4e6eKm~HV7r%YZt2g;rb9S8?Zdx3cIQexy6YwRW%xbk z$eD8ws>0wp@$z?|9t7Z=hvv4(;LwQ=c_|in|jqjcCwOkoI?q zEZLC;+)w?=cj;QMz&~I2@ZJJ6oIFS4Oy|-M@_`aU0Vj~>ZwaAX8cT;vOI4a0+6e}4 zFKHc2-4V{pfoDB}*J*$Cg*khm3#b#x+DkW7es+r}!1wJA8u5#HyC~FUSr=DFjF*dU zBYQ5UR7CJT9+G=Dw@-_bL708+J2Nvn<<7c@LU`1-jw=Yee(u&!j!fM-prG z@!A>508a^@+!^Gs%xefBR2uMMkuqbl87`+2ior*y=P>Q8g$q4~xmJAobVnX~vkaV$ z2r$h0N*`B-vcmBIOS&A;v8CQ%>K#mI1t-wxpr+Ra zGo{I#s)2vI{eH;S;h}TFZjF0CN$(06a)9SBk1xo5_nS10fOV8vYk;xe;-)tfEzbr# zX#u$v1Bj;Y2bQ z0dzLt_r*7E+Tc`tX(sF8&{1zDq8Cqry1U7P0VLl0mnxc0mRz91a%;Gy z96+~v;a+f>+-C-?r#BHCsGKVjLAV(W?cYXo;psJ^1>b65JP6dwuO@rmc#69N(s=y; zc;bQnws;7i{btnahSN}#aAH7m5EzHJ0BQjNdcK{RFFh;@;j|p!73Cp94eQ0!kh-6M zo}sbKARyl_y9W(@5BPBp%ObP9aitIMmwvg6 zdUMg)XQOBf!BI4e>cxf}v$YbM$9awytZOxkcS~L)_#P+*lm$xZ3RgeA?X0qRX{=MhFc=UGHEzhaa4Ox(f zra(mdPM(he?ge1-1TePgznR@`D>JH)nPbElIxHmEgdU(KfA?hnXFjLa%mwJ(Gv#tm%&g z5U`gwB!ci#G~E7}bSWC(K7njQVarfsgbBDu#u~6BO)0~L;}y_WeIOcU#vvIz;fvqi z+3y@-EhW&E2hamRUUM2bh)D%>fZVG~^fC0fb?CIxdWYc=wDrmzGqgVsNqeu3#)2>T zf-Qi|{b{@cedj(fU|gD0Xr}x`24}?W+hy*rGG)1pj!Yz@9S1;%(Vc7b{`Auy^|YWD z^vYABo3TL~!!#7s6)h|bt)&J=a8|7xqk8-bS`Te}h+N(AcnZ^+=E`)NI^s85MnePY zmoyQ-WI(7`CxEH3k|s3UKI$o>!O^0%++DXKUGS9!T3W!!V)Sm6M9qB2vAcs#I1GRC z0@}l1vI1T-2H?1s^y#BCii`AxU;It~8J!sFR5yHzF+(%``IZ;b(F9e4rH)(cMLvhs z72p+Eix&99)nFamO1<_5Ezegv*C7GDItCzad#{eSazy98>3 zzfaE$8%r1j4E+(%ex^Hp4e0U^X8-2%q5ya8;|LNK%*ALNKK-e;!zKhe2dH@MyIyF5 zy&S+hm>nn8SFjKYL1vJy%57dc)8?Q%00Hvn6p@*54cvw`jh0~e1GPNTijF#nQJ~CG z<`BSrW#*5A(;#(WU;?TDx=PPU6yRl`i2ZG&KP{5UP;gfa#~nk{6baE+{m^1MXFP*Q zJ@*_7cHhu0t5jKg$%;EP<7Sinu>AG%NbFMOJ#(_34V0rs_!xmIQ zJe|CJJD9Gd>P_t~=sR4xl^F}fyMfi8|3-myr-%F0)Ad;@!%Sc`!|o$I`Wjd01OdmT zVdwykL5bO%6QBcFI=<3?SJ(Q-6`{@G4pV4JPmrJz4fn6>7bHfg0W&14tu8+lkA6j( z&HU=DF1nW=fM&}%iB3N0e;QnHNnM?)wBqg!@OZ$?j4gOqYoT;mI|fCZLBrS&0d3LX z0?wvClP+dG@u@D>Q|9Wd9$ldMHG7}e@49?7#ErsW@Q*n-fV-)W)WwpXdF$>r7JOiz zNh|PQYje~|9Wjwai97wv&CfC5HtITLt;3u~Yhf|utU^K6cCK&Tina*^sCRaAdA9Vy z2Sl<*LqmW1EZDh6v|~^9PHr$|X|~$wIu~9Q0C^xr*=sjt>ySq$;uILL%56PB8)?&V z+@N|w{oR6L1~k@!{rSnCgR|x^iWcj7%Q&;mBNKGzdv}Aq{u3~OmQvNW)(=vq4l#)N zARs^*;o)ZjA)M-~vKGml!GU?`8jRIeOb~0M;{06?Jx#OqOtalFj=v&mQVmd6cSXbW z;*f#XqQBkq)ZZU`NDqRC=I{W=0PUxea&9xQ69MiP#VF$Tf)+GkTU=3BC1VMMrBN`3 z*z1p}kK@Yhtm6x_U_h@lOq+eshU@FsZi}jaErHp|P}eT#9G4}o7rxoe%3uO%KlA(x zY)E79Hb@*T@X|xlo{(ml!|n%_GDov?;gDs2x%vkpFd!X^-STM}Z_suP$2hD92(cB! zp#%LKES>2neRaPWM2^fRVUO9sDRbOBY*9ct(+Q~JtXwNkGp@gQ=2}tGoIze-Zm(P~ z8gOf+Y1dgb(41W*i}R!@F;-sU1c3q`QJ{m0!?#?PGrK>0Mix^)keo7thMruYYq*mH zke@yb#87Q3A6C>$*XnfTG(ET?XuCEE{1lzh8$K~U>i_Az44fk%fJSAWwo7+T8t}zk zFvbc%Q_aQfEDv*ldfbrImvr7O0lfjuO!vOhGH~XY0hY^s8hc(lx({-*u{fO`3qJ$K z0M5us1~-EQRn}eUND9OP<4-^fOJ^8N+Ha7)6gB8KrGu@5t^mg%=ypcSd2)gtc{Htn zj>}})$$#h!z5N12vr{66JKkmn@0!V=q3Q5r+#5Rr=Uo~l1N64mSdFU(&A~C=HcH#Z z8_){Y&~S-g-+CAtI*WS({6SsgHsBw;I0jDV366jCun0+MjUr6a>m#M-??A&a0D7mu zd_=aq|BoI3XuIxyCSMrni^pCBr#IX|0-8w&FMXK?It3~`z!f*zCXgW6??m^q`ZbA% z3)|$xLRTBOJrH3k1^3dKam;nWJjj{oG)?3AmOCrjKM^d>|$#j&tKVnbwV^5qPK$eE^8SW-8cNNnf~h zSs2pka09F5z60E%eY&T5oGS|k(Qp9v>JEDdj_J(az&QGgWg{}QG7}Au>NzX|IQK7T zP2*r_!jtFH5)j`!%XQ8!)-?j5TBjRna-c?PGa{_5lmAzBtN%Rav< zOFGqfilmYth$sDf!C4g8ahgdyHA)YBz<2^CPCL5~ z<-L0g#1W7aC^0Z26^#Ksf-0s4eG4!oSeJ{Xdmv3y-1?`t7Sc!Z9nyF=L>|m6%loH7 zcnsZPNm>_~rIjWCSYy@#L|kpt?6*|teW2t!$*V?pb- zGxK+~`|OXcGGhgWOe;>Xo72-pUKZu(kdKZ~f0P3qZZAJrP=yW?(ST=xa`Kqh>*@iMw7R1cBabGvMiM%l4dAvVt#A{X`mK(|G$S?%u9u4?bZryAqdo;YnDY0m*)I$;vw+7rd~(KY16ndAi!Ok+7cZ)mDyt@U+2M8 z>e+SPv`Y_@)AF{FulBSiY|x5_VLin-;H-eJwjsle6ZleUSsqdMm3Aa68GJU~F8k{| z5i7R69?$CwV*)pR`8iwv{3(Ua#d1T{h{N>P*{s1Bs0c+#+PPodT$`4smSGRhZ(Q}& zH>?Q5kT<$Z;=k5ozi<>j`0ETa^XqKLI$mTM>NY+*qR)MO1?RFrv&6i58(4IP{nyyE z!fr#JOUTUlD#tRMhxL6J{kfppF{iIEOk-2lKxfda4}t-iUnlRdzPdd*dhVm|L3aP= zVRyw+Y(T%Tf?a{H9aa_Cf%-NUEek+PS8ea8U(|7LqND!D(f`*uy4`Y*guTANK=|lOzUDT(ltn$Q13LAy5wC)H zd9-I<+ec#t0hZqgr3XG!5~HscJM}71vXUBTYZI|}iI<7M2*`hA`(Gb;UizPYTUzkT zzZdb1pGs@_>?OVAwvT@IZ})vu51AiBW9-6~odIo}c;mR6Y5WWKrtb%$h$qV_|_P_sv z6XRvB33M17x@k)^U;0pBhVBpF0W09{__;yADVJ(HAEWaq2cG{4c*w?}?*J!eAphbm zhAhG;nk*6VHYhh-Z4{aK5R&}xc`_#P25bYL{q8&j>Mzs`eSIlJjs?gKt0{c#nE?H%D_@jh zXgsa^+2!X>!Ax29yEpgJhR5h&K+l8$3|zhs4_VBGU>z5~mIDIEm?WGaz>Ts^dbelX z&N!j3{uR|k!3@TfBmpbZ&|@6tzyI=kU-A=F!Ju`XzCO8wYY8Hx(NW%q-j_a)(C!>4 zpl+c+8)0(b7HbFYY5RuiN$n>n#O-|fHPhXt=TFf-tlqoAxzq6PKF0UXjw-y83 zcxpMCtoz^03=7JgndS!|4_OV|I$ka+G41~C-g_bG$nihHxCWSLi(xhNtvYMunne$u z%JS851S~8Yj43lp&At!Uw)|0I5pp6o8cmttfb_t12>OoXpp#NG?Y!Y#k>lu4yTl1Dc!AmM4BMfh#7MV4S|Gh7X$thG za_QwuQ-kKU>%j0&47!2&&HYA2VpI?=uX!c$a#Lg)9Tt z)^5}L=L|||;K{dsW2|Ec+;^sqKv8--RzXl(aZe2T0T7J=aDKq&cZ7>TCm3)M$@8GW z-Ivi}=4ZE0n|Sp%|T^cynZ)&lySlhN|+Lk3t+%h)@Ki(83tKi{19aC=Iieb z1#3$}BA8%;+x53i#c_lzzJc$)0Db@xaB70JJ-XAo-cX3Z;0Q430D$bU?Rm85-|zbQ zg7$4Hbe}u{Hp0ow&@4bYC`!aFG#xqv^>K3|U%tCmmLBFS^zmEIusi`DJO<{;aU1yC zukMD-lLnA&Eg&oChVeKou`OmDOBe4%Z=E*F1uZ9o0kNRw5nu572s394>|T2E7@AAp z0^@R^Z9mJ5rkeJfq#H|}R#@g-m>)chz#5h4uQQ?IK`a3#(5(7@@!nchg0Ba|r!<&* z#{=TP)%#jbL$F*CK?LY*mZf?1gQ(~Z^Aeq=UenON`BPcss_sJALnlTQ9#wKj3wpGT zuHb{*hGYBtLywikx9&p%e0$R!90u2Oq=%nSDs|LCL93`V(Z z2RN(uP*VOq!<^b+U67jfZW%S|-lsm>-({1_zm=A=is%vf;;qsK>S2UQw^C23wgP>R;|#06 z-~J2m9Q8LD2x*{V4|iIP>n=xVhfdLzNgv$=-a7~e*n%pV*DBI`7Iq* z_--#TuJTHnMLQpe_T`7c3NGIB>MyP9meUh7vwyT@P8bVuv<~w&?J17P&Hhi%J&jhC1$vzZ zDg_y+uHu21(*hmMMxWU-!YI+X)uEmY7Sv#qoX0fNt52w#j%jXA(-)Kp0%)@yPBfb* zAVDOWKY#l=qwhDc?!+Ka?!YCOCn>x>4gKbaXDh`^Es%s7G3LSdj#5I|$K z_vfjH>o~h|p&>Gx)#rEypZ7{&%hARPm>;@zQ7d}$BH zDiF9EpuLyJuV*Qy1F8ZBCeZ--Aet}IfoFCB!vTnp<{#JJuH#zz;HjXE1RwSJ8KT~j zIS~YEsObl9a{1j`S=B zO)?|%#PNWgAziW14fYL*3bk%!3)Kv0tnp1AV$a{%H5&=szR9^WSAk6TgN|(xt7umP zG?Fu@HyVr-?jNl~j=@<6(nmUJdi#lBbOCPg(T|t1?40PIg89zLs3z)w9#A@{9K=*L z*B62dHW;V?t)1w(!c_o{kcSbJqG5TWpWW@i^jeVecnEs;1jt>TfGU~1?16L)(jR{6 zkDF1DIHR2Rf6Z8x8@A{&;QaT&T$}tq`92SyMOVq&j>7Hwy&~UWxN`b{di;)vtC8wq zG3b=qgE7Z)6T#X+z|LRqxR=e#+_!;TnvXFjSTQ6K3c|)y=DmYkU%ARyp#E5bUa5L^ zBZO>bJfoMhVNC_-^b$|7Y7q4_ihxxEr~Fh-Kai`et3K9K8HgXe zkR1}u!Ryqz_)ExXA2H^EwXQRtP;6ziwA)1cu!;!2Gy$j>`vU)LwCZiqX!FkBnkQ-& zgMmdF=rB}9K;aUoL#xsby;^!r;LWh3Ng3acD$#qAAbxTS*g6P^saNNWbZ%8ao4M0V zjsf%F;byS?xyh)5s-uutBZ^r3mO+Az1o|^ZJ2UU>O2jKm~|v(@#tthTfc)_86_;1rn&I#@L8F zW{Wmp(c$ckXj^l^Odw9VmFEWF&{;tg;ATV8mqSM1yma=L*`Ds`&%XalLpxzRn6j&? zaXB{)D>IgiCEK~7qoyOu0*|+DHx?U0dFl8-gFd}LIWov8Sqpc^ycjZtq7?3dra%qwl)9@PT`%=A&BH6v zf{a~&kup~FhnxJkYq~vr_teCD&p{BhIXBex7`=?Djw*C2L{Zd^){CLh1?sbx`>kYI zv^1mA2xsLHOrg2(u*meqh=^Bkj*WX}g0w_|adN=YUd*^i;9-Uo&v@Mk1@i?}f>`Br zDcaF(aqi$cv~-?z@BXC7bH5i^c;3paDJ-O>Xpw*841|uM-8|$w$H?FAtp)D}Rh!wn5{=eIE2X@cOl2&R)GHPP3+-jtV{n2GGeg+G*}8RozZ=)Q&6UgLpO43IYcE z!KRh5lUnISdOATpg9ZYy2zuvPFk0>e2zd8aUKY^>0isBi2Ao0I@YXM4oww8nm}*%- zbH}TPlaB1pLM{Of)0O`m0rsrl0(IY=%4+m2w}2jd>8{AW^*>+pc(SY%ow>Z|8W*eob(uAY1}+q`cqE6u3J?)zviLi zpw;uUtJ4Q%6>c&WTN&nDO%Dpx8ezdxwd#z%UGKc19frl;^`YN5i z_{R-*R*WzTQg`$wjJhe1sWy=?U&?oN+2xMx!3otOKrA)cSzZsX%k~VV@80 z=yiGdS=T6r=dLU`Ygk!7tru7 zG_TGpV*v!O2o4_L@XqN_EQtc`baV>x==Bs)pG3pZ z1I}IN1^)JPU`{r)Nc0%k+3#@r0*5-ZbOI+kQ^@UBE3?5TKy~%L$=Ye?b$5a>J-Vjo z38y6PYl!RRQhUkrv#sy^Y8d*LZ!HA}ST}(94BEPuRu2WSYza7R9%VVZH()tebZ*js z^B`V2;~-Wy1bO(_GfW!oU;3Sw`}yZi*OxjCs|O84Li;a||DKbpIa#3|jLy+jhradW zGNu3{XT5vU5e+bZT>IX%eVk<@GE~=W$RFLiv0S!&(ka@Np*vMn*8lch5Mpy&z%Trm z_8I;7x)f`IKq2spr&mr(M}2RUQJluQo&e$ijt2LBa?|BIWd(4S^o@C=yTI7>cRd5% z0cy{eB{Stsqmb0Q=JvZFv4EZXwJdz|kwFGEz|#@H2-p)5{75n9CkF>A@X#>AF?PQ(* zY^SJPczoZ1Ve-|jN>PG)+(Gr7+k_y;kIla7^Mbr>H2~G53b@&IP3O& zZWKaes|U=W#cENfChKat)XRCGC=0LAMnwncY71i#1d2fkbKK8*!m3*e+Gu9Lxto}3 z(TpGU_Av^KL3zv@1odaia$goNfFt1P*9WvhqcfQetUwB4wCYeg85x4kDNr>C2#}_` zw}%N$H+gMxm)gUS81JF=j5lEU__PS%C0$+MWzrvNS@9|ovD8T?Ai4?Z(+mPZPJ%vf zcrAZ?6I4|TyG&YuiM9`NcH4IkLwoib&$52H*$e_AK*c8d_WVy$>Vib+jAm=h@;X(~ zz4fsAQohBNaRaP)^K1t2gys+8ELpyQZpPnMS^+o4v(%?g zLcLmD_sOlc5P-O?iiiwfuL-4JvqC%Gn0>=DpxJE4N#+A688Qx5N8-026Q+y zM|n*Q`VC}rRefAp#dP__F_Df(=)|KvyrMZQ^(iaAy{z_>dTS9huIKKTwRLLdqIYs# zRK`Qr;yC^A4aP&r1g*`y(P0QF0|X>?O>(xwCNSEZa|XCHxKbBqbqx;1xRZM5Zz?UX z>IcoQ3~=`G>>A^thfubGq9T@i(Nur%EI1H<;lW{%z*n{VAr5(sPm}M?LA7iv+aJQ4qZ*Uy2nrZrl76t-S!93b0 zr8lA@Bi5@uN~>gn@PSW%SDII1iw@(RNw#7)!W~rU(0~XyJ9&H2OdK4*7~=eaz-h4K z(eFH_-p{}zc+I}?2%3Emhuo%3Nu#l^PamUYkh?(Y!S`qd48f|!{#S2f0MHvcXXG_L z_}fPubg#Y8m(Ge9xdkTBra~uPy)Vc~)O*GX5XQ%X!MS9`Goc)0IoZpv(A-)rV4qYB@05BsD%5?bumT+9DKin9*1@zRJF7<(C(x?UXG3m zF|;EfuDWa84SGfId>{PW&0r^48Fh7J&N(t0rggH6LZ%m8`f1}{=F_XcWTK~!4b^e5 zfWI`mA3St|nO6W>TI&nuIb}5qtZTA|b9M&>s>_b7KU|kJg&sZ6%rUTNND#Kw7dsHf zK^#~#_wEG(qIA?qFRK!5)d2&T?r11xF#X%_?*a$Xoax|{etSph3$I&K&vr*Ag8tF_ zg%quRUFB$jlC|9SdY)Zk-g$ue-~!TzrkQrV+;|dL0RhejF3t?maOoPHRe^8fe=&5rz(k;)KN_lBSi>yu@|bJ6HYzqatGSB>X#o*aLUVR z=7AbPwIG3dS`ya->i7Kh{&*n3l@$Z%kU5Zn zgbmM9bkxKAIBCWTa1zv6CbGe4JUWV*>;cja(zSv0F3{?w&BW{~X6~wE4@X@a!~z>& z57yB=*@5O{M#Wg3<>fNyAI2g&EJQ#C&wShrD2t)z7-4?uxpcdIoIi(k7_HLD)LvfN za7Y!6zxLyv5Bl0-Ii;8Gv|&b=Idl`q-SX7|EotF>;Llt~XLoN+tmLI&)jmr4qnO~Lk&UGYmI7exmy^M0ClNu3vR7fM- z1lMzz&S}%iB*O3TWB(+Fq6qKz)}n z$9b*pynBb}z;g${QX0VCx*LpB6ZkXFHbthz&iz>Avp->SkLieR?La3vwQi-(A3`gm zFC&B%;B-sW78(I-sAKkqADUin*b+6ml+jlM>7B>__WXTl1D$gQn)6NBJkna*5pdwW z+ZRR9xlLnfN0?>7=7zNDG5$4MK+iZ$*boi&E|QJu*i z)jH##o1UO|@tGTZtyoWs>*=G{Lp0E;9%n~qb0Z*L8eNUhIeLOF$prOhd+zd}rOlB` zTXe)4$WeV@n4IE*Fpxm+EHftD!QXP9YmXDZUG^a*jK+)}fxLDU(L!BPV-tO3rW z5i|?pY9Z5LognUn3tH5L@JII)nz@RKpA!&({Lg>;KFi}V?vbg>6syQaXb-=AGsFT= zHi+8+HH$o90t^p;(_cRhR<4VS`rm6Vs~@mA#h~Vhh6BKCxi(M@>Z8K)oDYcJuipQ+ zyT1z|dVUq$0Mh~3XwkNZ%#do?@U|<_f&1F6vJN;S_b9`ILVPRBUQplxxPayu^w0qV zxmDD;(hR$;^6bN^I8_41adThl{~{;ooP&$^f**!y9KCWTMTR}t2u(Ex0)}eA81-9v z4iGOr1Lj`qzi6}ELkD!l$AH)DfNiw~)?$IvvNjS_<)HCIuyhWbEht(xuCpk*D(MH+?G%g|v*XI>JpAUZ@xzev%wRhQ zVCo+{I^b;2{T3QQDd!uIM^gYCN%PCkb3cp_pgxlOVY-cEFM&_@cv~L-$0*h*9}E#F zmd0|UbN_@oero#O>g5+07TN`w>dw|uSy{*Yp7ajtqy^^`1%YTB^?I|S>Slxq=xb*d z&p%!Na8PfCC|lEO{naJ8%`qsLO1d!AGMacQh)!0|;DiK>oRL<$K-*DtbX5`OdG;n+ z=}WPpb5sxJQa^@4 z05-CxMKHMkib_-Iv);FX)bY`#g47?}oFHf`_j3LnQ~(a+M>v%ZH!zl;mdjT!GmRV1 z{}8+)&%b=(SbH->4zB1(IryPhg8jejz4v!r<(W2YHLG{Y>XIc{&5~8EuA^%8-X*JB z-RfPkbfi$7P(46`2@C{6a6$-#9ta_8VOtp6*bW94F!r0|Ba@lTOum`<2fpjt_oID| z4N363*7w`2b=~E;cYB_@?R`}C$zpCegX-$Gb+P9ZYkiNw1>!nc;~%`tH3FT?NnQ-d zY7tpdRshAAKT_ws=KAUP#i$&KL5DRMn}`EwyWQnSAb^tS`r&#=x)jI|q5bZ2Cv-nh zVJ<`N%(Dxk45L-^Os7QP`lO z2Q#$9mMaw0+;~{mM>YrZ+_ioQ&@afQ8 zkd9W-m@mLUG*}rY)+dGp5!}oh?$(Q@6B$y5F+AG2UUe3X90rZ2ehq1kH8!Do>kRPJ zcBXbQi-D^x2Kj0~te#}tm<#hDSMDlEzwj26fB(%LELosT#0-S?F>6i6_7K` z_in8MKYk~e7l3wSChmy54!026Q?-72x6@+<)Bw*d6< zOWJlZGGhRf|9R%d86Mov*)<&Nru5Km@psL(c1u1eZjTBw zTDCPfHPWSmSRSP#2wLJW7}prxRS#ZTH0?7s3nj%#?Ya+xb8mnFIUsH7Sx{U-_7%KT zm)6+!LKEcU2X3ONxyD)2|BR~O5Gw)b@p3r+g5T_#zQJCp>;*dFpmA>I2>j}4rjzz3 zTD2_oD($}@_WIB z32*m>^dJMcUKVT6@q1#8Umg*o_OZ{0*)mn3J>W)7G0naa4UG&Fex(OqzB7cgIAlqG zHi3XF_G6w(M_|ElV}3j1aNz~*UxJ#U&Vw?}z3a~@+cm32GkqB&&-wxI8c;_%*vD`0 zElgUiE^et|;{%!ol0=zWbvguXPDbDc<@pil)u1qa3zQi)jSinkUhU*>+IX+A(7rbh ziXZ}Fc10h?cIhflG|cJ%QcgSoz6@HcG=k5v?^J*?y)rsOPQM6^J~9;;Q;nr9h*}65 z6)hj)04TW7Qzab3!=$s?uKh_RG++^QP|Ye>Ox(xG0c@m`GhiL?84Izh z_kw0;A?|$jnCy6{U!?DeAC>v5e1*k^`*32MZzjl;HES3F zj5Sa{xIFgJ!;B_xByq{yA7G9G?h8TNhK6|S3fRR9<%(JRH)g2Xhr(S@rY}4T?#NYn zd#Gcf11Z<;5~Z%a-@xQJ2y*wo_$u?#Zl0k94$Pc*S2XbJZ~W^EPs)gHWg-MGgffkQ zNvAdzf`HQoc#@~r*9y;+OUf8dB&DDK&aO|@mreJj92Vn7SV1mx( zQx8Ks`@RgbhxRXZW;4^*zI<9t;DxMzOHh2%up=*X?G5Hx;#c?F9}4MOm|$a_GSJrR zZ{dxh4JeP*0BEl#=S0on#}%=WtZ6^{VjV27Si)|+2hLiXE@a6vM>x}@;gT?JkM!WK zf-Zp-`9QmiRQ9l(EDoeCU%MmCd@ECO>nn(? zrZDh3RBew!saykLv^_W-AGW^5+zux>OsOzjqrtWIBu^b{Yj2Br81YE7+?SycrPw5RoE<|9m1LN8&7F+6xpv~__)$-t&?VM&1 zh|`!=PJtT#$vdt_NFY`O9R%!D)=tNkloE_NRhAI2LrED`wutP|-!SPmNuOqkIS4}K z`4ZF>W-g%phaG#>1gN*~Uw`s=jcfpSYEfP+a>eN4-LiqPZK2QEQwBQO?K(j`sW}|n z{_@5XFbs5RkGA0{i~~TyX#q7{k6`@7<7zSV56=k<4}!&cwOaH1087(NRPb=o3RVoK z?|{lWU@`9%bLmPk6m_CivGs^pgJ|79%H-2QpR2srZIzD9TMrq}SVHzP)tQyqAfwwd zIFMY{>&vZ;pbJlejd2%dv7lzdp9|sME4opX%5>F(+YB&VI|`Pp{k%ptGjo_Hz^8}7 zB0+^R<|e^Tl|<|2 zT@iih-XR{{ytd(+h79d6hr)2&P z(h9mQAcWy<@?l*8iP>DS0Tjmu26(H)=FR|T!{P(BLIL&MF)%!Ns&JWK1sgB~2e`1U zSCp*wn7;;Aw$ls1H0b1=qJ!H;crdsswYxY9!CWzWmC5lrkgS4|>Mokc$y^9N2 zhrPOw9NUo@YaO64TqG|yWv zt$q5Dk8II$l&bn~eXtxQT>F_UZ!F*LXb;~(&1WiDXMg=I50i_TznobtxATVPXz|$T zpYY}oaT~8`k7LP%jqOlbaXW7?SL>9?*2f#5;8mqx56Rz%{7{m-=N*Sl8~1+uH5V!FfY3yAenHbOeT67-RZKV|*V6+BfBNMdUMq23y=~WTI zJJ)SRnKxXm_wJChMchW9ZjN(80nU(c8;1|Y=yC2f#@%MLq$;$p#n`x=Ln|fBn5qq^ z|288Xg<~{tRYC#&-o5P>Phju%3hie`aeLZAD*xVX=ax#rky+xYZn-Lc2_?$6=p)#% zK^pd`9Mksgk1e~UVSDf~?luqi;GHJ5lI{A#N5HMzRFwZ_Bzcqkz(+&F8F&~%(>AJS6 z9?q6v=@9Dy2=!oCnV|0|N$z;IAEYA^Ziy7q3EZW-hG_w&PMif&Vv)W@Q`RiMu;yT2kWBa1We&f<}#rD3DPcpLe-H0EeFK2FXzf1XvIKZ1evCe zfsQrpTgf^3!*?k~LmWzSEfbstjr2#A<4p`;DMmz)ShN?p2wNK|p1{wUaSXcBAjQ{8e+p8<{cW zBx^wDzr%*Xb8}+3Am&r&AKz3friIr_QZa$5{X5k@D8OG{{ZxM-A)Kg-Ns}Qa7Mr?s7u`~ z#4-aB$n;`yCYZf0)pZR9&NcF{uRjX`REjX?HM=oVvtE)7-Nld#qVxDTkXIlW^Ha?z z7-~89F?+MN8@=GxH4ZM=$rbBJ8c~gR*<1S)PgN2r*nw97w*G7jYToLwCZTa;HI_h*u#oWk*2yehuXifa%tUL6y6oScP-#S>}de^T7&} z4PbIJ1er1vzVz4ESqnfn4>E#QIKgp;EGLL4(DVn-r6uq{)nopjd*8wU2;{A$52U@) z{K^q%0iYU{P*8Pr>Y8J~c2*jbF_yXDEn65XLE3i6tdFfj_AQCdAaLP4;_QT8HI<$;GJO2 z*-QfMhwx)i1b?~nQ7BW)I{|MCEA-Q)cfo2wj2>VuB1nYrnHANBFyw_SZL4Ji{j0j< zE~qRJ9ZW1n3kVSr2w}_lr@2K}Io+Q3)`>6?1; zcn037!}sdOsbDYY>z0{k?u~QYQ&v`r(O69M9teL>bV{UeG^cgq88Cots3C{}U>%Ku z0QMpJ9+)3yMptWJF=TpF_JK|Np}T5RoK$*tH+c8ZXGe;I?Rh;NH~C+;=B$Nmvc?3> zx+Y3LMD-ysvqr?3jH{bg!;tK!@Fu0WT{k=B*2A4TCoF6~+Dq&^+MsdK z0w#>7_4Oxog9>M{T|hi+bk!UKHV;Zsr(-$jh-ltxiexCvV~1E|xgh&_b}9h_%xlpV z9$VjF;vm~RN?k90UsPUwH!r%-%KQfG-C5vs?8QK0=}^RuIeL4pdroC_)EtIf6)=0| z&P?zAjg~IxWr-O<6_Dyg(0Hb=V>8qx;b*?N7rqpc$s z$SwrbEjkXCi3)!DyI{#)J!0T4%vgZV8(slBDB~Rv9c(!Ov(P9ic#&&RS;KB#LD>em z;B`hJWxclI#yIBQH z>h%55%p0DNMFH^qouYzsmKW~@v!|nZCMES%>{bAj%eft(hEH0p`qMwe?;mWxRRL066 zAA;fF;uzh_JflZ6XS(%no3`B;_^35N1-M#|&8NU4g2d9`FA|((;$O+-o1+y2=gk+u z*UZhLbJDE9OHa^wqo$b$b4xuaUZozlr-A^_ks|PXp4bRd2W`E1B7_U(>VT1?=2h++ z4f0V{C|m4BTw%V&fF^fziACqhri>WZjg9lO(9=Nn^Ty!JGe6zBN6)v9z&P#!mUaE; z$G@(F(qKIv4AFe~@hk`(3~nN*yprqtQu_lJCMyqd1gV{={?E_fe;lI9&U2p%p7ym= zwmY@|uuI;FZLSr}n@(`h<5 z8=oqDn8OMK@P#Yg(LJ;2>UsxGJ=d98s_*4w&o!~gMwvL3vcYESw~`o!6?a{H^WUF- z8S1~i_^xa>ieS0%PU;3SM1Ot<9nTPQrPO2SJstn=KmFh}4A`Lo=4`U4g??PvoFNeW z`q3uspP9aT^pP1183@)OC;ZXdU~@w}i1z_=q_@gwDFM^DvN}rCUDe#YN>J-rV=qT%>eo~9K?ky6Erq?Kvlic@G5pM zsI%9fAx|}V$cwY`26ciOwmyQ|VgPU(P!eBRCfa%w9UliWg7`$?0%9Izg4jg0&j4&4 zz*X(PSE2aGIHr~7?E)={VR7yB(S`_KxhJvHFAA~&G;P_}ar(%~*Hu>j+fV-4s%RpP zna4doWlGF)3g?gAcr@a|o6y>rN%W|@bzw*^Bu~R#06Kdb@<5u+)6a<{hyz|wP4Jtg<&IB ztjV_OsUls*IuEdGyKxsr2LhM?K%$YJ1_D5(^DGw~wzGj*v%FIYx}Dmi)nr(>*)=M` zp|b<>1i+YD7pKGJ>Iijoooh@}dx_O?LEjnyO)V({s_OZ!vYi2sUIeSj; zud)DIn)jKz#V*{F$G539{rf`?V}vh9c~oV$z9;KV^=78JDi*toZ-mJ%oRD2MFSQnR z;W@DxAaX$6TQ48jg@M!W9;JTaJvbUik z%oAGne}CukANVwKrlu=0m)NyfhT|X|PRvs_j{{tJVz+W0MgKNdZDLM86&kR_klX~eW5t5?dL6HE|6zf0yPbCPSH2C`&2(vb6Ba|tV1kF9d3a*fO4Td9ni0qtGYEp*Ov^54yad4R12ACBn3REq z@;u0F5D>et2b|q}w-{O=3_RCD+V%S(R8(bShy@SL?*U^nrL<)psFE$Q3q~k;B{yDC zFCgf89O67`^J*$Y4yf=+7Jja|HqRD(5K}1i%vX z!BZ}(6uU7hW!C)p=zGd(E?hmZZUBql{2{ns1uLRsJp>$}%eWYoU(W5!YttCo#Ye#C zy02rL{Q)or_w7~pD~wbWpcz2Bg50zp06^8girN&u5k5bt0RHvCM==2C=yK5K?}9CF zYXWy|IS9TmYh((hEQ8UFOsoL<_3Eo&Bd4C@#dHjdwkRU7rzZ8(~to1N=E2@2y0Mek@BDF60`x@zt_+7 zK+C!MbicZ(o_K1V$!_*v_@%P9Dfjt-{DT+w%++v5chD3lcR+hTJ2cKFopsGD0%k>k z-_%t)dB4(;vH4Tzl}s9Df)1X#Rz?A85h(+G+RrIqWt>`ND|iqn_ok!uO||sM*Ku9@ zj1{0}$Or_r#?be^>^eYqxvvp|0bjfROR+J0Up?}s44gTIaL+FAPNxPt-uqb8fPvGk zcb^Dh(<`wDqoabSz5 znM#0v_P_n~$EwURG@U=BF=wWSF{#^rU>`q{-vU*z%!9yTQ)w~f_BgGvs4RKouXCEI z>ePM-v3KWUE-%eDkxPh_R}Vlh3vpG19)|JOzrnD62Vz z&+;mXtD$E9^~f(7PIs4#bsJUFjNdlxW~l!V8v1-eb(CEQPZLQ+iD?3i&1H^DG0&{^6c&AUJ=`4|Md6+P-1x9ImeDF?Ma~@ z1Hi%L05N2W0FO?{NMc^t?=^@mmz|CI0x(b3*maVx9|h0e&y)jtxW-+^?M7x3FcDB6 zqTSoI>te2Sz^0*GMF#1{E0oK}Y zW#Yt^Z1(L~JXkQnn&KY&V-f;FIA#R=OA;!zf1=85U202)qYa{hu=KE#?#UU zFlT+&c&o`~t_t$b{TzxPeIK;6&deN%vk!t`l%guirXYX`->tWuV8HefIpL%IYl!`F zzcYrhpdpYz&EFRz)=Psa6IyDjU|^;ufbae)(4K$;+*2-2UjGzu8X}wVUW}6+* z>0#|R)ibhvqcnI$v`L8b@A-lIhVv=MkF#QRXESwKGM?JcvZ^@)yZL$#uGbtZQB0Dq znDA7Hk(QDL-UwDFREj}o?Z?wt-p-}6C*Fdn3*Vo({JujJug^u#b8ogl!RdGZ_{J;R zOL;$gOE+?g+xq_PF&YM9CAkI z9|TK`vj9(E83Fz(-BSjFn0P>Qk2knwQa-pkEuF6=+-exKxa;2o*TE&Ih@L#cvoQ>S zXK|Xtffmt)$8NXnDulZA<(?nk7lqM{pEB95ACY~ywg^#$hvzZ}7(?eGUW!v<=>nm< zU1EG8nT`UEfNRfB@H~L2>m;sWsM1{ZT^KSsK^7;2y{&>F<7@=o_K?i;Tp2Uf6Is&~Cg92JG>s@BS8e5grd1oHM}_&;MG6Y$iZziwk&dj%qY%7_e_tf*MT@z=&7A zDUns)FdYFV=cOZq;HhE&4^#^lgF&}L@uh20Ja0S&ad0PCVE5T0OgA?FO|>+rI#|x^ zQqv%Y$ss+^kAVbsc7u6!vTyx&?SV~z()b&?o6FZ4L`|#GkM?_Dmrv_F4dy!Jlteu6Y^2|fgtA0igl{@{5QCQt=B^^Wi52mC{C zsv_BX(}$@h-nRgigVky%#;KUC&wnyjSHgf7X6piiq*XwRUNVRYoVy0mVJyZJKQKTw zNuyE^sBBSv+}mFjE?ws!M=rMF zw26&bFIWgOrB{_-h4vn|PT4e}O7!aMoBuqgCdU>fIQNRKLiy->={q0}o_fvPdi>|$O1U4nMDJ%I*J4jtZL&SXEe9%gErYY8f(-y z^Kf(2X0Kz2-;98>c`}0p&f4FfXur{#wT%cYX$@S=$-^dQ_2>uBG|FC;BEXXdl2ldE zZ+_gaqROIV#N0ure+34dXDBMQ7v)S%LY>%QcS{T5yl#Q=*H#PBx3WAvYq|BHk8Tv@ zcTe}(C2qb>=P~79o(%y!?paYDhEPewoRf#+4&rnV_J-YtKL=m@*Ng6A#pb zTYv-yF2AhnrMd=X>(9aC#K=s!1vVI~dvb^QCI&+L>J>;jbPtFfL#;0hst#67BYxbG z#tB!6*{ihr{I6QqF8yY}*s1Tx5JZ9uZ2G=pw)pgA?FYITsgtfiO_z-mBEWk@8}C2$ zTmm!c78uRP%V`tZhkessh;CJWXF8iBpR@OF-Fnf_qHG(4h}#avCF!6Of>^A;dU5*T z{;0I;-z}1H5Ckvb+8Pkx7450Lf{h(xkr}|*YrmRqhhmP(^WWR3LWW+Dn${#!T&Hgr7bUIXJO4+y#RUMnJ&J0lOV} zioG&m^rz1~ree9|o1a&-3#PVduJB);d`tEMUk3f#i$4dC^#|L$4@^yZIDU7z=g6Vp zQX_2UU@+EHDvm%;;#l9|<6l6v-g@UA^o{&j-XK_9m+xWG08xjydL65}rYl@u2nfHO z)@{kU#%c4GJgiPruW!k#ibUDVP~{IZwj`Wg!V=V0Japq%TPIC&?PihytIR7dDwbFE z0rd`m3A{y&{O(xjnZvqOC8R#yfaa%GFROJs%U9udi>9iOyv*SVL z;Em(VPqu0zZzqd-*L-mduf&oN1VeDjwV<6KeA)B@y%sl~2VU)?eK*PwRLTnw_&)iy`Xr+Pc(@KFsnnY>72irj zFyjY=M1zHew#cYV1U*}h!grc+w5cd6fZ(!$P>_+! z28Mi8O+dE=nMN3_ve~Tu_tS6R4Otb#Z29O(*JcPonF{~MKhjNfN*u@^R3cU;*JRtp zqY!yC_g;A8+?`8M>Nn<+Ap{I|jRly^@t00L7ieT2r+_lTuf5EcapwN0R;wLQ{uaiu zfr}b3xw|5P4FJe9f^mnGMdmi}R$)5vjP_%^N!Sj^zTorrV4ppavGwJN_4*bl3{QTeS{B_8u%`fQ z5-F=-Dj%UOf^tFOAfPL+gf%M&g5b>o2r;G^>BK9)O61Qb1t@?mz{|i7LMyi#2LQmu!R;5aFFm!++V)O8mVFC-YO?5t~yo{g^ zB86u>xcMty%T>(QBCg#J=_R`Gd>#-ntbMAT?QA`H2ZnC7F?1%gPzUo4ur);pqv0vB z9^i}tPN)eK2-*(<7AZN2-fpmODr1=2hEKf<0gQpBMP+XAm~RAq;cL?XZrHqc>$8;X zdFE}AqbzkDGdGQNvnamuQ(3wK;N6gE8b8h;aA~n1fI$FMLPSqM?_pNG@d-*5V6Wdub+D{6k?S{Ay^bq zd*hXKXiOz@@Qw>`Hq1Oe1e${x4xFHC)$SYW=`?PuRc*>UR4?)>KxZ$!Yz8TqF~DjU51BU{3wU z&kDj=76hJRYF~BHA04CD0UqMG0v15G0K++TewWyrwfC-z4V)JQW_g**Z)+hFOHd9Z z$F^r*8DO-ck83x>M5u8+OKlMzPy{_p=3Y=Ph;Pp3*BT)vlbMmc7?1(t$#~u#R-KI%m)rvaP98b=DraN^;u#9M z4d7Q^2h;J{k<`;1)z8xb3!pf*&VuV(3EGg1p}?DU#hlRI@}HpF;}})#|B+Zag00ua z$*4g4?F>Cr8n5pnic!Sbw?sxnu-UfxgKd3nt*=)wMh4m5Q@#K5*PxoNB=#}wRn<4$ zyRl(l2UrUacbDD3RUd>l@WDe}V&>Vte7oNXq7hWDJs&}7b!d-PuEr>b^}h!MF#WtY zp8?N{4FC^igtI`v3THnxH9xUm#z7q;Q0QO(39*3pTGWs-Z+fW;1aTdH z8^#vXuj=|R${((va|LU3N8vV4aBc&*m>N3ULdJBTt*G=#Ivdu8Ez>_G)D*^OUI%EK zXeosFpBH{}hZxvL)S1=t(oAXQE&FV~)92iN?eQ3lG81CEcnc6PygIgtwk{&FDMa5Q zN`*5dlj6#==353o`&nKOXJF}2w5@{O0N^D&L|qfX4g@SOcrddW9e|hE1L8he@?$1w z>;_jJ0~2JW4d^-)HU)RC8lWoxHB)cphf0_({+9b`f0`ouw2J_rWvT$JT#F8SURB4a zY1^KbplPnWuL>-~TFo3fTRY<@h=%y+Nkh-fNP#ae4fKN!fztdLM<%}xPO&Yi*q7O5 z&k8mNyN&kZHOQsxTsHeJ51X#*#WVG1GES+Np1<+{WEed<{o%84#8(ieh2 z2!rQe7oUfTON<3&UVb{4A<|Lt!EaMa zG3wSWoKrTat_5tFj@}zvl3@ zIb-kkZd-_E?w>NSVn~DJ>TB}A4szl;STBN|x~3~5jBXKAc^EV)c0R)~*s8d_4Idrj zY5(=P_rO1Y9ZUyr8#;V+t%7smGUHeZ#iL-XYhcf$T0K*at+mvk{bIb>r>hC0?#4c@ zzMx&@gliB+=O1oxZ-fN)Xg`P%y_$>P%ku!dcAcog8ULQlnD|`C+PEz6y}_JJa@05{ z-P{ZoA2F!?qlN_?02oCuw7h?04*%%KX8-IR5GFUr_F#DW zxz2WvPu_&atDX5}V==_M>xVyqFb7=_KxZ>gbVNIg+66HyD!t=u+pw}|jmK?Olnhr67fFje)B0acFts8b;3+OBZ)80}fW8NEBZ9w0iNzNP?x_U$u9;E%R|5kE0 zCAO-B9-BM=SS{zx!mc)T`DHnhK?MRt&=<4eUEslD5FRh$g1Q<;OfGkp4$!2QCnKJz zjA;ODV`TsVQ-O@nwI9e3e1sXmRZMm<9|l3DMuEX>vUUVC;DCs&->D_cpMPaTOyJp| zDJHRiSLPXcHm(L>?7hEw@g4}3CD7CpGk0a1AK+#is+`VDnTtEAPwX- zxXg+)N5>9~EO;VzLmmm_CiM7v8bk;G_$`)>?Sx7&&bEtsF{s(!8XRD51IsRI;p!gX z5F3|O3cE@J*-1F3Iwk7jZn}1w5!b#K7jg(R8s*yx;go6*?qZz1b}T&zn&1nzxjB$; zGS}dpUgH!0u1@mP-)5=Q)HQFj7E zaSPVE#V{NJdGfJdeREs6isKIEfjRcBswL@09?L$mM=Ymbh2*qZHhE%OVs>)B$Z9bx zPSTycNo(J0B*u9N_RlpLDxtj(Zww~J5=ik836B4V4uG&}tf z7*`pj1+oe5LKzZkXKB(`f zwpob5w!JY3+Lw2@gKs_rR!hGD2esdphz{O<{ApdUFnLLA6A%TU{&9MlsRXq3d98B; zz^11C+-NN{3Th0<7Q~nfXiYcTPOL!IF{1^U>Ou=Em)|eY`bUs#U?eU)BD!gJ8H2N- z>;W1e1gp5lrgZQj2PDIl3v`nVuJ@6fQ~k-}-;y zhI$vCr@6I@w+0>B*V8(c_(a<}!ZsGj-lS&0eCI>~*U9>Lh=jI}?}1=f7y*6o{CK|h zp$&RVj}aW`5)mL`FFj=rMwRVkp2Y#IgYrRpt4vj342Qu6l*c*g7*2~7-K&MU2xwni zFvhixi-Gq9CS)56)LU%4V&@;6f8r`;6W7ytJaV+IX&Um-gkkPY@rG#?UQ4& z8>0?%o%=Z_oC`9FrNy~|X9Jq}Od;6(342W^(=ZUU!&I?O82a(#7hkz@AA~uq$~Y(X zkGU@T|Ln0?Z~CN1RnBr+x|$Y@tJ+_n)z$ZP+OP{WGU){SMa<7T%tb-9;S3f*wBHnl z4OU)7v?X{%Vnxr%qfq7t9Oo*TueV@H#?BDTQo0MM+cB8&*CXoj1Y_-?Gxz%Um?S_K z$d8Q=@C1c?_?Qd~B6QBK+rBTKXI0LWO%pX2c}{<6Usp`;_*VpE*XTg551! zZ}$E0Ay}qWCe)|M=3?HP?6qHBG_zuXgPzCHK`@(iweghI`ERzSS>2o?k`#TGkW;&fJUp8G6k+}`wRGu%V3>A^c zMg0TFp6p(stbnzJ#eZ~!OgGg2)REL(89|ZX6!%+_A+Z}SlM4xt|27Bhvhb6=Reeb= z{C$oCDwPk+uUD+4(LX$N*UlpC7h&@__BPxuSNawQEYoM**qwLWZ?XSv?}F1>)6xEn zr;F!548DcCZ^D+BaYXB|&$l@Qdjd}Fyk!A>-&`j*=K%JL*vSJ_2}AofhZRX;mQ80> zu@U?O^x~}6yNfQp3E9mXwz@Zdn=6G)v-6oFj-EWY56t@8oWRJY&cfr}>i?Gj=O<{v z#p!K8HDTYPh_GM!R)AIGx$UI-M+X$Dg6rmOV%Pt=>hF~H|GMh$I0xBdmTLd&s+-(# zvvY19{a?Dup6yuu_~igp?aKxr1$;>6Lz)4^XKM8W31|~j`~buw_JFj{;gFf2+6^!r zv3!7JOXjY8_~Gr`D}~QS>h~MafV{X#UJT%e6ZPS0D4yrK>3lmT*I)`r2Rr=?m^t`h z%TE)6n+}6%6105wdEQF|eB=X8J~+u+JC=X7Al`+r_?5> z3>+62mBY^*Rp(v<7i_+O*EK@T191+XHGF;s@D{-)(ZDyHbyZi^c#k!JI5kh*tSjm~ zf4u({hCtoURrL7T)0K}C>c&kks{OSW)hs0U=s%}U;4%DAqhoCs)S>+WbdCIlkiJRH zlT+g9`5+<4)wMm7^_&n8D54*NC7@%2;S@MP%!yhDvGuN=jK93}x>$`6^ep3DFnj3% zh}8CwQ6ESy@?*7I2y}t`0|)eA(1pVcAwS}kgY@tGgS}Ib*|*Fcqei%O1iDTo2XJ<7%t+?^_u1oI0X%-A{iXBds@F zykC_!=l$W2PQ1OyX)l1-N(94~Ug(LZ&)tI5bWCwuiD$y^tON_1JpQ9fe<&BWe%IrI z0ox5g&jA&@8Yy;-Y24EZ21Ia4fGda}w^HL|9Lx+~0kIF6@*V>tNX^`2PlFMt_N8eZ zqaz3!!_@)e1dJf& zMte0FC)@-2TX2~@kmQ?!txV0OKR2)wRk4r!zV?mse8rB@RR3^qMY4I=~|yWvDAuSs_c7(grM z)%l@!gA#b83YF_RLrOY-3~Z2W zm8UZYt-B&~7zOQbDnx=a8}^m~QuVJBj0!_lR3 zUviRaB|R!y*k}aBZGh2n{dr?~|8@66KQ4q0xNd0gy;TQ^jpzSc$d(Q9gXiuM&3@~j zuv5SOB_DJfedX9A`+~DlVrpY#Y>K#A3isP>T047yCvTrXmI&dy)S59;ed5Z)bX~Yg z{0Qjf+rYl~HJD&G1W)~c-n%<#ouw>2!dU&O>JdCs5T_JrT-N^b!4J9}W638jz*5A5 z_FH=2oi&XzE-Y5@J}BcBo~Q)G6ml0E9yp=ogN8u|SdoGb#zhd7C(@K%AzDLGb*T0i z(IUb^ggI|5lToM$CY`{$4U*@gBT9YQOz1Md;ijs#iDGB~FWsItpuOo`EW&^P{trZ% z^ex(9W4;CYN$soGe4%+n%+Xb>FpEIdBh{i{x&>YOnE9stDu5N_sr`$P<)TUvdVo2T z%Ra#yLHU>W1wokBHC2QmKyR~z0M13Ay&P$GXr1!1_6983G8_vT)DKo`1XHnuMyKot z32J%2_XIJxswYG)U-@AJhRs}TEy#D$W22^0l|~#4Ky-#kygy!)OcCs50P|weL2GY4 zEIu1H_OF9&ZB7OIt6x5h(GF0`_3wfkL0kVS2~|zblJ}SRg>i6!3m00)F36==g{b_H zja(TZpGVdkO?fGHotGAGW7|*H4Wjt`>4z_cF{=UVIhH$+#uRtgKE-44!GHS6OMiIv zUD;&L$3|B%+!i}L%}i-#%#wK|X9@(W2IT{Ne?}BHx5*Z8rZ7u536#J|tDO%rzm?2r`10 z9wmKY;DU7Zs34=o1ADWyFDD9Pb%dnv74wwYi-z(T2tmtEU4^0&gq=3MA~VH&6ZC?} zjKT@XuYT~XGsJ~^z*HN$^cr*0&D*9Kz8%d~he?#FsufY!F0HPDj zVUWQt)qgJDr*Djh#$0M?Z> z468K=xJ<_GpS@iSOk^XdXJC*Axuw+fZ-d-?1T0L=#v=;@HW6!EdMaTT{>w6xnFE9@ za?OIAlEg)5eKJZ5+(@eV{Af^@4C%*`WPkAZ!CYoA8{9aPami5g$s>zuf*9+VVcZfx zOhcBu>Os&1bWWS7QM!?aU7W|(_q5S9F_vIArP*8>EINFlWyKeP^BRl_oeyX)F&*G- z#M={Z78av$?_m05Wk^P2Obhy)RXweYy(|;D*p0h>a1ZN%wRGWdZd2ei6bn!{m)Q!U zKhY6XGI@kEUoD}@2yW)ya+(=1FQ}= zQ26?A2L!Vjh*eHy;s7NR+Ks}IUQk&T0+s9P0WlLwzc?J`Ht5al1P(Tc-8rK=w>f5(Ismm)jdm)Q_42DEZe}jIWb9w_PSM9K6#+0+F>7t z!;WAFK`X1JAH2s31D0)~f*=-HX~2iCh*WFj0t75*wt@lBwmCHkyOO9#j9C;wf39lD z(8sjD(-b>ne~{&pCu4Bc3%JD`As8@f9%76T@$vV0(F%7NO)NLC*Y9H&0$MqhYmXRZ z1WqRbDl?%q+g^HKx!quqlEF%2d3w!paoUSx;vD^|11jk77Lc0d1XS+p%Ulgq7k7nj z4tkAQN;?Cm$_d)X?=YVjV1SMO!joVM?APjC$d+TWHzVe+d;hO*VuajB&t#`#2^el?8kgMOJ<2K)3LV+Mr4 z=j0<$s{Px*S#kpUIFnrlSoQ4P{E?{OE!zK{)rCS$P`S0BEf9w21ZI#9wl=N|@NsZ) zTI@RU_(qMs6^cOy!|mfx3Q1jHs;kI0;Y@UZTb)~6`LI^R-39$;?}9>OvZS_I&_TLTdM+6YL^&+VN4m= zXGg$*L{N$c7p?tu9}jf_0W$$$Zk*WJ?{EUF4ogr088A>)B8u=3E>LSaS6{gDoa?Zq zujBdm&372vd@Umg24-sGhWqB-8D4$cO4z?07!e?3f(gzRVL5CNlklsL7;_Jp8=z+= zh+#NjLe&qAjG9f0Rm#S&Yz>NJ@ZuWROi_~!!h6jJ`N1LQkLm`V38I-^sB9A1?PAMA)2BgzHCDo1NX+G}(CN z9%tC{QU-X2I#8%j*F1G|Y<+9{Xva9rGyn4V9XuPr64PGp$2A=Ur85SJMLR;n811j4 zb8Xh(IJl3i2XeEkdKoYdRqAN7VKd+g1%G_&zO%c!q!-D(tf#SD*da^bdg<;B9_dgF&2LiO$y?;(at!gW&b6gjY=7Xz2G6;Y_ z>vzzratF_wDK?$+Isy;6ew{A#9-Z6H;FW_sK};Bd{Sai>@D3TVz0^3Bf;XLb4xC=x z3j!L_+(a=IOtBbrUL;x%SIyuZe+rB}Lg3%bLo*}75}ho>!sMJ$=j$sIThh7d`t^hE;E9(qO78ke-0!r*x4M|06#Wm_C8n$a3 zn|P&d+Y%zZs_0qf8^Z=WvW)#>PBXhujSgtrX`qI&T;Q_o{Jtzzm1EL?@K~X&39MQ~yTi~4A zw9e`QxSDbvTu0&s0PPq1yaIzBm(vA4>(vJiY;f&BR8_DjTw`0@xyPD$DqgHmdAhgR zjdc%9HGvsGR-nlRuntf}Vc@)1iEHX1DJWMqJpofH22(&fp6Dn$1G$Vs94pJ z8&BQr&eg0IXz#rVWr#;WI>w+`Of$H29AK-|0CqTl{wrK=IQOs&!E3J{P8yhzVH!ks zkPrCI944Z`J^(VojnfDf3&W(Q3r!^k1J9&?6=;8Vht-@?b_1V z4*@Kkc(=5!Q1n~~^>h$#WM*?_OQT?1GO)@xbG1OjB$%7%9$T+H$&)eoRN^Ef*#b&C z^Gt@=H<+>jSJHRkHC|6}^`+16gcf5V!;+EVc8lRyuDy0-juGQ*dGKlx*n#QbR<^o) z&>pg3o___DWE_` z8KQkrDvWe}`dp=K=32J?gS&NXcCwe{wJh6ozGrHsvTZ`82drt!=Z-PD`?HMQ5IaBy z%E9fIUZp`0t82@~cIeybe$F3Y0&yAptl4V>3~$(23-4*hTwl*Wy!2hM$2uleh15bH zIy6q-3YacI6ZCTqr;<{`I6Be+I@le!SOI4Vm;z;*smcqL(Tn;aBdhD(A3WvJ#Z=n7 z11#;cr)1->2$egY$!u&>2f(y<@HjzXo+q-E+b?Oah5ho`ehdm}VN#}n=0WuXO$nTm zFNi5Rr6N}k9cTk_H(-GG1VOC#@-QbmS-27uup3IDzRA|K;65hYE%57YgoP(3UDzxYk4Bb1=AD0=(&cf<}p8s#pE ziIpVq)FQTepi~5S8dDC~&HWjp)r6VMb5Anff;9WSwPvVIAZJistUAYrY(M$xc3Ug$ zn`Z39FwGSRW*~w@oq*s(X1Zi7e2HP!k-;VDU@@&T=0(cPnHmNMR0FY>9qAwME+$sG z{Tle@yI`yd27qg-1OZm`jb|9W3~=r_43j|Btqk^cK(>k=!0H4J?wQ&pn(zbWgFFnF zBLF=gDJuA6{>G4zmBnIo<}uEW0p;2G)&Z-o804`igF%8Nh%%7MLEWz1$~?yKMa2V%R9lfUzThCD4UIu)u1gn zpt2d-?(L)PY$B>_$YwRajuaWHy{o=_3FB@Huymj9F4hgHVRR+BjyEyD#}r3qC(8$@ z2eEOetXyIP4bpzW2F1|4sD0Ak9nGs9@MZFGl^F?OmLQ-sjm?CvY8|-xe0U4p4P1TG z$o0&D&fEpoQZWJ!T>g0l|r&5eacsqLI{Qo z$U1qa_S;IR1|}Z+jP}i3Tz|H&_U{191^h-198e)0?&i|iQEG4853Vwb>*)poi=b-8 z3Sa_rMLav;!{)3beUO`h)gtY?EbU~(b7z87%v_WimRitM+Gt~UOn=IUuhW?=>Uw<8z%ggpby9PH$sen>_Ix>n0P?u+9BgqDU@=$1gHnO zgP6UHe?$hDj_Xg9_V~oE=wt0Y$Q%05Ka{6E`Gjsb_oDVn+y8v|F)?jDqUt+vQpH@2 zfDa{X5Oh-!C#_}$ja4*u;bCxJ-kP@Oft|h=%<=4-;L1VJbf~Osg`qJm^S>Ut=T&IY z^olv`@V~z*DmAdvs~>-#?aQiOc4I-#42dp0H_N%kd!m~+S#t!H@UWZwrA~oANRSlae zTg*|Fbks$!JPJJ)lnW9x8%1Wf zwvI!KTR&7(e(f!Nv%PDMV=6|xl0bWYg<9Cd{W)oXp;5EU6U*hpJ4_F93+(&=oxPr} zHR+F^jZPOM_&Cc<)sYyvXBcZ*2Ok)$RuW`vu`*O2s8$Wj|GaVdEf219uTi!O->A6A z<0DVLvtb+c*Jpnj?W5McGz~LTHWPH^pTU6GB<-a{m+x+7lzm3T3eG4FT)RWYfxKDf zmeSQtV!8UH_Itgc3qM;iz@h~nGM$;qGzJ3K)mnKy-j?IV4&I%>eOf>~CEFGEVU8W3 ze|0b$nMg5Qb?BJtDO?B76$b)FL#pYZ_)nfP`#aCv16=9l$7i#d6aY^Suz3c8CS_O+ zvBVfufr_>7C*R!b>{g8du(n;Rt{~+JRCr0?BVJuU`YrPQxn3@{Y-C)vM)P@*=2j-?pT%_ zN3VU4B3rsmS!K!1fZslEZ&+7Xj&igYLAzLiK%)qzW3Q-NxmiLbCu=#wU~EOsh$+`S z(UGy4r>8$aD=3DxEpPI{3*eSzcP>}}`vq$vG9c7K4y~=nPt40)@3f~L@8ukS>-H&^PU+eG}IMy7m*7 zCB{hAIJEGa51hK2HA!8>QWy_0bNnrERcO3!Hc4;8*?UB>hhUxqR-h$N>DqKUi(Pw= zwi;#LvPG}*MqbCrWwmpLZv(bFpz3Nw`H#ce0}9dJA`FF@AXv5QfKmzi^dK1H#L0T7 z>s-KqEn2plX6RjYU=?Fyt{0y>^B7c~>co%07D9$@JP5o3v3+g*H66TC0p6VY|M#97c;0p!MvF$EE8a^mvicq2Gf>mH|W--2nI{- z)*On9BezQg0jx*n3|9-K7Dq!-MbVtj!5E~|4?;x^TIyUcmx-O6CI<*KD9JQ!W5TQ( zr6bk$>M_j~^pfE7Hz9ZPRDmP9Q4K2Wk59g*=Ih6F=c;C@Z=@@!*$h&fYcTt+KBwEE z&~b^$^8ZEYxvbgjAtv2+Q1VVNfrD}|CjwkK#K}Z~u0J6bx(CQ&nY4ind&4uDnfpnT zV7ov77YT0Io4{Cfa^l*DB(Y2xH4kWMgU-yiB6{ClxfCjpZ)-nDKXXqe4C4o^g1GX; zP7i{wdF8O-=>*n7iUf<~z;r10qn)xVjAPNBI;s5`t3CHTuH3%q=D~r}FQaP@j|p&$ zI!3($e55e};X_ZXUC_Qc0Zqj&#sxY(-g)+(b_jYS1Vs1j0Sz_o15c_>U2ctXh34Z8 z7G%d^m#gxp-<0p5?X>_CY<`)YGZ$E#dlHIQ8Q2EMw3iXB0?^1dHDWa%m^P0BwFU%m z9N&avDGvb5`ioTSievh1+lHtGEo~Yd+E33QJ3w5Via`J`my4j~-yOLhVse1N1iZLS zMHv%IN2pm4GtMc;SDlIDP*=Hvp#x}4H1b#*-t=0TOBItM69h1Ybf`q(ts5}F6-FLV z&5i%?=wW9Fx<`!#T6sY>IH0a$(Kp`CPd1c_O_g7^Iu(rS$m3A8S2H!sh7J|pv2ly* z6MoX%%2!+URz+FFZ=cDWUS-jFij~#euvC#VFzdSnb@Q7ql^nS_NEKpYySkLJ%IFDT zJ`OCrq_x9i_^(eKVkS+hbdr8%ED$XMe4~TY2RO=JSH|x6t71tDTj!KFK&_@7q>pP# zZjpMb?Zm6wz%7mR5)gz*Gf-M3~J7F%ryOUIDZaj_f$`4s=gthX7nS%Wr~5 z>rxJiEw0cW2XFG1Y@f0Bw8 zzQrn}X!>vf3@tDWN(a#cNnISgiuNAsd1D%yfdgP!{a~FSpgj+a1M31^e;f>8L{>oU z%&Rq!fEfyGEQs`ja0Z=w18m`dIrwDe=1*Tb&^q-9m_;{pg(=bR3MOE3=B{wG0Ix4@ ze|qUoh%vDUU-|ehC&pmy{82IRgh1|?+w|~8z7eYHs0!Xk&*hsLoi9AJduKZfy#;iF zx#YpIaxH2O<66r+08al@`+Z|>LKMf834z$h6(?K$6HGH#_@9ytn*;l?vO{G20LR#FK#(@~{HE0i{a;n`alhI{oA1LLXL%)BAb>_AoY=+}u9KC)gYa z&>qImn4t2*#Uz{|9CdbG9k8^O2 zYS4wyji7vxU{%DDby$Kw5B#l&Ph%!Ijc=aaWt!0gjE!T4BvBbuzzIHM){sQeI_xx+G>IM!mY{B0s22STT%rk(35>w_x z8^d(79W<+cM|B0z&7xd9xYe*&pysKPGDA95lo{dK&D>j)E>1b0C0DYx%b0NdZDuwS45ykZ4>@Fj?{ka#!s#aw>{-ejlq*l1> z5|24+Shg{{(-}1#InWlnz=p)s#wCmsN(3==;#f>8onXKyD>)1wXwO)PL3k%moq6&- za8-^CmRVajfPMhP)Y#y(LqH!tT$|06r>f6+2ZRoP(RPVoSa|DX-wf8d1ZELKo<693 zGs4r+bphV(45T1!T-`4vw+5J^z4f!LnG7^N4^? ze5EKH-TAxh${}vNm*eRt){NAsIMKkpI>1&g92G+j1Q%%=Pzgx8_Y8nyZ4WkIJX*Ax z?#1?PUa+(kNB5XUYK8~FGE7H38W$wjm&)~lGfzOy?rxmhPxqWs{)7r>zX%S;BY=NXB?jS) z&Q^r}`;U%EdjoHi(;h5mr~SRjX{e&_gPI9qc5KB^ZArJZ89I(>NA`%v;C?9FMKJ&2 zk$NtTcB}~j9R{Q+;bLJ~N$nwBiWU}^uE{PR|j395gIT$XB z2UZSrbEz09E&{g(MNM!OXb1i6^+S6hwyTp>g|WnZL5ub5MO=BL-JlrI7eC}N0TtPd zLg>fuWlFn(d>PORkPgn4Ne5{EX~&^mzYC1xN#;H}`cL0^^?}k|t>+$viOJs7XFdlJ zYt`T-8~N{#b+AmTNTQ3nm?m_-87PCRBF9(&;!F&i|0N1tN5OwS{w~K06!~y<MA4pUcqg>E10h{)bOFw~h1SNx3n+7Z))YZEfs-U-1R}-PJm}cF<=r45+-ZbP!s;h_EoOwT_-RMCX zWWc#6PzP$@cm)hs%?>cXB_@bRAlN{w9XK$zCsmZKe11||+!J8t0Ln$u`1~DSl9co8 z;-IraxP%F~pwNYs=YJU#)9L{wkf|#0D$RouZ69khpSs`=)nAnp)Fjy?>EWPqa4&H00Wctbk+x`QxRrB81I(|`9F`lv zz}$vJx6R+bp`(+h269{rL(|Iab5vYrnip3-+g(g!?_V_#5I z2PejKQa)z_>e&2JMcXt?u3ta$CPde757P02+LLmi*hhGEY0rFE<8**o(3j8A<=VYz z{6H<#(!U)2V4bTrs%trsTe5G7XXj1~bFdSbwp_7}r~n|uV8j%Lv2SKXK?I`Cp$P7Cnr;doe_nE_x~AYe?ywO1gQlLwh3 zz*_AkPH(rgcyt|!lZDVb^)?h9_wzTv%6-7N(}Wi6{m=kzd+oWeUdA6H%mZ5x{hmwb z31}|ezaoucf9>0Bc1BYYqGM1x3ud^A3f$?v01HQZ2rhYo+R_DGPiOWDDs&BsJ5NPd z4~vz1aFR7k!Qh-k6A0)6rHQ&;&k<$RcbZqCp2qS1`nV7-TmmEc}Th?t*n0#%4_C~MF zB~BxXcN}zwkC_eU=hlL~>@@SMecjHb$*fQ5exI0d-Q#{1z;sQ9UtMuHvVxFc4z6qvebp;sjrL(NT>ce;Om;kQ@Dh@2*(mf23bt7}5 z7i7;sloc^u_JM*pud?f^WT308X9y>2qqv8PN3J6S1k^G2vO&qxX6O?WM@oa4`hxU% z&O)G)y>0XNY8L~w>gWa@5t!^`tpVB(jZb_Jngb}`2v!K96JtPt>;CFu(R9u!S%8*4x{t%( zt9ML7WB#zr7^?}^x4=AjeRG9Lyl^jbmm2{bnPDc-nIe>t1rk}>0E1X(E;>_%rE~KB z?Z(X)nb8SZ{*JOiK)cA0!OdTq^BxEozIh+`S3h$#ra-!i5xfPY7J|&(+1e_f8(ZjI z`o+EISh0)q;wdj2yzn>#XC>IHPNN&trB!c3QPY5yu)(|p`a-oIhR;LY4&p^s_4qrq zvM>Q=$ZeR`+dj@i0mt79(%VqD7G`WUSY^?hfd7=pT8OaL04`v_#mDK9WJA{sglQr0 zD`nj{)0tI%72Yk{r%HVmuF0)_FzMKCHR*+h@RY2O!B4)Un0wti!YT!_3Ji)U6 z_u<13fPaYvxHZVeo--9BWi#YDF1^B4w}2_@V)Szy{rK_3ZjMZQ^Ei*!Ia{%xmm?KO zu2`d0(=75Nf9-3>(EcmNHe{Fh;otO%+p5CRZFAKhreLr^{GT>;%+ah?BUgHX#z~Z1I3x2yN?Imv%28{ zp&X;?n?dD10$UgDbz;)scoI2n0T&G5t4F{_`sW#D9-Y|)lvsy@SAz^gW8nR!)L{*` ze*0_6(6b>g8`FxEG2uuB@>;ocG`RYzdFDu86VDb9wg$did(TvFDvJPMxMC6*;`$z@ zUb}gL149rEYIV*9w>fn;c!WO(D`?R*DB5MA-62Q&CPv1LZ%wyzyWP0 z-~KW+dXqnK7!Q+8W2?*~Sp+JYOyAp3GsSW4EH{EXq}#BB4aUVAuf@iG`5WaMm|Q`< z2fzT8v`hppf+O@j*N(YM$C7+~RCS^4+2ecj=Ow-@20^U@B*VF9MYoNM_D@7Xh1cG2 z=^5<%&+q>oPBqa$@*Vg1*BSnyqy80#f9N7?@IUS)oRHUc9gCvvh;if1=_mIM^Y%^M&ICYpQg^hc z?wjVl8;}KBuAKqz0L{iOg9F^XK~ztONd4v9^3~0D>C*XsxhzG^3ZLVgk-%E9h3hL$`6{{h*9^G4aNW zZHJ5r&Oem*aDX2uQdHqpBGMqDx<$X`TvZ%{_lTuOfOHDyU85mWbbmD#>jK8>oEm11 zmkrlsb?bSF&Q#9r^P~s0UoAF6`T7_4#79G#=Fh*P1Thds&?*DYKR|ef!VFdf{+4?YdESfK|8Tg68hO0i@b%L_aEz|658?2EE)fe~{92UU zm>y9+zuQj5SK8oqAQb^;}xx*8ViK2pEc+f@_CadY5gyuSE#Jme!#S)q+ z_0V7)gzp$S`xqn;5+D+sVHyC@5y0+V?F-5u-_In`5y1~??BsPLC7Um8lHQU0fdbJw zO$$e4jmaMi%0sdd|LzrVaxS9U#CC9_?$t?Ycdxp$O4erMz%3kEoJ*ifPthsEAcjh? z$Yq1m7le&*N7f?$kQL;_GX~~7!aT@e@~S~L$b;Y?J%Cew=7jP!OWq+}%zr!bev@_L zQt-6tOn-j=82Y+EtoV|8f3>A{m?@Lh3S$Rw+22CUQF5kAplQcwZUId3ydA9DJs@w84liBfdZN}FX!}7m z8F8j=*^a4n^8OhHI-MIcn)Erq(ll@g>O%I}Q4gjD^OF<<~w4e)F0nr|_!K(Nqn>B@Nl@eKWM4>C6e1zRgIfq7>~cR6sh0M{)6 zvx}_`&~8r6l2sQ_lY8RbUJeCt%!$o}Fccp!QVNnGV)+MmFY_!`djgsCfM?9+gU|7d z0JqwA@>#xeJYy$x&Ksy(pB06N;_ZWN-4as@SOKZDSM|Y79i)?kRJm5mjXdq*_OwId ztK&a!=ebkFk}3>=i;!mE%uG0J_5-SzciubEzHG9?3S+Q&>Bu!_UVA!X$(3KRRsrTuh(Fk#q>vJb&wdETW4nO>_RGsY z^2ETS3YeTJqH<4%j}Ozo0-hG2o#i^=D(&jCTnAWU=B+oM4vjcqFLqXVW@>s&1mjpK zwjEcV*za4LnISE3?cM` T}c95yfzp6Qb-s+0Y^MF`X=P>-0}cqCJ>1BZ!Fo3mJDynMLK-wEDGMR?JJc#oZsN?8uQ6e}P zfO)(~*K)p7bhQiJ6b|B!<)F=XNNNYQ*%CW5?s)u9LfO1wu#MA z?=?c}1T~3eN<=HoTd~dPszRmeZ!dc3AZ9>aJP+H<4d@*PNZ?0PLyI(O)lxi;=7e)3 zc&c<%tBDt?sljR|J>b!znU1OVq(kfEWo8tVUai}e+tKE7D>(lXNKd`F0Bx;YnoKdT z1m1E1+=J;e0V)HPavuhSCmZqh0MBL|+<1(wFsXMg}Yw$3d~?(c%p*nAVviMLEq)BcwOeJ+tD zJ6a(>yr0t$^fBJMKwo?hEGO23U6+0q4}SAOyXx*?`t0LF{@`{SU_cLuQy!D1=~#>d zBkU2yDpyhQ{L7<<*U+>F0WoJer7UUCSA%N1RTcRC(xo<+MlOUqsJ^Bje0*H2kyVEJ z@Th?fX*G%f_X05zGg{gNHRd%%g~drP)(CYPw2QbQ;LWVjehM|+#CdB^pz`NAISt0l z2-0!>-5_26rZUp0{T`lcC{t@!+ty9A*KEy;h4Zv;bEEe*|C(=YRoO`ICdy{ub!+`h zs0EOfnJ+z14FZ@}N`x`gi#9WoI5^;QH$x z5GB;<10W@ai{6-v*Zg|w`D0=Ow-k-*0s|BQ>eJRUjvUytD00gmJ7NImR7z*S;#t6s zAnoU$_yN)~^zEQg?Y}aiu)0^1z@k7ZU7!K1>%sw=fmji~5LHVB;C-x1f#a!rx&pS3 z)mD!&^K6y^!BtXJ=o>+y-4?X@&eF4wORrZeWQOR%xoUFwz?2B#bCr;4{?NLbzℑ z_TU3Q(c76dYDIKx+oEq&m2Tw47WZ@elNo@)ifJYrTdV9Y|n#kWMp@$v$wktL}3 zar8Pq*9MGNSgLc?7`4Cn()A7K00DEE3r%8`4=2gN7Mju#Eg)C~%W<{((vjc=D26Rh zUSN1X`M%o7(I&#eSy2~G& zRT=@JuQxzC)K*5_kTVqsJwMem$SUr@Zml3&r+VvQJNtTtam0M^_}$Q(nNuFCi^EISj=5wl&eAVOM2!)QZd-R8ZaQ1ZWH9Vdh2?ZXglPq z@DXM#zy;8xCm2HvOlU8*q67Et6bpD8Xn;QF5hm+QQ5r;>_H>3j$k|-w-0^47)KTFF zbxvSwA`gm zPDl8ke1I2JAX`OUdBobfO=SuXJenmN1^@O*D9izXlVyf;N*~?}CSZ`XCx|9T&|Sbr zgb|!W035h>&~1PLG!4oH`B@nDK>&%_U}~|0#ziOw1ne63J@qS-gE;MoZK>eAfY@Am zpb-KXRRP&GV$bCU(0(pBGN}C`{o=zgXdkCGXtxImt+EYx;aHLAC+}K>lE|fU^G9<~ z1G+~!SB{n=2RJt7o>L_+*4%kpq7xH@!(XOz0RL66B+wA33gph|Br=Sgy!I`l9Z&@M z+Ed1@?KmYb`fT^{r}fTcbEvK13$+kK*N<_KTVYF&>^%E@@Z_NZ@TTGJF$nGJ^c;GE zMId;|mIuId1ORh^Va=50$+sbQgP1XZsZ7wYnXbar1CWRo%s-BU`&g z&+X7{-lB~c9RKU94)m}w-rT0LRyIxX9XSB4E7Q@}#|%mWNV#}Uwg6`gfU&4Pa5vDr_T3<+ z5Hk*7ebmg0&fx+hjAsH%cY{fA1G>VfE#UO3lT4r0K#; zswpha;$+n^nI9|bU^DR>;9F5xQ@pXk99aJ2;in+hR@>YSCk}PVHa7aE?tlIM9V##l zjKo%S{^jtG(GDmTqniG*w0OM-R3>vVJNXi8FHrl939~m3BsldhM^g3KTf;mBeKRQ? z)sc|dL+x30oJC3R)psGLS(RKa+l#@thG?$@Tnbf{4g*i#1zPoMjyv~Mpb8cYfwp#e zrDiNzCPwm&*UrBW33Pyl$H6=uJjC+p+mOtYc&5h4npkH!x^DXVs~5!*raeH6={ISL z%xWFlJ*Jvv(aIExYRCVZ05zNC7?b6V>w$MY&}GoJ5BE=ie9m0w=vs^gcK3zagZd z^Nm%g4mY0$57Ld%PP(cx!pMAMgn;cJ9v5J^O*QA3+!VAImU9nCFL)ypTMaV74tkz+ z`9ko_f-_HfUc4uakr1e8GTK#4_QBFCN2O=bBJKPUW{1J#&@42kJlzeloK@yWBXAi>wv}=!$fnwiy`N+BNr*ndW z7G2|!XFz$NLFGn2=qmisfJPDE%yMoHxYlrixcRaVcq&MH_)@U}q6nVq9rnC6UI zyJkbC3ec`qLEqXXIy(|IpF%NRbz;zB{HnMvgcofy->Filzh-QoD*EPtKfd+`x_Obz zYgKgzoe6&Gng8=|zdoXzXb;0j4{aZA+NCaC6!aC44^oF_z|CjD>YJ9qfjSnkAk?j` zv6CM2*q$5D*#{PRtW|dRKr7=6c@EPgWA`+cz~F5P^LR7;FNiLMv}AkZ`kS20yx749 zF5kJLH;~Xg^p=#ibWptxu-AjEyG)z6atvo40&DUN8UOIde$W^&E`b6g1n+{Xq-haI z17KU)3a-5n1M-&GqRm(!C04-r)%Gc1Y z9|9|*TQ0od1}?b%R-51PH=VT?aZPp7F*9XNYGUd*YGItrMyCp07SBHe z!Rem;fmp{jHvhYzTns#?DG+FQ)q(Bj5z|!o#-WdY_B^<=-0moUs71PJ89iK1SVa5E z=s0`twoy(Vs9-7sL997}3hq_TJV(_s&zBk=z&up-pU$(>x$dAwkU7ZG%%-r{&2n=w zfMqzw68ykE@9hx4RPB_~q`l(*_qX4@GtmEccRx5A;zGY`PnCkSZIdl@-t2DRd6slT z8ni|xR&@nSNDWN7nQQUkYPsQ@mG`CBMXlaF8`6%07+7;D$d9*f8Dz2pNg}{mMXSh6c29KhHKKttvuO)W!x*}-iF?dWC;OvGNd2MnimphSa z=JMt7(qG>uD2kTI1tY%1KL27oNJV0mj7=UN@lIc3mV+-aWuKd#eme`xY z#_>f?-grC7z*Uc13C7Mm6vo1J0Tp8nhbZ9o(V%J&KyOWe4uF_8z8>iS^{h6|Iq=L6S#*NY z${16jq{9|@?G6f{3Pe}-fU-dDKHI>RGm@v%$I$}RG6K!yLg~>Su(>v?#>>Mrd-zBf z`1Rk6HF04at^w;4^RR_TI-)JfCB`){+;yKP8m0kjFMv*F1o~8^<+5LUU`N09!91G# z>4}~VBRIfwDi@7@@(X(2xg2b}-R6`0Y4TeP1>Xv_cELo4aNaFEAdln~+5`?vo;wsr zKl#ovzH~)jYx>w{FK{t&t=%)sxo0?mE#=43JuT)Q;B1Qk4?O=f6?ZY=OM{G39ep#^ zMfKWK(sTWY$hmlxqAUoW^EOmWUV1N$(P6sRcI=Yw9&R}V(r)=6hs6#aAK|ma*?U*+ zvv6Xl0lyrPA?FT4S>49}?zZs@ks=4x-b z!b1bIBJ3e-RSI)X+qOB+R_rcfK`!r00ly^y0v2JJSvPnArxX z(dDz*x6><6tXnzH7rb9A5j+W?9Lf`X`7oIGg=Zan(i!?4%@d|UWVQ}4wJLbM<{*n& z!K^^l6Qw-F=2MTM0=q^&d?p`Kd6t26mL66$y|p(0Ech=k9)*au1at8Ps{;Ym`CKP~ z^|C_n1?5MmJg8ut#uk>Tj&l#`DwIlZb=P9bD9ttKI{%_N;rkfy=Xs_ReY(BG%-;fM@VVB2`m5c39b0qcbGtkNwV z*`jh*K#bl2g-7SIRFm=LPaxW&=t0hN=e|9;rM#REdVp!K0cNZW>~G*9QZ3HB48gQd zYXt*Xfj|VP*H8&=;gp*Q!Qp@YOR&Zr(PYPo;`;LSA9aQY=5(YQzQs`swC{s?-hnv2<}^1`5O z0zwMMAK*Q&;|9}^`jS7IV<4ibHP=rzydZnc3_l?sW+(7MGTxT zFw2+%EG(eXD%GPcMRf2mQ-PtiTXi_qqtP@Upr*?38|j8fxp+HX#*@l# zpBClod!8~v^Kt;|1C@aW<9913pz+1eO=~(s1_V0s&H~s8Qn!j%o*5Mzc!y}tAOA>{ z&@sI$QnV(zi@vKL98)eSSl)*=!yOH}bT?RO*fO}F9ioUA3e(=SclO6QSyvknXbmpHC6yX-By+mxBYwLh1sM=?LOLPl> zFHNYkp)q#--^8j$8LZ~T;#vcBfvjZL29*EJuqB41_sbw-$a3(<2rW< zQ0EKV(0&GQ8j$S5WZC=xih%(9LB*g=$0vz-$|K2@alLRq*w=SG2<{o3ZkoDmC49QB zB{?JC6%DSXD!-tu)q-hHdg4^0Ksu6&T1?x)T6^M<_OecW11KA;m&nW*E{Z@3uO8nr z00Ycw&^Hf3SIG>`k=eMCuWFeqFt-KjeZ$mrRiMQTzWJSpHnHG;>SFAm<}$`B1p~Liz1v0g%7A`C?9+(H%H9EH~%A%@t`l_L!3{&4>?esqK2ou02{YS2Qg6e z;danAqqAvcKpSx4+_;VQ^bjry$)N0OT#|fTV?KEIbfZyss~zYQNf zBL<762qOGBL7RxIjnpI+Qu?s%DLb zfJ+>3;b`Mv)E?2xmmz1?JfsD_&N-id_VN#PizqyWpb26C6o`A;Hb}08?%Yd_9vUl+ zo?xiMKsr>l%oxSu2a%J^nJ+Se7HMMM%GE+zKo5u6gXQ+Ge)0~zYt4cJII9{EbJRPd z^vhpC0;VaU!SxErJ5Pv!Pfxbb0vH9fX`=rVItNYuV_g?-96ng2K@C@J`4lpFb!pUtO+KjZo=~#eX zINxbz7Fr!n{;aki;jB&*|z3gA&H`ckL#K! z^ot6p3Z>X%MReBBiZcA?M_*&GYSX=jHdqcEh=48N=5vhM3W&*?3<9dz`N_NLtgJvC z^t%eUZZoqC6rHI*-3?~8#v=lF)uQ|tm~}m;2xuRj`Nlxo$N*bn3UgE{YS45rN-RlD z+`X_ zysb`vhCe8d+3N;c0&VwCAL?(+P34W-C@2_o?R799+8vA~R{}coLni+ui211f&-*k> z1K=h=18A=)p7za)vsExLj@2xL0y<=P3=D7qT|GDsZoWOnj~)~-s7yD<<-foDgC~A} z_q*{dt^m3baIMQt8cB17X6MkcAhzL?4?=0r24k*`L@n;t&I_Gk0#sC_L3dVy<~TWK zBe3aps1~X)rGwXB;LNkV%rHODWEpn`qL};hoLmcN`Z(`7=%;*hE{j+017oJy@!F_; zw6#=hC``ebi4<^V6T4WIJij+6mDS4Z z02pFApP{j_&*7$@sv{4L&BjH3{=H9My!-~Y7>TOg2O$+SYSoy9Mr%_+(`=Ek94e3t zVp{Af)gI`$`Ark))Hz3>{TXO9PcI3x=3#VnIz#2)%l+n?mpNa6w?=E{4_Qmg#+0Kl zOn>!$UVxAGC8pjhkKS@D^BvkNYOv7(e8vhf%;Pa_wxW}X>K@Z7HuudR7ooY_#l_By zmy3Go<#NP;C|7G~ie_W;;N;a;f|#^!DP=n#W*c`Zm!JVS@In$lBaM?UIf){C72^C) z0{kP&Cr-T`&(hJc2U-v4@+YUe4#Ca=(~WOUMXfq=9gbF7{HW?th#oTUf=|^*Xe!n#yC`Z z(+JvLzA|D}1I7M}w`T?*lgGos`_3H!2llZF1qFUMbbrUYi&=rmawU!Wjtssv)QJCY162c=!-R;1|y`q~~9fzV<3e z`aU9Y0{o+2fw5wM5l|-azDYpSPPG7atj9wgdDZuK_DNy<1!Sp-hc;fv54rnE~2;ZDD~W z;}&R2-Z?Y@W(Baak1iWL^Nciw-*O>^>Dt2^TwVa{%M1z+!Np`N27Z74`)K>=HY)>E z?R{>2myRt%XQ2UmJ#)Ytq`@2Y{e6z<`Pc3h1H)p?BHSkJItc!yA}F$EzbTkvots=) zO3v`$Ie=65h%{C5fO(^sI*cNjDwAKn2OhwJ0(c8$yN4%QEWPqb1mpqkled~aIGji` zX?{N#z@7s7;8UhwwD!TMC$#Mx*tKVwqRh$9p2(Uzdl1U%F8)!*8x584tJmLF#l|JW zHQrEKIKb6`6$*U#x=QJR>hpJ?oz1O61Fzj-@1Hi+B-?;re4ic$+%1QrIX)oPtu$RU z`$is3HN@20%_6l*Vk~R5-=J{|wsHgMZX>7=BryK*&Uc(`3gv$~w^{OYjN2Q2tTHCT ziAPGnwSz#q@*vtb5IUW;B>eWhcVeqTKQlMOJTuhMj1_08eIy12s44kx&rOG^By`_; z_8{09oD07h|NMwA6dlu|74L8P#!NVtTSh4Z3A}yA-`Gxe;_^-p0=2DnqTF)0|DvJ1 zo$@Wd{kAjVz!m0^;Ms;aw;lh$wVS6b^}r6BjeoFDVT5UkVo>TM|FNgEL%SJrExv8U z-!d^AmUUuUKZ?=%w(R\n\n# DataFusion in Python\n\nThis is a Python library that binds to [Apache Arrow](https://arrow.apache.org/) in-memory query engine [DataFusion](https://github.com/apache/datafusion).\n\nLike pyspark, it allows you to build a plan through SQL or a DataFrame API against in-memory data, parquet or CSV files, run it in a multi-threaded environment, and obtain the result back in Python.\n\nIt also allows you to use UDFs and UDAFs for complex operations.\n\nThe major advantage of this library over other execution engines is that this library achieves zero-copy between Python and its execution engine: there is no cost in using UDFs, UDAFs, and collecting the results to Python apart from having to lock the GIL when running those operations.\n\nIts query engine, DataFusion, is written in [Rust](https://www.rust-lang.org), which makes strong assumptions about thread safety and lack of memory leaks.\n\nTechnically, zero-copy is achieved via the [c data interface](https://arrow.apache.org/docs/format/CDataInterface.html).\n\n## Install\n\n```shell\npip install datafusion\n```\n\n## Example\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "\n", - "df = ctx.read_csv(\"pokemon.csv\")\n", - "\n", - "df.show()" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\n" - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/index.md b/docs/source/index.md new file mode 100644 index 000000000..9ce4d4aed --- /dev/null +++ b/docs/source/index.md @@ -0,0 +1,73 @@ +```python exec="1" session="index" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + +# DataFusion in Python + +This is a Python library that binds to [Apache Arrow](https://arrow.apache.org/) in-memory query engine [DataFusion](https://github.com/apache/datafusion). + +Like pyspark, it allows you to build a plan through SQL or a DataFrame API against in-memory data, parquet or CSV files, run it in a multi-threaded environment, and obtain the result back in Python. + +It also allows you to use UDFs and UDAFs for complex operations. + +The major advantage of this library over other execution engines is that this library achieves zero-copy between Python and its execution engine: there is no cost in using UDFs, UDAFs, and collecting the results to Python apart from having to lock the GIL when running those operations. + +Its query engine, DataFusion, is written in [Rust](https://www.rust-lang.org), which makes strong assumptions about thread safety and lack of memory leaks. + +Technically, zero-copy is achieved via the [c data interface](https://arrow.apache.org/docs/format/CDataInterface.html). + +## Install + +```shell +pip install datafusion +``` + +## Example + +```python exec="1" source="material-block" result="text" session="index" +ctx = SessionContext() + +df = ctx.read_csv("pokemon.csv") + +df.show() +``` diff --git a/docs/source/user-guide/common-operations/aggregations.ipynb b/docs/source/user-guide/common-operations/aggregations.ipynb deleted file mode 100644 index 0e7947b33..000000000 --- a/docs/source/user-guide/common-operations/aggregations.ipynb +++ /dev/null @@ -1,436 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n\n# Aggregation\n\nAn aggregate or aggregation is a function where the values of multiple rows are processed together\nto form a single summary value. For performing an aggregation, DataFusion provides the\n[`aggregate`][datafusion.dataframe.DataFrame.aggregate]\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "df = ctx.read_csv(\"pokemon.csv\")\n", - "\n", - "col_type_1 = col('\"Type 1\"')\n", - "col_type_2 = col('\"Type 2\"')\n", - "col_speed = col('\"Speed\"')\n", - "col_attack = col('\"Attack\"')\n", - "\n", - "df.aggregate(\n", - " [col_type_1],\n", - " [\n", - " f.approx_distinct(col_speed).alias(\"Count\"),\n", - " f.approx_median(col_speed).alias(\"Median Speed\"),\n", - " f.approx_percentile_cont(col_speed, 0.9).alias(\"90% Speed\"),\n", - " ],\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\nWhen `group_by` is `None` or an empty list, the aggregation is done over the whole\n[`DataFrame`][datafusion.dataframe.DataFrame]. For grouping the `group_by` list must contain at least one column.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "df.aggregate(\n", - " [col_type_1],\n", - " [\n", - " f.max(col_speed).alias(\"Max Speed\"),\n", - " f.avg(col_speed).alias(\"Avg Speed\"),\n", - " f.min(col_speed).alias(\"Min Speed\"),\n", - " ],\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\nMore than one column can be used for grouping\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "df.aggregate(\n", - " [col_type_1, col_type_2],\n", - " [\n", - " f.max(col_speed).alias(\"Max Speed\"),\n", - " f.avg(col_speed).alias(\"Avg Speed\"),\n", - " f.min(col_speed).alias(\"Min Speed\"),\n", - " ],\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\n## Setting Parameters\n\nEach of the built in aggregate functions provides arguments for the parameters that affect their\noperation. These can also be overridden using the builder approach to setting any of the following\nparameters. When you use the builder, you must call `build()` to finish. For example, these two\nexpressions are equivalent.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "first_1 = f.first_value(col(\"a\"), order_by=[col(\"a\")])\n", - "first_2 = f.first_value(col(\"a\")).order_by(col(\"a\")).build()" - ] - }, - { - "cell_type": "markdown", - "id": "7cdc8c89c7104fffa095e18ddfef8986", - "metadata": {}, - "source": "\n### Ordering\n\nYou can control the order in which rows are processed by window functions by providing\na list of `order_by` functions for the `order_by` parameter. In the following example, we\nsort the Pokemon by their attack in increasing order and take the first value, which gives us the\nPokemon with the smallest attack value in each `Type 1`.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b118ea5561624da68c537baed56e602f", - "metadata": {}, - "outputs": [], - "source": [ - "df.aggregate(\n", - " [col('\"Type 1\"')],\n", - " [\n", - " f.first_value(\n", - " col('\"Name\"'), order_by=[col('\"Attack\"').sort(ascending=True)]\n", - " ).alias(\"Smallest Attack\")\n", - " ],\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "938c804e27f84196a10c8828c723f798", - "metadata": {}, - "source": "\n### Distinct\n\nWhen you set the parameter `distinct` to `True`, then unique values will only be evaluated one\ntime each. Suppose we want to create an array of all of the `Type 2` for each `Type 1` of our\nPokemon set. Since there will be many entries of `Type 2` we only one each distinct value.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "504fb2a444614c0babb325280ed9130a", - "metadata": {}, - "outputs": [], - "source": [ - "df.aggregate(\n", - " [col_type_1], [f.array_agg(col_type_2, distinct=True).alias(\"Type 2 List\")]\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "59bbdb311c014d738909a11f9e486628", - "metadata": {}, - "source": "\nIn the output of the above we can see that there are some `Type 1` for which the `Type 2` entry\nis `null`. In reality, we probably want to filter those out. We can do this in two ways. First,\nwe can filter DataFrame rows that have no `Type 2`. If we do this, we might have some `Type 1`\nentries entirely removed. The second is we can use the `filter` argument described below.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b43b363d81ae4b689946ece5c682cd59", - "metadata": {}, - "outputs": [], - "source": [ - "df.filter(col_type_2.is_not_null()).aggregate(\n", - " [col_type_1], [f.array_agg(col_type_2, distinct=True).alias(\"Type 2 List\")]\n", - ")\n", - "\n", - "df.aggregate(\n", - " [col_type_1],\n", - " [\n", - " f.array_agg(col_type_2, distinct=True, filter=col_type_2.is_not_null()).alias(\n", - " \"Type 2 List\"\n", - " )\n", - " ],\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "8a65eabff63a45729fe45fb5ade58bdc", - "metadata": {}, - "source": "\nWhich approach you take should depend on your use case.\n\n### Null Treatment\n\nThis option allows you to either respect or ignore null values.\n\nOne common usage for handling nulls is the case where you want to find the first value within a\npartition. By setting the null treatment to ignore nulls, we can find the first non-null value\nin our partition.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c3933fab20d04ec698c2621248eb3be0", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion.common import NullTreatment\n", - "\n", - "df.aggregate(\n", - " [col_type_1],\n", - " [\n", - " f.first_value(\n", - " col_type_2,\n", - " order_by=[col_attack],\n", - " null_treatment=NullTreatment.RESPECT_NULLS,\n", - " ).alias(\"Lowest Attack Type 2\")\n", - " ],\n", - ")\n", - "\n", - "df.aggregate(\n", - " [col_type_1],\n", - " [\n", - " f.first_value(\n", - " col_type_2, order_by=[col_attack], null_treatment=NullTreatment.IGNORE_NULLS\n", - " ).alias(\"Lowest Attack Type 2\")\n", - " ],\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "4dd4641cc4064e0191573fe9c69df29b", - "metadata": {}, - "source": "\n### Filter\n\nUsing the filter option is useful for filtering results to include in the aggregate function. It can\nbe seen in the example above on how this can be useful to only filter rows evaluated by the\naggregate function without filtering rows from the entire DataFrame.\n\nFilter takes a single expression.\n\nSuppose we want to find the speed values for only Pokemon that have low Attack values.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8309879909854d7188b41380fd92a7c3", - "metadata": {}, - "outputs": [], - "source": [ - "df.aggregate(\n", - " [col_type_1],\n", - " [\n", - " f.avg(col_speed).alias(\"Avg Speed All\"),\n", - " f.avg(col_speed, filter=col_attack < lit(50)).alias(\"Avg Speed Low Attack\"),\n", - " ],\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "3ed186c9a28b402fb0bc4494df01f08d", - "metadata": {}, - "source": "\n### Comparing subsets within a group\n\nSometimes you need to compare the full membership of a group against a\nsubset that meets some condition \u2014 for example, \"which groups have at least\none failure, but not every member failed?\". The `filter` argument on an\naggregate restricts the rows that contribute to *that* aggregate without\ndropping the group, so a single pass can produce both the full set and the\nfiltered subset side by side. Pairing\n[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and\n`filter=` is a compact way to express this: collect the distinct values\nof the group, collect the distinct values that satisfy the condition, then\ncompare the two arrays.\n\nSuppose each row records a line item with the supplier that fulfilled it and\na flag for whether that supplier met the commit date. We want to identify\n*partially failed* orders \u2014 orders where at least one supplier failed but\nnot every supplier failed:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "cb1e1581032b452c9409d6c6813c49d1", - "metadata": {}, - "outputs": [], - "source": [ - "orders_df = ctx.from_pydict(\n", - " {\n", - " \"order_id\": [1, 1, 1, 2, 2, 3, 4, 4],\n", - " \"supplier_id\": [100, 101, 102, 200, 201, 300, 400, 401],\n", - " \"failed\": [False, True, False, False, False, True, True, True],\n", - " },\n", - ")\n", - "\n", - "grouped = orders_df.aggregate(\n", - " [col(\"order_id\")],\n", - " [\n", - " f.array_agg(col(\"supplier_id\"), distinct=True).alias(\"all_suppliers\"),\n", - " f.array_agg(\n", - " col(\"supplier_id\"),\n", - " filter=col(\"failed\"),\n", - " distinct=True,\n", - " ).alias(\"failed_suppliers\"),\n", - " ],\n", - ")\n", - "\n", - "grouped.filter(\n", - " (f.array_length(col(\"failed_suppliers\")) > lit(0))\n", - " & (f.array_length(col(\"failed_suppliers\")) < f.array_length(col(\"all_suppliers\")))\n", - ").select(col(\"order_id\"), col(\"failed_suppliers\"))" - ] - }, - { - "cell_type": "markdown", - "id": "379cbbc1e968416e875cc15c1202d7eb", - "metadata": {}, - "source": "\nOrder 1 is partial (one of three suppliers failed). Order 2 is excluded\nbecause no supplier failed, order 3 because its only supplier failed, and\norder 4 because both of its suppliers failed.\n\n## Grouping Sets\n\nThe default style of aggregation produces one row per group. Sometimes you want a single query to\nproduce rows at multiple levels of detail \u2014 for example, totals per type *and* an overall grand\ntotal, or subtotals for every combination of two columns plus the individual column totals. Writing\nseparate queries and concatenating them is tedious and runs the data multiple times. Grouping sets\nsolve this by letting you specify several grouping levels in one pass.\n\nDataFusion supports three grouping set styles through the\n[`GroupingSet`][datafusion.expr.GroupingSet] class:\n\n- [`rollup`][datafusion.expr.GroupingSet.rollup] \u2014 hierarchical subtotals, like a drill-down report\n- [`cube`][datafusion.expr.GroupingSet.cube] \u2014 every possible subtotal combination, like a pivot table\n- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] \u2014 explicitly list exactly which grouping levels you want\n\nBecause result rows come from different grouping levels, a column that is *not* part of a\nparticular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to\ndistinguish a real `null` in the data from one that means \"this column was aggregated across.\"\nIt returns `0` when the column is a grouping key for that row, and `1` when it is not.\n\n### Rollup\n\n[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces\ngrouping sets `(a, b)`, `(a)`, and `()` \u2014 like nested subtotals in a report. This is useful\nwhen your columns have a natural hierarchy, such as region \u2192 city or type \u2192 subtype.\n\nSuppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With\nthe default aggregation style we would need two separate queries. With `rollup` we get it all at\nonce:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "277c27b1587741f2af2001be3712ef0d", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion.expr import GroupingSet\n", - "\n", - "df.aggregate(\n", - " [GroupingSet.rollup(col_type_1)],\n", - " [\n", - " f.count(col_speed).alias(\"Count\"),\n", - " f.avg(col_speed).alias(\"Avg Speed\"),\n", - " f.max(col_speed).alias(\"Max Speed\"),\n", - " ],\n", - ").sort(col_type_1.sort(ascending=True, nulls_first=True))" - ] - }, - { - "cell_type": "markdown", - "id": "db7b79bc585a40fcaf58bf750017e135", - "metadata": {}, - "source": "\nThe first row \u2014 where `Type 1` is `null` \u2014 is the grand total across all types. But how do you\ntell a grand-total `null` apart from a Pokemon that genuinely has no type? The\n[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key\nfor that row and `1` when it is aggregated across.\n\nUse `.alias()` to give the column a readable name:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "916684f9a58a4a2aa5f864670399430d", - "metadata": {}, - "outputs": [], - "source": "df.aggregate(\n [GroupingSet.rollup(col_type_1)],\n [\n f.count(col_speed).alias(\"Count\"),\n f.avg(col_speed).alias(\"Avg Speed\"),\n f.grouping(col_type_1).alias(\"Is Total\"),\n ],\n).sort(col_type_1.sort(ascending=True, nulls_first=True))" - }, - { - "cell_type": "markdown", - "id": "1671c31a24314836a5b85d7ef7fbf015", - "metadata": {}, - "source": "\nWith two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces:\n\n- one row per `(Type 1, Type 2)` pair \u2014 the most detailed level\n- one row per `Type 1` \u2014 subtotals\n- one grand total row\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "33b0902fd34d4ace834912fa1002cf8e", - "metadata": {}, - "outputs": [], - "source": [ - "df.aggregate(\n", - " [GroupingSet.rollup(col_type_1, col_type_2)],\n", - " [f.count(col_speed).alias(\"Count\"), f.avg(col_speed).alias(\"Avg Speed\")],\n", - ").sort(\n", - " col_type_1.sort(ascending=True, nulls_first=True),\n", - " col_type_2.sort(ascending=True, nulls_first=True),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "f6fa52606d8c4a75a9b52967216f8f3f", - "metadata": {}, - "source": "\n### Cube\n\n[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)`\nproduces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` \u2014 one more than `rollup` because\nit also includes `(b)` alone. This is useful when neither column is \"above\" the other in a\nhierarchy and you want all cross-tabulations.\n\nFor our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair,\nby `Type 1` alone, by `Type 2` alone, and a grand total \u2014 all in one query:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f5a1fa73e5044315a093ec459c9be902", - "metadata": {}, - "outputs": [], - "source": [ - "df.aggregate(\n", - " [GroupingSet.cube(col_type_1, col_type_2)],\n", - " [f.count(col_speed).alias(\"Count\"), f.avg(col_speed).alias(\"Avg Speed\")],\n", - ").sort(\n", - " col_type_1.sort(ascending=True, nulls_first=True),\n", - " col_type_2.sort(ascending=True, nulls_first=True),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "cdf66aed5cc84ca1b48e60bad68798a8", - "metadata": {}, - "source": "\nCompared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but\n`Type 2` has a value \u2014 those are the per-`Type 2` subtotals that `rollup` does not include.\n\n### Explicit Grouping Sets\n\n[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels\nyou need when `rollup` or `cube` would produce too many or too few. Each argument is a list of\ncolumns forming one grouping set.\n\nFor example, if we want only the per-`Type 1` totals and per-`Type 2` totals \u2014 but *not* the\nfull `(Type 1, Type 2)` detail rows or the grand total \u2014 we can ask for exactly that:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "28d3efd5258a48a79c179ea5c6759f01", - "metadata": {}, - "outputs": [], - "source": [ - "df.aggregate(\n", - " [GroupingSet.grouping_sets([col_type_1], [col_type_2])],\n", - " [f.count(col_speed).alias(\"Count\"), f.avg(col_speed).alias(\"Avg Speed\")],\n", - ").sort(\n", - " col_type_1.sort(ascending=True, nulls_first=True),\n", - " col_type_2.sort(ascending=True, nulls_first=True),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "3f9bc0b9dd2c44919cc8dcca39b469f8", - "metadata": {}, - "source": "\nEach row belongs to exactly one grouping level. The [`grouping`][datafusion.functions.grouping]\nfunction tells you which level each row comes from:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0e382214b5f147d187d36a2058b9c724", - "metadata": {}, - "outputs": [], - "source": "df.aggregate(\n [GroupingSet.grouping_sets([col_type_1], [col_type_2])],\n [\n f.count(col_speed).alias(\"Count\"),\n f.avg(col_speed).alias(\"Avg Speed\"),\n f.grouping(col_type_1).alias(\"grouping(Type 1)\"),\n f.grouping(col_type_2).alias(\"grouping(Type 2)\"),\n ],\n).sort(\n col_type_1.sort(ascending=True, nulls_first=True),\n col_type_2.sort(ascending=True, nulls_first=True),\n)" - }, - { - "cell_type": "markdown", - "id": "5b09d5ef5b5e4bb6ab9b829b10b6a29f", - "metadata": {}, - "source": "\nWhere `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`).\nWhere `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`).\n\n## Aggregate Functions\n\nThe available aggregate functions are:\n\n01. Comparison Functions\n : - [`min`][datafusion.functions.min]\n - [`max`][datafusion.functions.max]\n02. Math Functions\n : - [`sum`][datafusion.functions.sum]\n - [`avg`][datafusion.functions.avg]\n - [`median`][datafusion.functions.median]\n03. Array Functions\n : - [`array_agg`][datafusion.functions.array_agg]\n04. Logical Functions\n : - [`bit_and`][datafusion.functions.bit_and]\n - [`bit_or`][datafusion.functions.bit_or]\n - [`bit_xor`][datafusion.functions.bit_xor]\n - [`bool_and`][datafusion.functions.bool_and]\n - [`bool_or`][datafusion.functions.bool_or]\n05. Statistical Functions\n : - [`count`][datafusion.functions.count]\n - [`corr`][datafusion.functions.corr]\n - [`covar_samp`][datafusion.functions.covar_samp]\n - [`covar_pop`][datafusion.functions.covar_pop]\n - [`stddev`][datafusion.functions.stddev]\n - [`stddev_pop`][datafusion.functions.stddev_pop]\n - [`var_samp`][datafusion.functions.var_samp]\n - [`var_pop`][datafusion.functions.var_pop]\n - [`var_population`][datafusion.functions.var_population]\n06. Linear Regression Functions\n : - [`regr_count`][datafusion.functions.regr_count]\n - [`regr_slope`][datafusion.functions.regr_slope]\n - [`regr_intercept`][datafusion.functions.regr_intercept]\n - [`regr_r2`][datafusion.functions.regr_r2]\n - [`regr_avgx`][datafusion.functions.regr_avgx]\n - [`regr_avgy`][datafusion.functions.regr_avgy]\n - [`regr_sxx`][datafusion.functions.regr_sxx]\n - [`regr_syy`][datafusion.functions.regr_syy]\n - [`regr_slope`][datafusion.functions.regr_slope]\n07. Positional Functions\n : - [`first_value`][datafusion.functions.first_value]\n - [`last_value`][datafusion.functions.last_value]\n - [`nth_value`][datafusion.functions.nth_value]\n08. String Functions\n : - [`string_agg`][datafusion.functions.string_agg]\n09. Percentile Functions\n : - [`percentile_cont`][datafusion.functions.percentile_cont]\n - [`quantile_cont`][datafusion.functions.quantile_cont]\n - [`approx_distinct`][datafusion.functions.approx_distinct]\n - [`approx_median`][datafusion.functions.approx_median]\n - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont]\n - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight]\n10. Grouping Set Functions\n \\- [`grouping`][datafusion.functions.grouping]\n \\- [`rollup`][datafusion.expr.GroupingSet.rollup]\n \\- [`cube`][datafusion.expr.GroupingSet.cube]\n \\- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]\n\n## User-Defined Aggregate Functions\n\nYou can ship custom aggregations to the engine by subclassing\n[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via\n[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for\nthe accumulator interface and worked examples.\n\n
\n

Note

\n\nSerialization\n\n
\n\n Python aggregate UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions \u2014\n the accumulator class is captured by value via [`cloudpickle`][cloudpickle],\n so worker processes do not need to pre-register the UDF. Any names\n the accumulator resolves via `import` are captured **by reference**\n and must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/common-operations/aggregations.md b/docs/source/user-guide/common-operations/aggregations.md new file mode 100644 index 000000000..0026f9d2b --- /dev/null +++ b/docs/source/user-guide/common-operations/aggregations.md @@ -0,0 +1,497 @@ +```python exec="1" session="aggregations" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + + +# Aggregation + +An aggregate or aggregation is a function where the values of multiple rows are processed together +to form a single summary value. For performing an aggregation, DataFusion provides the +[`aggregate`][datafusion.dataframe.DataFrame.aggregate] + +```python exec="1" source="material-block" result="text" session="aggregations" +ctx = SessionContext() +df = ctx.read_csv("pokemon.csv") + +col_type_1 = col('"Type 1"') +col_type_2 = col('"Type 2"') +col_speed = col('"Speed"') +col_attack = col('"Attack"') + +df.aggregate( + [col_type_1], + [ + f.approx_distinct(col_speed).alias("Count"), + f.approx_median(col_speed).alias("Median Speed"), + f.approx_percentile_cont(col_speed, 0.9).alias("90% Speed"), + ], +) +``` + + +When `group_by` is `None` or an empty list, the aggregation is done over the whole +[`DataFrame`][datafusion.dataframe.DataFrame]. For grouping the `group_by` list must contain at least one column. + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [col_type_1], + [ + f.max(col_speed).alias("Max Speed"), + f.avg(col_speed).alias("Avg Speed"), + f.min(col_speed).alias("Min Speed"), + ], +) +``` + + +More than one column can be used for grouping + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [col_type_1, col_type_2], + [ + f.max(col_speed).alias("Max Speed"), + f.avg(col_speed).alias("Avg Speed"), + f.min(col_speed).alias("Min Speed"), + ], +) +``` + + +## Setting Parameters + +Each of the built in aggregate functions provides arguments for the parameters that affect their +operation. These can also be overridden using the builder approach to setting any of the following +parameters. When you use the builder, you must call `build()` to finish. For example, these two +expressions are equivalent. + +```python exec="1" source="material-block" result="text" session="aggregations" +first_1 = f.first_value(col("a"), order_by=[col("a")]) +first_2 = f.first_value(col("a")).order_by(col("a")).build() +``` + + +### Ordering + +You can control the order in which rows are processed by window functions by providing +a list of `order_by` functions for the `order_by` parameter. In the following example, we +sort the Pokemon by their attack in increasing order and take the first value, which gives us the +Pokemon with the smallest attack value in each `Type 1`. + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [col('"Type 1"')], + [ + f.first_value( + col('"Name"'), order_by=[col('"Attack"').sort(ascending=True)] + ).alias("Smallest Attack") + ], +) +``` + + +### Distinct + +When you set the parameter `distinct` to `True`, then unique values will only be evaluated one +time each. Suppose we want to create an array of all of the `Type 2` for each `Type 1` of our +Pokemon set. Since there will be many entries of `Type 2` we only one each distinct value. + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [col_type_1], [f.array_agg(col_type_2, distinct=True).alias("Type 2 List")] +) +``` + + +In the output of the above we can see that there are some `Type 1` for which the `Type 2` entry +is `null`. In reality, we probably want to filter those out. We can do this in two ways. First, +we can filter DataFrame rows that have no `Type 2`. If we do this, we might have some `Type 1` +entries entirely removed. The second is we can use the `filter` argument described below. + +```python exec="1" source="material-block" result="text" session="aggregations" +df.filter(col_type_2.is_not_null()).aggregate( + [col_type_1], [f.array_agg(col_type_2, distinct=True).alias("Type 2 List")] +) + +df.aggregate( + [col_type_1], + [ + f.array_agg(col_type_2, distinct=True, filter=col_type_2.is_not_null()).alias( + "Type 2 List" + ) + ], +) +``` + + +Which approach you take should depend on your use case. + +### Null Treatment + +This option allows you to either respect or ignore null values. + +One common usage for handling nulls is the case where you want to find the first value within a +partition. By setting the null treatment to ignore nulls, we can find the first non-null value +in our partition. + +```python exec="1" source="material-block" result="text" session="aggregations" +from datafusion.common import NullTreatment + +df.aggregate( + [col_type_1], + [ + f.first_value( + col_type_2, + order_by=[col_attack], + null_treatment=NullTreatment.RESPECT_NULLS, + ).alias("Lowest Attack Type 2") + ], +) + +df.aggregate( + [col_type_1], + [ + f.first_value( + col_type_2, order_by=[col_attack], null_treatment=NullTreatment.IGNORE_NULLS + ).alias("Lowest Attack Type 2") + ], +) +``` + + +### Filter + +Using the filter option is useful for filtering results to include in the aggregate function. It can +be seen in the example above on how this can be useful to only filter rows evaluated by the +aggregate function without filtering rows from the entire DataFrame. + +Filter takes a single expression. + +Suppose we want to find the speed values for only Pokemon that have low Attack values. + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [col_type_1], + [ + f.avg(col_speed).alias("Avg Speed All"), + f.avg(col_speed, filter=col_attack < lit(50)).alias("Avg Speed Low Attack"), + ], +) +``` + + +### Comparing subsets within a group + +Sometimes you need to compare the full membership of a group against a +subset that meets some condition — for example, "which groups have at least +one failure, but not every member failed?". The `filter` argument on an +aggregate restricts the rows that contribute to *that* aggregate without +dropping the group, so a single pass can produce both the full set and the +filtered subset side by side. Pairing +[`array_agg`][datafusion.functions.array_agg] with `distinct=True` and +`filter=` is a compact way to express this: collect the distinct values +of the group, collect the distinct values that satisfy the condition, then +compare the two arrays. + +Suppose each row records a line item with the supplier that fulfilled it and +a flag for whether that supplier met the commit date. We want to identify +*partially failed* orders — orders where at least one supplier failed but +not every supplier failed: + +```python exec="1" source="material-block" result="text" session="aggregations" +orders_df = ctx.from_pydict( + { + "order_id": [1, 1, 1, 2, 2, 3, 4, 4], + "supplier_id": [100, 101, 102, 200, 201, 300, 400, 401], + "failed": [False, True, False, False, False, True, True, True], + }, +) + +grouped = orders_df.aggregate( + [col("order_id")], + [ + f.array_agg(col("supplier_id"), distinct=True).alias("all_suppliers"), + f.array_agg( + col("supplier_id"), + filter=col("failed"), + distinct=True, + ).alias("failed_suppliers"), + ], +) + +grouped.filter( + (f.array_length(col("failed_suppliers")) > lit(0)) + & (f.array_length(col("failed_suppliers")) < f.array_length(col("all_suppliers"))) +).select(col("order_id"), col("failed_suppliers")) +``` + + +Order 1 is partial (one of three suppliers failed). Order 2 is excluded +because no supplier failed, order 3 because its only supplier failed, and +order 4 because both of its suppliers failed. + +## Grouping Sets + +The default style of aggregation produces one row per group. Sometimes you want a single query to +produce rows at multiple levels of detail — for example, totals per type *and* an overall grand +total, or subtotals for every combination of two columns plus the individual column totals. Writing +separate queries and concatenating them is tedious and runs the data multiple times. Grouping sets +solve this by letting you specify several grouping levels in one pass. + +DataFusion supports three grouping set styles through the +[`GroupingSet`][datafusion.expr.GroupingSet] class: + +- [`rollup`][datafusion.expr.GroupingSet.rollup] — hierarchical subtotals, like a drill-down report +- [`cube`][datafusion.expr.GroupingSet.cube] — every possible subtotal combination, like a pivot table +- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] — explicitly list exactly which grouping levels you want + +Because result rows come from different grouping levels, a column that is *not* part of a +particular level will be `null` in that row. Use [`grouping`][datafusion.functions.grouping] to +distinguish a real `null` in the data from one that means "this column was aggregated across." +It returns `0` when the column is a grouping key for that row, and `1` when it is not. + +### Rollup + +[`rollup`][datafusion.expr.GroupingSet.rollup] creates a hierarchy. `rollup(a, b)` produces +grouping sets `(a, b)`, `(a)`, and `()` — like nested subtotals in a report. This is useful +when your columns have a natural hierarchy, such as region → city or type → subtype. + +Suppose we want to summarize Pokemon stats by `Type 1` with subtotals and a grand total. With +the default aggregation style we would need two separate queries. With `rollup` we get it all at +once: + +```python exec="1" source="material-block" result="text" session="aggregations" +from datafusion.expr import GroupingSet + +df.aggregate( + [GroupingSet.rollup(col_type_1)], + [ + f.count(col_speed).alias("Count"), + f.avg(col_speed).alias("Avg Speed"), + f.max(col_speed).alias("Max Speed"), + ], +).sort(col_type_1.sort(ascending=True, nulls_first=True)) +``` + + +The first row — where `Type 1` is `null` — is the grand total across all types. But how do you +tell a grand-total `null` apart from a Pokemon that genuinely has no type? The +[`grouping`][datafusion.functions.grouping] function returns `0` when the column is a grouping key +for that row and `1` when it is aggregated across. + +Use `.alias()` to give the column a readable name: + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [GroupingSet.rollup(col_type_1)], + [ + f.count(col_speed).alias("Count"), + f.avg(col_speed).alias("Avg Speed"), + f.grouping(col_type_1).alias("Is Total"), + ], +).sort(col_type_1.sort(ascending=True, nulls_first=True)) +``` + + +With two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` produces: + +- one row per `(Type 1, Type 2)` pair — the most detailed level +- one row per `Type 1` — subtotals +- one grand total row + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [GroupingSet.rollup(col_type_1, col_type_2)], + [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], +).sort( + col_type_1.sort(ascending=True, nulls_first=True), + col_type_2.sort(ascending=True, nulls_first=True), +) +``` + + +### Cube + +[`cube`][datafusion.expr.GroupingSet.cube] produces every possible subset. `cube(a, b)` +produces grouping sets `(a, b)`, `(a)`, `(b)`, and `()` — one more than `rollup` because +it also includes `(b)` alone. This is useful when neither column is "above" the other in a +hierarchy and you want all cross-tabulations. + +For our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the type pair, +by `Type 1` alone, by `Type 2` alone, and a grand total — all in one query: + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [GroupingSet.cube(col_type_1, col_type_2)], + [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], +).sort( + col_type_1.sort(ascending=True, nulls_first=True), + col_type_2.sort(ascending=True, nulls_first=True), +) +``` + + +Compared to the `rollup` example above, notice the extra rows where `Type 1` is `null` but +`Type 2` has a value — those are the per-`Type 2` subtotals that `rollup` does not include. + +### Explicit Grouping Sets + +[`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] lets you list exactly which grouping levels +you need when `rollup` or `cube` would produce too many or too few. Each argument is a list of +columns forming one grouping set. + +For example, if we want only the per-`Type 1` totals and per-`Type 2` totals — but *not* the +full `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that: + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [GroupingSet.grouping_sets([col_type_1], [col_type_2])], + [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], +).sort( + col_type_1.sort(ascending=True, nulls_first=True), + col_type_2.sort(ascending=True, nulls_first=True), +) +``` + + +Each row belongs to exactly one grouping level. The [`grouping`][datafusion.functions.grouping] +function tells you which level each row comes from: + +```python exec="1" source="material-block" result="text" session="aggregations" +df.aggregate( + [GroupingSet.grouping_sets([col_type_1], [col_type_2])], + [ + f.count(col_speed).alias("Count"), + f.avg(col_speed).alias("Avg Speed"), + f.grouping(col_type_1).alias("grouping(Type 1)"), + f.grouping(col_type_2).alias("grouping(Type 2)"), + ], +).sort( + col_type_1.sort(ascending=True, nulls_first=True), + col_type_2.sort(ascending=True, nulls_first=True), +) +``` + + +Where `grouping(Type 1)` is `0` the row is a per-`Type 1` total (and `Type 2` is `null`). +Where `grouping(Type 2)` is `0` the row is a per-`Type 2` total (and `Type 1` is `null`). + +## Aggregate Functions + +The available aggregate functions are: + +01. Comparison Functions + : - [`min`][datafusion.functions.min] + - [`max`][datafusion.functions.max] +02. Math Functions + : - [`sum`][datafusion.functions.sum] + - [`avg`][datafusion.functions.avg] + - [`median`][datafusion.functions.median] +03. Array Functions + : - [`array_agg`][datafusion.functions.array_agg] +04. Logical Functions + : - [`bit_and`][datafusion.functions.bit_and] + - [`bit_or`][datafusion.functions.bit_or] + - [`bit_xor`][datafusion.functions.bit_xor] + - [`bool_and`][datafusion.functions.bool_and] + - [`bool_or`][datafusion.functions.bool_or] +05. Statistical Functions + : - [`count`][datafusion.functions.count] + - [`corr`][datafusion.functions.corr] + - [`covar_samp`][datafusion.functions.covar_samp] + - [`covar_pop`][datafusion.functions.covar_pop] + - [`stddev`][datafusion.functions.stddev] + - [`stddev_pop`][datafusion.functions.stddev_pop] + - [`var_samp`][datafusion.functions.var_samp] + - [`var_pop`][datafusion.functions.var_pop] + - [`var_population`][datafusion.functions.var_population] +06. Linear Regression Functions + : - [`regr_count`][datafusion.functions.regr_count] + - [`regr_slope`][datafusion.functions.regr_slope] + - [`regr_intercept`][datafusion.functions.regr_intercept] + - [`regr_r2`][datafusion.functions.regr_r2] + - [`regr_avgx`][datafusion.functions.regr_avgx] + - [`regr_avgy`][datafusion.functions.regr_avgy] + - [`regr_sxx`][datafusion.functions.regr_sxx] + - [`regr_syy`][datafusion.functions.regr_syy] + - [`regr_slope`][datafusion.functions.regr_slope] +07. Positional Functions + : - [`first_value`][datafusion.functions.first_value] + - [`last_value`][datafusion.functions.last_value] + - [`nth_value`][datafusion.functions.nth_value] +08. String Functions + : - [`string_agg`][datafusion.functions.string_agg] +09. Percentile Functions + : - [`percentile_cont`][datafusion.functions.percentile_cont] + - [`quantile_cont`][datafusion.functions.quantile_cont] + - [`approx_distinct`][datafusion.functions.approx_distinct] + - [`approx_median`][datafusion.functions.approx_median] + - [`approx_percentile_cont`][datafusion.functions.approx_percentile_cont] + - [`approx_percentile_cont_with_weight`][datafusion.functions.approx_percentile_cont_with_weight] +10. Grouping Set Functions + \- [`grouping`][datafusion.functions.grouping] + \- [`rollup`][datafusion.expr.GroupingSet.rollup] + \- [`cube`][datafusion.expr.GroupingSet.cube] + \- [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets] + +## User-Defined Aggregate Functions + +You can ship custom aggregations to the engine by subclassing +[`Accumulator`][datafusion.user_defined.Accumulator] and registering it via +[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for +the accumulator interface and worked examples. + +
+

Note

+ +Serialization + +
+ + Python aggregate UDFs travel inline inside pickled or + [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions — + the accumulator class is captured by value via [`cloudpickle`][cloudpickle], + so worker processes do not need to pre-register the UDF. Any names + the accumulator resolves via `import` are captured **by reference** + and must be importable on the receiving worker. See + [`ipc`][datafusion.ipc] for the full IPC model and security caveats. diff --git a/docs/source/user-guide/common-operations/basic-info.ipynb b/docs/source/user-guide/common-operations/basic-info.ipynb deleted file mode 100644 index 5d77c2ce6..000000000 --- a/docs/source/user-guide/common-operations/basic-info.ipynb +++ /dev/null @@ -1,143 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f # noqa: F401\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n# Basic Operations\n\nIn this section, you will learn how to display essential details of DataFrames using specific functions.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "import random\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict(\n", - " {\n", - " \"nrs\": [1, 2, 3, 4, 5],\n", - " \"names\": [\"python\", \"ruby\", \"java\", \"haskell\", \"go\"],\n", - " \"random\": random.sample(range(1000), 5),\n", - " \"groups\": [\"A\", \"A\", \"B\", \"C\", \"B\"],\n", - " }\n", - ")\n", - "df" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\nUse [`limit`][datafusion.dataframe.DataFrame.limit] to view the top rows of the frame:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "df.limit(2)" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\nDisplay the columns of the DataFrame using [`schema`][datafusion.dataframe.DataFrame.schema]:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "df.schema()" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\nThe method [`to_pandas`][datafusion.dataframe.DataFrame.to_pandas] uses pyarrow to convert to pandas DataFrame, by collecting the batches,\npassing them to an Arrow table, and then converting them to a pandas DataFrame.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "df.to_pandas()" - ] - }, - { - "cell_type": "markdown", - "id": "7cdc8c89c7104fffa095e18ddfef8986", - "metadata": {}, - "source": "\n[`describe`][datafusion.dataframe.DataFrame.describe] shows a quick statistic summary of your data:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b118ea5561624da68c537baed56e602f", - "metadata": {}, - "outputs": [], - "source": [ - "df.describe()" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/common-operations/basic-info.md b/docs/source/user-guide/common-operations/basic-info.md new file mode 100644 index 000000000..703e319e2 --- /dev/null +++ b/docs/source/user-guide/common-operations/basic-info.md @@ -0,0 +1,91 @@ +```python exec="1" session="basic-info" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + +# Basic Operations + +In this section, you will learn how to display essential details of DataFrames using specific functions. + +```python exec="1" source="material-block" result="text" session="basic-info" +import random + +ctx = SessionContext() +df = ctx.from_pydict( + { + "nrs": [1, 2, 3, 4, 5], + "names": ["python", "ruby", "java", "haskell", "go"], + "random": random.sample(range(1000), 5), + "groups": ["A", "A", "B", "C", "B"], + } +) +df +``` + + +Use [`limit`][datafusion.dataframe.DataFrame.limit] to view the top rows of the frame: + +```python exec="1" source="material-block" result="text" session="basic-info" +df.limit(2) +``` + + +Display the columns of the DataFrame using [`schema`][datafusion.dataframe.DataFrame.schema]: + +```python exec="1" source="material-block" result="text" session="basic-info" +df.schema() +``` + + +The method [`to_pandas`][datafusion.dataframe.DataFrame.to_pandas] uses pyarrow to convert to pandas DataFrame, by collecting the batches, +passing them to an Arrow table, and then converting them to a pandas DataFrame. + +```python exec="1" source="material-block" result="text" session="basic-info" +df.to_pandas() +``` + + +[`describe`][datafusion.dataframe.DataFrame.describe] shows a quick statistic summary of your data: + +```python exec="1" source="material-block" result="text" session="basic-info" +df.describe() +``` diff --git a/docs/source/user-guide/common-operations/expressions.ipynb b/docs/source/user-guide/common-operations/expressions.ipynb deleted file mode 100644 index a3eebb71e..000000000 --- a/docs/source/user-guide/common-operations/expressions.ipynb +++ /dev/null @@ -1,387 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n\n# Expressions\n\nIn DataFusion an expression is an abstraction that represents a computation.\nExpressions are used as the primary inputs and outputs for most functions within\nDataFusion. As such, expressions can be combined to create expression trees, a\nconcept shared across most compilers and databases.\n\n## Column\n\nThe first expression most new users will interact with is the Column, which is created by calling [`col`][datafusion.col.col].\nThis expression represents a column within a DataFrame. The function [`col`][datafusion.col.col] takes as in input a string\nand returns an expression as it's output.\n\n## Literal\n\nLiteral expressions represent a single value. These are helpful in a wide range of operations where\na specific, known value is of interest. You can create a literal expression using the function [`lit`][datafusion.lit].\nThe type of the object passed to the [`lit`][datafusion.lit] function will be used to convert it to a known data type.\n\nIn the following example we create expressions for the column named `color` and the literal scalar string `red`.\nThe resultant variable `red_units` is itself also an expression.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "red_units = col(\"color\") == lit(\"red\")" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\n## Boolean\n\nWhen combining expressions that evaluate to a boolean value, you can combine these expressions using boolean operators.\nIt is important to note that in order to combine these expressions, you *must* use bitwise operators. See the following\nexamples for the and, or, and not operations.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "red_or_green_units = (col(\"color\") == lit(\"red\")) | (col(\"color\") == lit(\"green\"))\n", - "heavy_red_units = (col(\"color\") == lit(\"red\")) & (col(\"weight\") > lit(42))\n", - "not_red_units = ~(col(\"color\") == lit(\"red\"))" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\n## Arrays\n\nFor columns that contain arrays of values, you can access individual elements of the array by index\nusing bracket indexing. This is similar to calling the function\n[`array_element`][datafusion.functions.array_element], except that array indexing using brackets is 0 based,\nsimilar to Python arrays and `array_element` is 1 based indexing to be compatible with other SQL\napproaches.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import col\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict({\"a\": [[1, 2, 3], [4, 5, 6]]})\n", - "df.select(col(\"a\")[0].alias(\"a0\"))" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\n
\n

Warning

\n\nIndexing an element of an array via `[]` starts at index 0 whereas\n[`array_element`][datafusion.functions.array_element] starts at index 1.\n\n
\n\nStarting in DataFusion 49.0.0 you can also create slices of array elements using\nslice syntax from Python.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(col(\"a\")[1:3].alias(\"second_two_elements\"))" - ] - }, - { - "cell_type": "markdown", - "id": "7cdc8c89c7104fffa095e18ddfef8986", - "metadata": {}, - "source": "\nTo check if an array is empty, you can use the function [`array_empty`][datafusion.functions.array_empty] or `datafusion.functions.empty`.\nThis function returns a boolean indicating whether the array is empty.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b118ea5561624da68c537baed56e602f", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import SessionContext, col\n", - "from datafusion.functions import array_empty\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict({\"a\": [[], [1, 2, 3]]})\n", - "df.select(array_empty(col(\"a\")).alias(\"is_empty\"))" - ] - }, - { - "cell_type": "markdown", - "id": "938c804e27f84196a10c8828c723f798", - "metadata": {}, - "source": "\nIn this example, the `is_empty` column will contain `True` for the first row and `False` for the second row.\n\nTo get the total number of elements in an array, you can use the function [`cardinality`][datafusion.functions.cardinality].\nThis function returns an integer indicating the total number of elements in the array.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "504fb2a444614c0babb325280ed9130a", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import SessionContext, col\n", - "from datafusion.functions import cardinality\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict({\"a\": [[1, 2, 3], [4, 5, 6]]})\n", - "df.select(cardinality(col(\"a\")).alias(\"num_elements\"))" - ] - }, - { - "cell_type": "markdown", - "id": "59bbdb311c014d738909a11f9e486628", - "metadata": {}, - "source": "\nIn this example, the `num_elements` column will contain `3` for both rows.\n\nTo concatenate two arrays, you can use the function [`array_cat`][datafusion.functions.array_cat] or [`array_concat`][datafusion.functions.array_concat].\nThese functions return a new array that is the concatenation of the input arrays.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b43b363d81ae4b689946ece5c682cd59", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import SessionContext, col\n", - "from datafusion.functions import array_cat\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict({\"a\": [[1, 2, 3]], \"b\": [[4, 5, 6]]})\n", - "df.select(array_cat(col(\"a\"), col(\"b\")).alias(\"concatenated_array\"))" - ] - }, - { - "cell_type": "markdown", - "id": "8a65eabff63a45729fe45fb5ade58bdc", - "metadata": {}, - "source": "\nIn this example, the `concatenated_array` column will contain `[1, 2, 3, 4, 5, 6]`.\n\nTo repeat the elements of an array a specified number of times, you can use the function [`array_repeat`][datafusion.functions.array_repeat].\nThis function returns a new array with the elements repeated.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c3933fab20d04ec698c2621248eb3be0", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import SessionContext, col\n", - "from datafusion.functions import array_repeat\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict({\"a\": [[1, 2, 3]]})\n", - "df.select(array_repeat(col(\"a\"), literal(2)).alias(\"repeated_array\"))" - ] - }, - { - "cell_type": "markdown", - "id": "4dd4641cc4064e0191573fe9c69df29b", - "metadata": {}, - "source": "\nIn this example, the `repeated_array` column will contain `[[1, 2, 3], [1, 2, 3]]`.\n\n## Lambda functions\n\nSome array functions take a *lambda function*: a small function that runs once\nper element. [`array_transform`][datafusion.functions.array_transform] maps a lambda over\nevery element, [`array_filter`][datafusion.functions.array_filter] keeps the elements\nfor which a predicate lambda is true, and\n[`array_any_match`][datafusion.functions.array_any_match] returns whether any element\nsatisfies a predicate lambda. (Functions that take another function as an\nargument are sometimes called *higher-order* functions.)\n\nThe simplest way to supply a lambda is a Python `lambda`. Its parameter names\nbecome the lambda parameters, and its return value becomes the body.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8309879909854d7188b41380fd92a7c3", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import SessionContext, col\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict({\"a\": [[1, 2, 3], [4, 5]]})\n", - "df.select(f.array_transform(col(\"a\"), lambda v: v * 2).alias(\"doubled\"))\n", - "df.select(f.array_filter(col(\"a\"), lambda v: v > 2).alias(\"big_only\"))\n", - "df.select(f.array_any_match(col(\"a\"), lambda v: v > 3).alias(\"has_big\"))" - ] - }, - { - "cell_type": "markdown", - "id": "3ed186c9a28b402fb0bc4494df01f08d", - "metadata": {}, - "source": "\nIf you need explicit control over parameter names, build the lambda with\n[`lambda_`][datafusion.functions.lambda_] and reference its parameters with\n[`lambda_var`][datafusion.functions.lambda_var]. The following is equivalent to the\n`array_transform` call above.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "cb1e1581032b452c9409d6c6813c49d1", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import lit\n", - "\n", - "double_fn = f.lambda_([\"v\"], f.lambda_var(\"v\") * lit(2))\n", - "df.select(f.array_transform(col(\"a\"), double_fn).alias(\"doubled\"))" - ] - }, - { - "cell_type": "markdown", - "id": "379cbbc1e968416e875cc15c1202d7eb", - "metadata": {}, - "source": "\n
\n

Note

\n\nLambda expressions cannot yet be serialized: calling\n[`to_bytes`][datafusion.expr.Expr.to_bytes] or pickling an expression that\ncontains a lambda raises `Lambda not implemented`. SQL lambda syntax is\nonly parsed by dialects that support lambdas; set\n`datafusion.sql_parser.dialect` to one of `DuckDB`, `ClickHouse`,\n`Snowflake`, or `Databricks`. Both arrow syntax (`x -> x * 2`) and\nkeyword syntax (`lambda x: x * 2`) parse. DuckDB will drop the arrow\nform in v2.1, so prefer `lambda x: x * 2` for forward compatibility.\nThe Python expression builder shown above works regardless of dialect.\n\n
\n\n## Testing membership in a list\n\nA common need is filtering rows where a column equals *any* of a small set of\nvalues. DataFusion offers three forms; they differ in readability and in how\nthey scale:\n\n1. A compound boolean using `|` across explicit equalities.\n2. [`in_list`][datafusion.functions.in_list], which accepts a list of\n expressions and tests equality against all of them in one call.\n3. A trick with [`array_position`][datafusion.functions.array_position] and\n [`make_array`][datafusion.functions.make_array], which returns the 1-based\n index of the value in a constructed array, or null if it is not present.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "277c27b1587741f2af2001be3712ef0d", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import SessionContext, col, lit\n", - "from datafusion import functions as f\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict({\"shipmode\": [\"MAIL\", \"SHIP\", \"AIR\", \"TRUCK\", \"RAIL\"]})\n", - "\n", - "# Option 1: compound boolean. Fine for two values; awkward past three.\n", - "df.filter((col(\"shipmode\") == lit(\"MAIL\")) | (col(\"shipmode\") == lit(\"SHIP\")))\n", - "\n", - "# Option 2: in_list. Preferred for readability as the set grows.\n", - "df.filter(f.in_list(col(\"shipmode\"), [lit(\"MAIL\"), lit(\"SHIP\")]))\n", - "\n", - "# Option 3: array_position / make_array. Useful when you already have the\n", - "# set as an array column and want \"is in that array\" semantics.\n", - "df.filter(\n", - " ~f.array_position(f.make_array(lit(\"MAIL\"), lit(\"SHIP\")), col(\"shipmode\")).is_null()\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "db7b79bc585a40fcaf58bf750017e135", - "metadata": {}, - "source": "\nUse `in_list` as the default. It is explicit, readable, and matches the\nsemantics users expect from SQL's `IN (...)`. Reach for the\n`array_position` form only when the membership set is itself an array\ncolumn rather than a literal list.\n\n## Conditional expressions\n\nDataFusion provides [`case`][datafusion.functions.case] for the SQL\n`CASE` expression in both its switched and searched forms, along with\n[`when`][datafusion.functions.when] as a standalone builder for the\nsearched form.\n\n**Switched CASE** (one expression compared against several literal values):\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "916684f9a58a4a2aa5f864670399430d", - "metadata": {}, - "outputs": [], - "source": [ - "df = ctx.from_pydict(\n", - " {\"priority\": [\"1-URGENT\", \"2-HIGH\", \"3-MEDIUM\", \"5-LOW\"]},\n", - ")\n", - "\n", - "df.select(\n", - " col(\"priority\"),\n", - " f.case(col(\"priority\"))\n", - " .when(lit(\"1-URGENT\"), lit(1))\n", - " .when(lit(\"2-HIGH\"), lit(1))\n", - " .otherwise(lit(0))\n", - " .alias(\"is_high_priority\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "1671c31a24314836a5b85d7ef7fbf015", - "metadata": {}, - "source": "\n**Searched CASE** (an independent boolean predicate per branch). Use this\nform whenever a branch tests more than simple equality — for example,\nchecking whether a joined column is `NULL` to gate a computed value:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "33b0902fd34d4ace834912fa1002cf8e", - "metadata": {}, - "outputs": [], - "source": [ - "df = ctx.from_pydict(\n", - " {\"volume\": [10.0, 20.0, 30.0], \"supplier_id\": [1, None, 2]},\n", - ")\n", - "\n", - "df.select(\n", - " col(\"volume\"),\n", - " col(\"supplier_id\"),\n", - " f.when(col(\"supplier_id\").is_not_null(), col(\"volume\"))\n", - " .otherwise(lit(0.0))\n", - " .alias(\"attributed_volume\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "f6fa52606d8c4a75a9b52967216f8f3f", - "metadata": {}, - "source": "\nThis searched-CASE pattern is idiomatic for \"attribute the measure to the\nmatching side of a left join, otherwise contribute zero\" — a shape that\nappears in TPC-H Q08 and similar market-share calculations.\n\nIf a switched CASE only groups several equality matches into one bucket,\n`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often\nsimpler than the full `case` builder.\n\n## Structs\n\nColumns that contain struct elements can be accessed using the bracket notation as if they were\nPython dictionary style objects. This expects a string key as the parameter passed.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f5a1fa73e5044315a093ec459c9be902", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "data = {\"a\": [{\"size\": 15, \"color\": \"green\"}, {\"size\": 10, \"color\": \"blue\"}]}\n", - "df = ctx.from_pydict(data)\n", - "df.select(col(\"a\")[\"size\"].alias(\"a_size\"))" - ] - }, - { - "cell_type": "markdown", - "id": "cdf66aed5cc84ca1b48e60bad68798a8", - "metadata": {}, - "source": "\n## Functions\n\nAs mentioned before, most functions in DataFusion return an expression at their output. This allows us to create\na wide variety of expressions built up from other expressions. For example, [`alias`][datafusion.expr.Expr.alias] is a function that takes\nas it input a single expression and returns an expression in which the name of the expression has changed.\n\nThe following example shows a series of expressions that are built up from functions operating on expressions.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "28d3efd5258a48a79c179ea5c6759f01", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import SessionContext, lit\n", - "from datafusion import functions as f\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_pydict(\n", - " {\n", - " \"name\": [\"Albert\", \"Becca\", \"Carlos\", \"Dante\"],\n", - " \"age\": [42, 67, 27, 71],\n", - " \"years_in_position\": [13, 21, 10, 54],\n", - " },\n", - " name=\"employees\",\n", - ")\n", - "\n", - "age_col = col(\"age\")\n", - "renamed_age = age_col.alias(\"age_in_years\")\n", - "start_age = age_col - col(\"years_in_position\")\n", - "started_young = start_age < lit(18)\n", - "can_retire = age_col > lit(65)\n", - "long_timer = started_young & can_retire\n", - "\n", - "df.filter(long_timer).select(col(\"name\"), renamed_age, col(\"years_in_position\"))" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/common-operations/expressions.md b/docs/source/user-guide/common-operations/expressions.md new file mode 100644 index 000000000..47ec644bf --- /dev/null +++ b/docs/source/user-guide/common-operations/expressions.md @@ -0,0 +1,365 @@ +```python exec="1" session="expressions" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + + +# Expressions + +In DataFusion an expression is an abstraction that represents a computation. +Expressions are used as the primary inputs and outputs for most functions within +DataFusion. As such, expressions can be combined to create expression trees, a +concept shared across most compilers and databases. + +## Column + +The first expression most new users will interact with is the Column, which is created by calling [`col`][datafusion.col.col]. +This expression represents a column within a DataFrame. The function [`col`][datafusion.col.col] takes as in input a string +and returns an expression as it's output. + +## Literal + +Literal expressions represent a single value. These are helpful in a wide range of operations where +a specific, known value is of interest. You can create a literal expression using the function [`lit`][datafusion.lit]. +The type of the object passed to the [`lit`][datafusion.lit] function will be used to convert it to a known data type. + +In the following example we create expressions for the column named `color` and the literal scalar string `red`. +The resultant variable `red_units` is itself also an expression. + +```python exec="1" source="material-block" result="text" session="expressions" +red_units = col("color") == lit("red") +``` + + +## Boolean + +When combining expressions that evaluate to a boolean value, you can combine these expressions using boolean operators. +It is important to note that in order to combine these expressions, you *must* use bitwise operators. See the following +examples for the and, or, and not operations. + +```python exec="1" source="material-block" result="text" session="expressions" +red_or_green_units = (col("color") == lit("red")) | (col("color") == lit("green")) +heavy_red_units = (col("color") == lit("red")) & (col("weight") > lit(42)) +not_red_units = ~(col("color") == lit("red")) +``` + + +## Arrays + +For columns that contain arrays of values, you can access individual elements of the array by index +using bracket indexing. This is similar to calling the function +[`array_element`][datafusion.functions.array_element], except that array indexing using brackets is 0 based, +similar to Python arrays and `array_element` is 1 based indexing to be compatible with other SQL +approaches. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import col + +ctx = SessionContext() +df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5, 6]]}) +df.select(col("a")[0].alias("a0")) +``` + + +
+

Warning

+ +Indexing an element of an array via `[]` starts at index 0 whereas +[`array_element`][datafusion.functions.array_element] starts at index 1. + +
+ +Starting in DataFusion 49.0.0 you can also create slices of array elements using +slice syntax from Python. + +```python exec="1" source="material-block" result="text" session="expressions" +df.select(col("a")[1:3].alias("second_two_elements")) +``` + + +To check if an array is empty, you can use the function [`array_empty`][datafusion.functions.array_empty] or `datafusion.functions.empty`. +This function returns a boolean indicating whether the array is empty. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import SessionContext, col +from datafusion.functions import array_empty + +ctx = SessionContext() +df = ctx.from_pydict({"a": [[], [1, 2, 3]]}) +df.select(array_empty(col("a")).alias("is_empty")) +``` + + +In this example, the `is_empty` column will contain `True` for the first row and `False` for the second row. + +To get the total number of elements in an array, you can use the function [`cardinality`][datafusion.functions.cardinality]. +This function returns an integer indicating the total number of elements in the array. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import SessionContext, col +from datafusion.functions import cardinality + +ctx = SessionContext() +df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5, 6]]}) +df.select(cardinality(col("a")).alias("num_elements")) +``` + + +In this example, the `num_elements` column will contain `3` for both rows. + +To concatenate two arrays, you can use the function [`array_cat`][datafusion.functions.array_cat] or [`array_concat`][datafusion.functions.array_concat]. +These functions return a new array that is the concatenation of the input arrays. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import SessionContext, col +from datafusion.functions import array_cat + +ctx = SessionContext() +df = ctx.from_pydict({"a": [[1, 2, 3]], "b": [[4, 5, 6]]}) +df.select(array_cat(col("a"), col("b")).alias("concatenated_array")) +``` + + +In this example, the `concatenated_array` column will contain `[1, 2, 3, 4, 5, 6]`. + +To repeat the elements of an array a specified number of times, you can use the function [`array_repeat`][datafusion.functions.array_repeat]. +This function returns a new array with the elements repeated. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import SessionContext, col +from datafusion.functions import array_repeat + +ctx = SessionContext() +df = ctx.from_pydict({"a": [[1, 2, 3]]}) +df.select(array_repeat(col("a"), literal(2)).alias("repeated_array")) +``` + + +In this example, the `repeated_array` column will contain `[[1, 2, 3], [1, 2, 3]]`. + +## Lambda functions + +Some array functions take a *lambda function*: a small function that runs once +per element. [`array_transform`][datafusion.functions.array_transform] maps a lambda over +every element, [`array_filter`][datafusion.functions.array_filter] keeps the elements +for which a predicate lambda is true, and +[`array_any_match`][datafusion.functions.array_any_match] returns whether any element +satisfies a predicate lambda. (Functions that take another function as an +argument are sometimes called *higher-order* functions.) + +The simplest way to supply a lambda is a Python `lambda`. Its parameter names +become the lambda parameters, and its return value becomes the body. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import SessionContext, col + +ctx = SessionContext() +df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5]]}) +df.select(f.array_transform(col("a"), lambda v: v * 2).alias("doubled")) +df.select(f.array_filter(col("a"), lambda v: v > 2).alias("big_only")) +df.select(f.array_any_match(col("a"), lambda v: v > 3).alias("has_big")) +``` + + +If you need explicit control over parameter names, build the lambda with +[`lambda_`][datafusion.functions.lambda_] and reference its parameters with +[`lambda_var`][datafusion.functions.lambda_var]. The following is equivalent to the +`array_transform` call above. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import lit + +double_fn = f.lambda_(["v"], f.lambda_var("v") * lit(2)) +df.select(f.array_transform(col("a"), double_fn).alias("doubled")) +``` + + +
+

Note

+ +Lambda expressions cannot yet be serialized: calling +[`to_bytes`][datafusion.expr.Expr.to_bytes] or pickling an expression that +contains a lambda raises `Lambda not implemented`. SQL lambda syntax is +only parsed by dialects that support lambdas; set +`datafusion.sql_parser.dialect` to one of `DuckDB`, `ClickHouse`, +`Snowflake`, or `Databricks`. Both arrow syntax (`x -> x * 2`) and +keyword syntax (`lambda x: x * 2`) parse. DuckDB will drop the arrow +form in v2.1, so prefer `lambda x: x * 2` for forward compatibility. +The Python expression builder shown above works regardless of dialect. + +
+ +## Testing membership in a list + +A common need is filtering rows where a column equals *any* of a small set of +values. DataFusion offers three forms; they differ in readability and in how +they scale: + +1. A compound boolean using `|` across explicit equalities. +2. [`in_list`][datafusion.functions.in_list], which accepts a list of + expressions and tests equality against all of them in one call. +3. A trick with [`array_position`][datafusion.functions.array_position] and + [`make_array`][datafusion.functions.make_array], which returns the 1-based + index of the value in a constructed array, or null if it is not present. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import SessionContext, col, lit +from datafusion import functions as f + +ctx = SessionContext() +df = ctx.from_pydict({"shipmode": ["MAIL", "SHIP", "AIR", "TRUCK", "RAIL"]}) + +# Option 1: compound boolean. Fine for two values; awkward past three. +df.filter((col("shipmode") == lit("MAIL")) | (col("shipmode") == lit("SHIP"))) + +# Option 2: in_list. Preferred for readability as the set grows. +df.filter(f.in_list(col("shipmode"), [lit("MAIL"), lit("SHIP")])) + +# Option 3: array_position / make_array. Useful when you already have the +# set as an array column and want "is in that array" semantics. +df.filter( + ~f.array_position(f.make_array(lit("MAIL"), lit("SHIP")), col("shipmode")).is_null() +) +``` + + +Use `in_list` as the default. It is explicit, readable, and matches the +semantics users expect from SQL's `IN (...)`. Reach for the +`array_position` form only when the membership set is itself an array +column rather than a literal list. + +## Conditional expressions + +DataFusion provides [`case`][datafusion.functions.case] for the SQL +`CASE` expression in both its switched and searched forms, along with +[`when`][datafusion.functions.when] as a standalone builder for the +searched form. + +**Switched CASE** (one expression compared against several literal values): + +```python exec="1" source="material-block" result="text" session="expressions" +df = ctx.from_pydict( + {"priority": ["1-URGENT", "2-HIGH", "3-MEDIUM", "5-LOW"]}, +) + +df.select( + col("priority"), + f.case(col("priority")) + .when(lit("1-URGENT"), lit(1)) + .when(lit("2-HIGH"), lit(1)) + .otherwise(lit(0)) + .alias("is_high_priority"), +) +``` + + +**Searched CASE** (an independent boolean predicate per branch). Use this +form whenever a branch tests more than simple equality — for example, +checking whether a joined column is `NULL` to gate a computed value: + +```python exec="1" source="material-block" result="text" session="expressions" +df = ctx.from_pydict( + {"volume": [10.0, 20.0, 30.0], "supplier_id": [1, None, 2]}, +) + +df.select( + col("volume"), + col("supplier_id"), + f.when(col("supplier_id").is_not_null(), col("volume")) + .otherwise(lit(0.0)) + .alias("attributed_volume"), +) +``` + + +This searched-CASE pattern is idiomatic for "attribute the measure to the +matching side of a left join, otherwise contribute zero" — a shape that +appears in TPC-H Q08 and similar market-share calculations. + +If a switched CASE only groups several equality matches into one bucket, +`f.when(f.in_list(col(...), [...]), value).otherwise(default)` is often +simpler than the full `case` builder. + +## Structs + +Columns that contain struct elements can be accessed using the bracket notation as if they were +Python dictionary style objects. This expects a string key as the parameter passed. + +```python exec="1" source="material-block" result="text" session="expressions" +ctx = SessionContext() +data = {"a": [{"size": 15, "color": "green"}, {"size": 10, "color": "blue"}]} +df = ctx.from_pydict(data) +df.select(col("a")["size"].alias("a_size")) +``` + + +## Functions + +As mentioned before, most functions in DataFusion return an expression at their output. This allows us to create +a wide variety of expressions built up from other expressions. For example, [`alias`][datafusion.expr.Expr.alias] is a function that takes +as it input a single expression and returns an expression in which the name of the expression has changed. + +The following example shows a series of expressions that are built up from functions operating on expressions. + +```python exec="1" source="material-block" result="text" session="expressions" +from datafusion import SessionContext, lit +from datafusion import functions as f + +ctx = SessionContext() +df = ctx.from_pydict( + { + "name": ["Albert", "Becca", "Carlos", "Dante"], + "age": [42, 67, 27, 71], + "years_in_position": [13, 21, 10, 54], + }, + name="employees", +) + +age_col = col("age") +renamed_age = age_col.alias("age_in_years") +start_age = age_col - col("years_in_position") +started_young = start_age < lit(18) +can_retire = age_col > lit(65) +long_timer = started_young & can_retire + +df.filter(long_timer).select(col("name"), renamed_age, col("years_in_position")) +``` diff --git a/docs/source/user-guide/common-operations/functions.ipynb b/docs/source/user-guide/common-operations/functions.ipynb deleted file mode 100644 index fe9c803a6..000000000 --- a/docs/source/user-guide/common-operations/functions.ipynb +++ /dev/null @@ -1,242 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n# Functions\n\nDataFusion provides a large number of built-in functions for performing complex queries without requiring user-defined functions.\nIn here we will cover some of the more popular use cases. If you want to view all the functions go to the [`Functions`][datafusion.functions] API Reference.\n\nWe'll use the pokemon dataset in the following examples.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "ctx.register_csv(\"pokemon\", \"pokemon.csv\")\n", - "df = ctx.table(\"pokemon\")" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\n## Mathematical\n\nDataFusion offers mathematical functions such as [`pow`][datafusion.functions.pow] or [`log`][datafusion.functions.log]\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import str_lit, string_literal\n", - "\n", - "df.select(\n", - " f.pow(col('\"Attack\"'), literal(2)) - f.pow(col('\"Defense\"'), literal(2))\n", - ").limit(10)" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\n## Conditional\n\nThere 3 conditional functions in DataFusion [`coalesce`][datafusion.functions.coalesce], [`nullif`][datafusion.functions.nullif] and [`case`][datafusion.functions.case].\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(f.coalesce(col('\"Type 1\"'), col('\"Type 2\"')).alias(\"dominant_type\")).limit(10)" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\n## Temporal\n\nFor selecting the current time use [`now`][datafusion.functions.now]\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(f.now())" - ] - }, - { - "cell_type": "markdown", - "id": "7cdc8c89c7104fffa095e18ddfef8986", - "metadata": {}, - "source": "\nConvert to timestamps using [`to_timestamp`][datafusion.functions.to_timestamp]\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b118ea5561624da68c537baed56e602f", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(f.to_timestamp(col('\"Total\"')).alias(\"timestamp\"))" - ] - }, - { - "cell_type": "markdown", - "id": "938c804e27f84196a10c8828c723f798", - "metadata": {}, - "source": "\nExtracting parts of a date using [`date_part`][datafusion.functions.date_part] (alias [`extract`][datafusion.functions.extract])\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "504fb2a444614c0babb325280ed9130a", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(\n", - " f.date_part(literal(\"month\"), f.to_timestamp(col('\"Total\"'))).alias(\"month\"),\n", - " f.extract(literal(\"day\"), f.to_timestamp(col('\"Total\"'))).alias(\"day\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "59bbdb311c014d738909a11f9e486628", - "metadata": {}, - "source": "\n## String\n\nIn the field of data science, working with textual data is a common task. To make string manipulation easier,\nDataFusion offers a range of helpful options.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b43b363d81ae4b689946ece5c682cd59", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(\n", - " f.char_length(col('\"Name\"')).alias(\"len\"),\n", - " f.lower(col('\"Name\"')).alias(\"lower\"),\n", - " f.left(col('\"Name\"'), literal(4)).alias(\"code\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "8a65eabff63a45729fe45fb5ade58bdc", - "metadata": {}, - "source": "\nThis also includes the functions for regular expressions like [`regexp_replace`][datafusion.functions.regexp_replace] and [`regexp_match`][datafusion.functions.regexp_match]\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c3933fab20d04ec698c2621248eb3be0", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(\n", - " f.regexp_match(col('\"Name\"'), literal(\"Char\")).alias(\"dragons\"),\n", - " f.regexp_replace(col('\"Name\"'), literal(\"saur\"), literal(\"fleur\")).alias(\"flowers\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "4dd4641cc4064e0191573fe9c69df29b", - "metadata": {}, - "source": "\n## Casting\n\nCasting expressions to different data types using [`arrow_cast`][datafusion.functions.arrow_cast]\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8309879909854d7188b41380fd92a7c3", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(\n", - " f.arrow_cast(col('\"Total\"'), string_literal(\"Float64\")).alias(\"total_as_float\"),\n", - " f.arrow_cast(col('\"Total\"'), str_lit(\"Int32\")).alias(\"total_as_int\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "3ed186c9a28b402fb0bc4494df01f08d", - "metadata": {}, - "source": "\n## Other\n\nThe function [`in_list`][datafusion.functions.in_list] allows to check a column for the presence of multiple values:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "cb1e1581032b452c9409d6c6813c49d1", - "metadata": {}, - "outputs": [], - "source": [ - "types = [literal(\"Grass\"), literal(\"Fire\"), literal(\"Water\")]\n", - "(\n", - " df.select(f.in_list(col('\"Type 1\"'), types, negated=False).alias(\"basic_types\"))\n", - " .limit(20)\n", - " .to_pandas()\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "379cbbc1e968416e875cc15c1202d7eb", - "metadata": {}, - "source": "\n# Handling Missing Values\n\nDataFusion provides methods to handle missing values in DataFrames:\n\n## fill_null\n\nThe `fill_null()` method replaces NULL values in specified columns with a provided value:\n\n```python\n# Fill all NULL values with 0 where possible\ndf = df.fill_null(0)\n\n# Fill NULL values only in specific string columns\ndf = df.fill_null(\"missing\", subset=[\"name\", \"category\"])\n```\n\nThe fill value will be cast to match each column's type. If casting fails for a column, that column remains unchanged.\n" - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/common-operations/functions.md b/docs/source/user-guide/common-operations/functions.md new file mode 100644 index 000000000..43f90f6fb --- /dev/null +++ b/docs/source/user-guide/common-operations/functions.md @@ -0,0 +1,173 @@ +```python exec="1" session="functions" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + +# Functions + +DataFusion provides a large number of built-in functions for performing complex queries without requiring user-defined functions. +In here we will cover some of the more popular use cases. If you want to view all the functions go to the [`Functions`][datafusion.functions] API Reference. + +We'll use the pokemon dataset in the following examples. + +```python exec="1" source="material-block" result="text" session="functions" +ctx = SessionContext() +ctx.register_csv("pokemon", "pokemon.csv") +df = ctx.table("pokemon") +``` + + +## Mathematical + +DataFusion offers mathematical functions such as [`pow`][datafusion.functions.pow] or [`log`][datafusion.functions.log] + +```python exec="1" source="material-block" result="text" session="functions" +from datafusion import str_lit, string_literal + +df.select( + f.pow(col('"Attack"'), literal(2)) - f.pow(col('"Defense"'), literal(2)) +).limit(10) +``` + + +## Conditional + +There 3 conditional functions in DataFusion [`coalesce`][datafusion.functions.coalesce], [`nullif`][datafusion.functions.nullif] and [`case`][datafusion.functions.case]. + +```python exec="1" source="material-block" result="text" session="functions" +df.select(f.coalesce(col('"Type 1"'), col('"Type 2"')).alias("dominant_type")).limit(10) +``` + + +## Temporal + +For selecting the current time use [`now`][datafusion.functions.now] + +```python exec="1" source="material-block" result="text" session="functions" +df.select(f.now()) +``` + + +Convert to timestamps using [`to_timestamp`][datafusion.functions.to_timestamp] + +```python exec="1" source="material-block" result="text" session="functions" +df.select(f.to_timestamp(col('"Total"')).alias("timestamp")) +``` + + +Extracting parts of a date using [`date_part`][datafusion.functions.date_part] (alias [`extract`][datafusion.functions.extract]) + +```python exec="1" source="material-block" result="text" session="functions" +df.select( + f.date_part(literal("month"), f.to_timestamp(col('"Total"'))).alias("month"), + f.extract(literal("day"), f.to_timestamp(col('"Total"'))).alias("day"), +) +``` + + +## String + +In the field of data science, working with textual data is a common task. To make string manipulation easier, +DataFusion offers a range of helpful options. + +```python exec="1" source="material-block" result="text" session="functions" +df.select( + f.char_length(col('"Name"')).alias("len"), + f.lower(col('"Name"')).alias("lower"), + f.left(col('"Name"'), literal(4)).alias("code"), +) +``` + + +This also includes the functions for regular expressions like [`regexp_replace`][datafusion.functions.regexp_replace] and [`regexp_match`][datafusion.functions.regexp_match] + +```python exec="1" source="material-block" result="text" session="functions" +df.select( + f.regexp_match(col('"Name"'), literal("Char")).alias("dragons"), + f.regexp_replace(col('"Name"'), literal("saur"), literal("fleur")).alias("flowers"), +) +``` + + +## Casting + +Casting expressions to different data types using [`arrow_cast`][datafusion.functions.arrow_cast] + +```python exec="1" source="material-block" result="text" session="functions" +df.select( + f.arrow_cast(col('"Total"'), string_literal("Float64")).alias("total_as_float"), + f.arrow_cast(col('"Total"'), str_lit("Int32")).alias("total_as_int"), +) +``` + + +## Other + +The function [`in_list`][datafusion.functions.in_list] allows to check a column for the presence of multiple values: + +```python exec="1" source="material-block" result="text" session="functions" +types = [literal("Grass"), literal("Fire"), literal("Water")] +( + df.select(f.in_list(col('"Type 1"'), types, negated=False).alias("basic_types")) + .limit(20) + .to_pandas() +) +``` + + +# Handling Missing Values + +DataFusion provides methods to handle missing values in DataFrames: + +## fill_null + +The `fill_null()` method replaces NULL values in specified columns with a provided value: + +```python +# Fill all NULL values with 0 where possible +df = df.fill_null(0) + +# Fill NULL values only in specific string columns +df = df.fill_null("missing", subset=["name", "category"]) +``` + +The fill value will be cast to match each column's type. If casting fails for a column, that column remains unchanged. diff --git a/docs/source/user-guide/common-operations/index.md b/docs/source/user-guide/common-operations/index.md index 384bce3e0..3a314085b 100644 --- a/docs/source/user-guide/common-operations/index.md +++ b/docs/source/user-guide/common-operations/index.md @@ -23,19 +23,19 @@ The contents of this section are designed to guide a new user through how to use ## Contents -- [Basic Info](basic-info.ipynb) — inspecting schema, row counts, and +- [Basic Info](basic-info.md) — inspecting schema, row counts, and summary statistics. - [Views](views.md) — saving and reusing query fragments as views. -- [Select and Filter](select-and-filter.ipynb) — projecting columns and +- [Select and Filter](select-and-filter.md) — projecting columns and applying predicates. -- [Expressions](expressions.ipynb) — `col`, `lit`, boolean operators, +- [Expressions](expressions.md) — `col`, `lit`, boolean operators, array indexing, and chaining. -- [Joins](joins.ipynb) — inner / outer / semi / anti joins. -- [Functions](functions.ipynb) — scalar functions across math, string, +- [Joins](joins.md) — inner / outer / semi / anti joins. +- [Functions](functions.md) — scalar functions across math, string, date/time, and array families. -- [Aggregations](aggregations.ipynb) — `group_by`, rollup, cube, +- [Aggregations](aggregations.md) — `group_by`, rollup, cube, grouping sets. -- [Windows](windows.ipynb) — partitioned and ranking window functions. -- [User-Defined Functions](udf-and-udfa.ipynb) — scalar (UDF), +- [Windows](windows.md) — partitioned and ranking window functions. +- [User-Defined Functions](udf-and-udfa.md) — scalar (UDF), aggregate (UDAF), window (UDWF), and table (UDTF) user-defined functions. diff --git a/docs/source/user-guide/common-operations/joins.ipynb b/docs/source/user-guide/common-operations/joins.ipynb deleted file mode 100644 index 71ec27f9e..000000000 --- a/docs/source/user-guide/common-operations/joins.ipynb +++ /dev/null @@ -1,242 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f # noqa: F401\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n# Joins\n\nDataFusion supports the following join variants via the method [`join`][datafusion.dataframe.DataFrame.join]\n\n- Inner Join\n- Left Join\n- Right Join\n- Full Join\n- Left Semi Join\n- Left Anti Join\n\nFor the examples in this section we'll use the following two DataFrames\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "\n", - "left = ctx.from_pydict(\n", - " {\n", - " \"customer_id\": [1, 2, 3],\n", - " \"customer\": [\"Alice\", \"Bob\", \"Charlie\"],\n", - " }\n", - ")\n", - "\n", - "right = ctx.from_pylist(\n", - " [\n", - " {\"id\": 1, \"name\": \"CityCabs\"},\n", - " {\"id\": 2, \"name\": \"MetroRide\"},\n", - " {\"id\": 5, \"name\": \"UrbanGo\"},\n", - " ]\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\n## Inner Join\n\nWhen using an inner join, only rows containing the common values between the two join columns present in both DataFrames\nwill be included in the resulting DataFrame.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"inner\")" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\nThe parameter `join_keys` specifies the columns from the left DataFrame and right DataFrame that contains the values\nthat should match.\n\n## Left Join\n\nA left join combines rows from two DataFrames using the key columns. It returns all rows from the left DataFrame and\nmatching rows from the right DataFrame. If there's no match in the right DataFrame, it returns null\nvalues for the corresponding columns.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"left\")" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\n## Full Join\n\nA full join merges rows from two tables based on a related column, returning all rows from both tables, even if there\nis no match. Unmatched rows will have null values.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"full\")" - ] - }, - { - "cell_type": "markdown", - "id": "7cdc8c89c7104fffa095e18ddfef8986", - "metadata": {}, - "source": "\n## Left Semi Join\n\nA left semi join retrieves matching rows from the left table while\nomitting duplicates with multiple matches in the right table.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b118ea5561624da68c537baed56e602f", - "metadata": {}, - "outputs": [], - "source": [ - "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"semi\")" - ] - }, - { - "cell_type": "markdown", - "id": "938c804e27f84196a10c8828c723f798", - "metadata": {}, - "source": "\n## Left Anti Join\n\nA left anti join shows all rows from the left table without any matching rows in the right table,\nbased on a the specified matching columns. It excludes rows from the left table that have at least one matching row in\nthe right table.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "504fb2a444614c0babb325280ed9130a", - "metadata": {}, - "outputs": [], - "source": [ - "left.join(right, left_on=\"customer_id\", right_on=\"id\", how=\"anti\")" - ] - }, - { - "cell_type": "markdown", - "id": "59bbdb311c014d738909a11f9e486628", - "metadata": {}, - "source": "\n## Duplicate Keys\n\nIt is common to join two DataFrames on a common column name. Starting in\nversion 51.0.0, `` datafusion-python` `` will now coalesce on column with identical names by\ndefault. This reduces problems with ambiguous column selection after joins.\nYou can disable this feature by setting the parameter `coalesce_duplicate_keys`\nto `False`.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b43b363d81ae4b689946ece5c682cd59", - "metadata": {}, - "outputs": [], - "source": [ - "left = ctx.from_pydict(\n", - " {\n", - " \"id\": [1, 2, 3],\n", - " \"customer\": [\"Alice\", \"Bob\", \"Charlie\"],\n", - " }\n", - ")\n", - "\n", - "right = ctx.from_pylist(\n", - " [\n", - " {\"id\": 1, \"name\": \"CityCabs\"},\n", - " {\"id\": 2, \"name\": \"MetroRide\"},\n", - " {\"id\": 5, \"name\": \"UrbanGo\"},\n", - " ]\n", - ")\n", - "\n", - "left.join(right, \"id\", how=\"inner\")" - ] - }, - { - "cell_type": "markdown", - "id": "8a65eabff63a45729fe45fb5ade58bdc", - "metadata": {}, - "source": "\nIn contrast to the above example, if we wish to get both columns:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c3933fab20d04ec698c2621248eb3be0", - "metadata": {}, - "outputs": [], - "source": [ - "left.join(right, \"id\", how=\"inner\", coalesce_duplicate_keys=False)" - ] - }, - { - "cell_type": "markdown", - "id": "4dd4641cc4064e0191573fe9c69df29b", - "metadata": {}, - "source": "\n## Disambiguating Columns with `DataFrame.col()`\n\nWhen both DataFrames contain non-key columns with the same name, you can use\n[`col`][datafusion.dataframe.DataFrame.col] on each DataFrame **before** the\njoin to create fully qualified column references. These references can then be\nused in the join predicate and when selecting from the result.\n\nThis is especially useful with [`join_on`][datafusion.dataframe.DataFrame.join_on],\nwhich accepts expression-based predicates.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8309879909854d7188b41380fd92a7c3", - "metadata": {}, - "outputs": [], - "source": [ - "left = ctx.from_pydict(\n", - " {\n", - " \"id\": [1, 2, 3],\n", - " \"val\": [10, 20, 30],\n", - " }\n", - ")\n", - "\n", - "right = ctx.from_pydict(\n", - " {\n", - " \"id\": [1, 2, 3],\n", - " \"val\": [40, 50, 60],\n", - " }\n", - ")\n", - "\n", - "joined = left.join_on(right, left.col(\"id\") == right.col(\"id\"), how=\"inner\")\n", - "\n", - "joined.select(left.col(\"id\"), left.col(\"val\"), right.col(\"val\"))" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/common-operations/joins.md b/docs/source/user-guide/common-operations/joins.md new file mode 100644 index 000000000..a999803bb --- /dev/null +++ b/docs/source/user-guide/common-operations/joins.md @@ -0,0 +1,196 @@ +```python exec="1" session="joins" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + +# Joins + +DataFusion supports the following join variants via the method [`join`][datafusion.dataframe.DataFrame.join] + +- Inner Join +- Left Join +- Right Join +- Full Join +- Left Semi Join +- Left Anti Join + +For the examples in this section we'll use the following two DataFrames + +```python exec="1" source="material-block" result="text" session="joins" +ctx = SessionContext() + +left = ctx.from_pydict( + { + "customer_id": [1, 2, 3], + "customer": ["Alice", "Bob", "Charlie"], + } +) + +right = ctx.from_pylist( + [ + {"id": 1, "name": "CityCabs"}, + {"id": 2, "name": "MetroRide"}, + {"id": 5, "name": "UrbanGo"}, + ] +) +``` + + +## Inner Join + +When using an inner join, only rows containing the common values between the two join columns present in both DataFrames +will be included in the resulting DataFrame. + +```python exec="1" source="material-block" result="text" session="joins" +left.join(right, left_on="customer_id", right_on="id", how="inner") +``` + + +The parameter `join_keys` specifies the columns from the left DataFrame and right DataFrame that contains the values +that should match. + +## Left Join + +A left join combines rows from two DataFrames using the key columns. It returns all rows from the left DataFrame and +matching rows from the right DataFrame. If there's no match in the right DataFrame, it returns null +values for the corresponding columns. + +```python exec="1" source="material-block" result="text" session="joins" +left.join(right, left_on="customer_id", right_on="id", how="left") +``` + + +## Full Join + +A full join merges rows from two tables based on a related column, returning all rows from both tables, even if there +is no match. Unmatched rows will have null values. + +```python exec="1" source="material-block" result="text" session="joins" +left.join(right, left_on="customer_id", right_on="id", how="full") +``` + + +## Left Semi Join + +A left semi join retrieves matching rows from the left table while +omitting duplicates with multiple matches in the right table. + +```python exec="1" source="material-block" result="text" session="joins" +left.join(right, left_on="customer_id", right_on="id", how="semi") +``` + + +## Left Anti Join + +A left anti join shows all rows from the left table without any matching rows in the right table, +based on a the specified matching columns. It excludes rows from the left table that have at least one matching row in +the right table. + +```python exec="1" source="material-block" result="text" session="joins" +left.join(right, left_on="customer_id", right_on="id", how="anti") +``` + + +## Duplicate Keys + +It is common to join two DataFrames on a common column name. Starting in +version 51.0.0, `` datafusion-python` `` will now coalesce on column with identical names by +default. This reduces problems with ambiguous column selection after joins. +You can disable this feature by setting the parameter `coalesce_duplicate_keys` +to `False`. + +```python exec="1" source="material-block" result="text" session="joins" +left = ctx.from_pydict( + { + "id": [1, 2, 3], + "customer": ["Alice", "Bob", "Charlie"], + } +) + +right = ctx.from_pylist( + [ + {"id": 1, "name": "CityCabs"}, + {"id": 2, "name": "MetroRide"}, + {"id": 5, "name": "UrbanGo"}, + ] +) + +left.join(right, "id", how="inner") +``` + + +In contrast to the above example, if we wish to get both columns: + +```python exec="1" source="material-block" result="text" session="joins" +left.join(right, "id", how="inner", coalesce_duplicate_keys=False) +``` + + +## Disambiguating Columns with `DataFrame.col()` + +When both DataFrames contain non-key columns with the same name, you can use +[`col`][datafusion.dataframe.DataFrame.col] on each DataFrame **before** the +join to create fully qualified column references. These references can then be +used in the join predicate and when selecting from the result. + +This is especially useful with [`join_on`][datafusion.dataframe.DataFrame.join_on], +which accepts expression-based predicates. + +```python exec="1" source="material-block" result="text" session="joins" +left = ctx.from_pydict( + { + "id": [1, 2, 3], + "val": [10, 20, 30], + } +) + +right = ctx.from_pydict( + { + "id": [1, 2, 3], + "val": [40, 50, 60], + } +) + +joined = left.join_on(right, left.col("id") == right.col("id"), how="inner") + +joined.select(left.col("id"), left.col("val"), right.col("val")) +``` diff --git a/docs/source/user-guide/common-operations/select-and-filter.ipynb b/docs/source/user-guide/common-operations/select-and-filter.ipynb deleted file mode 100644 index dc0daaa5a..000000000 --- a/docs/source/user-guide/common-operations/select-and-filter.ipynb +++ /dev/null @@ -1,120 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f # noqa: F401\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n# Column Selections\n\nUse [`select`][datafusion.dataframe.DataFrame.select] for basic column selection.\n\nDataFusion can work with several file types, to start simple we can use a subset of the\n[TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page),\nwhich you can download [here](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet).\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "df = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n", - "df.select(\"trip_distance\", \"passenger_count\")" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\nFor mathematical or logical operations use [`col`][datafusion.col.col] to select columns, and give meaningful names to the resulting\noperations using [`alias`][datafusion.expr.Expr.alias]\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "df.select((col(\"tip_amount\") + col(\"tolls_amount\")).alias(\"tips_plus_tolls\"))" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\n
\n

Warning

\n\nPlease be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters\n(ex: Name) you must put your column name in double quotes or the selection won’t work. As an alternative for simple\ncolumn selection use [`select`][datafusion.dataframe.DataFrame.select] without double quotes\n\n
\n\nFor selecting columns with capital letters use `'\"VendorID\"'`\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(col('\"VendorID\"'))" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\nTo combine it with literal values use the [`lit`][datafusion.lit]\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "large_trip_distance = col(\"trip_distance\") > lit(5.0)\n", - "low_passenger_count = col(\"passenger_count\") < lit(4)\n", - "df.select((large_trip_distance & low_passenger_count).alias(\"lonely_trips\"))" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/common-operations/select-and-filter.md b/docs/source/user-guide/common-operations/select-and-filter.md new file mode 100644 index 000000000..7df6c5889 --- /dev/null +++ b/docs/source/user-guide/common-operations/select-and-filter.md @@ -0,0 +1,90 @@ +```python exec="1" session="select-and-filter" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + +# Column Selections + +Use [`select`][datafusion.dataframe.DataFrame.select] for basic column selection. + +DataFusion can work with several file types, to start simple we can use a subset of the +[TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page), +which you can download [here](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet). + +```python exec="1" source="material-block" result="text" session="select-and-filter" +ctx = SessionContext() +df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") +df.select("trip_distance", "passenger_count") +``` + + +For mathematical or logical operations use [`col`][datafusion.col.col] to select columns, and give meaningful names to the resulting +operations using [`alias`][datafusion.expr.Expr.alias] + +```python exec="1" source="material-block" result="text" session="select-and-filter" +df.select((col("tip_amount") + col("tolls_amount")).alias("tips_plus_tolls")) +``` + + +
+

Warning

+ +Please be aware that all identifiers are effectively made lower-case in SQL, so if your file has capital letters +(ex: Name) you must put your column name in double quotes or the selection won’t work. As an alternative for simple +column selection use [`select`][datafusion.dataframe.DataFrame.select] without double quotes + +
+ +For selecting columns with capital letters use `'"VendorID"'` + +```python exec="1" source="material-block" result="text" session="select-and-filter" +df.select(col('"VendorID"')) +``` + + +To combine it with literal values use the [`lit`][datafusion.lit] + +```python exec="1" source="material-block" result="text" session="select-and-filter" +large_trip_distance = col("trip_distance") > lit(5.0) +low_passenger_count = col("passenger_count") < lit(4) +df.select((large_trip_distance & low_passenger_count).alias("lonely_trips")) +``` diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb b/docs/source/user-guide/common-operations/udf-and-udfa.ipynb deleted file mode 100644 index 676f5a6b3..000000000 --- a/docs/source/user-guide/common-operations/udf-and-udfa.ipynb +++ /dev/null @@ -1,210 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f # noqa: F401\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n# User-Defined Functions\n\nDataFusion provides powerful expressions and functions, reducing the need for custom Python\nfunctions. However you can still incorporate your own functions, i.e. User-Defined Functions (UDFs).\n\n## Scalar Functions\n\nWhen writing a user-defined function that can operate on a row by row basis, these are called Scalar\nFunctions. You can define your own scalar function by calling\n[`udf`][datafusion.user_defined.ScalarUDF.udf] .\n\nThe basic definition of a scalar UDF is a python function that takes one or more\n[pyarrow](https://arrow.apache.org/docs/python/index.html) arrays and returns a single array as\noutput. DataFusion scalar UDFs operate on an entire batch of records at a time, though the\nevaluation of those records should be on a row by row basis. In the following example, we compute\nif the input array contains null values.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "import pyarrow\n", - "from datafusion import udf\n", - "\n", - "\n", - "def is_null(array: pyarrow.Array) -> pyarrow.Array:\n", - " return array.is_null()\n", - "\n", - "\n", - "is_null_arr = udf(is_null, [pyarrow.int64()], pyarrow.bool_(), \"stable\")\n", - "\n", - "ctx = datafusion.SessionContext()\n", - "\n", - "batch = pyarrow.RecordBatch.from_arrays(\n", - " [pyarrow.array([1, None, 3]), pyarrow.array([4, 5, 6])],\n", - " names=[\"a\", \"b\"],\n", - ")\n", - "df = ctx.create_dataframe([[batch]], name=\"batch_array\")\n", - "\n", - "df.select(col(\"a\"), is_null_arr(col(\"a\")).alias(\"is_null\")).show()" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\nIn the previous example, we used the fact that pyarrow provides a variety of built in array\nfunctions such as `is_null()`. There are additional pyarrow\n[compute functions](https://arrow.apache.org/docs/python/compute.html) available. When possible,\nit is highly recommended to use these functions because they can perform computations without doing\nany copy operations from the original arrays. This leads to greatly improved performance.\n\nIf you need to perform an operation in python that is not available with the pyarrow compute\nfunctions, you will need to convert the record batch into python values, perform your operation,\nand construct an array. This operation of converting the built in data type of the array into a\npython object can be one of the slowest operations in DataFusion, so it should be done sparingly.\n\nThe following example performs the same operation as before with `is_null` but demonstrates\nconverting to Python objects to do the evaluation.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "import datafusion\n", - "import pyarrow\n", - "from datafusion import col, udf\n", - "\n", - "\n", - "def is_null(array: pyarrow.Array) -> pyarrow.Array:\n", - " return pyarrow.array([value.as_py() is None for value in array])\n", - "\n", - "\n", - "is_null_arr = udf(is_null, [pyarrow.int64()], pyarrow.bool_(), \"stable\")\n", - "\n", - "ctx = datafusion.SessionContext()\n", - "\n", - "batch = pyarrow.RecordBatch.from_arrays(\n", - " [pyarrow.array([1, None, 3]), pyarrow.array([4, 5, 6])],\n", - " names=[\"a\", \"b\"],\n", - ")\n", - "df = ctx.create_dataframe([[batch]], name=\"batch_array\")\n", - "\n", - "df.select(col(\"a\"), is_null_arr(col(\"a\")).alias(\"is_null\")).show()" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\nIn this example we passed the PyArrow `DataType` when we defined the function\nby calling `udf()`. If you need additional control, such as specifying\nmetadata or nullability of the input or output, you can instead specify a\nPyArrow `Field`.\n\nIf you need to write a custom function but do not want to incur the performance\ncost of converting to Python objects and back, a more advanced approach is to\nwrite Rust based UDFs and to expose them to Python. There is an example in the\n[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/)\ndescribing how to do this.\n\n### When not to use a UDF\n\nA UDF is the right tool when the per-row computation genuinely cannot be\nexpressed with DataFusion's built-in expressions. It is often the *wrong*\ntool for a predicate that *can* be written as an `Expr` tree but feels\neasier to write as a Python function — for example, a filter that keeps\na row if it matches any one of several rule sets, where each rule set\nchecks its own combination of columns (the worked example at the end of\nthis section keeps a row when it matches any one of several brand-specific\nrules). Looping over the rules in Python and returning a boolean per row\nreads naturally and is tempting to wrap in a UDF, but a UDF is opaque to\nthe optimizer: filters expressed as UDFs lose several rewrites that the\nengine applies to filters built from native expressions. The most visible\nof these is **predicate pushdown into the table provider**: a native\npredicate can be handed to the source so it skips data before it is read,\nwhile a UDF predicate cannot. The example below uses Parquet, where\npushdown prunes whole row groups using the min/max statistics in the\nfooter, but the same mechanism applies to any table provider that\nadvertises filter support — including custom providers.\n\nThe following example writes a small Parquet file, then filters it two\nways: first with a native expression, then with a UDF that computes the\nsame result. The filter itself is simple on purpose so we can compare\nthe plans side by side.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "import os\n", - "import tempfile\n", - "\n", - "import pyarrow as pa\n", - "import pyarrow.parquet as pq\n", - "from datafusion import col, udf\n", - "\n", - "tmpdir = tempfile.mkdtemp()\n", - "parquet_path = os.path.join(tmpdir, \"items.parquet\")\n", - "pq.write_table(\n", - " pa.table(\n", - " {\n", - " \"id\": list(range(100)),\n", - " \"brand\": [\"A\", \"B\", \"C\", \"D\"] * 25,\n", - " \"qty\": [i * 10 for i in range(100)],\n", - " }\n", - " ),\n", - " parquet_path,\n", - ")\n", - "\n", - "ctx = SessionContext()\n", - "items = ctx.read_parquet(parquet_path)" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\n**Native-expression predicate.** The filter is a plain boolean tree\nover column references and literals, so the optimizer can analyze it:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "native_filtered = items.filter((col(\"brand\") == lit(\"A\")) & (col(\"qty\") >= lit(150)))\n", - "print(native_filtered.execution_plan().display_indent())" - ] - }, - { - "cell_type": "markdown", - "id": "7cdc8c89c7104fffa095e18ddfef8986", - "metadata": {}, - "source": "\nNotice the `DataSourceExec` line. It carries three annotations the\noptimizer computed from the predicate:\n\n- `predicate=brand@1 = A AND qty@2 >= 150` — the filter is pushed\n into the Parquet scan itself, so the scan only reads matching rows.\n- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ...\n qty_max@4 >= 150` — the scan prunes whole row groups by consulting\n the Parquet min/max statistics in the footer *before* reading any\n column data.\n- `required_guarantees=[brand in (A)]` — the scan uses this when a\n bloom filter or dictionary is available to skip pages.\n\n**UDF predicate.** Now wrap the same logic in a Python UDF:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b118ea5561624da68c537baed56e602f", - "metadata": {}, - "outputs": [], - "source": [ - "def brand_qty_filter(brand_arr: pa.Array, qty_arr: pa.Array) -> pa.Array:\n", - " return pa.array(\n", - " [b.as_py() == \"A\" and q.as_py() >= 150 for b, q in zip(brand_arr, qty_arr)]\n", - " )\n", - "\n", - "\n", - "pred_udf = udf(\n", - " brand_qty_filter,\n", - " [pa.string(), pa.int64()],\n", - " pa.bool_(),\n", - " \"stable\",\n", - ")\n", - "udf_filtered = items.filter(pred_udf(col(\"brand\"), col(\"qty\")))\n", - "print(udf_filtered.execution_plan().display_indent())" - ] - }, - { - "cell_type": "markdown", - "id": "938c804e27f84196a10c8828c723f798", - "metadata": {}, - "source": "\nThe `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`.\nThere is no `pruning_predicate` and no `required_guarantees`: the\nscan has to materialize every row group and hand each row to the\nPython callback just to decide whether to keep it.\n\nAt small scale the cost difference is invisible; on a Parquet file with\nmany row groups, or data whose min/max statistics line up well with\nthe predicate, the native form can skip most of the file. The UDF form\nreads all of it.\n\n**Takeaway.** Reach for a UDF when the per-row computation is genuinely\nnot expressible as a tree of built-in functions (custom numerical work,\nexternal lookups, complex business rules). When it *is* expressible —\neven if the native form is a little more verbose — build the `Expr`\ntree directly so the optimizer can see through it. For disjunctive\npredicates the idiom is to produce one clause per bucket and combine\nthem with `|`:\n\n```python\nfrom functools import reduce\nfrom operator import or_\nfrom datafusion import col, lit, functions as f\n\nbuckets = {\n \"Brand#12\": {\"containers\": [\"SM CASE\", \"SM BOX\"], \"min_qty\": 1, \"max_size\": 5},\n \"Brand#23\": {\"containers\": [\"MED BAG\", \"MED BOX\"], \"min_qty\": 10, \"max_size\": 10},\n}\n\ndef bucket_clause(brand, spec):\n return (\n (col(\"brand\") == lit(brand))\n & f.in_list(col(\"container\"), [lit(c) for c in spec[\"containers\"]])\n & (col(\"quantity\") >= lit(spec[\"min_qty\"]))\n & (col(\"quantity\") <= lit(spec[\"min_qty\"] + 10))\n & (col(\"size\") >= lit(1))\n & (col(\"size\") <= lit(spec[\"max_size\"]))\n )\n\npredicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items()))\ndf = df.filter(predicate)\n```\n\n## Aggregate Functions\n\nThe [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined\nAggregate Functions (UDAFs). To use this you must implement an\n[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed.\n\nWhen defining a UDAF there are four methods you need to implement. The `update` function takes the\narray(s) of input and updates the internal state of the accumulator. You should define this function\nto have as many input arguments as you will pass when calling the UDAF. Since aggregation may be\nsplit into multiple batches, we must have a method to combine multiple batches. For this, we have\ntwo functions, `state` and `merge`. `state` will return an array of scalar values that contain\nthe current state of a single batch accumulation. Then we must `merge` the results of these\ndifferent states. Finally `evaluate` is the call that will return the final result after the\n`merge` is complete.\n\nIn the following example we want to define a custom aggregate function that will return the\ndifference between the sum of two columns. The state can be represented by a single value and we can\nalso see how the inputs to `update` and `merge` differ.\n\n```python\nimport pyarrow as pa\nimport pyarrow.compute\nimport datafusion\nfrom datafusion import col, udaf, Accumulator\nfrom typing import List\n\nclass MyAccumulator(Accumulator):\n \"\"\"\n Interface of a user-defined accumulation.\n \"\"\"\n def __init__(self):\n self._sum = 0.0\n\n def update(self, values_a: pa.Array, values_b: pa.Array) -> None:\n self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py()\n\n def merge(self, states: list[pa.Array]) -> None:\n self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py()\n\n def state(self) -> list[pa.Scalar]:\n return [pyarrow.scalar(self._sum)]\n\n def evaluate(self) -> pa.Scalar:\n return pyarrow.scalar(self._sum)\n\nctx = datafusion.SessionContext()\ndf = ctx.from_pydict(\n {\n \"a\": [4, 5, 6],\n \"b\": [1, 2, 3],\n }\n)\n\nmy_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable')\n\ndf.aggregate([], [my_udaf(col(\"a\"), col(\"b\")).alias(\"col_diff\")])\n```\n\n### FAQ\n\n**How do I return a list from a UDAF?**\n\nBoth the `evaluate` and the `state` functions expect to return scalar values.\nIf you wish to return a list array as a scalar value, the best practice is to\nwrap the values in a `pyarrow.Scalar` object. For example, you can return a\ntimestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp(\"ms\")))` and\nregister the appropriate return or state types as\n`return_type=pa.list_(pa.timestamp(\"ms\"))` and\n`state_type=[pa.list_(pa.timestamp(\"ms\"))]`, respectively.\n\nAs of DataFusion 52.0.0 , you can pass return any Python object, including a\nPyArrow array, as the return value(s) for these functions and DataFusion will\nattempt to create a scalar type from the value. DataFusion has been tested to\nconvert PyArrow, nanoarrow, and arro3 objects as well as primitive data types\nlike integers, strings, and so on.\n\n## Window Functions\n\nTo implement a User-Defined Window Function (UDWF) you must call the\n[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract\nclass [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator].\n\nThere are three methods of evaluation of UDWFs.\n\n- `evaluate` is the simplest case, where you are given an array and are expected to calculate the\n value for a single row of that array. This is the simplest case, but also the least performant.\n- `evaluate_all` computes the values for all rows for an input array at a single time.\n- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank\n information for the rows.\n\nWhich methods you implement are based upon which of these options are set.\n\n| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement |\n|---|---|---|---|\n| False (default) | False (default) | False (default) | `evaluate_all` |\n| False | True | False | `evaluate` |\n| False | True | False | `evaluate_all_with_rank` |\n| True | True/False | True/False | `evaluate` |\n\n### UDWF options\n\nWhen you define your UDWF you can override the functions that return these values. They will\ndetermine which evaluate functions are called.\n\n- `uses_window_frame` is set for functions that compute based on the specified window frame. If\n your function depends upon the specified frame, set this to `True`.\n- `supports_bounded_execution` specifies if your function can be incrementally computed.\n- `include_rank` is set to `True` for window functions that can be computed only using the rank\n information.\n\n```python\nimport pyarrow as pa\nfrom datafusion import udwf, col, SessionContext\nfrom datafusion.user_defined import WindowEvaluator\n\nclass ExponentialSmooth(WindowEvaluator):\n def __init__(self, alpha: float) -> None:\n self.alpha = alpha\n\n def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array:\n results = []\n curr_value = 0.0\n values = values[0]\n for idx in range(num_rows):\n if idx == 0:\n curr_value = values[idx].as_py()\n else:\n curr_value = values[idx].as_py() * self.alpha + curr_value * (\n 1.0 - self.alpha\n )\n results.append(curr_value)\n\n return pa.array(results)\n\nexp_smooth = udwf(\n ExponentialSmooth(0.9),\n pa.float64(),\n pa.float64(),\n volatility=\"immutable\",\n)\n\nctx = SessionContext()\n\ndf = ctx.from_pydict({\n \"a\": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0]\n})\n\ndf.select(\"a\", exp_smooth(col(\"a\")).alias(\"smooth_a\")).show()\n```\n\n## Table Functions\n\nUser Defined Table Functions are slightly different than the other functions\ndescribed here. These functions take any number of `Expr` arguments, but only\nliteral expressions are supported. Table functions must return a Table\nProvider as described in the ref:`_io_custom_table_provider` page.\n\nOnce you have a table function, you can register it with the session context\nby using [`register_udtf`][datafusion.context.SessionContext.register_udtf].\n\nThere are examples of both rust backed and python based table functions in the\nexamples folder of the repository. If you have a rust backed table function\nthat you wish to expose via PyO3, you need to expose it as a `PyCapsule`.\n\n```rust\n#[pymethods]\nimpl MyTableFunction {\n fn __datafusion_table_function__<'py>(\n &self,\n py: Python<'py>,\n ) -> PyResult> {\n let name = cr\"datafusion_table_function\".into();\n\n let func = self.clone();\n let provider = FFI_TableFunction::new(Arc::new(func), None);\n\n PyCapsule::new(py, provider, Some(name))\n }\n}\n```\n\n### Accessing the Calling Session\n\nPure-Python UDTFs can opt into receiving the calling\n[`SessionContext`][datafusion.context.SessionContext] by registering with\n`with_session=True`. The context is passed as a `session` keyword\nargument on every invocation. Use it to look up registered tables,\nUDFs, or session configuration from inside the callback.\n\n```python\nfrom datafusion import SessionContext, Table, udtf\nfrom datafusion.context import TableProviderExportable\nimport pyarrow as pa\nimport pyarrow.dataset as ds\n\n@udtf(\"list_tables\", with_session=True)\ndef list_tables(*, session: SessionContext) -> TableProviderExportable:\n names = sorted(session.catalog().schema().names())\n batch = pa.RecordBatch.from_pydict({\"name\": names})\n return Table(ds.dataset([batch]))\n\nctx = SessionContext()\nctx.register_batch(\"t1\", pa.RecordBatch.from_pydict({\"x\": [1]}))\nctx.register_udtf(list_tables)\nctx.sql(\"SELECT * FROM list_tables()\").show()\n```\n\nWithout `with_session=True`, the callback receives only the positional\nexpression arguments. The flag is opt-in so existing UDTFs keep working\nunchanged.\n\nThe injected `session` is a fresh [`SessionContext`][datafusion.context.SessionContext]\nwrapper backed by the same underlying state as the caller, so registries\n(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering\na new table or UDF) propagate to the live session because the registries\nare reference-counted and shared. Configuration changes made through the\nwrapper (e.g. setting session options) do **not** propagate — the wrapper\nholds its own clone of the session config.\n" - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.md b/docs/source/user-guide/common-operations/udf-and-udfa.md new file mode 100644 index 000000000..91ab531d9 --- /dev/null +++ b/docs/source/user-guide/common-operations/udf-and-udfa.md @@ -0,0 +1,485 @@ +```python exec="1" session="udf-and-udfa" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + +# User-Defined Functions + +DataFusion provides powerful expressions and functions, reducing the need for custom Python +functions. However you can still incorporate your own functions, i.e. User-Defined Functions (UDFs). + +## Scalar Functions + +When writing a user-defined function that can operate on a row by row basis, these are called Scalar +Functions. You can define your own scalar function by calling +[`udf`][datafusion.user_defined.ScalarUDF.udf] . + +The basic definition of a scalar UDF is a python function that takes one or more +[pyarrow](https://arrow.apache.org/docs/python/index.html) arrays and returns a single array as +output. DataFusion scalar UDFs operate on an entire batch of records at a time, though the +evaluation of those records should be on a row by row basis. In the following example, we compute +if the input array contains null values. + +```python exec="1" source="material-block" result="text" session="udf-and-udfa" +import pyarrow +from datafusion import udf + + +def is_null(array: pyarrow.Array) -> pyarrow.Array: + return array.is_null() + + +is_null_arr = udf(is_null, [pyarrow.int64()], pyarrow.bool_(), "stable") + +ctx = datafusion.SessionContext() + +batch = pyarrow.RecordBatch.from_arrays( + [pyarrow.array([1, None, 3]), pyarrow.array([4, 5, 6])], + names=["a", "b"], +) +df = ctx.create_dataframe([[batch]], name="batch_array") + +df.select(col("a"), is_null_arr(col("a")).alias("is_null")).show() +``` + + +In the previous example, we used the fact that pyarrow provides a variety of built in array +functions such as `is_null()`. There are additional pyarrow +[compute functions](https://arrow.apache.org/docs/python/compute.html) available. When possible, +it is highly recommended to use these functions because they can perform computations without doing +any copy operations from the original arrays. This leads to greatly improved performance. + +If you need to perform an operation in python that is not available with the pyarrow compute +functions, you will need to convert the record batch into python values, perform your operation, +and construct an array. This operation of converting the built in data type of the array into a +python object can be one of the slowest operations in DataFusion, so it should be done sparingly. + +The following example performs the same operation as before with `is_null` but demonstrates +converting to Python objects to do the evaluation. + +```python exec="1" source="material-block" result="text" session="udf-and-udfa" +import datafusion +import pyarrow +from datafusion import col, udf + + +def is_null(array: pyarrow.Array) -> pyarrow.Array: + return pyarrow.array([value.as_py() is None for value in array]) + + +is_null_arr = udf(is_null, [pyarrow.int64()], pyarrow.bool_(), "stable") + +ctx = datafusion.SessionContext() + +batch = pyarrow.RecordBatch.from_arrays( + [pyarrow.array([1, None, 3]), pyarrow.array([4, 5, 6])], + names=["a", "b"], +) +df = ctx.create_dataframe([[batch]], name="batch_array") + +df.select(col("a"), is_null_arr(col("a")).alias("is_null")).show() +``` + + +In this example we passed the PyArrow `DataType` when we defined the function +by calling `udf()`. If you need additional control, such as specifying +metadata or nullability of the input or output, you can instead specify a +PyArrow `Field`. + +If you need to write a custom function but do not want to incur the performance +cost of converting to Python objects and back, a more advanced approach is to +write Rust based UDFs and to expose them to Python. There is an example in the +[DataFusion blog](https://datafusion.apache.org/blog/2024/11/19/datafusion-python-udf-comparisons/) +describing how to do this. + +### When not to use a UDF + +A UDF is the right tool when the per-row computation genuinely cannot be +expressed with DataFusion's built-in expressions. It is often the *wrong* +tool for a predicate that *can* be written as an `Expr` tree but feels +easier to write as a Python function — for example, a filter that keeps +a row if it matches any one of several rule sets, where each rule set +checks its own combination of columns (the worked example at the end of +this section keeps a row when it matches any one of several brand-specific +rules). Looping over the rules in Python and returning a boolean per row +reads naturally and is tempting to wrap in a UDF, but a UDF is opaque to +the optimizer: filters expressed as UDFs lose several rewrites that the +engine applies to filters built from native expressions. The most visible +of these is **predicate pushdown into the table provider**: a native +predicate can be handed to the source so it skips data before it is read, +while a UDF predicate cannot. The example below uses Parquet, where +pushdown prunes whole row groups using the min/max statistics in the +footer, but the same mechanism applies to any table provider that +advertises filter support — including custom providers. + +The following example writes a small Parquet file, then filters it two +ways: first with a native expression, then with a UDF that computes the +same result. The filter itself is simple on purpose so we can compare +the plans side by side. + +```python exec="1" source="material-block" result="text" session="udf-and-udfa" +import os +import tempfile + +import pyarrow as pa +import pyarrow.parquet as pq +from datafusion import col, udf + +tmpdir = tempfile.mkdtemp() +parquet_path = os.path.join(tmpdir, "items.parquet") +pq.write_table( + pa.table( + { + "id": list(range(100)), + "brand": ["A", "B", "C", "D"] * 25, + "qty": [i * 10 for i in range(100)], + } + ), + parquet_path, +) + +ctx = SessionContext() +items = ctx.read_parquet(parquet_path) +``` + + +**Native-expression predicate.** The filter is a plain boolean tree +over column references and literals, so the optimizer can analyze it: + +```python exec="1" source="material-block" result="text" session="udf-and-udfa" +native_filtered = items.filter((col("brand") == lit("A")) & (col("qty") >= lit(150))) +print(native_filtered.execution_plan().display_indent()) +``` + + +Notice the `DataSourceExec` line. It carries three annotations the +optimizer computed from the predicate: + +- `predicate=brand@1 = A AND qty@2 >= 150` — the filter is pushed + into the Parquet scan itself, so the scan only reads matching rows. +- `pruning_predicate=... brand_min@0 <= A AND A <= brand_max@1 ... + qty_max@4 >= 150` — the scan prunes whole row groups by consulting + the Parquet min/max statistics in the footer *before* reading any + column data. +- `required_guarantees=[brand in (A)]` — the scan uses this when a + bloom filter or dictionary is available to skip pages. + +**UDF predicate.** Now wrap the same logic in a Python UDF: + +```python exec="1" source="material-block" result="text" session="udf-and-udfa" +def brand_qty_filter(brand_arr: pa.Array, qty_arr: pa.Array) -> pa.Array: + return pa.array( + [b.as_py() == "A" and q.as_py() >= 150 for b, q in zip(brand_arr, qty_arr)] + ) + + +pred_udf = udf( + brand_qty_filter, + [pa.string(), pa.int64()], + pa.bool_(), + "stable", +) +udf_filtered = items.filter(pred_udf(col("brand"), col("qty"))) +print(udf_filtered.execution_plan().display_indent()) +``` + + +The `DataSourceExec` now carries only `predicate=brand_qty_filter(...)`. +There is no `pruning_predicate` and no `required_guarantees`: the +scan has to materialize every row group and hand each row to the +Python callback just to decide whether to keep it. + +At small scale the cost difference is invisible; on a Parquet file with +many row groups, or data whose min/max statistics line up well with +the predicate, the native form can skip most of the file. The UDF form +reads all of it. + +**Takeaway.** Reach for a UDF when the per-row computation is genuinely +not expressible as a tree of built-in functions (custom numerical work, +external lookups, complex business rules). When it *is* expressible — +even if the native form is a little more verbose — build the `Expr` +tree directly so the optimizer can see through it. For disjunctive +predicates the idiom is to produce one clause per bucket and combine +them with `|`: + +```python +from functools import reduce +from operator import or_ +from datafusion import col, lit, functions as f + +buckets = { + "Brand#12": {"containers": ["SM CASE", "SM BOX"], "min_qty": 1, "max_size": 5}, + "Brand#23": {"containers": ["MED BAG", "MED BOX"], "min_qty": 10, "max_size": 10}, +} + +def bucket_clause(brand, spec): + return ( + (col("brand") == lit(brand)) + & f.in_list(col("container"), [lit(c) for c in spec["containers"]]) + & (col("quantity") >= lit(spec["min_qty"])) + & (col("quantity") <= lit(spec["min_qty"] + 10)) + & (col("size") >= lit(1)) + & (col("size") <= lit(spec["max_size"])) + ) + +predicate = reduce(or_, (bucket_clause(b, s) for b, s in buckets.items())) +df = df.filter(predicate) +``` + +## Aggregate Functions + +The [`udaf`][datafusion.user_defined.AggregateUDF.udaf] function allows you to define User-Defined +Aggregate Functions (UDAFs). To use this you must implement an +[`Accumulator`][datafusion.user_defined.Accumulator] that determines how the aggregation is performed. + +When defining a UDAF there are four methods you need to implement. The `update` function takes the +array(s) of input and updates the internal state of the accumulator. You should define this function +to have as many input arguments as you will pass when calling the UDAF. Since aggregation may be +split into multiple batches, we must have a method to combine multiple batches. For this, we have +two functions, `state` and `merge`. `state` will return an array of scalar values that contain +the current state of a single batch accumulation. Then we must `merge` the results of these +different states. Finally `evaluate` is the call that will return the final result after the +`merge` is complete. + +In the following example we want to define a custom aggregate function that will return the +difference between the sum of two columns. The state can be represented by a single value and we can +also see how the inputs to `update` and `merge` differ. + +```python +import pyarrow as pa +import pyarrow.compute +import datafusion +from datafusion import col, udaf, Accumulator +from typing import List + +class MyAccumulator(Accumulator): + """ + Interface of a user-defined accumulation. + """ + def __init__(self): + self._sum = 0.0 + + def update(self, values_a: pa.Array, values_b: pa.Array) -> None: + self._sum = self._sum + pyarrow.compute.sum(values_a).as_py() - pyarrow.compute.sum(values_b).as_py() + + def merge(self, states: list[pa.Array]) -> None: + self._sum = self._sum + pyarrow.compute.sum(states[0]).as_py() + + def state(self) -> list[pa.Scalar]: + return [pyarrow.scalar(self._sum)] + + def evaluate(self) -> pa.Scalar: + return pyarrow.scalar(self._sum) + +ctx = datafusion.SessionContext() +df = ctx.from_pydict( + { + "a": [4, 5, 6], + "b": [1, 2, 3], + } +) + +my_udaf = udaf(MyAccumulator, [pa.float64(), pa.float64()], pa.float64(), [pa.float64()], 'stable') + +df.aggregate([], [my_udaf(col("a"), col("b")).alias("col_diff")]) +``` + +### FAQ + +**How do I return a list from a UDAF?** + +Both the `evaluate` and the `state` functions expect to return scalar values. +If you wish to return a list array as a scalar value, the best practice is to +wrap the values in a `pyarrow.Scalar` object. For example, you can return a +timestamp list with `pa.scalar([...], type=pa.list_(pa.timestamp("ms")))` and +register the appropriate return or state types as +`return_type=pa.list_(pa.timestamp("ms"))` and +`state_type=[pa.list_(pa.timestamp("ms"))]`, respectively. + +As of DataFusion 52.0.0 , you can pass return any Python object, including a +PyArrow array, as the return value(s) for these functions and DataFusion will +attempt to create a scalar type from the value. DataFusion has been tested to +convert PyArrow, nanoarrow, and arro3 objects as well as primitive data types +like integers, strings, and so on. + +## Window Functions + +To implement a User-Defined Window Function (UDWF) you must call the +[`udwf`][datafusion.user_defined.WindowUDF.udwf] function using a class that implements the abstract +class [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator]. + +There are three methods of evaluation of UDWFs. + +- `evaluate` is the simplest case, where you are given an array and are expected to calculate the + value for a single row of that array. This is the simplest case, but also the least performant. +- `evaluate_all` computes the values for all rows for an input array at a single time. +- `evaluate_all_with_rank` computes the values for all rows, but you only have the rank + information for the rows. + +Which methods you implement are based upon which of these options are set. + +| `uses_window_frame` | `supports_bounded_execution` | `include_rank` | function_to_implement | +|---|---|---|---| +| False (default) | False (default) | False (default) | `evaluate_all` | +| False | True | False | `evaluate` | +| False | True | False | `evaluate_all_with_rank` | +| True | True/False | True/False | `evaluate` | + +### UDWF options + +When you define your UDWF you can override the functions that return these values. They will +determine which evaluate functions are called. + +- `uses_window_frame` is set for functions that compute based on the specified window frame. If + your function depends upon the specified frame, set this to `True`. +- `supports_bounded_execution` specifies if your function can be incrementally computed. +- `include_rank` is set to `True` for window functions that can be computed only using the rank + information. + +```python +import pyarrow as pa +from datafusion import udwf, col, SessionContext +from datafusion.user_defined import WindowEvaluator + +class ExponentialSmooth(WindowEvaluator): + def __init__(self, alpha: float) -> None: + self.alpha = alpha + + def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array: + results = [] + curr_value = 0.0 + values = values[0] + for idx in range(num_rows): + if idx == 0: + curr_value = values[idx].as_py() + else: + curr_value = values[idx].as_py() * self.alpha + curr_value * ( + 1.0 - self.alpha + ) + results.append(curr_value) + + return pa.array(results) + +exp_smooth = udwf( + ExponentialSmooth(0.9), + pa.float64(), + pa.float64(), + volatility="immutable", +) + +ctx = SessionContext() + +df = ctx.from_pydict({ + "a": [1.0, 2.1, 2.9, 4.0, 5.1, 6.0, 6.9, 8.0] +}) + +df.select("a", exp_smooth(col("a")).alias("smooth_a")).show() +``` + +## Table Functions + +User Defined Table Functions are slightly different than the other functions +described here. These functions take any number of `Expr` arguments, but only +literal expressions are supported. Table functions must return a Table +Provider as described in the ref:`_io_custom_table_provider` page. + +Once you have a table function, you can register it with the session context +by using [`register_udtf`][datafusion.context.SessionContext.register_udtf]. + +There are examples of both rust backed and python based table functions in the +examples folder of the repository. If you have a rust backed table function +that you wish to expose via PyO3, you need to expose it as a `PyCapsule`. + +```rust +#[pymethods] +impl MyTableFunction { + fn __datafusion_table_function__<'py>( + &self, + py: Python<'py>, + ) -> PyResult> { + let name = cr"datafusion_table_function".into(); + + let func = self.clone(); + let provider = FFI_TableFunction::new(Arc::new(func), None); + + PyCapsule::new(py, provider, Some(name)) + } +} +``` + +### Accessing the Calling Session + +Pure-Python UDTFs can opt into receiving the calling +[`SessionContext`][datafusion.context.SessionContext] by registering with +`with_session=True`. The context is passed as a `session` keyword +argument on every invocation. Use it to look up registered tables, +UDFs, or session configuration from inside the callback. + +```python +from datafusion import SessionContext, Table, udtf +from datafusion.context import TableProviderExportable +import pyarrow as pa +import pyarrow.dataset as ds + +@udtf("list_tables", with_session=True) +def list_tables(*, session: SessionContext) -> TableProviderExportable: + names = sorted(session.catalog().schema().names()) + batch = pa.RecordBatch.from_pydict({"name": names}) + return Table(ds.dataset([batch])) + +ctx = SessionContext() +ctx.register_batch("t1", pa.RecordBatch.from_pydict({"x": [1]})) +ctx.register_udtf(list_tables) +ctx.sql("SELECT * FROM list_tables()").show() +``` + +Without `with_session=True`, the callback receives only the positional +expression arguments. The flag is opt-in so existing UDTFs keep working +unchanged. + +The injected `session` is a fresh [`SessionContext`][datafusion.context.SessionContext] +wrapper backed by the same underlying state as the caller, so registries +(tables, UDFs, catalogs) are visible. Registry mutations (e.g. registering +a new table or UDF) propagate to the live session because the registries +are reference-counted and shared. Configuration changes made through the +wrapper (e.g. setting session options) do **not** propagate — the wrapper +holds its own clone of the session config. diff --git a/docs/source/user-guide/common-operations/windows.ipynb b/docs/source/user-guide/common-operations/windows.ipynb deleted file mode 100644 index 6b227a210..000000000 --- a/docs/source/user-guide/common-operations/windows.ipynb +++ /dev/null @@ -1,230 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n\n# Window Functions\n\nIn this section you will learn about window functions. A window function utilizes values from one or\nmultiple rows to produce a result for each individual row, unlike an aggregate function that\nprovides a single value for multiple rows.\n\nThe window functions are available in the [`functions`][datafusion.functions] module.\n\nWe'll use the pokemon dataset (from Ritchie Vink) in the following examples.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "df = ctx.read_csv(\"pokemon.csv\")" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\nHere is an example that shows how you can compare each pokemon's speed to the speed of the\nprevious row in the DataFrame.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(col('\"Name\"'), col('\"Speed\"'), f.lag(col('\"Speed\"')).alias(\"Previous Speed\"))" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\n## Setting Parameters\n\n### Ordering\n\nYou can control the order in which rows are processed by window functions by providing\na list of `order_by` functions for the `order_by` parameter.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(\n", - " col('\"Name\"'),\n", - " col('\"Attack\"'),\n", - " col('\"Type 1\"'),\n", - " f.rank(\n", - " partition_by=[col('\"Type 1\"')],\n", - " order_by=[col('\"Attack\"').sort(ascending=True)],\n", - " ).alias(\"rank\"),\n", - ").sort(col('\"Type 1\"'), col('\"Attack\"'))" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\n### Partitions\n\nA window function can take a list of `partition_by` columns similar to an\n[Aggregation Function](../aggregations/). This will cause the window values to be evaluated\nindependently for each of the partitions. In the example above, we found the rank of each\nPokemon per `Type 1` partitions. We can see the first couple of each partition if we do\nthe following:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(\n", - " col('\"Name\"'),\n", - " col('\"Attack\"'),\n", - " col('\"Type 1\"'),\n", - " f.rank(\n", - " partition_by=[col('\"Type 1\"')],\n", - " order_by=[col('\"Attack\"').sort(ascending=True)],\n", - " ).alias(\"rank\"),\n", - ").filter(col(\"rank\") < lit(3)).sort(col('\"Type 1\"'), col(\"rank\"))" - ] - }, - { - "cell_type": "markdown", - "id": "7cdc8c89c7104fffa095e18ddfef8986", - "metadata": {}, - "source": "\n### Window Frame\n\nWhen using aggregate functions, the Window Frame of defines the rows over which it operates.\nIf you do not specify a Window Frame, the frame will be set depending on the following\ncriteria.\n\n- If an `order_by` clause is set, the default window frame is defined as the rows between\n unbounded preceding and the current row.\n- If an `order_by` is not set, the default frame is defined as the rows between unbounded\n and unbounded following (the entire partition).\n\nWindow Frames are defined by three parameters: unit type, starting bound, and ending bound.\n\nThe unit types available are:\n\n- Rows: The starting and ending boundaries are defined by the number of rows relative to the\n current row.\n- Range: When using Range, the `order_by` clause must have exactly one term. The boundaries\n are defined bow how close the rows are to the value of the expression in the `order_by`\n parameter.\n- Groups: A \"group\" is the set of all rows that have equivalent values for all terms in the\n `order_by` clause.\n\nIn this example we perform a \"rolling average\" of the speed of the current Pokemon and the\ntwo preceding rows.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b118ea5561624da68c537baed56e602f", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion.expr import Window, WindowFrame\n", - "\n", - "df.select(\n", - " col('\"Name\"'),\n", - " col('\"Speed\"'),\n", - " f.avg(col('\"Speed\"'))\n", - " .over(Window(window_frame=WindowFrame(\"rows\", 2, 0), order_by=[col('\"Speed\"')]))\n", - " .alias(\"Previous Speed\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "938c804e27f84196a10c8828c723f798", - "metadata": {}, - "source": "\n### Null Treatment\n\nWhen using aggregate functions as window functions, it is often useful to specify how null values\nshould be treated. In order to do this you need to use the builder function. In future releases\nwe expect this to be simplified in the interface.\n\nOne common usage for handling nulls is the case where you want to find the last value up to the\ncurrent row. In the following example we demonstrate how setting the null treatment to ignore\nnulls will fill in with the value of the most recent non-null row. To do this, we also will set\nthe window frame so that we only process up to the current row.\n\nIn this example, we filter down to one specific type of Pokemon that does have some entries in\nit's `Type 2` column that are null.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "504fb2a444614c0babb325280ed9130a", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion.common import NullTreatment\n", - "\n", - "df.filter(col('\"Type 1\"') == lit(\"Bug\")).select(\n", - " '\"Name\"',\n", - " '\"Type 2\"',\n", - " f.last_value(col('\"Type 2\"'))\n", - " .over(\n", - " Window(\n", - " window_frame=WindowFrame(\"rows\", None, 0),\n", - " order_by=[col('\"Speed\"')],\n", - " null_treatment=NullTreatment.IGNORE_NULLS,\n", - " )\n", - " )\n", - " .alias(\"last_wo_null\"),\n", - " f.last_value(col('\"Type 2\"'))\n", - " .over(\n", - " Window(\n", - " window_frame=WindowFrame(\"rows\", None, 0),\n", - " order_by=[col('\"Speed\"')],\n", - " null_treatment=NullTreatment.RESPECT_NULLS,\n", - " )\n", - " )\n", - " .alias(\"last_with_null\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "59bbdb311c014d738909a11f9e486628", - "metadata": {}, - "source": "\n## Aggregate Functions\n\nYou can use any [Aggregation Function](../aggregations/) as a window function. Here\nis an example that shows how to compare each pokemons’s attack power with the average attack\npower in its `\"Type 1\"` using the [`avg`][datafusion.functions.avg] function.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b43b363d81ae4b689946ece5c682cd59", - "metadata": {}, - "outputs": [], - "source": [ - "df.select(\n", - " col('\"Name\"'),\n", - " col('\"Attack\"'),\n", - " col('\"Type 1\"'),\n", - " f.avg(col('\"Attack\"'))\n", - " .over(\n", - " Window(\n", - " window_frame=WindowFrame(\"rows\", None, None),\n", - " partition_by=[col('\"Type 1\"')],\n", - " )\n", - " )\n", - " .alias(\"Average Attack\"),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "8a65eabff63a45729fe45fb5ade58bdc", - "metadata": {}, - "source": "\n## Available Functions\n\nThe possible window functions are:\n\n1. Rank Functions\n : - [`rank`][datafusion.functions.rank]\n - [`dense_rank`][datafusion.functions.dense_rank]\n - [`ntile`][datafusion.functions.ntile]\n - [`row_number`][datafusion.functions.row_number]\n2. Analytical Functions\n : - [`cume_dist`][datafusion.functions.cume_dist]\n - [`percent_rank`][datafusion.functions.percent_rank]\n - [`lag`][datafusion.functions.lag]\n - [`lead`][datafusion.functions.lead]\n3. Aggregate Functions\n : - All [Aggregation Functions](../aggregations/) can be used as window functions.\n\n## User-Defined Window Functions\n\nYou can ship custom window functions to the engine by subclassing\n[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it\nvia [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`][datafusion.user_defined]\nfor the evaluator interface and worked examples.\n\n
\n

Note

\n\nSerialization\n\n
\n\n Python window UDFs travel inline inside pickled or\n [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions —\n the evaluator class is captured by value via [`cloudpickle`][cloudpickle], so\n worker processes do not need to pre-register the UDF. Any names the\n evaluator resolves via `import` are captured **by reference** and\n must be importable on the receiving worker. See\n [`ipc`][datafusion.ipc] for the full IPC model and security caveats.\n" - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/common-operations/windows.md b/docs/source/user-guide/common-operations/windows.md new file mode 100644 index 000000000..1ad5e79bc --- /dev/null +++ b/docs/source/user-guide/common-operations/windows.md @@ -0,0 +1,252 @@ +```python exec="1" session="windows" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + + +# Window Functions + +In this section you will learn about window functions. A window function utilizes values from one or +multiple rows to produce a result for each individual row, unlike an aggregate function that +provides a single value for multiple rows. + +The window functions are available in the [`functions`][datafusion.functions] module. + +We'll use the pokemon dataset (from Ritchie Vink) in the following examples. + +```python exec="1" source="material-block" result="text" session="windows" +ctx = SessionContext() +df = ctx.read_csv("pokemon.csv") +``` + + +Here is an example that shows how you can compare each pokemon's speed to the speed of the +previous row in the DataFrame. + +```python exec="1" source="material-block" result="text" session="windows" +df.select(col('"Name"'), col('"Speed"'), f.lag(col('"Speed"')).alias("Previous Speed")) +``` + + +## Setting Parameters + +### Ordering + +You can control the order in which rows are processed by window functions by providing +a list of `order_by` functions for the `order_by` parameter. + +```python exec="1" source="material-block" result="text" session="windows" +df.select( + col('"Name"'), + col('"Attack"'), + col('"Type 1"'), + f.rank( + partition_by=[col('"Type 1"')], + order_by=[col('"Attack"').sort(ascending=True)], + ).alias("rank"), +).sort(col('"Type 1"'), col('"Attack"')) +``` + + +### Partitions + +A window function can take a list of `partition_by` columns similar to an +[Aggregation Function](../aggregations/). This will cause the window values to be evaluated +independently for each of the partitions. In the example above, we found the rank of each +Pokemon per `Type 1` partitions. We can see the first couple of each partition if we do +the following: + +```python exec="1" source="material-block" result="text" session="windows" +df.select( + col('"Name"'), + col('"Attack"'), + col('"Type 1"'), + f.rank( + partition_by=[col('"Type 1"')], + order_by=[col('"Attack"').sort(ascending=True)], + ).alias("rank"), +).filter(col("rank") < lit(3)).sort(col('"Type 1"'), col("rank")) +``` + + +### Window Frame + +When using aggregate functions, the Window Frame of defines the rows over which it operates. +If you do not specify a Window Frame, the frame will be set depending on the following +criteria. + +- If an `order_by` clause is set, the default window frame is defined as the rows between + unbounded preceding and the current row. +- If an `order_by` is not set, the default frame is defined as the rows between unbounded + and unbounded following (the entire partition). + +Window Frames are defined by three parameters: unit type, starting bound, and ending bound. + +The unit types available are: + +- Rows: The starting and ending boundaries are defined by the number of rows relative to the + current row. +- Range: When using Range, the `order_by` clause must have exactly one term. The boundaries + are defined bow how close the rows are to the value of the expression in the `order_by` + parameter. +- Groups: A "group" is the set of all rows that have equivalent values for all terms in the + `order_by` clause. + +In this example we perform a "rolling average" of the speed of the current Pokemon and the +two preceding rows. + +```python exec="1" source="material-block" result="text" session="windows" +from datafusion.expr import Window, WindowFrame + +df.select( + col('"Name"'), + col('"Speed"'), + f.avg(col('"Speed"')) + .over(Window(window_frame=WindowFrame("rows", 2, 0), order_by=[col('"Speed"')])) + .alias("Previous Speed"), +) +``` + + +### Null Treatment + +When using aggregate functions as window functions, it is often useful to specify how null values +should be treated. In order to do this you need to use the builder function. In future releases +we expect this to be simplified in the interface. + +One common usage for handling nulls is the case where you want to find the last value up to the +current row. In the following example we demonstrate how setting the null treatment to ignore +nulls will fill in with the value of the most recent non-null row. To do this, we also will set +the window frame so that we only process up to the current row. + +In this example, we filter down to one specific type of Pokemon that does have some entries in +it's `Type 2` column that are null. + +```python exec="1" source="material-block" result="text" session="windows" +from datafusion.common import NullTreatment + +df.filter(col('"Type 1"') == lit("Bug")).select( + '"Name"', + '"Type 2"', + f.last_value(col('"Type 2"')) + .over( + Window( + window_frame=WindowFrame("rows", None, 0), + order_by=[col('"Speed"')], + null_treatment=NullTreatment.IGNORE_NULLS, + ) + ) + .alias("last_wo_null"), + f.last_value(col('"Type 2"')) + .over( + Window( + window_frame=WindowFrame("rows", None, 0), + order_by=[col('"Speed"')], + null_treatment=NullTreatment.RESPECT_NULLS, + ) + ) + .alias("last_with_null"), +) +``` + + +## Aggregate Functions + +You can use any [Aggregation Function](../aggregations/) as a window function. Here +is an example that shows how to compare each pokemons’s attack power with the average attack +power in its `"Type 1"` using the [`avg`][datafusion.functions.avg] function. + +```python exec="1" source="material-block" result="text" session="windows" +df.select( + col('"Name"'), + col('"Attack"'), + col('"Type 1"'), + f.avg(col('"Attack"')) + .over( + Window( + window_frame=WindowFrame("rows", None, None), + partition_by=[col('"Type 1"')], + ) + ) + .alias("Average Attack"), +) +``` + + +## Available Functions + +The possible window functions are: + +1. Rank Functions + : - [`rank`][datafusion.functions.rank] + - [`dense_rank`][datafusion.functions.dense_rank] + - [`ntile`][datafusion.functions.ntile] + - [`row_number`][datafusion.functions.row_number] +2. Analytical Functions + : - [`cume_dist`][datafusion.functions.cume_dist] + - [`percent_rank`][datafusion.functions.percent_rank] + - [`lag`][datafusion.functions.lag] + - [`lead`][datafusion.functions.lead] +3. Aggregate Functions + : - All [Aggregation Functions](../aggregations/) can be used as window functions. + +## User-Defined Window Functions + +You can ship custom window functions to the engine by subclassing +[`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it +via [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`][datafusion.user_defined] +for the evaluator interface and worked examples. + +
+

Note

+ +Serialization + +
+ + Python window UDFs travel inline inside pickled or + [`to_bytes`][datafusion.expr.Expr.to_bytes]-serialized expressions — + the evaluator class is captured by value via [`cloudpickle`][cloudpickle], so + worker processes do not need to pre-register the UDF. Any names the + evaluator resolves via `import` are captured **by reference** and + must be importable on the receiving worker. See + [`ipc`][datafusion.ipc] for the full IPC model and security caveats. diff --git a/docs/source/user-guide/concepts.ipynb b/docs/source/user-guide/concepts.ipynb deleted file mode 100644 index 83bf49550..000000000 --- a/docs/source/user-guide/concepts.ipynb +++ /dev/null @@ -1,86 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n\n# Concepts\n\nIn this section, we will cover a basic example to introduce a few key concepts. We will use the\n2021 Yellow Taxi Trip Records ([download](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet)),\nfrom the [TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page).\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "\n", - "df = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n", - "\n", - "df = df.select(\n", - " \"trip_distance\",\n", - " col(\"total_amount\").alias(\"total\"),\n", - " (f.round(lit(100.0) * col(\"tip_amount\") / col(\"total_amount\"), lit(1))).alias(\n", - " \"tip_percent\"\n", - " ),\n", - ")\n", - "\n", - "df.show()" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\n## Session Context\n\nThe first statement group creates a [`SessionContext`][datafusion.context.SessionContext].\n\n```python\n# create a context\nctx = datafusion.SessionContext()\n```\n\nA Session Context is the main interface for executing queries with DataFusion. It maintains the state\nof the connection between a user and an instance of the DataFusion engine. Additionally it provides\nthe following functionality:\n\n- Create a DataFrame from a data source.\n- Register a data source as a table that can be referenced from a SQL query.\n- Execute a SQL query\n\n## DataFrame\n\nThe second statement group creates a [`DataFrame`][datafusion.dataframe.DataFrame],\n\n```python\n# Create a DataFrame from a file\ndf = ctx.read_parquet(\"yellow_tripdata_2021-01.parquet\")\n```\n\nA DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html).\nDataFrames are typically created by calling a method on [`SessionContext`][datafusion.context.SessionContext], such as [`read_csv`][datafusion.context.SessionContext.read_csv], and can then be modified by\ncalling the transformation methods, such as [`filter`][datafusion.dataframe.DataFrame.filter], [`select`][datafusion.dataframe.DataFrame.select], [`aggregate`][datafusion.dataframe.DataFrame.aggregate],\nand [`limit`][datafusion.dataframe.DataFrame.limit] to build up a query definition.\n\nFor more details on working with DataFrames, including visualization options and conversion to other formats, see [dataframe/index](dataframe/index.md).\n\n## Expressions\n\nThe third statement uses [Expressions](../common-operations/expressions/) to build up a query definition. You can find\nexplanations for what the functions below do in the user documentation for\n[`col`][datafusion.col.col], [`lit`][datafusion.lit], [`round`][datafusion.functions.round],\nand [`alias`][datafusion.expr.Expr.alias].\n\n```python\ndf = df.select(\n \"trip_distance\",\n col(\"total_amount\").alias(\"total\"),\n (f.round(lit(100.0) * col(\"tip_amount\") / col(\"total_amount\"), lit(1))).alias(\"tip_percent\"),\n)\n```\n\nFinally the [`show`][datafusion.dataframe.DataFrame.show] method converts the logical plan\nrepresented by the DataFrame into a physical plan and execute it, collecting all results and\ndisplaying them to the user. It is important to note that DataFusion performs lazy evaluation\nof the DataFrame. Until you call a method such as [`show`][datafusion.dataframe.DataFrame.show]\nor [`collect`][datafusion.dataframe.DataFrame.collect], DataFusion will not perform the query.\n" - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/concepts.md b/docs/source/user-guide/concepts.md new file mode 100644 index 000000000..b76c41783 --- /dev/null +++ b/docs/source/user-guide/concepts.md @@ -0,0 +1,121 @@ +```python exec="1" session="concepts" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + + +# Concepts + +In this section, we will cover a basic example to introduce a few key concepts. We will use the +2021 Yellow Taxi Trip Records ([download](https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet)), +from the [TLC Trip Record Data](https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page). + +```python exec="1" source="material-block" result="text" session="concepts" +ctx = SessionContext() + +df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") + +df = df.select( + "trip_distance", + col("total_amount").alias("total"), + (f.round(lit(100.0) * col("tip_amount") / col("total_amount"), lit(1))).alias( + "tip_percent" + ), +) + +df.show() +``` + + +## Session Context + +The first statement group creates a [`SessionContext`][datafusion.context.SessionContext]. + +```python +# create a context +ctx = datafusion.SessionContext() +``` + +A Session Context is the main interface for executing queries with DataFusion. It maintains the state +of the connection between a user and an instance of the DataFusion engine. Additionally it provides +the following functionality: + +- Create a DataFrame from a data source. +- Register a data source as a table that can be referenced from a SQL query. +- Execute a SQL query + +## DataFrame + +The second statement group creates a [`DataFrame`][datafusion.dataframe.DataFrame], + +```python +# Create a DataFrame from a file +df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") +``` + +A DataFrame refers to a (logical) set of rows that share the same column names, similar to a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html). +DataFrames are typically created by calling a method on [`SessionContext`][datafusion.context.SessionContext], such as [`read_csv`][datafusion.context.SessionContext.read_csv], and can then be modified by +calling the transformation methods, such as [`filter`][datafusion.dataframe.DataFrame.filter], [`select`][datafusion.dataframe.DataFrame.select], [`aggregate`][datafusion.dataframe.DataFrame.aggregate], +and [`limit`][datafusion.dataframe.DataFrame.limit] to build up a query definition. + +For more details on working with DataFrames, including visualization options and conversion to other formats, see [dataframe/index](dataframe/index.md). + +## Expressions + +The third statement uses [Expressions](../common-operations/expressions/) to build up a query definition. You can find +explanations for what the functions below do in the user documentation for +[`col`][datafusion.col.col], [`lit`][datafusion.lit], [`round`][datafusion.functions.round], +and [`alias`][datafusion.expr.Expr.alias]. + +```python +df = df.select( + "trip_distance", + col("total_amount").alias("total"), + (f.round(lit(100.0) * col("tip_amount") / col("total_amount"), lit(1))).alias("tip_percent"), +) +``` + +Finally the [`show`][datafusion.dataframe.DataFrame.show] method converts the logical plan +represented by the DataFrame into a physical plan and execute it, collecting all results and +displaying them to the user. It is important to note that DataFusion performs lazy evaluation +of the DataFrame. Until you call a method such as [`show`][datafusion.dataframe.DataFrame.show] +or [`collect`][datafusion.dataframe.DataFrame.collect], DataFusion will not perform the query. diff --git a/docs/source/user-guide/data-sources.ipynb b/docs/source/user-guide/data-sources.ipynb deleted file mode 100644 index 30f7bf199..000000000 --- a/docs/source/user-guide/data-sources.ipynb +++ /dev/null @@ -1,119 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f # noqa: F401\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n\n# Data Sources\n\nDataFusion provides a wide variety of ways to get data into a DataFrame to perform operations.\n\n## Local file\n\nDataFusion has the ability to read from a variety of popular file formats, such as [Parquet](../io/parquet/),\n[CSV](../io/csv/), [JSON](../io/json/), and [AVRO](../io/avro/).\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "df = ctx.read_csv(\"pokemon.csv\")\n", - "df.show()" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\n## Create in-memory\n\nSometimes it can be convenient to create a small DataFrame from a Python list or dictionary object.\nTo do this in DataFusion, you can use one of the three functions\n[`from_pydict`][datafusion.context.SessionContext.from_pydict],\n[`from_pylist`][datafusion.context.SessionContext.from_pylist], or\n[`create_dataframe`][datafusion.context.SessionContext.create_dataframe].\n\nAs their names suggest, [`from_pydict`][datafusion.context.SessionContext.from_pydict] and [`from_pylist`][datafusion.context.SessionContext.from_pylist] will create DataFrames from Python\ndictionary and list objects, respectively. [`create_dataframe`][datafusion.context.SessionContext.create_dataframe] assumes you will pass in a list\nof list of [PyArrow Record Batches](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html).\n\nThe following three examples all will create identical DataFrames:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "import pyarrow as pa\n", - "\n", - "ctx.from_pylist(\n", - " [\n", - " {\"a\": 1, \"b\": 10.0, \"c\": \"alpha\"},\n", - " {\"a\": 2, \"b\": 20.0, \"c\": \"beta\"},\n", - " {\"a\": 3, \"b\": 30.0, \"c\": \"gamma\"},\n", - " ]\n", - ").show()\n", - "\n", - "ctx.from_pydict(\n", - " {\n", - " \"a\": [1, 2, 3],\n", - " \"b\": [10.0, 20.0, 30.0],\n", - " \"c\": [\"alpha\", \"beta\", \"gamma\"],\n", - " }\n", - ").show()\n", - "\n", - "batch = pa.RecordBatch.from_arrays(\n", - " [\n", - " pa.array([1, 2, 3]),\n", - " pa.array([10.0, 20.0, 30.0]),\n", - " pa.array([\"alpha\", \"beta\", \"gamma\"]),\n", - " ],\n", - " names=[\"a\", \"b\", \"c\"],\n", - ")\n", - "\n", - "ctx.create_dataframe([[batch]]).show()" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\n## Object Store\n\nDataFusion has support for multiple storage options in addition to local files.\nThe example below requires an appropriate S3 account with access credentials.\n\nSupported Object Stores are\n\n- [`AmazonS3`][datafusion.object_store.AmazonS3]\n- [`GoogleCloud`][datafusion.object_store.GoogleCloud]\n- [`Http`][datafusion.object_store.Http]\n- [`LocalFileSystem`][datafusion.object_store.LocalFileSystem]\n- [`MicrosoftAzure`][datafusion.object_store.MicrosoftAzure]\n\n```python\nfrom datafusion.object_store import AmazonS3\n\nregion = \"us-east-1\"\nbucket_name = \"yellow-trips\"\n\ns3 = AmazonS3(\n bucket_name=bucket_name,\n region=region,\n access_key_id=os.getenv(\"AWS_ACCESS_KEY_ID\"),\n secret_access_key=os.getenv(\"AWS_SECRET_ACCESS_KEY\"),\n)\n\npath = f\"s3://{bucket_name}/\"\nctx.register_object_store(\"s3://\", s3, None)\n\nctx.register_parquet(\"trips\", path)\n\nctx.table(\"trips\").show()\n```\n\n## Other DataFrame Libraries\n\nDataFusion can import DataFrames directly from other libraries, such as\n[Polars](https://pola.rs/) and [Pandas](https://pandas.pydata.org/).\nSince DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule\ninterface can be imported to DataFusion using the\n[`from_arrow`][datafusion.context.SessionContext.from_arrow] function. Older versions of Polars may\nnot support the arrow interface. In those cases, you can still import via the\n[`from_polars`][datafusion.context.SessionContext.from_polars] function.\n\n```python\nimport pandas as pd\n\ndata = { \"a\": [1, 2, 3], \"b\": [10.0, 20.0, 30.0], \"c\": [\"alpha\", \"beta\", \"gamma\"] }\npandas_df = pd.DataFrame(data)\n\ndatafusion_df = ctx.from_arrow(pandas_df)\ndatafusion_df.show()\n```\n\n```python\nimport polars as pl\npolars_df = pl.DataFrame(data)\n\ndatafusion_df = ctx.from_arrow(polars_df)\ndatafusion_df.show()\n```\n\n## Delta Lake\n\nDataFusion 43.0.0 and later support the ability to register table providers from sources such\nas Delta Lake. This will require a recent version of\n[deltalake](https://delta-io.github.io/delta-rs/) to provide the required interfaces.\n\n```python\nfrom deltalake import DeltaTable\n\ndelta_table = DeltaTable(\"path_to_table\")\nctx.register_table(\"my_delta_table\", delta_table)\ndf = ctx.table(\"my_delta_table\")\ndf.show()\n```\n\nOn older versions of [`deltalake`](https://delta-io.github.io/delta-rs/) (prior to 0.22) you can use the\n[Arrow DataSet](https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Dataset.html)\ninterface to import to DataFusion, but this does not support features such as filter push down\nwhich can lead to a significant performance difference.\n\n```python\nfrom deltalake import DeltaTable\n\ndelta_table = DeltaTable(\"path_to_table\")\nctx.register_dataset(\"my_delta_table\", delta_table.to_pyarrow_dataset())\ndf = ctx.table(\"my_delta_table\")\ndf.show()\n```\n\n## Apache Iceberg\n\nDataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface.\n\nThis requires either the [pyiceberg](https://pypi.org/project/pyiceberg/) library (>=0.10.0) or the [pyiceberg-core](https://pypi.org/project/pyiceberg-core/) library (>=0.5.0).\n\n- The `pyiceberg-core` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings.\n- The `pyiceberg` library utilizes the `pyiceberg-core` python bindings under the hood and provides a native way for Python users to interact with the DataFusion.\n\n```python\nfrom datafusion import SessionContext\nfrom pyiceberg.catalog import load_catalog\nimport pyarrow as pa\n\n# Load catalog and create/load a table\ncatalog = load_catalog(\"catalog\", type=\"in-memory\")\ncatalog.create_namespace_if_not_exists(\"default\")\n\n# Create some sample data\ndata = pa.table({\"x\": [1, 2, 3], \"y\": [4, 5, 6]})\niceberg_table = catalog.create_table(\"default.test\", schema=data.schema)\niceberg_table.append(data)\n\n# Register the table with DataFusion\nctx = SessionContext()\nctx.register_table_provider(\"test\", iceberg_table)\n\n# Query the table using DataFusion\nctx.table(\"test\").show()\n```\n\nNote that the Datafusion integration rely on features from the [Iceberg Rust](https://github.com/apache/iceberg-rust/) implementation instead of the [PyIceberg](https://github.com/apache/iceberg-python/) implementation.\nFeatures that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion.\n\n## Custom Table Provider\n\nYou can implement a custom Data Provider in Rust and expose it to DataFusion through the\nthe interface as describe in the [Custom Table Provider](../io/table_provider/)\nsection. This is an advanced topic, but a\n[user example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example)\nis provided in the DataFusion repository.\n\n# Catalog\n\nA common technique for organizing tables is using a three level hierarchical approach. DataFusion\nsupports this form of organizing using the [`Catalog`][datafusion.catalog.Catalog],\n[`Schema`][datafusion.catalog.Schema], and [`Table`][datafusion.catalog.Table]. By default,\na [`SessionContext`][datafusion.context.SessionContext] comes with a single Catalog and a single Schema\nwith the names `datafusion` and `public`, respectively.\n\nThe default implementation uses an in-memory approach to the catalog and schema. We have support\nfor adding additional in-memory catalogs and schemas. You can access tables registered in a schema\neither through the Dataframe API or via sql commands. This can be done like in the following\nexample:\n\n```python\nimport pyarrow as pa\nfrom datafusion.catalog import Catalog, Schema\nfrom datafusion import SessionContext\n\nctx = SessionContext()\n\nmy_catalog = Catalog.memory_catalog()\nmy_schema = Schema.memory_schema()\nmy_catalog.register_schema('my_schema_name', my_schema)\nctx.register_catalog_provider('my_catalog_name', my_catalog)\n\n# Create an in-memory table\ntable = pa.table({\n 'name': ['Bulbasaur', 'Charmander', 'Squirtle'],\n 'type': ['Grass', 'Fire', 'Water'],\n 'hp': [45, 39, 44],\n})\ndf = ctx.create_dataframe([table.to_batches()], name='pokemon')\n\nmy_schema.register_table('pokemon', df)\n\nctx.sql('SELECT * FROM my_catalog_name.my_schema_name.pokemon').show()\n```\n\n## User Defined Catalog and Schema\n\nIf the in-memory catalogs are insufficient for your uses, there are two approaches you can take\nto implementing a custom catalog and/or schema. In the below discussion, we describe how to\nimplement these for a Catalog, but the approach to implementing for a Schema is nearly\nidentical.\n\nDataFusion supports Catalogs written in either Rust or Python. If you write a Catalog in Rust,\nyou will need to export it as a Python library via PyO3. There is a complete example of a\ncatalog implemented this way in the\n[examples folder](https://github.com/apache/datafusion-python/tree/main/examples/)\nof our repository. Writing catalog providers in Rust provides typically can lead to significant\nperformance improvements over the Python based approach.\n\nTo implement a Catalog in Python, you will need to inherit from the abstract base class\n[`CatalogProvider`][datafusion.catalog.CatalogProvider]. There are examples in the\n[unit tests](https://github.com/apache/datafusion-python/tree/main/python/tests) of\nimplementing a basic Catalog in Python where we simply keep a dictionary of the\nregistered Schemas.\n\nOne important note for developers is that when we have a Catalog defined in Python, we have\ntwo different ways of accessing this Catalog. First, we register the catalog with a Rust\nwrapper. This allows for any rust based code to call the Python functions as necessary.\nSecond, if the user access the Catalog via the Python API, we identify this and return back\nthe original Python object that implements the Catalog. This is an important distinction\nfor developers because we do *not* return a Python wrapper around the Rust wrapper of the\noriginal Python object.\n" - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/data-sources.md b/docs/source/user-guide/data-sources.md new file mode 100644 index 000000000..0a1e53f50 --- /dev/null +++ b/docs/source/user-guide/data-sources.md @@ -0,0 +1,305 @@ +```python exec="1" session="data-sources" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + + +# Data Sources + +DataFusion provides a wide variety of ways to get data into a DataFrame to perform operations. + +## Local file + +DataFusion has the ability to read from a variety of popular file formats, such as [Parquet](../io/parquet/), +[CSV](../io/csv/), [JSON](../io/json/), and [AVRO](../io/avro/). + +```python exec="1" source="material-block" result="text" session="data-sources" +ctx = SessionContext() +df = ctx.read_csv("pokemon.csv") +df.show() +``` + + +## Create in-memory + +Sometimes it can be convenient to create a small DataFrame from a Python list or dictionary object. +To do this in DataFusion, you can use one of the three functions +[`from_pydict`][datafusion.context.SessionContext.from_pydict], +[`from_pylist`][datafusion.context.SessionContext.from_pylist], or +[`create_dataframe`][datafusion.context.SessionContext.create_dataframe]. + +As their names suggest, [`from_pydict`][datafusion.context.SessionContext.from_pydict] and [`from_pylist`][datafusion.context.SessionContext.from_pylist] will create DataFrames from Python +dictionary and list objects, respectively. [`create_dataframe`][datafusion.context.SessionContext.create_dataframe] assumes you will pass in a list +of list of [PyArrow Record Batches](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html). + +The following three examples all will create identical DataFrames: + +```python exec="1" source="material-block" result="text" session="data-sources" +import pyarrow as pa + +ctx.from_pylist( + [ + {"a": 1, "b": 10.0, "c": "alpha"}, + {"a": 2, "b": 20.0, "c": "beta"}, + {"a": 3, "b": 30.0, "c": "gamma"}, + ] +).show() + +ctx.from_pydict( + { + "a": [1, 2, 3], + "b": [10.0, 20.0, 30.0], + "c": ["alpha", "beta", "gamma"], + } +).show() + +batch = pa.RecordBatch.from_arrays( + [ + pa.array([1, 2, 3]), + pa.array([10.0, 20.0, 30.0]), + pa.array(["alpha", "beta", "gamma"]), + ], + names=["a", "b", "c"], +) + +ctx.create_dataframe([[batch]]).show() +``` + + +## Object Store + +DataFusion has support for multiple storage options in addition to local files. +The example below requires an appropriate S3 account with access credentials. + +Supported Object Stores are + +- [`AmazonS3`][datafusion.object_store.AmazonS3] +- [`GoogleCloud`][datafusion.object_store.GoogleCloud] +- [`Http`][datafusion.object_store.Http] +- [`LocalFileSystem`][datafusion.object_store.LocalFileSystem] +- [`MicrosoftAzure`][datafusion.object_store.MicrosoftAzure] + +```python +from datafusion.object_store import AmazonS3 + +region = "us-east-1" +bucket_name = "yellow-trips" + +s3 = AmazonS3( + bucket_name=bucket_name, + region=region, + access_key_id=os.getenv("AWS_ACCESS_KEY_ID"), + secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"), +) + +path = f"s3://{bucket_name}/" +ctx.register_object_store("s3://", s3, None) + +ctx.register_parquet("trips", path) + +ctx.table("trips").show() +``` + +## Other DataFrame Libraries + +DataFusion can import DataFrames directly from other libraries, such as +[Polars](https://pola.rs/) and [Pandas](https://pandas.pydata.org/). +Since DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule +interface can be imported to DataFusion using the +[`from_arrow`][datafusion.context.SessionContext.from_arrow] function. Older versions of Polars may +not support the arrow interface. In those cases, you can still import via the +[`from_polars`][datafusion.context.SessionContext.from_polars] function. + +```python +import pandas as pd + +data = { "a": [1, 2, 3], "b": [10.0, 20.0, 30.0], "c": ["alpha", "beta", "gamma"] } +pandas_df = pd.DataFrame(data) + +datafusion_df = ctx.from_arrow(pandas_df) +datafusion_df.show() +``` + +```python +import polars as pl +polars_df = pl.DataFrame(data) + +datafusion_df = ctx.from_arrow(polars_df) +datafusion_df.show() +``` + +## Delta Lake + +DataFusion 43.0.0 and later support the ability to register table providers from sources such +as Delta Lake. This will require a recent version of +[deltalake](https://delta-io.github.io/delta-rs/) to provide the required interfaces. + +```python +from deltalake import DeltaTable + +delta_table = DeltaTable("path_to_table") +ctx.register_table("my_delta_table", delta_table) +df = ctx.table("my_delta_table") +df.show() +``` + +On older versions of [`deltalake`](https://delta-io.github.io/delta-rs/) (prior to 0.22) you can use the +[Arrow DataSet](https://arrow.apache.org/docs/python/generated/pyarrow.dataset.Dataset.html) +interface to import to DataFusion, but this does not support features such as filter push down +which can lead to a significant performance difference. + +```python +from deltalake import DeltaTable + +delta_table = DeltaTable("path_to_table") +ctx.register_dataset("my_delta_table", delta_table.to_pyarrow_dataset()) +df = ctx.table("my_delta_table") +df.show() +``` + +## Apache Iceberg + +DataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface. + +This requires either the [pyiceberg](https://pypi.org/project/pyiceberg/) library (>=0.10.0) or the [pyiceberg-core](https://pypi.org/project/pyiceberg-core/) library (>=0.5.0). + +- The `pyiceberg-core` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings. +- The `pyiceberg` library utilizes the `pyiceberg-core` python bindings under the hood and provides a native way for Python users to interact with the DataFusion. + +```python +from datafusion import SessionContext +from pyiceberg.catalog import load_catalog +import pyarrow as pa + +# Load catalog and create/load a table +catalog = load_catalog("catalog", type="in-memory") +catalog.create_namespace_if_not_exists("default") + +# Create some sample data +data = pa.table({"x": [1, 2, 3], "y": [4, 5, 6]}) +iceberg_table = catalog.create_table("default.test", schema=data.schema) +iceberg_table.append(data) + +# Register the table with DataFusion +ctx = SessionContext() +ctx.register_table_provider("test", iceberg_table) + +# Query the table using DataFusion +ctx.table("test").show() +``` + +Note that the Datafusion integration rely on features from the [Iceberg Rust](https://github.com/apache/iceberg-rust/) implementation instead of the [PyIceberg](https://github.com/apache/iceberg-python/) implementation. +Features that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion. + +## Custom Table Provider + +You can implement a custom Data Provider in Rust and expose it to DataFusion through the +the interface as describe in the [Custom Table Provider](../io/table_provider/) +section. This is an advanced topic, but a +[user example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example) +is provided in the DataFusion repository. + +# Catalog + +A common technique for organizing tables is using a three level hierarchical approach. DataFusion +supports this form of organizing using the [`Catalog`][datafusion.catalog.Catalog], +[`Schema`][datafusion.catalog.Schema], and [`Table`][datafusion.catalog.Table]. By default, +a [`SessionContext`][datafusion.context.SessionContext] comes with a single Catalog and a single Schema +with the names `datafusion` and `public`, respectively. + +The default implementation uses an in-memory approach to the catalog and schema. We have support +for adding additional in-memory catalogs and schemas. You can access tables registered in a schema +either through the Dataframe API or via sql commands. This can be done like in the following +example: + +```python +import pyarrow as pa +from datafusion.catalog import Catalog, Schema +from datafusion import SessionContext + +ctx = SessionContext() + +my_catalog = Catalog.memory_catalog() +my_schema = Schema.memory_schema() +my_catalog.register_schema('my_schema_name', my_schema) +ctx.register_catalog_provider('my_catalog_name', my_catalog) + +# Create an in-memory table +table = pa.table({ + 'name': ['Bulbasaur', 'Charmander', 'Squirtle'], + 'type': ['Grass', 'Fire', 'Water'], + 'hp': [45, 39, 44], +}) +df = ctx.create_dataframe([table.to_batches()], name='pokemon') + +my_schema.register_table('pokemon', df) + +ctx.sql('SELECT * FROM my_catalog_name.my_schema_name.pokemon').show() +``` + +## User Defined Catalog and Schema + +If the in-memory catalogs are insufficient for your uses, there are two approaches you can take +to implementing a custom catalog and/or schema. In the below discussion, we describe how to +implement these for a Catalog, but the approach to implementing for a Schema is nearly +identical. + +DataFusion supports Catalogs written in either Rust or Python. If you write a Catalog in Rust, +you will need to export it as a Python library via PyO3. There is a complete example of a +catalog implemented this way in the +[examples folder](https://github.com/apache/datafusion-python/tree/main/examples/) +of our repository. Writing catalog providers in Rust provides typically can lead to significant +performance improvements over the Python based approach. + +To implement a Catalog in Python, you will need to inherit from the abstract base class +[`CatalogProvider`][datafusion.catalog.CatalogProvider]. There are examples in the +[unit tests](https://github.com/apache/datafusion-python/tree/main/python/tests) of +implementing a basic Catalog in Python where we simply keep a dictionary of the +registered Schemas. + +One important note for developers is that when we have a Catalog defined in Python, we have +two different ways of accessing this Catalog. First, we register the catalog with a Rust +wrapper. This allows for any rust based code to call the Python functions as necessary. +Second, if the user access the Catalog via the Python API, we identify this and return back +the original Python object that implements the Catalog. This is an important distinction +for developers because we do *not* return a Python wrapper around the Rust wrapper of the +original Python object. diff --git a/docs/source/user-guide/dataframe/index.md b/docs/source/user-guide/dataframe/index.md index 787f76ace..084f57fb3 100644 --- a/docs/source/user-guide/dataframe/index.md +++ b/docs/source/user-guide/dataframe/index.md @@ -302,7 +302,7 @@ streams = list(df.execute_stream_partitioned()) await asyncio.gather(*(consume(s) for s in streams)) ``` -See [../io/arrow](../io/arrow.ipynb) for additional details on the Arrow interface. +See [../io/arrow](../io/arrow.md) for additional details on the Arrow interface. ## HTML Rendering diff --git a/docs/source/user-guide/index.md b/docs/source/user-guide/index.md index c689e1861..34a29ea98 100644 --- a/docs/source/user-guide/index.md +++ b/docs/source/user-guide/index.md @@ -24,11 +24,11 @@ with the DataFrame API or SQL, reading and writing data, and tuning execution. ## Contents -- [Introduction](introduction.ipynb) — what DataFusion in Python is and +- [Introduction](introduction.md) — what DataFusion in Python is and when to reach for it. -- [Concepts](concepts.ipynb) — `SessionContext`, `DataFrame`, and +- [Concepts](concepts.md) — `SessionContext`, `DataFrame`, and `Expr` at a glance. -- [Data Sources](data-sources.ipynb) — reading Parquet / CSV / JSON / +- [Data Sources](data-sources.md) — reading Parquet / CSV / JSON / Avro, in-memory DataFrames, object stores, Delta Lake, Iceberg, custom table providers, and catalogs. - [DataFrame](dataframe/index.md) — building queries with the DataFrame @@ -41,7 +41,7 @@ with the DataFrame API or SQL, reading and writing data, and tuning execution. - [Distributing Work](distributing-work.md) — shipping expressions to worker processes via pickle / cloudpickle, FFI-capsule UDFs, and the sender/worker context model. -- [SQL](sql.ipynb) — registering tables and running SQL queries. +- [SQL](sql.md) — registering tables and running SQL queries. - [Upgrade Guides](upgrade-guides.md) — notes on cross-version migrations. - [AI Coding Assistants](ai-coding-assistants.md) — agent-facing diff --git a/docs/source/user-guide/introduction.ipynb b/docs/source/user-guide/introduction.ipynb deleted file mode 100644 index 3a0666529..000000000 --- a/docs/source/user-guide/introduction.ipynb +++ /dev/null @@ -1,104 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f # noqa: F401\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n\n# Introduction\n\nWelcome to the User Guide for the Python bindings of Arrow DataFusion. This guide aims to provide an introduction to\nDataFusion through various examples and highlight the most effective ways of using it.\n\n## Installation\n\nDataFusion is a Python library and, as such, can be installed via pip from [PyPI](https://pypi.org/project/datafusion).\n\n```shell\npip install datafusion\n```\n\nYou can verify the installation by running:\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "datafusion.__version__" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\nIn this documentation we will also show some examples for how DataFusion integrates\nwith Jupyter notebooks. To install and start a Jupyter labs session use\n\n```shell\npip install jupyterlab\njupyter lab\n```\n\nTo demonstrate working with DataFusion, we need a data source. Later in the tutorial we will show\noptions for data sources. For our first example, we demonstrate using a Pokemon dataset that you\ncan download\n[here](https://gist.githubusercontent.com/ritchie46/cac6b337ea52281aa23c049250a4ff03/raw/89a957ff3919d90e6ef2d34235e6bf22304f3366/pokemon.csv).\n\nWith that file in place you can use the following python example to view the DataFrame in\nDataFusion.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "ctx = SessionContext()\n", - "\n", - "df = ctx.read_csv(\"pokemon.csv\")\n", - "\n", - "df.show()" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\nIf you are working in a Jupyter notebook, you can also use the following to give you a table\ndisplay that may be easier to read.\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "display(df)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/introduction.md b/docs/source/user-guide/introduction.md new file mode 100644 index 000000000..d159ac596 --- /dev/null +++ b/docs/source/user-guide/introduction.md @@ -0,0 +1,98 @@ +```python exec="1" session="introduction" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + + +# Introduction + +Welcome to the User Guide for the Python bindings of Arrow DataFusion. This guide aims to provide an introduction to +DataFusion through various examples and highlight the most effective ways of using it. + +## Installation + +DataFusion is a Python library and, as such, can be installed via pip from [PyPI](https://pypi.org/project/datafusion). + +```shell +pip install datafusion +``` + +You can verify the installation by running: + +```python exec="1" source="material-block" result="text" session="introduction" +datafusion.__version__ +``` + + +In this documentation we will also show some examples for how DataFusion integrates +with Jupyter notebooks. To install and start a Jupyter labs session use + +```shell +pip install jupyterlab +jupyter lab +``` + +To demonstrate working with DataFusion, we need a data source. Later in the tutorial we will show +options for data sources. For our first example, we demonstrate using a Pokemon dataset that you +can download +[here](https://gist.githubusercontent.com/ritchie46/cac6b337ea52281aa23c049250a4ff03/raw/89a957ff3919d90e6ef2d34235e6bf22304f3366/pokemon.csv). + +With that file in place you can use the following python example to view the DataFrame in +DataFusion. + +```python exec="1" source="material-block" result="text" session="introduction" +ctx = SessionContext() + +df = ctx.read_csv("pokemon.csv") + +df.show() +``` + + +If you are working in a Jupyter notebook, you can also use the following to give you a table +display that may be easier to read. + +```python +display(df) +``` + +![Rendered table showing Pokemon DataFrame](../images/jupyter_lab_df_view.png) diff --git a/docs/source/user-guide/io/arrow.ipynb b/docs/source/user-guide/io/arrow.ipynb deleted file mode 100644 index 243266b73..000000000 --- a/docs/source/user-guide/io/arrow.ipynb +++ /dev/null @@ -1,92 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion # noqa: F401\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f # noqa: F401\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n# Arrow\n\nDataFusion implements the\n[Apache Arrow PyCapsule interface](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html)\nfor importing and exporting DataFrames with zero copy. With this feature, any Python\nproject that implements this interface can share data back and forth with DataFusion\nwith zero copy.\n\nWe can demonstrate using [pyarrow](https://arrow.apache.org/docs/python/index.html).\n\n## Importing to DataFusion\n\nHere we will create an Arrow table and import it to DataFusion.\n\nTo import an Arrow table, use [`from_arrow`][datafusion.context.SessionContext.from_arrow].\nThis will accept any Python object that implements\n[\\_\\_arrow_c_stream\\_\\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowstream-export)\nor [\\_\\_arrow_c_array\\_\\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowarray-export)\nand returns a `StructArray`. Common pyarrow sources you can use are:\n\n- [Array](https://arrow.apache.org/docs/python/generated/pyarrow.Array.html) (but it must return a Struct Array)\n- [Record Batch](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html)\n- [Record Batch Reader](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatchReader.html)\n- [Table](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html)\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "import pyarrow as pa\n", - "\n", - "data = {\"a\": [1, 2, 3], \"b\": [4, 5, 6]}\n", - "table = pa.Table.from_pydict(data)\n", - "\n", - "ctx = SessionContext()\n", - "df = ctx.from_arrow(table)\n", - "df" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\n## Exporting from DataFusion\n\nDataFusion DataFrames implement `__arrow_c_stream__` PyCapsule interface, so any\nPython library that accepts these can import a DataFusion DataFrame directly.\n\nInvoking `__arrow_c_stream__` triggers execution of the underlying query, but\nbatches are yielded incrementally rather than materialized all at once in memory.\nConsumers can process the stream as it arrives. The stream executes lazily,\nletting downstream readers pull batches on demand.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "df = df.select((col(\"a\") * lit(1.5)).alias(\"c\"), lit(\"df\").alias(\"d\"))\n", - "pa.table(df)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/io/arrow.md b/docs/source/user-guide/io/arrow.md new file mode 100644 index 000000000..4dc9a725c --- /dev/null +++ b/docs/source/user-guide/io/arrow.md @@ -0,0 +1,95 @@ +```python exec="1" session="arrow" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + +# Arrow + +DataFusion implements the +[Apache Arrow PyCapsule interface](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) +for importing and exporting DataFrames with zero copy. With this feature, any Python +project that implements this interface can share data back and forth with DataFusion +with zero copy. + +We can demonstrate using [pyarrow](https://arrow.apache.org/docs/python/index.html). + +## Importing to DataFusion + +Here we will create an Arrow table and import it to DataFusion. + +To import an Arrow table, use [`from_arrow`][datafusion.context.SessionContext.from_arrow]. +This will accept any Python object that implements +[\_\_arrow_c_stream\_\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowstream-export) +or [\_\_arrow_c_array\_\_](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html#arrowarray-export) +and returns a `StructArray`. Common pyarrow sources you can use are: + +- [Array](https://arrow.apache.org/docs/python/generated/pyarrow.Array.html) (but it must return a Struct Array) +- [Record Batch](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html) +- [Record Batch Reader](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatchReader.html) +- [Table](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html) + +```python exec="1" source="material-block" result="text" session="arrow" +import pyarrow as pa + +data = {"a": [1, 2, 3], "b": [4, 5, 6]} +table = pa.Table.from_pydict(data) + +ctx = SessionContext() +df = ctx.from_arrow(table) +df +``` + + +## Exporting from DataFusion + +DataFusion DataFrames implement `__arrow_c_stream__` PyCapsule interface, so any +Python library that accepts these can import a DataFusion DataFrame directly. + +Invoking `__arrow_c_stream__` triggers execution of the underlying query, but +batches are yielded incrementally rather than materialized all at once in memory. +Consumers can process the stream as it arrives. The stream executes lazily, +letting downstream readers pull batches on demand. + +```python exec="1" source="material-block" result="text" session="arrow" +df = df.select((col("a") * lit(1.5)).alias("c"), lit("df").alias("d")) +pa.table(df) +``` diff --git a/docs/source/user-guide/io/index.md b/docs/source/user-guide/io/index.md index 6b5478186..5aa9e3992 100644 --- a/docs/source/user-guide/io/index.md +++ b/docs/source/user-guide/io/index.md @@ -26,7 +26,7 @@ through Arrow-compatible Python objects. | Format | Reader | Notes | |---|---|---| -| [Apache Arrow](arrow.ipynb) | [`SessionContext.read_arrow`][datafusion.context.SessionContext.read_arrow] | Single Arrow IPC file. | +| [Apache Arrow](arrow.md) | [`SessionContext.read_arrow`][datafusion.context.SessionContext.read_arrow] | Single Arrow IPC file. | | [Avro](avro.md) | [`SessionContext.read_avro`][datafusion.context.SessionContext.read_avro] | Schema-on-read; requires the Avro feature in the wheel. | | [CSV](csv.md) | [`SessionContext.read_csv`][datafusion.context.SessionContext.read_csv] | Header inference, custom delimiters, gzip/bz2 compression. | | [JSON](json.md) | [`SessionContext.read_json`][datafusion.context.SessionContext.read_json] | Newline-delimited JSON; one record per line. | @@ -40,6 +40,6 @@ through Arrow-compatible Python objects. ## See also -- [Data Sources](../data-sources.ipynb) — concept overview, including +- [Data Sources](../data-sources.md) — concept overview, including in-memory DataFrame creation from `pyarrow` / `pandas` / `polars` and object-store integration. diff --git a/docs/source/user-guide/sql.ipynb b/docs/source/user-guide/sql.ipynb deleted file mode 100644 index 25c673b78..000000000 --- a/docs/source/user-guide/sql.ipynb +++ /dev/null @@ -1,153 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "7fb27b941602401d91542211134fc71a", - "metadata": { - "tags": [ - "nb-setup", - "remove-input", - "remove-output" - ] - }, - "outputs": [], - "source": [ - "import os\n", - "import pathlib\n", - "\n", - "import datafusion\n", - "from datafusion import ( # noqa: F401\n", - " SessionContext,\n", - " col,\n", - " column,\n", - " lit,\n", - " literal,\n", - ")\n", - "from datafusion import functions as f # noqa: F401\n", - "from datafusion.dataframe_formatter import configure_formatter\n", - "\n", - "_p = pathlib.Path.cwd()\n", - "while _p != _p.parent and not (_p / \"pokemon.csv\").exists():\n", - " _p = _p.parent\n", - "if (_p / \"pokemon.csv\").exists():\n", - " os.chdir(_p)\n", - "\n", - "configure_formatter(max_rows=10, show_truncation_message=False)" - ] - }, - { - "cell_type": "markdown", - "id": "acae54e37e7d407bbb7b55eff062a284", - "metadata": {}, - "source": "\n\n# SQL\n\nDataFusion also offers a SQL API, read the full reference [here](https://arrow.apache.org/datafusion/user-guide/sql/index.html)\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9a63283cbaf04dbcab1f6479b197f3a8", - "metadata": {}, - "outputs": [], - "source": [ - "from datafusion import DataFrame\n", - "\n", - "# create a context\n", - "ctx = datafusion.SessionContext()\n", - "\n", - "# register a CSV\n", - "ctx.register_csv(\"pokemon\", \"pokemon.csv\")\n", - "\n", - "# create a new statement via SQL\n", - "df = ctx.sql('SELECT \"Attack\"+\"Defense\", \"Attack\"-\"Defense\" FROM pokemon')\n", - "\n", - "# collect and convert to pandas DataFrame\n", - "df.to_pandas()" - ] - }, - { - "cell_type": "markdown", - "id": "8dd0d8092fe74a7c96281538738b07e2", - "metadata": {}, - "source": "\n## Parameterized queries\n\nIn DataFusion-Python 51.0.0 we introduced the ability to pass parameters\nin a SQL query. These are similar in concept to\n[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html),\nbut allow passing named parameters into a SQL query. Consider this simple\nexample.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "72eea5119410473aa328ad9291626812", - "metadata": {}, - "outputs": [], - "source": [ - "def show_attacks(ctx: SessionContext, threshold: int) -> None:\n", - " ctx.sql(\n", - " 'SELECT \"Name\", \"Attack\" FROM pokemon WHERE \"Attack\" > $val', val=threshold\n", - " ).show(num=5)\n", - "\n", - "\n", - "show_attacks(ctx, 75)" - ] - }, - { - "cell_type": "markdown", - "id": "8edb47106e1a46a883d545849b8ab81b", - "metadata": {}, - "source": "\nWhen passing parameters like the example above we convert the Python objects\ninto their string representation. We also have special case handling\nfor [`DataFrame`][datafusion.dataframe.DataFrame] objects, since they cannot simply\nbe turned into string representations for an SQL query. In these cases we\nwill register a temporary view in the [`SessionContext`][datafusion.context.SessionContext]\nusing a generated table name.\n\nThe formatting for passing string replacement objects is to precede the\nvariable name with a single `$`. This works for all dialects in\nthe SQL parser except `hive` and `mysql`. Since these dialects do not\nsupport named placeholders, we are unable to do this type of replacement.\nWe recommend either switching to another dialect or using Python\nf-string style replacement.\n\n
\n

Warning

\n\nTo support DataFrame parameterized queries, your session must support\nregistration of temporary views. The default\n[`CatalogProvider`][datafusion.catalog.CatalogProvider] and\n[`SchemaProvider`][datafusion.catalog.SchemaProvider] do have this capability.\nIf you have implemented custom providers, it is important that temporary\nviews do not persist across [`SessionContext`][datafusion.context.SessionContext]\nor you may get unintended consequences.\n\n
\n\nThe following example shows passing in both a [`DataFrame`][datafusion.dataframe.DataFrame]\nobject as well as a Python object to be used in parameterized replacement.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "10185d26023b46108eb7d9f57d49d2b3", - "metadata": {}, - "outputs": [], - "source": [ - "def show_column(\n", - " ctx: SessionContext, column: str, df: DataFrame, threshold: int\n", - ") -> None:\n", - " ctx.sql(\n", - " 'SELECT \"Name\", $col FROM $df WHERE $col > $val',\n", - " col=column,\n", - " df=df,\n", - " val=threshold,\n", - " ).show(num=5)\n", - "\n", - "\n", - "df = ctx.table(\"pokemon\")\n", - "show_column(ctx, '\"Defense\"', df, 75)" - ] - }, - { - "cell_type": "markdown", - "id": "8763a12b2bbd4a93a75aff182afb95dc", - "metadata": {}, - "source": "\nThe approach implemented for conversion of variables into a SQL query\nrelies on string conversion. This has the potential for data loss,\nspecifically for cases like floating point numbers. If you need to pass\nvariables into a parameterized query and it is important to maintain the\noriginal value without conversion to a string, then you can use the\noptional parameter `param_values` to specify these. This parameter\nexpects a dictionary mapping from the parameter name to a Python\nobject. Those objects will be cast into a\n[PyArrow Scalar Value](https://arrow.apache.org/docs/python/generated/pyarrow.Scalar.html).\n\nUsing `param_values` will rely on the SQL dialect you have configured\nfor your session. This can be set using the [configuration options](../configuration/)\nof your [`SessionContext`][datafusion.context.SessionContext]. Similar to how\n[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html)\nwork, these parameters are limited to places where you would pass in a\nscalar value, such as a comparison.\n\n" - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7623eae2785240b9bd12b16a66d81610", - "metadata": {}, - "outputs": [], - "source": [ - "def param_attacks(ctx: SessionContext, threshold: int) -> None:\n", - " ctx.sql(\n", - " 'SELECT \"Name\", \"Attack\" FROM pokemon WHERE \"Attack\" > $val',\n", - " param_values={\"val\": threshold},\n", - " ).show(num=5)\n", - "\n", - "\n", - "param_attacks(ctx, 75)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/docs/source/user-guide/sql.md b/docs/source/user-guide/sql.md new file mode 100644 index 000000000..df3e15406 --- /dev/null +++ b/docs/source/user-guide/sql.md @@ -0,0 +1,158 @@ +```python exec="1" session="sql" +import os +import pathlib + +import datafusion # noqa: F401 +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs runs from the repo root; the demo data lives at docs/source/. +for candidate in ("docs/source", ".."): + p = pathlib.Path(candidate) + if (p / "pokemon.csv").exists(): + os.chdir(p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) +``` + + + +# SQL + +DataFusion also offers a SQL API, read the full reference [here](https://arrow.apache.org/datafusion/user-guide/sql/index.html) + +```python exec="1" source="material-block" result="text" session="sql" +from datafusion import DataFrame + +# create a context +ctx = datafusion.SessionContext() + +# register a CSV +ctx.register_csv("pokemon", "pokemon.csv") + +# create a new statement via SQL +df = ctx.sql('SELECT "Attack"+"Defense", "Attack"-"Defense" FROM pokemon') + +# collect and convert to pandas DataFrame +df.to_pandas() +``` + + +## Parameterized queries + +In DataFusion-Python 51.0.0 we introduced the ability to pass parameters +in a SQL query. These are similar in concept to +[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html), +but allow passing named parameters into a SQL query. Consider this simple +example. + +```python exec="1" source="material-block" result="text" session="sql" +def show_attacks(ctx: SessionContext, threshold: int) -> None: + ctx.sql( + 'SELECT "Name", "Attack" FROM pokemon WHERE "Attack" > $val', val=threshold + ).show(num=5) + + +show_attacks(ctx, 75) +``` + + +When passing parameters like the example above we convert the Python objects +into their string representation. We also have special case handling +for [`DataFrame`][datafusion.dataframe.DataFrame] objects, since they cannot simply +be turned into string representations for an SQL query. In these cases we +will register a temporary view in the [`SessionContext`][datafusion.context.SessionContext] +using a generated table name. + +The formatting for passing string replacement objects is to precede the +variable name with a single `$`. This works for all dialects in +the SQL parser except `hive` and `mysql`. Since these dialects do not +support named placeholders, we are unable to do this type of replacement. +We recommend either switching to another dialect or using Python +f-string style replacement. + +
+

Warning

+ +To support DataFrame parameterized queries, your session must support +registration of temporary views. The default +[`CatalogProvider`][datafusion.catalog.CatalogProvider] and +[`SchemaProvider`][datafusion.catalog.SchemaProvider] do have this capability. +If you have implemented custom providers, it is important that temporary +views do not persist across [`SessionContext`][datafusion.context.SessionContext] +or you may get unintended consequences. + +
+ +The following example shows passing in both a [`DataFrame`][datafusion.dataframe.DataFrame] +object as well as a Python object to be used in parameterized replacement. + +```python exec="1" source="material-block" result="text" session="sql" +def show_column( + ctx: SessionContext, column: str, df: DataFrame, threshold: int +) -> None: + ctx.sql( + 'SELECT "Name", $col FROM $df WHERE $col > $val', + col=column, + df=df, + val=threshold, + ).show(num=5) + + +df = ctx.table("pokemon") +show_column(ctx, '"Defense"', df, 75) +``` + + +The approach implemented for conversion of variables into a SQL query +relies on string conversion. This has the potential for data loss, +specifically for cases like floating point numbers. If you need to pass +variables into a parameterized query and it is important to maintain the +original value without conversion to a string, then you can use the +optional parameter `param_values` to specify these. This parameter +expects a dictionary mapping from the parameter name to a Python +object. Those objects will be cast into a +[PyArrow Scalar Value](https://arrow.apache.org/docs/python/generated/pyarrow.Scalar.html). + +Using `param_values` will rely on the SQL dialect you have configured +for your session. This can be set using the [configuration options](../configuration/) +of your [`SessionContext`][datafusion.context.SessionContext]. Similar to how +[prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html) +work, these parameters are limited to places where you would pass in a +scalar value, such as a comparison. + +```python exec="1" source="material-block" result="text" session="sql" +def param_attacks(ctx: SessionContext, threshold: int) -> None: + ctx.sql( + 'SELECT "Name", "Attack" FROM pokemon WHERE "Attack" > $val', + param_values={"val": threshold}, + ).show(num=5) + + +param_attacks(ctx, 75) +``` diff --git a/mkdocs.yml b/mkdocs.yml index 2b4d0d490..ae2444aad 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -55,15 +55,7 @@ extra_css: plugins: - search - - mkdocs-jupyter: - execute: true - allow_errors: false - include_source: false - ignore_h1_titles: true - remove_tag_config: - remove_cell_tags: ["remove-cell"] - remove_input_tags: ["remove-input"] - remove_all_outputs_tags: ["remove-output"] + - markdown-exec - mkdocstrings: default_handler: python handlers: @@ -113,34 +105,32 @@ markdown_extensions: watch: - python/datafusion -hooks: - - docs/hooks.py nav: - - Home: index.ipynb + - Home: index.md - User Guide: - user-guide/index.md - - Introduction: user-guide/introduction.ipynb - - Concepts: user-guide/concepts.ipynb - - Data Sources: user-guide/data-sources.ipynb + - Introduction: user-guide/introduction.md + - Concepts: user-guide/concepts.md + - Data Sources: user-guide/data-sources.md - DataFrame: - user-guide/dataframe/index.md - Rendering: user-guide/dataframe/rendering.md - Execution Metrics: user-guide/dataframe/execution-metrics.md - Common Operations: - user-guide/common-operations/index.md - - Basic Info: user-guide/common-operations/basic-info.ipynb + - Basic Info: user-guide/common-operations/basic-info.md - Views: user-guide/common-operations/views.md - - Select and Filter: user-guide/common-operations/select-and-filter.ipynb - - Expressions: user-guide/common-operations/expressions.ipynb - - Joins: user-guide/common-operations/joins.ipynb - - Functions: user-guide/common-operations/functions.ipynb - - Aggregations: user-guide/common-operations/aggregations.ipynb - - Windows: user-guide/common-operations/windows.ipynb - - User-Defined Functions: user-guide/common-operations/udf-and-udfa.ipynb + - Select and Filter: user-guide/common-operations/select-and-filter.md + - Expressions: user-guide/common-operations/expressions.md + - Joins: user-guide/common-operations/joins.md + - Functions: user-guide/common-operations/functions.md + - Aggregations: user-guide/common-operations/aggregations.md + - Windows: user-guide/common-operations/windows.md + - User-Defined Functions: user-guide/common-operations/udf-and-udfa.md - I/O: - user-guide/io/index.md - - Arrow: user-guide/io/arrow.ipynb + - Arrow: user-guide/io/arrow.md - Avro: user-guide/io/avro.md - CSV: user-guide/io/csv.md - JSON: user-guide/io/json.md @@ -148,7 +138,7 @@ nav: - Table Provider: user-guide/io/table_provider.md - Configuration: user-guide/configuration.md - Distributing Work: user-guide/distributing-work.md - - SQL: user-guide/sql.ipynb + - SQL: user-guide/sql.md - Upgrade Guides: user-guide/upgrade-guides.md - AI Coding Assistants: user-guide/ai-coding-assistants.md - Contributor Guide: diff --git a/pyproject.toml b/pyproject.toml index 274ed5890..69dde845e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -226,12 +226,10 @@ dev = [ # from sdist under free-threaded interpreters (PyO3 < 3.14 support). release = ["pygithub==2.5.0"] docs = [ - "ipython>=8.12.3", + "markdown-exec[ansi]>=1.10", "mkdocs>=1.6,<2", - "mkdocs-jupyter>=0.25", "mkdocs-material>=9.5,<10", "mkdocs-redirects>=1.2", "mkdocstrings[python]>=0.27", "pandas>=2.0.3", - "pickleshare>=0.7.5", ] diff --git a/uv.lock b/uv.lock index 22c38c9d0..77b94882d 100644 --- a/uv.lock +++ b/uv.lock @@ -8,15 +8,6 @@ resolution-markers = [ "python_full_version < '3.11'", ] -[[package]] -name = "appnope" -version = "0.1.4" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/35/5d/752690df9ef5b76e169e68d6a129fa6d08a7100ca7f754c89495db3c6019/appnope-0.1.4.tar.gz", hash = "sha256:1de3860566df9caf38f01f86f65e0e13e379af54f9e4bee1e66b48f2efffd1ee", size = 4170, upload-time = "2024-02-06T09:43:11.258Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/81/29/5ecc3a15d5a33e31b26c11426c45c501e439cb865d0bff96315d86443b78/appnope-0.1.4-py2.py3-none-any.whl", hash = "sha256:502575ee11cd7a28c0205f379b525beefebab9d161b7c964670864014ed7213c", size = 4321, upload-time = "2024-02-06T09:43:09.663Z" }, -] - [[package]] name = "arro3-core" version = "0.6.5" @@ -79,24 +70,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/0b/fd/4f8dac58ea17e05978bf35cb9a3e485b1ff3cdd6e2cc29deb08f54080de4/arro3_core-0.6.5-pp310-pypy310_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:9a58acbc61480b533aa84d735db04b1e68fc7f6807ab694d606c03b5e694d83d", size = 2954405, upload-time = "2025-10-13T23:12:35.328Z" }, ] -[[package]] -name = "asttokens" -version = "3.0.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/4a/e7/82da0a03e7ba5141f05cce0d302e6eed121ae055e0456ca228bf693984bc/asttokens-3.0.0.tar.gz", hash = "sha256:0dcd8baa8d62b0c1d118b399b2ddba3c4aff271d0d7a9e0d4c1681c79035bbc7", size = 61978, upload-time = "2024-11-30T04:30:14.439Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/25/8a/c46dcc25341b5bce5472c718902eb3d38600a903b14fa6aeecef3f21a46f/asttokens-3.0.0-py3-none-any.whl", hash = "sha256:e3078351a059199dd5138cb1c706e6430c05eff2ff136af5eb4790f9d28932e2", size = 26918, upload-time = "2024-11-30T04:30:10.946Z" }, -] - -[[package]] -name = "attrs" -version = "26.1.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/9a/8e/82a0fe20a541c03148528be8cac2408564a6c9a0cc7e9171802bc1d26985/attrs-26.1.0.tar.gz", hash = "sha256:d03ceb89cb322a8fd706d4fb91940737b6642aa36998fe130a9bc96c985eff32", size = 952055, upload-time = "2026-03-19T14:22:25.026Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/64/b4/17d4b0b2a2dc85a6df63d1157e028ed19f90d4cd97c36717afef2bc2f395/attrs-26.1.0-py3-none-any.whl", hash = "sha256:c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309", size = 67548, upload-time = "2026-03-19T14:22:23.645Z" }, -] - [[package]] name = "babel" version = "2.16.0" @@ -119,35 +92,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/3e/5c/fb93d3092640a24dfb7bd7727a24016d7c01774ca013e60efd3f683c8002/backrefs-7.0-py314-none-any.whl", hash = "sha256:a6448b28180e3ca01134c9cf09dcebafad8531072e09903c5451748a05f24bc9", size = 412349, upload-time = "2026-04-28T16:28:02.412Z" }, ] -[[package]] -name = "beautifulsoup4" -version = "4.12.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "soupsieve" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/b3/ca/824b1195773ce6166d388573fc106ce56d4a805bd7427b624e063596ec58/beautifulsoup4-4.12.3.tar.gz", hash = "sha256:74e3d1928edc070d21748185c46e3fb33490f22f52a3addee9aee0f4f7781051", size = 581181, upload-time = "2024-01-17T16:53:17.902Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b1/fe/e8c672695b37eecc5cbf43e1d0638d88d66ba3a44c4d321c796f4e59167f/beautifulsoup4-4.12.3-py3-none-any.whl", hash = "sha256:b80878c9f40111313e55da8ba20bdba06d8fa3969fc68304167741bbf9e082ed", size = 147925, upload-time = "2024-01-17T16:53:12.779Z" }, -] - -[[package]] -name = "bleach" -version = "6.4.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "webencodings" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/48/3c/e12ac860709702bd5ebeb9b56a4fe334f1001246ee1b8f2b7ee28912df7d/bleach-6.4.0.tar.gz", hash = "sha256:4202482733d85cedd04e59fcb2f89f4e4c7c385a78d3c3c23c30446843a37452", size = 204857, upload-time = "2026-06-05T13:01:13.734Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/58/9d/40b6267367182187139a4000b82a3b287d84d745bccd808e75d916920e9d/bleach-6.4.0-py3-none-any.whl", hash = "sha256:4b6b6a54fff2e69a3dde9d21cc6301220bee3c3cb792187d11403fd795031081", size = 165109, upload-time = "2026-06-05T13:01:12.504Z" }, -] - -[package.optional-dependencies] -css = [ - { name = "tinycss2" }, -] - [[package]] name = "certifi" version = "2024.12.14" @@ -323,15 +267,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, ] -[[package]] -name = "comm" -version = "0.2.3" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/4c/13/7d740c5849255756bc17888787313b61fd38a0a8304fc4f073dfc46122aa/comm-0.2.3.tar.gz", hash = "sha256:2dc8048c10962d55d7ad693be1e7045d891b7ce8d999c97963a5e3e99c055971", size = 6319, upload-time = "2025-07-25T14:02:04.452Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/60/97/891a0971e1e4a8c5d2b20bbe0e524dc04548d2307fee33cdeba148fd4fc7/comm-0.2.3-py3-none-any.whl", hash = "sha256:c615d91d75f7f04f095b30d1c1711babd43bdc6419c1be9886a85f2f4e489417", size = 7294, upload-time = "2025-07-25T14:02:02.896Z" }, -] - [[package]] name = "cryptography" version = "44.0.0" @@ -396,14 +331,12 @@ dev = [ { name = "toml" }, ] docs = [ - { name = "ipython" }, + { name = "markdown-exec", extra = ["ansi"] }, { name = "mkdocs" }, - { name = "mkdocs-jupyter" }, { name = "mkdocs-material" }, { name = "mkdocs-redirects" }, { name = "mkdocstrings", extra = ["python"] }, { name = "pandas" }, - { name = "pickleshare" }, ] release = [ { name = "pygithub" }, @@ -435,64 +368,15 @@ dev = [ { name = "toml", specifier = ">=0.10.2" }, ] docs = [ - { name = "ipython", specifier = ">=8.12.3" }, + { name = "markdown-exec", extras = ["ansi"], specifier = ">=1.10" }, { name = "mkdocs", specifier = ">=1.6,<2" }, - { name = "mkdocs-jupyter", specifier = ">=0.25" }, { name = "mkdocs-material", specifier = ">=9.5,<10" }, { name = "mkdocs-redirects", specifier = ">=1.2" }, { name = "mkdocstrings", extras = ["python"], specifier = ">=0.27" }, { name = "pandas", specifier = ">=2.0.3" }, - { name = "pickleshare", specifier = ">=0.7.5" }, ] release = [{ name = "pygithub", specifier = "==2.5.0" }] -[[package]] -name = "debugpy" -version = "1.8.21" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/f2/aa/12037145b7a56eaa5b29b41872f7a21b538e807e13f32c4d3c46e59be084/debugpy-1.8.21.tar.gz", hash = "sha256:a3c53278e84c94e11bd87c53970ec391d1a67396c8b22609fcac576520e611a6", size = 1697577, upload-time = "2026-06-01T19:30:35.156Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/fc/f3/6b1d4c71f4cbb5360009f928934a03b42906f28fc7b3f7f35f04e58acead/debugpy-1.8.21-cp310-cp310-macosx_15_0_x86_64.whl", hash = "sha256:8eeab7b5462f683452c57c0126aaa5ec4e974ddb705f39ba87dff8818c8e08f9", size = 2113873, upload-time = "2026-06-01T19:30:37.148Z" }, - { url = "https://files.pythonhosted.org/packages/1c/f2/17c3bf91cebc173bfbf5734cd2669723d0a35c0cf9d2fd2124546efeae83/debugpy-1.8.21-cp310-cp310-manylinux_2_34_x86_64.whl", hash = "sha256:0fddfdc130ac6d8bfc0415b0409822fa901c8f310e5c945ac5653a0352532344", size = 3004715, upload-time = "2026-06-01T19:30:38.888Z" }, - { url = "https://files.pythonhosted.org/packages/5a/22/1f8efd80c7b5909e760f9cfd0c9e8681d2d35d532f7c0a40760cd4da4a19/debugpy-1.8.21-cp310-cp310-win32.whl", hash = "sha256:72b5d676c4cbfac3bac5bb01c138a4656e843f93f03ce2a5f4e394ad49fbee73", size = 5303455, upload-time = "2026-06-01T19:30:40.52Z" }, - { url = "https://files.pythonhosted.org/packages/da/ce/54c79abd6cccef92fa7b43d97e3acafedf4d645557267ece05e948b5e4b8/debugpy-1.8.21-cp310-cp310-win_amd64.whl", hash = "sha256:a7fe47fd23da57b9e0bec3f4a8ee65a2dc55782455ed7f2141d75ab5d2eaeef5", size = 5331751, upload-time = "2026-06-01T19:30:42.146Z" }, - { url = "https://files.pythonhosted.org/packages/89/fb/cbf306d6e07a313a91e7171a98669054502840931432c227cfd505ee367f/debugpy-1.8.21-cp311-cp311-macosx_15_0_universal2.whl", hash = "sha256:da456226c7b4c69e35dbe35dcee6623d912000a77816db7856a41af1c72a0264", size = 2203120, upload-time = "2026-06-01T19:30:43.964Z" }, - { url = "https://files.pythonhosted.org/packages/aa/57/aa739bd4ad2cbf96aeb1b20b56918ddd5ae4c28b68709bfcd327f02123ee/debugpy-1.8.21-cp311-cp311-manylinux_2_34_x86_64.whl", hash = "sha256:f68b891688e61bdc08b8d364d919ff0051e0b94657b39dcd027bc3173edb7cdc", size = 3059958, upload-time = "2026-06-01T19:30:45.622Z" }, - { url = "https://files.pythonhosted.org/packages/a8/31/453d2c9a23d133fe2c8ec7ca1d816ded52a913487fe3ffef7c01b4b706af/debugpy-1.8.21-cp311-cp311-win32.whl", hash = "sha256:f843a8b08c2edeaf9b1582eed4f25441af21a297c22ff16bf76a662557aa9c9e", size = 5236515, upload-time = "2026-06-01T19:30:47.461Z" }, - { url = "https://files.pythonhosted.org/packages/60/94/6660de2f2d7bf388f229335ba4637646eebabdbf38564cb439a95a9193c9/debugpy-1.8.21-cp311-cp311-win_amd64.whl", hash = "sha256:84c564d8cc701d41843b29a92814c1f1bef6798724ca9d675c284ad9f6a547d7", size = 5256138, upload-time = "2026-06-01T19:30:49.113Z" }, - { url = "https://files.pythonhosted.org/packages/a2/df/bf625547431a9cadc9f4cbfeda38866e2b17f6aed147b625377e87834449/debugpy-1.8.21-cp312-cp312-macosx_15_0_universal2.whl", hash = "sha256:9f96713896f39c3dff0ee841f47320c3f2983d33c341e009361bb0ebc79adc4e", size = 2483609, upload-time = "2026-06-01T19:30:50.794Z" }, - { url = "https://files.pythonhosted.org/packages/bf/09/59324b903599031ff9faaec1758292409f6561a0ec2492fe4b703327705a/debugpy-1.8.21-cp312-cp312-manylinux_2_34_x86_64.whl", hash = "sha256:c193d474f0a211191f2b4449d2d06157c689013035bd952f3b617e0ef422b176", size = 3968900, upload-time = "2026-06-01T19:30:52.341Z" }, - { url = "https://files.pythonhosted.org/packages/14/cd/27f65b805d7fe005c44e1a36b9183ecdfbcdbf9d3e721a5115d461ecc7ee/debugpy-1.8.21-cp312-cp312-win32.whl", hash = "sha256:4743373c1cac7f9e74a1b9915bf1dbe0e900eca657ffb170ae07ac8363205ae9", size = 5336340, upload-time = "2026-06-01T19:30:54.047Z" }, - { url = "https://files.pythonhosted.org/packages/77/1d/c84e30c0c674184948b66f076ab271c01d940618a2824c23cd035a27bc20/debugpy-1.8.21-cp312-cp312-win_amd64.whl", hash = "sha256:bd7ba9dd3daa7c2f942c6ca8d4695a16bf9ac16b63615261c7982bc74f7ed20c", size = 5374751, upload-time = "2026-06-01T19:30:55.891Z" }, - { url = "https://files.pythonhosted.org/packages/77/6b/d817e1f8cc77aa055d37fba092e0febfdff40fe652d8d53d4cd7a86ad98d/debugpy-1.8.21-cp313-cp313-macosx_15_0_universal2.whl", hash = "sha256:13678151fc401e2d68c9880b91e28714f797d40422994572b24560ef80910a88", size = 2477398, upload-time = "2026-06-01T19:30:57.644Z" }, - { url = "https://files.pythonhosted.org/packages/48/57/412421516afc3055fa577516f00beec3d663f9b0ab330639547ae6c57720/debugpy-1.8.21-cp313-cp313-manylinux_2_34_x86_64.whl", hash = "sha256:ecbd158386c31ffe71d46f72d44d56e66331ab9b16cad649156d514368f23ab2", size = 3962096, upload-time = "2026-06-01T19:30:59.235Z" }, - { url = "https://files.pythonhosted.org/packages/c1/62/2c616337cf6ba7b07ebbc97f02c6c945a8e2f76b365e33ee809c32ee36d1/debugpy-1.8.21-cp313-cp313-win32.whl", hash = "sha256:2c2ae706dec41d99a9ca1f7ebc987a83e65578363be6f6b3ac9067504917fae1", size = 5336288, upload-time = "2026-06-01T19:31:00.79Z" }, - { url = "https://files.pythonhosted.org/packages/f8/99/9175103392f84c4b1bf7622888cdc68da07f0ff7d9e581266428f6776033/debugpy-1.8.21-cp313-cp313-win_amd64.whl", hash = "sha256:aa648733047443eb1d07682c4ef287d36a54507b643ffdf38b09a3ef002c72a0", size = 5376567, upload-time = "2026-06-01T19:31:02.56Z" }, - { url = "https://files.pythonhosted.org/packages/ce/3d/f4bbb323a548bfab2af3d6b4ffd9bf22636e55956a1285d317a1de643aad/debugpy-1.8.21-cp314-cp314-macosx_15_0_universal2.whl", hash = "sha256:9bb2a685287a2ac9b181cde89edcec64845cb51de7faaa75badb9a698bc24782", size = 2477209, upload-time = "2026-06-01T19:31:04.157Z" }, - { url = "https://files.pythonhosted.org/packages/8c/2d/6e7ec524984a1702777868de49a4c53202bddac2a432a76a093469587750/debugpy-1.8.21-cp314-cp314-manylinux_2_34_x86_64.whl", hash = "sha256:3d6922439bf33fd38a3e2c447869ebc7b97da5cd3d329ff1ef9bc06c4903437e", size = 3927115, upload-time = "2026-06-01T19:31:05.863Z" }, - { url = "https://files.pythonhosted.org/packages/97/47/d1aa6d64005a98a9144647d99306b419396f9ad7bf1d73c119e17a81fb4d/debugpy-1.8.21-cp314-cp314-win32.whl", hash = "sha256:15d4963bd5ffa48f0da0947fd06757fa7621945048a14ad7705431566d3c0e7c", size = 5336724, upload-time = "2026-06-01T19:31:07.711Z" }, - { url = "https://files.pythonhosted.org/packages/5f/67/b905b90d163af11878c1af8abafa4a25206335e112e284e413454543a6da/debugpy-1.8.21-cp314-cp314-win_amd64.whl", hash = "sha256:fe0744a12353406de0ae8ccff0d0a4a666f00801a3db8fd04e7a5f761cd520e8", size = 5373803, upload-time = "2026-06-01T19:31:09.469Z" }, - { url = "https://files.pythonhosted.org/packages/95/51/67e7cf11a53e40694f720457d5b3a1cdaaa3d5a9a633e482f225456b93ff/debugpy-1.8.21-py2.py3-none-any.whl", hash = "sha256:b1e37d333663c8851516a47364ef473da127f9caebe4417e6df6f5825a7e9a92", size = 5352888, upload-time = "2026-06-01T19:31:25.186Z" }, -] - -[[package]] -name = "decorator" -version = "5.1.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/66/0c/8d907af351aa16b42caae42f9d6aa37b900c67308052d10fdce809f8d952/decorator-5.1.1.tar.gz", hash = "sha256:637996211036b6385ef91435e4fae22989472f9d571faba8927ba8253acbc330", size = 35016, upload-time = "2022-01-07T08:20:05.666Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d5/50/83c593b07763e1161326b3b8c6686f0f4b0f24d5526546bee538c89837d6/decorator-5.1.1-py3-none-any.whl", hash = "sha256:b8c3f85900b9dc423225913c5aace94729fe1fa9763b38939a95226f02d37186", size = 9073, upload-time = "2022-01-07T08:20:03.734Z" }, -] - -[[package]] -name = "defusedxml" -version = "0.7.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/0f/d5/c66da9b79e5bdb124974bfe172b4daf3c984ebd9c2a06e2b8a4dc7331c72/defusedxml-0.7.1.tar.gz", hash = "sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69", size = 75520, upload-time = "2021-03-08T10:59:26.269Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/07/6c/aa3f2f849e01cb6a001cd8554a88d4c77c5c1a31c95bdf1cf9301e6d9ef4/defusedxml-0.7.1-py2.py3-none-any.whl", hash = "sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61", size = 25604, upload-time = "2021-03-08T10:59:24.45Z" }, -] - [[package]] name = "deprecated" version = "1.2.18" @@ -523,24 +407,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/02/cc/b7e31358aac6ed1ef2bb790a9746ac2c69bcb3c8588b41616914eb106eaf/exceptiongroup-1.2.2-py3-none-any.whl", hash = "sha256:3111b9d131c238bec2f8f516e123e14ba243563fb135d3fe885990585aa7795b", size = 16453, upload-time = "2024-07-12T22:25:58.476Z" }, ] -[[package]] -name = "executing" -version = "2.1.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/8c/e3/7d45f492c2c4a0e8e0fad57d081a7c8a0286cdd86372b070cca1ec0caa1e/executing-2.1.0.tar.gz", hash = "sha256:8ea27ddd260da8150fa5a708269c4a10e76161e2496ec3e587da9e3c0fe4b9ab", size = 977485, upload-time = "2024-09-01T12:37:35.708Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b5/fd/afcd0496feca3276f509df3dbd5dae726fcc756f1a08d9e25abe1733f962/executing-2.1.0-py2.py3-none-any.whl", hash = "sha256:8d63781349375b5ebccc3142f4b30350c0cd9c79f921cde38be2be4637e98eaf", size = 25805, upload-time = "2024-09-01T12:37:33.007Z" }, -] - -[[package]] -name = "fastjsonschema" -version = "2.21.2" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/20/b5/23b216d9d985a956623b6bd12d4086b60f0059b27799f23016af04a74ea1/fastjsonschema-2.21.2.tar.gz", hash = "sha256:b1eb43748041c880796cd077f1a07c3d94e93ae84bba5ed36800a33554ae05de", size = 374130, upload-time = "2025-08-14T18:49:36.666Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/cb/a8/20d0723294217e47de6d9e2e40fd4a9d2f7c4b6ef974babd482a59743694/fastjsonschema-2.21.2-py3-none-any.whl", hash = "sha256:1c797122d0a86c5cace2e54bf4e819c36223b552017172f32c5c024a6b77e463", size = 24024, upload-time = "2025-08-14T18:49:34.776Z" }, -] - [[package]] name = "filelock" version = "3.18.0" @@ -598,64 +464,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ef/a6/62565a6e1cf69e10f5727360368e451d4b7f58beeac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl", hash = "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374", size = 5892, upload-time = "2023-01-07T11:08:09.864Z" }, ] -[[package]] -name = "ipykernel" -version = "7.2.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "appnope", marker = "sys_platform == 'darwin'" }, - { name = "comm" }, - { name = "debugpy" }, - { name = "ipython" }, - { name = "jupyter-client" }, - { name = "jupyter-core" }, - { name = "matplotlib-inline" }, - { name = "nest-asyncio" }, - { name = "packaging" }, - { name = "psutil" }, - { name = "pyzmq" }, - { name = "tornado" }, - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/ca/8d/b68b728e2d06b9e0051019640a40a9eb7a88fcd82c2e1b5ce70bef5ff044/ipykernel-7.2.0.tar.gz", hash = "sha256:18ed160b6dee2cbb16e5f3575858bc19d8f1fe6046a9a680c708494ce31d909e", size = 176046, upload-time = "2026-02-06T16:43:27.403Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/82/b9/e73d5d9f405cba7706c539aa8b311b49d4c2f3d698d9c12f815231169c71/ipykernel-7.2.0-py3-none-any.whl", hash = "sha256:3bbd4420d2b3cc105cbdf3756bfc04500b1e52f090a90716851f3916c62e1661", size = 118788, upload-time = "2026-02-06T16:43:25.149Z" }, -] - -[[package]] -name = "ipython" -version = "8.31.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "colorama", marker = "sys_platform == 'win32'" }, - { name = "decorator" }, - { name = "exceptiongroup", marker = "python_full_version < '3.11'" }, - { name = "jedi" }, - { name = "matplotlib-inline" }, - { name = "pexpect", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" }, - { name = "prompt-toolkit" }, - { name = "pygments" }, - { name = "stack-data" }, - { name = "traitlets" }, - { name = "typing-extensions", marker = "python_full_version < '3.12'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/01/35/6f90fdddff7a08b7b715fccbd2427b5212c9525cd043d26fdc45bee0708d/ipython-8.31.0.tar.gz", hash = "sha256:b6a2274606bec6166405ff05e54932ed6e5cfecaca1fc05f2cacde7bb074d70b", size = 5501011, upload-time = "2024-12-20T12:34:22.61Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/04/60/d0feb6b6d9fe4ab89fe8fe5b47cbf6cd936bfd9f1e7ffa9d0015425aeed6/ipython-8.31.0-py3-none-any.whl", hash = "sha256:46ec58f8d3d076a61d128fe517a51eb730e3aaf0c184ea8c17d16e366660c6a6", size = 821583, upload-time = "2024-12-20T12:34:17.106Z" }, -] - -[[package]] -name = "jedi" -version = "0.19.2" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "parso" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/72/3a/79a912fbd4d8dd6fbb02bf69afd3bb72cf0c729bb3063c6f4498603db17a/jedi-0.19.2.tar.gz", hash = "sha256:4770dc3de41bde3966b02eb84fbcf557fb33cce26ad23da12c742fb50ecb11f0", size = 1231287, upload-time = "2024-11-11T01:41:42.873Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/c0/5a/9cac0c82afec3d09ccd97c8b6502d48f165f9124db81b4bcb90b4af974ee/jedi-0.19.2-py2.py3-none-any.whl", hash = "sha256:a8ef22bde8490f57fe5c7681a3c83cb58874daf72b4784de3cce5b6ef6edb5b9", size = 1572278, upload-time = "2024-11-11T01:41:40.175Z" }, -] - [[package]] name = "jinja2" version = "3.1.5" @@ -668,89 +476,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/bd/0f/2ba5fbcd631e3e88689309dbe978c5769e883e4b84ebfe7da30b43275c5a/jinja2-3.1.5-py3-none-any.whl", hash = "sha256:aba0f4dc9ed8013c424088f68a5c226f7d6097ed89b246d7749c2ec4175c6adb", size = 134596, upload-time = "2024-12-21T18:30:19.133Z" }, ] -[[package]] -name = "jsonschema" -version = "4.26.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "attrs" }, - { name = "jsonschema-specifications" }, - { name = "referencing" }, - { name = "rpds-py", version = "0.30.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "rpds-py", version = "2026.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/b3/fc/e067678238fa451312d4c62bf6e6cf5ec56375422aee02f9cb5f909b3047/jsonschema-4.26.0.tar.gz", hash = "sha256:0c26707e2efad8aa1bfc5b7ce170f3fccc2e4918ff85989ba9ffa9facb2be326", size = 366583, upload-time = "2026-01-07T13:41:07.246Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/69/90/f63fb5873511e014207a475e2bb4e8b2e570d655b00ac19a9a0ca0a385ee/jsonschema-4.26.0-py3-none-any.whl", hash = "sha256:d489f15263b8d200f8387e64b4c3a75f06629559fb73deb8fdfb525f2dab50ce", size = 90630, upload-time = "2026-01-07T13:41:05.306Z" }, -] - -[[package]] -name = "jsonschema-specifications" -version = "2025.9.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "referencing" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/19/74/a633ee74eb36c44aa6d1095e7cc5569bebf04342ee146178e2d36600708b/jsonschema_specifications-2025.9.1.tar.gz", hash = "sha256:b540987f239e745613c7a9176f3edb72b832a4ac465cf02712288397832b5e8d", size = 32855, upload-time = "2025-09-08T01:34:59.186Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/41/45/1a4ed80516f02155c51f51e8cedb3c1902296743db0bbc66608a0db2814f/jsonschema_specifications-2025.9.1-py3-none-any.whl", hash = "sha256:98802fee3a11ee76ecaca44429fda8a41bff98b00a0f2838151b113f210cc6fe", size = 18437, upload-time = "2025-09-08T01:34:57.871Z" }, -] - -[[package]] -name = "jupyter-client" -version = "8.8.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "jupyter-core" }, - { name = "python-dateutil" }, - { name = "pyzmq" }, - { name = "tornado" }, - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/05/e4/ba649102a3bc3fbca54e7239fb924fd434c766f855693d86de0b1f2bec81/jupyter_client-8.8.0.tar.gz", hash = "sha256:d556811419a4f2d96c869af34e854e3f059b7cc2d6d01a9cd9c85c267691be3e", size = 348020, upload-time = "2026-01-08T13:55:47.938Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/2d/0b/ceb7694d864abc0a047649aec263878acb9f792e1fec3e676f22dc9015e3/jupyter_client-8.8.0-py3-none-any.whl", hash = "sha256:f93a5b99c5e23a507b773d3a1136bd6e16c67883ccdbd9a829b0bbdb98cd7d7a", size = 107371, upload-time = "2026-01-08T13:55:45.562Z" }, -] - -[[package]] -name = "jupyter-core" -version = "5.9.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "platformdirs" }, - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/02/49/9d1284d0dc65e2c757b74c6687b6d319b02f822ad039e5c512df9194d9dd/jupyter_core-5.9.1.tar.gz", hash = "sha256:4d09aaff303b9566c3ce657f580bd089ff5c91f5f89cf7d8846c3cdf465b5508", size = 89814, upload-time = "2025-10-16T19:19:18.444Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e7/e7/80988e32bf6f73919a113473a604f5a8f09094de312b9d52b79c2df7612b/jupyter_core-5.9.1-py3-none-any.whl", hash = "sha256:ebf87fdc6073d142e114c72c9e29a9d7ca03fad818c5d300ce2adc1fb0743407", size = 29032, upload-time = "2025-10-16T19:19:16.783Z" }, -] - -[[package]] -name = "jupyterlab-pygments" -version = "0.3.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/90/51/9187be60d989df97f5f0aba133fa54e7300f17616e065d1ada7d7646b6d6/jupyterlab_pygments-0.3.0.tar.gz", hash = "sha256:721aca4d9029252b11cfa9d185e5b5af4d54772bb8072f9b7036f4170054d35d", size = 512900, upload-time = "2023-11-23T09:26:37.44Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b1/dd/ead9d8ea85bf202d90cc513b533f9c363121c7792674f78e0d8a854b63b4/jupyterlab_pygments-0.3.0-py3-none-any.whl", hash = "sha256:841a89020971da1d8693f1a99997aefc5dc424bb1b251fd6322462a1b8842780", size = 15884, upload-time = "2023-11-23T09:26:34.325Z" }, -] - -[[package]] -name = "jupytext" -version = "1.19.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "markdown-it-py" }, - { name = "mdit-py-plugins" }, - { name = "nbformat" }, - { name = "packaging" }, - { name = "pyyaml" }, - { name = "tomli", marker = "python_full_version < '3.11'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/ef/2d/15624c3d9440d85a280ff13d2d23afd989802f25470ac59932f4fef6f0c6/jupytext-1.19.3.tar.gz", hash = "sha256:713c3ed4441afe0f31474d28ea2e6b61a268c04c40fd78e5ccfd7f7ac9e9f766", size = 4305350, upload-time = "2026-05-17T09:09:29.294Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/aa/ec/d9be3bd1db141e76b2f525c265f70e66edd30a51a3307d8edf0ef1909c54/jupytext-1.19.3-py3-none-any.whl", hash = "sha256:acf75492f80895ad8e664fd8db1708b617008dd0e71c341a1abc3d0d07310ed0", size = 170579, upload-time = "2026-05-17T09:09:27.478Z" }, -] - [[package]] name = "markdown" version = "3.10.2" @@ -761,15 +486,20 @@ wheels = [ ] [[package]] -name = "markdown-it-py" -version = "3.0.0" +name = "markdown-exec" +version = "1.12.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "mdurl" }, + { name = "pymdown-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/38/71/3b932df36c1a044d397a1f92d1cf91ee0a503d91e470cbd670aa66b07ed0/markdown-it-py-3.0.0.tar.gz", hash = "sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb", size = 74596, upload-time = "2023-06-03T06:41:14.443Z" } +sdist = { url = "https://files.pythonhosted.org/packages/96/73/1f20927d075c83c0e2bc814d3b8f9bd254d919069f78c5423224b4407944/markdown_exec-1.12.1.tar.gz", hash = "sha256:eee8ba0df99a5400092eeda80212ba3968f3cbbf3a33f86f1cd25161538e6534", size = 78105, upload-time = "2025-11-11T19:25:05.44Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/42/d7/1ec15b46af6af88f19b8e5ffea08fa375d433c998b8a7639e76935c14f1f/markdown_it_py-3.0.0-py3-none-any.whl", hash = "sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1", size = 87528, upload-time = "2023-06-03T06:41:11.019Z" }, + { url = "https://files.pythonhosted.org/packages/ea/22/7b684ddb01b423b79eaba9726954bbe559540d510abc7a72a84d8eee1b26/markdown_exec-1.12.1-py3-none-any.whl", hash = "sha256:a645dce411fee297f5b4a4169c245ec51e20061d5b71e225bef006e87f3e465f", size = 38046, upload-time = "2025-11-11T19:25:03.878Z" }, +] + +[package.optional-dependencies] +ansi = [ + { name = "pygments-ansi-color" }, ] [[package]] @@ -830,18 +560,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/4f/65/6079a46068dfceaeabb5dcad6d674f5f5c61a6fa5673746f42a9f4c233b3/MarkupSafe-3.0.2-cp313-cp313t-win_amd64.whl", hash = "sha256:e444a31f8db13eb18ada366ab3cf45fd4b31e4db1236a4448f68778c1d1a5a2f", size = 15739, upload-time = "2024-10-18T15:21:42.784Z" }, ] -[[package]] -name = "matplotlib-inline" -version = "0.1.7" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/99/5b/a36a337438a14116b16480db471ad061c36c3694df7c2084a0da7ba538b7/matplotlib_inline-0.1.7.tar.gz", hash = "sha256:8423b23ec666be3d16e16b60bdd8ac4e86e840ebd1dd11a30b9f117f2fa0ab90", size = 8159, upload-time = "2024-04-15T13:44:44.803Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/8f/8e/9ad090d3553c280a8060fbf6e24dc1c0c29704ee7d1c372f0c174aa59285/matplotlib_inline-0.1.7-py3-none-any.whl", hash = "sha256:df192d39a4ff8f21b1895d72e6a13f5fcc5099f00fa84384e0ea28c2cc0653ca", size = 9899, upload-time = "2024-04-15T13:44:43.265Z" }, -] - [[package]] name = "maturin" version = "1.13.3" @@ -866,27 +584,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/6c/93/e32e79333f0902ba292b996f504f5f06be59587f7d02ab8d5ed1e3066445/maturin-1.13.3-py3-none-win_arm64.whl", hash = "sha256:2389fe92d017cea9d94e521fa0175314a4c52f79a1057b901fbc9f8686ef7d0b", size = 9706562, upload-time = "2026-05-11T07:43:31.743Z" }, ] -[[package]] -name = "mdit-py-plugins" -version = "0.4.2" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "markdown-it-py" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/19/03/a2ecab526543b152300717cf232bb4bb8605b6edb946c845016fa9c9c9fd/mdit_py_plugins-0.4.2.tar.gz", hash = "sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5", size = 43542, upload-time = "2024-09-09T20:27:49.564Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a7/f7/7782a043553ee469c1ff49cfa1cdace2d6bf99a1f333cf38676b3ddf30da/mdit_py_plugins-0.4.2-py3-none-any.whl", hash = "sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636", size = 55316, upload-time = "2024-09-09T20:27:48.397Z" }, -] - -[[package]] -name = "mdurl" -version = "0.1.2" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, -] - [[package]] name = "mergedeep" version = "1.3.4" @@ -896,18 +593,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/2c/19/04f9b178c2d8a15b076c8b5140708fa6ffc5601fb6f1e975537072df5b2a/mergedeep-1.3.4-py3-none-any.whl", hash = "sha256:70775750742b25c0d8f36c55aed03d24c3384d17c951b3175d898bd778ef0307", size = 6354, upload-time = "2021-02-05T18:55:29.583Z" }, ] -[[package]] -name = "mistune" -version = "3.2.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "typing-extensions", marker = "python_full_version < '3.11'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/ca/84/620cc3f7e3adf6f5067e10f4dbae71295d8f9e16d5d3f9ef97c40f2f592c/mistune-3.2.1.tar.gz", hash = "sha256:7c8e5501d38bac1582e067e46c8343f17d57ea1aaa735823f3aba1fd59c88a28", size = 98003, upload-time = "2026-05-03T14:33:22.312Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/2a/7f/a946aa4f8752b37102b41e64dca18a1976ac705c3a0d1dfe74d820a02552/mistune-3.2.1-py3-none-any.whl", hash = "sha256:78cdb0ba5e938053ccf63651b352508d2efa9411dc8810bfb05f2dc5140c0048", size = 53749, upload-time = "2026-05-03T14:33:20.551Z" }, -] - [[package]] name = "mkdocs" version = "1.6.1" @@ -960,23 +645,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/88/29/744136411e785c4b0b744d5413e56555265939ab3a104c6a4b719dad33fd/mkdocs_get_deps-0.2.2-py3-none-any.whl", hash = "sha256:e7878cbeac04860b8b5e0ca31d3abad3df9411a75a32cde82f8e44b6c16ff650", size = 9555, upload-time = "2026-03-10T02:46:32.256Z" }, ] -[[package]] -name = "mkdocs-jupyter" -version = "0.26.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "ipykernel" }, - { name = "jupytext" }, - { name = "mkdocs" }, - { name = "mkdocs-material" }, - { name = "nbconvert" }, - { name = "pygments" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/00/aa/f8d15409a9a3112486994a80d5a975694c7d145c4f8b5b484aeb383420ef/mkdocs_jupyter-0.26.3.tar.gz", hash = "sha256:e1e8bd48a1b96542e84e3028e3066112bac7b94d95ab69f8b91305c84003ca26", size = 1628353, upload-time = "2026-04-17T18:56:31.517Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/13/95/cf3f7fe4910cf0365fa8ea0c731f4b8a624d97cd76ea777913ac8d0868e2/mkdocs_jupyter-0.26.3-py3-none-any.whl", hash = "sha256:cd6644fb578131157194d750fd4d10fc2fd8f1e84e00036ee62df3b5b4b84c82", size = 1459740, upload-time = "2026-04-17T18:56:30.031Z" }, -] - [[package]] name = "mkdocs-material" version = "9.7.6" @@ -1128,70 +796,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b9/e5/c740ea047b5ada76175327360d0406ae283159cb1745cbcb51443d90d53b/nanoarrow-0.8.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:8c5529abc4e75b7764ffc6d2fbabd0c676f75ca2ece71a8671c4724207cfb697", size = 591889, upload-time = "2026-02-10T03:33:58.891Z" }, ] -[[package]] -name = "nbclient" -version = "0.11.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "jupyter-client" }, - { name = "jupyter-core" }, - { name = "nbformat" }, - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/28/a5/b3bae4b590c0cbcada2c63a34f7580024e834a8ba213e949a2f906705787/nbclient-0.11.0.tar.gz", hash = "sha256:04a134a5b087f2c5887f228aca155db50169b8cd9334dee6942c8e927e56081a", size = 62535, upload-time = "2026-06-05T07:52:41.746Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/36/c9/94d73e5a01c5b926c3fa2496e97d7a8dc28ed5a77c0b2ed712f1a62e6694/nbclient-0.11.0-py3-none-any.whl", hash = "sha256:ef7fa0d59d6e1d41103933d8a445a18d5de860ca6b613b87b8574accdb3c2895", size = 25288, upload-time = "2026-06-05T07:52:40.115Z" }, -] - -[[package]] -name = "nbconvert" -version = "7.17.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "beautifulsoup4" }, - { name = "bleach", extra = ["css"] }, - { name = "defusedxml" }, - { name = "jinja2" }, - { name = "jupyter-core" }, - { name = "jupyterlab-pygments" }, - { name = "markupsafe" }, - { name = "mistune" }, - { name = "nbclient" }, - { name = "nbformat" }, - { name = "packaging" }, - { name = "pandocfilters" }, - { name = "pygments" }, - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/01/b1/708e53fe2e429c103c6e6e159106bcf0357ac41aa4c28772bd8402339051/nbconvert-7.17.1.tar.gz", hash = "sha256:34d0d0a7e73ce3cbab6c5aae8f4f468797280b01fd8bd2ca746da8569eddd7d2", size = 865311, upload-time = "2026-04-08T00:44:14.914Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/67/f8/bb0a9d5f46819c821dc1f004aa2cc29b1d91453297dbf5ff20470f00f193/nbconvert-7.17.1-py3-none-any.whl", hash = "sha256:aa85c087b435e7bf1ffd03319f658e285f2b89eccab33bc1ba7025495ab3e7c8", size = 261927, upload-time = "2026-04-08T00:44:12.845Z" }, -] - -[[package]] -name = "nbformat" -version = "5.10.4" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "fastjsonschema" }, - { name = "jsonschema" }, - { name = "jupyter-core" }, - { name = "traitlets" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/6d/fd/91545e604bc3dad7dca9ed03284086039b294c6b3d75c0d2fa45f9e9caf3/nbformat-5.10.4.tar.gz", hash = "sha256:322168b14f937a5d11362988ecac2a4952d3d8e3a2cbeb2319584631226d5b3a", size = 142749, upload-time = "2024-04-04T11:20:37.371Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a9/82/0340caa499416c78e5d8f5f05947ae4bc3cba53c9f038ab6e9ed964e22f1/nbformat-5.10.4-py3-none-any.whl", hash = "sha256:3b48d6c8fbca4b299bf3982ea7db1af21580e4fec269ad087b9e81588891200b", size = 78454, upload-time = "2024-04-04T11:20:34.895Z" }, -] - -[[package]] -name = "nest-asyncio" -version = "1.6.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/83/f8/51569ac65d696c8ecbee95938f89d4abf00f47d58d48f6fbabfe8f0baefe/nest_asyncio-1.6.0.tar.gz", hash = "sha256:6f172d5449aca15afd6c646851f4e31e02c598d553a667e38cafa997cfec55fe", size = 7418, upload-time = "2024-01-21T14:25:19.227Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a0/c4/c2971a3ba4c6103a3d10c4b0f24f461ddc027f0f09763220cf35ca1401b3/nest_asyncio-1.6.0-py3-none-any.whl", hash = "sha256:87af6efd6b5e897c81050477ef65c62e2b2f35d51703cae01aff2905b1852e1c", size = 5195, upload-time = "2024-01-21T14:25:17.223Z" }, -] - [[package]] name = "nodeenv" version = "1.9.1" @@ -1419,24 +1023,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ab/5f/b38085618b950b79d2d9164a711c52b10aefc0ae6833b96f626b7021b2ed/pandas-2.2.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:ad5b65698ab28ed8d7f18790a0dc58005c7629f227be9ecc1072aa74c0c1d43a", size = 13098436, upload-time = "2024-09-20T13:09:48.112Z" }, ] -[[package]] -name = "pandocfilters" -version = "1.5.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/70/6f/3dd4940bbe001c06a65f88e36bad298bc7a0de5036115639926b0c5c0458/pandocfilters-1.5.1.tar.gz", hash = "sha256:002b4a555ee4ebc03f8b66307e287fa492e4a77b4ea14d3f934328297bb4939e", size = 8454, upload-time = "2024-01-18T20:08:13.726Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/ef/af/4fbc8cab944db5d21b7e2a5b8e9211a03a79852b1157e2c102fcc61ac440/pandocfilters-1.5.1-py2.py3-none-any.whl", hash = "sha256:93be382804a9cdb0a7267585f157e5d1731bbe5545a85b268d6f5fe6232de2bc", size = 8663, upload-time = "2024-01-18T20:08:11.28Z" }, -] - -[[package]] -name = "parso" -version = "0.8.4" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/66/94/68e2e17afaa9169cf6412ab0f28623903be73d1b32e208d9e8e541bb086d/parso-0.8.4.tar.gz", hash = "sha256:eb3a7b58240fb99099a345571deecc0f9540ea5f4dd2fe14c2a99d6b281ab92d", size = 400609, upload-time = "2024-04-05T09:43:55.897Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/c6/ac/dac4a63f978e4dcb3c6d3a78c4d8e0192a113d288502a1216950c41b1027/parso-0.8.4-py2.py3-none-any.whl", hash = "sha256:a418670a20291dacd2dddc80c377c5c3791378ee1e8d12bffc35420643d43f18", size = 103650, upload-time = "2024-04-05T09:43:53.299Z" }, -] - [[package]] name = "pathspec" version = "1.1.1" @@ -1446,27 +1032,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f1/d9/7fb5aa316bc299258e68c73ba3bddbc499654a07f151cba08f6153988714/pathspec-1.1.1-py3-none-any.whl", hash = "sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189", size = 57328, upload-time = "2026-04-27T01:46:07.06Z" }, ] -[[package]] -name = "pexpect" -version = "4.9.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "ptyprocess" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/42/92/cc564bf6381ff43ce1f4d06852fc19a2f11d180f23dc32d9588bee2f149d/pexpect-4.9.0.tar.gz", hash = "sha256:ee7d41123f3c9911050ea2c2dac107568dc43b2d3b0c7557a33212c398ead30f", size = 166450, upload-time = "2023-11-25T09:07:26.339Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/9e/c3/059298687310d527a58bb01f3b1965787ee3b40dce76752eda8b44e9a2c5/pexpect-4.9.0-py2.py3-none-any.whl", hash = "sha256:7236d1e080e4936be2dc3e326cec0af72acf9212a7e1d060210e70a47e253523", size = 63772, upload-time = "2023-11-25T06:56:14.81Z" }, -] - -[[package]] -name = "pickleshare" -version = "0.7.5" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d8/b6/df3c1c9b616e9c0edbc4fbab6ddd09df9535849c64ba51fcb6531c32d4d8/pickleshare-0.7.5.tar.gz", hash = "sha256:87683d47965c1da65cdacaf31c8441d12b8044cdec9aca500cd78fc2c683afca", size = 6161, upload-time = "2018-09-25T19:17:37.249Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/9a/41/220f49aaea88bc6fa6cba8d05ecf24676326156c23b991e80b3f2fc24c77/pickleshare-0.7.5-py2.py3-none-any.whl", hash = "sha256:9649af414d74d4df115d5d718f82acb59c9d418196b7b4290ed47a12ce62df56", size = 6877, upload-time = "2018-09-25T19:17:35.817Z" }, -] - [[package]] name = "platformdirs" version = "4.3.8" @@ -1501,18 +1066,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/5b/a5/987a405322d78a73b66e39e4a90e4ef156fd7141bf71df987e50717c321b/pre_commit-4.3.0-py2.py3-none-any.whl", hash = "sha256:2b0747ad7e6e967169136edffee14c16e148a778a54e4f967921aa1ebf2308d8", size = 220965, upload-time = "2025-08-09T18:56:13.192Z" }, ] -[[package]] -name = "prompt-toolkit" -version = "3.0.48" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "wcwidth" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/2d/4f/feb5e137aff82f7c7f3248267b97451da3644f6cdc218edfe549fb354127/prompt_toolkit-3.0.48.tar.gz", hash = "sha256:d6623ab0477a80df74e646bdbc93621143f5caf104206aa29294d53de1a03d90", size = 424684, upload-time = "2024-09-25T10:20:57.609Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a9/6a/fd08d94654f7e67c52ca30523a178b3f8ccc4237fce4be90d39c938a831a/prompt_toolkit-3.0.48-py3-none-any.whl", hash = "sha256:f49a827f90062e411f1ce1f854f2aedb3c23353244f8108b89283587397ac10e", size = 386595, upload-time = "2024-09-25T10:20:53.932Z" }, -] - [[package]] name = "properdocs" version = "1.6.7" @@ -1536,52 +1089,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/bd/4d/fc923f5c85318ee8cc903566dc4e0ebe41b2dfc1d2ecf5546db232397ed6/properdocs-1.6.7-py3-none-any.whl", hash = "sha256:6fa0cfa2e01bf338f684892c8a506cf70ea88ae7f3479c933b6fa20168101cbd", size = 225406, upload-time = "2026-03-20T20:07:46.875Z" }, ] -[[package]] -name = "psutil" -version = "7.2.2" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/aa/c6/d1ddf4abb55e93cebc4f2ed8b5d6dbad109ecb8d63748dd2b20ab5e57ebe/psutil-7.2.2.tar.gz", hash = "sha256:0746f5f8d406af344fd547f1c8daa5f5c33dbc293bb8d6a16d80b4bb88f59372", size = 493740, upload-time = "2026-01-28T18:14:54.428Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/51/08/510cbdb69c25a96f4ae523f733cdc963ae654904e8db864c07585ef99875/psutil-7.2.2-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:2edccc433cbfa046b980b0df0171cd25bcaeb3a68fe9022db0979e7aa74a826b", size = 130595, upload-time = "2026-01-28T18:14:57.293Z" }, - { url = "https://files.pythonhosted.org/packages/d6/f5/97baea3fe7a5a9af7436301f85490905379b1c6f2dd51fe3ecf24b4c5fbf/psutil-7.2.2-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:e78c8603dcd9a04c7364f1a3e670cea95d51ee865e4efb3556a3a63adef958ea", size = 131082, upload-time = "2026-01-28T18:14:59.732Z" }, - { url = "https://files.pythonhosted.org/packages/37/d6/246513fbf9fa174af531f28412297dd05241d97a75911ac8febefa1a53c6/psutil-7.2.2-cp313-cp313t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1a571f2330c966c62aeda00dd24620425d4b0cc86881c89861fbc04549e5dc63", size = 181476, upload-time = "2026-01-28T18:15:01.884Z" }, - { url = "https://files.pythonhosted.org/packages/b8/b5/9182c9af3836cca61696dabe4fd1304e17bc56cb62f17439e1154f225dd3/psutil-7.2.2-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:917e891983ca3c1887b4ef36447b1e0873e70c933afc831c6b6da078ba474312", size = 184062, upload-time = "2026-01-28T18:15:04.436Z" }, - { url = "https://files.pythonhosted.org/packages/16/ba/0756dca669f5a9300d0cbcbfae9a4c30e446dfc7440ffe43ded5724bfd93/psutil-7.2.2-cp313-cp313t-win_amd64.whl", hash = "sha256:ab486563df44c17f5173621c7b198955bd6b613fb87c71c161f827d3fb149a9b", size = 139893, upload-time = "2026-01-28T18:15:06.378Z" }, - { url = "https://files.pythonhosted.org/packages/1c/61/8fa0e26f33623b49949346de05ec1ddaad02ed8ba64af45f40a147dbfa97/psutil-7.2.2-cp313-cp313t-win_arm64.whl", hash = "sha256:ae0aefdd8796a7737eccea863f80f81e468a1e4cf14d926bd9b6f5f2d5f90ca9", size = 135589, upload-time = "2026-01-28T18:15:08.03Z" }, - { url = "https://files.pythonhosted.org/packages/81/69/ef179ab5ca24f32acc1dac0c247fd6a13b501fd5534dbae0e05a1c48b66d/psutil-7.2.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:eed63d3b4d62449571547b60578c5b2c4bcccc5387148db46e0c2313dad0ee00", size = 130664, upload-time = "2026-01-28T18:15:09.469Z" }, - { url = "https://files.pythonhosted.org/packages/7b/64/665248b557a236d3fa9efc378d60d95ef56dd0a490c2cd37dafc7660d4a9/psutil-7.2.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7b6d09433a10592ce39b13d7be5a54fbac1d1228ed29abc880fb23df7cb694c9", size = 131087, upload-time = "2026-01-28T18:15:11.724Z" }, - { url = "https://files.pythonhosted.org/packages/d5/2e/e6782744700d6759ebce3043dcfa661fb61e2fb752b91cdeae9af12c2178/psutil-7.2.2-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1fa4ecf83bcdf6e6c8f4449aff98eefb5d0604bf88cb883d7da3d8d2d909546a", size = 182383, upload-time = "2026-01-28T18:15:13.445Z" }, - { url = "https://files.pythonhosted.org/packages/57/49/0a41cefd10cb7505cdc04dab3eacf24c0c2cb158a998b8c7b1d27ee2c1f5/psutil-7.2.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e452c464a02e7dc7822a05d25db4cde564444a67e58539a00f929c51eddda0cf", size = 185210, upload-time = "2026-01-28T18:15:16.002Z" }, - { url = "https://files.pythonhosted.org/packages/dd/2c/ff9bfb544f283ba5f83ba725a3c5fec6d6b10b8f27ac1dc641c473dc390d/psutil-7.2.2-cp314-cp314t-win_amd64.whl", hash = "sha256:c7663d4e37f13e884d13994247449e9f8f574bc4655d509c3b95e9ec9e2b9dc1", size = 141228, upload-time = "2026-01-28T18:15:18.385Z" }, - { url = "https://files.pythonhosted.org/packages/f2/fc/f8d9c31db14fcec13748d373e668bc3bed94d9077dbc17fb0eebc073233c/psutil-7.2.2-cp314-cp314t-win_arm64.whl", hash = "sha256:11fe5a4f613759764e79c65cf11ebdf26e33d6dd34336f8a337aa2996d71c841", size = 136284, upload-time = "2026-01-28T18:15:19.912Z" }, - { url = "https://files.pythonhosted.org/packages/e7/36/5ee6e05c9bd427237b11b3937ad82bb8ad2752d72c6969314590dd0c2f6e/psutil-7.2.2-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:ed0cace939114f62738d808fdcecd4c869222507e266e574799e9c0faa17d486", size = 129090, upload-time = "2026-01-28T18:15:22.168Z" }, - { url = "https://files.pythonhosted.org/packages/80/c4/f5af4c1ca8c1eeb2e92ccca14ce8effdeec651d5ab6053c589b074eda6e1/psutil-7.2.2-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:1a7b04c10f32cc88ab39cbf606e117fd74721c831c98a27dc04578deb0c16979", size = 129859, upload-time = "2026-01-28T18:15:23.795Z" }, - { url = "https://files.pythonhosted.org/packages/b5/70/5d8df3b09e25bce090399cf48e452d25c935ab72dad19406c77f4e828045/psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:076a2d2f923fd4821644f5ba89f059523da90dc9014e85f8e45a5774ca5bc6f9", size = 155560, upload-time = "2026-01-28T18:15:25.976Z" }, - { url = "https://files.pythonhosted.org/packages/63/65/37648c0c158dc222aba51c089eb3bdfa238e621674dc42d48706e639204f/psutil-7.2.2-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b0726cecd84f9474419d67252add4ac0cd9811b04d61123054b9fb6f57df6e9e", size = 156997, upload-time = "2026-01-28T18:15:27.794Z" }, - { url = "https://files.pythonhosted.org/packages/8e/13/125093eadae863ce03c6ffdbae9929430d116a246ef69866dad94da3bfbc/psutil-7.2.2-cp36-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:fd04ef36b4a6d599bbdb225dd1d3f51e00105f6d48a28f006da7f9822f2606d8", size = 148972, upload-time = "2026-01-28T18:15:29.342Z" }, - { url = "https://files.pythonhosted.org/packages/04/78/0acd37ca84ce3ddffaa92ef0f571e073faa6d8ff1f0559ab1272188ea2be/psutil-7.2.2-cp36-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b58fabe35e80b264a4e3bb23e6b96f9e45a3df7fb7eed419ac0e5947c61e47cc", size = 148266, upload-time = "2026-01-28T18:15:31.597Z" }, - { url = "https://files.pythonhosted.org/packages/b4/90/e2159492b5426be0c1fef7acba807a03511f97c5f86b3caeda6ad92351a7/psutil-7.2.2-cp37-abi3-win_amd64.whl", hash = "sha256:eb7e81434c8d223ec4a219b5fc1c47d0417b12be7ea866e24fb5ad6e84b3d988", size = 137737, upload-time = "2026-01-28T18:15:33.849Z" }, - { url = "https://files.pythonhosted.org/packages/8c/c7/7bb2e321574b10df20cbde462a94e2b71d05f9bbda251ef27d104668306a/psutil-7.2.2-cp37-abi3-win_arm64.whl", hash = "sha256:8c233660f575a5a89e6d4cb65d9f938126312bca76d8fe087b947b3a1aaac9ee", size = 134617, upload-time = "2026-01-28T18:15:36.514Z" }, -] - -[[package]] -name = "ptyprocess" -version = "0.7.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/20/e5/16ff212c1e452235a90aeb09066144d0c5a6a8c0834397e03f5224495c4e/ptyprocess-0.7.0.tar.gz", hash = "sha256:5c5d0a3b48ceee0b48485e0c26037c0acd7d29765ca3fbb5cb3831d347423220", size = 70762, upload-time = "2020-12-28T15:15:30.155Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/22/a6/858897256d0deac81a172289110f31629fc4cee19b6f01283303e18c8db3/ptyprocess-0.7.0-py2.py3-none-any.whl", hash = "sha256:4b41f3967fce3af57cc7e94b888626c18bf37a083e3651ca8feeb66d492fef35", size = 13993, upload-time = "2020-12-28T15:15:28.35Z" }, -] - -[[package]] -name = "pure-eval" -version = "0.2.3" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/cd/05/0a34433a064256a578f1783a10da6df098ceaa4a57bbeaa96a6c0352786b/pure_eval-0.2.3.tar.gz", hash = "sha256:5f4e983f40564c576c7c8635ae88db5956bb2229d7e9237d03b3c0b0190eaf42", size = 19752, upload-time = "2024-07-21T12:58:21.801Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/8e/37/efad0257dc6e593a18957422533ff0f87ede7c9c6ea010a2177d738fb82f/pure_eval-0.2.3-py3-none-any.whl", hash = "sha256:1db8e35b67b3d218d818ae653e27f06c3aa420901fa7b081ca98cbedc874e0d0", size = 11842, upload-time = "2024-07-21T12:58:20.04Z" }, -] - [[package]] name = "pyarrow" version = "22.0.0" @@ -1674,6 +1181,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/8a/0b/9fcc47d19c48b59121088dd6da2488a49d5f72dacf8262e2790a1d2c7d15/pygments-2.19.1-py3-none-any.whl", hash = "sha256:9ea1544ad55cecf4b8242fab6dd35a93bbce657034b0611ee383099054ab6d8c", size = 1225293, upload-time = "2025-01-06T17:26:25.553Z" }, ] +[[package]] +name = "pygments-ansi-color" +version = "0.3.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pygments" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/50/f9/7f417aaee98a74b4f757f2b72971245181fcf25d824d2e7a190345669eaf/pygments-ansi-color-0.3.0.tar.gz", hash = "sha256:7018954cf5b11d1e734383a1bafab5af613213f246109417fee3f76da26d5431", size = 7317, upload-time = "2023-05-18T22:44:35.792Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e6/17/8306a0bcd8c88d7761c2e73e831b0be026cd6873ce1f12beb3b4c9a03ffa/pygments_ansi_color-0.3.0-py3-none-any.whl", hash = "sha256:7eb063feaecadad9d4d1fd3474cbfeadf3486b64f760a8f2a00fc25392180aba", size = 10242, upload-time = "2023-05-18T22:44:34.287Z" }, +] + [[package]] name = "pyjwt" version = "2.10.1" @@ -1859,94 +1378,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/04/11/432f32f8097b03e3cd5fe57e88efb685d964e2e5178a48ed61e841f7fdce/pyyaml_env_tag-1.1-py3-none-any.whl", hash = "sha256:17109e1a528561e32f026364712fee1264bc2ea6715120891174ed1b980d2e04", size = 4722, upload-time = "2025-05-13T15:23:59.629Z" }, ] -[[package]] -name = "pyzmq" -version = "27.1.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "cffi", marker = "implementation_name == 'pypy'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/04/0b/3c9baedbdf613ecaa7aa07027780b8867f57b6293b6ee50de316c9f3222b/pyzmq-27.1.0.tar.gz", hash = "sha256:ac0765e3d44455adb6ddbf4417dcce460fc40a05978c08efdf2948072f6db540", size = 281750, upload-time = "2025-09-08T23:10:18.157Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/67/b9/52aa9ec2867528b54f1e60846728d8b4d84726630874fee3a91e66c7df81/pyzmq-27.1.0-cp310-cp310-macosx_10_15_universal2.whl", hash = "sha256:508e23ec9bc44c0005c4946ea013d9317ae00ac67778bd47519fdf5a0e930ff4", size = 1329850, upload-time = "2025-09-08T23:07:26.274Z" }, - { url = "https://files.pythonhosted.org/packages/99/64/5653e7b7425b169f994835a2b2abf9486264401fdef18df91ddae47ce2cc/pyzmq-27.1.0-cp310-cp310-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:507b6f430bdcf0ee48c0d30e734ea89ce5567fd7b8a0f0044a369c176aa44556", size = 906380, upload-time = "2025-09-08T23:07:29.78Z" }, - { url = "https://files.pythonhosted.org/packages/73/78/7d713284dbe022f6440e391bd1f3c48d9185673878034cfb3939cdf333b2/pyzmq-27.1.0-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bf7b38f9fd7b81cb6d9391b2946382c8237fd814075c6aa9c3b746d53076023b", size = 666421, upload-time = "2025-09-08T23:07:31.263Z" }, - { url = "https://files.pythonhosted.org/packages/30/76/8f099f9d6482450428b17c4d6b241281af7ce6a9de8149ca8c1c649f6792/pyzmq-27.1.0-cp310-cp310-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:03ff0b279b40d687691a6217c12242ee71f0fba28bf8626ff50e3ef0f4410e1e", size = 854149, upload-time = "2025-09-08T23:07:33.17Z" }, - { url = "https://files.pythonhosted.org/packages/59/f0/37fbfff06c68016019043897e4c969ceab18bde46cd2aca89821fcf4fb2e/pyzmq-27.1.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:677e744fee605753eac48198b15a2124016c009a11056f93807000ab11ce6526", size = 1655070, upload-time = "2025-09-08T23:07:35.205Z" }, - { url = "https://files.pythonhosted.org/packages/47/14/7254be73f7a8edc3587609554fcaa7bfd30649bf89cd260e4487ca70fdaa/pyzmq-27.1.0-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:dd2fec2b13137416a1c5648b7009499bcc8fea78154cd888855fa32514f3dad1", size = 2033441, upload-time = "2025-09-08T23:07:37.432Z" }, - { url = "https://files.pythonhosted.org/packages/22/dc/49f2be26c6f86f347e796a4d99b19167fc94503f0af3fd010ad262158822/pyzmq-27.1.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:08e90bb4b57603b84eab1d0ca05b3bbb10f60c1839dc471fc1c9e1507bef3386", size = 1891529, upload-time = "2025-09-08T23:07:39.047Z" }, - { url = "https://files.pythonhosted.org/packages/a3/3e/154fb963ae25be70c0064ce97776c937ecc7d8b0259f22858154a9999769/pyzmq-27.1.0-cp310-cp310-win32.whl", hash = "sha256:a5b42d7a0658b515319148875fcb782bbf118dd41c671b62dae33666c2213bda", size = 567276, upload-time = "2025-09-08T23:07:40.695Z" }, - { url = "https://files.pythonhosted.org/packages/62/b2/f4ab56c8c595abcb26b2be5fd9fa9e6899c1e5ad54964e93ae8bb35482be/pyzmq-27.1.0-cp310-cp310-win_amd64.whl", hash = "sha256:c0bb87227430ee3aefcc0ade2088100e528d5d3298a0a715a64f3d04c60ba02f", size = 632208, upload-time = "2025-09-08T23:07:42.298Z" }, - { url = "https://files.pythonhosted.org/packages/3b/e3/be2cc7ab8332bdac0522fdb64c17b1b6241a795bee02e0196636ec5beb79/pyzmq-27.1.0-cp310-cp310-win_arm64.whl", hash = "sha256:9a916f76c2ab8d045b19f2286851a38e9ac94ea91faf65bd64735924522a8b32", size = 559766, upload-time = "2025-09-08T23:07:43.869Z" }, - { url = "https://files.pythonhosted.org/packages/06/5d/305323ba86b284e6fcb0d842d6adaa2999035f70f8c38a9b6d21ad28c3d4/pyzmq-27.1.0-cp311-cp311-macosx_10_15_universal2.whl", hash = "sha256:226b091818d461a3bef763805e75685e478ac17e9008f49fce2d3e52b3d58b86", size = 1333328, upload-time = "2025-09-08T23:07:45.946Z" }, - { url = "https://files.pythonhosted.org/packages/bd/a0/fc7e78a23748ad5443ac3275943457e8452da67fda347e05260261108cbc/pyzmq-27.1.0-cp311-cp311-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:0790a0161c281ca9723f804871b4027f2e8b5a528d357c8952d08cd1a9c15581", size = 908803, upload-time = "2025-09-08T23:07:47.551Z" }, - { url = "https://files.pythonhosted.org/packages/7e/22/37d15eb05f3bdfa4abea6f6d96eb3bb58585fbd3e4e0ded4e743bc650c97/pyzmq-27.1.0-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c895a6f35476b0c3a54e3eb6ccf41bf3018de937016e6e18748317f25d4e925f", size = 668836, upload-time = "2025-09-08T23:07:49.436Z" }, - { url = "https://files.pythonhosted.org/packages/b1/c4/2a6fe5111a01005fc7af3878259ce17684fabb8852815eda6225620f3c59/pyzmq-27.1.0-cp311-cp311-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5bbf8d3630bf96550b3be8e1fc0fea5cbdc8d5466c1192887bd94869da17a63e", size = 857038, upload-time = "2025-09-08T23:07:51.234Z" }, - { url = "https://files.pythonhosted.org/packages/cb/eb/bfdcb41d0db9cd233d6fb22dc131583774135505ada800ebf14dfb0a7c40/pyzmq-27.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:15c8bd0fe0dabf808e2d7a681398c4e5ded70a551ab47482067a572c054c8e2e", size = 1657531, upload-time = "2025-09-08T23:07:52.795Z" }, - { url = "https://files.pythonhosted.org/packages/ab/21/e3180ca269ed4a0de5c34417dfe71a8ae80421198be83ee619a8a485b0c7/pyzmq-27.1.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:bafcb3dd171b4ae9f19ee6380dfc71ce0390fefaf26b504c0e5f628d7c8c54f2", size = 2034786, upload-time = "2025-09-08T23:07:55.047Z" }, - { url = "https://files.pythonhosted.org/packages/3b/b1/5e21d0b517434b7f33588ff76c177c5a167858cc38ef740608898cd329f2/pyzmq-27.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:e829529fcaa09937189178115c49c504e69289abd39967cd8a4c215761373394", size = 1894220, upload-time = "2025-09-08T23:07:57.172Z" }, - { url = "https://files.pythonhosted.org/packages/03/f2/44913a6ff6941905efc24a1acf3d3cb6146b636c546c7406c38c49c403d4/pyzmq-27.1.0-cp311-cp311-win32.whl", hash = "sha256:6df079c47d5902af6db298ec92151db82ecb557af663098b92f2508c398bb54f", size = 567155, upload-time = "2025-09-08T23:07:59.05Z" }, - { url = "https://files.pythonhosted.org/packages/23/6d/d8d92a0eb270a925c9b4dd039c0b4dc10abc2fcbc48331788824ef113935/pyzmq-27.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:190cbf120fbc0fc4957b56866830def56628934a9d112aec0e2507aa6a032b97", size = 633428, upload-time = "2025-09-08T23:08:00.663Z" }, - { url = "https://files.pythonhosted.org/packages/ae/14/01afebc96c5abbbd713ecfc7469cfb1bc801c819a74ed5c9fad9a48801cb/pyzmq-27.1.0-cp311-cp311-win_arm64.whl", hash = "sha256:eca6b47df11a132d1745eb3b5b5e557a7dae2c303277aa0e69c6ba91b8736e07", size = 559497, upload-time = "2025-09-08T23:08:02.15Z" }, - { url = "https://files.pythonhosted.org/packages/92/e7/038aab64a946d535901103da16b953c8c9cc9c961dadcbf3609ed6428d23/pyzmq-27.1.0-cp312-abi3-macosx_10_15_universal2.whl", hash = "sha256:452631b640340c928fa343801b0d07eb0c3789a5ffa843f6e1a9cee0ba4eb4fc", size = 1306279, upload-time = "2025-09-08T23:08:03.807Z" }, - { url = "https://files.pythonhosted.org/packages/e8/5e/c3c49fdd0f535ef45eefcc16934648e9e59dace4a37ee88fc53f6cd8e641/pyzmq-27.1.0-cp312-abi3-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:1c179799b118e554b66da67d88ed66cd37a169f1f23b5d9f0a231b4e8d44a113", size = 895645, upload-time = "2025-09-08T23:08:05.301Z" }, - { url = "https://files.pythonhosted.org/packages/f8/e5/b0b2504cb4e903a74dcf1ebae157f9e20ebb6ea76095f6cfffea28c42ecd/pyzmq-27.1.0-cp312-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3837439b7f99e60312f0c926a6ad437b067356dc2bc2ec96eb395fd0fe804233", size = 652574, upload-time = "2025-09-08T23:08:06.828Z" }, - { url = "https://files.pythonhosted.org/packages/f8/9b/c108cdb55560eaf253f0cbdb61b29971e9fb34d9c3499b0e96e4e60ed8a5/pyzmq-27.1.0-cp312-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:43ad9a73e3da1fab5b0e7e13402f0b2fb934ae1c876c51d0afff0e7c052eca31", size = 840995, upload-time = "2025-09-08T23:08:08.396Z" }, - { url = "https://files.pythonhosted.org/packages/c2/bb/b79798ca177b9eb0825b4c9998c6af8cd2a7f15a6a1a4272c1d1a21d382f/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:0de3028d69d4cdc475bfe47a6128eb38d8bc0e8f4d69646adfbcd840facbac28", size = 1642070, upload-time = "2025-09-08T23:08:09.989Z" }, - { url = "https://files.pythonhosted.org/packages/9c/80/2df2e7977c4ede24c79ae39dcef3899bfc5f34d1ca7a5b24f182c9b7a9ca/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_i686.whl", hash = "sha256:cf44a7763aea9298c0aa7dbf859f87ed7012de8bda0f3977b6fb1d96745df856", size = 2021121, upload-time = "2025-09-08T23:08:11.907Z" }, - { url = "https://files.pythonhosted.org/packages/46/bd/2d45ad24f5f5ae7e8d01525eb76786fa7557136555cac7d929880519e33a/pyzmq-27.1.0-cp312-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:f30f395a9e6fbca195400ce833c731e7b64c3919aa481af4d88c3759e0cb7496", size = 1878550, upload-time = "2025-09-08T23:08:13.513Z" }, - { url = "https://files.pythonhosted.org/packages/e6/2f/104c0a3c778d7c2ab8190e9db4f62f0b6957b53c9d87db77c284b69f33ea/pyzmq-27.1.0-cp312-abi3-win32.whl", hash = "sha256:250e5436a4ba13885494412b3da5d518cd0d3a278a1ae640e113c073a5f88edd", size = 559184, upload-time = "2025-09-08T23:08:15.163Z" }, - { url = "https://files.pythonhosted.org/packages/fc/7f/a21b20d577e4100c6a41795842028235998a643b1ad406a6d4163ea8f53e/pyzmq-27.1.0-cp312-abi3-win_amd64.whl", hash = "sha256:9ce490cf1d2ca2ad84733aa1d69ce6855372cb5ce9223802450c9b2a7cba0ccf", size = 619480, upload-time = "2025-09-08T23:08:17.192Z" }, - { url = "https://files.pythonhosted.org/packages/78/c2/c012beae5f76b72f007a9e91ee9401cb88c51d0f83c6257a03e785c81cc2/pyzmq-27.1.0-cp312-abi3-win_arm64.whl", hash = "sha256:75a2f36223f0d535a0c919e23615fc85a1e23b71f40c7eb43d7b1dedb4d8f15f", size = 552993, upload-time = "2025-09-08T23:08:18.926Z" }, - { url = "https://files.pythonhosted.org/packages/60/cb/84a13459c51da6cec1b7b1dc1a47e6db6da50b77ad7fd9c145842750a011/pyzmq-27.1.0-cp313-cp313-android_24_arm64_v8a.whl", hash = "sha256:93ad4b0855a664229559e45c8d23797ceac03183c7b6f5b4428152a6b06684a5", size = 1122436, upload-time = "2025-09-08T23:08:20.801Z" }, - { url = "https://files.pythonhosted.org/packages/dc/b6/94414759a69a26c3dd674570a81813c46a078767d931a6c70ad29fc585cb/pyzmq-27.1.0-cp313-cp313-android_24_x86_64.whl", hash = "sha256:fbb4f2400bfda24f12f009cba62ad5734148569ff4949b1b6ec3b519444342e6", size = 1156301, upload-time = "2025-09-08T23:08:22.47Z" }, - { url = "https://files.pythonhosted.org/packages/a5/ad/15906493fd40c316377fd8a8f6b1f93104f97a752667763c9b9c1b71d42d/pyzmq-27.1.0-cp313-cp313t-macosx_10_15_universal2.whl", hash = "sha256:e343d067f7b151cfe4eb3bb796a7752c9d369eed007b91231e817071d2c2fec7", size = 1341197, upload-time = "2025-09-08T23:08:24.286Z" }, - { url = "https://files.pythonhosted.org/packages/14/1d/d343f3ce13db53a54cb8946594e567410b2125394dafcc0268d8dda027e0/pyzmq-27.1.0-cp313-cp313t-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:08363b2011dec81c354d694bdecaef4770e0ae96b9afea70b3f47b973655cc05", size = 897275, upload-time = "2025-09-08T23:08:26.063Z" }, - { url = "https://files.pythonhosted.org/packages/69/2d/d83dd6d7ca929a2fc67d2c3005415cdf322af7751d773524809f9e585129/pyzmq-27.1.0-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d54530c8c8b5b8ddb3318f481297441af102517602b569146185fa10b63f4fa9", size = 660469, upload-time = "2025-09-08T23:08:27.623Z" }, - { url = "https://files.pythonhosted.org/packages/3e/cd/9822a7af117f4bc0f1952dbe9ef8358eb50a24928efd5edf54210b850259/pyzmq-27.1.0-cp313-cp313t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6f3afa12c392f0a44a2414056d730eebc33ec0926aae92b5ad5cf26ebb6cc128", size = 847961, upload-time = "2025-09-08T23:08:29.672Z" }, - { url = "https://files.pythonhosted.org/packages/9a/12/f003e824a19ed73be15542f172fd0ec4ad0b60cf37436652c93b9df7c585/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c65047adafe573ff023b3187bb93faa583151627bc9c51fc4fb2c561ed689d39", size = 1650282, upload-time = "2025-09-08T23:08:31.349Z" }, - { url = "https://files.pythonhosted.org/packages/d5/4a/e82d788ed58e9a23995cee70dbc20c9aded3d13a92d30d57ec2291f1e8a3/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:90e6e9441c946a8b0a667356f7078d96411391a3b8f80980315455574177ec97", size = 2024468, upload-time = "2025-09-08T23:08:33.543Z" }, - { url = "https://files.pythonhosted.org/packages/d9/94/2da0a60841f757481e402b34bf4c8bf57fa54a5466b965de791b1e6f747d/pyzmq-27.1.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:add071b2d25f84e8189aaf0882d39a285b42fa3853016ebab234a5e78c7a43db", size = 1885394, upload-time = "2025-09-08T23:08:35.51Z" }, - { url = "https://files.pythonhosted.org/packages/4f/6f/55c10e2e49ad52d080dc24e37adb215e5b0d64990b57598abc2e3f01725b/pyzmq-27.1.0-cp313-cp313t-win32.whl", hash = "sha256:7ccc0700cfdf7bd487bea8d850ec38f204478681ea02a582a8da8171b7f90a1c", size = 574964, upload-time = "2025-09-08T23:08:37.178Z" }, - { url = "https://files.pythonhosted.org/packages/87/4d/2534970ba63dd7c522d8ca80fb92777f362c0f321900667c615e2067cb29/pyzmq-27.1.0-cp313-cp313t-win_amd64.whl", hash = "sha256:8085a9fba668216b9b4323be338ee5437a235fe275b9d1610e422ccc279733e2", size = 641029, upload-time = "2025-09-08T23:08:40.595Z" }, - { url = "https://files.pythonhosted.org/packages/f6/fa/f8aea7a28b0641f31d40dea42d7ef003fded31e184ef47db696bc74cd610/pyzmq-27.1.0-cp313-cp313t-win_arm64.whl", hash = "sha256:6bb54ca21bcfe361e445256c15eedf083f153811c37be87e0514934d6913061e", size = 561541, upload-time = "2025-09-08T23:08:42.668Z" }, - { url = "https://files.pythonhosted.org/packages/87/45/19efbb3000956e82d0331bafca5d9ac19ea2857722fa2caacefb6042f39d/pyzmq-27.1.0-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:ce980af330231615756acd5154f29813d553ea555485ae712c491cd483df6b7a", size = 1341197, upload-time = "2025-09-08T23:08:44.973Z" }, - { url = "https://files.pythonhosted.org/packages/48/43/d72ccdbf0d73d1343936296665826350cb1e825f92f2db9db3e61c2162a2/pyzmq-27.1.0-cp314-cp314t-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:1779be8c549e54a1c38f805e56d2a2e5c009d26de10921d7d51cfd1c8d4632ea", size = 897175, upload-time = "2025-09-08T23:08:46.601Z" }, - { url = "https://files.pythonhosted.org/packages/2f/2e/a483f73a10b65a9ef0161e817321d39a770b2acf8bcf3004a28d90d14a94/pyzmq-27.1.0-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7200bb0f03345515df50d99d3db206a0a6bee1955fbb8c453c76f5bf0e08fb96", size = 660427, upload-time = "2025-09-08T23:08:48.187Z" }, - { url = "https://files.pythonhosted.org/packages/f5/d2/5f36552c2d3e5685abe60dfa56f91169f7a2d99bbaf67c5271022ab40863/pyzmq-27.1.0-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01c0e07d558b06a60773744ea6251f769cd79a41a97d11b8bf4ab8f034b0424d", size = 847929, upload-time = "2025-09-08T23:08:49.76Z" }, - { url = "https://files.pythonhosted.org/packages/c4/2a/404b331f2b7bf3198e9945f75c4c521f0c6a3a23b51f7a4a401b94a13833/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:80d834abee71f65253c91540445d37c4c561e293ba6e741b992f20a105d69146", size = 1650193, upload-time = "2025-09-08T23:08:51.7Z" }, - { url = "https://files.pythonhosted.org/packages/1c/0b/f4107e33f62a5acf60e3ded67ed33d79b4ce18de432625ce2fc5093d6388/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:544b4e3b7198dde4a62b8ff6685e9802a9a1ebf47e77478a5eb88eca2a82f2fd", size = 2024388, upload-time = "2025-09-08T23:08:53.393Z" }, - { url = "https://files.pythonhosted.org/packages/0d/01/add31fe76512642fd6e40e3a3bd21f4b47e242c8ba33efb6809e37076d9b/pyzmq-27.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cedc4c68178e59a4046f97eca31b148ddcf51e88677de1ef4e78cf06c5376c9a", size = 1885316, upload-time = "2025-09-08T23:08:55.702Z" }, - { url = "https://files.pythonhosted.org/packages/c4/59/a5f38970f9bf07cee96128de79590bb354917914a9be11272cfc7ff26af0/pyzmq-27.1.0-cp314-cp314t-win32.whl", hash = "sha256:1f0b2a577fd770aa6f053211a55d1c47901f4d537389a034c690291485e5fe92", size = 587472, upload-time = "2025-09-08T23:08:58.18Z" }, - { url = "https://files.pythonhosted.org/packages/70/d8/78b1bad170f93fcf5e3536e70e8fadac55030002275c9a29e8f5719185de/pyzmq-27.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:19c9468ae0437f8074af379e986c5d3d7d7bfe033506af442e8c879732bedbe0", size = 661401, upload-time = "2025-09-08T23:08:59.802Z" }, - { url = "https://files.pythonhosted.org/packages/81/d6/4bfbb40c9a0b42fc53c7cf442f6385db70b40f74a783130c5d0a5aa62228/pyzmq-27.1.0-cp314-cp314t-win_arm64.whl", hash = "sha256:dc5dbf68a7857b59473f7df42650c621d7e8923fb03fa74a526890f4d33cc4d7", size = 575170, upload-time = "2025-09-08T23:09:01.418Z" }, - { url = "https://files.pythonhosted.org/packages/f3/81/a65e71c1552f74dec9dff91d95bafb6e0d33338a8dfefbc88aa562a20c92/pyzmq-27.1.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:c17e03cbc9312bee223864f1a2b13a99522e0dc9f7c5df0177cd45210ac286e6", size = 836266, upload-time = "2025-09-08T23:09:40.048Z" }, - { url = "https://files.pythonhosted.org/packages/58/ed/0202ca350f4f2b69faa95c6d931e3c05c3a397c184cacb84cb4f8f42f287/pyzmq-27.1.0-pp310-pypy310_pp73-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:f328d01128373cb6763823b2b4e7f73bdf767834268c565151eacb3b7a392f90", size = 800206, upload-time = "2025-09-08T23:09:41.902Z" }, - { url = "https://files.pythonhosted.org/packages/47/42/1ff831fa87fe8f0a840ddb399054ca0009605d820e2b44ea43114f5459f4/pyzmq-27.1.0-pp310-pypy310_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9c1790386614232e1b3a40a958454bdd42c6d1811837b15ddbb052a032a43f62", size = 567747, upload-time = "2025-09-08T23:09:43.741Z" }, - { url = "https://files.pythonhosted.org/packages/d1/db/5c4d6807434751e3f21231bee98109aa57b9b9b55e058e450d0aef59b70f/pyzmq-27.1.0-pp310-pypy310_pp73-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:448f9cb54eb0cee4732b46584f2710c8bc178b0e5371d9e4fc8125201e413a74", size = 747371, upload-time = "2025-09-08T23:09:45.575Z" }, - { url = "https://files.pythonhosted.org/packages/26/af/78ce193dbf03567eb8c0dc30e3df2b9e56f12a670bf7eb20f9fb532c7e8a/pyzmq-27.1.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:05b12f2d32112bf8c95ef2e74ec4f1d4beb01f8b5e703b38537f8849f92cb9ba", size = 544862, upload-time = "2025-09-08T23:09:47.448Z" }, - { url = "https://files.pythonhosted.org/packages/4c/c6/c4dcdecdbaa70969ee1fdced6d7b8f60cfabe64d25361f27ac4665a70620/pyzmq-27.1.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:18770c8d3563715387139060d37859c02ce40718d1faf299abddcdcc6a649066", size = 836265, upload-time = "2025-09-08T23:09:49.376Z" }, - { url = "https://files.pythonhosted.org/packages/3e/79/f38c92eeaeb03a2ccc2ba9866f0439593bb08c5e3b714ac1d553e5c96e25/pyzmq-27.1.0-pp311-pypy311_pp73-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:ac25465d42f92e990f8d8b0546b01c391ad431c3bf447683fdc40565941d0604", size = 800208, upload-time = "2025-09-08T23:09:51.073Z" }, - { url = "https://files.pythonhosted.org/packages/49/0e/3f0d0d335c6b3abb9b7b723776d0b21fa7f3a6c819a0db6097059aada160/pyzmq-27.1.0-pp311-pypy311_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:53b40f8ae006f2734ee7608d59ed661419f087521edbfc2149c3932e9c14808c", size = 567747, upload-time = "2025-09-08T23:09:52.698Z" }, - { url = "https://files.pythonhosted.org/packages/a1/cf/f2b3784d536250ffd4be70e049f3b60981235d70c6e8ce7e3ef21e1adb25/pyzmq-27.1.0-pp311-pypy311_pp73-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f605d884e7c8be8fe1aa94e0a783bf3f591b84c24e4bc4f3e7564c82ac25e271", size = 747371, upload-time = "2025-09-08T23:09:54.563Z" }, - { url = "https://files.pythonhosted.org/packages/01/1b/5dbe84eefc86f48473947e2f41711aded97eecef1231f4558f1f02713c12/pyzmq-27.1.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:c9f7f6e13dff2e44a6afeaf2cf54cee5929ad64afaf4d40b50f93c58fc687355", size = 544862, upload-time = "2025-09-08T23:09:56.509Z" }, -] - -[[package]] -name = "referencing" -version = "0.37.0" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "attrs" }, - { name = "rpds-py", version = "0.30.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "rpds-py", version = "2026.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "typing-extensions", marker = "python_full_version < '3.13'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/22/f5/df4e9027acead3ecc63e50fe1e36aca1523e1719559c499951bb4b53188f/referencing-0.37.0.tar.gz", hash = "sha256:44aefc3142c5b842538163acb373e24cce6632bd54bdb01b21ad5863489f50d8", size = 78036, upload-time = "2025-10-13T15:30:48.871Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/2c/58/ca301544e1fa93ed4f80d724bf5b194f6e4b945841c5bfd555878eea9fcb/referencing-0.37.0-py3-none-any.whl", hash = "sha256:381329a9f99628c9069361716891d34ad94af76e461dcb0335825aecc7692231", size = 26766, upload-time = "2025-10-13T15:30:47.625Z" }, -] - [[package]] name = "requests" version = "2.32.3" @@ -1962,273 +1393,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f9/9b/335f9764261e915ed497fcdeb11df5dfd6f7bf257d4a6a2a686d80da4d54/requests-2.32.3-py3-none-any.whl", hash = "sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6", size = 64928, upload-time = "2024-05-29T15:37:47.027Z" }, ] -[[package]] -name = "rpds-py" -version = "0.30.0" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.11'", -] -sdist = { url = "https://files.pythonhosted.org/packages/20/af/3f2f423103f1113b36230496629986e0ef7e199d2aa8392452b484b38ced/rpds_py-0.30.0.tar.gz", hash = "sha256:dd8ff7cf90014af0c0f787eea34794ebf6415242ee1d6fa91eaba725cc441e84", size = 69469, upload-time = "2025-11-30T20:24:38.837Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/06/0c/0c411a0ec64ccb6d104dcabe0e713e05e153a9a2c3c2bd2b32ce412166fe/rpds_py-0.30.0-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:679ae98e00c0e8d68a7fda324e16b90fd5260945b45d3b824c892cec9eea3288", size = 370490, upload-time = "2025-11-30T20:21:33.256Z" }, - { url = "https://files.pythonhosted.org/packages/19/6a/4ba3d0fb7297ebae71171822554abe48d7cab29c28b8f9f2c04b79988c05/rpds_py-0.30.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:4cc2206b76b4f576934f0ed374b10d7ca5f457858b157ca52064bdfc26b9fc00", size = 359751, upload-time = "2025-11-30T20:21:34.591Z" }, - { url = "https://files.pythonhosted.org/packages/cd/7c/e4933565ef7f7a0818985d87c15d9d273f1a649afa6a52ea35ad011195ea/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:389a2d49eded1896c3d48b0136ead37c48e221b391c052fba3f4055c367f60a6", size = 389696, upload-time = "2025-11-30T20:21:36.122Z" }, - { url = "https://files.pythonhosted.org/packages/5e/01/6271a2511ad0815f00f7ed4390cf2567bec1d4b1da39e2c27a41e6e3b4de/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:32c8528634e1bf7121f3de08fa85b138f4e0dc47657866630611b03967f041d7", size = 403136, upload-time = "2025-11-30T20:21:37.728Z" }, - { url = "https://files.pythonhosted.org/packages/55/64/c857eb7cd7541e9b4eee9d49c196e833128a55b89a9850a9c9ac33ccf897/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f207f69853edd6f6700b86efb84999651baf3789e78a466431df1331608e5324", size = 524699, upload-time = "2025-11-30T20:21:38.92Z" }, - { url = "https://files.pythonhosted.org/packages/9c/ed/94816543404078af9ab26159c44f9e98e20fe47e2126d5d32c9d9948d10a/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:67b02ec25ba7a9e8fa74c63b6ca44cf5707f2fbfadae3ee8e7494297d56aa9df", size = 412022, upload-time = "2025-11-30T20:21:40.407Z" }, - { url = "https://files.pythonhosted.org/packages/61/b5/707f6cf0066a6412aacc11d17920ea2e19e5b2f04081c64526eb35b5c6e7/rpds_py-0.30.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0c0e95f6819a19965ff420f65578bacb0b00f251fefe2c8b23347c37174271f3", size = 390522, upload-time = "2025-11-30T20:21:42.17Z" }, - { url = "https://files.pythonhosted.org/packages/13/4e/57a85fda37a229ff4226f8cbcf09f2a455d1ed20e802ce5b2b4a7f5ed053/rpds_py-0.30.0-cp310-cp310-manylinux_2_31_riscv64.whl", hash = "sha256:a452763cc5198f2f98898eb98f7569649fe5da666c2dc6b5ddb10fde5a574221", size = 404579, upload-time = "2025-11-30T20:21:43.769Z" }, - { url = "https://files.pythonhosted.org/packages/f9/da/c9339293513ec680a721e0e16bf2bac3db6e5d7e922488de471308349bba/rpds_py-0.30.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e0b65193a413ccc930671c55153a03ee57cecb49e6227204b04fae512eb657a7", size = 421305, upload-time = "2025-11-30T20:21:44.994Z" }, - { url = "https://files.pythonhosted.org/packages/f9/be/522cb84751114f4ad9d822ff5a1aa3c98006341895d5f084779b99596e5c/rpds_py-0.30.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:858738e9c32147f78b3ac24dc0edb6610000e56dc0f700fd5f651d0a0f0eb9ff", size = 572503, upload-time = "2025-11-30T20:21:46.91Z" }, - { url = "https://files.pythonhosted.org/packages/a2/9b/de879f7e7ceddc973ea6e4629e9b380213a6938a249e94b0cdbcc325bb66/rpds_py-0.30.0-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:da279aa314f00acbb803da1e76fa18666778e8a8f83484fba94526da5de2cba7", size = 598322, upload-time = "2025-11-30T20:21:48.709Z" }, - { url = "https://files.pythonhosted.org/packages/48/ac/f01fc22efec3f37d8a914fc1b2fb9bcafd56a299edbe96406f3053edea5a/rpds_py-0.30.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:7c64d38fb49b6cdeda16ab49e35fe0da2e1e9b34bc38bd78386530f218b37139", size = 560792, upload-time = "2025-11-30T20:21:50.024Z" }, - { url = "https://files.pythonhosted.org/packages/e2/da/4e2b19d0f131f35b6146425f846563d0ce036763e38913d917187307a671/rpds_py-0.30.0-cp310-cp310-win32.whl", hash = "sha256:6de2a32a1665b93233cde140ff8b3467bdb9e2af2b91079f0333a0974d12d464", size = 221901, upload-time = "2025-11-30T20:21:51.32Z" }, - { url = "https://files.pythonhosted.org/packages/96/cb/156d7a5cf4f78a7cc571465d8aec7a3c447c94f6749c5123f08438bcf7bc/rpds_py-0.30.0-cp310-cp310-win_amd64.whl", hash = "sha256:1726859cd0de969f88dc8673bdd954185b9104e05806be64bcd87badbe313169", size = 235823, upload-time = "2025-11-30T20:21:52.505Z" }, - { url = "https://files.pythonhosted.org/packages/4d/6e/f964e88b3d2abee2a82c1ac8366da848fce1c6d834dc2132c3fda3970290/rpds_py-0.30.0-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:a2bffea6a4ca9f01b3f8e548302470306689684e61602aa3d141e34da06cf425", size = 370157, upload-time = "2025-11-30T20:21:53.789Z" }, - { url = "https://files.pythonhosted.org/packages/94/ba/24e5ebb7c1c82e74c4e4f33b2112a5573ddc703915b13a073737b59b86e0/rpds_py-0.30.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:dc4f992dfe1e2bc3ebc7444f6c7051b4bc13cd8e33e43511e8ffd13bf407010d", size = 359676, upload-time = "2025-11-30T20:21:55.475Z" }, - { url = "https://files.pythonhosted.org/packages/84/86/04dbba1b087227747d64d80c3b74df946b986c57af0a9f0c98726d4d7a3b/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:422c3cb9856d80b09d30d2eb255d0754b23e090034e1deb4083f8004bd0761e4", size = 389938, upload-time = "2025-11-30T20:21:57.079Z" }, - { url = "https://files.pythonhosted.org/packages/42/bb/1463f0b1722b7f45431bdd468301991d1328b16cffe0b1c2918eba2c4eee/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:07ae8a593e1c3c6b82ca3292efbe73c30b61332fd612e05abee07c79359f292f", size = 402932, upload-time = "2025-11-30T20:21:58.47Z" }, - { url = "https://files.pythonhosted.org/packages/99/ee/2520700a5c1f2d76631f948b0736cdf9b0acb25abd0ca8e889b5c62ac2e3/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:12f90dd7557b6bd57f40abe7747e81e0c0b119bef015ea7726e69fe550e394a4", size = 525830, upload-time = "2025-11-30T20:21:59.699Z" }, - { url = "https://files.pythonhosted.org/packages/e0/ad/bd0331f740f5705cc555a5e17fdf334671262160270962e69a2bdef3bf76/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:99b47d6ad9a6da00bec6aabe5a6279ecd3c06a329d4aa4771034a21e335c3a97", size = 412033, upload-time = "2025-11-30T20:22:00.991Z" }, - { url = "https://files.pythonhosted.org/packages/f8/1e/372195d326549bb51f0ba0f2ecb9874579906b97e08880e7a65c3bef1a99/rpds_py-0.30.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:33f559f3104504506a44bb666b93a33f5d33133765b0c216a5bf2f1e1503af89", size = 390828, upload-time = "2025-11-30T20:22:02.723Z" }, - { url = "https://files.pythonhosted.org/packages/ab/2b/d88bb33294e3e0c76bc8f351a3721212713629ffca1700fa94979cb3eae8/rpds_py-0.30.0-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:946fe926af6e44f3697abbc305ea168c2c31d3e3ef1058cf68f379bf0335a78d", size = 404683, upload-time = "2025-11-30T20:22:04.367Z" }, - { url = "https://files.pythonhosted.org/packages/50/32/c759a8d42bcb5289c1fac697cd92f6fe01a018dd937e62ae77e0e7f15702/rpds_py-0.30.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:495aeca4b93d465efde585977365187149e75383ad2684f81519f504f5c13038", size = 421583, upload-time = "2025-11-30T20:22:05.814Z" }, - { url = "https://files.pythonhosted.org/packages/2b/81/e729761dbd55ddf5d84ec4ff1f47857f4374b0f19bdabfcf929164da3e24/rpds_py-0.30.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d9a0ca5da0386dee0655b4ccdf46119df60e0f10da268d04fe7cc87886872ba7", size = 572496, upload-time = "2025-11-30T20:22:07.713Z" }, - { url = "https://files.pythonhosted.org/packages/14/f6/69066a924c3557c9c30baa6ec3a0aa07526305684c6f86c696b08860726c/rpds_py-0.30.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:8d6d1cc13664ec13c1b84241204ff3b12f9bb82464b8ad6e7a5d3486975c2eed", size = 598669, upload-time = "2025-11-30T20:22:09.312Z" }, - { url = "https://files.pythonhosted.org/packages/5f/48/905896b1eb8a05630d20333d1d8ffd162394127b74ce0b0784ae04498d32/rpds_py-0.30.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:3896fa1be39912cf0757753826bc8bdc8ca331a28a7c4ae46b7a21280b06bb85", size = 561011, upload-time = "2025-11-30T20:22:11.309Z" }, - { url = "https://files.pythonhosted.org/packages/22/16/cd3027c7e279d22e5eb431dd3c0fbc677bed58797fe7581e148f3f68818b/rpds_py-0.30.0-cp311-cp311-win32.whl", hash = "sha256:55f66022632205940f1827effeff17c4fa7ae1953d2b74a8581baaefb7d16f8c", size = 221406, upload-time = "2025-11-30T20:22:13.101Z" }, - { url = "https://files.pythonhosted.org/packages/fa/5b/e7b7aa136f28462b344e652ee010d4de26ee9fd16f1bfd5811f5153ccf89/rpds_py-0.30.0-cp311-cp311-win_amd64.whl", hash = "sha256:a51033ff701fca756439d641c0ad09a41d9242fa69121c7d8769604a0a629825", size = 236024, upload-time = "2025-11-30T20:22:14.853Z" }, - { url = "https://files.pythonhosted.org/packages/14/a6/364bba985e4c13658edb156640608f2c9e1d3ea3c81b27aa9d889fff0e31/rpds_py-0.30.0-cp311-cp311-win_arm64.whl", hash = "sha256:47b0ef6231c58f506ef0b74d44e330405caa8428e770fec25329ed2cb971a229", size = 229069, upload-time = "2025-11-30T20:22:16.577Z" }, - { url = "https://files.pythonhosted.org/packages/03/e7/98a2f4ac921d82f33e03f3835f5bf3a4a40aa1bfdc57975e74a97b2b4bdd/rpds_py-0.30.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:a161f20d9a43006833cd7068375a94d035714d73a172b681d8881820600abfad", size = 375086, upload-time = "2025-11-30T20:22:17.93Z" }, - { url = "https://files.pythonhosted.org/packages/4d/a1/bca7fd3d452b272e13335db8d6b0b3ecde0f90ad6f16f3328c6fb150c889/rpds_py-0.30.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6abc8880d9d036ecaafe709079969f56e876fcf107f7a8e9920ba6d5a3878d05", size = 359053, upload-time = "2025-11-30T20:22:19.297Z" }, - { url = "https://files.pythonhosted.org/packages/65/1c/ae157e83a6357eceff62ba7e52113e3ec4834a84cfe07fa4b0757a7d105f/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ca28829ae5f5d569bb62a79512c842a03a12576375d5ece7d2cadf8abe96ec28", size = 390763, upload-time = "2025-11-30T20:22:21.661Z" }, - { url = "https://files.pythonhosted.org/packages/d4/36/eb2eb8515e2ad24c0bd43c3ee9cd74c33f7ca6430755ccdb240fd3144c44/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:a1010ed9524c73b94d15919ca4d41d8780980e1765babf85f9a2f90d247153dd", size = 408951, upload-time = "2025-11-30T20:22:23.408Z" }, - { url = "https://files.pythonhosted.org/packages/d6/65/ad8dc1784a331fabbd740ef6f71ce2198c7ed0890dab595adb9ea2d775a1/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f8d1736cfb49381ba528cd5baa46f82fdc65c06e843dab24dd70b63d09121b3f", size = 514622, upload-time = "2025-11-30T20:22:25.16Z" }, - { url = "https://files.pythonhosted.org/packages/63/8e/0cfa7ae158e15e143fe03993b5bcd743a59f541f5952e1546b1ac1b5fd45/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d948b135c4693daff7bc2dcfc4ec57237a29bd37e60c2fabf5aff2bbacf3e2f1", size = 414492, upload-time = "2025-11-30T20:22:26.505Z" }, - { url = "https://files.pythonhosted.org/packages/60/1b/6f8f29f3f995c7ffdde46a626ddccd7c63aefc0efae881dc13b6e5d5bb16/rpds_py-0.30.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:47f236970bccb2233267d89173d3ad2703cd36a0e2a6e92d0560d333871a3d23", size = 394080, upload-time = "2025-11-30T20:22:27.934Z" }, - { url = "https://files.pythonhosted.org/packages/6d/d5/a266341051a7a3ca2f4b750a3aa4abc986378431fc2da508c5034d081b70/rpds_py-0.30.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:2e6ecb5a5bcacf59c3f912155044479af1d0b6681280048b338b28e364aca1f6", size = 408680, upload-time = "2025-11-30T20:22:29.341Z" }, - { url = "https://files.pythonhosted.org/packages/10/3b/71b725851df9ab7a7a4e33cf36d241933da66040d195a84781f49c50490c/rpds_py-0.30.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a8fa71a2e078c527c3e9dc9fc5a98c9db40bcc8a92b4e8858e36d329f8684b51", size = 423589, upload-time = "2025-11-30T20:22:31.469Z" }, - { url = "https://files.pythonhosted.org/packages/00/2b/e59e58c544dc9bd8bd8384ecdb8ea91f6727f0e37a7131baeff8d6f51661/rpds_py-0.30.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:73c67f2db7bc334e518d097c6d1e6fed021bbc9b7d678d6cc433478365d1d5f5", size = 573289, upload-time = "2025-11-30T20:22:32.997Z" }, - { url = "https://files.pythonhosted.org/packages/da/3e/a18e6f5b460893172a7d6a680e86d3b6bc87a54c1f0b03446a3c8c7b588f/rpds_py-0.30.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:5ba103fb455be00f3b1c2076c9d4264bfcb037c976167a6047ed82f23153f02e", size = 599737, upload-time = "2025-11-30T20:22:34.419Z" }, - { url = "https://files.pythonhosted.org/packages/5c/e2/714694e4b87b85a18e2c243614974413c60aa107fd815b8cbc42b873d1d7/rpds_py-0.30.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:7cee9c752c0364588353e627da8a7e808a66873672bcb5f52890c33fd965b394", size = 563120, upload-time = "2025-11-30T20:22:35.903Z" }, - { url = "https://files.pythonhosted.org/packages/6f/ab/d5d5e3bcedb0a77f4f613706b750e50a5a3ba1c15ccd3665ecc636c968fd/rpds_py-0.30.0-cp312-cp312-win32.whl", hash = "sha256:1ab5b83dbcf55acc8b08fc62b796ef672c457b17dbd7820a11d6c52c06839bdf", size = 223782, upload-time = "2025-11-30T20:22:37.271Z" }, - { url = "https://files.pythonhosted.org/packages/39/3b/f786af9957306fdc38a74cef405b7b93180f481fb48453a114bb6465744a/rpds_py-0.30.0-cp312-cp312-win_amd64.whl", hash = "sha256:a090322ca841abd453d43456ac34db46e8b05fd9b3b4ac0c78bcde8b089f959b", size = 240463, upload-time = "2025-11-30T20:22:39.021Z" }, - { url = "https://files.pythonhosted.org/packages/f3/d2/b91dc748126c1559042cfe41990deb92c4ee3e2b415f6b5234969ffaf0cc/rpds_py-0.30.0-cp312-cp312-win_arm64.whl", hash = "sha256:669b1805bd639dd2989b281be2cfd951c6121b65e729d9b843e9639ef1fd555e", size = 230868, upload-time = "2025-11-30T20:22:40.493Z" }, - { url = "https://files.pythonhosted.org/packages/ed/dc/d61221eb88ff410de3c49143407f6f3147acf2538c86f2ab7ce65ae7d5f9/rpds_py-0.30.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:f83424d738204d9770830d35290ff3273fbb02b41f919870479fab14b9d303b2", size = 374887, upload-time = "2025-11-30T20:22:41.812Z" }, - { url = "https://files.pythonhosted.org/packages/fd/32/55fb50ae104061dbc564ef15cc43c013dc4a9f4527a1f4d99baddf56fe5f/rpds_py-0.30.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:e7536cd91353c5273434b4e003cbda89034d67e7710eab8761fd918ec6c69cf8", size = 358904, upload-time = "2025-11-30T20:22:43.479Z" }, - { url = "https://files.pythonhosted.org/packages/58/70/faed8186300e3b9bdd138d0273109784eea2396c68458ed580f885dfe7ad/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2771c6c15973347f50fece41fc447c054b7ac2ae0502388ce3b6738cd366e3d4", size = 389945, upload-time = "2025-11-30T20:22:44.819Z" }, - { url = "https://files.pythonhosted.org/packages/bd/a8/073cac3ed2c6387df38f71296d002ab43496a96b92c823e76f46b8af0543/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:0a59119fc6e3f460315fe9d08149f8102aa322299deaa5cab5b40092345c2136", size = 407783, upload-time = "2025-11-30T20:22:46.103Z" }, - { url = "https://files.pythonhosted.org/packages/77/57/5999eb8c58671f1c11eba084115e77a8899d6e694d2a18f69f0ba471ec8b/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:76fec018282b4ead0364022e3c54b60bf368b9d926877957a8624b58419169b7", size = 515021, upload-time = "2025-11-30T20:22:47.458Z" }, - { url = "https://files.pythonhosted.org/packages/e0/af/5ab4833eadc36c0a8ed2bc5c0de0493c04f6c06de223170bd0798ff98ced/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:692bef75a5525db97318e8cd061542b5a79812d711ea03dbc1f6f8dbb0c5f0d2", size = 414589, upload-time = "2025-11-30T20:22:48.872Z" }, - { url = "https://files.pythonhosted.org/packages/b7/de/f7192e12b21b9e9a68a6d0f249b4af3fdcdff8418be0767a627564afa1f1/rpds_py-0.30.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9027da1ce107104c50c81383cae773ef5c24d296dd11c99e2629dbd7967a20c6", size = 394025, upload-time = "2025-11-30T20:22:50.196Z" }, - { url = "https://files.pythonhosted.org/packages/91/c4/fc70cd0249496493500e7cc2de87504f5aa6509de1e88623431fec76d4b6/rpds_py-0.30.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:9cf69cdda1f5968a30a359aba2f7f9aa648a9ce4b580d6826437f2b291cfc86e", size = 408895, upload-time = "2025-11-30T20:22:51.87Z" }, - { url = "https://files.pythonhosted.org/packages/58/95/d9275b05ab96556fefff73a385813eb66032e4c99f411d0795372d9abcea/rpds_py-0.30.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a4796a717bf12b9da9d3ad002519a86063dcac8988b030e405704ef7d74d2d9d", size = 422799, upload-time = "2025-11-30T20:22:53.341Z" }, - { url = "https://files.pythonhosted.org/packages/06/c1/3088fc04b6624eb12a57eb814f0d4997a44b0d208d6cace713033ff1a6ba/rpds_py-0.30.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:5d4c2aa7c50ad4728a094ebd5eb46c452e9cb7edbfdb18f9e1221f597a73e1e7", size = 572731, upload-time = "2025-11-30T20:22:54.778Z" }, - { url = "https://files.pythonhosted.org/packages/d8/42/c612a833183b39774e8ac8fecae81263a68b9583ee343db33ab571a7ce55/rpds_py-0.30.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:ba81a9203d07805435eb06f536d95a266c21e5b2dfbf6517748ca40c98d19e31", size = 599027, upload-time = "2025-11-30T20:22:56.212Z" }, - { url = "https://files.pythonhosted.org/packages/5f/60/525a50f45b01d70005403ae0e25f43c0384369ad24ffe46e8d9068b50086/rpds_py-0.30.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:945dccface01af02675628334f7cf49c2af4c1c904748efc5cf7bbdf0b579f95", size = 563020, upload-time = "2025-11-30T20:22:58.2Z" }, - { url = "https://files.pythonhosted.org/packages/0b/5d/47c4655e9bcd5ca907148535c10e7d489044243cc9941c16ed7cd53be91d/rpds_py-0.30.0-cp313-cp313-win32.whl", hash = "sha256:b40fb160a2db369a194cb27943582b38f79fc4887291417685f3ad693c5a1d5d", size = 223139, upload-time = "2025-11-30T20:23:00.209Z" }, - { url = "https://files.pythonhosted.org/packages/f2/e1/485132437d20aa4d3e1d8b3fb5a5e65aa8139f1e097080c2a8443201742c/rpds_py-0.30.0-cp313-cp313-win_amd64.whl", hash = "sha256:806f36b1b605e2d6a72716f321f20036b9489d29c51c91f4dd29a3e3afb73b15", size = 240224, upload-time = "2025-11-30T20:23:02.008Z" }, - { url = "https://files.pythonhosted.org/packages/24/95/ffd128ed1146a153d928617b0ef673960130be0009c77d8fbf0abe306713/rpds_py-0.30.0-cp313-cp313-win_arm64.whl", hash = "sha256:d96c2086587c7c30d44f31f42eae4eac89b60dabbac18c7669be3700f13c3ce1", size = 230645, upload-time = "2025-11-30T20:23:03.43Z" }, - { url = "https://files.pythonhosted.org/packages/ff/1b/b10de890a0def2a319a2626334a7f0ae388215eb60914dbac8a3bae54435/rpds_py-0.30.0-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:eb0b93f2e5c2189ee831ee43f156ed34e2a89a78a66b98cadad955972548be5a", size = 364443, upload-time = "2025-11-30T20:23:04.878Z" }, - { url = "https://files.pythonhosted.org/packages/0d/bf/27e39f5971dc4f305a4fb9c672ca06f290f7c4e261c568f3dea16a410d47/rpds_py-0.30.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:922e10f31f303c7c920da8981051ff6d8c1a56207dbdf330d9047f6d30b70e5e", size = 353375, upload-time = "2025-11-30T20:23:06.342Z" }, - { url = "https://files.pythonhosted.org/packages/40/58/442ada3bba6e8e6615fc00483135c14a7538d2ffac30e2d933ccf6852232/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cdc62c8286ba9bf7f47befdcea13ea0e26bf294bda99758fd90535cbaf408000", size = 383850, upload-time = "2025-11-30T20:23:07.825Z" }, - { url = "https://files.pythonhosted.org/packages/14/14/f59b0127409a33c6ef6f5c1ebd5ad8e32d7861c9c7adfa9a624fc3889f6c/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:47f9a91efc418b54fb8190a6b4aa7813a23fb79c51f4bb84e418f5476c38b8db", size = 392812, upload-time = "2025-11-30T20:23:09.228Z" }, - { url = "https://files.pythonhosted.org/packages/b3/66/e0be3e162ac299b3a22527e8913767d869e6cc75c46bd844aa43fb81ab62/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1f3587eb9b17f3789ad50824084fa6f81921bbf9a795826570bda82cb3ed91f2", size = 517841, upload-time = "2025-11-30T20:23:11.186Z" }, - { url = "https://files.pythonhosted.org/packages/3d/55/fa3b9cf31d0c963ecf1ba777f7cf4b2a2c976795ac430d24a1f43d25a6ba/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:39c02563fc592411c2c61d26b6c5fe1e51eaa44a75aa2c8735ca88b0d9599daa", size = 408149, upload-time = "2025-11-30T20:23:12.864Z" }, - { url = "https://files.pythonhosted.org/packages/60/ca/780cf3b1a32b18c0f05c441958d3758f02544f1d613abf9488cd78876378/rpds_py-0.30.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:51a1234d8febafdfd33a42d97da7a43f5dcb120c1060e352a3fbc0c6d36e2083", size = 383843, upload-time = "2025-11-30T20:23:14.638Z" }, - { url = "https://files.pythonhosted.org/packages/82/86/d5f2e04f2aa6247c613da0c1dd87fcd08fa17107e858193566048a1e2f0a/rpds_py-0.30.0-cp313-cp313t-manylinux_2_31_riscv64.whl", hash = "sha256:eb2c4071ab598733724c08221091e8d80e89064cd472819285a9ab0f24bcedb9", size = 396507, upload-time = "2025-11-30T20:23:16.105Z" }, - { url = "https://files.pythonhosted.org/packages/4b/9a/453255d2f769fe44e07ea9785c8347edaf867f7026872e76c1ad9f7bed92/rpds_py-0.30.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:6bdfdb946967d816e6adf9a3d8201bfad269c67efe6cefd7093ef959683c8de0", size = 414949, upload-time = "2025-11-30T20:23:17.539Z" }, - { url = "https://files.pythonhosted.org/packages/a3/31/622a86cdc0c45d6df0e9ccb6becdba5074735e7033c20e401a6d9d0e2ca0/rpds_py-0.30.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c77afbd5f5250bf27bf516c7c4a016813eb2d3e116139aed0096940c5982da94", size = 565790, upload-time = "2025-11-30T20:23:19.029Z" }, - { url = "https://files.pythonhosted.org/packages/1c/5d/15bbf0fb4a3f58a3b1c67855ec1efcc4ceaef4e86644665fff03e1b66d8d/rpds_py-0.30.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:61046904275472a76c8c90c9ccee9013d70a6d0f73eecefd38c1ae7c39045a08", size = 590217, upload-time = "2025-11-30T20:23:20.885Z" }, - { url = "https://files.pythonhosted.org/packages/6d/61/21b8c41f68e60c8cc3b2e25644f0e3681926020f11d06ab0b78e3c6bbff1/rpds_py-0.30.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:4c5f36a861bc4b7da6516dbdf302c55313afa09b81931e8280361a4f6c9a2d27", size = 555806, upload-time = "2025-11-30T20:23:22.488Z" }, - { url = "https://files.pythonhosted.org/packages/f9/39/7e067bb06c31de48de3eb200f9fc7c58982a4d3db44b07e73963e10d3be9/rpds_py-0.30.0-cp313-cp313t-win32.whl", hash = "sha256:3d4a69de7a3e50ffc214ae16d79d8fbb0922972da0356dcf4d0fdca2878559c6", size = 211341, upload-time = "2025-11-30T20:23:24.449Z" }, - { url = "https://files.pythonhosted.org/packages/0a/4d/222ef0b46443cf4cf46764d9c630f3fe4abaa7245be9417e56e9f52b8f65/rpds_py-0.30.0-cp313-cp313t-win_amd64.whl", hash = "sha256:f14fc5df50a716f7ece6a80b6c78bb35ea2ca47c499e422aa4463455dd96d56d", size = 225768, upload-time = "2025-11-30T20:23:25.908Z" }, - { url = "https://files.pythonhosted.org/packages/86/81/dad16382ebbd3d0e0328776d8fd7ca94220e4fa0798d1dc5e7da48cb3201/rpds_py-0.30.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:68f19c879420aa08f61203801423f6cd5ac5f0ac4ac82a2368a9fcd6a9a075e0", size = 362099, upload-time = "2025-11-30T20:23:27.316Z" }, - { url = "https://files.pythonhosted.org/packages/2b/60/19f7884db5d5603edf3c6bce35408f45ad3e97e10007df0e17dd57af18f8/rpds_py-0.30.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:ec7c4490c672c1a0389d319b3a9cfcd098dcdc4783991553c332a15acf7249be", size = 353192, upload-time = "2025-11-30T20:23:29.151Z" }, - { url = "https://files.pythonhosted.org/packages/bf/c4/76eb0e1e72d1a9c4703c69607cec123c29028bff28ce41588792417098ac/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f251c812357a3fed308d684a5079ddfb9d933860fc6de89f2b7ab00da481e65f", size = 384080, upload-time = "2025-11-30T20:23:30.785Z" }, - { url = "https://files.pythonhosted.org/packages/72/87/87ea665e92f3298d1b26d78814721dc39ed8d2c74b86e83348d6b48a6f31/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ac98b175585ecf4c0348fd7b29c3864bda53b805c773cbf7bfdaffc8070c976f", size = 394841, upload-time = "2025-11-30T20:23:32.209Z" }, - { url = "https://files.pythonhosted.org/packages/77/ad/7783a89ca0587c15dcbf139b4a8364a872a25f861bdb88ed99f9b0dec985/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3e62880792319dbeb7eb866547f2e35973289e7d5696c6e295476448f5b63c87", size = 516670, upload-time = "2025-11-30T20:23:33.742Z" }, - { url = "https://files.pythonhosted.org/packages/5b/3c/2882bdac942bd2172f3da574eab16f309ae10a3925644e969536553cb4ee/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4e7fc54e0900ab35d041b0601431b0a0eb495f0851a0639b6ef90f7741b39a18", size = 408005, upload-time = "2025-11-30T20:23:35.253Z" }, - { url = "https://files.pythonhosted.org/packages/ce/81/9a91c0111ce1758c92516a3e44776920b579d9a7c09b2b06b642d4de3f0f/rpds_py-0.30.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:47e77dc9822d3ad616c3d5759ea5631a75e5809d5a28707744ef79d7a1bcfcad", size = 382112, upload-time = "2025-11-30T20:23:36.842Z" }, - { url = "https://files.pythonhosted.org/packages/cf/8e/1da49d4a107027e5fbc64daeab96a0706361a2918da10cb41769244b805d/rpds_py-0.30.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:b4dc1a6ff022ff85ecafef7979a2c6eb423430e05f1165d6688234e62ba99a07", size = 399049, upload-time = "2025-11-30T20:23:38.343Z" }, - { url = "https://files.pythonhosted.org/packages/df/5a/7ee239b1aa48a127570ec03becbb29c9d5a9eb092febbd1699d567cae859/rpds_py-0.30.0-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:4559c972db3a360808309e06a74628b95eaccbf961c335c8fe0d590cf587456f", size = 415661, upload-time = "2025-11-30T20:23:40.263Z" }, - { url = "https://files.pythonhosted.org/packages/70/ea/caa143cf6b772f823bc7929a45da1fa83569ee49b11d18d0ada7f5ee6fd6/rpds_py-0.30.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:0ed177ed9bded28f8deb6ab40c183cd1192aa0de40c12f38be4d59cd33cb5c65", size = 565606, upload-time = "2025-11-30T20:23:42.186Z" }, - { url = "https://files.pythonhosted.org/packages/64/91/ac20ba2d69303f961ad8cf55bf7dbdb4763f627291ba3d0d7d67333cced9/rpds_py-0.30.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:ad1fa8db769b76ea911cb4e10f049d80bf518c104f15b3edb2371cc65375c46f", size = 591126, upload-time = "2025-11-30T20:23:44.086Z" }, - { url = "https://files.pythonhosted.org/packages/21/20/7ff5f3c8b00c8a95f75985128c26ba44503fb35b8e0259d812766ea966c7/rpds_py-0.30.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:46e83c697b1f1c72b50e5ee5adb4353eef7406fb3f2043d64c33f20ad1c2fc53", size = 553371, upload-time = "2025-11-30T20:23:46.004Z" }, - { url = "https://files.pythonhosted.org/packages/72/c7/81dadd7b27c8ee391c132a6b192111ca58d866577ce2d9b0ca157552cce0/rpds_py-0.30.0-cp314-cp314-win32.whl", hash = "sha256:ee454b2a007d57363c2dfd5b6ca4a5d7e2c518938f8ed3b706e37e5d470801ed", size = 215298, upload-time = "2025-11-30T20:23:47.696Z" }, - { url = "https://files.pythonhosted.org/packages/3e/d2/1aaac33287e8cfb07aab2e6b8ac1deca62f6f65411344f1433c55e6f3eb8/rpds_py-0.30.0-cp314-cp314-win_amd64.whl", hash = "sha256:95f0802447ac2d10bcc69f6dc28fe95fdf17940367b21d34e34c737870758950", size = 228604, upload-time = "2025-11-30T20:23:49.501Z" }, - { url = "https://files.pythonhosted.org/packages/e8/95/ab005315818cc519ad074cb7784dae60d939163108bd2b394e60dc7b5461/rpds_py-0.30.0-cp314-cp314-win_arm64.whl", hash = "sha256:613aa4771c99f03346e54c3f038e4cc574ac09a3ddfb0e8878487335e96dead6", size = 222391, upload-time = "2025-11-30T20:23:50.96Z" }, - { url = "https://files.pythonhosted.org/packages/9e/68/154fe0194d83b973cdedcdcc88947a2752411165930182ae41d983dcefa6/rpds_py-0.30.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:7e6ecfcb62edfd632e56983964e6884851786443739dbfe3582947e87274f7cb", size = 364868, upload-time = "2025-11-30T20:23:52.494Z" }, - { url = "https://files.pythonhosted.org/packages/83/69/8bbc8b07ec854d92a8b75668c24d2abcb1719ebf890f5604c61c9369a16f/rpds_py-0.30.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:a1d0bc22a7cdc173fedebb73ef81e07faef93692b8c1ad3733b67e31e1b6e1b8", size = 353747, upload-time = "2025-11-30T20:23:54.036Z" }, - { url = "https://files.pythonhosted.org/packages/ab/00/ba2e50183dbd9abcce9497fa5149c62b4ff3e22d338a30d690f9af970561/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0d08f00679177226c4cb8c5265012eea897c8ca3b93f429e546600c971bcbae7", size = 383795, upload-time = "2025-11-30T20:23:55.556Z" }, - { url = "https://files.pythonhosted.org/packages/05/6f/86f0272b84926bcb0e4c972262f54223e8ecc556b3224d281e6598fc9268/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5965af57d5848192c13534f90f9dd16464f3c37aaf166cc1da1cae1fd5a34898", size = 393330, upload-time = "2025-11-30T20:23:57.033Z" }, - { url = "https://files.pythonhosted.org/packages/cb/e9/0e02bb2e6dc63d212641da45df2b0bf29699d01715913e0d0f017ee29438/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9a4e86e34e9ab6b667c27f3211ca48f73dba7cd3d90f8d5b11be56e5dbc3fb4e", size = 518194, upload-time = "2025-11-30T20:23:58.637Z" }, - { url = "https://files.pythonhosted.org/packages/ee/ca/be7bca14cf21513bdf9c0606aba17d1f389ea2b6987035eb4f62bd923f25/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e5d3e6b26f2c785d65cc25ef1e5267ccbe1b069c5c21b8cc724efee290554419", size = 408340, upload-time = "2025-11-30T20:24:00.2Z" }, - { url = "https://files.pythonhosted.org/packages/c2/c7/736e00ebf39ed81d75544c0da6ef7b0998f8201b369acf842f9a90dc8fce/rpds_py-0.30.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:626a7433c34566535b6e56a1b39a7b17ba961e97ce3b80ec62e6f1312c025551", size = 383765, upload-time = "2025-11-30T20:24:01.759Z" }, - { url = "https://files.pythonhosted.org/packages/4a/3f/da50dfde9956aaf365c4adc9533b100008ed31aea635f2b8d7b627e25b49/rpds_py-0.30.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:acd7eb3f4471577b9b5a41baf02a978e8bdeb08b4b355273994f8b87032000a8", size = 396834, upload-time = "2025-11-30T20:24:03.687Z" }, - { url = "https://files.pythonhosted.org/packages/4e/00/34bcc2565b6020eab2623349efbdec810676ad571995911f1abdae62a3a0/rpds_py-0.30.0-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fe5fa731a1fa8a0a56b0977413f8cacac1768dad38d16b3a296712709476fbd5", size = 415470, upload-time = "2025-11-30T20:24:05.232Z" }, - { url = "https://files.pythonhosted.org/packages/8c/28/882e72b5b3e6f718d5453bd4d0d9cf8df36fddeb4ddbbab17869d5868616/rpds_py-0.30.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:74a3243a411126362712ee1524dfc90c650a503502f135d54d1b352bd01f2404", size = 565630, upload-time = "2025-11-30T20:24:06.878Z" }, - { url = "https://files.pythonhosted.org/packages/3b/97/04a65539c17692de5b85c6e293520fd01317fd878ea1995f0367d4532fb1/rpds_py-0.30.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:3e8eeb0544f2eb0d2581774be4c3410356eba189529a6b3e36bbbf9696175856", size = 591148, upload-time = "2025-11-30T20:24:08.445Z" }, - { url = "https://files.pythonhosted.org/packages/85/70/92482ccffb96f5441aab93e26c4d66489eb599efdcf96fad90c14bbfb976/rpds_py-0.30.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:dbd936cde57abfee19ab3213cf9c26be06d60750e60a8e4dd85d1ab12c8b1f40", size = 556030, upload-time = "2025-11-30T20:24:10.956Z" }, - { url = "https://files.pythonhosted.org/packages/20/53/7c7e784abfa500a2b6b583b147ee4bb5a2b3747a9166bab52fec4b5b5e7d/rpds_py-0.30.0-cp314-cp314t-win32.whl", hash = "sha256:dc824125c72246d924f7f796b4f63c1e9dc810c7d9e2355864b3c3a73d59ade0", size = 211570, upload-time = "2025-11-30T20:24:12.735Z" }, - { url = "https://files.pythonhosted.org/packages/d0/02/fa464cdfbe6b26e0600b62c528b72d8608f5cc49f96b8d6e38c95d60c676/rpds_py-0.30.0-cp314-cp314t-win_amd64.whl", hash = "sha256:27f4b0e92de5bfbc6f86e43959e6edd1425c33b5e69aab0984a72047f2bcf1e3", size = 226532, upload-time = "2025-11-30T20:24:14.634Z" }, - { url = "https://files.pythonhosted.org/packages/69/71/3f34339ee70521864411f8b6992e7ab13ac30d8e4e3309e07c7361767d91/rpds_py-0.30.0-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:c2262bdba0ad4fc6fb5545660673925c2d2a5d9e2e0fb603aad545427be0fc58", size = 372292, upload-time = "2025-11-30T20:24:16.537Z" }, - { url = "https://files.pythonhosted.org/packages/57/09/f183df9b8f2d66720d2ef71075c59f7e1b336bec7ee4c48f0a2b06857653/rpds_py-0.30.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:ee6af14263f25eedc3bb918a3c04245106a42dfd4f5c2285ea6f997b1fc3f89a", size = 362128, upload-time = "2025-11-30T20:24:18.086Z" }, - { url = "https://files.pythonhosted.org/packages/7a/68/5c2594e937253457342e078f0cc1ded3dd7b2ad59afdbf2d354869110a02/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3adbb8179ce342d235c31ab8ec511e66c73faa27a47e076ccc92421add53e2bb", size = 391542, upload-time = "2025-11-30T20:24:20.092Z" }, - { url = "https://files.pythonhosted.org/packages/49/5c/31ef1afd70b4b4fbdb2800249f34c57c64beb687495b10aec0365f53dfc4/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:250fa00e9543ac9b97ac258bd37367ff5256666122c2d0f2bc97577c60a1818c", size = 404004, upload-time = "2025-11-30T20:24:22.231Z" }, - { url = "https://files.pythonhosted.org/packages/e3/63/0cfbea38d05756f3440ce6534d51a491d26176ac045e2707adc99bb6e60a/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9854cf4f488b3d57b9aaeb105f06d78e5529d3145b1e4a41750167e8c213c6d3", size = 527063, upload-time = "2025-11-30T20:24:24.302Z" }, - { url = "https://files.pythonhosted.org/packages/42/e6/01e1f72a2456678b0f618fc9a1a13f882061690893c192fcad9f2926553a/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:993914b8e560023bc0a8bf742c5f303551992dcb85e247b1e5c7f4a7d145bda5", size = 413099, upload-time = "2025-11-30T20:24:25.916Z" }, - { url = "https://files.pythonhosted.org/packages/b8/25/8df56677f209003dcbb180765520c544525e3ef21ea72279c98b9aa7c7fb/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:58edca431fb9b29950807e301826586e5bbf24163677732429770a697ffe6738", size = 392177, upload-time = "2025-11-30T20:24:27.834Z" }, - { url = "https://files.pythonhosted.org/packages/4a/b4/0a771378c5f16f8115f796d1f437950158679bcd2a7c68cf251cfb00ed5b/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_31_riscv64.whl", hash = "sha256:dea5b552272a944763b34394d04577cf0f9bd013207bc32323b5a89a53cf9c2f", size = 406015, upload-time = "2025-11-30T20:24:29.457Z" }, - { url = "https://files.pythonhosted.org/packages/36/d8/456dbba0af75049dc6f63ff295a2f92766b9d521fa00de67a2bd6427d57a/rpds_py-0.30.0-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ba3af48635eb83d03f6c9735dfb21785303e73d22ad03d489e88adae6eab8877", size = 423736, upload-time = "2025-11-30T20:24:31.22Z" }, - { url = "https://files.pythonhosted.org/packages/13/64/b4d76f227d5c45a7e0b796c674fd81b0a6c4fbd48dc29271857d8219571c/rpds_py-0.30.0-pp311-pypy311_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:dff13836529b921e22f15cb099751209a60009731a68519630a24d61f0b1b30a", size = 573981, upload-time = "2025-11-30T20:24:32.934Z" }, - { url = "https://files.pythonhosted.org/packages/20/91/092bacadeda3edf92bf743cc96a7be133e13a39cdbfd7b5082e7ab638406/rpds_py-0.30.0-pp311-pypy311_pp73-musllinux_1_2_i686.whl", hash = "sha256:1b151685b23929ab7beec71080a8889d4d6d9fa9a983d213f07121205d48e2c4", size = 599782, upload-time = "2025-11-30T20:24:35.169Z" }, - { url = "https://files.pythonhosted.org/packages/d1/b7/b95708304cd49b7b6f82fdd039f1748b66ec2b21d6a45180910802f1abf1/rpds_py-0.30.0-pp311-pypy311_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:ac37f9f516c51e5753f27dfdef11a88330f04de2d564be3991384b2f3535d02e", size = 562191, upload-time = "2025-11-30T20:24:36.853Z" }, -] - -[[package]] -name = "rpds-py" -version = "2026.5.1" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version >= '3.12' and python_full_version < '3.14'", - "python_full_version == '3.11.*'", -] -sdist = { url = "https://files.pythonhosted.org/packages/2e/43/25a8dcd3feedd735039a8f0b5b7e3b118232b5eae288c4fd9ab200d41094/rpds_py-2026.5.1.tar.gz", hash = "sha256:07b24fea40541e28570e5b795a4a38fbdcd12550c06bd0748005ecc8116ca256", size = 64459, upload-time = "2026-05-28T12:02:13.232Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/4f/a0/acf8b6fc20bfdcd3a45bd3f57680fb198e157b7e997b9123b10763798bd2/rpds_py-2026.5.1-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:3397a5ed7174dc2786bb214030232fc36fe8e5584fec43a9952cc542b1a12036", size = 355609, upload-time = "2026-05-28T11:58:50.78Z" }, - { url = "https://files.pythonhosted.org/packages/b6/95/f8203fd997484b1690a6869cd0e503b6c3c6be55b0ecc36d1a491fe742f0/rpds_py-2026.5.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:99ab6ba7bfa2cb0f96a04e3652355bf04e3f51aceb1e943b8541dab7ba4828cc", size = 348460, upload-time = "2026-05-28T11:58:52.374Z" }, - { url = "https://files.pythonhosted.org/packages/33/8c/b47326ad2f0be545a5e5c1a55937a12afaea7d392ba2837bb9680f57e6c9/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d0efbe45632665e53e3db8fe1e5692db58fc5cb9bab4459d570b83efefe11164", size = 381031, upload-time = "2026-05-28T11:58:53.775Z" }, - { url = "https://files.pythonhosted.org/packages/22/0b/e83bbd97ffac6f6389b605cd4e1c8ac5761dc7e977769c9255d8c5adb7bd/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:01d17b29c0c23d82b1f4751147ec49cf451f1fc2554eb9ef5f957e55d2656ead", size = 387121, upload-time = "2026-05-28T11:58:55.243Z" }, - { url = "https://files.pythonhosted.org/packages/fd/0e/d285d1bc8864245919c61e1ca82263e4a66d337759c3a4cef72766ff9afc/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7559f72b94ae52659086c595dfa017cde03155f7832071d30959049052cb3ece", size = 501026, upload-time = "2026-05-28T11:58:56.788Z" }, - { url = "https://files.pythonhosted.org/packages/86/06/ccb2109a1e543437b5e43816f2b43b9554cc6783145528a4e3711e05c011/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9e25b7088f9ccbfc0dfcaa52bf969300ca229e10ecf758974ebcbb080a4b37bb", size = 391865, upload-time = "2026-05-28T11:58:58.298Z" }, - { url = "https://files.pythonhosted.org/packages/3d/33/237173db1cfef10105b3839a24de00eb8d2a523711add4632447cdf0aedd/rpds_py-2026.5.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:613fc4ee9eaef26dc5840666214dd6fbcebcf32f46e76f4abc473059f4e13dda", size = 378012, upload-time = "2026-05-28T11:58:59.589Z" }, - { url = "https://files.pythonhosted.org/packages/97/64/1eae54e34d5161f9969295e80bd6b62a55f2b6ac5f2a5b60d02c2140e758/rpds_py-2026.5.1-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:85264a90ff4c05c1568dd65f5921c837614b67c60358fb4c17df3b7f2e90690a", size = 391111, upload-time = "2026-05-28T11:59:01.104Z" }, - { url = "https://files.pythonhosted.org/packages/d8/34/5bb334a5a0f65d77869217c4654f34c78a7d11b93938a3c076a2edeafc52/rpds_py-2026.5.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fe71bca7d547acb17027c7fd1624ff8aae623499c498d3e7011182c4de5c25e0", size = 409225, upload-time = "2026-05-28T11:59:02.433Z" }, - { url = "https://files.pythonhosted.org/packages/16/0f/007ec21283b5b040b4ec3bd95e0402591e22bfa7d5c93dfe01c465c2d2d7/rpds_py-2026.5.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:a05fa4f41f37ec97c9c260441a940450a192f78d774d2b097eee1379f1e1246a", size = 556487, upload-time = "2026-05-28T11:59:04.012Z" }, - { url = "https://files.pythonhosted.org/packages/ff/10/5437c94508169b6b22d8418fef7a66e9ffb5f3b9e9c94460f2eedafe06ff/rpds_py-2026.5.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:df1d2a1996755b24b9ecee92cb4d36c28f86f464a6a173349c26bab41e94b8c2", size = 620798, upload-time = "2026-05-28T11:59:05.485Z" }, - { url = "https://files.pythonhosted.org/packages/e0/d5/9937dce4d6bda74157b954e7d1460db05a22f5929dccfeeba1ed27a93df0/rpds_py-2026.5.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:8895840ac4809e5f60c88fd07617cd71326e73d6e5a8aa783c5c0f7c24985de2", size = 584053, upload-time = "2026-05-28T11:59:06.837Z" }, - { url = "https://files.pythonhosted.org/packages/6c/31/750617dd0ae1752471bf43f9e41d263398fae7cde7849d23b8574a70e617/rpds_py-2026.5.1-cp311-cp311-win32.whl", hash = "sha256:3684a59b158a7683aaeb8e25352e9a9dd2122cec78f2d8530266e4f91b4c7b3f", size = 214390, upload-time = "2026-05-28T11:59:08.402Z" }, - { url = "https://files.pythonhosted.org/packages/3c/bb/3dcab0e1d9516303f2eb672a5d6f62eca5a69e2886301e9c8c54b520c39b/rpds_py-2026.5.1-cp311-cp311-win_amd64.whl", hash = "sha256:7bd530e6a530bb3ea892f194fafa455f3516ac25ecf7143fd33c09be62b0470a", size = 231097, upload-time = "2026-05-28T11:59:09.786Z" }, - { url = "https://files.pythonhosted.org/packages/49/d6/c6bbf5cb1cf12b9732df8074b57f6ef8341ba884c95d40632ae8bddb44e4/rpds_py-2026.5.1-cp311-cp311-win_arm64.whl", hash = "sha256:0a5ae4dbe43c1076983b72616496919872ae7bbe7a1e21cc48336bc3154d130b", size = 226361, upload-time = "2026-05-28T11:59:11.079Z" }, - { url = "https://files.pythonhosted.org/packages/d4/e7/a78582dc57caa592dcc7d4fb69b61390561e908eb3d2f5df5928a8e354c0/rpds_py-2026.5.1-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:3abe24a66e57adcfa645d718063a5fa5103ecc71ddbf26d78af8f9368018ff1d", size = 353040, upload-time = "2026-05-28T11:59:12.531Z" }, - { url = "https://files.pythonhosted.org/packages/a3/43/35e3f136343aef451e545ce8c38d36c2f93c0ed88703db8b64ba2b205c68/rpds_py-2026.5.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:58b1d94308ddf0b1982f61f2eb54bf92997c9ece8a8093ef014250f4a517906c", size = 345775, upload-time = "2026-05-28T11:59:13.827Z" }, - { url = "https://files.pythonhosted.org/packages/20/e1/0f2160c5982d3157734d5cb3ed63d8b2d583a73c9864f77b666449f32cf8/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0fa92420128dadce7f54bd73ba1825a273e9268fe9e35dbf7e6362890efa4e08", size = 376329, upload-time = "2026-05-28T11:59:15.271Z" }, - { url = "https://files.pythonhosted.org/packages/d0/11/ee0ba42aff83bf4effdbc576673c6be64c5e173978c3f6d537e94482f77d/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ca653c6546386227cd9800d1bef6a348099acf8db4250341da6d90f663d6dfcb", size = 383539, upload-time = "2026-05-28T11:59:16.665Z" }, - { url = "https://files.pythonhosted.org/packages/11/df/d94aa6a499d4ac40afe2d7620f2c597fd3c0f182e854ad7cf3f596a81cb6/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:66c93681c4729e4e3ecba31b8179fae083ff3118841672835140338b4b9867c1", size = 494674, upload-time = "2026-05-28T11:59:17.991Z" }, - { url = "https://files.pythonhosted.org/packages/1f/75/33d30f43bb2f458de11979486a591b1bf6e5651765ed1704c6197c2dc773/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:40ff257542e04796880e011e15cd4dc21c2599975df2aaa8f2c8495ca574e1a5", size = 389268, upload-time = "2026-05-28T11:59:19.434Z" }, - { url = "https://files.pythonhosted.org/packages/f4/1e/2c9096fc19d5fd084b0184ca2b651e659aa0a37e6fdbecf6ece47f147fe1/rpds_py-2026.5.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b6825cc329b290e93c5f6a9be2393118a763f6ccf6abd83704e0c102ca583644", size = 376280, upload-time = "2026-05-28T11:59:21Z" }, - { url = "https://files.pythonhosted.org/packages/b9/e5/61ec9f8be8211ea7f48448195549e4aaf02004083475493b0e137702ecb2/rpds_py-2026.5.1-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:de42116e69cb53b911cc34aee5ab98f36c597b822545045d49e938818b99e5e4", size = 387233, upload-time = "2026-05-28T11:59:22.454Z" }, - { url = "https://files.pythonhosted.org/packages/0d/ca/bcec1005c4f4a234f92a29078631fee49206c7265ccae966f18fd332e80e/rpds_py-2026.5.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c0f920015df2a504bebaba6d4c31ccf3fcf942f92655c086da30b671aad19aa6", size = 405009, upload-time = "2026-05-28T11:59:23.845Z" }, - { url = "https://files.pythonhosted.org/packages/72/e6/4d5718c5cf26c522dc7c9999e238da1e77380b81d0c5d1df11e271ddfeb1/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:0408a24e44feb919423dc6d9da677cb5cddb894d2ca9e763967d156d9c60fab4", size = 553113, upload-time = "2026-05-28T11:59:25.184Z" }, - { url = "https://files.pythonhosted.org/packages/d4/25/2ee807bdb3e1f0b7eddf7782acd5665a8b5205a331a7d7244a52c4812fd9/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:cea68bcd53467561ae2f96a6bdad1544299ba97b5b0ddcd5ac3d376e5c781c24", size = 618838, upload-time = "2026-05-28T11:59:26.749Z" }, - { url = "https://files.pythonhosted.org/packages/6a/c1/7d4c26f167f8c41501cc073d30ee22082b16ce358cf5b00ec97cbc7804ea/rpds_py-2026.5.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:4be8b1d2a705cc37d08256004e1d07de143fa0075c8e85a3df020b776f62b732", size = 582436, upload-time = "2026-05-28T11:59:28.11Z" }, - { url = "https://files.pythonhosted.org/packages/04/1d/9d12b0a337bab46f4769f8857f4007e3b2d639e14f9a44a0efe157696e64/rpds_py-2026.5.1-cp312-cp312-win32.whl", hash = "sha256:6736718bd4fc49cbcb538ba30516fdbef161522acefb739657d48b97bd864fed", size = 212734, upload-time = "2026-05-28T11:59:29.689Z" }, - { url = "https://files.pythonhosted.org/packages/c5/93/e4116f2de7f56bc7406a76033dc501811ddeb22b7f056b92d632871ebb0c/rpds_py-2026.5.1-cp312-cp312-win_amd64.whl", hash = "sha256:0a7d1eec967df0e9b22614a5e177622e0c89611d03727fa0cb48e45028907870", size = 229045, upload-time = "2026-05-28T11:59:31.033Z" }, - { url = "https://files.pythonhosted.org/packages/cb/53/6c3419d85eb2ec5938a37627c585b42d76a63bb731d6e42ed4b079ebf486/rpds_py-2026.5.1-cp312-cp312-win_arm64.whl", hash = "sha256:1841d067089e117142d79b98aa0df2f08b52f2ecc1819dd2700636c0db74a473", size = 223967, upload-time = "2026-05-28T11:59:32.318Z" }, - { url = "https://files.pythonhosted.org/packages/6c/32/14c961ad295f490eb0849ada8b79683e93a59b9de3afdd983eaf55fa6867/rpds_py-2026.5.1-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:efef4ac29c6ff495531eb17ee705b62841ecaa291b7c7077e848ea03e237164d", size = 352787, upload-time = "2026-05-28T11:59:33.655Z" }, - { url = "https://files.pythonhosted.org/packages/ca/bb/d1b85117967c11191441a7274ae616c65d93901d082c588f89a50a8da5ae/rpds_py-2026.5.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c39f5b67a8a2e67179ada2a954227d670fe65fa9098457f698f56ddf248709b3", size = 345179, upload-time = "2026-05-28T11:59:35Z" }, - { url = "https://files.pythonhosted.org/packages/7c/46/d84105f062e626a1b233f863907288a4708c2d833b8b4c6fb2764bc080c0/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b5c30f3f04eef4fbd362226a6f31d7c8895ca4fbb6e0b790f6890a98d8da8559", size = 376173, upload-time = "2026-05-28T11:59:36.43Z" }, - { url = "https://files.pythonhosted.org/packages/e2/ae/469d7959ce5b1201e1de135dc735b86db3b35dd0d1734f6a44246d5f061c/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:277f6c82f0580848796c7ecc8a7173aa3bfb928e4ff831261c2f60a81dc270db", size = 383162, upload-time = "2026-05-28T11:59:37.995Z" }, - { url = "https://files.pythonhosted.org/packages/dc/a2/57853d31a1116a561aa072794602ad3f6341e18d70a8523f1bd5b9fc1e5a/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:63c2c4c213f1a4e3f3de28ecab029dbdee976324e729c0d7a55211be72576b02", size = 495093, upload-time = "2026-05-28T11:59:39.453Z" }, - { url = "https://files.pythonhosted.org/packages/99/63/3a8eabcad9314b7daf5c65f451d2c33d989235cd8a5762186cf2c3f5a4f8/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3350ec808fb538fe71a1f94dfaa0e29c598dfad805ce49f0caec5ae3183c652b", size = 389829, upload-time = "2026-05-28T11:59:40.896Z" }, - { url = "https://files.pythonhosted.org/packages/4b/25/05678d97fc25e2622df14dc530fb82023174ecfff6733991ed0d78f167bd/rpds_py-2026.5.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b1b964e3ab599e718dc46c018d104b1ebc007cbc6567d827c94a687fca56d77e", size = 374786, upload-time = "2026-05-28T11:59:42.626Z" }, - { url = "https://files.pythonhosted.org/packages/88/d1/8c90b6431e80a3b91b284a5c7c8c0c4f9c006444d90477a740d6e0f9c694/rpds_py-2026.5.1-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:19cb09fab7b7fc96b2a6e28f2e34b72a3705ff27b37edb77455316e5d3f3dc9b", size = 386920, upload-time = "2026-05-28T11:59:44.124Z" }, - { url = "https://files.pythonhosted.org/packages/ff/99/4638f672ab356682d633ee0da9255f5b67ce6efd0b85eb94ad3e255e65a5/rpds_py-2026.5.1-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:abe76bcdba31e576cb83eeb8797aa0d882b738fef6dc65d0601fc753806a5b46", size = 405059, upload-time = "2026-05-28T11:59:47.177Z" }, - { url = "https://files.pythonhosted.org/packages/66/3f/3546524b6eb4cc2e1f363a3d638fa52f6c24faae3500c25fb488b02f1740/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:8bff7073db3899158fff55ebf57b113a67030af26f80a18978f9f0aa60250ddf", size = 553030, upload-time = "2026-05-28T11:59:48.603Z" }, - { url = "https://files.pythonhosted.org/packages/c6/c3/7b3388c796fcf471bd17194242d4dc1a7608567c0fa422bcc1c5e79f9c1e/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:8ba264fa49be666cd9cc56bf34ec7002fb3d27a4aee5bcb4d43d0d18feb1bb6f", size = 618975, upload-time = "2026-05-28T11:59:50.314Z" }, - { url = "https://files.pythonhosted.org/packages/61/1e/a3cb07f2795075d1d88efddae2f541359fde5f08c81ee114c29c2949c90a/rpds_py-2026.5.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:4860b603ddda0475a8885499b3729e90229d480105b42651962a5397d995fa89", size = 581178, upload-time = "2026-05-28T11:59:51.673Z" }, - { url = "https://files.pythonhosted.org/packages/a1/74/e758c03a5ef46f04c37f2651a2893db846d569ba8a7bca469d4b58939bcd/rpds_py-2026.5.1-cp313-cp313-win32.whl", hash = "sha256:7944270ae71383f6e2657dd7d5ce4eeb4ac2d0059a6738f0510583d462ab4842", size = 212481, upload-time = "2026-05-28T11:59:53.148Z" }, - { url = "https://files.pythonhosted.org/packages/70/ec/a2aca432db9c7359b40fa393eeeaa0d166c2f70175be956e75fa24197c44/rpds_py-2026.5.1-cp313-cp313-win_amd64.whl", hash = "sha256:88647f43a73c4e01be19b04ceef0c8d3a1958153604d13c773becd8016f2a0cf", size = 228519, upload-time = "2026-05-28T11:59:54.505Z" }, - { url = "https://files.pythonhosted.org/packages/29/60/a73bfdd45b096574556acf303bbd9fa9eed36ca8a818b514e2a5d5fe2b9d/rpds_py-2026.5.1-cp313-cp313-win_arm64.whl", hash = "sha256:453895624ecf7db7063b1004e44037522bbaef9ff6a945e59bc71662d7a03abd", size = 223446, upload-time = "2026-05-28T11:59:56.081Z" }, - { url = "https://files.pythonhosted.org/packages/18/e2/408105fd611823f00882aea810f3989a30d26b1bab8b6beb20f98c724e0e/rpds_py-2026.5.1-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:b4e4bc98639ec915f512fde3aa7a95e0041d95d9c3cc86eea841fa63cb1e8600", size = 355287, upload-time = "2026-05-28T11:59:57.448Z" }, - { url = "https://files.pythonhosted.org/packages/8d/58/5c4a43436843c90d0f6d19f82c200c80e3843ca9fa07b237623327f6d384/rpds_py-2026.5.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:cacedb7a6e167680acba45ad5716e89067d225dc80da0d7040cae8c81d4572fa", size = 347033, upload-time = "2026-05-28T11:59:58.881Z" }, - { url = "https://files.pythonhosted.org/packages/fb/c2/1a71acdacaf4e259b10278fb87b039ded3cf80041bcd89dd8a3ea702ded6/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:68700371c5d7ae1412862ddfa719090925c93ecf351c566d66f09d04b136ea00", size = 376891, upload-time = "2026-05-28T12:00:00.516Z" }, - { url = "https://files.pythonhosted.org/packages/c2/c8/535f3d9b65addd8e28aa87b83c6e526799c3717a88273db8ea795beeef7a/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:296c799becfa849c779c8725494fe9ed94959ed886787df4364b058465bad7f0", size = 385646, upload-time = "2026-05-28T12:00:02.394Z" }, - { url = "https://files.pythonhosted.org/packages/1c/91/dc033f313345c354ade914dbe73cdb90b615a4409ea02430d5356794f3d8/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d3858b908218ee108d0bbfb2095ccc237648053c9bf98affad7cb079acaf1d97", size = 498830, upload-time = "2026-05-28T12:00:04.189Z" }, - { url = "https://files.pythonhosted.org/packages/27/fc/90fcbea459dbb8ddc18a2e0fd1de9412b48bc84ffff2db771cf714bacfd6/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4fb8d2e7cb2f850b169806d61d1b991738acec96500a75c30f49caf064ce7cef", size = 392830, upload-time = "2026-05-28T12:00:05.797Z" }, - { url = "https://files.pythonhosted.org/packages/b2/1d/46cd11a228c9750684a798d98f878be6f614aa762438da7378f035e79e35/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:27b74c10ed6a8f190f4287f53bcfea348b92a84a9c9f70d30183d1e6172d580d", size = 379613, upload-time = "2026-05-28T12:00:07.433Z" }, - { url = "https://files.pythonhosted.org/packages/24/4a/d9b0c6af3a1de03eb93741bbe8be2bdce84d8fda8224f3005451d86df389/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_31_riscv64.whl", hash = "sha256:b9a6528956191c48c52294a592dbd4a8386d7048bdb25c0efcb6b966466c6d83", size = 388183, upload-time = "2026-05-28T12:00:09.227Z" }, - { url = "https://files.pythonhosted.org/packages/c5/b4/db7aaabdda6d020afc87d981bcc2f57a434c7dec60ecfc2ab3dd50b20351/rpds_py-2026.5.1-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:af03e34e860047bc7a352b842856fcf78798fbb81132cc98bd2f907ab4eb9cd2", size = 408578, upload-time = "2026-05-28T12:00:10.779Z" }, - { url = "https://files.pythonhosted.org/packages/08/d6/070f6a41cbb343e2ac4171859bf3f3623e0ab002f72619d6d505313ec2de/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:fea6e836d10abbe191d557d33bd58bd5987725fe63aa1eefe557d230209855bd", size = 553573, upload-time = "2026-05-28T12:00:12.443Z" }, - { url = "https://files.pythonhosted.org/packages/75/ab/1a71ea3589c4345dac0a0518f0e6a031cb42689277851b683c46d27463a5/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:fc0c0f878ea770a0a8a462456c5ad36fc9fe6358e6b76fdadc7f17575e0b8bf1", size = 620861, upload-time = "2026-05-28T12:00:14.09Z" }, - { url = "https://files.pythonhosted.org/packages/8a/22/9bf80a56069c0c443fcfefac639a86a744550a2898817a6dfd3e26654924/rpds_py-2026.5.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:e0b360f316d966b048b085857630b3cc51f3db2f07b06f440eac8f695374d1e3", size = 585633, upload-time = "2026-05-28T12:00:15.66Z" }, - { url = "https://files.pythonhosted.org/packages/da/68/3b2c0a75c9e04125696f84ebdbbf304acf5a40b58ba4481cdb98a922c3ba/rpds_py-2026.5.1-cp313-cp313t-win32.whl", hash = "sha256:a2999883eedf72fdfb7520b92c7d4ec2572a71ff40239377aa604cc529eecafc", size = 210074, upload-time = "2026-05-28T12:00:17.291Z" }, - { url = "https://files.pythonhosted.org/packages/e7/8b/609157d5a25d37d4f29f92840ba531f416907c34ae5c5739dd21fc2bef98/rpds_py-2026.5.1-cp313-cp313t-win_amd64.whl", hash = "sha256:e07be2a9d7122bd6e82dea89814ef8dc893feb1aae97fec1630f3263bbb30e55", size = 228635, upload-time = "2026-05-28T12:00:18.73Z" }, - { url = "https://files.pythonhosted.org/packages/d4/6f/19c1918a4b590d8de87e712e4abe4b3875771eff60216fb6153cf6665c68/rpds_py-2026.5.1-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:1f2c391c3059798093b65df23aca2cac150460ae9c630d99dec83d703d9485b9", size = 349756, upload-time = "2026-05-28T12:00:20.217Z" }, - { url = "https://files.pythonhosted.org/packages/e5/60/a06fe7da34eca79dacbf958a2ba0c6eea85bc2b29de20080bf40f72f66fa/rpds_py-2026.5.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:413b424f7c4ee65ab5e5be91f5731be0f8b41a1ee2b12dfe810d716312e95a78", size = 343831, upload-time = "2026-05-28T12:00:21.711Z" }, - { url = "https://files.pythonhosted.org/packages/bf/ec/b2333b97b90e2a6ef6ca8ad386ee284968e74bcfe113b3f1a8d9036429a9/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2c595a1d9255dce0599e13130d1440ab2506654f2b50294226ee06402f8fef63", size = 375127, upload-time = "2026-05-28T12:00:23.326Z" }, - { url = "https://files.pythonhosted.org/packages/14/7f/e00aae54067f2b488c4637961d5f58204d470795fc791085fa3f15060d2e/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:1c27c5f6102eac8c03e7595a00827a53b271ba40a53b59ff8709170e0855ea4a", size = 379034, upload-time = "2026-05-28T12:00:24.89Z" }, - { url = "https://files.pythonhosted.org/packages/be/cc/423999bbb8ae8dc93c77fc1d5e984ade5eb89d237d3bb884ccfa72ae2890/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6c7fcf61d44cacecaf3aea542b0e053db77972a4573e7ceda16fb2b399161195", size = 490823, upload-time = "2026-05-28T12:00:26.676Z" }, - { url = "https://files.pythonhosted.org/packages/0f/aa/c671bf660f12e68d3c52ff86c7066ed1372df5a0f4f2ff584e419b8207e7/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2c817a189d4ee14290420e5ff051e4dd6baa13f3edf84685071dee07a6d538ee", size = 388144, upload-time = "2026-05-28T12:00:28.577Z" }, - { url = "https://files.pythonhosted.org/packages/19/c8/d63bb75b68afe77b229e3021c6031bcaf01da5db5b0e69d0d10f9ba679a7/rpds_py-2026.5.1-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:21846aac0ed2e0589f38c12dc44e77bb64e494b771eadbcf169cba00566ba7ba", size = 371959, upload-time = "2026-05-28T12:00:30.304Z" }, - { url = "https://files.pythonhosted.org/packages/82/35/c51122014d8274ff37dc606d60049c3db7d83da02b5b282511e5a906a9a6/rpds_py-2026.5.1-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:b317c87a13f769a4e787819bd508aaa5d69aa09b0880de9af6d3a8a54571cdec", size = 383558, upload-time = "2026-05-28T12:00:31.764Z" }, - { url = "https://files.pythonhosted.org/packages/e3/f9/2790cb99c136a5363acdeacf5c27c56f3de0d4118a1f48fca83404c99c89/rpds_py-2026.5.1-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ce87129d9f2c14fa6c4a8601fb80eb4488c80d38a20cd13758ef11123e14995d", size = 402789, upload-time = "2026-05-28T12:00:33.247Z" }, - { url = "https://files.pythonhosted.org/packages/e5/1b/e4fb584f8c75d35c38150ff6a332cda949e6f97acba1f4fd123b14ab56fe/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:9cdddb6c1207d284d94fd1530adf57fbd797fe7c4b8704ba85f49414f2557e7d", size = 551405, upload-time = "2026-05-28T12:00:34.819Z" }, - { url = "https://files.pythonhosted.org/packages/d8/f7/a6731b4216cb3793ea1af5391da240f5683dacc0d13e034fe5fc3503f240/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:4e237e139f94d3c036fd28eb9f564c99055476ff4ff05cd42be55ce349b5aa02", size = 616975, upload-time = "2026-05-28T12:00:36.268Z" }, - { url = "https://files.pythonhosted.org/packages/2c/ea/2e051a81d95d8e63f4b35a1c463a87e8766bc3d083c067c5dfb6bf220747/rpds_py-2026.5.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ed0954b524873214369184a9c82b0eaa45a3fbb9a798cd95b17e0d98499e7ea0", size = 578701, upload-time = "2026-05-28T12:00:37.82Z" }, - { url = "https://files.pythonhosted.org/packages/65/56/b5f6fdb2083e32bca8a8993d89e70db114b4756c9e2c38421328126689d2/rpds_py-2026.5.1-cp314-cp314-win32.whl", hash = "sha256:2d88621d6a7d4dfa633d21abe90f280bb205274e16b1d1e61c6ad4640b2453b7", size = 209806, upload-time = "2026-05-28T12:00:39.492Z" }, - { url = "https://files.pythonhosted.org/packages/fb/80/65a5aa96c155e611d1ed844e4e1f57f3e36b021f396d9f8585d756e6b90d/rpds_py-2026.5.1-cp314-cp314-win_amd64.whl", hash = "sha256:cef8ac28d26f4dda3533060c20fbf80a325458fa9fd23ea72a73cdfa8e978838", size = 225985, upload-time = "2026-05-28T12:00:40.94Z" }, - { url = "https://files.pythonhosted.org/packages/27/7c/ad185212e87b05f196daef92bc5f3caf07298eb47c295b5585c3dd3093ac/rpds_py-2026.5.1-cp314-cp314-win_arm64.whl", hash = "sha256:eaaea962c68cdc68d4a533ba985ab8e9484277910bbfaa2ab3ef7732667bfed8", size = 221219, upload-time = "2026-05-28T12:00:43.15Z" }, - { url = "https://files.pythonhosted.org/packages/23/58/e14ae18759020334646b031e708ab4158d653a938822bfb7b95ef2e93aa3/rpds_py-2026.5.1-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:21942f52dbbd5f8758bf021213d28bd45c39e873e65e2407faf5f1846f5761ad", size = 352148, upload-time = "2026-05-28T12:00:44.638Z" }, - { url = "https://files.pythonhosted.org/packages/31/9b/5f4a1e2f960bca3ac5d052b139dd31eed97b259f9d909173821760d542e8/rpds_py-2026.5.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f414556f6e3958300ff941e40c9f97e3dc9774ddd1b3434c475d73dd354bbed3", size = 345196, upload-time = "2026-05-28T12:00:46.14Z" }, - { url = "https://files.pythonhosted.org/packages/1a/71/1d9574d6a2fa20ab60eaa55c7467f5aa20cbc770f341a05f09c0876f59e2/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ef1013a8625c74043210190b246f5b1551e09757c1f356c6e4160ef96c5bc081", size = 374981, upload-time = "2026-05-28T12:00:47.531Z" }, - { url = "https://files.pythonhosted.org/packages/0c/9a/37e99f4915a80aa71670263c1267f7ae0af95f53a3f61e6c3bdc016d4515/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:cc68e231a77a5f0d774ae278a1f8e55c0456501820847c1e4efb3829f3441df6", size = 379961, upload-time = "2026-05-28T12:00:49.216Z" }, - { url = "https://files.pythonhosted.org/packages/a8/ff/6e73f74b89d2e0715e0fc86b7dde893f9a61ae2f9b256ff3bdfe41ac4e94/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9baffb505aff33acc69b422a19f77806680f3c8632227d79f48de8a810d1c2c5", size = 495965, upload-time = "2026-05-28T12:00:51.111Z" }, - { url = "https://files.pythonhosted.org/packages/ea/e0/425faba25f59d74d4638b267f7c7a80e8649d2ef4db10a19b0c4a71e6e6f/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b8d2f912928d426e8cfa396f7f3f8d29a59e6689c86dcca3c420730c1096322b", size = 389526, upload-time = "2026-05-28T12:00:52.77Z" }, - { url = "https://files.pythonhosted.org/packages/c6/76/7a41960e3fddae47fab43a28684d5da981401dffd88253de0944148654cb/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:90f628283be835db980c941767d41c9a27b5239e54ba0a9c1335247e82406964", size = 376190, upload-time = "2026-05-28T12:00:54.215Z" }, - { url = "https://files.pythonhosted.org/packages/27/60/5f38dc70824fc6951b51d35377e577a3a3a4c81a6769cc5a2de25ebe0ad1/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:1ebb2f0ab7e16132995a72de805170e0203df0c3dd22e1ef1cd1fdd90bd7a131", size = 383921, upload-time = "2026-05-28T12:00:55.673Z" }, - { url = "https://files.pythonhosted.org/packages/60/1a/d60a38caa1505f4b9483c3fbbde12c94e1079154f4f401a6da96f7e77621/rpds_py-2026.5.1-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:f3df3d16ded76f1f8c9cdebd0e1ea55fdf4c23b812de189814da7cf229c22a81", size = 404766, upload-time = "2026-05-28T12:00:57.518Z" }, - { url = "https://files.pythonhosted.org/packages/87/ff/602fd3f174d6425f0bce05ad0dfbec0e96b38d0f7d08a79af5aa20083885/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:9af8905b8f854990e40d5206aa5ac58d9b0fe0b7f351ff2bb086c20f6c8c6a47", size = 551343, upload-time = "2026-05-28T12:00:58.978Z" }, - { url = "https://files.pythonhosted.org/packages/b8/c1/1be13327acdbead3eca1fde03b6a34dbb011f1e864e217f0d32cc1779a7f/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:036a36a87fb1cd3b214d11c4b3c4f7d2ddad933625dca1c900b56a057c07740a", size = 618502, upload-time = "2026-05-28T12:01:00.656Z" }, - { url = "https://files.pythonhosted.org/packages/f3/d7/afb49b49d7f2be8b7ba1a9f0977fa5168003437b93086726f066544e8351/rpds_py-2026.5.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:62ae3853454fe9ef283a03c96c2d835d39e84b14643a9d62c82ef0fb87d702ca", size = 581916, upload-time = "2026-05-28T12:01:02.22Z" }, - { url = "https://files.pythonhosted.org/packages/25/d1/dbef8c1f8a10f07beb62b5f054e20099fd9924b3ec001b8f0b6ac7813a85/rpds_py-2026.5.1-cp314-cp314t-win32.whl", hash = "sha256:6c3d771a46ec18b12af06ce36243a9a80b07a5d0515236332d90863ca8bb326a", size = 207855, upload-time = "2026-05-28T12:01:03.821Z" }, - { url = "https://files.pythonhosted.org/packages/2a/72/bfa4e61ab8e7dc1c8adf397e05e6cbdd4239357bd72b248d3de662f23915/rpds_py-2026.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:c93c629be4636cf54337bd5f06c104d55e42ced54d681f6fe21ae510a65116f6", size = 225422, upload-time = "2026-05-28T12:01:05.194Z" }, - { url = "https://files.pythonhosted.org/packages/27/3a/7b5da92b640f67b6717ccafc83cdd06bfa7ff2395c3685c68922bb54d703/rpds_py-2026.5.1-cp315-cp315-macosx_10_12_x86_64.whl", hash = "sha256:3574b55c604b8f75dacb007136508bbc0db406e626301778096a133327e7f2fb", size = 349576, upload-time = "2026-05-28T12:01:06.722Z" }, - { url = "https://files.pythonhosted.org/packages/d7/8a/2aafd7ad355a1bd48ca76e2262b74b15e6432b5a1efe150efd4d779cd55d/rpds_py-2026.5.1-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:94068eb3ae6d43f5a786b7db96a406a34e6d5c24489feef32fd6e8946ea7b291", size = 343640, upload-time = "2026-05-28T12:01:08.441Z" }, - { url = "https://files.pythonhosted.org/packages/f7/7d/6c9523c1abbe840a1b7fba3c516d48e1d3487cc80fea4366c4071cf56784/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f3a5b10e8ce894825f380a8f1b6444cf73c294dfea62afbb2d13e3a9e630cec1", size = 375322, upload-time = "2026-05-28T12:01:09.934Z" }, - { url = "https://files.pythonhosted.org/packages/5a/5d/0b7b03fb1dc509321f01de3149784ab773e34c8573022029af8076afcb9c/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:fc09f82e63d4bcd58149572f857a431bae851dc747e313c3b5bdf7abb907fda8", size = 379066, upload-time = "2026-05-28T12:01:11.48Z" }, - { url = "https://files.pythonhosted.org/packages/d7/e2/8ef6012999ebf1cb1c22f876d9ce5e63d960fd4631d2af3202d3f480aa25/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e10464d17df3b582745c25cec695cb9558bca2cb6ddb631aee1787fc72c767b2", size = 494586, upload-time = "2026-05-28T12:01:13.051Z" }, - { url = "https://files.pythonhosted.org/packages/80/af/1eeb029bec67582c226b7809172207cd005073af4ebd906e65ff494f4983/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ba05adbf15d994c38ec0b7ab32e858e5110c21e9009a00a86545fd220f84e038", size = 388415, upload-time = "2026-05-28T12:01:14.631Z" }, - { url = "https://files.pythonhosted.org/packages/18/23/ffbe10711c4d766c1cab0557d6906c074f795814863c67b351355d29354a/rpds_py-2026.5.1-cp315-cp315-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:77c004fdc7b891967106f78ddfd7b076bfe6813c6139c6fff6aed3bcaa960b26", size = 372427, upload-time = "2026-05-28T12:01:16.153Z" }, - { url = "https://files.pythonhosted.org/packages/bd/3a/30ba4a6ad457e5b070c18d742a33fb77d8d922b565cc881f8a5313d63bfe/rpds_py-2026.5.1-cp315-cp315-manylinux_2_31_riscv64.whl", hash = "sha256:83bcf894486c9d78dd290d3c0124ff6dd8875d3025e2090a8ec49fcc37c55fdd", size = 383615, upload-time = "2026-05-28T12:01:17.809Z" }, - { url = "https://files.pythonhosted.org/packages/d3/69/62e242b53ce39c0814bd24e1a6e6eba6c92be716277745f317f9540a2e7b/rpds_py-2026.5.1-cp315-cp315-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c3df104083952a0e0c6f10de33e440eabe98fb6317d23e1a58c68f6df08d01b9", size = 402786, upload-time = "2026-05-28T12:01:19.419Z" }, - { url = "https://files.pythonhosted.org/packages/38/c1/a770b9c186928a1ed0f7e6d7ae50e7f3950ed23e3f9e366dbc8e38cb55de/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:980450826cf22e133c57e0835070bdd0dd3f73b9b708c3ce223def2cb9469e14", size = 551583, upload-time = "2026-05-28T12:01:21.013Z" }, - { url = "https://files.pythonhosted.org/packages/21/7c/68e8579b95375b70d2a963103c42e705856cdb98569258bd807f4423891c/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_i686.whl", hash = "sha256:205dde846f24332ab0c1188699a043b8d165b79bb84529ce272c45048ff6be01", size = 616941, upload-time = "2026-05-28T12:01:22.548Z" }, - { url = "https://files.pythonhosted.org/packages/70/a1/a6135aed5730ff03ab957182259987ac11e55fb392a28dc6f0592048a280/rpds_py-2026.5.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:3966b82dd563176396df030f3dd52a6e54cb69b718e95e78bd555ed3d1e0185d", size = 578349, upload-time = "2026-05-28T12:01:24.118Z" }, - { url = "https://files.pythonhosted.org/packages/09/6e/f24201a76a84e6c49d0bdfdfcb735210e21701e9b21c5bfc0ba497dd62f6/rpds_py-2026.5.1-cp315-cp315-win32.whl", hash = "sha256:7818f8d0a415be74d2be3590b0a1c1f463a642f4d0217e7d10602dceef5b79aa", size = 209922, upload-time = "2026-05-28T12:01:25.522Z" }, - { url = "https://files.pythonhosted.org/packages/9e/e4/966bc240bb0485fc265278f6de44d05834bf0b3618886e0b22e33d54c49a/rpds_py-2026.5.1-cp315-cp315-win_amd64.whl", hash = "sha256:b3cc20c0d800af78fd0fac68086e28c1856cec51ea528bb81ea851aa40d39325", size = 226003, upload-time = "2026-05-28T12:01:27.062Z" }, - { url = "https://files.pythonhosted.org/packages/5c/5c/a15a59269cd5e74472734516c73795c15eccfc841b3d4b0228c3f53f19d0/rpds_py-2026.5.1-cp315-cp315-win_arm64.whl", hash = "sha256:3609e9939a8a76cd904cf98a3f1f13b5dc7e150adeaee89e0ea09652ea213e16", size = 221245, upload-time = "2026-05-28T12:01:28.51Z" }, - { url = "https://files.pythonhosted.org/packages/e0/22/135ce03804e179a71ceb13be095deda4a279bc88f7a6b8fa161c5ad44e12/rpds_py-2026.5.1-cp315-cp315t-macosx_10_12_x86_64.whl", hash = "sha256:5d333a7127d4b307601ac37792bee01bb95c867cbfacf21b6375b804d6bbd723", size = 352015, upload-time = "2026-05-28T12:01:30.214Z" }, - { url = "https://files.pythonhosted.org/packages/3b/5f/f1f6d2652eb9d848f6eb369d8db83a2da6249bb49ad2c2a48f45d54538d3/rpds_py-2026.5.1-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:b5f077b44a4f7808520f66dae234988d867deb9aed9be5da057ce9ba831b2a41", size = 345016, upload-time = "2026-05-28T12:01:31.656Z" }, - { url = "https://files.pythonhosted.org/packages/88/66/b74182775691ea2290c99e52ac8d5db844e56fbec90ce421f107658c8314/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:55d8f9b7b78c9538fc9e04e82ec0e888ff0c3cffcfad152c77e57cd09351a98a", size = 374775, upload-time = "2026-05-28T12:01:33.136Z" }, - { url = "https://files.pythonhosted.org/packages/ff/8f/15e5a61d9f0a43902d36561d4f07cae6ae9f4716be825159fd72717f33af/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e3a8ae58895ac107ed934a6bf51e5846f95c53b9b940c2c6d310838fd5846358", size = 380270, upload-time = "2026-05-28T12:01:34.574Z" }, - { url = "https://files.pythonhosted.org/packages/02/c3/f859b12763a80540cdf2af0f15b19904cf756a71d7bdd3f82ff3e5b1bbf9/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0957cf3c2b8632ec7aaebffebea8005b353cc2a237b6e2ae3c2cac0820704cfb", size = 495285, upload-time = "2026-05-28T12:01:36.127Z" }, - { url = "https://files.pythonhosted.org/packages/1c/c7/ff27c2ac8411d30b03b1829fd88cae8dad1a4d0da48dd25e57c4038042e6/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c396c1304de421050b3681ea70f371874b54d41b0151e96109758144c231e30b", size = 389581, upload-time = "2026-05-28T12:01:37.635Z" }, - { url = "https://files.pythonhosted.org/packages/6e/67/fe92ee32a6cc05c77228a2f8b1762e7124f386ec20ff83d0757b762d58d0/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aad1bff7f666b9598e573815affd666aac6a13a585dde336f843e33350c7fadc", size = 376041, upload-time = "2026-05-28T12:01:39.307Z" }, - { url = "https://files.pythonhosted.org/packages/f8/91/b4d6685c27aba55bd82f25b278be8237038117d05f9659a6213ad3408130/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_31_riscv64.whl", hash = "sha256:656a042550878f12d45752452d47094b7cfe5ad1e9d7b87b5a22ad3ae5ff8015", size = 383946, upload-time = "2026-05-28T12:01:41.043Z" }, - { url = "https://files.pythonhosted.org/packages/bd/79/2c1d832a53c8e0f8e98fc970ec257b950fecd4f62be2ab7182b500a0cbc8/rpds_py-2026.5.1-cp315-cp315t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:73c4bd4f70294737b5206a3e8e30ccadbf8a60301831c8ea23eec5dbeea1ecfa", size = 405526, upload-time = "2026-05-28T12:01:43.032Z" }, - { url = "https://files.pythonhosted.org/packages/78/c4/c98117b03c6a8581ab2c2dfccfe9a5ad82bd8128a3c28b46a6ad2d97c393/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:43bca78665423cabae77146f2fe7ce55272b6c8d55d82cca83effd42c7e13972", size = 551165, upload-time = "2026-05-28T12:01:44.648Z" }, - { url = "https://files.pythonhosted.org/packages/3b/c1/bc479ca069200af730881b1bd525e3114b2b391a351509fcb1b772f28086/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_i686.whl", hash = "sha256:42d0f20e85e549c870749d0e247f0c10d318a45b7e9676d575d2dcb04a1b2e66", size = 618778, upload-time = "2026-05-28T12:01:46.337Z" }, - { url = "https://files.pythonhosted.org/packages/77/65/38ab2f90df44c2febfb63cc10ced40763d9b4bc94d173e734528663fe7f5/rpds_py-2026.5.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:b1be5c35683684d5331b93600c210e8367c254683d8a6df6bd21bd2da3a334fb", size = 581839, upload-time = "2026-05-28T12:01:48.109Z" }, - { url = "https://files.pythonhosted.org/packages/15/2d/ce1f605fe036aadd460e5822e578c6c7ec3a860936cca37d6e0f299daa77/rpds_py-2026.5.1-cp315-cp315t-win32.whl", hash = "sha256:75808f6c38ce7749bb68cc2770161aae5045e6c6f6781a9782e74b93304399df", size = 207866, upload-time = "2026-05-28T12:01:49.648Z" }, - { url = "https://files.pythonhosted.org/packages/79/cb/966040123eb102371559746908ef2c9471f4d43e17ec9a645a2258dab64b/rpds_py-2026.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:90bd6630002a1c7f09e7843dd79f0d24f3d2897cc25a753480917865d14f15b3", size = 225441, upload-time = "2026-05-28T12:01:51.408Z" }, - { url = "https://files.pythonhosted.org/packages/42/56/3fe0fb34820ff667be791b3a3c22b85e8bcba54e9c832f47438c191fa7be/rpds_py-2026.5.1-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:edf2765d84e42447f112ad877af8fe1db0089aaec5b28e88d6eab45e7fe99cea", size = 357151, upload-time = "2026-05-28T12:01:53.43Z" }, - { url = "https://files.pythonhosted.org/packages/8b/f2/3eb9ccdb9f143b8c9b003978898cb497f942a324c077401e6b8834238e63/rpds_py-2026.5.1-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:ad3773236e95f7f33991eb125224b7da66f206504d032a253a02da7e134519fb", size = 350195, upload-time = "2026-05-28T12:01:54.901Z" }, - { url = "https://files.pythonhosted.org/packages/a7/24/dbda232bc4f3ed732120692ab0d2c8402cb020516556d8bee622dcef2413/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a04df86b3f0fade39ec8fd0e0aab089b1da9fbd2b48df778a57ef96f5e7d38df", size = 381850, upload-time = "2026-05-28T12:01:56.601Z" }, - { url = "https://files.pythonhosted.org/packages/40/30/32e769839a358f78810c234f160f2cc21d1e4e47e1c0e0e0d535be5a0219/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:6142dbd80c4df62a5d899f0d616d417f84e0bc8d32526c8e5589019d75d028a7", size = 387899, upload-time = "2026-05-28T12:01:58.212Z" }, - { url = "https://files.pythonhosted.org/packages/ab/86/ec84d243aadb3b34b71dd26a010d0930b2d284ff5fc9a69fec53810ee6fd/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0b35217adefe87f2fe4db7e9766cabe84744bfe9616d9667be18988928c7f2dc", size = 501618, upload-time = "2026-05-28T12:01:59.888Z" }, - { url = "https://files.pythonhosted.org/packages/74/25/b60e52686bbff777a64f9e4f4d3dd57980dc846913777177a2c92e4937aa/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b95d5e11fc712b752081183a55a244c03cd00570489edd7014d8899f8ceb8162", size = 394003, upload-time = "2026-05-28T12:02:01.482Z" }, - { url = "https://files.pythonhosted.org/packages/9b/c7/b3a6a588cc2219510ef3f42e207483a93950bedd1e3a0fd4015c95cff9e5/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:141c9498daf2ace9eda35d2b0e376f9ea8b058d84f2aef4f96fccfd449a2f251", size = 379778, upload-time = "2026-05-28T12:02:03.197Z" }, - { url = "https://files.pythonhosted.org/packages/31/00/c7dba3fc8a3da8cb3f6db1eb3386be4d79c2e97c6890d20eb9ac66ae8c43/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_31_riscv64.whl", hash = "sha256:6f249f8b860a200ad35193af961183ebe9132710484e6f6ce0cf89fd83c63a9a", size = 392359, upload-time = "2026-05-28T12:02:04.817Z" }, - { url = "https://files.pythonhosted.org/packages/93/dd/472ba494c70753f93745992c99855bee0636daf74e6984e5e003f150316f/rpds_py-2026.5.1-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e4abbf391a70be864920858bf360f4fb380577c9a0f732438a1996726e2c195b", size = 412820, upload-time = "2026-05-28T12:02:06.401Z" }, - { url = "https://files.pythonhosted.org/packages/1d/6f/93831a3bfe789542ed0c1d0d74b78b440f055d6dc3ea4640eba2d95e6e23/rpds_py-2026.5.1-pp311-pypy311_pp73-musllinux_1_2_aarch64.whl", hash = "sha256:c74005a7bb87752acf351c93897ec63ad77a07a0da7ecad9c050e32e7286ba34", size = 557243, upload-time = "2026-05-28T12:02:08.013Z" }, - { url = "https://files.pythonhosted.org/packages/1f/ff/0b3d604614ffc77522c6b288fdbce68957eb583da1002aa65ba38ac0ee40/rpds_py-2026.5.1-pp311-pypy311_pp73-musllinux_1_2_i686.whl", hash = "sha256:8213afbe8a3a906fb9acb2014423fe3359ee783d0bf90995f70623a3217bfa6c", size = 623541, upload-time = "2026-05-28T12:02:09.661Z" }, - { url = "https://files.pythonhosted.org/packages/ea/ea/e7b0251441da9adfeaebcf29601d10f2a1455fcf0772fae9e7e19032bd96/rpds_py-2026.5.1-pp311-pypy311_pp73-musllinux_1_2_x86_64.whl", hash = "sha256:8c43a8a973270fd173bf48cdf80bbe66312421cba68d40845034f174f2389049", size = 586326, upload-time = "2026-05-28T12:02:11.47Z" }, -] - [[package]] name = "ruff" version = "0.15.6" @@ -2263,41 +1427,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" }, ] -[[package]] -name = "soupsieve" -version = "2.6" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d7/ce/fbaeed4f9fb8b2daa961f90591662df6a86c1abf25c548329a86920aedfb/soupsieve-2.6.tar.gz", hash = "sha256:e2e68417777af359ec65daac1057404a3c8a5455bb8abc36f1a9866ab1a51abb", size = 101569, upload-time = "2024-08-13T13:39:12.166Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/d1/c2/fe97d779f3ef3b15f05c94a2f1e3d21732574ed441687474db9d342a7315/soupsieve-2.6-py3-none-any.whl", hash = "sha256:e72c4ff06e4fb6e4b5a9f0f55fe6e81514581fca1515028625d0f299c602ccc9", size = 36186, upload-time = "2024-08-13T13:39:10.986Z" }, -] - -[[package]] -name = "stack-data" -version = "0.6.3" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "asttokens" }, - { name = "executing" }, - { name = "pure-eval" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/28/e3/55dcc2cfbc3ca9c29519eb6884dd1415ecb53b0e934862d3559ddcb7e20b/stack_data-0.6.3.tar.gz", hash = "sha256:836a778de4fec4dcd1dcd89ed8abff8a221f58308462e1c4aa2a3cf30148f0b9", size = 44707, upload-time = "2023-09-30T13:58:05.479Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/f1/7b/ce1eafaf1a76852e2ec9b22edecf1daa58175c090266e9f6c64afcd81d91/stack_data-0.6.3-py3-none-any.whl", hash = "sha256:d5558e0c25a4cb0853cddad3d77da9891a08cb85dd9f9f91b9f8cd66e511e695", size = 24521, upload-time = "2023-09-30T13:58:03.53Z" }, -] - -[[package]] -name = "tinycss2" -version = "1.5.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "webencodings" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/a3/ae/2ca4913e5c0f09781d75482874c3a95db9105462a92ddd303c7d285d3df2/tinycss2-1.5.1.tar.gz", hash = "sha256:d339d2b616ba90ccce58da8495a78f46e55d4d25f9fd71dfd526f07e7d53f957", size = 88195, upload-time = "2025-11-23T10:29:10.082Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/60/45/c7b5c3168458db837e8ceab06dc77824e18202679d0463f0e8f002143a97/tinycss2-1.5.1-py3-none-any.whl", hash = "sha256:3415ba0f5839c062696996998176c4a3751d18b7edaaeeb658c9ce21ec150661", size = 28404, upload-time = "2025-11-23T10:29:08.676Z" }, -] - [[package]] name = "toml" version = "0.10.2" @@ -2346,32 +1475,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/6e/c2/61d3e0f47e2b74ef40a68b9e6ad5984f6241a942f7cd3bbfbdbd03861ea9/tomli-2.2.1-py3-none-any.whl", hash = "sha256:cb55c73c5f4408779d0cf3eef9f762b9c9f147a77de7b258bef0a5628adc85cc", size = 14257, upload-time = "2024-11-27T22:38:35.385Z" }, ] -[[package]] -name = "tornado" -version = "6.5.6" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/50/57/6d7303a77ae439d9189108f76c0c4fd89ee5e2cc8387bffb55232565c4ed/tornado-6.5.6.tar.gz", hash = "sha256:9a365179fe8ff6b8766f602c0f67c185d778193e9bdd828b19f0b6ed7764177d", size = 518139, upload-time = "2026-05-27T15:35:54.646Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/1b/0d/b4f481e18c5a51864e6d12b9a05ecf72919696680b747c958c3fc1f4fbae/tornado-6.5.6-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:65fcfaafb079435c2c19dc9e07c0f1cf0fa9051759ed0a7d0a3ba7ea7f64919c", size = 447737, upload-time = "2026-05-27T15:35:38.122Z" }, - { url = "https://files.pythonhosted.org/packages/9e/9c/5430c39fcab1144d35860f457b15e9c08b4bc7ac86764354204e983d6183/tornado-6.5.6-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:38bc01b4acacded2de63ae78023548e41ebe6fbed3ec05a796d7ae3ad893887e", size = 445899, upload-time = "2026-05-27T15:35:40.519Z" }, - { url = "https://files.pythonhosted.org/packages/8b/79/fa7e14a2f939c807a8d30619b4eb604eab219601b78792516ebe22d40cf9/tornado-6.5.6-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b942e6a137fda31ff54bf8e6e2c8d1c37f1f50583f3ed53fb840b53b9601d104", size = 448964, upload-time = "2026-05-27T15:35:42.106Z" }, - { url = "https://files.pythonhosted.org/packages/a7/71/bd67d5f5199f937dafe03a49a37989f60f600ff6fef34c79412a829d97bd/tornado-6.5.6-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8666946e70171b8c3f1fc9b7876fac492e84822c4c7f3746f4e8f8bc9ac92a79", size = 449935, upload-time = "2026-05-27T15:35:43.906Z" }, - { url = "https://files.pythonhosted.org/packages/cc/a4/c24388c9cf5b3c3a513b56a158af9f23092c9a2810d789e294310797df21/tornado-6.5.6-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:1c34cfab7ad6d104f052f55de06d39bbafc5885cfeb4da688803308dbcfa90b7", size = 449767, upload-time = "2026-05-27T15:35:45.793Z" }, - { url = "https://files.pythonhosted.org/packages/a5/eb/6a07ad550c3f7b37244bd0becdf293ec3d3e961783d8b720a97df50de1b2/tornado-6.5.6-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:385f35e4e22fb52551dfcda4cdc8c30c61c2c001aef5ddad99cdfe116952efd3", size = 449174, upload-time = "2026-05-27T15:35:47.485Z" }, - { url = "https://files.pythonhosted.org/packages/bb/84/3469e098dccdb6763130e06aacd786bb4363fca7b590a55c101ddf34ed30/tornado-6.5.6-cp39-abi3-win32.whl", hash = "sha256:db475f1b67b2809b10bb16264829087724ca8d24fe4ed47f7b8675cae453ef86", size = 450230, upload-time = "2026-05-27T15:35:49.322Z" }, - { url = "https://files.pythonhosted.org/packages/d2/3c/273a04e0b9dd9016f1685cca0c1c8795a71ac88a34a8c889a0b443483226/tornado-6.5.6-cp39-abi3-win_amd64.whl", hash = "sha256:6739bf1e8eb09230f1280ddbd3236f0309db70f2c551a8dbc40f62babdf82f79", size = 450667, upload-time = "2026-05-27T15:35:51.194Z" }, - { url = "https://files.pythonhosted.org/packages/02/98/0cffe22a224f60c5fb1e3aa0b76f9da2e1ca78b0e9545e3d077c68ce60a7/tornado-6.5.6-cp39-abi3-win_arm64.whl", hash = "sha256:2543597b24a695d72338a9a77818362d72387c03ae173f1f169eadc5c91466ac", size = 449690, upload-time = "2026-05-27T15:35:52.902Z" }, -] - -[[package]] -name = "traitlets" -version = "5.14.3" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/eb/79/72064e6a701c2183016abbbfedaba506d81e30e232a68c9f0d6f6fcd1574/traitlets-5.14.3.tar.gz", hash = "sha256:9ed0579d3502c94b4b3732ac120375cda96f923114522847de4b3bb98b96b6b7", size = 161621, upload-time = "2024-04-19T11:11:49.746Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/00/c0/8f5d070730d7836adc9c9b6408dec68c6ced86b304a9b26a14df072a6e8c/traitlets-5.14.3-py3-none-any.whl", hash = "sha256:b74e89e397b1ed28cc831db7aea759ba6640cb3de13090ca145426688ff1ac4f", size = 85359, upload-time = "2024-04-19T11:11:46.763Z" }, -] - [[package]] name = "typing-extensions" version = "4.12.2" @@ -2445,24 +1548,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/33/e8/e40370e6d74ddba47f002a32919d91310d6074130fe4e17dabcafc15cbf1/watchdog-6.0.0-py3-none-win_ia64.whl", hash = "sha256:a1914259fa9e1454315171103c6a30961236f508b9b623eae470268bbcc6a22f", size = 79067, upload-time = "2024-11-01T14:07:11.845Z" }, ] -[[package]] -name = "wcwidth" -version = "0.2.13" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/6c/63/53559446a878410fc5a5974feb13d31d78d752eb18aeba59c7fef1af7598/wcwidth-0.2.13.tar.gz", hash = "sha256:72ea0c06399eb286d978fdedb6923a9eb47e1c486ce63e9b4e64fc18303972b5", size = 101301, upload-time = "2024-01-06T02:10:57.829Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/fd/84/fd2ba7aafacbad3c4201d395674fc6348826569da3c0937e75505ead3528/wcwidth-0.2.13-py2.py3-none-any.whl", hash = "sha256:3da69048e4540d84af32131829ff948f1e022c1c6bdb8d6102117aac784f6859", size = 34166, upload-time = "2024-01-06T02:10:55.763Z" }, -] - -[[package]] -name = "webencodings" -version = "0.5.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/0b/02/ae6ceac1baeda530866a85075641cec12989bd8d31af6d5ab4a3e8c92f47/webencodings-0.5.1.tar.gz", hash = "sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923", size = 9721, upload-time = "2017-04-05T20:21:34.189Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/f4/24/2a3e3df732393fed8b3ebf2ec078f05546de641fe1b667ee316ec1dcf3b7/webencodings-0.5.1-py2.py3-none-any.whl", hash = "sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78", size = 11774, upload-time = "2017-04-05T20:21:32.581Z" }, -] - [[package]] name = "wrapt" version = "1.17.2" From d6ede7f07c0ca8f8bee2013305cdb655642d93b9 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Mon, 8 Jun 2026 09:01:19 +0200 Subject: [PATCH 10/18] docs: centralize markdown-exec setup, fix output capture and truncation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add `docs/hooks.py` with an `on_page_markdown` mkdocs hook that auto-prepends a shared `markdown-exec` setup block to every page containing `python exec="1"` fences. The block matches the first fence's `session=""` so kernel state carries over. Authors now write pages with no setup boilerplate at all — a single source of truth lives in the hook, so the setup can never drift across pages. * Register the hook in `mkdocs.yml` (`hooks: - docs/hooks.py`). * Strip the duplicated inline setup fence (imports + chdir + `configure_formatter`) from all 14 executable user-guide pages. * `DataFrame.show()` writes through Rust's libc stdout (fd 1), which markdown-exec doesn't capture (it only redirects `sys.stdout`). The hook monkey-patches `DataFrame.show()` to call `print(self)` instead so the table appears in the rendered output. * `DataFrame.__repr__()` returns a Rust-formatted ASCII table that hardcodes a trailing `"Data truncated."` line — the HTML-only `configure_formatter(show_truncation_message=False)` option does not affect this path. The hook monkey-patches `__repr__` to strip the suffix so example DataFrames don't advertise truncation in every block. * Auto-wrap the final bare expression of each exec block in `print(...)`. Code cells in the original notebooks routinely ended with a bare `df` and relied on Jupyter's auto-display via `_repr_html_`. `exec()` doesn't echo last expressions, so without the wrap markdown-exec captures nothing. The AST-driven rewriter skips lines that are already `print(...)` / `display(...)` calls, and skips calls that return `None` (e.g. `df.show()`). * Unwrap `print(.show())` back to `.show()` everywhere it slipped in — `df.show()` is now a side-effect call that prints via the monkey-patch, so wrapping it in `print(...)` would print the table once and then literal `None`. `dev/check_api_coverage.py`: index the reference page's `:::` directive targets so the coverage check still validates documented symbols after the move from notebook-based pages back to plain Markdown. Co-Authored-By: Claude Opus 4.7 --- docs/hooks.py | 105 ++++++++++++++++++ docs/source/images/jupyter_lab_df_view.png | Bin 267052 -> 150303 bytes docs/source/index.md | 25 ----- .../common-operations/aggregations.md | 85 +++++--------- .../common-operations/basic-info.md | 35 +----- .../common-operations/expressions.md | 57 +++------- .../user-guide/common-operations/functions.md | 55 +++------ .../user-guide/common-operations/joins.md | 41 ++----- .../common-operations/select-and-filter.md | 33 +----- .../common-operations/udf-and-udfa.md | 25 ----- .../user-guide/common-operations/windows.md | 47 ++------ docs/source/user-guide/concepts.md | 25 ----- docs/source/user-guide/data-sources.md | 25 ----- docs/source/user-guide/introduction.md | 27 +---- docs/source/user-guide/io/arrow.md | 29 +---- docs/source/user-guide/sql.md | 33 +----- mkdocs.yml | 2 + 17 files changed, 203 insertions(+), 446 deletions(-) create mode 100644 docs/hooks.py diff --git a/docs/hooks.py b/docs/hooks.py new file mode 100644 index 000000000..7a42542ad --- /dev/null +++ b/docs/hooks.py @@ -0,0 +1,105 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""MkDocs hooks for datafusion-python docs. + +Auto-injects a shared `markdown-exec` setup block at the top of every +user-guide page that contains executable Python code blocks. Page +authors write `.md` files with just prose plus `python exec="1" +session="..."` fences — they never have to copy/paste the shared +imports, `chdir`, or formatter configuration. + +The injected block uses the same session slug as the first executable +block on the page, so its imports and `chdir` carry over to the rest of +the page through markdown-exec's per-session globals. +""" + +from __future__ import annotations + +import re +from typing import Any + +# Matches `python exec="1" ... session=""` (slug captured). +# Tolerates attribute reordering. +_EXEC_FENCE = re.compile( + r'```python\s+[^\n]*?exec="1"[^\n]*?session="(?P[\w-]+)"', +) + +_SETUP_TEMPLATE = """```python exec="1" session="{slug}" +import os +import pathlib + +import datafusion # noqa: F401 +import datafusion.dataframe +from datafusion import ( # noqa: F401 + SessionContext, + col, + column, + lit, + literal, +) +from datafusion import functions as f # noqa: F401 +from datafusion.dataframe_formatter import configure_formatter + +# mkdocs build runs from the repo root; mkdocs serve from `docs/`. Walk +# the local candidates to find the demo data so pages resolve +# `pokemon.csv` regardless of which one is in use. +for _candidate in ("docs/source", "source", "."): + _p = pathlib.Path(_candidate) + if (_p / "pokemon.csv").exists(): + os.chdir(_p) + break + +configure_formatter(max_rows=10, show_truncation_message=False) + + +# `DataFrame.show()` writes through Rust's libc stdout (fd 1), bypassing +# Python's `sys.stdout` redirect that markdown-exec installs. Override +# it to route through Python's print() so the table appears in the +# captured output. The Python `__repr__` produces the same ASCII table. +def _show(self, *_args, **_kwargs): + print(self) + + +datafusion.dataframe.DataFrame.show = _show + + +# `DataFrame.__repr__` appends a literal "Data truncated." footer that +# the HTML-side `show_truncation_message=False` option does not affect. +# Strip it so the rendered docs do not advertise truncation on every +# example DataFrame. +_orig_repr = datafusion.dataframe.DataFrame.__repr__ + + +def _repr(self): + text = _orig_repr(self).rstrip() + if text.endswith("Data truncated."): + text = text[: -len("Data truncated.")].rstrip() + return text + + +datafusion.dataframe.DataFrame.__repr__ = _repr +``` +""" + + +def on_page_markdown(markdown: str, **_: Any) -> str: + """Prepend a setup `markdown-exec` block when the page uses code execution.""" + match = _EXEC_FENCE.search(markdown) + if match is None: + return markdown + return _SETUP_TEMPLATE.format(slug=match.group("slug")) + "\n" + markdown diff --git a/docs/source/images/jupyter_lab_df_view.png b/docs/source/images/jupyter_lab_df_view.png index 31245ce15e29f06cb76f161d2cd4e676cf1a1773..9dafb4f61d3a82b6f51e03fa43f3c97a4f8bde0f 100644 GIT binary patch literal 150303 zcma&M1yozj)&NQkDGtTGP~6=q1b5fs1P{SoT8b5?c(I}_UR;W_xCgfYDeeTfATRyy z{bjwk*1P{%=j6kBs( z6cj8e2LM1*2>_tgbO+lyI0I2o6hEaJpc(2666cy|)77Iq)s_K=zE45Xma%_QMDkl+ z@o5-=HEq=VgxrfcPU0W=<*%cu@Wia_@DGBtWTlNrFuga`UhcG32ZB4mM|W3we2*d? zMi+BE^z3nFk zo>9Q4bjbze;nA4?CRCgnMUlV9;ZEF)vnLYe7n`4zk|c^O8CX)loW1+UQxFCg-dZ?s zcK)wa-t3ZJhz-G$8t162C$#16EuXParWa`;%zuDyu^S|#a%2PZ(NVlmdpcTLl5g<7 z$v8g)_23PkXF5pj*xae{ffSR6M@hS)TLv&iG&7jSR@D(wQzxA&F7ck@kwg_`nV-iE z7qTr^T>HQHR&sy=sPFg;!;!(&@ zvJZWV)jW0B?Xtk>aSi_(-W8}W-s6wQ^=jiqFAGReg|VMCDBo+BKu~4S{EzZRFtCGK zxsK3K`F<|*X}mU#4%<+yMrg*%8o3YLJOow(QB-HO>s_gWg)zJfNgo9X94gp@F{7+s z6eOaAOX??bV#Mn$rf+RwkC8yLSvF)@Pd?m@Tx-e&|FEStpq9$UoMptTAwczd?q^Mp zy4^+c9F=7&-V0L}!%aB{ckS-IVXt8}>)0p##dX`zY&wC^#H_UR94S_>4G3I_|`fu~)wE4OC8-*zxi-_brHdy7JuGV&*i z!7l;bZ2c|%HR0{EV^MFEG)d&Js`@AJ0X z?!8bxH^^_lz$PZ~H}!j>gL{L57-dg}UCxb^`xV#WpK1NwWD+2)E zkqDvSG_##|cin%r6B?*ZbsMhjx>iDf`L0(Tm=BrNU#0NIu4&ZV_!WK0`2tOhW}lPV zQowgw)a+OQNJt^<9utEkox4LCdx)*jYR={z4qkg>m7`%DFwDMhu;>T++IvYPRaZ%a zYfgP@^W%?c9#Hct*pvO#1=Jdfg7U}1-NNF}jt}Pn0}4Xo&!yC9PMu|!M&wHrlKA7I z&re@O4s$-gPScDOQ==h$=3(({d;RIs2ccs$X<97V559?5j9oi?tD@=eOD#|{FqPku z8KUZcys#jgf9V}6@dxEQTK_KxSDbgKpg*Y1Xa*W2zE5s{i5-(k;}OUJcA_M$LJePE zMqzfz1}4!mJpCm1IZ2n{89tr14Jt1#3qYTCKZ!$8pq54?G+UnY_2B^32L9y7HhIZ> z(z0mBLiz7Rq6EMKc7Dn@1*WfLJFn`|a}qeE=L+<8T_g1U#WI5>uH{4yex^y zw89}Jw6@kVe3=%n^ux%IML%B8vcZr(aBS`eP6Y`fiqK7T>=)IampHwIuGEq*^1`|L z=GMhdMBB(ZBc!?z$KdDWzY{Gqj$gZ?TVs%gm4*?>h**lPa<8&4shEcO$iCJ}w^uuV zAHb8DA(yTGPIc++l8U&TGm|#~LW$wCWr-jP`=|D2RxN_=q%8Ns01@C{r6)5xSh;#kFs~mlZpH8`7t$X4Nw-Xa!uP z>`~1K@sp26Z)vFHZ%(C+mpRzk<2ukx?>NRgQVMh|Iyby*5N#-TK#o8mzlQ*c)6rzx_k~Ki$=b=6lWR_+ zbBkqO^5x0#6QjG%q6v}-9dW^FE5&Nljk8J{vEja#gH+I#2w z&D)DxTrAgTVjoFAmZDN%kCC6_%uydnWDB)V+H8#e!##J@q;G319Mb<&b z4|?r-O%1sM{jCG7eYYLhqkRzDXtmT)FOvd}DM6_BM^F}(Y`8`6b`rV=Fiz*6J@Xv|mzGe%S;J4LATFL9Sp&kFmhvA*(~%b}ZMIiO(r8oRi2rAtPdp!okf$t62u%sufjOObi>S~c4tk&Bd2Z^KPT>w>R#9Nxa|_} zvW*akERX09uf`3bv?QVXWKa1q9)qIJedJzoSW(L+=vCC3Y-OxJ9*9cf&8n(>0kAv9xGq>|*CAXhzE4^qvTydT07Rj_~_R6{%lzE0Z*ODyU4szQ4^GVir@$K2p ziR`K_igd*cCC(bVG$i~k7tK{CF6%0Nuxty7eAlktPpE_*jMRBWlApwOEm-LUSiB7O zNL*euMMLxYl*(22*K7Qoe7UY4?)2`cKO|U$|1oOSxQ#wP!pe(HC6mYu&fLswmuYlu zKi1Q?pDn4lTF_b$t3`AF?0&iR*+a)8A*x<>pE&00K*nU223ZgQAP z<2Ici;VJoP!)iic%(VP%%(>PFu(aIo$kIcCB)Iq?~>`R&ZjTtg=gyOQ~2 z>vYg=AFu~l02E3`ASH;5hulQ;SgN)DY^7BNtLcUJMbQk&u<5W+Wqwss)vh5$O`~N_ zZ!2)WqXP0Sr114uD{!&pX2H-E#Wh8h%Q?*O=b*jLLWxFJcorCpv7l`L)YI!oZEmpG zp8|b?jy?9>Rr>$@eOY^G?i1zhc@Vq2@a|B{w0+ohZj!uS-gv4m;O?kV;zIzrBb8&* z!OXHrBkn=+Q9y^3*W>83RkSZSytuQ}wc@&d8NtooBa^8*Ctd=H%&j`8%us(O@EHI13ZohkJ2T_eKjA(k1c0t z8bb~aUE#tH%*R^8>&^XFqJBACxf>EUlLC({hYmM<$#yf#oXh?Gp#h7z2>~k?GIwhp z;K95vA^IKlettV#kDbQ1eV1PCpWDcN+ppsuvZ%nOFu@CjE2+SA#L*dI68DzIJJ=>z z{E=mMst%&lnb=Wr?}%tPF}ZF+JZpLO%&IDk?UCr0h1}HC6W{h{D8{@fp1zWW@jBU$ z8!xo7jW*Hmm@0(nhhoG~8jc?&N=P1-m!4X{1by#9Z}--pM3p?jgJX} zei0xa->*;q3mUcd>$CqVKeb1;p-Afhl$4Nf9cyBFlfPxn9%$1LEl<@!C*LlNJDW2hs|1a&vOOmV8M|ODpbfV=JmHEC28A$S;Z4 z_MV=uqFh`)K0cg2e4Jo+J1!m(5fLtKUM^l<4kQGJho6h5r7wqz2mQYW`LA(gfgaZG z4z8XKU>DlI<62sQy*wpezy3SXe_#LlpFm%S|C!0f$vS|LG(*lIDaa!z{tx{Br{#ZU{9io{|EDL95D)+V?)tx){@-2oJb>;1 zFbFxPr{w>v*S|ae@5X<36zBSj`u`%uzv%o=Et1fZFU7h3d(k9cPI6u|A=i=8K~_T- z`9|)tztfT|r4ZxbN56!z!=s5(Ab0597X#U~hl+n)c~`yseeqMpV=UvJa)8N4@`#Hj;!Re2j`M$*96k08614BMhFRx7qJ;$Dwb`Zm&t!dqPpX0h zjoY-=D*6A%tnbq=sfUwt6>ZBo2*14S6>-+Cv z`r>|dKHkF*GVOA;?Chd&x6({p202@I3TBDs%tP)EQmX@j`%U_?7P`+n9xkfwq#kb| zCh0Ix({e45!MmCZ6SmFg!58J`K%r$%oxW3_PK2r8enp*ab;$jG9Y@nr?H$7*8S|69 zrf+${q5(%9d1B-Hs0c@VLkkGx3U~?azzKJEvlyr|V|66M{LC zLH>e>gSsnkYn}6*7_JUs#I8ojV(U*+?}axez*Ifs&VI|y#|QXhv>oHKA^Y4uZzw2z zIT&%++iQM1nQlvB=DTXoTTrfU^fWo2lKjP8AGQ&4UPJFmB_^UPnK!;W`H{r?AveS3 zwd)B|9fti@_p7)e!U0HL;)@QL6_9)T+}8DJg4BcWC350cIm^O0!689d;Mpv^`E=j> zQ512S-nq1RSDW6>9x&=;Hg#K>>i@jsG_~5s2NvXG{;BQAt9n@9Jm{ihEXSECpH}p0 z$%Ai2A-R{>gdGa*yg%3;PnWu#NT?m=>TIjtA5(-)#0jlrn1telZS~k;$ zcg`z*9jhv-$cx97jqygH}AJ)c77$xyL7d|5m)Z>U?aUZ$ljYgdRD^i);<( zn}8ae7wa7sV$9A9W-r~0{Qi7EtK}OLE*uyW+0IVpsOcdfi^vYTTK0KYD|{T-cGV6C z@zv%A-|kXD>7PL~2VZFZlDgSQ9&G1>h=O+p_X1 z)>qy2YUU@=1~y{wHVI52(w2~x5{A?alV=^iDIOfxljKmux2Mzb%@`D7J z^#vU}3qhUh3WCXg#%_~oXU^Q0j#q!Z|J5{{$Fkv8@|ArX+)w4hzDTgFljW}v_~#kT zfPGVDw*Ou!y-%psb37Qsp)ppWey(9&s345k#5sxM$FHYpAPzLy_opX%)&6=WdvotR z)EcL?RT@B9dr}2vaUBX}L4q?PSbGnBHI(Gj?U$cj)*~v)t2PA{i zX8TpX+rHRVulzkB@_hL~hkrc^4%Xp6z1!?SEW>Sw+3R9q)5e{bi%xv6@)&*Zg4YQ1 ze&iX?GN71{UePn<`mSMF>fYVML-O*E`QIefoL*FbD7H>9&97y=r?nI=yasc;7g@x% zGOTY!eRy-itXiQ#9k(;pobhDg3Y*{aR*~EXjTO&myYA&0yPUViMJx=szr!+OUZ^x3 ziKBAs!C#<&UH44}+|Bn(rUjkEm^&YaY>UA%PwdFk4+9=dmgj)Hp#|bs&(VMFi1BhR$@uhixj378)h6i7TdFwZw zl^xmXDrP>5!7|cgnU%tMb3LIzv#c*)tlfJoz3_3WuNkxTr6;jjS$$nu8ej5YVt8^- zz{(kQ!t0iJ5BDiA?^78G7v0PmvUx%c@gyqWKOYEJu6&8%k9Jas1+4xSGPORn53zhp zCM&0DX(wd|et!&Q5M{}Qx(h9f59}XUfue;8$mJW=hoT?tG?P(D(km%fd&AqW#uU*} z{j1mYaj?&A0~Fib0K|ngk{6{KL`GkEt|tkS|8wx#ld?$&9zW}6mlvko?rHIC@A zM0Nkly)X2MWv&k2*S!RgTTiVy5zB}$k-t?sqGC+A^CS*z0}p^9KBMVr{b0~=wqO4q zM~l7O_v`b`fP&=bpj{R7vJ^?Pl&~VwOZy8gscxFb>;8qRpGS*UVYSIgk!sOcMvzI_ z{1N77D-CXVvC2qxgoRc7VGiw9%sf9G8;4zQFi9;azwAADf<`%hE3ubaYCphR!n{O^ zp~Mi+7_LhDqHMxxJBVHz{?M;~3fan3BjOD_~bne9HrvM_4`J`{?Z zxh+hP{L+56$)&MXnp{W0q|0H)7(<82#FJ~Fx8MPhsO!d+YTH`{h*XgP(t!5zEOqF; z${B46)P6lK?dzX;FSB5Gwx!*6{mD!xV{Y2CeA@MM=#;V13wC)yj3w7>W&&>B?K0EV764w~oq!mE$OC+hhp1I3m{ReG`NCO<=6EQs?=&L@UpG zasSObgo=92HXm@XHg#T+)xJ)$4dG*@i=XUW7xn&uo`n~_y$JyI-Y4G(t&B0h!=Te* zEEKd^t%><&`-{FS>v1;2c`=DqDPcsY2s*bhPxSII^Q)l3;D~n8 zH{#`Hvvv#sVmcsR*RPu&e7Pj_IK;;Gbe#@6`<+Pg@X5a}c@E@7ZQ|YC%qcC4W&Sz) zwMj!CKumv^!M1Jwcxy08>aY|qemXD``Lq7eD=p05_Epqi*2;Ma*0(2v$LN8FPUczT zW@grL0@1gJ;aq{1-VlkaS9C4+%pF=y2CWmncZ&`r*2IDkP?*^l*%^0RUea3aAchW~ z*#5BZK}ca1eW~Ipvm^Mq@zu(EZ&^UvP{reizWk4kc_cDhg<(v(?1u)6YGy09vrV?l zx^@g?Brka7y-VI6Zv%Cv#KPyzoF zK1y#c?*7|CVlLV$x$RiqG$$|e(GQl-KjQl#x@jIX`2$JJ15aW_y81v?E>;cjAKf?+ z#Tg;=5i@l;O)LIReKt*uDx57gXsuScn+UZ+0f;)J&VVn)!y{YFth7qlO?dP`eln*5BbJN4><<1GS|05A^ z1up=7Q+sPz#_XF=AkMSe^1FW5Zf-+JWt67X z+fey*6x=1E;#;nN8OTiFv|C69Jks9S87z+iC~)rSy?I{{cY@A)ReMy%=JvW^YcKsJ-NZWOy&%U%|{{f2>0-B?qN3qXVTP z^E;_Q9wD8L)pPD8S#+4TOHCwLVatm-57({xVRgloRx(g*I~%My#(E7hs;(wb*LOwv z8(JU?55dfKO>EyeQB8NZK)Z#JH3B5gMtxI1%KWnpPkOKfrRH#iWE&u_y{&nCH0E=pN$HU0BLSE?hD%M#zgmRcC56uFXMiH04E{VpliYF zLOIG{O{j&fx~Dhx;&{7qHM~$G&nMXj#q^0N$4bBu!=fSS*2FIDUCFZTW`;)L+`iC_ z1iu!}1o1c6(}2Ys6Irv(yRd{ONa6#-<(-?@D60Tcl5TJY647@}SFr)pG(mXH|<=3fnbs5IDfZL?JfG z(Kki#=HSbNd_h41zTwG&LOA}=TTF$g#%sVJN)+U%B$Km zvNq1=O$SBW&#*3jz!xeA!LwKRN^2tCSRlbgud^|+YLFv`n|_z3rmbOqd!Kz%3U5j3 z0E&T?;rR5=(M^oFgYEC#JN6#H;>Awpr#gow9TI!PugV!+2seMyG`tnV>&B+*SL5E2 zedbiXsmcT(Mr}iyhmCri?xv@*na|QKG2$8O6V$lX7tZk-YpW24X%q~urf}@41kBW+h>hk(dm!M9lL zDZBL(J1}1*GS_Q$(g_L}pIE|eM29Q?T`xFR#GA##ZR?YaRA#W-9z{&Q;JsL}fr{IX z=W_6hf(qB=iL;pcAK`h~Nb+1Xd1|ivcb%p1d>4^0>1h1Mu&axBz{3OcM+ZLtZlAJT z`E4b!o^KzYq;)7#nPx#$boLvtjeLy)ijBF*_KPwO-cV7o{q>Tx<018&hrkrfk$pSt z9+wFASSqZ;tMCpDNc9xTi5XjsR^JY@{8&r0;pD>1)$a$I3tN>y1EYuP}Nb)&X`}rht6X|}Z`kBko!g-?>Ct7Rm z?0}6Wiosvu5ZEK?Q%g~6zw>2s_DF0@?gS~q>W-Of@7dMzr>E;EdW2-Q^X#I=2YTpP zy}6MVWk%O89m#ZtVpMRW0?_m#)HY!&nFFF;6+-^++oe_Vdm)gbby{0&2s9`w}_=teP1hvPGCb&d!i@(P`-osR;FhU1J45yV|J_oWYN0Jr@m3;W{Wb+4jzsDN&LQE1fRws-sekfmk zF5%ETRvd_1EO zz?1iw$0uBC%WNwY9+wNR+{T;Jf#LNzgoK<>9(mng%q3(Y9wf+yD>}f9k%T%!J{@+H zIQ}<{1)tlYR}}k_wM$@AizXYX=t4@EdKm@enjDNMuYtUv!72n|@u2fIrPF8`Z)f(4((9FE(3Qg^0bD@;J(FnE zY^3d7_Z>3q`SMbATtW-i4*?6Thhy zG<)!e$Q&k>8@!U30_e@|(8D`(x$s$;VaB&dWpc=US}oY z;R}DF14`itU#4RO`%#RRqHk5!IPIB7+4y^zQahP`bR0!Oh>I>v@f>A4Sd%!jNwiae zMGwn}Kh#gZCiyKHfNUd=?W(J;8$eU-|Ekm1i6B=rKCp-`?#~58QM*W8M63$Q+S4n) z%$B|*y<_6sY6Y*E$-Vz>lR1-S`N;z3(82Ev-mRMN*OQb7q+oVmAI?z=%~r^|GCU+N z+N)LkxG(}f{+g1vi1%WjyI7q;jyQ2W;5SWi z2|5nQ(AHy7&9%qv#itZU<7@5#XL`6EOYB7IU)`rxp zx8&|EB`btE-CXT1R2pe17=>hjgc&R-1@hn7sSC^NojJGLHPY1dHIRMU>3mi-*fiQ< z|6ZNU;NZ*0ck%KDnR0Z0b6GEKZLl@eQDq%oS!Zbf1$Yq-{0Pn}V+P z?yBz|PZ^TE)NG5CE<=fOiom161oa}EO$inq{ke9S`nH0?GqE(@&!S|EDjW^E3WW;a zzyp&pDl0THUo-1emSFqvL4Df`{}8Wu7*eTW5>`7M(~{TH=ZP-K(h_A|DBoZjo0|rk z{%Y7uIXSOpw$r)r_=X@2r(;V?7XPN3F{OTUiP$?g5W8p@+Uwh4Fxk#jAb#j@MsB?m z(T(b8my&eecBGmqfmwKLo%KOb_>dHaL;E;=R*$HRXF>E08R{q$7knYjlIGCyZzw~a zb#Qu|!fBNxa1-Jj`88LK_^WsE5PkA_y0KDP=q8@uPKlT!Jw&gJ(yjb9(LEl#-F`6{ zQ#G15H}CJ#09HK>we~k_CJyoI9ak3O!ZU`RC6ZMh8sqH86)Id zJ|}#293}}A8~p2RX6TyGi*-F4&`vAtB9_tQJ ziIaH$LBL_d;Wsr)?W={PG3Jh>C11WAyKRvZoP^}-wp593@f204ge&D{v64$-?hh%0 z=krHaWWzrKJq@722(53~qZCIVq`yVfs1{&aGL$sXG6n8 zZZ_lnh;&r_WTF(Rz@C;$seX_=8yD%_+@-%hZ27>JSy=|hV%lDPSezW8+F>kGk^p%L zs|?ZZnI~UAY(#h=ZLg*!_u)pTV~%dBL9ys553at(`wpxaFThGE=N=Q}!$n|`chbuD zhhDohF{2dQ@K(MpDIZ~$cnA?L;pJw|p}z5rebH{<{zIU7i7V~JPZ-?KSjKC}YtddZ zXx3wW#Pc*>^QvDkygFw6=EvD#mG~>|6SdQoRZ zI2^`~Z7c^F6=$zL&$xCvIXF$60zrlQuILvz`NL>Zf?7@+I5Z?76=w%$6An`z-wFzH zDC>wipNQp&#+>G!zW1&LHN83fUBb|jhji-;c)|202jdP1=b?N*-T;k{Qn)4Z5|}b= zj>7op;>6GYoG%$lN#v;%+hZCVH=Z9ZwR%bC@Jh}IeOs{;0}6O)-|4qRBHiWUgI0+Y z)(AS>6MtXAS^t zlL9krV!fgx1nC9ZyL8bAZy^Zdp6UVFNiMBdwfR+meF5cA4gh+~RU<)(APo*Nl2FTH z$ir>?qj&=bRUxeY!{|Cm(-7+r z`zw-`^!~6uUw#{}pXu{>Cpo3_%rVi5qNn=7i@e^hXM?`b_(6%o7BHK86`Q%PbOB#COg7llMJ#J#Jd3 z7_!jQ_wHphpW=9{#rK)orE}K(>Q)95<-Q3~YcS0-kO$ho%bX7KF1(iCIqn6=!(ZX7 zGn`}=<>rH7Zh?}Ji7=C2q^nZ}Q@h0^g<(=$RSG2?*>Mq3rH%t#zhs!wq*j;~mq<}1 zLT0tB;*{5fVhZNNz|d_0y)<`cF1RWYFwC^mnM^Dou$r+RKQPxwNG_TxS>wR8o{9PD zrf^K~Q;up!x*^z_o4TP#{07dp#i3VXef_pzjgE$lN`oubJD2693K#MS_8PvpZW~mQ z{L~ETI=6Q{g+Et*l@~JWkY3ZCB`YT_DFf_cxyi-_?Ix#dp~K$>a;*hH!9)3hR_Be* zeV1Ggo78)t{-ReBrLEr&i>O03%pO4E?NcC0N%F?x zk@~w{K@wOq+{Epjr}vsEPpJ;cG{<@&U4p+zn#RpQPa~L+vyj`S*W26EK9p%_ciN*4 zvB)-vWB1-jzs#f;rTp`#mi2PG_5bW zNVN-a`mZ#ss0-F?bbFJBYb%B(7OfjX)YsS!S+J=1oAwpxq*s*Lx(6& z{ra0hxvZFVDMAso?*6dy@p|vt?Th(yj{ziFZb6HEZfZP zG?#=`CE?RVgn`f#KcGpNP%R9L3BqQEFk)Zd*b{`KC5LavwKZCE#D0clP$=9%V(+$c z;u=BS4-+c((x+(#eIUB)d3i&=En?bO!c}`!_JjBL@qlifSP?<+w5=anm-_JOm|R)_ zfG9sWG%)cEkA;(ukNu z>Xy)PnngE>2;x{YoubX)F zP#zN5Yn(@of=dNCVum}+#e*72_a@^h`gnyVi3uS(eYfM;rA||3medTGM#Rw0mMLZY zGOk<-qg=N>GP$UqzVDMcgjgzGyw zGy>7WJcd2ZF+GsI0u1DBEX7Y=KtlO~G9Xv#~k6hg2aKBar+1)%aMh5H z^DUvw@(MEQTe;z2!s``5vgK-Gm*aI;GD~UnkAR4Rf%c$`>nCLCserKZqQG8Gz~dXC zF_McW-1Z&7__sX=8J)E7@Tx*5rvMb}`|rSkA&H3Sr=Qx!rBAG)1tk$EZ0@oF4OF=ONe6hPAAC@3iOXrQ^Ws1T8w>txP*&4}tObSIXS4I3>rx&QY?( znad&0$6phz*4!R^UD~5rF8xKoW#qMMf!u=dex&n$vE-{wf!F0Aw2?+ zf926>_@(<@!E^!5A2B`8an0E*&0K%W_j!poz>20d4AKuCTpP`PJDmYqFZ)lkN4~6mIE9MQ>$W8A{2Cuk)N6XD z;dONAXA*p-slM=%t-?YordSH~1rIzlWDbfm7Gg2~H!A<~8!41zSQonDU{rFq<2goq z>GoK^d|g<%mP2{q-!{B8j|c4x#B6E$LA7WszsYo_Hm9%zM49x0}p`b9wx0WP6?%HFpxh-X?N39!ALcvB|@}xo7Ox4=f{3^3oq|MQ!XOgfo{D z7GvG4w^I=vD+zRL=oI~}Gjar)9%hAbvqpE^jDm>^kAqfpqFJR{^P{y2T>dfPbvS*^f+a** z&YcDp?F-|g@oAac?6N?DA)oTA)|c}l)%SP5q!b5k3x*~nm$&hGC$SEp4{dQY&RfZt zf*gwW-1ZN8Phma;hj;43^DL@VS&`U7q?NqNpJ1)v{+_^Itu=8#)_cls2CuyI+S|#- z!iwX*Fk}>`b<fg?swg2uLE7cxz zbru2;_iL*6-KHO|Gx+)T%M8L5R&S!jkq+I71*BK-4C?aHvjDAwre@jOH9=~viGK{J zpU@&WjU+~>VDBKo`ag%U8mc`qlZmQW7n*dbKlnz6GNeWk$w;bsD z9@YI3<1i`9rIjTM>3;y}^qmcLPd5Y?(q5@+UOHO^gcDX!JLF_nr(76V=MCTfepoU* z3IOQm*o_HTQYu6||Kn$G5$#wJ}4TzVi&X z9c(G15YxY~U(4R-3Co=L^f%#4GwE&Yrjv)w!fua%t%*@fnt`~tJr7HR6sHjTD-SyE z1fpG9w|D}YPT{@)$-kjdhdLQbSVl1P$qla6Q-rm9F~|-5%=7wi15~S6?P23)-g2b} z4a8PG|KbycRO?r&2=RKZe{Y6K8Jny z;LehZg(4`pylOjVtg%Ah&+Z^>iKqDRON)D#cSD+6E_uhCIK$z&xU4G=89!lZf7{|9 zPoKnvy7KA7Os~y`s$wpMfN^AxNNqC?gvX3aD7t=(tXFv?8Qgz;w?T!#@Pp)7Bintx zh(!6PE*+11V@vr7ILvU&>71eVh)=FWL71tnJN-i36Bc%toFqTnclrHj5L95-xryjk z`#$9!ZoHr1->m9;nVSp-_g_}$aY4TIw?#|FEiv!;T(#qP!1|8ocIwB@hKSyR<|>if zTPAgXVs`bOJXN#gX4?sFJEC!9IDPrf!uBkyq$A%jKH03w8LZOXNps@g{*a42v8%82 zw|#Hr9&XZ7H{X3k2F)-#unD%|CmzE_58vT;Hx*|tv|f9mt-!zvKt?&on-ePXUP`0s zCrOsmMAvQG`R9z3U=Q=T!@o!Kg;9$5JA3Gb2U^)IomZ~NrykM5SirL6v$-u1PU`JdX zo06wPvmZ-)8kKU>{}K!qxR zQuMV0Cg;!Xo`>T8RIGTMF)(jG6f97{7F*9Ioc8&fK?6m+jpE$WwWi7EQ$7CLan!mW zebeGhp(1YJM}sH6%id;FlfTxvgOTi0Thc>mv=n@<90Q5}VLqa=ReV%OaW3>(C@n@a zKlUx}>b(UHbJoZ1#sny@f7sbkZz6VzSFAVhZ^HN=D#1P0lx_q7pS}r!H4*HvDe#14DWn=U>7pGG40L9pT)I819sxl?RI{+4*FB0YM47J1Z%78p!@v7 zV?4IZrHZ%pgMXH5X3cUBC3)XR(To5=B|p#g&5<4KJd`datRwRd^bFQ#ebN9hFF_(p0Iy&?(z8l=N;ib$7!c#OnH1|vI z(nrhRtG~y0V1$C2Or5GG<{vgmS56w{%%C`2U7Ebw!L%qO?}qVA(na7vQZGS}AIc(tTQ|>_g(GCCJyFQ?@4~4&o{z=}e(V(C zG9gcw<3Hi%q~1o5lBq5@0!cmQG^=jGIX~j-*W2+PlI-4+twCQrWK7>8RekLnn(4-Q z>H>~06!wdYpm!8o)1Y%V)u?$(FH1@e8EBm6-&_WxM9XFJ(=Ud1xF)sav=JQ!OwBe%b@0QOWGGgn;u5?zeOS0 zh$t^qpd8kVVYiyN@p6|4{<4(a9C@4K6|F9tsXo5^%0!A7=$=P)NEFOaIg&3^!Dc3i z1~SrJeby0RIn8_sy3JQ5vD!?P>XWyb`O8prxf;{~ERovqyydKAuGTU_b>ZK^#I0)k z*2R0f^Ru9w9sS#hhL$M`lS<$~?YpnX0?&w`NX<3=^+WK{;JhHNl4Gct3P9zb#2Mlb ztP$*|1VsC3Qs4n)FLX`qsjuI#ilo=JG3q@>qb4e~^!pQg8!{69r0A&ERFL324~zNH z+~v8k+Z@nKNc4#G%0fl$Ua{Si*J^uY8$Dp;y)C5k4%5NL7_-DdWki3|^HdqLTG&d; zlERtba(-SGV@RCEwJ2XYa=>Q{A@EiW2q4DxZ%)DJ``t@o-UjkPMl&>F=rE}qjg)ya z&AYFzDj1e9?)dnRx;!?2lT``VrgU=g>T(0}17qbsE?ILHVhPYS`ba+Q3no_S;;=}e zLlu3w*t+Q~E=4ZC$SL`YhI#2s%c+u$pUT)Z8{0^0Hd{L}tPZ8+2-C~ESenp330iIv(ed2$opXNDjtQ_^qt^~WZ zC4}6qldb+q^abf8DxgwkeJT8K*q!+ORbT1*7H#*0jWz1hEO*=u45I~PM#7@#M2%WE zMj#0(;p*tSP-ezJW|2Lsq36lhs)ckh>w78tdZJf3SVr|X%+{**0mlz#9dta|MnVmnr7?RwgSr=2lSg-C5Zp`9tq8joRW9(q_e&Kb9_NG^4cur5p*@TBJniZAB+P!lr$`Q@Y6-}L2G12@?S5xB6tGFQsSS(suF zfO43J*~Nk6EKbM8_auU#u@LXufdaF}-^4>z?&eowgpZ6SnW@d;ADpeIiLmg@d?7p4 zR?}UHDpHw^CX#EvX!OrxDj&%a5|oM*xvqir%jkoVHN7!HHglDj$oNRebvF%Tpvg3J z8o-{(G!g>y#x3NvSw?0B4|Y7c$G9z>=BSs>pkdjgy!Lj(;{ODxOn-1Z@VIasC_{`` z=SX*o0tcc&!7KY_o_s%$BJCR%MhD3E?xf$&mY@57A7J!T?Gj9>}URzBLBd*5*xJ1Crna4EL3oe2tXfXKTQ*1!XU*U#~ve&%o zr-5CTghgQpjrHln37Kjss^GKf@z$CZ=gG z9bu#T%Yr+RZ}kAg4*eqj^IOR6N%uQAj2E)SA_<-vqOKi|1{OfsjN765Eo$uE} z35WdN4(IX4w*of#7WrB<@E8OgszCHF<{CPJt8c*wKk9jiSnu<#nhDuRX4bbNxr1)j z3ZD;`!_+(BiPxe{(3b8c^Ir7NL%3(wf7)qYXe>=%GUhIS?Z6lPT^jG&)^z<-nVvV7 zFAAoYOBZMV%2mrqD~*PH+*jm_bm)49)Ze7-K^@f^!=~)G9AmQD&&_D9Jn{ZA3trc& zkxbQ}cP8|%WRjMK97Q9W~-qFa~!kt3)1JZFa& zK)hYFNE7U-qw15`S*YDS)NtIPa6+konJR+e7U)U3D#|R&^CeUEh0ai--nVzvlPz$Y z0hyL?E8z|?=Wat1qByr#%0u5jSVdg%G%{(j7A~Cv%iPE}J|{JiR{7(f!WGIDifvNs zzw7XXWms$g+}2+J(phTnygPFv`&qI<_d?K8(=wR4oyXN3bx)>i9q`U$JJg$bv zzX_IlkVLIf_uC>fXh(&$&TN)~znh4*4X1zu=@k#qepIIW$uXH^_TFQwxW4*rR6t@W z`Q>jCyO**qN|U#z;fb<8PB~J1u&lRuQ#00ZGyE{qlLrpG4iT2r135wOE$%k6PA>pC z5030;0RlEj)uIsbSu!BfN(e$dH7Ztx+YS^2=LaWb1lJyFu<}q^H^Q)y%D0n2|ORqZ#l51Fn4({lXVY^ zq8ny&YI9E*!hw*7C8{!WG5j?oxE-JB8vAx<;MJnZ*(uvI_glhN5_?5>+R`*4<9Cob z%QLegR3&sVZGWDmLd2%Fc#h8O_rvbLF6Da78iA$;(rTVzDs+kI;ft?oDatAdpolOE z${zPgYwbzw1R3*3pNjZI<1+;Z=mBk&cekc?>m_HfjhI`kqNsdRy!5~Ldw;x^w~8#E z^0+{>g)V_{kL zF<-6jS9b*QutVtSBb}Gz!=GCnUj3AS@YHn3Y%>NxgoJ91P zxgv5gMe64A;7M{Gb!oOtggtzy+6uSAwiBkJ99mht>77Sy_%bhkDs~V0g?$N^C6S`? zXZNaFvnzMbNXAf2b}3~<{Q#Z42dv@fD=9=Lt|Rff)6c`>a(k`0eZ| z@(_rFMhed(^PTR1(&ZX&mtgh7lNn{O@Vr;eQ3*bhWfsFMAxHPWN>clqe|lrzQ{;HX z+#LLMNmJP6P&lO7awLE*yxL>@%j8jtv5?zqLXanqgtD8*3WQcOXZpHM=uvOUxvNuZI6Sq+pLaW zKXchvaMzPe)9LUW;N5QFU6JS~0YrX;tftuW2Wc9F&skU4G8$L1g-yrUtQSI3Al$Pn zu;`6p8yPIY?F!XT)|!MLJ0jocVw?`Cy9|hMu6bS5>-LZYDi-IwC+mI!iAc$9uB-5KG$P}jm7t(YbJGKXVTmfryFu znsD+sC~nECjUne7-x}$=ii~$@uC`0@`ZbGjgDGE;Qw}a>0xrW3I)?oHp(Uf5vBcW( zl1`~{p@*zGS|8t)74qG`%Ndl$*B-v~a_GsFQRBq$!sO>Z&sOh|<7DsSNjt|1RW>GY zE9kGCIGw4}Z*K1iJ8N^B-E25M2(LEbybN)4C`?jaAU_}1FrYfeSdnO_1v81-X-LKnR$|h(kyIqX zARYy?+&Za{{0!6UUwP;|>ift2w7IzD!4j!+X2wC!aRFjeLt;-}OXwoz%a86wajrO@ z4YZMs0(h;LndQSVDu(7JgXd9vVJ`mIK|cYTHsTX$8LIAgUe-SuyF>i@&iPMu?{lB? zVh7V!-4y--wy;O&3|0uAboWhtg|yHXXqu#d-e{?6nRyRA7gBpB4}l+K4?pJ`C;#R! zC1t&XW#_&qF}uv@dIpzv|a|BR+*R9%)1m|taWb53S zm>1t089Yikf3Th)C$hTm%ld^|J@>lsxk+(JrHT=4Tdh%1wew?!mYHH8{g@u}UxDNgmE{rN4oQArUn>jf!H49i;gqjJ~X)&{C|IA2A*EvpyYrvx)Q&+U#GZargRHj;Jg&agGypY=A+2PxH zqnp~jpwmL};dnJCip=gdX`>IR%K9mVH&w~0zo;F{HM052$1vin@yfN$6SIDqR_!lb zbA7r=e+d2j%zp*guY>aB;CB5=diHOBv`}|tn;V2OG8%S}w%ceU*BWmBY4qZoaNp&x zM&W}&Dc{MZps!6vqWok1V~voU9XD641g(=1vaXOmR#`{Rki;dI5@pVT)oL29Fk8S zaLo4ZJ(0PH6AM2nO`$^91 zmUAB+5`CEkNOv=g5O)eC)mz-dSB{Eo!&ij&jDkv-S=sSWNh#rn7-F+U$_P117B_Ma z0xL13ssVM=2d?%sIGqbT!Jpy1wRjq@*svRe=MMkpbw!>o}c*mma(QmUi`h$floGu+1a*t6HMzP3vl= zm~h4>l219d8&|J>4@T1!i!;#B{q;q8meJ=^DI^;LR_LiMlRJVm*_T}bT4kCc{mSiY zo{C$qI`7HRCLED~%4kqYjXPFPw4i(X(=yR`x?+9vxE(j}?PvLQA{EToiOdkxS^L^^ zg*qpp7W^-|u>HB#(xhojjCS-ld;oy*n_>;?AM3!&qN!I;ER}LPoEEgz2jQCvHnQrO z0@BRzdSS6Zq2Vg3t*X&+fN!GSluTNH&K}9xy-Q_~`?2Vz?)(mT2hn(l>4Ohpqp2!? zyVTTw?(N*YnB;rVL83vOvfEC>FP<;Hu{wp}@PyA2&#!2N6h#0NLD+9IFFXdf_R9L@ zj~om&;=W~oeC1FR(NGd+w5`4W z!_BYsBe(x!DO}JsB}^Q@3zm8fTN%0jb~bAH!sm`n$afyX5ZROT{9Y^F8o1)5H45jZ zYCL>>wKr+b=u728cp?*6v19Y1wzq}5%V!h!215z zBB28HzyI?|4UpVZSn>a!;>HU3_7D{;SVby2e}g5@6av`wD+b+v#`-@O`M>$e{Q=Pa zgd4=;fK<(K;ZY458Agn$4l!t+DfAOHIwnD7G>MEE!s zl!^Xtgzb<3i13fhMTh@~pYVVCnJMWW%IPsc;zRzeTLMtp4x5=A#vsY%-&qP?!D=kIOyl^ zf4l+BV0t_2eF6iz9n+fmFb*s6&2s;8RB(RFrIVh9fzn}`z}ZtM=9Bj|xbxF>{!Q81 zJAvQ81v&%XVrZ3|mpJOQ$w9kQ$4mHv&S{HFih&V(zXqH`Hya|f86oj@tg!K>Cq+- z=Y%vmxoI4gG6>(xq(B(CV-*8+vFZ_ zvSt9Ei&2%Mw#O&G;?jJt&jPcfU9A9$$@%^6L=JA<)$w#y#?_{E=aE~to{97L>vlZh ze&DA-9-l5ADLw#NzMCkEj2J=OecJP(V3qhuFI(2>uf69RWf^uglUBd+N3SW-fv3ea z_r+m>)NhXmPdF>&Yq}qJBJ!1fA8pzDieGdIH)}bW-&~)xSl{ES*3^=m&#mfZPC=v> z)5Q6I18GpAH$YkJVJ<+dF_>WLu?GAq+FWnO%X|bcfJ4IQD?pE&uPYdNf*|_7{L%Ka zTbK_)%(vO=cgAO8Gu zzRs{ESo#}svCMnRw(}rQ65Q?5>N53T7is*AoIDbIHM+#dBL!zHU|s`rr`*5t--KU{6vqP!=_UTV8$5hQc+ zkp*#bzNbC?hKUpC_d%{VgijW1m^}5^#weo7#LniW(at$BXhMlQyF^p%+y9}q|Y4XD+8 z6exG-_znF{!&c4fo2!#SV6zaq7@vO9xB>%k?5i5ET+AmL8Iq6X=$sZ~6l;zxLL}F6 zSWtTUx@N>{>nLnf=6e+Z%vZN6ht8kxdKNWLx=$! z)Cp!kfN&=Fx@j{?0ru%r^&?pRR{Q;+p2`(DAxU5+VugAhEQ_o8DR0=*w*qKF@iM@v zi-QU8cw|TU@HQL)R0kS!ggP!1Q%O6%Hy3d^)c`>khK-H?&c$Ye)!O!lTzI0M)JLM( z=w9E!rsF8)8ybx(@o%knF-79^fxijG0B`l(jSyTz?~(TKO#ok~3=IKhUn5gY4ZRC4 zty^se-oaApk!bmf)w?iUoRoZda3(6LZ#!jc-@bdcVS4hljYqbQ9Bu?GT3!@Cq70a` zd`Cj9y4}R*3OrU`1tPO=U-vLXkunf$B3YN45A(PMt@`P!A3_PC{>{sRWfgrkqKs&W zE7o&W@Xn-|lajUYzBTbVTdFCB*eO{2?nCV!o4f4<_6MpKzR7eP?3lGb&BM(F?#)fs zJ;<_BQB)|MR2 zZtXYx>`_|!Fj)aK_={L8>_hFDqOCDb&paVFH1G&9G8jym2B_>S!tHyVS=h*6luOAexCv_wYojv@-?s0eN7p_qx#@d;9eahcka3=>o90t6 zmSVK*Sr48X6AP}FcbR%wpl3r)D(S|qsPO1$1e+yQ(@B<2GoLWynG@7gcqaM?>=x#- zO4Z;6VsBqa8aBX6yAFT!)uuZ6$2b+8G?oNh(JQzh^_@x$FgvNu+lY5}lWYMVC~vm9 zSry#w>X7ZJ>DXtAa9+I8Vm9&G>r|i&Xy}Yy>ZJM#xvK@s&6M)}xdE4DNTv7F#`VY0 z*7F5UGijyK;bVOedB%1YY3~4%yDt(QlB7dJWvlz61L@{7{qfoPRDQBgY2D*2m*!Y# zJP3%fc%&KNe#7%l834?Ej~YtyO1j#QfsYa^Uk0CLmVCixWqA+nr{y>4Rn=%vNm;5ya9 zHr?m-Ynt>eOI-FZMa@mXOHGpBS3Y#?HbD}k$tQuR?HOa8PAQQOFDvfwD%)|QHr9)w+Ea2Z*y20V!l89HWdMaO&tlU`8UKs50 zkGt0{c$ffzrw6z^-s2XPU_luoSt^C-K1iBaLUkr16?vD$^9mR2s8q0_cX(aN9Rr>| zCb%m#=xt|f3_sM+(S=_-3$3|@G1iOmLts=3B%}?>17@PWb`w` zdp9x(x%5DN9!e6Z7w=arIf{c!-L%by3|Ie4smtAVik-draBkDHX9*K8xEm-cIi&(< zYph3)vzzHjxefGO?_mV>J*pIw1vo5g+CdYjuRAiym zoHB>Q#5rrM(r+uue%Po#)rv`*I(MYnp3cV$>&f;q1w}+0IdYoY@Eo=m0g0jpb7s82 zbEyEI>Z~TkzPVm7gLqDxZ7>p;PtiXIuQc>krm>thbiq9M8ZY(M&P*>$hOu!l4;^bM zMW~Kdk1Y0eT*hUtNP4{su>PsQcF!%TAasJQsfs95b@|;=u}6v`<3bHaV!#qfuT=IJ~QKxNp;H+KNq&iso@3g+tbE48bNJn?uLt3esGj zO@CsSU8szx$0;6&hpoL5H9)Ig;gv|HJ8PJ|dh_mq#-VY58q{D)`1=m%ZqsrVs0V%t z>eA7h8M3|kbbUX*tsHB}KEL#IS-7SC?eeCJ3EUL*f9`n?0Shog z54WdsdM)RK7+jjxv6$4SRw;HhBaMfZIV!A2$?rOUIuR=vP9T!#!qo-aJ+;K#39!M1 zgTB;B-mDrO=zbf8y$%z)t4IlS%&$i2G(rQhLsXDOK%2=>|Cz0CixM1}M!W?ole;FQ znr>s0@&l>C`(CJIjFi~PQpXQd2*c~1+KX;ND0=5>6~jRD3y^}!Zr`1$q_pO?^Llw{mtQ; z<_oM(@eEPMTVZEdb(n~6Wc1&+Lp4=cz1)OMS@~_o3btoSy}im1Qd&H2=IBeJA9qWM z2DVIgJoJYjR&DuAYM5SnD){iO>(sctkSE6JFACE1-L5OmQRUHa#MMBsu+3d8cq&m; z@;uOiPI?tt9jZ`N+IW>!bWs8?SJ(BjjR;(XJ?w0M1=lmpa}pSq-OHh2aDrs@u%?2( z4YY+9C(<>^!;sGl4U&(2FJx%hERx!`0YQ;jvM^;QP?RS(1w%ePR_+RaPut|bfa`4s zjb>nzBRh%>oZS}4prza7Ov{&Zi6U>^L*nIFey{5R5ZOdin5eD+#qh?Nn9o~upR0}M zwX#~H_?7~%%Vcl9Pu(m`4XhreGxzN^CDi^(oKlZlT4T>3%p{n6a-%fXmGTYbyE=zw z$afTSVHN_}_WeCRWdzwRN;eiQSD@2oJo-I)yi77c$x}qvordj1CIxX;v>`Sh1rEYr zA}6!HzI_adDY10-aEOVLo}r=%E72mX&h~SCK2JI%^vzw)?z#AZFWD8vpq)2BuVQge z2R8AANyQOvJX{>BB`Nu%iN^j3=^`oz>pK{zvG{a@I*z1RI7JEURcCKQ-IY~=KU#}- z+g{CW*baqt>M%qc&eN|f@1u!kWNi9b*bvunhIRZ&2o0_r1iz3GmHJW6o6kUSsT!@F zY_7x??k_s2b5(0>ESCFy2(@0%1AAArwo|^he`i6jSeh`j6rQ3hXxaK}dGp0KmQ=;d z`#;zMi6c1f)d!3WqDL9QYi2K(h3!TInDAsr;1T6M2W?FbQIPrM=j~L3m!0D7e=ZjC z+s=W}oN50>Ta&y$Uzi8{WTw3&jxJmP&O3-}!H{Ft#PjN2*-Lm9!X2Vvk-QT{J1*vo zgY&v$#`d{yLPU=kFej)4OwA)B@1yZ>Eop>3f#j*#Txl(kU8}$KgN$FPjc;gwHVauk z+x6MQ=bInI)dX=dh*bay5M3;ZK$r)7pjo<0oc2Mx+je>t`uufO$L{33-<(4!yB(S1Lbv1mRllEdB)?H8o0xTX0$2DYQpJGOfOOXc3WTYIam{3K9Fvicc z#QWtn888~Tggk%!J4kx5E0j4<5K%v_q7x^9$e&x{0tXL>sWn!dJMZiF^Nh^CZO03w z3-hP&wm3-IO^LSW_MGf9;DXCQ+BJ2vs+Stu@x@DilCRjDO-(c>pe)ilC2?&bOiAcT zy)(&Ax}dS0^_J`9{Bn8@!Wc}AP(2RkJda6b4R-75Z|{~6C>IK(JsJFTzwD5M-;&7X zd)qg&%K*AT@)auF2#9OUYJ?FB_g<51VXtTB#;6QYr56`v^Dtg)HjZk1Fl`DjZ8fNYMw5Q zux1{63ddsBZWE#0<;;k4?(Gr9UF->EN&mUu12iJ@22}AqJvOOc$c~BbLiaf2!07QX z;VuJ*gDib68(k^L3@%qKaYrFMI`8_pYAx-(7O0FG8mpkTIeL^VW*D&4mbZx3wfFmz z_i{LrS2&j_zB`a9yeFzgl{Sp%u(@7QKk zq=9g(c5OaLBS}sBX@5mLB_uy-CMl4z5PcG#VDI2++oGH zse){y_m7JS_O=YpAFXOQ{Af?c^d#{1K+mDv`U#9SCo4xrE5p=nA79wr392xiJw!W9-zs2tIWJUIlV=A*I7c#h6e?C$vgj`F zGdxtCR9tCamTn^%yV6rBXNtH^6lb4$>vJ(T8qNQwq@VUP5mT z%g}Fz*sX}b$#U-u+K3dl4^UO|><>?Ve2t?n)mDhj?QH+{kEoejM4SAKm9)&{>zziL z6}ds4{Wt!2LaZ)F2X8Jx4Z@bf8S=Z6m76;N#<2K+$j_a;v2v7FxabH77bzX!-(+9D z=TQt1G`&2?j3L`0=;S_$=!kX%372f(R3oWFIwO*+`5rFklcgX_QZ;7h{rN#APNJnq za`+WwfDkcfWpmK2y6lY9p`4YlrTk{K+UeFQuHE@oKQE`N7P|%FH}X@fc^)y;d+|Mz z<~Y_q$v$=Gz1N795Q7^KsqC<)t{obrrF;$UefFbZ8)jUyCpp^n-jkZ**Z1h9J~sF& z8U5&N3V~sd%SiQ^;ibbfT3R$8OQsx&*NbWTw{oGDd-#)=eh>kkgbo)9*xj6KyJo3v z%Ju~4KJN9I5vwMy+894?Zsg!?I&*Ap$_!c@W+eO#OmOgLo8r0C7fBJc>Wa0CVWHM~ z0?b0bRvCN%4%xf+EseRXlYCXDqC$Imu4oZw z2+Kvh|H7^9XgT+-4~uom7ZUL{V0XTKp18By35QxIpJP*>)_g!@%zm{I)OKEG#LGs# z+|mi>9C4BEq|Lv#?o@|sn6_#Ij54GZ(T1a+y6`-(aeKh8{O(YigTu`GnKPcDsp$6T z(V1d>#C&GhDG_6H^zlsfuS9oUUFE7#-euk{-hfh`oglHzM!I6;FYUB_;Ue#VXC3bY z3E3kFpJ{2L^HNx;@ED(4TVi0cTjL_SM4jJE)__V}nF40g6`6X-k!p^P zvj*9kWlx(;2P(g*u5uxD+ z)iLcZ!82LNnwOKDx2HIZKYx8?l#TczpZbva&Xxl58Psx0qIU1>;%j<EK9ih#j{FePk%Mn;aqDKJc2Yi!Wj_!G4sbX|m}R={_p*dbLG6?0 zk)B!?L;|_(SbU^JiyyR1eB=!pBd?gzX)4B)!=>2K8^!EM@6fOv?3T0-;2J{gG~es> zrYt!aC-}%GoOm3cRbiyhT5Y(Q@a?yD;NqwHqh*B=zKI?7mf)BnNDzv}+@U!5_@W`O zpW%1WPEcU_JoSEG4|ITOEUyW3lS&a_Va@LL0sIP4K(Oau?gx6Q`W4uLgeQ?5FZMgT zZmtg;DUHGXdeZOSBZEygeoJ?Mddz)qVhpT2DY-)(ZDHaSAxha4-MFCrIu#HV*Q=xel=vz->H<~$@^Nk{`11lYrS zJ>49=!A+TV)!^5a=?QhUJ` z+Bq)|+VR%ZU2Xix0P^lrn*$uhl{-=`?~twd7R)%(%f<>tV!DRu)vsC121{o5_3R2< z($59Yu$(q%)tbdW@;d$kUY}`P{a5uGO zm%Q8WPNcZ~JzAl;uDh4jXNF!@k#C+GXU|`YGN^E5po|I&;ZifFxwwX2 zRGW%C{Ra$CTP{Y8d$^*(BeQU=-teyzBNAo|G;lzEMIB7k((mrKAEHi|^_nPv35mExF0}^)w-wzzl??QIL^*Za0 z1;lZe9&rF0K==8~Y>&mEKGyPqX!pg!OC3afuMerF6X!QW%oQQTTiBK!^^^WRk{k#_ zE0BKJWTQ46*KIM2Ml7dw+6aC1_!M>Y67Lust{(Sl+9AcE?e^hPRgja4wCHF8ch@$d5E8h{cEZIpufw!Xr3no7o{x+R$dL3f%QNZ#AdMm^)7Z=}2DI`!G z!U-i$a^L`8g|_So<~ko@Q>6VK}<8#&i~MKt@JQ zXPkC@IK(I>gS`LveMmP;eI;9u@TbG8pK(5j=>xB0{0`#+)F!%lbWf2 zp>OaB`Dj}bjG>~7NrH+Y!Qrj6+c88k`&;M$hAhSV{Wy!dduXxjU8j&(X1p zsUK!6~o4-ArgEx3{{C51z zfFbjq6+HkwFS6h&$bm1iM&?Y~=UW>~vw+MgMCIdfa+@jtTOTf=mr ze7Q3ix^evI$kRvF9(wWH$$dIBV@e$)8?&FVQE?$=|JBDC4ImT!F@d826SwEa0=9dd z;>ap%$mPiDHxy{+k3O#|Bxf@)!8w5|`gj!(=jLJsr-|iH3sF>lB^vPohto534_?a` z9hL5)UUeA!mYp%rBrTj+d9XU$Cdoey>->gVKE>k-A?HG>p=%h7$Z2-vTpp;n9Zxor z0bP&7yzkBMu#Bj}N)J^oPeK-ach8C-sJ!&L1EGaD zL~9dXpJ^dtcUX7D=JA_T3%fL}TdXZw#=O1x zp|lZ})6k~FAkLc^z1~EEveC7f#@1pLo7S-}dzmm(bhti5obX0%{QHym6QmfuPHS|a zY`Rq2`EUpqiIk|FFf3zFnWD0h+*x?XS@tB$#_A0*X+REHvxRxc3H$Zqtp19aEQCJ{ z_VC0m%PHA$fONgd)TmqP$+zW|jd;wQmyoo{@KcX&akQh@phz0d+o?vH>L={g7pk|l zj*O@9SsxQlW$W0Pb{0_wd8ghiJFtJ77GZ*qZokCCS0ua)Ktgz7wi&nX9W-_{b!BI`)wj^o1xd9Y^kC zXk_Z#*zD?0-2aFYlP#sW2m{7Bd$ta=S-zN1rjbRy)zW|E@q^u2P1rkR!WUnxG8os; zE=pet>;)g0hr;Rls4P&$V?DRWRE2?Pk%!qsL%Af1-H*uzz?PRAtmsrEeRw9k@G+yW zCu(H>Y@gv1NW?DPO(|eVchXfdJwLZIw@GLPCv-FTnW?xY&

m^m=cWXDZ7b!E{*~tMYDl7vA zp)oGU6v$w9Z$V{jIP|TD6C7+Ca(<2(_>rh2rv7~Skjy$<28jsPEV4j=7B4!5S@AU~ zu+`%>-qnV(rC?NV51GZvzyA{&owCV=fF_(G5<&z8v$cmE?)@P37Rmc8VWGj^9!|F0 zY^;sY2;z-Jr%7!>hyxoo@4!n(dWia3%L;c|aj~Ts+2tfAs@fgv!!ZFh^w6$yeIO4_c#AMjQR%c~H1iEKRxYS*Rs{?WZCX2%f|$tK3|u=GUoFoon&EjekxZA- z-g(7!TLm1|F^!@;dc0(NOeUUVc9DsAcF!%|zFawJdpgI08wM$=k!B`d#Jc8xPi_H%E(1y{o* zuVT}@+lXLzFlQZW!U5}kQ1N~FzzEnxZ;H6u$n)?QXH(g>dTyGw6x`*G0~9tj`X%L} zhwX&s^L0+OujNMr{(ueB>>Ad+=c=qITEZ`_vF)17+c7ipz~=!}NcqD?)r9 z69Y9Ix5VR9)<4S7Z(w7@n^2%OzY&A)zxZNr}VZQ%)?3-c-;thvZzx(n5WhE%6C#m@ZwpVvOIeIvm zRKeC#Z!WLdNIyHt+^VJ6{s^y=m=Zxoq|Q2dwy+;pDQl0ozJ$FBWqZBClpZg0MJ?@; zrpCm@P{nwj`jN5LnUBs$BxkVWmiE%liWf1;S5kdy+2j$=Omew%m)~#I%le7MVc-af zY>hPT3>ll-KVhBp9Vr~El#McFQ?Q6djI0@&vcR6Yil~acI83_lP&kji?UCesRdIJv zSy|$zFIhF@N1woJ8-@&#U_uv&N&lX#g`Yz^B~9g|>gICyO+3hR0FEBHg~3{rd6R0p z*z|&if*3lZHxVz;QWf?E8SB2zSc@Zw?t4sJa_oK9H;)&ztaFIj)mLwb`Q1TG>!xjd zgru;bbU37S$CM%1>F<$a<=jj(v%UR|+fNUiqPP(rpCL)oEWe=xX5jwJaPLikU;a0j28*UFhKP(EKY5YRd%{W zp_YA#g?(nG(sC5MsN?QSyPKlP7vs#-*cnFgSTg76_Z6o%S`fzeUx`wA9<=iQJ~MTM z633|=3T)39H47Z;MD{(L9bTiTILjFZ{(;VCbW=$I}ov)ai&)3{X) zy*Bff*3eiLc0z6o3(xg39<++~=_}hZQVGH=DNC*`+2i(%wmVI6SxD%Y)XTvsmw}k^ zwjT4(3#*>4<~u74#Tsb^g{s98&ALA%S68EMv$52Ud~W5-PXk^AVxp)pcua9HvXw^Yx`P49gRA5ty%$GF>DmVp&og zl@nihK#WYdkWP{MojydBD=&&V+>U&0dAwNgi8S#(y@zQ{;B?Sn)8+(zX(cxk8! zIy6U&YVsJV=Z=F&5>gz8qipyY64h|mt*4`tfW~c+ra(>-R0Od|6{fxC3-cFi4l07J z``6-Qh=Ig_&plK`K=mJgq+$kn-L_bXVekisgkq7;z1Fqx|BGY6fKVY3L7j}I@Q^Te z;!u^0cH_nea;z8J@-dqG7C>e!#bQs6&d+&}(-SY!w9f4cgAJOFXVpEAg-<$};BL|Ff2 z7ySG8NRkZw`;Y(Sv0^Nk>D^mKl}fDt&6WP!l@LLNQU19#|EaktjG#lK$6sdr_m>LJ o2s#hve>(3!+?fB*citbA9?S~|bmwJBx3rQj2ZvX%Q literal 267052 zcmeFa^_N}OnI>E%m5P~}nVD}fGc#A+0u?h?rDG>{Y}r=gfE`oYaS}ruQ;c!qwQLIw zl4X=-Nvk{5i8I|Z-~0pL^PG3rxg{l?7_e@c=D1IhZY#LjgWNX(kG|JJ00@ zG7oUp&L|`KDRH^E+*qo#b#CL_y^Th(-XTg)y2$)Mvnr}np!|WvcW&cKahoI61!iSF z^931$@oztOWY)&s8QnKbeM19-nV=aEU{2F4iEgiU&aP;XvB&|uKnBZpAg#?-E#LXI zbO`5KB{N57A*c$bXqYijecG&Y8XDej-66wvIZaTaq|pGu%YoWCu$7Cq(WHmbb@{!M4k3gl6Yy+N${M55P-7E%XmyW4rzFNyszb%sC3-oG0DRekK0#=f8yp zH^3QyAx^p=y6~Nbd?;JaN$wDgmzdei+DORQU9eS@&}srw3^9rFiLdVv>~vHBwZ9;y~}*2?h**c*h2WF_^|E4jwY& zic>?3MfT_z;8lQ)WnZwlPm!6wJ_03t6U=%5tcVHf7HSUe=j+7~WtecL{eTgVPAe#h zpahV>vM~e-tr*O~T2L8d#1-TQvIYTmaZaZnV36E|b+mc2bwpTz3^fHohFiLrE*?Th z0)tk^hJ~&|Nn4cGCUYk$KLXq>&~45+*LppY-i#Xw*8XyJ5s)#CAODD~#;*mSp zY{+OZGz_}$%4i8ho-bIGw0SlkQSCh;L#@6284`;(Sloj~1Ik5gxrITQ3kFNL{@Ug~ zN0*m|xIti@HVi^g zWsp3ieIyt_XXJ_sKE&hN)fD-3S*qk3VkFwGfpH7min8t0f@7a0t154?>TLAdVB`ZV zWj@8ut|g^V3P!-XLcst#kXEIG!7kMIpW84m{!mjof9_MH`*i6Y<2z}iO3SCn3LkDYr0Dlp& z5N(NI`J7nCanU*gAY)h8{2j zvIWsLz;QY-Ms(U=paK{Tt52SluH#KSGs(9PtlQVaw=#Wh3MwlXCnu#ZXLzxF=?$>U z&y-(!0g}rB><7MhYL#Jq6K1#{M!$gN_}Y)_KYm%7lM`Hs1kj_0v4}=BDCXlQ!E-<# zJqiZU?n^7nE7A(QsBTm9Fyu%8&r2)y2Y?OCgXv(jwW%IH`5Qs@Am%e|c^FRi3r88u zDuXTkc!Mv9NfQV%fLO28j-ahOG?((0E)dY`(96knl!&Gb#(6m%$!~&5lv==~9WCLc z87^S#%TWg$N?G!r_IdwAK`Y~>iQx`-f|%~WILSui+fcc~*7xrR7rgg?$oM)&M%R1K z=_Av$_gK$34GiSpV6j;A(3tItAb^ZBub+KgRB#$bz}1=Oz-lj80w@!-2ATy0a4KFj z)j7Nl!A(+nfw=5^P%+JtQ^L^XwFy@3Ol}@orAb(2KN_r-BcQD_KL|^xfCBUthyoM{G8j#pk*E+jXy3~B@Q1(o}GjAT&pViQ>-(-fB@LTnpaxVhHL)O{E2NI^Cqa9ljeCmh8|CK^+#Zpb`|q}kie*jzsi-F z^|1+E6NNEgI>O;Wqcmtfy25_J7Tm($al_O<29>=>N{`eYr6UEdBCHJdnRZa=1RIdu zB3pq=vEa_WJZwJ5^UUqwKpPDXu#*e2v}^VqJt z4Dq!9*m9l?FhoF^v`>koTlwr|o~V_e(Lq49lG38BlSBm{0tMGk*0eQpVU@E5G09Wq zEoguK?NO6S>7e+!xhigxkEoeOupqDW8>$x@wV>+{4jCX8K&QTC>9!oH+|TgL0}aK7 zWW=m?Li3ta=eqI;T1>+_5U@GDVcI8CPffus1Kl9*K!?4q`Ecny(C_a573&Rm0kn5a ze)E|rLiMI zB_tYLX}-8zx`nqoc^>zKYOCU^zL&|~-OQcN%O%3=)9NPPdkFeu!WdTa$ zAttXrX4*upvB8Kw;~@9zIS3~B>?KIY`0Bw+uS40DPVGvl#w=;$MY&|T{KGpw_}*y8 zr_V6kV*9~ZfdpKWJ?eVBA4(Tf0O0fB)a%Bte$Auz?SnDWN4P@{33x3@1)~b&$=1t1 zF~!LIJQKC0)7Z{v(xKvsrTH(hbyL^iIy%-?^xi&b&Hd6;VMYs9^nCQ9Nfn(14j?qM<#KFS#fJ=bv0z`=#tSm zw>&w&QfI!=g$|pAs_tboGtUT;d36i(9IJs(wRNp@)z{^Y&d(+ekIj!56E|iTIG0)e zMlRG@+y<5~cKwBMuRT7T8$hf_d}S!%i={vnppFnU zZ6mIZFk01RNS4%SKDysStcs4gqA z`1=^b^{g=ArJvqsa4}Z?1GnJn;NVj50Sgyr^S=jN$LX~jx(YmvHUGd0MYtm#bTl;h z$lt|iH7^(J#Gn5zPXqi@g?;yy)!(;ek;UonU}gJm21pku=Ib5yq z#K8j4qg(&3HE}^IV>saNd)C~=yH+m7^`m@P{B0AR{uVAN1PS;w1ja#ApqrmSjOWGk z4}*suQBOttZl$0T&{sQsmvkKFe06A^Am_uocMi;yNPGwn^1=4!?>)Q2?CmHUYO{dU zF=5)LdYJPy_Ua2Bo$AG}m)o~cPtZZ_wR}bGPNipm=2t&^k*8Bn&;+Poz5vE2jZd`s zpI-SXl)YTO%&pt3$ z8hWW`lowfyFK`WBy8!|kIZL@ZM<#UA%`bLbA{X>m5zo1&Rl$`jwp4_S(ZRdxyj`H) zEjtExrTxJ}Gf~ij)#S_EYNZT$_Ohq;fIX*abz72YQS|c?Q_eEKfVd6QbF6TN zF~a`>)pM=7WH22()?s}9#^Dy{&5@3dUd}S}#V@8V-+6;K7Q59_Rv703Dy*U9{NcfS znQefpkH}pS>&#wWctb4ADqE=Y1zS@+* zXGR9MW%M=&7xe2I6jd9b8H!Y1=)+5VJpmfkK5C$=WKZIxwK7b5lzaMU-w|immUIh` zb$|O8^^DBUu{`i)K8b*yX_4SoDaF+VkonSc;rU)DJP}~p9lbkSe&rjSoNI5>_Pd~& zEHV(Vr;H2Lr$xGiUwrm(>)l8Ava={%2jL&EPR$k63fiiqLyiW4sT#vhP#Cu~V&M$V zve{z~)&c4Q0o=D|FSn!oO`FapSd~Et+99^uc}Adbxn7hO8jgkp=v*q~M>3NcA8z^pF9|p6^@a#z+2IxT!)@S#eO*iECJ$rQPg-`2m=gNDnI>30 z_Yn0M?w$dZyRK^a2pYhyh)NPAq;k1I`j{ulyM90T#1fcb!YW`zv_HT4xG1I&DwK{f+GXLb zTX)tOY=RH|b&O}_`|k)Njd2Zx4K{d|dfI9wi$(w#qyicqSnJSxCfZoZSS;+j+ZpDC zj>^cDL1=Z*^DQ8t`0_2_r(d-Lw_Wj6Atz(tX`pi_^`0jVWB(RzR#%NJTweO}$(m}-_LAHqbF-TLaS+zH#9o{%I z`I{&13g*6VLM1o83#uKIk%9sJ3U6qPyJqg4t`}VXxvr|@(c9dgJuLvJdKG>2+DnV9 z5;P~Ue*IQaqD9b(e-%+&wMIQZ%&q+8BEz4)_ayg}3gQgjHrltqBcbx7^9&iko@#K@ zIjWM;>|16i_k#o-*2;DR+p8?70GD7u@^nlY`I+7ZH|Kz=B`9-xQOmGV! zc2V?wCnsb`89;y&k0L0|f%GZ`Gfq};A#0E_O=aLF8tS!!2h^cLW#Y`e%z9w4B{;1` zRBL4!v?f5i+=rWY|EEvOa8~97bIE4t^B+QlgE~QsHUW#p`A0?H<3{{MFL6yu5G{;0 z-2*YZVnOOOP9D%LKrDg>j{>(Uo_!4DXb;xv?gP$jAKy=lHLZ${T;>)}&)^W0&gv=@ zW%}qon~-K6Ds$}Lp1%`v++D@uE*e~fCSN=b2B;-?PQZ%+%3J|*tdo0qr65}SaUY%v z@Q$E~7%&szrJf;1(~i)U2J#AX^z51B+*2SONof^4nuRgd5`rH4-9t~xC}Z=}e+6so zDD_n79Eawh4ATeRQ(xZAb%FP86A8X5+Sa%`lyeX0{XD3-w5)d^r`1dY1EXkV$Ow4S z&M*yY0J$q;a4$>_E_xWeCT~~o;rYuzOXI1`{FYrBaUFE#RFk>zyHy#QEl;1xpX}j06u+XJi2Wn+{f!!Y6Z8dzSLo`g05m3 zS`=Lej52xaK=f6L_d74p(FLGsR?YNQ+Dj`r1}KXvb)f>|Eyv$EDXI!tG#K8)(z*Hj z`=5d4w3o+RosmxPVVd5RU+3^<-0`_I4N`M33mB)TPM&%&2(pp?mH6yUu#B)ePX>Zu zfR77?2VmoK<`J7Sk22Q)-qrQak1AatUCd5B%>1DdJd=XK%RvmK=~dtN|7F+$nhVPU z!26ogcf1Daf7TITc4ST&3Jt!m3-%dupS-BIw|IatP-HD%aH!n`n%8N=g_Q|d0R_j7ccPvnNuaX5p%R098J=_@Z;R|3%7w)62bT9yP3>QEx z=B9V(6hYwZ^1j}HvLuZVyYVky@nVVqUpyod{Ht44nvXF49VW(j2}7jOC*ZADX5V_E zssfE*`oU*!fyyVR4N)AGuGIv?<_ zWGn#c6wr=$OHV=eXFq|~5HPU|q2hU)x^dMq*8wu577XxaOaWT*8*^umL62c|>>iH^ zkfA(A1CZrq3=-`oo{Khb8EK(abf_@Y+k4P>vYL4sZfLw=WsN(zc<}buh{ig}tlvDB z<%P!!2X!%V1oV_(9D;WUIIkhS*%5*c1deQf@erSC22N=Tr+&0rI@FE4)GvMzT;R!* z@-)B(D3V*w232A>(+qdADl%E}^$%&*L|v~pzUG-yU2 z5T)HC_1a0Oj9WSaV6@Ut00VQx!Zl>3ZpfX<0?f{D_x39NI88`n2`Y5WrAt&^$7Cqt z@-ubnf>S5U(PS+EGW%KCnvy%gJI85->))zAeLv*Y$HA7?Dl*3Sj1h2eLCc?>;61>+ z2ulXzFo^3l7KTc%@XP@`of!5^4aQS1Xf%=@q49yLqgU<`4ZIMY8&)j(DJNh9(NnI zs@#x<2G9p$HRzWn_<97Gseqx8&Id;T$n~hZOEqN9n*A&{sme$m)Y;<+iye?nP2 zXiS0t)9%eWM>zWNFJ-{AMQKkiGjG0lGg|vT5shk)0mgac9CL+dD9UpNcc4k@lflZ4 zG75pZs8a1!bQb#6m)jGyAGKk09|a9_H6|cXpRAIZ69Eww+&gnYMbM=uRTu=w1=&`n zc#)nG#)m2!J>#Feh88{bSCL0=A9xmOkyy-FYWMzzlZM1wb&A1F<27b>ujE|SaSti>#|P?T^J zd(?P6zyRZ2owe*_ZgH*3+9YsbCXX)62GPO3AghW%jxnMIb2*U3AQ!z$6vi_^LrtZz z{HrQOBUR_U^HT_O5KtA+FG_7Iq(k>c_cmltlFwI<-v=ckARvz4*M{n>eF*@{4#*A! zOgMnC-o=9gZZK2hg=f*n7mR^8ux1bt%^fgKn?O7pkYi0N(3+n;%7avbva6I8KK~k8 zT!Ba0W;kBGlNWG~DBsj*s}T-V!~`0Q7w(0oZc|r4w1EOZ00TM^M2~Z((U~a+`&=9P zYKn)-rIqL$mDbPzZjPlF*cSn=LNK0F0Uy7}+qD0wFq(CQ#jYizK{p9n{Y5GV>HSbH ze4BftpBYmCv*a)a-O-A+m>HuDfdG~Z0Z%g%A4a1FErGFeEFA?4;O?XCxlLNz00JV_ z_S2x#Y4LfE;9RyYdf(fSrt-X(!&Uj}L{X8Iivgu9Pk=i2wfmvxvPx!c^)ZS)Kmga- z-`E0=pOPMMF3S73YPx#d3`0)=0bD*mqFox%Ymil-u0~!$@94-h873*mN$<9R_Q^@G zdC;7y+;+Aye&pKSqJVR#zY{EkY5(r`!2kx~y>1bl^FuLBvHoid*Il6Pz1W(GN{fmIFBl@>|xv4J*Cs5qs_VHUr~5F1Lv> zJW`gseC>x&nCA53I4A=I&?rZE4DWDdkCL5ZXwJv7)9KTD%A)zSum(}kNJXHx?vjqM zk57n#K#CoKs;md?%7f>g<0kF2Z(n)(8P~LoR2N+SP7NQQ8z2?E-B4+{iDTMR8k)ex zBQ3f<26|-z7(L3(jDZAKZZnMZwxzoA>=g~CXfVNz`H#Mt%QdzNI~spl8wn`6~oEsKK+A_ z-q0_AqV}|&p55xj_l9U0G(fE0uu*gzd`k!FCdd!OK;#lF?J;!_lTKhR{`pq)0l+G< z%&aX=shqg{%CtHFy}%#~wFh$m@o~aO`Dn|9Uo=1pzPz^*%Cz3T{7iRqWj9OLYgwkPSYafBxYueHBo-$xu-Cp>H&(LK)Z4 zx*20y5f|}%j64)vf! zM{{oM<)D`Vp{ePl7G-h=9Kc&T1{HI=3{yRlU7*C-nCpk?t*?Yn}-(y2Ui~ z+-H6OHCai{J))|R>~WDFmS}dC?lUh#hM)i0m$z*549+x{j^MVjBqy{ky_^KjJTfxb zfJ_mcd~h-3H-Gc^ONG)WJOYAenw?g`D&03+v#q_38d%;0Ho7(et-}{A8Kk;ZSfgVK zbm28Hfc7wmDt!9NBkLnPG*e9n3$oV^gN4Cy7_5yzzKVe^=((|-G$t^*Kc_sCWr^k= ze)KQocYRGp8LA9&?qTUDQw`Vw^@|FA;Wk|rMP-ZL8>fz?{KK#ij2j?n^HLs$OYzw3 zq@(zfprd}6hq(N0FjYUHZKQ$GjyWI|jnD*~h6m#@4GaF@)haij7IyO86d3V&_qm-F zzH;z{&C&_(2U2yZGPfIz(tfb>-#$y9FxrnXUVsn22e!xeHz2RAZvT^~8J~QX0Gt$v z^3MWy0GXnrK!&(=L71*VQDs58?4u43i5>tE392^Cu4Xd@na_hD9dv{CO$x}RcVC`{ zU;*Tf3=&{D=`?c#um`P{UVAbTGU@^&fIj9St~~(;(9C%>gmNZLU$~wf0|9tAxrqYa zL1#`%ld}Lj+}Oem6=$=jgc~GH7VLu4{lE&SNmP}A(zuD|yE7hCB|bdH;9jHWJV9rE zd3cDXo-N!2FTL~PQOytD|$@|9SkJ(aZ^ju(0K+s8ddi2N@fKEUZ zs41b+i|gQqv!%r_NMykX0_{p)LSw;&pNT4*d7=y^9gJ@eka-?huJnP^ zgPfbSN)WDV1LGOAemm&gU0}+CX;34K^kK3A3}Ez)fztRa&Ijq}D&!@MzDc_t-Xi$& z_0?i%sSI%jJ}{AI%vhz%I;0KG1mbKXAU~ES?K?~vP-tLo>#t(4Ub-jOwJ)_5_XnWO zQ(u2Z23l8H$LP!QZobFGt}m%5MT4g+0;PiJr}Zn}2Nz6O`At_sjR(1aXpjqcak6mk zylXB$W~Geb1IRbpM4u{k;A#U7o~BLn@n+QrMqjMa2uxN6jSnbz6^uEt%{=ix|B$M6 zG>L((^xoU=*8ZnLcR7K8D$qQgsoWb2Ro&+dl~wu}2nY^u03S+nWC)e;A_dR(*>*jN=b)rH|>> zj~)X9l+_C9G=|Mz;<(;b!qp2wlz_LTVIQKIv0npXfB>H#d^IA2k+d|!sp~vgU;05a zK6%?akc=c3D&1M--~pgU5HE>Ru7V0=ieS_MUGt;Kfuhn>0DY6dm6(0Z3LqrNC!mq* z(mpY^+Lu1SJl!ahcT%F!lK3<-8`ELG`Xm(yttr*=TMpGH2cSUf6#1=j-nkd zwhNj+|L6|w0nCU<;vC%a?3u^i4brFcKrS-CL3Nth0$Or*kXy?DZGn~|RTzEvO=iwv zGz{?6ejp|hvr@36*|MUb6-^cYW##kwu&xA%w#Y{&ogv_ElUY~*V}Y+I@BKn#rq}gX zq4h5ItjUq+JnzQ2lbJMYYb%owDte$TgJw5BfBuMC>$oym z!S&ZMGC9N)8EN^${r57y(>CV7eaQg7cW;xQ$mH^Z>El6AY?HiM=NzT4afD|H^8{1J z;4XJ%(BZg26!2OPQE7L;MMU8z57+TDe~CRO^f6Ag1#TLzc=vI=hgBgLG$B0<`*rS_ zVUyv)9Sim*nLe%+DGc;PRL?N{gXSW*PGB3P9MK&(jFb?USpZp!zYM%D+ljP}A7$BVmp2m!OvO4L8x^#+tt z8WgyGpI;@!g%e=Ef9Ni7?p#ni$#T}oJ_ZO-(Zy$&Cts%cfSS}kGzAL4KxWwHWHMgI zH@R8^NWfaLT}o%|fh<6^9v|Ea0Yn5Ez}@Kn5^g*1zn*vnB9JMPT?GcLs3UJd4=JD% z&i#^){^%00T_TWgdk?nsoTJS0p63vv+?yt(AA;pG*357XZgzjuhpS~ z+;cp>)7fiz2L1JJpMH~bv!*m2eA(xi+%Ut-FMgdvhhd{PlutDCcxMboxh5vk!a$-Q zSj^xcIPDJ5*vX=@s5n%fQURKmIA!I`GzOwYfdBrUZ-IAx{#`yP1cAE7kXD+_s5ZrCrFn!Jz5Wo%) zgA=d>g~l(i@T<6WgPO4FupN~>4sG2OE@&|-nHMULc{5PBtNj+))Mt0ol|1LAyJ@!6 zEHF>b*9rnIJ_t4q8W;`?x9j%2@}~6HFfx)re%)Y;ARopiu+ditPIDx4oQ0x{Qw#b< zFpmnx^xiL@f|`~Brry7^WhkL>hYntrmhfs3+i$(1_j1z*udb8Qo38!Z`dF-fjjlLA zJN1S%*9-clW9d;Ht)rj6$oc_P{O5PSulGZtrJ5&PY25}MBX)n(08u{L%|~1(NWjcA z@&IRkWSv0M)=B?OJas*lE@UEH`#IPkXg3jz`w5}_n8cPBzL6QllwRhvX`pm-u*T}b zUCT>Pa$wyy0eq{nb!!1qu)$mXYQe&|f#xqBDF~O5JP6~*PcQPGKw4*Qr;~dH*0~io z#wi=1T~VicQyHCJpoQKEaJB^g+Dip@zAYWJ5x_DJ_=sRh6EI#A*^HpeI3vIa0ZK3a z6nv^g1)g+*Pfu@~IY>)Y1B0}V=$A0fM?jeaiQEgoQ`p*zuDv`%t=C?knIWM;iv@v@ zF8gu?c({5#+Swh|`>x(ypP-`6*VWHROA2^O)9SqB?AJ@IPTdVf9hpJGtkFSNU;j}T z_;FAus5xW)&nPoyZp5 z^yHRy6Hj9UVt6#2xet6pT?&;^X?j%Hc%!+@;i{1dZ7MG`G)EZN)kOH2a4}CiGFX4|a zKA3elUwL!KPHzQ3>!IVKK-E0jr~k4G&JCP5-GtWc!Wwug!=w(>16oNN8|D*^P63Pz zD|II6u~Q*|7IT>aMDojJjd9j~4JA%RG<$lqpO=Lqs55rxI+RT=#U0WVj2b}|L_fO) zmCyufRjMo#;m-?`_$!~L8)le?u2fn3nrcZfbJ;xKoqKlh@veX5dE z>vM`%wO^xOPuDNv1Ub@LAMD_o3?RVoU6#Jd9Fwon98L9+QSe4@8dBg3F~XW7h?<$0 z>FieLL)Vl(!U`7)qAwOF_3lV$+IJcPpwz^Hb?aj->h6wrPol5$=zywnoeWJCI_fL) zdp0ZD`Ns8|MWOF_$cwkSaV22cw6=8Ecdzgehrzp)%b=;0;^sg4CfK!mLgIWkz0233 zhk)EcISldie3lVF70fRB8`uO@f-*p?we&#*NbLw3qhJTbSyVI_pjv|ZCwN*x*Qc+F zQbCV%+KNW7e9%_PIgWLL06wAoKyk57;F%yDyn?Q8l^#Ak03I%xOvk*ZsJjrnHrAA% zOrg1hH{|%%#EpSHgZ6WL({hW+!Y;wl=we3AX70!|#Ig>^JNZ?KB#L zTe1cL>LD;v(m$)>;^w-w&fEf2_Quo&NSoGR#HQAOU4LEGMy|<|Xg^R6Ni&D5f(sS6 z^s-3s)3?s)I+U#%8#RmczRv(%^j-pYmLB0nwTD~K2ctu}q-*!45gG|dys2B7+$Rv^ zp^vgEGN!_7c-nT53R7rC*8uRD?ZR3{20dH^((!@zqP8%TrTzY+_EgcwpoDL3fR{)Y zmk@BO8_sOvH=M` zcpw!@L%&KX=tjDaPjhQqdb6&A&+E6I0DJouoB#3c2e^$G&@w1zR>e1T;NzG001XB; zaK4!ln$Hass6q*r*U5fsEs zS7w}l@7KH{4fGx>iC~nGSH6|v+FY>y#jVk$<7c1Z_5`KeCHKr~yshG37kZl^;L7VM zQ0T40Xix=&nLj^VV)7u`ENm;+@Bi>y{M;4JssdwrnNL?i#(1wP!JYYQlPR=lVPaOi z`*KJ!R7PO%BA6==m<`(Idm_+FQnj1d82vI?|3wm2v~ zjNl!II%%wnpqH=x+|5~9;M@RR;1o$;15@0|vgqKfh++FZdYg+**MsGN?0H@mMeV0T zc33M+W<~>WIP)57vM1*Olo_CH9UlPCX8hBgeGE-tr$S`JfBy6h(f9z`7*dyqQlQ#wyDoD}1?OEU{TV4dpNq-6lc#Q*r= zYc%_s^w3D^5tkn>UHN4TDL(0ptut!pyv+b=V3z4%Sk%!|ei_QFG$p{$P8mpI-U@sm zYTmo|h7u8gKAX)84~?pO=~v#r1=6wRP`H`F*hv7jf-FJ#pli2-MQiU`oBKSDj_CrZ zlb}uM%B+m%jRN``;0;`sYiYl*1qsaGu)(AYKh@69`G9DUiakeDRLPjmJ&I;Od!_}= z1yv*U5h%VO#+iW6!*Ng}h&~4ThQ|N*13&q-sbc`@%zW+JZ!(`uhS6VPS#~x+<)Fjp zLxX$WXJK$~)rYw6ZoPF_nG0V1)7M`&G743VN?X zydZgbeji-`EL({TS%u6IJy!%5DmePtQ?yrWCfLN=uT||-<|IO8Sdw^YDW~8`09wj| zQ}=+rd>$;p2`p@3`ucOU1}( zg=tx)QiK5m2#9NMY&7%cCV^>gh7Ye#Z;xj%dNW@E&I$}WIBlryqE#)Oex8twVs>%M z9iaz5Ib!aDW@f02xlZsx0gXw|dl;n6hfAEOI=2Ft2ZQ9LSnDML4 z@7xS>rm0w`Opg;ZY(Sh&poQ8VbQYQmcY|^LS`ff06Sp}^ZOi%i`T z34ZD%UBi%A!*1Z1}9K!4ffSHp3}aF5*)@rie`Ep#;G?dhNsp2j-n$yq%q#GPkdNA3}SbVEPm(Lotnby7`ix{J|tBz#uPR^a+w$Tf>e)?Ubgo2*y|A zvT`Z(_n!u90J%)@smo3DPlFwXx89(A9JyRmY74ksAXy^V(6rlyuR{VY+Rx) zW)|u2_F+mKH+nBas58#%qoOD~OcZeifF{zReQ}s43)22{0FOsc_)W?cvwd6|pqUm` z`VFrR`zMrKxr?6N-^-H=4s*K2WK#VbgRQ&h@6#_#Y!sEr<=KdRX)p1Irkkf))GL~4{n}$yWGt^=U#Xa99Xq9mfQ2Fi_*UMGy|3yA*fKbq8Ux4 zrs;#~&}h@H`_ixdy(C^!$?JDnkbt+J1=}75<2h(1picV~i9%=U|Z zp07RtmIMOa8hGqICT2Zo6m%GB)(-T2_>v6x^+FoYzM#yRanwV*XWPcoc4;T{K#3DH8sc7wPGlO|*htkr?dVMohI>j)@EYV5$XM+%4i zW?Z2O%6LtF%!0JkUJvSQQN6L$(_u@-c%i_H!7{t<20q6$u?EpSDg+1E2Q1Xu(36Zm zL0igL|Guca7ia}Ji^?(y)&2C7 z($TV{53qUwOWcPIol9c|fO0>4o^frnS)gu_2B2p zqetr|X?HF)rd1#v zWRS6Uzg-#F?ah1e{l6Iq7MB805Gb!w#hOaIWb_4q*7Ep_iCgPCPO~=FE92&)u?y;+ z00T-y2(miY_MfB~$2%amuIk`7vwH{=@6-Z88EHL#Kngv6BIsEa33~S}lbqpV{HLwT~`a4P-6btIpeopg$Nz%G|WsVU#*)KNY2SSwc%v z*Y3^#`CBJAhuWm}E177zgBWyz{o)|Me!A6lHuMIs>sR~@g;|-U_QYITas<=<=i$r) zdYYDr3*h4eeMRl1o>P!HD&J7An6cbrDhNo~;Q0kk(`BBWQ2BgQIW7Zhts>9g1DU?Y z8GAt5_s&=`o7K`Gmb5oOBZeGU2Kr{8^yfjSnHm4lu|+gFusB*N%GF!d(+bU8(3RJu ze-J_y2L>7ABIp<$Dq!YbAujcRq!BRv^KaG1{+Yw zkMD$RWs&QWKH)t+ddmtL9kIoduqxe51t5jNXTsBOaOH(`KFn!v+Q$^I?6e0P10@cF z0m=$~P_<83V&)u1>cJ<|1?{c0)|nm}Ex73XOJKC8e(+SYT`)*MuJL~ z84hQ^4|dd!+s$itp;eo!%BnNWaw0R!&;W{?!3?0?gFk_rLle0~?QORk$7`VKD1heB z&0RD612(``JHWP7yjG~JK$mIjXaMu9id*=M6vkq)^j)P_7<&T7%*S`Qax2`qkM#D7 z01tWhcJOf@?W+TJjLNT`eRGe6iBGuJYxjc#$FA^kndZ+U$5tEzcWIreeD5R_!P~c~ z3e6$Cm%-qmeLO3>esQlpP!5f8(O3b7pJk5HN3^d&_%HtP$3GZH-@f!GTz)iuyWJKI z#>~9-HIc0DH8&X*ZgOM-I$BLO)_%nnU74W7uwK?L3y=<)zTA@&XSOG{PI-O!bEw+K z0hY)$))4BQMa_EkR#)`77-(Gq>w=(G1m8UXdW8qQ_=|BCU?!`vrW#zwE;N7b=cS;0 z@f|SQBo?IoxTW$Knm|2hJqhg6BNtzmE|m@iT*PsiYp{^$2w`-Nf+`r-d=nDzav-Te znB96)Q9uIZmJv?FVVsJsfDPq-awmY2dlGL5s{0))0m<;Bv&u58{L8 zxQ%z7SD}3HmtHTO)c)jeE5U2{Vwasd_Lzw@l?p}=s0IFD-1Xf2*{7I(CXb`hl*Jr9 zsFRm&RUx?b$xG-K^l4uRg03PtZGao#Du6e=tSluxo)8R~D1H=1DHl^Q*jRLip}{RQ zRr8iIS^}_Q__{W+?!NE!cH{t8THQHb8VK8Km(&En>nWr$8EcT1j>}P-rj}u4q#ob59&%+ z&(4+S&@u-x)d1Fyxe3Q4%DzpogZzb>Tm@Ej6pZUOs5PJhrs{yk9;hW|s}&-7amOi| zZs7iPFere%Pk(fz0J8YXt`LeHg@Ol(%ZUX&^4MsIVcsX`;v@Mth2gw zO8deCBEc8ehSF)lK+wfoz<6Gu3FO)WR=~j45#7qes%cX_j`93J>f3jMPk~rrdOsd~wd=b9G1~X!ftH1k% ziUgkM^kZOxL3)E#P!Pl|0n?Xm6NQz(RUL^Ax(R`3muBmo6T=L!Bv2NpU+=H!4SMSp z709aw8u{AOVEV6|E3>1EMqlCYl?*{~qfdCuIq%%_--Q%Zs$_PTX81m99}Q>CoUvL~ zCza9aVMxyU`R&@@c7%4cT86++Jzc<1zjUAU(4#y-sP+q#(x-EVEc0QbnlU#6;`XjG zd;o6|;5$5j60c@)ssSnc@#Ym4I~{Jy)tsykG@$(wKNKHkzF<`4VEJ$%I_OZI=HQd% zq4EGeAmBfKbdTq(IqkCN&V>P=T|ZI4b7rH!&;J^1uUmS#Dll0~d(+u1AQkFuPywC- z82R$xSLUQG{0|R5cJ&tK(Ps6e^i-ce$+QM0OGTQB0@@4=;LA6ZQpmLa-8Wwc@6BO; z03W}hthuqy!z%-Lp1vK8=#u4lxk~3n(3mCzpf1qaUxFp9gwpc>Q)U^|xGimP1`M#- z2-Z%A>!^g51Ih!n#_#}!W&8Jjag@ol1LDzC_~a_v`JxL55uuKS1|w3(W`1*(_GzKg zv)cjb0OHvgulbVVP;8)>P$hb17Xr5Bv zs12_zr*Uq%uFD@BNO=1paP8YF(!otn(D2&RdLwKbs)~q8#=Ww4Im=ao`h#Oo=|mue z+u?@&*LcCf-)Nv=^DT0HARvzXn$XxcEi4Tx!7 z6wps!G353@+QXR>Ss%D9C8~+B>B?dQeE5(^aPQPFZtESFCL^U^RP9e{XMU)C2%PkQ zHU?-~LFSd8={n=g{J>dj7ESs6jK2)f4rsi09bEep=#Z(KHqI>E z0Jn4YzCk+(Ius~%5~&t$3|^2n!*6h$mIJIjMS{;K(7;1o7L&DTaWNgLIF6!&p>7i= zBMb<@Hg3$1rDgccYwFC>&nZ+rDgN@6L$7S<%9G;H|ArrDUhpWHB@q(3f14*;BPecm z8_ULj4@_RqY!z7lpA*5%InH0x`VagCL|m7t{o%gjVQs1+sd)HrEPxy2k1_^Bi>?1# zM^@Irb%4c~_T8Z-<}vWkHGsWL`VY7r7iNPF{?_IV7W%(C3dDpVUp@I#(S*DHjgU4C z;o4X}HFT^Q_`KG^cgcUQ$1x$Z$%2n2KE?$6gC~CvlX9sRD?@Fee>>V`wEmzxOyS;| z#v6i$Pl>+-u5&?&U?t^KS@6%bK>DF_g85?&fB)!wtEoVFfUvHJ2>l209ap9w{9R8z zlYqZ++P^rzZ%A-yhxR+6TuScy=ZXL4c(JJP|LYy^9d|^x$GooTN?qFf-<_4#8U$Z( zfDRGD=hgrB&{A)W+6Pmisa}+!*{e4{xq~l8d=1H$W*US6Q~`?MtIsZoldkhk8HgLt z`}1?Rhz_oD0a^%ynOAB*sFDt)y~2v`L;#;^S6&qzT)hGDHSm8wa^lTy^(HIhtgzvH zoddWV)TERE5RuIs} zHsC0OXn^GEvEgcA?eb2AO{6>RS)l_Wq}Uvm7&q;e)<`R+F@FgugPv8fdBE( zQ2>pWJroOQ<AJKoVW0 zL;E9ZbRZDVY4}>t2&(}B2GE!Hg57Wz+Uq>1{Kd^(yKG(=k%sUI?K>?^%?1DdIL{b)cq~9~@SOdiUA<@C z$S4EWFMNMsOYdd}=#yvn81t$Haa{WtXz42NWyG1LVqDRjdrmTb6Q5;zcG48UC(nX0 z_<#snVWR?!uPAKnXrIeb+eo7Tzy0;A;A`*QIVf%BD8Py(*oENgfcA$jXflSK)(Ofz8sE6=GGTGR;?pXy0YUo&<9D_fD^rRQ!>Ex{{MGIX21eZ9X|*J|kgMfuyK zFGg0ParR_8|2t#9=v>1IQrGcl>ziRP(+jXiVohmm#XW9&@#$-^`P{5<;IE< z>8b)3O-^e7H6?=aF~klFT28}78(u+5TSF{sx)cwv;cSi_BX)_P1p9gGu7z^`l>C#s)!-53(5sW*YWJ` zjK04_61rO2bJvVhlQL*6!b$r=`EiCT&_sU-cmpt7%euhjfKCxs5P6~s4?5h)-9zis z+se!!JV0k(04vR8WPNzgIQTATItuJRe)7Pa@=XAmrvLn-Var8;#97~9^!j&;l`8x3__AE+sy zV<7wPU2yGtagZvWF5b(hD{s%Ow>W*Gs12Gbmwd4KUplGk;4!1U3Io-F(W(Y8hE0n( z7XSuCfG3xLd4YQIciD#_558M5w&dnthj!{Z7?8I}3|gP5 zA6(&ViV{^7(LI?Hu1W-N;`{*b8F6Qpr!4f-wZD7$Y46!`M)$jqfZYVAd34l`(lH@% zvC4@fXpN-XfnR<#5PZD8As<3VJG5PCMQguE&gyN$E#jy9XpYSDceuznxbZD5RVq`x zxm&IW)b?@&6c6eG`7c$03;e?&t;4Ks5=si8ajLUV56T!g&oWipSr}=h4Cu`ll|w%@ zgx2K~oP#8gKQ71O+OEPx`fJm9*z(crI7ZN=pGa@12yl-2GNsh=y^Jmu`8)zGTmt&5 zU{FPXc)f}}>!!#wTI(j{N5y}{zH1MFdDAXA-e4*inxMAKX@AwCxzkvZPCGKZ##wq> zt~~_~#BYOTZg5aw2@%#0W^+WvFUtlkDo|QIt)zXaJ*=YSxHB4dryjLAf17@tDB3qt zI6;d=WR^MI7;2&3$K?W(gQQ(Jv--uWP*cnM3d6^wPdKX|P)2v~MXY1e%e`PDwz51? z25eYunpR3ecyMm0XQg60OZ&+?G`P-4i78sPR82y6mo=DxQ%&kED1ISD){J89&!5Y{ zQjLCrdWNIDxly3^LYb3Bbos`yR%TUwE03=Ic*R_EWIrDSK)J8TQzu3GL3kf)rXvCE z`(uzL%=A!D3nR+1rDGpr^A8U_1EIsajQ)r!+68wzMmH~l)EfswO$+ppfB)B89AmXl zeca}Jxk%r@QCwZw_`y*&jm2IKj1l|JG3b8trM+KvTkr& z6YT{g%unTv^fSZ(H_*G>Bf!0}-ISVd3VpO|n04;m+~PUU>u0j#;p0o(BTSg{Ok)eyc^JdiL?t!8i+msw)(-2Q<1h^Qcivp zo)Kt`aiO+A=?9lz2dPYfnx}Wp`qkJ}98FpGT1Hp9!AbjjP&8diW*s=qTyCBP4)FOS z;F;A1l*TQw80i=acZMDU3N2Ova_Q$h^@e9Y{RMO;`?^)^a6i`CUz*_A|NP5G)RU}G zE&^v^6eJBm_J}^51j(UzUdfPRz*Mtv9sGw;#RPgbGrSL-;=P*}*P7zir#L=pI`{C3SS#=!t{hkAkZ1;>H*cEio7FHb4TQgWP_61>N6Q+QO&g2ne7j7=QxSyuLoL zKWca#HSLIcj?vrC&OQtFT$?x$;D8 z$!}h`3%$0W55C9E9na}fNt%+6CIe5$+)`j88Z<$D1x=kLL;E6vxa|hf5A*-;`wu>e zcDnSjn9-7^dhUVNThCd8D<44v)a2@HYdpfh!Af?!7ayd5ySFylwa-3*?rfRR#L(6L z;5+>?3%aBN5^4>d1}uqKaV_uu?7Pr;#0G}Nc608glMUSpu&AwQKOaMLtA|I{jtri8 z(cP(4#YhNE0x$?_&OZgtI!vPjF(TCMX!O!$HPYMf-o*L>lzs4^C~E$WICs~Tu~zh1 zWm@jo)ib=EE4Y!Fnr?8lG>b*NeTt`=Py1wDbODnh-o9HTc#V}4c&YZKv5eykE`ce$ z(WYqv)+P-QvBPWNv;si0I*95bf|V~iA8Z|@0vtaVGiNAeU|K8RLFJ}bK!V~hw%fVk z&8*2_2rn5(U#hcaF7RYcWYO&o-Gz`J-U9|OJgh|ZRJ&@biQd`B9EeVv{P?Fb;=Zd4 zSlFDrF~X%dC!luS4|S^(j4K1&Ya?sncKIG1LyK<=m$}-$M?W_0|L3dEK-MKUF5A<~ zQOccAx2t@>lLK2>Ex0HHu7-V+Y70>5_6QJd1lWTPeHGW9k4h4E0oZ{8!n9wffyQ8+ z1I2TJ6k9OPvelisbHp2p4X+jqLHMg|;s$igs*Gd4Gqu}6)u-p-B~UW7IW}R zKWGSea}aZb;S7{A9+bDBF&xI;fA#F+kUQERsL5&TGA*CGw_u^pHHa%pn!y~$(pe8n zKz{>~82ewp`u;0l{s4-pn#7{T^2aEy1yzAgKLBO`0d$>U$%h9X<^JcTha2I20B;)= zRp))s-aq-tvjvb<+u3cK0rYdRJ}ycw#KHeyRDs`TIZ~MpEyT{&m4Vj7z@zUOvuZKr zA9N&x=)t*zA4L7&CA4(V;y`H4s9me^vzH=jmeK0_@M-Yof`U(f6)<1x{@b5EeDJlB z4_?0EG=(#TQSmuW+8>rNuh^`vli6s8T~W;$2|i3{K>IybZm7C!ujxo{-vHFCZtc5{ zj6d2rz`oSm5i*=^1iW}KCaAZ)ANYj!Io^5DSjATw62KclzkTlO;K0P6ethsI(!x5x zZJc7d(wM;JAAD%X*@5YxnS1q?C|R=~-5TLga_NLL>G`*` z*K?YjN8qLfd7HYbD}}TG@Rs&aPKCCmkG=gfsB{k_i`Bw@l)h*BCCeuWLlbOzH!8OT zp(A(I)|P<}Ob_mh!fyNxP-)~P_U=OaHOv^%7*RC=X9c59n;#f@yikUz64G0#MLPk<{_7QNv z62!?^3Ct>j{ULyNZu2QyFDjG<_@?R)@?a%ZDWvL(eo>uh@pL?3Ji}NAw9;|@gnf4m zD_NvkxJW(*uhc!476!5`KYs{`{ z%)wQV2c5|b2N*{VAZ2==T@|DAN!xNk7tLGDm}zR09^sC8BE2A#dp`R$^ngV$u3un3 z?HvJ0BLmm&uLWmrpL%T!Jgu~YR{}wsZ#^KI-$=@}JA=3br=z^JB%D#tfMO^u9r#cEdX zy;n=JdhczkciHN_+mdBjy==)RKp>$fB(x+1k~owAp_9OYF$^}w7z`LNwuyama?U07 zmUI8W_dN5RT3e8ExZ~rF@r`esF`l=~xz?I%&bO_#rM=hE&P&V~^H@fQK&LC`#-#;0 zgZXo8pf@fBoYMm6iy~3M>2dCQo?(=JWdA-0+VX%mSfTby{A59GP6avybf80fu1<0^ zn6n0SI!T*}a_SW@9U{>D@cKfo#g}QKeuK!OG|9{IL6Ac%t41!(r7c3^0C}9|*>A;3 z%i#1s(51Zrl@YWlz2I*>X31Wo(;Zq=;(U1|WWr9k8e7Md?NZy1qlXbYrr!Li9)%+N znCS3BfpJ52;+XY2SLZE--`|vGjIgCI>H z4rmM8Ti{d)We>;z9TFq0adpGg&>*b~wx&HAYkNrF_Ae8&*+v? z$W%)Q0Zw5@f-EB(MNe}g^NPtGzRG9^tX-x}oKhKsaTb&d;+Ra$DK-QQaGgb$ADD4! zDmnF-W6IJTO_(>t5dki5qO=Tr1{4C~`gs&j?vkzENvvBCbK6G(mi(ar@hP16cobMD zSF^?{A_&V7$$FL)NS^@Z(_pDI#m=wpIA=PL;{b2orQQ^u#IkecRc{^^4Prv=b55#< z81rsaZ@+*F#3b>w`mCIkyYx>mS^!#}rJ$WrPxOT16e3xn1w z-3EX6ju6r3AV>g1bRs)y6+e20#-$3n3{8}tPks(%yiZz;{WB*vf*D}yF}aY8XFz?5 zy#AnDH}vHXp9iNl-xC&Qy*0*=E2NJ)>3a2$C|4)~fAtU$=*t1)d!>tUQ348l{twvPE;Vhh*dqNn8C^ zXPFsYBOTxYqhO4sNzfXogZ`vn=hzY~LogYdf`Y6Wa zKe+4Qa%$+*YG_97arPu84w-wFT)GGH&@w=?0ss2rZ-aYqrGjhEe|GO$M@Ox+(jayS zDl;x0q%SSd7@hz#$qedw^8~m%!_2*>wZ%3qV4^OSF*P}6XTN_^`!{dt=;asDX`qTO z0@Sq=atq`lU9^Cpb#}>LwO1IrDDg*B(CiB11Ka zGh(a?4(ny1RS63Bv;nUI^{DDl2<@-YQlqVpOkY9xiZC6xi=xeaRmJ zP~U8dObQSE^Nl}{o>&oF0uYip{l+8VnbH7X-~zls+eX)y6hE{JSqVIJupGSlSl~iP z)_|?Cqqcb?I!fhn(#9qznXcIb(oWNLf}tHzyn7$fBO!**-+bwQ$ocoaxAQ*WIrVEL zljCZII};-3+RNZPJOl(3;N|M9g{H`BkEwUeI(4CS;aRYVeL4mH@Y3^W{`RHYz?n|^ z0+hG_oi&lZ2I4>^ra5VVtLIjGXmdHFdwZa-wSTLUClJKi>RGT_l&CL>=g)fEs6PW4 zJ2`jz&ociGz5O8gd>_~rBLP?!0d6PS1jK58y3y@SF9X&h+!@h;xsX=p>KKNs9rW-p zND%d#$N$GSZxTJ&=w;f(eDpJL9-&^f#V}ptiZd9Iz<=EE%Wu#uS)E{xAZ;sXmf_J5 zEClv#h<5oK`wq}92^nK+5Rp8C+0)JeqJ%-)-lo1Y#aKAQ8Tf)+IiV&HkD!HUV4vpn zaxU!sZzg~D>H$ZP8*BCne|qdr>Ew{W z$k}ICoKxn#N1-z?1o}XlG!<0C>CqGdu0}x9ICs;ZC4-D=;Eh+nHjZ4lRhraoMkI|Cf(Jj3*aYH*VWj!>DE*3+8S?R#@hp{%$4rvRY-5RkEO7twN zay^KaA1LDrfb@R#BU;+k@59$<6NYAXqvOIF7e2rhwU*b;iY%o*1V$~NESd#Tjr^YyO zfljdF&gvZ<4AAabC-ts!dim5NU^L%b_Xb$KyK`kj8;EDrM3z@G-&35x)@UhKBtZiN zsz=W6Z~MNZi#-}lm?+)^3RZXi_P&-Assqs2#&rSQnTcU82jc<&^)MDlb^*2T-g5dG z$kM%&(!t;Za#q0tKs8aUQw$z|E3m1^2LJ0f)&AzyE^Voe@ep)#3y5psT!Ng+OwkMz z`byy3y~*5EKNf{#tTK53yQp4u{Vu4JHn_#UoPN1~S{zMJiv;D56iu#;M{-g_Z$7uZ zGFws~2HkVCli35TuULx$?mTgnwZqfs=xTPtA1dfH59!lE)5H*)VP9|}(;nc|_Xj0Q z<9CmLn|p8t{n8^maa-g(L^DVSR>}eBg`jp2;Mm3e5|Pav+K?e@SFUTOsquniR#3hQ9U)Fn}NX80=P(^QhjL$ z%0P>BfU5_lNoz~btJ;x<0p+T>(k5}sh-3BDsli93;Bu@te`cFRx z79u-DeQ?+57YCrbfXd1mbL#{3(Yu~FmBLxHE?v8;YLx{@u+Pv^X`(wiB$V<4c=+aud=po4=IriF4pAe}o*bb!UB&_TKm#6#74*rh=qb_N?C*=%Ef zFsq+|dfNTeed;d;&|(VJV>jlHm9C=YUu^6L#jadimbn;2bLN4r-v$P(1(`+p=o!dQ zeh9V=;xIg|V3j6MGchf+@u4N`%?^d&qyw0Aku3H=qG|sKBj~rcy<+CV7C=#Uv`)M& z7-=`=)1}27N_+Q8!x4hhHF<(~7? z7r9&-bDRsXxOP=< z(3v`Q{ZU6+g4vKi2e!4uLg(Cl{%(kiH-cG#KB59}R~>17J6DP-9&4aVv?%&NY0wP* z@pJNEGtL^&%gfeoW;xnYH{?CG>uF=dZ$7)0M-$p*jGex_r+9QRK2;xxPKFUctC_|Q zx-fJ6$Qo^%yC5oB&in)dP(QK1{Hia0az2kHV?@Mo)P^h z#ON$o>mEjCYhT*iCn0H3#)!L?UK)=fcv^a9wp`UKVKU%>e^%GP=j z;Q5iXnxGt_MAQ#%N%ktqo89^FT&um}x0HkJOWYBdnmFp|Eh?Qqs)Kqz^diuCy%^bJ zd^{Wq*Ti610P)a)Ua%6&bkO3(X4)U@@~vRZp8|%Sfe`hnw^zNEiKhY@tXjeQK(6sc z?pGdZpB*^;crcn~kYHZ#n#%zE@TXw#`FTcG46D^g55zN-mIJ|P03SP!MElcOo$LWcg47pj z)hq0|1-Sk)n6_^}kEg+pwRRq+efBRk=IE7fyXK<@2~R5v|zwZ)I@{QV;%? z9S*G)an`^Wto!~?y=a&sj*puc*`t~+RnPLoco`;#Ff)Noe5PQZ&S2yZV=jXvHEPkvi?&?9$Ms`i zxd!*^cY?ctEU|$GkUek&yighjZ*PWbHkzVvse+n8?UTU9L8H@wp$BwUj)Zq{wsho| zz+5}lk-IVK9o67Z?*e%&2wgM|4>FCZN0UJ-WmE&n=NW6j{)#a(_XTY1>{1|T>DAU_ z3!H5ZuSbPErt!D8J`15^9$MKx#z;VVYZM(1Fr;-vrb*Av=fjQh92A?vl)yRk_c}O* z1}-JY9^m^#bJGr1cf&ddI}SR#PGIv$O^)ouwS4lkRB%rQAJaGSc1ahwo>nJQ>ErJY)j+f|POJvijG~>O(>EWz$~^-? z$3e6dlLg?_q)8`YlO6M9)kd~}kJ0Vtehjt(DgpsPl$?#N#s9KJ`1xlxqkMgR8HzN$ zV4Rg>$#&;Ef32KNWJFtm<5Kvz9+#5{F0TbYMt26~&J0<~5sTN_mG6M(`-9PQQJ`|} zDF}nGjo$|Uk9%Lg2ZD#t=OKsa2?3V@`15)I+n`9Qk}n_MnaXCJgmHW@MfH6h@@G0-w&EM7gM+Z~#E zsw&Gdpw9#C@bY2lN)~ZAJ7wbV0nxo6qUag}%XNlYTM3o(nevoAJsqwx3Dm3)+#F>5 z^0=K#dPP#wfpuDwb+=O|trc`2nyWa*N=+{T>Yx@VdH7{HgyU)yPt*ufA5lK}Rpzd@ z-=dc}*Eo`!12LBwt)5E1nwGxeigqk$)|b&Pppo{1I`bE#!*?tM3f*(+UND_z%q9Ut z|C5)%bfL>(gb0{qg2i0ck2IaNP&xSO-lUKwlYMx6e5y7sI^!)Wmw(J;?;FQ8)|P_V zfdF3j`g63A;?Rfy&9_zIxw}w~f^=w@s-SZi+fn+BMsfsoq}tIxAG{8h&B1};Zjn5V zuiF&(%3EAVDi0QF~H5hVb^qppH0t;y8kQZJjpjZYVQ zOXxNOM+}Ww+)v^_baE(YCK?P#$Ohvyf$Di(F);qw&Cuy8TDElaxHrV+#z#Ln#6*AZ!7^~c@l@NX-WGZs*dyYtS1pO_ z(_(5gs?uvFXN$uUm4rh47sPTczi7O<;n`sS+rRqU1Mzz-<6M-y76X z2T`W4Qq!`9b^;2+XbF}$VA%DzC>0`7MbvQf`J1bYr0Gxh=)t5XiSd2uPOu0w7|*5; zg%$%kFk(uAU`@>hMbMTs7E^(PH#2SlUr?*42B$MVzCoG+X^{IxZ(^{q4bX#AheiMB zHAvbsX;C`j+#y3i6}h!~+BQ|uoAFaNYfntIxYwdB6E=gyB1xww_~_fJBM_kszWMC4^M^%$j$M_1__JNu1&(0t^v zxzLt-Fj2VE|i7veQ-R5HT zLb*&G?CKvi9@ysN940%qp1n~N@P5$mUU-`0Fh`PT6$Ab}9Vga6cO-IZ4bd%}C1CO9 zg~#&Kq;Kj7kFNzagIbP*mE^NVn?bCfQ)AADIG}-OdW{qVVksEy0^^yEHHE7uosYL} zI;fY6Z;hfiHYwlk0OXKEhy3z9Q2mM`zt9?;0L$|`-va9Z;s4k>U&KI*t{AKwwA3H8 zf%A7Atfpbw`=uQV5zo{C0umXZz(@Cc4W%b|tV?UOYRIH(`wG*V?gyqXp7go)Qz+>= z8KE8v*|{gP(=$wM3O39BG-NR-_^t1{%yEp~qY3}*{$Fl>dW-r_md6`EK%aW6=eu_d zSK5WBx1iAo^lWGd%?vcMsC@9NVQ>z3f<=fo;Fqi?6Cb{T$Pd=}HdRE1UN-C4d3-R?dT+%vpe0vDwVU z=prKBj3JhSau5U4ro0{;$PfXZHX~}=xb~bh>06hNAL7!+8Mr(_Lmwk3Bw{RpE&$q2 zKRTifm@GZu+FH^MitQ663BnSSu|v$90?_Q%K*>&C$)3WsX+6-gn&W<;WK$MBPy}M> z;S`xmbQr+Ib`=%ey9ca#TiPaw1kfZ`0@#Z1KYcsHp$XJ4Jq$LF-t)Kes-d$$k3P0& z>ySL#>brShN_OSCK6~kdTgG_^PnMnQ%HX3Xx158S{w(0NCgv)~6R>imiOO9T+2$fg zN3PxsVPhW(&f#XbVxUL_xVB`>lF`E~0ZcR|J*B7CK{I%if&tnjc>&zM2<8u>)2u)~ z=3bK<zl&xUXuw7)(^rD9y=g-YGycd`uu$ibLM3M6VG}yb{Jv;#K;QpI(r`k zEn=@Fiv-Km!A_2N9JCuWAkZA%(c%S3tLgbxaCf$|!xox7Az9D_o6Ye~MgvMS6SCdt z(T?>)$EGUSJFo8zJpCB@@OS6HScb7{zq@PNL9Ugne_#EA4GOT}$o%{LBe!Nl9Fabn zF9LLqW>1r+M*d@t{WxzP)C#I=_*;*W{L+IJv0nn8Dp}R`hg(iCl!32FWY+22HO=o& z`)W^;sr@lt$j*v7*p}broOj^6_78|G#t@`Tx4x zZPY)!5GexQlJVXK$N zo(tgz(RqFUrM{nM>`@Y+hfPP<&ZZFPyj=u%d#yhgPUugJ96r27HiPpc$7bFGU3V1L7-mk&%GPqP5u65@@JWwt7kRdCd z-CcwpU=-9=&pFOY$6s^aCx`Om4WL&b4c;Iwf_kQ<4Q`V*CVF+rxu=X?C_%i1kB$?`qxkU$=o10XkFH$8zkP@Irv|ok)t8Px zKJlC=C0vl80HQjSuBq7xnRidpEnmXr@An?<_Lt7uor(X;<4>UH+#Py43=N=P#8DCa z6l8$w8dJZRe&<=~9DtY|oa?L&llCR`4KUHSSjTnu>jQo2eJ^P051+Du#{-ihHn@f) z(RcG8G;lu;z5rs3OqCo5_f&sf`MZ9nalGDp#sy;I=rVY}ULsUr1Vp17(DB8Op1*tP zb?{jp#Ml^LWN;p3MF;d}3LWMH8~nHdWH9~#nk+$7;m06c__^0WQA#hfo0k%^baG}( zL-Yi_^xlIse*zbf0aE`A%i4ejyhS@)cz~1WF4UhLo6z8=EntoV_6F#P9@K86-2g7x zk^%Htzii;$FHp>>{^{x0i>2AT-2+803tEbF?>u}PEr!yE_tI^vAoWex94LYSLxZxV zqYP>yh^@Jh7!L@+q24d}0-dosjXVO;pl>`s`bmlGe1Vz3rKvwj?;Grb8pT)|>t!_1 zu7ID2Q9H=-{8(w#TGR4P&0M0+gmsr%<%pXe;URV-pdKs9tHzuKf8;A z0F#r|!Rp2K+Lz0hpFwXZ$VqkpXP^Qctq!C^g%$y{M{=dLn-6eDy%)4SAcmjSpHA?S z;_6iP#~=Ud^B=te-FbKcIu{a%jts}-Nz^^(Qz5d#ka0=V*yx*84PIK0%JaG+tZY8+hA|p#6N^sDpOBl0UpMSF2J{E z3S37JM-z0?kUHj|F$M+tLS(en7l_Q-IqDsS&72N1TRr$NSp8ASjwLdYiv~D!D~N$4 zSW!PD=p>-EDTI+X3_3D8Bz=<>OG}5YeC^He-k~=gk`3UT2#{w-2j}B=F;(jw z(3mRwO{2-wzDP+H)$nOS?v4Z3`tddrv9F&*^g1N8JD1CkZ0wcEeBp$v)v&RjkH z)#wo=YFuX?W7lK9EyG4ISj++5WmOkgjKPf3S+!>~bk{EgTsJ=J7t z?GBNCZt@m2WTh{$T~y0T(O!w=7#-?SoSxl1jB&<<&TTg13%~@d=zyS8g_17LAVE4em!Hmb`O6gKYuCO2-9K2fQ;}($v4;pikN+ zw=+0Cd_jHfRCbuQ?uXK}EiG^k^f%!-8m%p7Yz~^t3SnCz0XXMCx4?@g?LU@kqjkVvf zLy#w;6vDPxbY`J|;{iNx;^qW+mUN75xu5-cc|00Cq#*@tA%>NniO!*p@Nxh+Uw;Ce zOX>hI@ZY=~M-QUC8{HQkrj5sw!R$Z|vayt_TI@EpUih(e2oHp? zBtUk76UQy)c?9%+?zv~d8EpZe&C@^JfT+^31yv9PVX{}R`fH&@4xfF*!`KKVIb?SW zdb*Ccne}>}g*$I9g6Zkhvx;)^jQxZH^{iNEK)wir_t06V9&M_D`0T+gPP8d@J)tQW z3P(Q0=?jA1d}y+N0eY5A5%{#R0SXtedi@^oOrx7dWZ0K5KRB_$rQoV|Nc#4O4%I>E zBe7xN*-R4>J#gH$3VaEq9&Zy01z^QUA6u(pAh}2vi&S!l4zSb-uGyPcqt#8Y!6dL| zPzVUfHGTfgJDQ+yMoD{laRKVqO2EfJbIF`Y>i`!CFs1-6O|)eZ3x7fs7%#4X`Yy>9 z6!iw9$YdQ++yj^tkpUo~ ziZ(<@Eu#eJ1Tl7PB2(+sGeD(V2New@;688Pm9_IK=Wb(zzNVm^QnszDJc=-8WgKcr}O-?^R4`1}=O=z)W z>ilz}%|m23a=Q+&Y9HusjAk3qJeS&NbM{BI3<&1PRFbpJ&QF0a(Ov;u6>#bVw~7v~ z^YWdCe6+?Km_7GP*#VD<=9R=e`^gIs)zT@Pp*+FKL`vJ^`mNd;_9d<8YIMQx${`1H zd%*q5TPLSSFFg};{pVbdz$SlCV*nV>3|#m=nEIh7BnQ`vubqJ&xTe0da!Gv$nPb<3zJztO z*DJu>tb#c%z-TF}D3%7D0<<420#7L)1m}bgfyS6@QJl!dXWqLD!oQLyX-=V?>^YcT zPsvf}86X2r!%*j?somL(Rxu|6a3KaBZ*RVtK3Bh2$PSKy3OMwOgLG(Y_w7_{^l^+- zF6=iqdql+YItXNe%4p9xX&am#7y|vz?>v12PiX}jBnJAZGdX$+JGbVK(}&8>vUmu{C8kre6Mom_2@9oReHtQ^uV zKx+;pXzi_FLIe3-7k+4TLy6@KTDn7`vh&{Jss#QY-OCuGy_WSF%we+ubhv;qKc)Rb zH#ma)TflljKwTM_x=3^K+)8ovMEY2xp4F@=kr^Y{_rdo>N!X@Y`}B!;&2W+p74JVL ziov;F13G(X2;i8A?2d5yIK~G|S7Kl!Fr z{rPvQ3#yJ6y^;j<08FlZMPT|Q=8jzEQ7Z^w0Og4azR%D^dF!S3?qbZvsK3%e`PbWD zpLXB9lRlR&k4#)o`bnUZp%g6_>1oOrseUrQmnR1n^nFeC92fDwe)RK)w$?v5Q3c&B zI|$cTK9{RcF*yye9Bm|wI;7FbbmzWK=Vfhkv}#7cRv9+B2CLVCOrxE;1qRUCnOv0h z{yyocr`+TO(zZtRB4kUv2=GKhOJn(=Nh?5%!Y!HsxOkt}#YZ^Gx#z*ooRD7OIyM5; z->TfuS1A3H5K(X5nk6l89m4ao(nF{Ecl+l1Lb|R>9}lH-ufEbL9R_FQ6mT}xJl~YF zvusJ~7b(2A2$J?4;F!YHM>99HX{ID5n zO9x}Ok^#IM+fFX)4F;=wF2SOV$5D13$%oC^zgz8^}ZzO+t~g8r;+P-sy|<0lX~R4 zB^A&!0}2AAspnsC5&}y$qOHpbwEj{0A(@LRuwztla@QUNyYMs^9S9_k59qw+)LNy} z;HiU4hE5MGxr5ajU2oqDRcB9v^j`brA6|b#)WxHPyf^`FtPrdvX(3>p`nu@KNdbv8 zas_7xoV!KmsWicL@>APXZ#sgep2#eF#EJ0u=mqAWdQnDY7>~)54J^Q=VYA%rzdil{ z_~kpfdD3xS8volh!*K#!ds$`25^%NZrf#bEINn6M(WqynIr%Y z>K8Sg4g+eOySMT~2d=%!3%nl)j{(FtiFji5SotDe8i5rMs{lYhe8H{K?f!+nc)C`= zrO$8IgPTEgP!NM2*zWRD;ldGH{j=!xcG=Ry-2jGeN<&meZcEX)+@QnrkT7KuSWzCl5mj?MWYpsH7d4bpXw3ATJZD4)*9i z==#()QN}K;jV;cfs(=3;3X^VT!)~;Nvlsz?+IdN`wx{%RhV;)Hyc*GD$)L>*+8tPA zq$JRW42RvwM#FD`-V2d#^>iSpZ#}-jUct(T&l!3|e?ra_ZEb+3dgmlq$o!}GHgH72 zM$$GE8i!GpCY@eGhx4w#COUXAZJ7(w5h)w;qtn-*F$lD;p&e)#+-iq>>m)6aoUv9J zn_)6THNaIKas;E(c+M}>m^=^vz3RVarZNtG;Zb~tnTi4RtNBSZ)srqDP1+a69-=Ua ze#3P8_;Kl=hk3P2D&(b+v*x&SAlA!3rUYaZrw5$7za{V8d)zH{svbZui*2#K-Kvot z9rum&Ldyej#SVpqR7sgq; zgQoj+6{X!uK>1{`pf0+41SH^k+3nfR%Fg5rr%wUSor_sL)YaTkv>=_FTI2xjclx%O z;_KgRzsQJiq0{%%V}d$}CA&8sKgL`-U@e{2Z+#0&i(gl$8zi%*AH>x+RDca}-!YIj zb6AZ&B^S)7QQw$ph63<>iy2_iaT&}0DNtB(OO|N8y`axdHqjojGriyjESXJ*#EtW~ zP}lq|2zrhd>H+;t&^kLG${5j0Oz^)(ZNJgiDVm>^(R1OKgV0x7jSf*NW-6IL%bnvYUp~HZMlY}j;0XYlF*D1Fo}|?UI*4d!AjvV&ju*2v-;ik2fj4iQhTae`1ODcV z*G@nP7(h#)BoL!Zz!g{q4xC!0Yv>Pvr7&qf7|8(7+K|ENefX_WKD3TE2rLj@N5BNb zq2ubYIs32OMY{GDi5R;0vCbh6TaA-!ocaYQj66i{vSFbIL^JGefZ{<5&d-5LH5U|WG9JZUsYXU$kGbUm193c7(JXgcx|I!BG}F?at%OaNMhk+#Ki8eqH%bUb8N zc%!{O6F3AxcW>ETx>4Os%WlHedpPNb*8l#^=jI_<3$H!wR=O?Qzr<&=&_P!=v1lDD zzkCv+3*-qhfq?X4y2wDh`i!L4)2cn!Et}Qr%AoeDF9BuHTG^&QKmAw)lpOz#FpfHJ zu>e}z&h6k6AoX&!7zc*m-e$a!T>ZW+T)lq}T|k_!C>!+DCr^Tfx~dzZI;h7Tt8y>h zbR>{-4+R0<>+BCP`8rRjC$*So#?Ssb(VkbQwCsdt*-1DJ=FA&{^4+~bN8_u}zWiKk z%ZI--HdLEMO&UqnUJh06UXyOUI?h#_jVl7!Lpum~GJt^9lZXG zbTJ{a%Y)ZA6j1A)`{^s9dX}*K=)r=Zp1K=s*`=2=U*#!d{&4e&&cmPl3~Eks2e^QT z0p}jj2^yX%9qthmemOFr*vbU0r#mEH2NYT+-nZI=%2SwDMIb@7$INtazAp0U z;wpIG8MyPS{7DA>Znzz6LUW6j0{r@e7a&+`1!3c7AAq)KbU+DWfnlk123aaWN-EdS z%S8LO3wMd+$Up^`)*ZqkP|?il0|Z(|tVi?6N`8s=&hr)%)Hh3Hzma^AM}4h47{IJ3 z9Ab~2;*7Rv!9jHL^8NWYXz7+A=1$hZTyTIBDHl~oUPionwl@0mxF;|TQVT1zZy!1H zyl%-h@NSTL4`9n0v`_Epi|{keKq~@Sr8GLb4tPLIj{>8=fP@K=7oLEmX?j4DAcnfG zT+lW^jITzJitP(8QRP_xKN0lk(bcLT`#A=O`Wv=-0;ByKx5y5{J9NmSBWrSPA}O_lm4bbMKQGlbM z5yp3dlXQSjz4b%zbr6GP+gg`F=rqX|2%sl{Y7d(F;zMAzAjYx4X^+SR%LLo3*6D%- zQdBZ{N5z%fj`czD@m&gq=sV)0o(uuiUgcz3Y(R;+VxeaN1e7fofp_aDedjjm70&YN z+QH;d57ksp@zUjejCnxXVpRu6oLwf$zWcFJlh$Gp#yg6*AZra`j3YI9G(Ye z9bk|yRr+@FFd(y@9fH_kNQ)UVX|AJ8%mz?bp7wTn<=W{N&_ScO=YRpU&3M&e*40{2 znjVuitOz7nuj2T?&O?;V0RczH-G=oU8+4!;`$HMxKpYit(01E1;3FMbagZU7VCu!Q zHUZP47w<E+9xya+G(O}pKBIuqxkh+5O7emt%;6ZP^ z24=vMAH)55af$`0ds9f$Xe*uGO4F{6P%Q_YyU7gRkOoF$EjSM}^!n`Yp{+JX84EU` z`9ZK_!#d%)kRlMEe&nQsZ;(N-8||%VanQ}QG4+J4{j4hVbaP!37XZY7Xio#NcJ}R| zXfcqc)kh;Jo!J{ZZ${=Ex+k??p<50PVjal^Ik4snmQ{SSrg6+8v|)3}c>kv-b}p1- z=wZU}J`9*J*W0~&=OQ%5>;ayPgE@l$4^Vc6uZ|Vz18)I!2Y@-`gSmhXf<8C_mRCWS z0Slm1hA%@j8Z_C*(gXbErbjOQTwO98Oe`7&(!Ju#2x9%=)EF*_pc;Ckq|auMCG_AH z^X!<;Z>}C->e3+}eiLk>p=B8&3nXAP(h?+&*#n|Gf&A1~ThZ)AG(zxnsT>lBrAZ1w z+8X9J?vV`~#6V-Ss8zajrdg~-&oiL8%F3u3*{{HF3rf)S`0JoGpeYmkA`IjBjW{Hk<~9x4JAe)3PXvYF9ffZ@bz3@}m(wi4VPB~4ywzJ#{8 zo#!LS2K9f>BSM2cWR1{nT=c!-@KG()e?JIoN*2@GvZ zJ!(1)n!w|DBMkuHaoxe?O&p+AI=Cu^NN2&BC#C1wD`R^_n}R?hOQTXm&ym#0H*bJW zZ#dI-v^uW}>T{(F6v28dT`^!|7rTi%Y4&a|UH=+rA=OAvZ$ zta>dwp9r}aGYEL|cJ1H;ENlj@{$h$*!n19v_eS93%)8$(Hv2&H8(3wq?3YfS(JqOu z1On_>KRToVzK~V{&e?c_xU0a?kCW|GzknHoHeCbe5du~QqKA_ki?qi$z2<2!0pq@S z|G=7wsUk=eVHtiB9qEH_-3C$23h&kwp6T5Nsmn*>dP`T50Jp?ttU7U zmr&L|z_O3Dv*?4lnuROhhK!rA9EhfRI-7P}*3S1+{a7N;J`6^q{o6mE1P7J})!%O` zbhx6Y-~bp`$uYU4Gd~9drkJLA@BIXP-_;lUA!wtD(fpAQUx1=x3>wf{0#;fR$UV-e zmw4tv>jMF~Az<_j!176fFHx-2v8c4Tf~oJL_n-Z-B`Z=HsT>WcFb}G~6Yc!-LV6n@ z=tgZ=XiN`5Qps`ViC}Vna@kNf1DrMO@*TR`bAkYi70?Y@Wu5{IBnMF`Nc0WkB`Ln? z^Pk)WC9Vjp7DT(NHw1HV!TWb{qd@(vb@{k7l5%wQSwOp7`i_WD|7L?44r26o7q5e> zZ_+|`c7h2CAxO>ARG(5rJqE!=S*m~EuxtT8y29{Xh+z=`)UQM#*AK690Sr2x;*k0k zFBb|#gLqJq<>Y9cdqK4)xdj-R-F)Z4*wwh4Jri@h2ms7YUUzn5E?oPgcLLqPv`5K7 zu+M+^qof@5rOuqAdfr^dhc|!$oST6lRC1~|mP#_Sps0Z2M`Vl#STjaKO9$R9tqxyb4vI0C`qkge|3q3|LBB=Ka)AXe4x z;t>WT-3(MS#xg;)48S2mMP&vUph>Gi2WW6c+(K#&_&5WW1+H*>tT<9VpNu&Z0y1!@ zYj|%#CNp?*j74lEoB0EDfYcYai=eDs_(gj0W*RhRCaV<7Q63232`5Xyv~BP+6+KWnX62KTm=cVO*a==J$&-Bj;dd>*v!ve`O(|A_o%-z9;L~E{uz-4rL^e! zrB}dpavWqLf1$25wEEE!TsojW3PL8WNEqGO zc57S}vyj0A)C_uwGL}zU7^pHdW@J$c*yyPAU3x7WN_FMnJj4cwegiK2prF)u-4j|& zY(F@QnVun48^$-YmJ7bdNY=Vp(Jai2(I-s0;d7hr8 zqsD4>< z>SDl%Yrc99xXu)=E`%N^1gS5wr1=l9QnQQzPEieY5G4$F^({U<;GS;A3wPuOP-?p9 z;6WSHc@TEPt zZ+CG|(mVR{I-XgE9>HW0WU7-cKs+8O6_E+iS(p>Xbut$P-8@;8=+iuUH>T~R)qY^4 zquEh?#n><3W{4rb^R=5nWB*)3+fQE&>oq~;qTan#Ht&4WOhr>0)5=EImP}U=bH>_ruTI2+A}6N z$$3*q9TVe^_uK@|cn9XpOf*239%xtY0Hay^{DT%a>2BsDP-dBU2?xvrRp`0ye4463 z->Lguz^g$iJxTO1D~W-*O~z-34r-`Jde7V}TMx^R`=p`1f4(BR!K>S$)>RI9s{gnc z-J-;?W&`b6R*QkTw2VNfk%<4{OGFIwMUU z>j}l&!Cmwcp#gM`fz5Y+3jKI5Pk&GyDx1{TY2%@5FG{a))+2%XkyWBDt^SGXO;Xrm zUnHB`gSf&*X{^zwI}Zo7GwW2)YjKl(z(4=?ql`+x?K~a|0{Pz4U^{={peL27AOT)2 zfWe0cOxf4)+8&aB_u50|U5)VF`qO@+dgn0YvBt_80db zY(+a+x~oLg=tAe{-4o)lOFsT!f@!fVFTb}_i=qZ3(7MPOb1nF>mn zwBz>eO`I)&9u3kV6+57wrXxFWFgp)0^}Ol83ok%r(-9OpU9GP(Z8n_r5p{P6h12s} zQRC9Td$b3#pmQEmw0;t|uhy?%s!)2u0G*DpM;ectw2Nyfk_t4*Bdn z%s~3|P$LYgFIX;DwwEz>fud3?{)qMO0eiTr$7Dg-V+l}>K4{$l4otER0_&_d!60U5 zQ@Q$o=-+)jR%##~Yli&{VE5ElrK6U+uwP;p!`i zjCwt{uT@mCghU)sr4pLw-*fz~ejZT2_N510{>|o4Bu$I`-+rV99Qa!2F3+$m4F5+~ z*8#^1yzQ9!7wK3l^sn;3Ifi3f=-*9+FIuDq;oHa~;@kdijPIIg`dET4ykD z+ouAZB^xOHFK7R43&1x2DZEkN$2(#`Ue2D}12w*v_nfSqdggZ@%^OfJKC9k%tgmXuHYjdesl0szdX9|m^VOHH zYF14kjx~$7tAh6CqAbzyWJNl`fck3)T6RG*Ef7VZZV+9>yKa80(ARakL-4N;z6K@8 zXhS*dyW(yArzc<1_xb24jP2$8vUGSv@#EF{$4`J~JE)(KK?CRv1H6em^=P=Xd6NxP zclOaE3DO#NqgDFtrRBCIQuw|UNZk{*^MkeK-Ft9j`|b|t&0EK%$G11y<>JqwAFl)Z z%`4Kon;(`24H2=t<_**AbE5n zm#Wjm40R#n${pIzTucZD;01IX)DIHOZ8N92-}c1^uiaB5y>4{$q0r85?&vRl4Ea7$ zx#+iV;$Q{_3~kQPM0w{mql2gZ5^6fpp$$FAL88moXn=spF*7}HLea+5A!5qvm^XkTzrR<2brtC)dFXzJ&^ zW+?jxjeaO|oOJ33H-K|&-vBVyB*0>jPhQd|c?5VV+5#)}}x&rqSH z?rvOV1eBLa*QM8adg~7#1aAW6NOvOyOP4DrZnYTc#t;Q&MAY|-EMlbq)W2-ZP;^$i zKy}+3l8w#V`AnXQu%3-s$bBHL&#P%_%yudent<{DB?Pbi+ub*%>nR!AT=PZMReEpA zY8lhP%3e7V8Ygl@r`FPk&vRKkseq%cj+!QIAXn;mAmfAU3R$k%KgQy#7YPlx`kY9E z(_R2$Yz4#vy!tQR39<9;!A|Ic?IL>9KmoM%KRx+fh^3L^4z$lYsQ21;z=7uNI!k?U zYtCnMhZV~&d-_)n-defX#gylm%^-RmtH+7X&9Vm6Zk;XXiL%Ac_5nAG8!5J&YsD0>R~X3 z+48Z}GKk$ciUw(qYxL%v`ra!VdU#{MUE>d+lU2undX=8EI0biO!v^cYPN@w!B>fpK z@7=}>*bV1yZTJhX8{Gj;b}ae7``(Ry5KgRz29lt$n0J+fF$$O}X=H=8Qsx#&T-+3@MhMza-~Sz3b!-g&B1)PN=R-YGQHi-zcD12i=~1_Z zY@+vrIms$6oEzJU)4}%*>yXPvBS{bWHzt6s`G5MwBM?Ba)0I27MFp?x@zfQbp$0H} zQhXbBex!ufCMY$3A!L@>)DQAzeM=1l6O0-fls5IEckW@JHv=jU2HyKFq_8&D zi(Y#5dr){COfYq(L8#xcB=Jx{y~q@j2c|I7Q#DHF+cI1lFx7AXcT z@a|(JjS+7=3hkJ_6iPohOKhV*vxq0jVO|PK3@wezb>v#<4=(Tv*M}=yUY0{$E<_i| z02d>eUh3j-o$+R_X3ufy^@GsS5(YJOYz@3?!|x4!ROy!bACc@lA3P4lUK;Kb8BiS5 ztH+^ik@i&1nn_mn;T^1O`-8zaB9Jg= z?iLMvMYK6O8Hi<}p^E_Z+gqI@+2dDkkzVk{XfT_orGpS0i8I3w!^0=rp>!lLbGc%x z`Vert78Y5D2K7+C$@ow-YDWKZ^F5HPX8|Co8hy#$y3+3CU)W%M7wH1b|t zon`b1sv%f6o4ESsIL3kc%FV@x8;hU=j41Vq5b{rt-YA_({p)}D*{zw3+XYaemJpoy z2R{X)e(E(H_Y+((6G$)u!O77LVc=nxt{>h2SMGy6*UbH)Pg7ADmnFt+C znP_n`fcn`pBVSi-X!*8abkzU;;ZJYg?^LL3@3q?w(QE*(k>S0GUW~-DLT1;Qo4~4> zVhn;67R`PT@WxNM#LOZv+J2pv17JEZM--l#^9L^cXdw}Tqj=WjCSStc!2LYVu{j=` zf!P5PghTWluce24Q;c4oiVNmZ+LX}tfdo!bt2!t-P4z!y1|Ww9oGim{F;@VdxQn9! zUb+o24_a0;x1=Y-g1fiYfdP(DmHsi1o1l&B4}yoPH`VX_O^0&@8t7PBSl@^-8E8J< zZCO&RNatyMPPC==Fz{JAc4M)$zz%J*(N_db+ZS3T*Bv4on*#JqqoYH3?Maeu@Jm0N zNBD<1%X2xS%*h(`zGh!Ofw0zId#hSU;Yug)nl$9SMrzhqi8&@ zsdm{rjBd(x)&|v?SrO%`ZZ$hPr_8dOr9f9m#{`eh1L-YXA*Yn!b?O!f&dCPUVM!gd zdTlF*@*Pt@{08)PclB_l#8Au3SuNxDsXtjUx<{EyN0Xcx_v(w*Emk^Y(ybV+>UYMF zbF3r5JC9G(>qggcXYShCvU^H4(z-};4Y zUK7DJGL8*!FP6bv>G7s%=wb$y70AX({i=)!$wUIUNBC+nZwDPf>T7y$-31NEU?*cS zCM}|b<_!S_)C__%2mbJ5^&EC+3!!d(DVz-8kaqgn638uSmWkNS_2eC+T0W*51T3(E zh8f!&?RG_)&sIVSj2mb4S?2Y?Arv7 z1aa>%Qh=XgRvg>r#!z+_vlju5RrsqWqw`=f{7+{l-28qV+p0HkNopw}cZK zthKEmo*mHpO2h2{{l<8w6&N;xaO=%Uo~jgNDMVbyDy8rz`^XNK%A}nDy+Qibg#<;L#<$)kZ#>nAXYmRgNKH5=zTM4q77Bb)oZQ0f? zSvrcU^g7UFhV`{e|3vP7QG~09qaF&Qk&Z6wP&tshj3$Q@ILATehe^ZSFzQ;R%PaGs zf(&5l0((H!jDA6!H-FjXX}VIDib&DjXTRB1P<@KT2V3TXYxlgI_jh(v}FF4u<&|K)cxoI%> ziK*~8^;g$Uv}zl;dTpEGxT_DL1(@@@z!{V*WsM-<^!G*59zazKBd?V9VrF{lyM^}4 z51_@l^3XJQumKQ|kuWk&x7U8*y2p8bdXMa3@RqCv=>4GC1?z*;TxKTd%v0>?2BV)B zcvGlTo^7tj#JUGC7I=L~0%gawna8#7yg$GH`=?(Wt{AWItH1^WIyh(0 z;gSueM<2-#@2yVl^Ft5)z%4l-z+v3Q%lptOv|&)GbP7)$^WjMaULl6cc+MSbH`=wI zeFu6=Qn;=>Vf|=&xA@xGHF7Zmnp0Vam;N-%!AFce=n5YfYwtEaOsB5ML5%Te^@G7x zPMq1#&<9q1!Ge4n_nR5;tz2~=NZ>ig*wo{nxefY6e`c3gv9Uu}7qcu&+UiGsU7~Z= z{w2fo{J^$!G4OT7I@ozRqosZf;Rz+t#P|nR>H zR}2)C>Y>1YnvKsF|ms4}u)bQxE_>(Mz*zNt;IlJ|OkfRcX4O zdlE|1k$4s%y(ctIW@@r4r+kFU92np)VrLp?jk$v{)B!6NJFZlZvJWc56kwHY1-VL- zD_&1wEO0WLlaZNYr%yr~Ui1PF2MJE!1?ji0UG#^O5Bf?!t(D_)>JbQDyEroAbT`V; z_~T7?m&I>;u;OJ$%0@Fipd%d`(*RI^)i{0UF{sI);;EgRa@ou%;4utjXExF&dIDKN z+d&3aoq16{{c0 zFi1GXP9Gca>fp>7h)m|s0*LX#ENTKJfr>CcYIN<&9CFBR9G+P&kLnNbIHWEtJF&=~u= zEQDBWCcDrY3G{bDb!R`#(hXWyIpYu-aS1>6L1 zd2~^32bkxY_k8@ZmgpLMSI^hYI^LTT@&RqPb2J0#6{WdGqW~JMJivv<7~N2;K`b5R zdvm}e{H=H*b_y1E-dD>5^PQRLt*yKy0!&U|jFFxIqB~P|9*hA!zjAvD;y@o5kjjO# zCIY<-ZGj$48k_?=a_wnifL^Y{V`L#5CNmEP?PcOze3s@E1VvdKu!1g@vwoIlCQbzB z8w?WIL9j*u+bddeOB+U$u?n;Mbi;J%W!WJS;^UK|A5ve7a%Do(Pk1^k zi&iu!3{>A8bpA#iHJe!;v{lrRT29It#Eht-*-Jt3$^Msr4GB!Fg3XJb;am*mxRXmc zSIfw!(e}|1z@iAIQtSj+ILJGnYo$FF7z#Anmrz}N=BbaLL8rrFt!!eJBW2QY|!SQC=iNysNV+q*}k0EL4)1gtb4ED}_U_3WKPXWx; zO;N#zrw45zs-&YKsMFX2#j%4~D4=l-&`E$ExBx2M#3_$>&~dRK0i%RQDz#+=1@xhQ z>i>NuJoj!irMKw6D)ky0gcVq3yZde#S@`XumsfUJg zF@f|4z_GY|Kv$Wmxk!f6D*d0JZsjLtpaw^%AJ?~6UwsuVfE6sUd{f7g?w})=F8vT3 zIH3K#tv<;_UVDwHZeTAomaQeD2gD4T_$gdWHv@*8l7AC!XTv>yIaYTxN+=4h}FT*Uy|h zb6Zuw0re&^9e(J(^bAid@N>(eZO%SF9Sy+@X#&Z+&!O~z^d%Kq4~X&H4?5~2XFuv$Rupy?t#t)%qZMlVnR41nbR$3wJGc18V^3kkSRC6FtrD z3&+QL7%g^;IUkTE(bS()uinOb;qm{l_ulVyRcF4qEz9b?%j%NVEXnG<_ulJK@5PcV z$&ytp9VrkxCKzHcp%_AFNf;n8m{9XP?zmvE!NxY&H<^LSkj%`z-}wi=pU-}m_7TqH z!aO(kr#sK{dCPj&-fOS*mbK53&e7y|%!$7V0bF%?aN@{mvORa>@0NeI_N ze{y1UbTJ5Eak+Q=)6;KF)^(QS_Ks%mX1h>3NFFg~%Ml~o0>A<77qtF#_^uu(gAM4?1;}``DC5?5WFd=Gu5t8`~5ex z01gj~WpTEEuZ;?aG{KXqnF`!;qCr0YbpJsXNCrq?3$bYO2J0vd6gCCMTx;Wi@d4Zp zIS>$M!>cY9w$SbB%qFPZR)Kb&sbiZ~E=y>0fVhQc& zF_aFr+Z&)V>pEvu$uV;{j=;(lvPt`{EEFCH4DDljw5i&s0TfmO284nvwLd+?Y0vA& zPw4LEd0ff$pLE1Q)W?!@uMPpl>`nt$Lt8WOiL1;u0h0;XlMChmve$Nh<~^_q;ZD86UxEGx(XcX~}&wuB57`K~B zyDXXslaMnn$AVjnMspBIWh=B*CQCje5=_S~aC(cNxhC`Vzi?LhOseB|(&t9{i96dg z2#E4+VyzX+3-K_cSWSBe+KpZ(pN2@_dI6TWipv}aix=}YIx=z=tklQP(haive?I;E zBhm>z$QL4lWC*iA{rXji3Q)gT)3h8_$yUWw2X=BGz}26*x4c(c|9psB@g~F4*N;88 zl>zQyhU_T0I@Rp7k@QiN#C@9=3qRo`YmKlciUy9_`&kb@l0Sw3%0UosgMxRGJ(0^RLdgxa2 zpbE6MGS~~9z<@r^PWvFYvp=&e*#KQRSgc8#=AhKi zg)rs=+#-)1+%%^>(H*)Ch}(r=l`CC;x@E3zPF?GQUYDg(4?07xcCB=%biO?+$MM*E zDNT^Kfq?;>O)0WG$)$|7?7GA)Bspx+F@wJGx%e_Tz#=$@C`#J#<&e#pYE{}}Hem>7dx8nS5zW#_O-P2)9=D>+RczG*`hoGf z#=Vc(31l!ic7d4CDw@z7s=PU9HVEhkv5J7ol$0T=L)ybiw0AhwXco4C9e*1PP=4oC zfb>`}wKwf#Ak;kQTwa{X| zvC|B4Q?FX^l^4)rJp=P2yhwKHa7HkX3GE%?dT+bymbk#id;{!7*c?AP%(yZdz)lh1 z|M9DzJfuCdHcJ%>H#8mnIi-#5XvI|c+2`?Rr}Z9Dx;u0OLk~E+G8XG!v)~Hy-Ds() zaNUGA&(JtVydXTTJ_E&(21hr!;@1h6LK4ozH zC(!KIRD69>&|=`bKmgN9M~^4(9V&(S&1W%d(RkK3Wz_U=-f8mk^{5{#ocYDW(#2`HbJK2a?E3_bC>2~yD;sK+9-6`#mJ|Ir-Hk7%{*TBS3Es=(DLngeepZ8e;90+_Jht@Ob8ZVu8(^@#AkP_ z41xZ)kco~qj-xI`kJy2VK>!PC4wS{J3oB=yGFI_q0MCwd5FOl>Nu}J&I5B)#AnxFR z(>^W^P`89rcm8M3rEV$25%l37v=2+Ct1YMp7LJE6#y!ZS%-B?{eYpcFXAuF?(I%P& z#Ob48OoNE*^;N2scC(b0Ll|0ha?FY+zM0Jiy;lNq0r51z|I43)?*T1UIrFL&S8lj00Gfg)e9r~)v+jX?tznRNhN?jox2@8rx1Yz0n384%ktGWne| z<`Co0^MEoC9+=H)1(*nBR$!(~qnBN6MqHU?ylZmm+HpjDtC}(GD;mN}Xe` z{hGye5lnP3U&>fD)-Yy*A$nS%wiX^HsJr+ZQ9=SlPC&S~stH$>kFWpGl>_W_GGOwp z9V2EYRxkG$D3$api&96jXgBXm*8Z=E)`D2t$61tR>80B$!=P*$@(Y=k9-XtyCqa8b zT_Dy?<5VM)x?kJdm|4KMIx+jBgO}g5jf|RT;wAw!nW`jQ`$enZLFoX$`BJy&VUu5Q zQz+~&;UfQ2v_~>&RDS-aN5JT^C271$I+?vX5?MKg17B-D=h-gikrvxL=zW$k&J#d(CAQeai!M zPPCi`?x|~{Ec%C|a%EE4pxK=7#*N7`mN7!-XCdopNT9 zi0vtyLXWx~bZ3?_G3%pLD$6m#8D=0vo!^s)hEdh7zvbR8=~f1MCWH=6x%cdb09~6d1KnJ(w?z{Bb`1QlOkr%B7rA%hJ zjP`4fP($m+gDFMLklZ_YSspc-1*T&lSfwf~s7!7L%i32>W}3%YU#qTi==mSHFmfrN zblx=tZQ0Db&?}GVmM&d8`Qb;ep|MXlK#8zQ;$5Z0*)bc!2c*3{>C?W7tUal zHuOgCX|h;Um4~JuNXJg~I+r`R!+4Zgu=Ft0cD)(j*?zcUwGL>TOm()1W&+3g=Dnl; z`05cTK#7lgt|LonPkvkG#mC32t2aBLWg5}5r*ly)%g!;&jjuzly#!_h0yyZWZ-Dtq zn~^)s4((sQg^auQ>j2Wb1;T|TF6iZGp8jAQ=JfkuGh7*ylGi-brcsbOLXHDh)h+G3J{B)GIeOib#{++S`N@GUn^Y9P4|*Gqk_@efmD-8&36;VB_V!yabEr$N)$p#wX$2I5@sxC5tK+|f1g06dxP+_?k3NIJyJIcg zjzJ?bG>b#j4(6sz)#++@+nq9<`BRqN6XSV2f*DV zIK^`9$KX_w(H*nWfAYYPvuMH{^CBi0_=JS(4?r+?L#1b@_FIL`>))b*u@Yo2oyNud zIB1*LY-#5fb>T6lNiy^6=8u-^A(#w+3y5a|tWbekcAbT8*#XTD)RtGplM}UHr0i77 z4M<$k59Br%AuZdIA}ALAt1Mhr4xlatjVU|5$ejVu{ssuC%9|&!KL)hntJ7m>a%x#1 zpqdlrGI({Si*lVKRKV6dra~248)F5O?n^fA-Efx?D|lh1u1<0V8^K_Q>yIr!gM|gm zaG6ZIV}}=Ny&)~FTftLUU`#1tU6!FjMSet!)$E;<;H(+8*R10er+;N9B}r2{j{R0+-Kot`vJPEBhu@C zpMf`wQe`gB(GB)?v;M&kF~#|Ih9FiI{!C$d!s*}8c{M&XxF{jQ<%a|>NSjdsvnb|= zJn`6h{dvcA+rKw%IG{kD3Cl?1yV&0s=%U_WJbgLY`J?B`Z~~X|_ZHfNbGZrs-xwTY z7CC{*gRi+keQ&sMfEAun>&~K^`u^y0bU+^7UBns<4*CI6b8_**HEzDj<2O7m`fuuS zHD>A-k$_WF5dP-T8%J~skp~~+G#!8UAj-&ghJjkMAF_QE1+G6K-OJt2^>5V8#z=W( zYDIDCu(@36@4f)F)7Tl}ci?r)_d7OjUdkxj+U*BSpGo4St)RbYuIh(W9Zt{+W81Pz z#1EKGpjkZTbMh6O_IG!Ucw9Z;eS6&xn0=mkKlY>t_<|_@2UieZ2nF%RTMqiau?UWd zy-dC09Qghi2bv9pYtQ8W;-`5(U>Vw9ta(!iiqf8O_5FLR_TTRs(@l=AiGRqng8z3l zU6qYdbmT&Fsf-5aYet|Ev~!>6%FBGIa`q=YQouDV^8TP>LbSbmr&xdLMhxw*o`nqC z$c);|sn3Hgg9c`J$)&5Wfdi&@=4>$XHK4LovmNJdf5is7pb8>Y)CTQ$g~xa5eL1?e zmZ+txg?c0s`vIlt;&$a+MKAE(90<|9_X29}T{F<#ue}J~Egc)Vzcflm{`~syP1o@m@lg># zcf6NjrH1eR{_?9|gY)&S06#~zEpkL}PVxN_-%0VcHKz|)W{(a+XuoA*zWy&6_C>jY zMR=Ei=Whcm4KU8pBlf9-Ne}`WIFID3`*TNiD-^CW8q^D_PWk9C1TgaXhxYc6TO2rB zLAD6K6H%*Y^4>rUM;GTN zQ14}DrRAXBRiLFJquvX_pxz%aCk3TB2`cyukh%gV%UAxu7w?OVWswDf_I~@zmp#O$ z{V}n=77eXh_R@>>r9*UUIR}lbU8;Z(m{y3N#1oTgdqCS1Ui}oVGD2Flrj!xG`j5`> zwWN;F^5btxW5*Uq&P=ciLjFF<80c8df%=yt4=Uri2hg+xaV{4Qa+XHW04s;PLoMHR zx?a26v%3w7$^AS?;G=6xSMShuC|mVLtS!{E3g#og5u-$%A2N;jpEzUvNXDhBHH$}3 zy>Jwa%Tr~vHK8KxqleK30-0=+hT$n`g0F;&$}80~;J}EAOouv*kz>ktNw@GGGe`zg z9SrT-A?Xrc%8vSa-6YR!MpOHk9^^IoJ*I4^eUAEJZteM^tK4PC1^p2Ga1Q?e7Uk25J&jeL+^tFv8yx*N_c|+n!>C9{KC83ebrQQ#m=?% zfhs{AVgjc(yXMQk^F(KQ@5Yfra~5HQxb|{ZStg|K+}R(C_3~rc zlicBVssiOKfK+uWfy(@e*>HOZk5f0X26V)OO#2D_El7p<;tstDC9(j_FMtKofE!o7 zwYRI@ZLMqELU{jZvo)hbm$X+x`IdTrgIW^|g#j!dL6mPre|@O2o*Q}~sAQ96;Pw3+ zY_1`J1}@%rkJ=X*FWxxoK|Uv+2Y0PfCB@CIrf?5q4~ zXJEPuV%O++CbC+T_83Rbuaj5){O(qp3Y9a)sUefY(6`}};oLap^1zxvEVi0NmCev2 zxC}t~D!ODFGfgnbdph8d87;~n1k-IXSNY3-nTezv_MIf5n*lTSefc<8o-0#-&Iktd zbgw^fxh7a!f-j8O(#IWQi1&#&yYT8izVo{-sP+cC5eA>PNFOgS!IG;TPpt1sXNm*L z8`3CHF3oE+)j4`peaJE392e-LdqO<6y7+&xUs-23s`gwrN+H+Ibpzbga+b88Z{~>0m!AS>s&4ZG11xhzf-jiz z5-n~a$WWH)4)=ql^erL~a{s1fN_fMr0}F}=86w$)nXT?Xx|ZAJ)= z`OVXuEa#zPIj4VD7+Uo0NN+!Qslh{fGa)!IV1Mk0DH^x#g5~02W;-KU9LPH69vlP$ zYC&D13eQR9=ws4g8b@)>{!2lrCk~0xGRnLmrIGB}v!a`sjeu`fMqHn{g*3vVMb>QPKH}U+57C`KQL%Sf z4}(&!Ue2tXebmea?SDP^z8nF*lc7ENR)}seLgO8i(EFc%GdLoHxU;4DtnKv+g@*!IIVa>=m>_eDr~HsY`7@JXG%QXYK%-(!K-&#S~*S zfHAeC4DC52d$ljRNKbnXFAOkVI!1LZ_R{0f^FgsUUs@dO2-RMll%t(f1a|3YR&*Y> zd|*4M-U`eU6u!jdQDK3$P@OgF2r2MW4*b(gKZ9X>MqQ67q^85Htz#ur7&uOk`?+Yry}aop*eqORpVf(*E#W_!BTL5H^r$64o+KzM-m)jViwEZ&!N zaG4p~hv`4Q0ut0PLu)}--|<|4;EV*Rys_oy*nnqoH4M2R1!CMm?STf}+@%uF+eXVF z*xf>qAx>?a9%)7Lc19R_XR`Fcp=oAzXW4M20?TZ%O+qF49HcI zP6s$|RZ-9yY1vD6EtpXk@Iq}`t|lAB(q`HO?6kkM`Q~R8XpbAvPg-9U=K&N>aWa0| zrxx{NL1LjB8Vn}mp9%t6M9{J7Q-)m(O!Ev_=GCXP=jA|EsmYrr zU`{-kSr>7ILiO*nRag*-)Z0 z&iw|$d#eZM%Yp|=KyAKY$@O4aAYlDAwW2PT2AeJD;yYjtAc3h|;&yTB%Mtt&ER^l; zoQBgVYl2HVbuYbCpkjDtKN`NNTPn{VD24{G+9Lv^S8lvyXJBUTz4EkltYnA;p9&Y1 zBl(D$z9imzan7># zjBYeT9SyyUMks=nS-xseL4kJh;P9!3q^mNT$sE6%AyGbxqJv1}}&( z2AqR=(d7c|)5F{`n7-v8PEE~>hpHjN`%SeKragxsg~^@0aOrIbfE!Djq1h8+g*tav zcTo1Xzj^|qQF+CDD;l0t+)@DL%dLjSykoaTC|xk+!;cPSDz{pIElnYzp1H zMk#q%@FjZ~^3NJeC3qB10NIBDR-<7f!Jx)+jO0~TVB-P08@xd>HVh3;$bWOrRBn4pyb z0{mIF`RCu17PyOOK@j$!tzBSDRYAjv!)JXTCdYBP#i&-Y9>1? z86f)m%;DolD|9Uyp`5ZBUJF6i+J&Fdym@Qp+ zlrxkdmIF|F@qt~c*0?fm__XG*q{cE&J?_jqP)0sj90;iPIeE|@^5#*d=F0lR-7Y?h zmJ!5pnXo`3XRm$4U)n0&czHp<=lh{tR+S+(=I^Yi;1TO@XQ4-f0%|H4wX1i5aX8ip z@Y&0eG&2I&L%{P!&%Bl}xOeE^A372lqCLI=#>SY-zdQRg-o9Eg*1wU!6#9t8diJ+r z;DW&9T~Nwd1jEC_E|(9bSZQDW$u3VdlOyO5X8Y%dcwGigaES?b8(ea@t9FGK&M8Na zPp>4x!r^r=lelypye;M0bBn0&sRm~{0_vow7HJA^gov`5n7|LBK38vYzuOnjZ9N;L zCS2lIvaPX*xl*sax#nQeoT)45UtWH;JHond8mhX%L$lQ40R@OkS?YnXv&goq6^o|` z;Z_g*P0ql9P zxE_0MUmrhkg4+Z=)iuBpEeE;s04|DVU=RIMux)8zjkz%F#wi8T=eeUr9S}f+KUhh( zG_E`cX{rD*f}ebL&xMyheu0Mq9L10r?lud_O^}&V4+5C>I=Y};d}N{5!w1UekBL2a zXHwn1Z$8|Q79F7@KrJ;(jih%#Sp_+3f5BDd@bROt)b@6QpL?pUiz)n_Edy;jp4*No zJ|mxMk1+DyR-M(A>w+#F+EZH_I7>kRFQa@e7H%p4Qw~`F*a6hC(M)i2vA2p4&Rpyh z^~AG~TNuy*F>K^qT>x2dt@}2{}M{a+JfyizxzvR{+#h z{F&yRX-tc8=>YdMg7t%}Kmm&^xa~>W&!usBoAST_n{c=CRh#Y705_ym^OA|6DDB4s z4eg7pjnpDuJ)X0226ZokHG?YI014E5cx5|oNpx5B#e4Q>PtBD>c(%ZFb9Cc00S>wV zD)j7}&iB@y62R06Q)6Xlq_}~p{J8l-H5#KGOq|5iA53z_^tXmWzhQFjZd=cfIqOWDHlt4tlv^*z5!^+r zN2@HohFpD2YnMJph~}xbCTR&b6{Pn!zkcm1#|&X=net&Rwz+W+LvZ1Z9`Lzs+2BA{ zy{9O`Z#@h-&$Tf^0&WypAP1HbXSom(2M*YCX{KvW7}4ZSTx}=LQ2WpPim3&#bD1d! z>;lCweDgM95N@tF);@lNjj1TzQZnKJy-qE=2>S9i>p8s}3NJ7&h8o>ooIWD@7msRx zD>0uejE(H->oAseeDx}N8tIL~^(QmiMmeu?O)RSK++{LhC;**(uoYau6wL}^g`IdG ztnlRFNa@giI3|Ee2k>RH%f+8Ka>>VjEwGQ%>k~>wHv-lY=nhreerSa55g=wz{yL+RGQ$twJqqaL|qfbbz|GK>BAat z$%)&5cCs)-#<=!5#%dl!=k5UkhDLXA?GdMIg~3p>L3A;o79ovv2(^pY8MtHLc$~3O zSuoXh^-cX5uv+HiqmywD0y&4Pgl23uwSx)f0+K{o$-eo9Xmm;>cL2asg!YTXv-6H< zEk*AM5aXCNsykV9#+FMH3_{TF86fRX8%%x9Bn`wnwt(FN(~ibGHmVir-J-aLKzhxa z_FzZtH7IvL;C~%_{w0WQhCkf*Cgf&N%_x|_f^pb&{LWDA@qnivM9X!6BgHSP%7=d9 z0q#}NATO28XAdM>P1(Y*d%V#HqBB4pJ!n%L+qEcPUazZK$%xx^{sr_0Y2SQ>!Zhj3 zNMS{?Am*1_CLkB=z18aVz8m@zllwwuXn!A9z8 z zc|O-E(5k0y*GSnl=J5{6}J86eh7OIkFx~l|EeX#2-F* z5Q3%1=IN(7xDMv5VELOj?|_zRN+(CwGGNsNMq-2#*63Dl>a~qvK&&0u7SVOSs(l2a zg5EJLRqtK@T5ATK=sL<;+y&yO0DUx<*1YP)>H^qZ!NO*eS%YeOheHM&HvDW=NykSY z@~F$NF(&|SZ>Qc7U3-Kqa};2r0Wl(k)0_B!h&nSh!zH(RSDIQ+OhK0%)L4=t+s=i1 zZa=r*85Zr~YQD@C<06o|z_Fh5Wr zh!L0rE%)E}Q2U#s_Oby6fH?+Sy~EPKZ826F6KgG^fLDTe?YPJ-T%&;LSi|CFcC!@Z zKe<^nqVg4<~b7GqwqNEZl8QKN<+*{v344>hTxzh`QX@v1z$qNNn3y|EXc z<@6i3feE(LC0kgRMo=#o$xBU6fOUZY=W6T=Xtf`;4OpBPFn+ z&y$h{xzrgkafyOBh{8+z!HuB!(0mTe7iQB1U;=Ie9Gzv8m&^4D<{`{QEsAFF0aGut z5m?>U03KcrX0B4&i|%1fa)T8SQN;SwLSg$?^W5O0XVZfb)gO=BIvr^r24ed+6?0-Z=JSm`7T)pQB|`0dzebySHB2vp48*x*>@- zxdyc~=>%sk4f!@oyT&Vv=dU*)RgnEAq4RtJHB#-x^+kR3M4R60xy1>3B5OVgluDYn z?|{%z$(GtXjZn=wtu`M8Vr%OZT)l>yp_oEUA|Fn#iZ_8ARmOgA7& z4cE9y+9w`{>Z=>bbE>_{%!+af}GX+bt^sXC+pr(Ur zT*0Oio4{vETfqet>3gC0YMU37d~Soaf=?vh#7Aj;BtUUl{hRJK_Enc(Fgq2 z+YO^mUY{LERj#FF>rHcpF>dlSubx`}xuUGf z+;1QHWlj~0UJOpk`r9<<1E4W>^sRs3;KqyE(`{i}0Wn%Npn8_U42NLy3l<<^rUFX7 z`cOBCk}ATf!qZ>HCaR~A^~HG=f>;P#B}cHfZ%AQS)-t`gL!=!0C71ZwV_>(zdFs@g zFtJ<5wF6s~GZ+EDCl*YVE*IAk%PFWtVo-ogQy|>5*Xd#gw_g)AHIj}3W|(NE%_U5% z<6pc8g)3SAU&3bS^kW8%U8f%hSB106GSG4AMOM5Htyp_U_u7v2!RG7BL|c|#&ea+; z3L0MjQwwzF(|14hJDol}RzC!DEr_la;GZT?xFBV__DU)P2=InA##?{_&O9xp`lI zcD>Be5adTd9#a{k3fsx+!~s~tjM3n6?Wv&<9B4NP*aIs6>UHqM#8vPP5HKSGylfgw z`#g}T(ACh_Uy-A_A=G&#E{TIneu*%gcmgbTA7@$z8UX<+#u>&kRfKVuj(Dy=9IP1! zkbr%loVrjB1UR`Bh{E{L&91JED&0nG&`eqXZ2peDa@-Dx?aHsX2xr9tAtHF^QQIb$-YeFfOA$;&KIXG**1T07jXDGt{0t7XYO`Rr?7NR;7aLWg^f_?~ zj%tv^s~-$iaTnVN?E!Q!T)8VKMtjVwbTR-Z4;DraFu3i$1w#-<^OZrmLj?!g^$&vn ztUbz&^{bK{S_v1w{=u+S=!}*FXaQ8sc$GM{X>S^p_x|n0pK2dBTsEQ)@XB3W)t*zA zo|_?-H4)LA2e6tbG9AL-$|7DJB}Uw_dQ-{wgN6jGK>N zybtQqKRZZ037 zM+6-3*aM;$jc$riF7She#RT-(Ts{b)N=o~gUNr68=Zz4m%DJ&}1IUP4<21D|WN1GW zrQNH$!nkdm)ZVCTuZ_JsT49I<`7uYow83+94T|>jP#hN6^6gKaSDJ{t0xKxmqn04^ zobuUH+ZEumE`=&gc4LW9Qzl-vrID1m<)8 z1y$u}2zbfeIX$}|K7CzG#SlZw$t@5q)=b|*(9IXX=qdqgRlph9B^zRI95#kPRhuhU zqn*pVi&xQc8DL5Y{2;dIt$o{|@*Hz9wL&b?0Rr{BnTC7`QVe&Q9|9I{Y_;wK87#F>*ZASY8=gKOY zC7_{P4{#m6(5$Dt`yf2k7R;@Ts=asGVyNlhXqafd_{?UWG;{0e^}nN|MQ?3HE>jmM z0`bn+nloW97I1e%oZ8Vs#?_8D5-hzI0kQgF+L7IngJQV)99Q6GXtRM>ium#kF@sxq zHqax2#y)Q8$pMVZ+25#0B=cTsK2v&eKN2D<_9XP=lU4-B2{v9 zuIU6c=0Rh7xrD6j_OTFS4>TQ~(C9LM(2Q=ax%m)NhRe5OW^CahH;=Y+TrO$K0xT26 z3K5(;boDS_1_*dkncfQKY&pd42;Bx{WP~prwT}va_Tt74=%3wp;pEA&XD z@3}K{fSf^epP;%xmD4A@I|1QiBD0EB4Gf{}B{oi<^H9^~h@39>D*?y7_0Y$!Krp{J zj|pBAU}kc&P6crs&b|fIR=?bn23a`NJTt{yU1L6)T!uz!7uanWVA`n2H;%9n)K;1k z0Dbp3xiQ!^A0nP5IN^WhH`>>Rp!fk@ML$_NJ&h822W$o@zod|9&ZnZ5)fm-hP`BBG=KRtH< zx}C$osr`^Q4)Hq2eh%ipErB5dxbX-tESe9)msw~oSWBphvKu{wmQOJr~ z+0zC--51lrM2Z?b%dD8cc?cX(+v1)I=yk(jejqj3d8nc4?ohcCjqRY1-;V|dmO)*h zJ)lVtP-R8$Giv9b&CTV`(lWE)u^LBBknLJ>^~1&yy$vO@66!$~ zwKG>=WqA~YfeBRg+R*F@8X}8Z0Q>S?u)6K)njGuWLsb#XLtv{2uKM_+p_NzOfwI}s zmub?<)%)Li5S)(Jei#x`FvJ?O83SWE|LGo@um2j1F}AQW6t+X^U=ZmEmTWX{7(iTi zLI*^YFZ~rHHcr=Jh^_~$ZA`nNSinOY6SR*P>jvf-!!tDvW}hPVVJ?0ePvn7E)!q;p z8HAJ=W?x3K$^DEOz%`sY$|V7*Q|&!bQ&4gfxArnsM(Zn^r3<{$_2LT#A1KxP`fnVt z_S8Fg22e|{d4sr?ubu-Fc=|$eZ_qKBX&M#A^*?h&LJyRKx`Jk*Of$zfmoNc<%zuCN z83^sWPZLl)6OP|^{X;0{j)Lvx%&bi}9<(kq{6M$6$lLf#K}zrL>xXZ%Ub_>97%}?O zNzP1nAkIDAY}2M4mgPnHO_$WjfBxBT@Q28`5Ok};Ym1Y1JR#=%AT%ygZ3+j%)q@jent-BNJbFAj$HF? z;I4EtGY=SH2{98>r@Ggam*=GmoC%D$-uf$T1sHS%OG z5EzLAW7Y$W!+FefAXt&KZsXdjVT%aQ@9vjAW|Ql+SG%Nj@l_6!G^BldduX7=X;kds zecX(iYBz!N2CcpKX>nnOHGHC>HQy(B#niKOA=VSC%F*|!^HMd@Ql1nmrgQ?Ol={ChkKgCq&Fa>}( z1lPC5CHsN2Pa8Adb&m^zYXgf5J zC&$p<05bo}+duPU=#}5Hoh{Lb-*oX|h>g(dqo(Qe$Kzog(0dx8(KV}L*~0tG=PyVj zb!ztrG|p-yY8IT;v=MY#8i*Ak;Qqww+jaG~+9!63#~0NKN|`~x77^NOL#WoE4v;Di zXjUMB3zMzKp6=#;xR=MKf@+<>!eWDZxPz!|C8&d!=4v*gbZke6<5(+q!S0QAmMOj0 z&Pvi==W_DZw7c% z6AX)Btl8}ex^M)H`;j0zx+9BMt>!#|Gw*UydmK7O0~J>$m`yS;FXP7FaY2&kD7sq=YV+o-7A(M`)H zx58EnVtfEjgjK-wa8hM7zdH5RD}0%>%z_5S*b6Mq_=>WTm(uwTD`3UE-|!T={qj$^ zJ$Z3@%xfLp+M|S$)#b%}{GoYU(dL;C>n{qS#v2_6ri=4=G`I4lbH4?jO>Ly#bNG{AJzsugb^ zfFH9Epi@6y>}UN6SOoyHv$?hY^qbO*6k$7tr*%VJ zHn(Ev44{%wi-OpKXf-X5y`Jaahjo|9nLKsq$~{aQ;Qv(EZQ9HMEkHpao8Y>w5JTr4 z2ag8TMcX(gF~F`618$rS*Uhb)FB5>j)6m};3=KDConZVseSfFx{r6J-%Z~i-sQg!w z{l63J|K$YBtJb%#_5Ta-#T)Qn_R0@9A%c4{P~wve-l1;e!3`Ea;8F5&rga%)v-WYC z@b9@l@u~53*4nL5-DC=W;N$1R0*hO48SUree!!~!Ylq~8@`c`+Uy<85|9~G#z;OaM zL$ho1(hqqUK91*|kKX2;a7WP(_<;f(x67-&H7Ut1DR{okI@wEj9l`zn>w ze+T<-M;)#d{J$OS!V29He4jFP{)zY)-Lg^P+leUM@V{Pv@u$)l-|&l~d=shtWur73 zTD|GLK$jeABLcjOZzJ4uqD=I#x|@n#88wA)v8S4C_DOn$9=W zJ5mZ_w?l?VA9xYp__>k++9U7z7HU=6oZ9ubOcUhGa@EC`=6{Xh3r~vio2Ty89?SmG ziyIG8e`G1pZM8dXp>Sq#Yir%Di}HT5Z^1)Cj!1_0J1^a(|@Vh ziEWwdy?odMe1-2J1j;F2y{?vV{QhvgkMFqll}VFtf2_5qwpeM;W>$a8a8#!;lNzi{ z^H?Y?o{ma#$o$J2YW;ltn~YSu>cT$=1b_xW+3k@9OGUk-*3t`}%QwmR{;+P~-<`A% zTe*p;U+>usZL8jL%t_<@ihq0f@^z+^4#%}LrjWp1*Pw6;I&xBUHFk0gdf8ejcq#w1 z0OW#r=dJzD*i4-W5U}m@UxJ^0OPIu-VM+H%oY6qJNWWl;W<$L{PZvUAPPYOW-8Fj1hX4amrYZdwSy(jtqAz$ zm0y5!*9foOy*Mx=R-N}onnA}ekC-tk!wi6q$=PmRn`v0!4HoAgJgIlaL*wNAJQ+Z> zk6h-TJPOU!+Ir9seEaLy9oSQzGT+}Vrc4$^r|3JQJ1v!$&0ApV_EQT{F_xniy2jG1 zMUAFswa>!(+L7Xs9BLbYS-+&>$!~iltrp|O@>)@yNC&HGP@1GhAxft8C zxobR0{gq(hqenwk8KF<0yq}HMN6$=@LwtT8*Fp_on#`a4y)>rk)70igXY!Pd$j#>F zxt`tis+w2)JYnr))-XI9!AM^Gk#1R|uTv)5cl+xmro8qiX-MUYt!f4*?>_r9uQ$RJ z1DGd?9MLBB(#u=K25#@T{^TMwvbZ^M_g8;A$b$;_g7WM4f!lUF)rl=3Jz|GSQ3i?= z0W7)hwYX~!LeMiO?%%u2&2VZJ3}_Vrt_(G{u}a%%3~-?XKEC(dQA>_IG^!sgN@Hyd zl-vp~Vv8M@)o|@YW=|DJppt`$1$2u@6=4s-z$JL>?DVo{X{SpEc!V=pe+XCwXzBVP z?Z0MFE7ce^+VVSK*#^>~e2%tCvLLgrFuj)>ExG7?m>-c&@sG^1HK7nd`t=} zTtG)9pL&)>!Rm_^>qed{4V9iFSD%R~Zt>F|&%vnirp=Rl2EYaxp^WY}a3)s&=< zLnK9Xft%97RQ--?oTKCB?{qUK=?6jVmTr}LIl2wXGH)h;qY;b;L1`Mz^xL>jZSZ1eyJ-~=Ox zAp<;4y`1C3SwO&^>bwOA?ZNkuj=5hxD;w-#Pu zh@Ns}EP6paNzDWr8%96vJFZ4)>YO8S-aH#%*a7XQc|`^HICpP8gg59D&M~>~nDRLq z3}sOdm#c1diRYfZTBbnhKh5#_2)ClIq$@z%^oF#DM9yfzgdqg@bK}WZ)HJ7w< zLYudPsYBFaL)^B(7MVDF{*})ULW`&ctKe}e)~du{0C-vL!#f;}8!vMkp!)!xGQLH0 zGl*PKRdzDqgONN-kSSss!V?s|80t3*882pVZXPif%*0_3kJHfu&4nHVe78H#Gg;RE zzbIB7p=Q@-S)fA3eak%6e9S|8u$gTI?=k?(?>6|NU6&-rGfWuz%bT^q9>DdR``&_3 zD;Ha+2)Zmptqy9KSXUq>mgB(Hu=C7JfEC$iDp2}D@IN1a`+W`>Vc|WsPy4BZ-Dp+y zw6;Q|fC@kY#zQa;G1>9yU7W9=GJ>9O2B~u=ZnAMA;4E;Q+8nZ zp_i|o5hM5#-MLws+6T(pX#{mu`2<4@nMohlODDy+RD$<1ifN1np!~fD>Ur(w?HBu< zRlK)re*)X?%yi^kruxLA;Kb42ft!O^vQ6{OqY#@wYA{~aF&2zh+lV$aEOh=<$L}~=z5GC42FPL2i!z#aa0PL ztI;Z0f!d7_7UoT_97vU=lozy_WA0KLD3|BnutUYKwZPTI#xiQU(vEh9>5M6d(W>2+ z3=S+B!8r2zf5f#rw)BB(KiLqVy>SbiwDJ)9NB1|Jeif>bsWHvHlY21dWdU*k0amqO zYan;+Tb#fG?->~Q})g~d6(;}D)oWL9-wms#xB-)9cYjF1bCwP(xM8pO6xdS&;Qh#uc4S) z(Nkb~AVC!EIX+zQpPqkk*SNXfvr+l@ICW%8Ptt%XZd#f zeaECKL2OJ4ph|=t1ox{n2Q_JxSTJ2x<63VvxT9GU%WyNCTtgEG;OT;kA2N5H62QEf zl>%<8476p&Osw|I2I#KFZ167a<2Ytod7ue!Xp>m~BZS)N(XlA~e?D?p6yXV*m)kGA zEjnLi0ELAjc>>EYpP>`jW-+66SO=Tcc5qn$>r}^hkV+Ybr+q!^p}W{GZwqr6Uk#|u z%$B2rMvYK-TwNxZk-L8Mwkg)14u%>q+dS~;UDlyqP%A8M{IcN0YXemiJgJHfX421s zfEv&?u4c&1z?jpu4D>D#<5wGA%PmTKB>>4p1h`hg^35Z<-lhFs60?RiD&S?ZQ+b;P zy1P{l4Wn`5{&}&27bN`Qt-D!Lp;um{OPfJC?Ky57i&hcVSN0!&bS3kDd~g))+=*&B z3z+L_Ut6nUWvRnWd(HAPbKP<00`hwfSRT>gcmSN7yt30yXkhO z+$ZmVsgS8W%7z7SV5WxWbyPxdJJ%lQ>gb;W)sibPo16*x_a}dAgsNjf*VN=jVd(GE z-qlH$_OnGC4k!U>PZW=tU@CDBG=Lm=bJzwFT>WsxEQP)qpxqy&9L?BHg;l8(8Al5U z24$Oh)dluIF_i!sJaac_Bh*Qph#*+|T9-Qw9w2VVz?pkQZsc@isTlgYMRH49TJ7m^ zGn}}GyREOkn-xTPm{;p4Z-Rm@MkhpczHm5q594&}*PrvgGR_+z5ExW6&^KENje|ye zFhj$+-JHD(EQE39xiy>tb6zmJ(^V}+{+Tokvwb;OydjXM&^cvEV2n;DTj?!eKqey^ z8o=Oo6oXACa<9tPUbjKv$xh5siS`38+D;eNxjPk`9Y4rB>bCA!T0{c!t(dN=8Bf;808?ie|S{1n)tY8PKp~mNUT#dV&f-TS3bp!KZgvr{7>D zJA%fTVx2$! zRBo(`O0cY{H9sRgvC4$trF!J39?wE+X!b7SQT-9epIja0FdH$qqbH_8d*n7#J&;M~ zY}zK*UypMq+Nyl9;ugiLdjI2rCl(U?`{^uTQ$lSbM5%S@iTi7q%=^S%P*&BcJtPjz zYB8O8U=d2K`Ht{jNCxwh17J)62GGttv26_!C}e4#{E5NJyVf-tDg)`IoMj1}yOFQU z0yww6w01Dw{+L<-iv(C%ni(`gnnHQ*-g%}qL%qkvESw{kaRBZ21UxHwu#N=|HXhhK zs9PMMapHa3ue@q#gi^)9Oxu~08Jbog%LbDp(LDZBFu|f-C=UQSLeC~NqP=o?kt$b;W5A$zu-D_^z-t& zq{$J+w&-0<$&o!OBcLpk8J+@weN zk}I>5=X~;f9C!;S03?Q&umi~1eDMG$tUMbfR-Y1)og%P*<)yC2z*(`RVnEt&P-vS-Ez=m-1affQ-Mduu#dDl3x3=POwTVC%#3-Mj#i1NL za_M!$X_in_iZ+`P zB2ebW?s4wB>p#=MwJn#q8+cJ*q(vln59rFh()8s#xm~0$6}Kc(x$!F4Fvu1JtbdN1 zNy-_H8NfV1-RnOQfBY9r;~v*g_nXgKtu^GcWf&_R^MqXf{8)%MBL@$mljWAEq&QFuW5EEprR zRcyi+dRQx*;iVVnIp){*9RSDwkHG~a5LSUjMb6dIW;P9h2Gt(r*V{Q2w2vFlWHlEs z0NMzi@8y^Ue(`xQJCJG7fvadTdb5lH1CzH5Ho}Vl%FpTs!Q(b!C6tAcs(oNo?B*g? zLyB9XTGhVk5h=zgS3gl0mYx*R_jy6`-Uo0A8?pZA%dxMIybOCZ2WNf(%cE_@)p0wy zRXBh8vQ-mgA0ng_Y(GU7F$M9@YA?3C8=*Xh1=PTsUl%&*s|J3X}#Ssg5PK^_EG z?-jm1%3}};>tZsjjP@e!4TPa`_^Z3aohQfZwI|SCd8>k5AT{$nP`_l-`-6ZY4$jy8 zKqzR(`ZrnI&~S0AzVu_q90p$7pCKJvz^p2&%RLjh!3d&s%~Sj1->!mGSc2BisdJu@ z1NL$vtRUvqTtgNer~Mge?2jM31U)n!Ob|4t0?7#jURNG4E+(o)SiqpV9l77_0BKKp zw}euts%pFTVrn}r8DKz3*;sC4aZf6Tbol0(Cm^f~YmObMg%St44O}+R%#;(1i%_nk z(*fpLzo-o&Hi-ad3aG>SSD>o|)cb3CnN{3u1$1fJ?k|7GGE2Dl9(^sy3s}D&m%`xQ z4dz$eqV62peVDVH_EB@@sDHTUS#Y3ZFsFLCA{d&ra?PA{2p{5p!<|G>AcBW;4>boh z40<~oA-NK^LqI?W`-lAUTQgggV|FbyXz$}IUPfDRZ#8w#fuUBQqn z(U(A__DU8CYmM9DYFLl<9KtH-&7gSgfAPDQwd6UO)e!-S$lLXHAZ3NdS@PnAN%j4$7 z$?eou$7L<-;|3Y)YTE$;us)Q#*$wz_2D4Ea;{F=w`NEHH+#chbbImn(T5G1g&pG?-&9P4!z33WF^!0nv!1eUG>c|`4 z9$N6+-MV}?Iw#6L{{R{V*p-GstxFh_Z`>^iobK%fHGz(Dwyeg$_RM%9 znq!~@klx`5Psr-^k!XQj`J*tTm{X!z8SoABq@Mv6at;k(fvW=*PSCgq$LfwcqXlZR z!ryrjZR|S_fiH^^yrL=?`~)49L$?ZAmuPuLxnsk~^2pREq>-+VEk|+GRpRq|p=Wj# zoqppNXx`JChcFsKL0KT+^k}AS_EAR6Xnt<#OOGuI+Aug zEsK(3Mss&Mc{B!D{Ulgm6U|De(X8PNU7%1sE}I?;VBF<2fHe$+l%+>)(I^~~uBecx zX$DP!1ik<3!G|7&=mjcm1FNlY0FV2ZM}FPW3E?op?26`tIUE5;GsnpZOtpr191;fu z@Qx#qUb#D5{Y#l{Y@Slj?>X|pw?YWA(5&NA*0U$=7b|4@gJvr6)eUiTfAO9J3M6F>j!zBg7~IQS!NL;8QPIdWImkcN72J)EG%zq{h=vd%QV54D;PGv^CSy>im)`-M zAHtQ*AWxl%5SiAE(G%e63&yt}gJiWDO{P9yMXP^=paZlJ>55mwzmYWg1l8I&ntY=1g zA|Rb(4%_4$bR{fkg)F#mcOR|D*#rN2&-EvG2FR1{Mps9uk85(eX%GEZz5zx9g)jHe zIKV83a~D{yc0T>xTyE#2p%&rQCk`vD$&`Fp4Y~0oB0xKg zvWln={%H4i8R7uLr26n5(a`5e3qb}reI9F;(Fg>nyV~Eqq3HZWdhTXfFv?s+9{hh= zqG^jIQ4p_lN_w6gqhJ8dZm-Cr(&VA}QoJugp`&5TM6Z|1is1erZDgmAdDe5m0S&-R z)K{3CdSQI6sV`iXbJ~EXo+QL1JtpqE5ya3q3Zr{w`28&C;H!`1~a#%1_9Kq{3b;OgH4((_y3 z^tEZZPY&z7TQ)!XnYBE1^~t(i#B=>&a99kTwbnsxa2>$LN;-RtKHYm5tSdmFGy#Vf zLFOBAM7196Tg@%rn$e5uq@WIpBOL}6^aUS~E zj(UVGP%XsffCtfR(CereK`%M6fK2`YI$s&WNK+>!CiPC>)-dLJ30#Uoba`zTAoG|byvqV4*qc{Y_ zGhF!YDR6GdInc@nAfQ+p;hxL`9^&#%=YMW2&{snyQDka!h3_Vm8mllC-uzJ*kLWha z;~6O6F-@DS7=Q~4@D(t-G#{~rE(p~2>XYDemb=@!j8#|NMWju#M5IR;$E*jhJ}Nqc zPmAO&&3k$>I)9!Opmxdk76MWOpnu!q%6uW(KWYxDGtIV=#<3mC>NoPW6(B;6&0O1a9m{J3m?7#LIfm8 z<5^WDf`hNLbbU5RpLsce9bGeIh1M!<{mVB;YfZtwe)9)ap3(tl;0b~_Hyd46UVem= z)Ey6q>fxpzSv_vj;m?Sse+1<;7EmaS{o6OcbHhXsItR3<<>HZ9F*r{%ov;IovH=4; zSfFb~2IrpF^n|lY9bBAOxEnr7cM^P$(Z-;xWBN8SL;)Re%-eVG$6CeQ#&PKS%oP8-k-q`_j0eIivo?F8%z~$@pq7$ZMbr)_jnqZt4L-^rHGJWL~TK2NAjlx-iio zKuWT-wAt>iL%jRh2N{3+_6zS_&!l6V2zbUMnQ1VyrbzsQ7vg(2(q(!skd{rK{d!fW zcAnpW`|c?RhW(#!eE#E`(K>*DQ>?F+R6P5RBjRm$U&5Z7h zNb6`?KpoL`#v%xl&i2yX-pjDt-W{K&*#&3NR39yzZ2tU7L$^WmG>(BWs9(K}vCrh# zPiOhkLy5Qiz6BKnVD6KvD`{XA9m8d_5P_p3)D6{gbCwK_Gukc62@MU0PpCUO+Uj^b zAT2|^-z+2P&F8&1YHmgQ>N5o-1_*s4P>FQnSd#P@7&<5VA&wVvDu8D)7$Zq1EhG~t zBZlK-#WgULV)^4SRQfP|BLl8o-Y04+SUdq|X?i(tf6fI^|F5>#AAO1|3q2mqXgxqN zAkWL+(uFBgJH(4t2-~EQT((?}quE;p;}mj02Ou4#J(XGLs2h>dc5%rH$iR?Di_+BT zIem6ROBpmtTTCT4Sq9GJZIIpt24-AF5Z%CFVBvQModyMh99i-8UN3zgGGMCTt}*p{ z3`6uf3HFzpACd*&)6?84gIo35`!jSH{;X=d`|;p{0%?5cuXYeG+T+y|{2>(3DE**p zP{}Z@ec^733oftUfcE1%tfv>+)6D(S!NWANfiv&@7&6TmKi*B_F+XjmL%`=iKti~* zIs`v?=zz&1@SN6xj&AUR-vpXZUZ^ecop6GrgK^&VrKdNz3G)ees~^#!hu+kwYVf_P zQX1*y>q5D+$@pysRO6Yq9-j53odNaF$(g*~4cgpRUQ$3~ICX=4`fa+cge;Cb(Blko zg$yKLeBh05qh-~wxo}haLB}Zf+U`NC0vt_|hz4KkNqgg2^j3;RwTGs$9IQ;e2@--^ zI)|)l>RhXJ2RfCokmjdn0lLXR0W&OtMS}ia=vMa_sPA**E1}o`g{dEKZ2AV7c%7gQ z&|+^BI1nohxON0%hfxV|AU(z23@nDP8|ba)wG}^j2t5HE#pOVBGguIax@qPZ{eAu> zFksh1+BaU9ZAr6*yt%3upwX7*d;mRjFX?TgiSh+wVA3t>*Yjgs*RL;>Gk^c?{+f%= zjjThYG3M6#7VVGcK=PhFvRr6+w$OziBs1avTZ}+!tFDijIe-_-Bdw7SDg$}%f&pw| zA_{YuxEIn|}gy|Il(orzmQPqc@ zi5fGPRe=Cq?fYqHAiGi8lJb2!k%YIQY0`mbhTJP^Zj}rgzwkoeM?V)G&I~D#Ub|7c zHPzD6_6FJxAa)sDPmvOvK77i%dlDVSTuyAHtxH3Noi?MxKkp9GAxs-uAwJ=2Y)Rue z;bL;3J;&p#A1{_?5Wq#6(_xfPeAUr%F4&;5qP920x25_k&QVzan{C+foHHa!NH`cFG4!h7-XJ0Gruvt|_?WKFh zay!1xgt6JUE-%3{c<`ow9!Ota+U#n4NXDi$abKsPVgt@KdafUQB>n3Y)Qm(Ve2Z(Z zzL@!nE}~ZrjJxpc_!YM_w)!9Ee#M}M#s2^7+-&ih3#gZ_ z>uXVT4|sVJ4yp$MouH<%L7m>{szy%xzz^gi5Zxv7L z3-8EdfiC*Ft+K|UA2gugY%SmH_6{b>D&YDu4N^I1#n3}|??&~KJ5lkXgSy+)KSHF# zYfSAENP1wL&8XEzvL^V9gZgk0rmrs#4yvGB1r>tywvYgtwXunls{!c?$lWgK6mkIx3vC{x3BT^>7Rm%3nPw%t+#XRf`~Fh^yZ0%73loU z9H#mJ|DHaqu@d77=;<@{jPQe>dKH`KMi0>P!ZL<>t7*}rMfAB#Pu3TjiO>trIIFh_ z=|Q}GKeUJ8*EgJ5YA9Oq$k~cPegrgNGNdDh)O7I~Ll(gj*wp9p<0)YAg!CfN@R}aT zGMPmH=jV+^qc=FX1)0~{%&5Z{qSrWrD4^^TZn6{c3Qij?fa)q_ z0s9AbUHa-X6TMJbz+83%F(P!>4K0~MWFh#BNF3qNrE7Rzi4slm2p4hG@t}DoAuy-i z;lm@%|2JdXi#2`8(mZ~%8;#dK$k__C<2WE@;s`$=EZc)cTZ5WM)V76Wpl_s*69=5= zAIeP3i}!&6x!j*1l4jWa!|%bZdu2*>scD zeH)}YQ3&_8e7LemmbUGVK2t2qRJqMf6 zAq!bro4Z6dI9;CxI{(x$@bEcfK@g-RqsNpA?JNaPH~`bmi12}2020hGNI$wqS8CCP zagXy}VA*MNuIW1vn%;XsmI?Q%>tYDCS*Mn6Eam~r-+B=|m1F6lS64?T!CM)6OdYc| zg?V6mr-Tj0IpdC1pObC?(RuP!`5Ql_iMv4&JAeMs9fzK`2^F;*C#Al;AQbb>+5A+sZ3u9{o|%9+;;jWJ)a z6Cfa{yM_M9tve1bH+Vo`hz^(>z=1Mp!09am`BOnWRAWC&c+a788%K0Bi31+h^Dkxy z)2xjkom}p$4fYuqu0{j6L`ge9vs=DUhL{if{Q5{*f*}S>(FEKHBY-mj=m219OAAbb ziK^S!DnY254E@ z?J9ID{Nr$nnQ&>a9-B7AZl9Uyuld>Tzepmf1G z#TfN`je&kc>FfXIU-lcSf({8lkQg)IkV%wz!-^+jHVd?CT z7jzivD}+Spuc2Z65=5ZUgEe$Izqab3@f;L2hb!2lB?U$OY8LY&hUN;vFAR#Epu<|m z`i{DPv4XSwkH_x^?{St@jxnJDrXzQ{=zrqFN6~I*ze2RUYCfhtAx(!Sv|FH3kj@%N zfHF$qE4R6rvmFB5HlO65K3G>$BT=*4II>p;PbGtLtgS!oS|~qU<45uizwITZSa9$x(FmgL&-E z9uIeikPhN7X{K17I)G*ntEZp^P4-w_L!F_~dqqEUCeqfSbUd{3{B!7?k~Pw~MT&c> z^irmIm>ubWw;ktVJu}fBF{5=G?Jqmvd}iKw65Ja2T>T1!p|8~}EP{TPt|8D-5bIXl z0vKaaz*9I2%4km)_^Z!!S$Ev@FTea=8k9ahv*0|nRFs0f|4i~U8f(;>mv$qzxuRg| zrDtOy>B!Y_aJ&pEMk=5Ox8^O~gKlYGxeaX?=#}SnmL*+!m~~-|RK$6C=wxQbPBN@~K!6TxrhA8Ibk##A2lVp2qCNXK+EtMU z|MkgdAhG&P*ocaLVoHQC{);?vMQiX?T7Z6J$e2JI!`v-T55Od~dbhpg(|1DmW!^9qhh*dH+z7ypgDoGMOs`N z(1`(2H_zf<#Do{ny>ts>`rO@M+PT#nLoEm}gY=<90iJ02tI75UzaH+QJJr{X)=_5g zL;^TRfiWKaYU*atR5BbYb;_qO$m>7=eWTO8wVfHLUEVbbd4+uw7|;kRTADp){2#PzU9$07R+NajdbEe0y;PyAoca1 zgVXZ^W5wV!s7|FG_2yh|&65MC2kg&RivXVNKNR7^7>fxNoeoTNI8k6Qj<(+qztXbm zFArG4K}5TO(F;0M<_mRVK?P_FL8AMrGQr-tD*^n!LcIumT-}g&AWJ#u&()uE@(-`x za3sT={o;%Fd7uc((fVnH>JXQT3W zvrkskF^JCIXTKKJ`3I~8=?qk1c7?J)c%n4#J-}o$U>Ob+UwrEUooMJV80b3zLs8Je zII@OAz-xC-Far<6j7~f#EA-UoccWd(8VhqAfk3mUkLzfs)~!21Hx;xO>b;?dgPwZ+ zo(Bh1!cb#I1DAgVcF;Sc9)Oo3I^hdnzODPM@AcAAhtWud<&J-du5N3xIL94rd|bHn z+@rW(b&3_mF*M16(@`EitLkh?`$evUc~dw=gP8ZgdO$58^`z=hzr?B{Z5euRjb8Nu zaV>y_1gHj8>Va-ucoDtAZdsx&p`$UB$1|Yaam05<=btAQ1+y~5dF~tuU?E92gEeu2 zrscogdkb1YbPJ2SAQMf`j=t{h5S{a`H|u-J%lCGNqSeDZ0@0r8x4+?+hBg@l^v{E> z9R*|A2D*Ai)CIH)HxT%7G|4$7&0}cQhtXMM8HdfIF_@XBjj#U<+(21I7i2v?Mif9> zW`W{17#028T9D>ZOXK&s)5s=}pWO~;KobBo8RJ<`lf92`0|)4teNZ6xZ6I~p)e9Yt z0O&q{7x-rE=>NXv>4!xHo@NqJk8Md8CsC$z!p)!kz8=kZGrc<6t@n258dSa7C<6;i zK`i@ynIGTCkOJ!2T4%UFAZqXtj@dEH=!>A!{md4+l}Bg|`mDB)Ri2(I_~Un;K*Jh4 z0s?q?%K_$$UQ8dKbeLm;rUFOIGQFM<+Xk8lB58mT|PpjiMRW7G^}Q_k-=O$pTeI)b|l< zwY?iFv(YD^r#JH%a$9i)0kWRuyQBo`WYsUcvh?&W+F6ED^DOe2qL{0| zl#-!68BhczP2IW#77YS)agk0Onp1P!pP&QH$w+QOGwX3d;yf9(baNfZ0F?(u&k(rv zmwz5<(?;bIAXS3GMm&!7AQh5&?P*=`3|mi;yJEVO~r_0O}(|#?=45@s{6; zg0>R4qIu)4(?*{u#S8z6J-R5M8jKDTFb04k7Uu+zHfnxp9}6n*+AZ8Rqz{Z541?cl32( zq!q<5y88SE#&g@a8S9PT#3HSg9u5XRUfL>aQ1YV19b%3$li5 zUCWS2CGOIyB;r853I+S%DlovBV$x4Pt4q1|9mER4OdCl>p4RPXaIUD0i?5w#^2I|@Tp^fY72(=Lk5oRn!pQP zn)BqQv6gK*oGVqBy1{|qSZ-H8Y<4oVb!ex-)UT=;5C9!%kP1X$yHe6O3GtM4n{5|Qa3>w0 zs^865(i_zbXK_<&d)1+V|C69TIpump0P316-3-Y+>|=P($ck=(F}MWM|a zIp}q@FeFMrbhwTqNIVtz4g@y5{LBaUKr~R)2XT$X=`d)AHISR3^l4*ms-MJpZUj04 z#|K1W=rypKxJ`U?H5&PQ)UN!1fIh9~zqPVK14Q(g*1Z@UdQ$tWIwMw>GtjO8ec!`E zQ3GOf5AT5isZ7|;|Gf5bG;4e6eKm~HV7r%YZt2g;rb9S8?Zdx3cIQexy6YwRW%xbk z$eD8ws>0wp@$z?|9t7Z=hvv4(;LwQ=c_|in|jqjcCwOkoI?q zEZLC;+)w?=cj;QMz&~I2@ZJJ6oIFS4Oy|-M@_`aU0Vj~>ZwaAX8cT;vOI4a0+6e}4 zFKHc2-4V{pfoDB}*J*$Cg*khm3#b#x+DkW7es+r}!1wJA8u5#HyC~FUSr=DFjF*dU zBYQ5UR7CJT9+G=Dw@-_bL708+J2Nvn<<7c@LU`1-jw=Yee(u&!j!fM-prG z@!A>508a^@+!^Gs%xefBR2uMMkuqbl87`+2ior*y=P>Q8g$q4~xmJAobVnX~vkaV$ z2r$h0N*`B-vcmBIOS&A;v8CQ%>K#mI1t-wxpr+Ra zGo{I#s)2vI{eH;S;h}TFZjF0CN$(06a)9SBk1xo5_nS10fOV8vYk;xe;-)tfEzbr# zX#u$v1Bj;Y2bQ z0dzLt_r*7E+Tc`tX(sF8&{1zDq8Cqry1U7P0VLl0mnxc0mRz91a%;Gy z96+~v;a+f>+-C-?r#BHCsGKVjLAV(W?cYXo;psJ^1>b65JP6dwuO@rmc#69N(s=y; zc;bQnws;7i{btnahSN}#aAH7m5EzHJ0BQjNdcK{RFFh;@;j|p!73Cp94eQ0!kh-6M zo}sbKARyl_y9W(@5BPBp%ObP9aitIMmwvg6 zdUMg)XQOBf!BI4e>cxf}v$YbM$9awytZOxkcS~L)_#P+*lm$xZ3RgeA?X0qRX{=MhFc=UGHEzhaa4Ox(f zra(mdPM(he?ge1-1TePgznR@`D>JH)nPbElIxHmEgdU(KfA?hnXFjLa%mwJ(Gv#tm%&g z5U`gwB!ci#G~E7}bSWC(K7njQVarfsgbBDu#u~6BO)0~L;}y_WeIOcU#vvIz;fvqi z+3y@-EhW&E2hamRUUM2bh)D%>fZVG~^fC0fb?CIxdWYc=wDrmzGqgVsNqeu3#)2>T zf-Qi|{b{@cedj(fU|gD0Xr}x`24}?W+hy*rGG)1pj!Yz@9S1;%(Vc7b{`Auy^|YWD z^vYABo3TL~!!#7s6)h|bt)&J=a8|7xqk8-bS`Te}h+N(AcnZ^+=E`)NI^s85MnePY zmoyQ-WI(7`CxEH3k|s3UKI$o>!O^0%++DXKUGS9!T3W!!V)Sm6M9qB2vAcs#I1GRC z0@}l1vI1T-2H?1s^y#BCii`AxU;It~8J!sFR5yHzF+(%``IZ;b(F9e4rH)(cMLvhs z72p+Eix&99)nFamO1<_5Ezegv*C7GDItCzad#{eSazy98>3 zzfaE$8%r1j4E+(%ex^Hp4e0U^X8-2%q5ya8;|LNK%*ALNKK-e;!zKhe2dH@MyIyF5 zy&S+hm>nn8SFjKYL1vJy%57dc)8?Q%00Hvn6p@*54cvw`jh0~e1GPNTijF#nQJ~CG z<`BSrW#*5A(;#(WU;?TDx=PPU6yRl`i2ZG&KP{5UP;gfa#~nk{6baE+{m^1MXFP*Q zJ@*_7cHhu0t5jKg$%;EP<7Sinu>AG%NbFMOJ#(_34V0rs_!xmIQ zJe|CJJD9Gd>P_t~=sR4xl^F}fyMfi8|3-myr-%F0)Ad;@!%Sc`!|o$I`Wjd01OdmT zVdwykL5bO%6QBcFI=<3?SJ(Q-6`{@G4pV4JPmrJz4fn6>7bHfg0W&14tu8+lkA6j( z&HU=DF1nW=fM&}%iB3N0e;QnHNnM?)wBqg!@OZ$?j4gOqYoT;mI|fCZLBrS&0d3LX z0?wvClP+dG@u@D>Q|9Wd9$ldMHG7}e@49?7#ErsW@Q*n-fV-)W)WwpXdF$>r7JOiz zNh|PQYje~|9Wjwai97wv&CfC5HtITLt;3u~Yhf|utU^K6cCK&Tina*^sCRaAdA9Vy z2Sl<*LqmW1EZDh6v|~^9PHr$|X|~$wIu~9Q0C^xr*=sjt>ySq$;uILL%56PB8)?&V z+@N|w{oR6L1~k@!{rSnCgR|x^iWcj7%Q&;mBNKGzdv}Aq{u3~OmQvNW)(=vq4l#)N zARs^*;o)ZjA)M-~vKGml!GU?`8jRIeOb~0M;{06?Jx#OqOtalFj=v&mQVmd6cSXbW z;*f#XqQBkq)ZZU`NDqRC=I{W=0PUxea&9xQ69MiP#VF$Tf)+GkTU=3BC1VMMrBN`3 z*z1p}kK@Yhtm6x_U_h@lOq+eshU@FsZi}jaErHp|P}eT#9G4}o7rxoe%3uO%KlA(x zY)E79Hb@*T@X|xlo{(ml!|n%_GDov?;gDs2x%vkpFd!X^-STM}Z_suP$2hD92(cB! zp#%LKES>2neRaPWM2^fRVUO9sDRbOBY*9ct(+Q~JtXwNkGp@gQ=2}tGoIze-Zm(P~ z8gOf+Y1dgb(41W*i}R!@F;-sU1c3q`QJ{m0!?#?PGrK>0Mix^)keo7thMruYYq*mH zke@yb#87Q3A6C>$*XnfTG(ET?XuCEE{1lzh8$K~U>i_Az44fk%fJSAWwo7+T8t}zk zFvbc%Q_aQfEDv*ldfbrImvr7O0lfjuO!vOhGH~XY0hY^s8hc(lx({-*u{fO`3qJ$K z0M5us1~-EQRn}eUND9OP<4-^fOJ^8N+Ha7)6gB8KrGu@5t^mg%=ypcSd2)gtc{Htn zj>}})$$#h!z5N12vr{66JKkmn@0!V=q3Q5r+#5Rr=Uo~l1N64mSdFU(&A~C=HcH#Z z8_){Y&~S-g-+CAtI*WS({6SsgHsBw;I0jDV366jCun0+MjUr6a>m#M-??A&a0D7mu zd_=aq|BoI3XuIxyCSMrni^pCBr#IX|0-8w&FMXK?It3~`z!f*zCXgW6??m^q`ZbA% z3)|$xLRTBOJrH3k1^3dKam;nWJjj{oG)?3AmOCrjKM^d>|$#j&tKVnbwV^5qPK$eE^8SW-8cNNnf~h zSs2pka09F5z60E%eY&T5oGS|k(Qp9v>JEDdj_J(az&QGgWg{}QG7}Au>NzX|IQK7T zP2*r_!jtFH5)j`!%XQ8!)-?j5TBjRna-c?PGa{_5lmAzBtN%Rav< zOFGqfilmYth$sDf!C4g8ahgdyHA)YBz<2^CPCL5~ z<-L0g#1W7aC^0Z26^#Ksf-0s4eG4!oSeJ{Xdmv3y-1?`t7Sc!Z9nyF=L>|m6%loH7 zcnsZPNm>_~rIjWCSYy@#L|kpt?6*|teW2t!$*V?pb- zGxK+~`|OXcGGhgWOe;>Xo72-pUKZu(kdKZ~f0P3qZZAJrP=yW?(ST=xa`Kqh>*@iMw7R1cBabGvMiM%l4dAvVt#A{X`mK(|G$S?%u9u4?bZryAqdo;YnDY0m*)I$;vw+7rd~(KY16ndAi!Ok+7cZ)mDyt@U+2M8 z>e+SPv`Y_@)AF{FulBSiY|x5_VLin-;H-eJwjsle6ZleUSsqdMm3Aa68GJU~F8k{| z5i7R69?$CwV*)pR`8iwv{3(Ua#d1T{h{N>P*{s1Bs0c+#+PPodT$`4smSGRhZ(Q}& zH>?Q5kT<$Z;=k5ozi<>j`0ETa^XqKLI$mTM>NY+*qR)MO1?RFrv&6i58(4IP{nyyE z!fr#JOUTUlD#tRMhxL6J{kfppF{iIEOk-2lKxfda4}t-iUnlRdzPdd*dhVm|L3aP= zVRyw+Y(T%Tf?a{H9aa_Cf%-NUEek+PS8ea8U(|7LqND!D(f`*uy4`Y*guTANK=|lOzUDT(ltn$Q13LAy5wC)H zd9-I<+ec#t0hZqgr3XG!5~HscJM}71vXUBTYZI|}iI<7M2*`hA`(Gb;UizPYTUzkT zzZdb1pGs@_>?OVAwvT@IZ})vu51AiBW9-6~odIo}c;mR6Y5WWKrtb%$h$qV_|_P_sv z6XRvB33M17x@k)^U;0pBhVBpF0W09{__;yADVJ(HAEWaq2cG{4c*w?}?*J!eAphbm zhAhG;nk*6VHYhh-Z4{aK5R&}xc`_#P25bYL{q8&j>Mzs`eSIlJjs?gKt0{c#nE?H%D_@jh zXgsa^+2!X>!Ax29yEpgJhR5h&K+l8$3|zhs4_VBGU>z5~mIDIEm?WGaz>Ts^dbelX z&N!j3{uR|k!3@TfBmpbZ&|@6tzyI=kU-A=F!Ju`XzCO8wYY8Hx(NW%q-j_a)(C!>4 zpl+c+8)0(b7HbFYY5RuiN$n>n#O-|fHPhXt=TFf-tlqoAxzq6PKF0UXjw-y83 zcxpMCtoz^03=7JgndS!|4_OV|I$ka+G41~C-g_bG$nihHxCWSLi(xhNtvYMunne$u z%JS851S~8Yj43lp&At!Uw)|0I5pp6o8cmttfb_t12>OoXpp#NG?Y!Y#k>lu4yTl1Dc!AmM4BMfh#7MV4S|Gh7X$thG za_QwuQ-kKU>%j0&47!2&&HYA2VpI?=uX!c$a#Lg)9Tt z)^5}L=L|||;K{dsW2|Ec+;^sqKv8--RzXl(aZe2T0T7J=aDKq&cZ7>TCm3)M$@8GW z-Ivi}=4ZE0n|Sp%|T^cynZ)&lySlhN|+Lk3t+%h)@Ki(83tKi{19aC=Iieb z1#3$}BA8%;+x53i#c_lzzJc$)0Db@xaB70JJ-XAo-cX3Z;0Q430D$bU?Rm85-|zbQ zg7$4Hbe}u{Hp0ow&@4bYC`!aFG#xqv^>K3|U%tCmmLBFS^zmEIusi`DJO<{;aU1yC zukMD-lLnA&Eg&oChVeKou`OmDOBe4%Z=E*F1uZ9o0kNRw5nu572s394>|T2E7@AAp z0^@R^Z9mJ5rkeJfq#H|}R#@g-m>)chz#5h4uQQ?IK`a3#(5(7@@!nchg0Ba|r!<&* z#{=TP)%#jbL$F*CK?LY*mZf?1gQ(~Z^Aeq=UenON`BPcss_sJALnlTQ9#wKj3wpGT zuHb{*hGYBtLywikx9&p%e0$R!90u2Oq=%nSDs|LCL93`V(Z z2RN(uP*VOq!<^b+U67jfZW%S|-lsm>-({1_zm=A=is%vf;;qsK>S2UQw^C23wgP>R;|#06 z-~J2m9Q8LD2x*{V4|iIP>n=xVhfdLzNgv$=-a7~e*n%pV*DBI`7Iq* z_--#TuJTHnMLQpe_T`7c3NGIB>MyP9meUh7vwyT@P8bVuv<~w&?J17P&Hhi%J&jhC1$vzZ zDg_y+uHu21(*hmMMxWU-!YI+X)uEmY7Sv#qoX0fNt52w#j%jXA(-)Kp0%)@yPBfb* zAVDOWKY#l=qwhDc?!+Ka?!YCOCn>x>4gKbaXDh`^Es%s7G3LSdj#5I|$K z_vfjH>o~h|p&>Gx)#rEypZ7{&%hARPm>;@zQ7d}$BH zDiF9EpuLyJuV*Qy1F8ZBCeZ--Aet}IfoFCB!vTnp<{#JJuH#zz;HjXE1RwSJ8KT~j zIS~YEsObl9a{1j`S=B zO)?|%#PNWgAziW14fYL*3bk%!3)Kv0tnp1AV$a{%H5&=szR9^WSAk6TgN|(xt7umP zG?Fu@HyVr-?jNl~j=@<6(nmUJdi#lBbOCPg(T|t1?40PIg89zLs3z)w9#A@{9K=*L z*B62dHW;V?t)1w(!c_o{kcSbJqG5TWpWW@i^jeVecnEs;1jt>TfGU~1?16L)(jR{6 zkDF1DIHR2Rf6Z8x8@A{&;QaT&T$}tq`92SyMOVq&j>7Hwy&~UWxN`b{di;)vtC8wq zG3b=qgE7Z)6T#X+z|LRqxR=e#+_!;TnvXFjSTQ6K3c|)y=DmYkU%ARyp#E5bUa5L^ zBZO>bJfoMhVNC_-^b$|7Y7q4_ihxxEr~Fh-Kai`et3K9K8HgXe zkR1}u!Ryqz_)ExXA2H^EwXQRtP;6ziwA)1cu!;!2Gy$j>`vU)LwCZiqX!FkBnkQ-& zgMmdF=rB}9K;aUoL#xsby;^!r;LWh3Ng3acD$#qAAbxTS*g6P^saNNWbZ%8ao4M0V zjsf%F;byS?xyh)5s-uutBZ^r3mO+Az1o|^ZJ2UU>O2jKm~|v(@#tthTfc)_86_;1rn&I#@L8F zW{Wmp(c$ckXj^l^Odw9VmFEWF&{;tg;ATV8mqSM1yma=L*`Ds`&%XalLpxzRn6j&? zaXB{)D>IgiCEK~7qoyOu0*|+DHx?U0dFl8-gFd}LIWov8Sqpc^ycjZtq7?3dra%qwl)9@PT`%=A&BH6v zf{a~&kup~FhnxJkYq~vr_teCD&p{BhIXBex7`=?Djw*C2L{Zd^){CLh1?sbx`>kYI zv^1mA2xsLHOrg2(u*meqh=^Bkj*WX}g0w_|adN=YUd*^i;9-Uo&v@Mk1@i?}f>`Br zDcaF(aqi$cv~-?z@BXC7bH5i^c;3paDJ-O>Xpw*841|uM-8|$w$H?FAtp)D}Rh!wn5{=eIE2X@cOl2&R)GHPP3+-jtV{n2GGeg+G*}8RozZ=)Q&6UgLpO43IYcE z!KRh5lUnISdOATpg9ZYy2zuvPFk0>e2zd8aUKY^>0isBi2Ao0I@YXM4oww8nm}*%- zbH}TPlaB1pLM{Of)0O`m0rsrl0(IY=%4+m2w}2jd>8{AW^*>+pc(SY%ow>Z|8W*eob(uAY1}+q`cqE6u3J?)zviLi zpw;uUtJ4Q%6>c&WTN&nDO%Dpx8ezdxwd#z%UGKc19frl;^`YN5i z_{R-*R*WzTQg`$wjJhe1sWy=?U&?oN+2xMx!3otOKrA)cSzZsX%k~VV@80 z=yiGdS=T6r=dLU`Ygk!7tru7 zG_TGpV*v!O2o4_L@XqN_EQtc`baV>x==Bs)pG3pZ z1I}IN1^)JPU`{r)Nc0%k+3#@r0*5-ZbOI+kQ^@UBE3?5TKy~%L$=Ye?b$5a>J-Vjo z38y6PYl!RRQhUkrv#sy^Y8d*LZ!HA}ST}(94BEPuRu2WSYza7R9%VVZH()tebZ*js z^B`V2;~-Wy1bO(_GfW!oU;3Sw`}yZi*OxjCs|O84Li;a||DKbpIa#3|jLy+jhradW zGNu3{XT5vU5e+bZT>IX%eVk<@GE~=W$RFLiv0S!&(ka@Np*vMn*8lch5Mpy&z%Trm z_8I;7x)f`IKq2spr&mr(M}2RUQJluQo&e$ijt2LBa?|BIWd(4S^o@C=yTI7>cRd5% z0cy{eB{Stsqmb0Q=JvZFv4EZXwJdz|kwFGEz|#@H2-p)5{75n9CkF>A@X#>AF?PQ(* zY^SJPczoZ1Ve-|jN>PG)+(Gr7+k_y;kIla7^Mbr>H2~G53b@&IP3O& zZWKaes|U=W#cENfChKat)XRCGC=0LAMnwncY71i#1d2fkbKK8*!m3*e+Gu9Lxto}3 z(TpGU_Av^KL3zv@1odaia$goNfFt1P*9WvhqcfQetUwB4wCYeg85x4kDNr>C2#}_` zw}%N$H+gMxm)gUS81JF=j5lEU__PS%C0$+MWzrvNS@9|ovD8T?Ai4?Z(+mPZPJ%vf zcrAZ?6I4|TyG&YuiM9`NcH4IkLwoib&$52H*$e_AK*c8d_WVy$>Vib+jAm=h@;X(~ zz4fsAQohBNaRaP)^K1t2gys+8ELpyQZpPnMS^+o4v(%?g zLcLmD_sOlc5P-O?iiiwfuL-4JvqC%Gn0>=DpxJE4N#+A688Qx5N8-026Q+y zM|n*Q`VC}rRefAp#dP__F_Df(=)|KvyrMZQ^(iaAy{z_>dTS9huIKKTwRLLdqIYs# zRK`Qr;yC^A4aP&r1g*`y(P0QF0|X>?O>(xwCNSEZa|XCHxKbBqbqx;1xRZM5Zz?UX z>IcoQ3~=`G>>A^thfubGq9T@i(Nur%EI1H<;lW{%z*n{VAr5(sPm}M?LA7iv+aJQ4qZ*Uy2nrZrl76t-S!93b0 zr8lA@Bi5@uN~>gn@PSW%SDII1iw@(RNw#7)!W~rU(0~XyJ9&H2OdK4*7~=eaz-h4K z(eFH_-p{}zc+I}?2%3Emhuo%3Nu#l^PamUYkh?(Y!S`qd48f|!{#S2f0MHvcXXG_L z_}fPubg#Y8m(Ge9xdkTBra~uPy)Vc~)O*GX5XQ%X!MS9`Goc)0IoZpv(A-)rV4qYB@05BsD%5?bumT+9DKin9*1@zRJF7<(C(x?UXG3m zF|;EfuDWa84SGfId>{PW&0r^48Fh7J&N(t0rggH6LZ%m8`f1}{=F_XcWTK~!4b^e5 zfWI`mA3St|nO6W>TI&nuIb}5qtZTA|b9M&>s>_b7KU|kJg&sZ6%rUTNND#Kw7dsHf zK^#~#_wEG(qIA?qFRK!5)d2&T?r11xF#X%_?*a$Xoax|{etSph3$I&K&vr*Ag8tF_ zg%quRUFB$jlC|9SdY)Zk-g$ue-~!TzrkQrV+;|dL0RhejF3t?maOoPHRe^8fe=&5rz(k;)KN_lBSi>yu@|bJ6HYzqatGSB>X#o*aLUVR z=7AbPwIG3dS`ya->i7Kh{&*n3l@$Z%kU5Zn zgbmM9bkxKAIBCWTa1zv6CbGe4JUWV*>;cja(zSv0F3{?w&BW{~X6~wE4@X@a!~z>& z57yB=*@5O{M#Wg3<>fNyAI2g&EJQ#C&wShrD2t)z7-4?uxpcdIoIi(k7_HLD)LvfN za7Y!6zxLyv5Bl0-Ii;8Gv|&b=Idl`q-SX7|EotF>;Llt~XLoN+tmLI&)jmr4qnO~Lk&UGYmI7exmy^M0ClNu3vR7fM- z1lMzz&S}%iB*O3TWB(+Fq6qKz)}n z$9b*pynBb}z;g${QX0VCx*LpB6ZkXFHbthz&iz>Avp->SkLieR?La3vwQi-(A3`gm zFC&B%;B-sW78(I-sAKkqADUin*b+6ml+jlM>7B>__WXTl1D$gQn)6NBJkna*5pdwW z+ZRR9xlLnfN0?>7=7zNDG5$4MK+iZ$*boi&E|QJu*i z)jH##o1UO|@tGTZtyoWs>*=G{Lp0E;9%n~qb0Z*L8eNUhIeLOF$prOhd+zd}rOlB` zTXe)4$WeV@n4IE*Fpxm+EHftD!QXP9YmXDZUG^a*jK+)}fxLDU(L!BPV-tO3rW z5i|?pY9Z5LognUn3tH5L@JII)nz@RKpA!&({Lg>;KFi}V?vbg>6syQaXb-=AGsFT= zHi+8+HH$o90t^p;(_cRhR<4VS`rm6Vs~@mA#h~Vhh6BKCxi(M@>Z8K)oDYcJuipQ+ zyT1z|dVUq$0Mh~3XwkNZ%#do?@U|<_f&1F6vJN;S_b9`ILVPRBUQplxxPayu^w0qV zxmDD;(hR$;^6bN^I8_41adThl{~{;ooP&$^f**!y9KCWTMTR}t2u(Ex0)}eA81-9v z4iGOr1Lj`qzi6}ELkD!l$AH)DfNiw~)?$IvvNjS_<)HCIuyhWbEht(xuCpk*D(MH+?G%g|v*XI>JpAUZ@xzev%wRhQ zVCo+{I^b;2{T3QQDd!uIM^gYCN%PCkb3cp_pgxlOVY-cEFM&_@cv~L-$0*h*9}E#F zmd0|UbN_@oero#O>g5+07TN`w>dw|uSy{*Yp7ajtqy^^`1%YTB^?I|S>Slxq=xb*d z&p%!Na8PfCC|lEO{naJ8%`qsLO1d!AGMacQh)!0|;DiK>oRL<$K-*DtbX5`OdG;n+ z=}WPpb5sxJQa^@4 z05-CxMKHMkib_-Iv);FX)bY`#g47?}oFHf`_j3LnQ~(a+M>v%ZH!zl;mdjT!GmRV1 z{}8+)&%b=(SbH->4zB1(IryPhg8jejz4v!r<(W2YHLG{Y>XIc{&5~8EuA^%8-X*JB z-RfPkbfi$7P(46`2@C{6a6$-#9ta_8VOtp6*bW94F!r0|Ba@lTOum`<2fpjt_oID| z4N363*7w`2b=~E;cYB_@?R`}C$zpCegX-$Gb+P9ZYkiNw1>!nc;~%`tH3FT?NnQ-d zY7tpdRshAAKT_ws=KAUP#i$&KL5DRMn}`EwyWQnSAb^tS`r&#=x)jI|q5bZ2Cv-nh zVJ<`N%(Dxk45L-^Os7QP`lO z2Q#$9mMaw0+;~{mM>YrZ+_ioQ&@afQ8 zkd9W-m@mLUG*}rY)+dGp5!}oh?$(Q@6B$y5F+AG2UUe3X90rZ2ehq1kH8!Do>kRPJ zcBXbQi-D^x2Kj0~te#}tm<#hDSMDlEzwj26fB(%LELosT#0-S?F>6i6_7K` z_in8MKYk~e7l3wSChmy54!026Q?-72x6@+<)Bw*d6< zOWJlZGGhRf|9R%d86Mov*)<&Nru5Km@psL(c1u1eZjTBw zTDCPfHPWSmSRSP#2wLJW7}prxRS#ZTH0?7s3nj%#?Ya+xb8mnFIUsH7Sx{U-_7%KT zm)6+!LKEcU2X3ONxyD)2|BR~O5Gw)b@p3r+g5T_#zQJCp>;*dFpmA>I2>j}4rjzz3 zTD2_oD($}@_WIB z32*m>^dJMcUKVT6@q1#8Umg*o_OZ{0*)mn3J>W)7G0naa4UG&Fex(OqzB7cgIAlqG zHi3XF_G6w(M_|ElV}3j1aNz~*UxJ#U&Vw?}z3a~@+cm32GkqB&&-wxI8c;_%*vD`0 zElgUiE^et|;{%!ol0=zWbvguXPDbDc<@pil)u1qa3zQi)jSinkUhU*>+IX+A(7rbh ziXZ}Fc10h?cIhflG|cJ%QcgSoz6@HcG=k5v?^J*?y)rsOPQM6^J~9;;Q;nr9h*}65 z6)hj)04TW7Qzab3!=$s?uKh_RG++^QP|Ye>Ox(xG0c@m`GhiL?84Izh z_kw0;A?|$jnCy6{U!?DeAC>v5e1*k^`*32MZzjl;HES3F zj5Sa{xIFgJ!;B_xByq{yA7G9G?h8TNhK6|S3fRR9<%(JRH)g2Xhr(S@rY}4T?#NYn zd#Gcf11Z<;5~Z%a-@xQJ2y*wo_$u?#Zl0k94$Pc*S2XbJZ~W^EPs)gHWg-MGgffkQ zNvAdzf`HQoc#@~r*9y;+OUf8dB&DDK&aO|@mreJj92Vn7SV1mx( zQx8Ks`@RgbhxRXZW;4^*zI<9t;DxMzOHh2%up=*X?G5Hx;#c?F9}4MOm|$a_GSJrR zZ{dxh4JeP*0BEl#=S0on#}%=WtZ6^{VjV27Si)|+2hLiXE@a6vM>x}@;gT?JkM!WK zf-Zp-`9QmiRQ9l(EDoeCU%MmCd@ECO>nn(? zrZDh3RBew!saykLv^_W-AGW^5+zux>OsOzjqrtWIBu^b{Yj2Br81YE7+?SycrPw5RoE<|9m1LN8&7F+6xpv~__)$-t&?VM&1 zh|`!=PJtT#$vdt_NFY`O9R%!D)=tNkloE_NRhAI2LrED`wutP|-!SPmNuOqkIS4}K z`4ZF>W-g%phaG#>1gN*~Uw`s=jcfpSYEfP+a>eN4-LiqPZK2QEQwBQO?K(j`sW}|n z{_@5XFbs5RkGA0{i~~TyX#q7{k6`@7<7zSV56=k<4}!&cwOaH1087(NRPb=o3RVoK z?|{lWU@`9%bLmPk6m_CivGs^pgJ|79%H-2QpR2srZIzD9TMrq}SVHzP)tQyqAfwwd zIFMY{>&vZ;pbJlejd2%dv7lzdp9|sME4opX%5>F(+YB&VI|`Pp{k%ptGjo_Hz^8}7 zB0+^R<|e^Tl|<|2 zT@iih-XR{{ytd(+h79d6hr)2&P z(h9mQAcWy<@?l*8iP>DS0Tjmu26(H)=FR|T!{P(BLIL&MF)%!Ns&JWK1sgB~2e`1U zSCp*wn7;;Aw$ls1H0b1=qJ!H;crdsswYxY9!CWzWmC5lrkgS4|>Mokc$y^9N2 zhrPOw9NUo@YaO64TqG|yWv zt$q5Dk8II$l&bn~eXtxQT>F_UZ!F*LXb;~(&1WiDXMg=I50i_TznobtxATVPXz|$T zpYY}oaT~8`k7LP%jqOlbaXW7?SL>9?*2f#5;8mqx56Rz%{7{m-=N*Sl8~1+uH5V!FfY3yAenHbOeT67-RZKV|*V6+BfBNMdUMq23y=~WTI zJJ)SRnKxXm_wJChMchW9ZjN(80nU(c8;1|Y=yC2f#@%MLq$;$p#n`x=Ln|fBn5qq^ z|288Xg<~{tRYC#&-o5P>Phju%3hie`aeLZAD*xVX=ax#rky+xYZn-Lc2_?$6=p)#% zK^pd`9Mksgk1e~UVSDf~?luqi;GHJ5lI{A#N5HMzRFwZ_Bzcqkz(+&F8F&~%(>AJS6 z9?q6v=@9Dy2=!oCnV|0|N$z;IAEYA^Ziy7q3EZW-hG_w&PMif&Vv)W@Q`RiMu;yT2kWBa1We&f<}#rD3DPcpLe-H0EeFK2FXzf1XvIKZ1evCe zfsQrpTgf^3!*?k~LmWzSEfbstjr2#A<4p`;DMmz)ShN?p2wNK|p1{wUaSXcBAjQ{8e+p8<{cW zBx^wDzr%*Xb8}+3Am&r&AKz3friIr_QZa$5{X5k@D8OG{{ZxM-A)Kg-Ns}Qa7Mr?s7u`~ z#4-aB$n;`yCYZf0)pZR9&NcF{uRjX`REjX?HM=oVvtE)7-Nld#qVxDTkXIlW^Ha?z z7-~89F?+MN8@=GxH4ZM=$rbBJ8c~gR*<1S)PgN2r*nw97w*G7jYToLwCZTa;HI_h*u#oWk*2yehuXifa%tUL6y6oScP-#S>}de^T7&} z4PbIJ1er1vzVz4ESqnfn4>E#QIKgp;EGLL4(DVn-r6uq{)nopjd*8wU2;{A$52U@) z{K^q%0iYU{P*8Pr>Y8J~c2*jbF_yXDEn65XLE3i6tdFfj_AQCdAaLP4;_QT8HI<$;GJO2 z*-QfMhwx)i1b?~nQ7BW)I{|MCEA-Q)cfo2wj2>VuB1nYrnHANBFyw_SZL4Ji{j0j< zE~qRJ9ZW1n3kVSr2w}_lr@2K}Io+Q3)`>6?1; zcn037!}sdOsbDYY>z0{k?u~QYQ&v`r(O69M9teL>bV{UeG^cgq88Cots3C{}U>%Ku z0QMpJ9+)3yMptWJF=TpF_JK|Np}T5RoK$*tH+c8ZXGe;I?Rh;NH~C+;=B$Nmvc?3> zx+Y3LMD-ysvqr?3jH{bg!;tK!@Fu0WT{k=B*2A4TCoF6~+Dq&^+MsdK z0w#>7_4Oxog9>M{T|hi+bk!UKHV;Zsr(-$jh-ltxiexCvV~1E|xgh&_b}9h_%xlpV z9$VjF;vm~RN?k90UsPUwH!r%-%KQfG-C5vs?8QK0=}^RuIeL4pdroC_)EtIf6)=0| z&P?zAjg~IxWr-O<6_Dyg(0Hb=V>8qx;b*?N7rqpc$s z$SwrbEjkXCi3)!DyI{#)J!0T4%vgZV8(slBDB~Rv9c(!Ov(P9ic#&&RS;KB#LD>em z;B`hJWxclI#yIBQH z>h%55%p0DNMFH^qouYzsmKW~@v!|nZCMES%>{bAj%eft(hEH0p`qMwe?;mWxRRL066 zAA;fF;uzh_JflZ6XS(%no3`B;_^35N1-M#|&8NU4g2d9`FA|((;$O+-o1+y2=gk+u z*UZhLbJDE9OHa^wqo$b$b4xuaUZozlr-A^_ks|PXp4bRd2W`E1B7_U(>VT1?=2h++ z4f0V{C|m4BTw%V&fF^fziACqhri>WZjg9lO(9=Nn^Ty!JGe6zBN6)v9z&P#!mUaE; z$G@(F(qKIv4AFe~@hk`(3~nN*yprqtQu_lJCMyqd1gV{={?E_fe;lI9&U2p%p7ym= zwmY@|uuI;FZLSr}n@(`h<5 z8=oqDn8OMK@P#Yg(LJ;2>UsxGJ=d98s_*4w&o!~gMwvL3vcYESw~`o!6?a{H^WUF- z8S1~i_^xa>ieS0%PU;3SM1Ot<9nTPQrPO2SJstn=KmFh}4A`Lo=4`U4g??PvoFNeW z`q3uspP9aT^pP1183@)OC;ZXdU~@w}i1z_=q_@gwDFM^DvN}rCUDe#YN>J-rV=qT%>eo~9K?ky6Erq?Kvlic@G5pM zsI%9fAx|}V$cwY`26ciOwmyQ|VgPU(P!eBRCfa%w9UliWg7`$?0%9Izg4jg0&j4&4 zz*X(PSE2aGIHr~7?E)={VR7yB(S`_KxhJvHFAA~&G;P_}ar(%~*Hu>j+fV-4s%RpP zna4doWlGF)3g?gAcr@a|o6y>rN%W|@bzw*^Bu~R#06Kdb@<5u+)6a<{hyz|wP4Jtg<&IB ztjV_OsUls*IuEdGyKxsr2LhM?K%$YJ1_D5(^DGw~wzGj*v%FIYx}Dmi)nr(>*)=M` zp|b<>1i+YD7pKGJ>Iijoooh@}dx_O?LEjnyO)V({s_OZ!vYi2sUIeSj; zud)DIn)jKz#V*{F$G539{rf`?V}vh9c~oV$z9;KV^=78JDi*toZ-mJ%oRD2MFSQnR z;W@DxAaX$6TQ48jg@M!W9;JTaJvbUik z%oAGne}CukANVwKrlu=0m)NyfhT|X|PRvs_j{{tJVz+W0MgKNdZDLM86&kR_klX~eW5t5?dL6HE|6zf0yPbCPSH2C`&2(vb6Ba|tV1kF9d3a*fO4Td9ni0qtGYEp*Ov^54yad4R12ACBn3REq z@;u0F5D>et2b|q}w-{O=3_RCD+V%S(R8(bShy@SL?*U^nrL<)psFE$Q3q~k;B{yDC zFCgf89O67`^J*$Y4yf=+7Jja|HqRD(5K}1i%vX z!BZ}(6uU7hW!C)p=zGd(E?hmZZUBql{2{ns1uLRsJp>$}%eWYoU(W5!YttCo#Ye#C zy02rL{Q)or_w7~pD~wbWpcz2Bg50zp06^8girN&u5k5bt0RHvCM==2C=yK5K?}9CF zYXWy|IS9TmYh((hEQ8UFOsoL<_3Eo&Bd4C@#dHjdwkRU7rzZ8(~to1N=E2@2y0Mek@BDF60`x@zt_+7 zK+C!MbicZ(o_K1V$!_*v_@%P9Dfjt-{DT+w%++v5chD3lcR+hTJ2cKFopsGD0%k>k z-_%t)dB4(;vH4Tzl}s9Df)1X#Rz?A85h(+G+RrIqWt>`ND|iqn_ok!uO||sM*Ku9@ zj1{0}$Or_r#?be^>^eYqxvvp|0bjfROR+J0Up?}s44gTIaL+FAPNxPt-uqb8fPvGk zcb^Dh(<`wDqoabSz5 znM#0v_P_n~$EwURG@U=BF=wWSF{#^rU>`q{-vU*z%!9yTQ)w~f_BgGvs4RKouXCEI z>ePM-v3KWUE-%eDkxPh_R}Vlh3vpG19)|JOzrnD62Vz z&+;mXtD$E9^~f(7PIs4#bsJUFjNdlxW~l!V8v1-eb(CEQPZLQ+iD?3i&1H^DG0&{^6c&AUJ=`4|Md6+P-1x9ImeDF?Ma~@ z1Hi%L05N2W0FO?{NMc^t?=^@mmz|CI0x(b3*maVx9|h0e&y)jtxW-+^?M7x3FcDB6 zqTSoI>te2Sz^0*GMF#1{E0oK}Y zW#Yt^Z1(L~JXkQnn&KY&V-f;FIA#R=OA;!zf1=85U202)qYa{hu=KE#?#UU zFlT+&c&o`~t_t$b{TzxPeIK;6&deN%vk!t`l%guirXYX`->tWuV8HefIpL%IYl!`F zzcYrhpdpYz&EFRz)=Psa6IyDjU|^;ufbae)(4K$;+*2-2UjGzu8X}wVUW}6+* z>0#|R)ibhvqcnI$v`L8b@A-lIhVv=MkF#QRXESwKGM?JcvZ^@)yZL$#uGbtZQB0Dq znDA7Hk(QDL-UwDFREj}o?Z?wt-p-}6C*Fdn3*Vo({JujJug^u#b8ogl!RdGZ_{J;R zOL;$gOE+?g+xq_PF&YM9CAkI z9|TK`vj9(E83Fz(-BSjFn0P>Qk2knwQa-pkEuF6=+-exKxa;2o*TE&Ih@L#cvoQ>S zXK|Xtffmt)$8NXnDulZA<(?nk7lqM{pEB95ACY~ywg^#$hvzZ}7(?eGUW!v<=>nm< zU1EG8nT`UEfNRfB@H~L2>m;sWsM1{ZT^KSsK^7;2y{&>F<7@=o_K?i;Tp2Uf6Is&~Cg92JG>s@BS8e5grd1oHM}_&;MG6Y$iZziwk&dj%qY%7_e_tf*MT@z=&7A zDUns)FdYFV=cOZq;HhE&4^#^lgF&}L@uh20Ja0S&ad0PCVE5T0OgA?FO|>+rI#|x^ zQqv%Y$ss+^kAVbsc7u6!vTyx&?SV~z()b&?o6FZ4L`|#GkM?_Dmrv_F4dy!Jlteu6Y^2|fgtA0igl{@{5QCQt=B^^Wi52mC{C zsv_BX(}$@h-nRgigVky%#;KUC&wnyjSHgf7X6piiq*XwRUNVRYoVy0mVJyZJKQKTw zNuyE^sBBSv+}mFjE?ws!M=rMF zw26&bFIWgOrB{_-h4vn|PT4e}O7!aMoBuqgCdU>fIQNRKLiy->={q0}o_fvPdi>|$O1U4nMDJ%I*J4jtZL&SXEe9%gErYY8f(-y z^Kf(2X0Kz2-;98>c`}0p&f4FfXur{#wT%cYX$@S=$-^dQ_2>uBG|FC;BEXXdl2ldE zZ+_gaqROIV#N0ure+34dXDBMQ7v)S%LY>%QcS{T5yl#Q=*H#PBx3WAvYq|BHk8Tv@ zcTe}(C2qb>=P~79o(%y!?paYDhEPewoRf#+4&rnV_J-YtKL=m@*Ng6A#pb zTYv-yF2AhnrMd=X>(9aC#K=s!1vVI~dvb^QCI&+L>J>;jbPtFfL#;0hst#67BYxbG z#tB!6*{ihr{I6QqF8yY}*s1Tx5JZ9uZ2G=pw)pgA?FYITsgtfiO_z-mBEWk@8}C2$ zTmm!c78uRP%V`tZhkessh;CJWXF8iBpR@OF-Fnf_qHG(4h}#avCF!6Of>^A;dU5*T z{;0I;-z}1H5Ckvb+8Pkx7450Lf{h(xkr}|*YrmRqhhmP(^WWR3LWW+Dn${#!T&Hgr7bUIXJO4+y#RUMnJ&J0lOV} zioG&m^rz1~ree9|o1a&-3#PVduJB);d`tEMUk3f#i$4dC^#|L$4@^yZIDU7z=g6Vp zQX_2UU@+EHDvm%;;#l9|<6l6v-g@UA^o{&j-XK_9m+xWG08xjydL65}rYl@u2nfHO z)@{kU#%c4GJgiPruW!k#ibUDVP~{IZwj`Wg!V=V0Japq%TPIC&?PihytIR7dDwbFE z0rd`m3A{y&{O(xjnZvqOC8R#yfaa%GFROJs%U9udi>9iOyv*SVL z;Em(VPqu0zZzqd-*L-mduf&oN1VeDjwV<6KeA)B@y%sl~2VU)?eK*PwRLTnw_&)iy`Xr+Pc(@KFsnnY>72irj zFyjY=M1zHew#cYV1U*}h!grc+w5cd6fZ(!$P>_+! z28Mi8O+dE=nMN3_ve~Tu_tS6R4Otb#Z29O(*JcPonF{~MKhjNfN*u@^R3cU;*JRtp zqY!yC_g;A8+?`8M>Nn<+Ap{I|jRly^@t00L7ieT2r+_lTuf5EcapwN0R;wLQ{uaiu zfr}b3xw|5P4FJe9f^mnGMdmi}R$)5vjP_%^N!Sj^zTorrV4ppavGwJN_4*bl3{QTeS{B_8u%`fQ z5-F=-Dj%UOf^tFOAfPL+gf%M&g5b>o2r;G^>BK9)O61Qb1t@?mz{|i7LMyi#2LQmu!R;5aFFm!++V)O8mVFC-YO?5t~yo{g^ zB86u>xcMty%T>(QBCg#J=_R`Gd>#-ntbMAT?QA`H2ZnC7F?1%gPzUo4ur);pqv0vB z9^i}tPN)eK2-*(<7AZN2-fpmODr1=2hEKf<0gQpBMP+XAm~RAq;cL?XZrHqc>$8;X zdFE}AqbzkDGdGQNvnamuQ(3wK;N6gE8b8h;aA~n1fI$FMLPSqM?_pNG@d-*5V6Wdub+D{6k?S{Ay^bq zd*hXKXiOz@@Qw>`Hq1Oe1e${x4xFHC)$SYW=`?PuRc*>UR4?)>KxZ$!Yz8TqF~DjU51BU{3wU z&kDj=76hJRYF~BHA04CD0UqMG0v15G0K++TewWyrwfC-z4V)JQW_g**Z)+hFOHd9Z z$F^r*8DO-ck83x>M5u8+OKlMzPy{_p=3Y=Ph;Pp3*BT)vlbMmc7?1(t$#~u#R-KI%m)rvaP98b=DraN^;u#9M z4d7Q^2h;J{k<`;1)z8xb3!pf*&VuV(3EGg1p}?DU#hlRI@}HpF;}})#|B+Zag00ua z$*4g4?F>Cr8n5pnic!Sbw?sxnu-UfxgKd3nt*=)wMh4m5Q@#K5*PxoNB=#}wRn<4$ zyRl(l2UrUacbDD3RUd>l@WDe}V&>Vte7oNXq7hWDJs&}7b!d-PuEr>b^}h!MF#WtY zp8?N{4FC^igtI`v3THnxH9xUm#z7q;Q0QO(39*3pTGWs-Z+fW;1aTdH z8^#vXuj=|R${((va|LU3N8vV4aBc&*m>N3ULdJBTt*G=#Ivdu8Ez>_G)D*^OUI%EK zXeosFpBH{}hZxvL)S1=t(oAXQE&FV~)92iN?eQ3lG81CEcnc6PygIgtwk{&FDMa5Q zN`*5dlj6#==353o`&nKOXJF}2w5@{O0N^D&L|qfX4g@SOcrddW9e|hE1L8he@?$1w z>;_jJ0~2JW4d^-)HU)RC8lWoxHB)cphf0_({+9b`f0`ouw2J_rWvT$JT#F8SURB4a zY1^KbplPnWuL>-~TFo3fTRY<@h=%y+Nkh-fNP#ae4fKN!fztdLM<%}xPO&Yi*q7O5 z&k8mNyN&kZHOQsxTsHeJ51X#*#WVG1GES+Np1<+{WEed<{o%84#8(ieh2 z2!rQe7oUfTON<3&UVb{4A<|Lt!EaMa zG3wSWoKrTat_5tFj@}zvl3@ zIb-kkZd-_E?w>NSVn~DJ>TB}A4szl;STBN|x~3~5jBXKAc^EV)c0R)~*s8d_4Idrj zY5(=P_rO1Y9ZUyr8#;V+t%7smGUHeZ#iL-XYhcf$T0K*at+mvk{bIb>r>hC0?#4c@ zzMx&@gliB+=O1oxZ-fN)Xg`P%y_$>P%ku!dcAcog8ULQlnD|`C+PEz6y}_JJa@05{ z-P{ZoA2F!?qlN_?02oCuw7h?04*%%KX8-IR5GFUr_F#DW zxz2WvPu_&atDX5}V==_M>xVyqFb7=_KxZ>gbVNIg+66HyD!t=u+pw}|jmK?Olnhr67fFje)B0acFts8b;3+OBZ)80}fW8NEBZ9w0iNzNP?x_U$u9;E%R|5kE0 zCAO-B9-BM=SS{zx!mc)T`DHnhK?MRt&=<4eUEslD5FRh$g1Q<;OfGkp4$!2QCnKJz zjA;ODV`TsVQ-O@nwI9e3e1sXmRZMm<9|l3DMuEX>vUUVC;DCs&->D_cpMPaTOyJp| zDJHRiSLPXcHm(L>?7hEw@g4}3CD7CpGk0a1AK+#is+`VDnTtEAPwX- zxXg+)N5>9~EO;VzLmmm_CiM7v8bk;G_$`)>?Sx7&&bEtsF{s(!8XRD51IsRI;p!gX z5F3|O3cE@J*-1F3Iwk7jZn}1w5!b#K7jg(R8s*yx;go6*?qZz1b}T&zn&1nzxjB$; zGS}dpUgH!0u1@mP-)5=Q)HQFj7E zaSPVE#V{NJdGfJdeREs6isKIEfjRcBswL@09?L$mM=Ymbh2*qZHhE%OVs>)B$Z9bx zPSTycNo(J0B*u9N_RlpLDxtj(Zww~J5=ik836B4V4uG&}tf z7*`pj1+oe5LKzZkXKB(`f zwpob5w!JY3+Lw2@gKs_rR!hGD2esdphz{O<{ApdUFnLLA6A%TU{&9MlsRXq3d98B; zz^11C+-NN{3Th0<7Q~nfXiYcTPOL!IF{1^U>Ou=Em)|eY`bUs#U?eU)BD!gJ8H2N- z>;W1e1gp5lrgZQj2PDIl3v`nVuJ@6fQ~k-}-;y zhI$vCr@6I@w+0>B*V8(c_(a<}!ZsGj-lS&0eCI>~*U9>Lh=jI}?}1=f7y*6o{CK|h zp$&RVj}aW`5)mL`FFj=rMwRVkp2Y#IgYrRpt4vj342Qu6l*c*g7*2~7-K&MU2xwni zFvhixi-Gq9CS)56)LU%4V&@;6f8r`;6W7ytJaV+IX&Um-gkkPY@rG#?UQ4& z8>0?%o%=Z_oC`9FrNy~|X9Jq}Od;6(342W^(=ZUU!&I?O82a(#7hkz@AA~uq$~Y(X zkGU@T|Ln0?Z~CN1RnBr+x|$Y@tJ+_n)z$ZP+OP{WGU){SMa<7T%tb-9;S3f*wBHnl z4OU)7v?X{%Vnxr%qfq7t9Oo*TueV@H#?BDTQo0MM+cB8&*CXoj1Y_-?Gxz%Um?S_K z$d8Q=@C1c?_?Qd~B6QBK+rBTKXI0LWO%pX2c}{<6Usp`;_*VpE*XTg551! zZ}$E0Ay}qWCe)|M=3?HP?6qHBG_zuXgPzCHK`@(iweghI`ERzSS>2o?k`#TGkW;&fJUp8G6k+}`wRGu%V3>A^c zMg0TFp6p(stbnzJ#eZ~!OgGg2)REL(89|ZX6!%+_A+Z}SlM4xt|27Bhvhb6=Reeb= z{C$oCDwPk+uUD+4(LX$N*UlpC7h&@__BPxuSNawQEYoM**qwLWZ?XSv?}F1>)6xEn zr;F!548DcCZ^D+BaYXB|&$l@Qdjd}Fyk!A>-&`j*=K%JL*vSJ_2}AofhZRX;mQ80> zu@U?O^x~}6yNfQp3E9mXwz@Zdn=6G)v-6oFj-EWY56t@8oWRJY&cfr}>i?Gj=O<{v z#p!K8HDTYPh_GM!R)AIGx$UI-M+X$Dg6rmOV%Pt=>hF~H|GMh$I0xBdmTLd&s+-(# zvvY19{a?Dup6yuu_~igp?aKxr1$;>6Lz)4^XKM8W31|~j`~buw_JFj{;gFf2+6^!r zv3!7JOXjY8_~Gr`D}~QS>h~MafV{X#UJT%e6ZPS0D4yrK>3lmT*I)`r2Rr=?m^t`h z%TE)6n+}6%6105wdEQF|eB=X8J~+u+JC=X7Al`+r_?5> z3>+62mBY^*Rp(v<7i_+O*EK@T191+XHGF;s@D{-)(ZDyHbyZi^c#k!JI5kh*tSjm~ zf4u({hCtoURrL7T)0K}C>c&kks{OSW)hs0U=s%}U;4%DAqhoCs)S>+WbdCIlkiJRH zlT+g9`5+<4)wMm7^_&n8D54*NC7@%2;S@MP%!yhDvGuN=jK93}x>$`6^ep3DFnj3% zh}8CwQ6ESy@?*7I2y}t`0|)eA(1pVcAwS}kgY@tGgS}Ib*|*Fcqei%O1iDTo2XJ<7%t+?^_u1oI0X%-A{iXBds@F zykC_!=l$W2PQ1OyX)l1-N(94~Ug(LZ&)tI5bWCwuiD$y^tON_1JpQ9fe<&BWe%IrI z0ox5g&jA&@8Yy;-Y24EZ21Ia4fGda}w^HL|9Lx+~0kIF6@*V>tNX^`2PlFMt_N8eZ zqaz3!!_@)e1dJf& zMte0FC)@-2TX2~@kmQ?!txV0OKR2)wRk4r!zV?mse8rB@RR3^qMY4I=~|yWvDAuSs_c7(grM z)%l@!gA#b83YF_RLrOY-3~Z2W zm8UZYt-B&~7zOQbDnx=a8}^m~QuVJBj0!_lR3 zUviRaB|R!y*k}aBZGh2n{dr?~|8@66KQ4q0xNd0gy;TQ^jpzSc$d(Q9gXiuM&3@~j zuv5SOB_DJfedX9A`+~DlVrpY#Y>K#A3isP>T047yCvTrXmI&dy)S59;ed5Z)bX~Yg z{0Qjf+rYl~HJD&G1W)~c-n%<#ouw>2!dU&O>JdCs5T_JrT-N^b!4J9}W638jz*5A5 z_FH=2oi&XzE-Y5@J}BcBo~Q)G6ml0E9yp=ogN8u|SdoGb#zhd7C(@K%AzDLGb*T0i z(IUb^ggI|5lToM$CY`{$4U*@gBT9YQOz1Md;ijs#iDGB~FWsItpuOo`EW&^P{trZ% z^ex(9W4;CYN$soGe4%+n%+Xb>FpEIdBh{i{x&>YOnE9stDu5N_sr`$P<)TUvdVo2T z%Ra#yLHU>W1wokBHC2QmKyR~z0M13Ay&P$GXr1!1_6983G8_vT)DKo`1XHnuMyKot z32J%2_XIJxswYG)U-@AJhRs}TEy#D$W22^0l|~#4Ky-#kygy!)OcCs50P|weL2GY4 zEIu1H_OF9&ZB7OIt6x5h(GF0`_3wfkL0kVS2~|zblJ}SRg>i6!3m00)F36==g{b_H zja(TZpGVdkO?fGHotGAGW7|*H4Wjt`>4z_cF{=UVIhH$+#uRtgKE-44!GHS6OMiIv zUD;&L$3|B%+!i}L%}i-#%#wK|X9@(W2IT{Ne?}BHx5*Z8rZ7u536#J|tDO%rzm?2r`10 z9wmKY;DU7Zs34=o1ADWyFDD9Pb%dnv74wwYi-z(T2tmtEU4^0&gq=3MA~VH&6ZC?} zjKT@XuYT~XGsJ~^z*HN$^cr*0&D*9Kz8%d~he?#FsufY!F0HPDj zVUWQt)qgJDr*Djh#$0M?Z> z468K=xJ<_GpS@iSOk^XdXJC*Axuw+fZ-d-?1T0L=#v=;@HW6!EdMaTT{>w6xnFE9@ za?OIAlEg)5eKJZ5+(@eV{Af^@4C%*`WPkAZ!CYoA8{9aPami5g$s>zuf*9+VVcZfx zOhcBu>Os&1bWWS7QM!?aU7W|(_q5S9F_vIArP*8>EINFlWyKeP^BRl_oeyX)F&*G- z#M={Z78av$?_m05Wk^P2Obhy)RXweYy(|;D*p0h>a1ZN%wRGWdZd2ei6bn!{m)Q!U zKhY6XGI@kEUoD}@2yW)ya+(=1FQ}= zQ26?A2L!Vjh*eHy;s7NR+Ks}IUQk&T0+s9P0WlLwzc?J`Ht5al1P(Tc-8rK=w>f5(Ismm)jdm)Q_42DEZe}jIWb9w_PSM9K6#+0+F>7t z!;WAFK`X1JAH2s31D0)~f*=-HX~2iCh*WFj0t75*wt@lBwmCHkyOO9#j9C;wf39lD z(8sjD(-b>ne~{&pCu4Bc3%JD`As8@f9%76T@$vV0(F%7NO)NLC*Y9H&0$MqhYmXRZ z1WqRbDl?%q+g^HKx!quqlEF%2d3w!paoUSx;vD^|11jk77Lc0d1XS+p%Ulgq7k7nj z4tkAQN;?Cm$_d)X?=YVjV1SMO!joVM?APjC$d+TWHzVe+d;hO*VuajB&t#`#2^el?8kgMOJ<2K)3LV+Mr4 z=j0<$s{Px*S#kpUIFnrlSoQ4P{E?{OE!zK{)rCS$P`S0BEf9w21ZI#9wl=N|@NsZ) zTI@RU_(qMs6^cOy!|mfx3Q1jHs;kI0;Y@UZTb)~6`LI^R-39$;?}9>OvZS_I&_TLTdM+6YL^&+VN4m= zXGg$*L{N$c7p?tu9}jf_0W$$$Zk*WJ?{EUF4ogr088A>)B8u=3E>LSaS6{gDoa?Zq zujBdm&372vd@Umg24-sGhWqB-8D4$cO4z?07!e?3f(gzRVL5CNlklsL7;_Jp8=z+= zh+#NjLe&qAjG9f0Rm#S&Yz>NJ@ZuWROi_~!!h6jJ`N1LQkLm`V38I-^sB9A1?PAMA)2BgzHCDo1NX+G}(CN z9%tC{QU-X2I#8%j*F1G|Y<+9{Xva9rGyn4V9XuPr64PGp$2A=Ur85SJMLR;n811j4 zb8Xh(IJl3i2XeEkdKoYdRqAN7VKd+g1%G_&zO%c!q!-D(tf#SD*da^bdg<;B9_dgF&2LiO$y?;(at!gW&b6gjY=7Xz2G6;Y_ z>vzzratF_wDK?$+Isy;6ew{A#9-Z6H;FW_sK};Bd{Sai>@D3TVz0^3Bf;XLb4xC=x z3j!L_+(a=IOtBbrUL;x%SIyuZe+rB}Lg3%bLo*}75}ho>!sMJ$=j$sIThh7d`t^hE;E9(qO78ke-0!r*x4M|06#Wm_C8n$a3 zn|P&d+Y%zZs_0qf8^Z=WvW)#>PBXhujSgtrX`qI&T;Q_o{Jtzzm1EL?@K~X&39MQ~yTi~4A zw9e`QxSDbvTu0&s0PPq1yaIzBm(vA4>(vJiY;f&BR8_DjTw`0@xyPD$DqgHmdAhgR zjdc%9HGvsGR-nlRuntf}Vc@)1iEHX1DJWMqJpofH22(&fp6Dn$1G$Vs94pJ z8&BQr&eg0IXz#rVWr#;WI>w+`Of$H29AK-|0CqTl{wrK=IQOs&!E3J{P8yhzVH!ks zkPrCI944Z`J^(VojnfDf3&W(Q3r!^k1J9&?6=;8Vht-@?b_1V z4*@Kkc(=5!Q1n~~^>h$#WM*?_OQT?1GO)@xbG1OjB$%7%9$T+H$&)eoRN^Ef*#b&C z^Gt@=H<+>jSJHRkHC|6}^`+16gcf5V!;+EVc8lRyuDy0-juGQ*dGKlx*n#QbR<^o) z&>pg3o___DWE_` z8KQkrDvWe}`dp=K=32J?gS&NXcCwe{wJh6ozGrHsvTZ`82drt!=Z-PD`?HMQ5IaBy z%E9fIUZp`0t82@~cIeybe$F3Y0&yAptl4V>3~$(23-4*hTwl*Wy!2hM$2uleh15bH zIy6q-3YacI6ZCTqr;<{`I6Be+I@le!SOI4Vm;z;*smcqL(Tn;aBdhD(A3WvJ#Z=n7 z11#;cr)1->2$egY$!u&>2f(y<@HjzXo+q-E+b?Oah5ho`ehdm}VN#}n=0WuXO$nTm zFNi5Rr6N}k9cTk_H(-GG1VOC#@-QbmS-27uup3IDzRA|K;65hYE%57YgoP(3UDzxYk4Bb1=AD0=(&cf<}p8s#pE ziIpVq)FQTepi~5S8dDC~&HWjp)r6VMb5Anff;9WSwPvVIAZJistUAYrY(M$xc3Ug$ zn`Z39FwGSRW*~w@oq*s(X1Zi7e2HP!k-;VDU@@&T=0(cPnHmNMR0FY>9qAwME+$sG z{Tle@yI`yd27qg-1OZm`jb|9W3~=r_43j|Btqk^cK(>k=!0H4J?wQ&pn(zbWgFFnF zBLF=gDJuA6{>G4zmBnIo<}uEW0p;2G)&Z-o804`igF%8Nh%%7MLEWz1$~?yKMa2V%R9lfUzThCD4UIu)u1gn zpt2d-?(L)PY$B>_$YwRajuaWHy{o=_3FB@Huymj9F4hgHVRR+BjyEyD#}r3qC(8$@ z2eEOetXyIP4bpzW2F1|4sD0Ak9nGs9@MZFGl^F?OmLQ-sjm?CvY8|-xe0U4p4P1TG z$o0&D&fEpoQZWJ!T>g0l|r&5eacsqLI{Qo z$U1qa_S;IR1|}Z+jP}i3Tz|H&_U{191^h-198e)0?&i|iQEG4853Vwb>*)poi=b-8 z3Sa_rMLav;!{)3beUO`h)gtY?EbU~(b7z87%v_WimRitM+Gt~UOn=IUuhW?=>Uw<8z%ggpby9PH$sen>_Ix>n0P?u+9BgqDU@=$1gHnO zgP6UHe?$hDj_Xg9_V~oE=wt0Y$Q%05Ka{6E`Gjsb_oDVn+y8v|F)?jDqUt+vQpH@2 zfDa{X5Oh-!C#_}$ja4*u;bCxJ-kP@Oft|h=%<=4-;L1VJbf~Osg`qJm^S>Ut=T&IY z^olv`@V~z*DmAdvs~>-#?aQiOc4I-#42dp0H_N%kd!m~+S#t!H@UWZwrA~oANRSlae zTg*|Fbks$!JPJJ)lnW9x8%1Wf zwvI!KTR&7(e(f!Nv%PDMV=6|xl0bWYg<9Cd{W)oXp;5EU6U*hpJ4_F93+(&=oxPr} zHR+F^jZPOM_&Cc<)sYyvXBcZ*2Ok)$RuW`vu`*O2s8$Wj|GaVdEf219uTi!O->A6A z<0DVLvtb+c*Jpnj?W5McGz~LTHWPH^pTU6GB<-a{m+x+7lzm3T3eG4FT)RWYfxKDf zmeSQtV!8UH_Itgc3qM;iz@h~nGM$;qGzJ3K)mnKy-j?IV4&I%>eOf>~CEFGEVU8W3 ze|0b$nMg5Qb?BJtDO?B76$b)FL#pYZ_)nfP`#aCv16=9l$7i#d6aY^Suz3c8CS_O+ zvBVfufr_>7C*R!b>{g8du(n;Rt{~+JRCr0?BVJuU`YrPQxn3@{Y-C)vM)P@*=2j-?pT%_ zN3VU4B3rsmS!K!1fZslEZ&+7Xj&igYLAzLiK%)qzW3Q-NxmiLbCu=#wU~EOsh$+`S z(UGy4r>8$aD=3DxEpPI{3*eSzcP>}}`vq$vG9c7K4y~=nPt40)@3f~L@8ukS>-H&^PU+eG}IMy7m*7 zCB{hAIJEGa51hK2HA!8>QWy_0bNnrERcO3!Hc4;8*?UB>hhUxqR-h$N>DqKUi(Pw= zwi;#LvPG}*MqbCrWwmpLZv(bFpz3Nw`H#ce0}9dJA`FF@AXv5QfKmzi^dK1H#L0T7 z>s-KqEn2plX6RjYU=?Fyt{0y>^B7c~>co%07D9$@JP5o3v3+g*H66TC0p6VY|M#97c;0p!MvF$EE8a^mvicq2Gf>mH|W--2nI{- z)*On9BezQg0jx*n3|9-K7Dq!-MbVtj!5E~|4?;x^TIyUcmx-O6CI<*KD9JQ!W5TQ( zr6bk$>M_j~^pfE7Hz9ZPRDmP9Q4K2Wk59g*=Ih6F=c;C@Z=@@!*$h&fYcTt+KBwEE z&~b^$^8ZEYxvbgjAtv2+Q1VVNfrD}|CjwkK#K}Z~u0J6bx(CQ&nY4ind&4uDnfpnT zV7ov77YT0Io4{Cfa^l*DB(Y2xH4kWMgU-yiB6{ClxfCjpZ)-nDKXXqe4C4o^g1GX; zP7i{wdF8O-=>*n7iUf<~z;r10qn)xVjAPNBI;s5`t3CHTuH3%q=D~r}FQaP@j|p&$ zI!3($e55e};X_ZXUC_Qc0Zqj&#sxY(-g)+(b_jYS1Vs1j0Sz_o15c_>U2ctXh34Z8 z7G%d^m#gxp-<0p5?X>_CY<`)YGZ$E#dlHIQ8Q2EMw3iXB0?^1dHDWa%m^P0BwFU%m z9N&avDGvb5`ioTSievh1+lHtGEo~Yd+E33QJ3w5Via`J`my4j~-yOLhVse1N1iZLS zMHv%IN2pm4GtMc;SDlIDP*=Hvp#x}4H1b#*-t=0TOBItM69h1Ybf`q(ts5}F6-FLV z&5i%?=wW9Fx<`!#T6sY>IH0a$(Kp`CPd1c_O_g7^Iu(rS$m3A8S2H!sh7J|pv2ly* z6MoX%%2!+URz+FFZ=cDWUS-jFij~#euvC#VFzdSnb@Q7ql^nS_NEKpYySkLJ%IFDT zJ`OCrq_x9i_^(eKVkS+hbdr8%ED$XMe4~TY2RO=JSH|x6t71tDTj!KFK&_@7q>pP# zZjpMb?Zm6wz%7mR5)gz*Gf-M3~J7F%ryOUIDZaj_f$`4s=gthX7nS%Wr~5 z>rxJiEw0cW2XFG1Y@f0Bw8 zzQrn}X!>vf3@tDWN(a#cNnISgiuNAsd1D%yfdgP!{a~FSpgj+a1M31^e;f>8L{>oU z%&Rq!fEfyGEQs`ja0Z=w18m`dIrwDe=1*Tb&^q-9m_;{pg(=bR3MOE3=B{wG0Ix4@ ze|qUoh%vDUU-|ehC&pmy{82IRgh1|?+w|~8z7eYHs0!Xk&*hsLoi9AJduKZfy#;iF zx#YpIaxH2O<66r+08al@`+Z|>LKMf834z$h6(?K$6HGH#_@9ytn*;l?vO{G20LR#FK#(@~{HE0i{a;n`alhI{oA1LLXL%)BAb>_AoY=+}u9KC)gYa z&>qImn4t2*#Uz{|9CdbG9k8^O2 zYS4wyji7vxU{%DDby$Kw5B#l&Ph%!Ijc=aaWt!0gjE!T4BvBbuzzIHM){sQeI_xx+G>IM!mY{B0s22STT%rk(35>w_x z8^d(79W<+cM|B0z&7xd9xYe*&pysKPGDA95lo{dK&D>j)E>1b0C0DYx%b0NdZDuwS45ykZ4>@Fj?{ka#!s#aw>{-ejlq*l1> z5|24+Shg{{(-}1#InWlnz=p)s#wCmsN(3==;#f>8onXKyD>)1wXwO)PL3k%moq6&- za8-^CmRVajfPMhP)Y#y(LqH!tT$|06r>f6+2ZRoP(RPVoSa|DX-wf8d1ZELKo<693 zGs4r+bphV(45T1!T-`4vw+5J^z4f!LnG7^N4^? ze5EKH-TAxh${}vNm*eRt){NAsIMKkpI>1&g92G+j1Q%%=Pzgx8_Y8nyZ4WkIJX*Ax z?#1?PUa+(kNB5XUYK8~FGE7H38W$wjm&)~lGfzOy?rxmhPxqWs{)7r>zX%S;BY=NXB?jS) z&Q^r}`;U%EdjoHi(;h5mr~SRjX{e&_gPI9qc5KB^ZArJZ89I(>NA`%v;C?9FMKJ&2 zk$NtTcB}~j9R{Q+;bLJ~N$nwBiWU}^uE{PR|j395gIT$XB z2UZSrbEz09E&{g(MNM!OXb1i6^+S6hwyTp>g|WnZL5ub5MO=BL-JlrI7eC}N0TtPd zLg>fuWlFn(d>PORkPgn4Ne5{EX~&^mzYC1xN#;H}`cL0^^?}k|t>+$viOJs7XFdlJ zYt`T-8~N{#b+AmTNTQ3nm?m_-87PCRBF9(&;!F&i|0N1tN5OwS{w~K06!~y<MA4pUcqg>E10h{)bOFw~h1SNx3n+7Z))YZEfs-U-1R}-PJm}cF<=r45+-ZbP!s;h_EoOwT_-RMCX zWWc#6PzP$@cm)hs%?>cXB_@bRAlN{w9XK$zCsmZKe11||+!J8t0Ln$u`1~DSl9co8 z;-IraxP%F~pwNYs=YJU#)9L{wkf|#0D$RouZ69khpSs`=)nAnp)Fjy?>EWPqa4&H00Wctbk+x`QxRrB81I(|`9F`lv zz}$vJx6R+bp`(+h269{rL(|Iab5vYrnip3-+g(g!?_V_#5I z2PejKQa)z_>e&2JMcXt?u3ta$CPde757P02+LLmi*hhGEY0rFE<8**o(3j8A<=VYz z{6H<#(!U)2V4bTrs%trsTe5G7XXj1~bFdSbwp_7}r~n|uV8j%Lv2SKXK?I`Cp$P7Cnr;doe_nE_x~AYe?ywO1gQlLwh3 zz*_AkPH(rgcyt|!lZDVb^)?h9_wzTv%6-7N(}Wi6{m=kzd+oWeUdA6H%mZ5x{hmwb z31}|ezaoucf9>0Bc1BYYqGM1x3ud^A3f$?v01HQZ2rhYo+R_DGPiOWDDs&BsJ5NPd z4~vz1aFR7k!Qh-k6A0)6rHQ&;&k<$RcbZqCp2qS1`nV7-TmmEc}Th?t*n0#%4_C~MF zB~BxXcN}zwkC_eU=hlL~>@@SMecjHb$*fQ5exI0d-Q#{1z;sQ9UtMuHvVxFc4z6qvebp;sjrL(NT>ce;Om;kQ@Dh@2*(mf23bt7}5 z7i7;sloc^u_JM*pud?f^WT308X9y>2qqv8PN3J6S1k^G2vO&qxX6O?WM@oa4`hxU% z&O)G)y>0XNY8L~w>gWa@5t!^`tpVB(jZb_Jngb}`2v!K96JtPt>;CFu(R9u!S%8*4x{t%( zt9ML7WB#zr7^?}^x4=AjeRG9Lyl^jbmm2{bnPDc-nIe>t1rk}>0E1X(E;>_%rE~KB z?Z(X)nb8SZ{*JOiK)cA0!OdTq^BxEozIh+`S3h$#ra-!i5xfPY7J|&(+1e_f8(ZjI z`o+EISh0)q;wdj2yzn>#XC>IHPNN&trB!c3QPY5yu)(|p`a-oIhR;LY4&p^s_4qrq zvM>Q=$ZeR`+dj@i0mt79(%VqD7G`WUSY^?hfd7=pT8OaL04`v_#mDK9WJA{sglQr0 zD`nj{)0tI%72Yk{r%HVmuF0)_FzMKCHR*+h@RY2O!B4)Un0wti!YT!_3Ji)U6 z_u<13fPaYvxHZVeo--9BWi#YDF1^B4w}2_@V)Szy{rK_3ZjMZQ^Ei*!Ia{%xmm?KO zu2`d0(=75Nf9-3>(EcmNHe{Fh;otO%+p5CRZFAKhreLr^{GT>;%+ah?BUgHX#z~Z1I3x2yN?Imv%28{ zp&X;?n?dD10$UgDbz;)scoI2n0T&G5t4F{_`sW#D9-Y|)lvsy@SAz^gW8nR!)L{*` ze*0_6(6b>g8`FxEG2uuB@>;ocG`RYzdFDu86VDb9wg$did(TvFDvJPMxMC6*;`$z@ zUb}gL149rEYIV*9w>fn;c!WO(D`?R*DB5MA-62Q&CPv1LZ%wyzyWP0 z-~KW+dXqnK7!Q+8W2?*~Sp+JYOyAp3GsSW4EH{EXq}#BB4aUVAuf@iG`5WaMm|Q`< z2fzT8v`hppf+O@j*N(YM$C7+~RCS^4+2ecj=Ow-@20^U@B*VF9MYoNM_D@7Xh1cG2 z=^5<%&+q>oPBqa$@*Vg1*BSnyqy80#f9N7?@IUS)oRHUc9gCvvh;if1=_mIM^Y%^M&ICYpQg^hc z?wjVl8;}KBuAKqz0L{iOg9F^XK~ztONd4v9^3~0D>C*XsxhzG^3ZLVgk-%E9h3hL$`6{{h*9^G4aNW zZHJ5r&Oem*aDX2uQdHqpBGMqDx<$X`TvZ%{_lTuOfOHDyU85mWbbmD#>jK8>oEm11 zmkrlsb?bSF&Q#9r^P~s0UoAF6`T7_4#79G#=Fh*P1Thds&?*DYKR|ef!VFdf{+4?YdESfK|8Tg68hO0i@b%L_aEz|658?2EE)fe~{92UU zm>y9+zuQj5SK8oqAQb^;}xx*8ViK2pEc+f@_CadY5gyuSE#Jme!#S)q+ z_0V7)gzp$S`xqn;5+D+sVHyC@5y0+V?F-5u-_In`5y1~??BsPLC7Um8lHQU0fdbJw zO$$e4jmaMi%0sdd|LzrVaxS9U#CC9_?$t?Ycdxp$O4erMz%3kEoJ*ifPthsEAcjh? z$Yq1m7le&*N7f?$kQL;_GX~~7!aT@e@~S~L$b;Y?J%Cew=7jP!OWq+}%zr!bev@_L zQt-6tOn-j=82Y+EtoV|8f3>A{m?@Lh3S$Rw+22CUQF5kAplQcwZUId3ydA9DJs@w84liBfdZN}FX!}7m z8F8j=*^a4n^8OhHI-MIcn)Erq(ll@g>O%I}Q4gjD^OF<<~w4e)F0nr|_!K(Nqn>B@Nl@eKWM4>C6e1zRgIfq7>~cR6sh0M{)6 zvx}_`&~8r6l2sQ_lY8RbUJeCt%!$o}Fccp!QVNnGV)+MmFY_!`djgsCfM?9+gU|7d z0JqwA@>#xeJYy$x&Ksy(pB06N;_ZWN-4as@SOKZDSM|Y79i)?kRJm5mjXdq*_OwId ztK&a!=ebkFk}3>=i;!mE%uG0J_5-SzciubEzHG9?3S+Q&>Bu!_UVA!X$(3KRRsrTuh(Fk#q>vJb&wdETW4nO>_RGsY z^2ETS3YeTJqH<4%j}Ozo0-hG2o#i^=D(&jCTnAWU=B+oM4vjcqFLqXVW@>s&1mjpK zwjEcV*za4LnISE3?cM` T}c95yfzp6Qb-s+0Y^MF`X=P>-0}cqCJ>1BZ!Fo3mJDynMLK-wEDGMR?JJc#oZsN?8uQ6e}P zfO)(~*K)p7bhQiJ6b|B!<)F=XNNNYQ*%CW5?s)u9LfO1wu#MA z?=?c}1T~3eN<=HoTd~dPszRmeZ!dc3AZ9>aJP+H<4d@*PNZ?0PLyI(O)lxi;=7e)3 zc&c<%tBDt?sljR|J>b!znU1OVq(kfEWo8tVUai}e+tKE7D>(lXNKd`F0Bx;YnoKdT z1m1E1+=J;e0V)HPavuhSCmZqh0MBL|+<1(wFsXMg}Yw$3d~?(c%p*nAVviMLEq)BcwOeJ+tD zJ6a(>yr0t$^fBJMKwo?hEGO23U6+0q4}SAOyXx*?`t0LF{@`{SU_cLuQy!D1=~#>d zBkU2yDpyhQ{L7<<*U+>F0WoJer7UUCSA%N1RTcRC(xo<+MlOUqsJ^Bje0*H2kyVEJ z@Th?fX*G%f_X05zGg{gNHRd%%g~drP)(CYPw2QbQ;LWVjehM|+#CdB^pz`NAISt0l z2-0!>-5_26rZUp0{T`lcC{t@!+ty9A*KEy;h4Zv;bEEe*|C(=YRoO`ICdy{ub!+`h zs0EOfnJ+z14FZ@}N`x`gi#9WoI5^;QH$x z5GB;<10W@ai{6-v*Zg|w`D0=Ow-k-*0s|BQ>eJRUjvUytD00gmJ7NImR7z*S;#t6s zAnoU$_yN)~^zEQg?Y}aiu)0^1z@k7ZU7!K1>%sw=fmji~5LHVB;C-x1f#a!rx&pS3 z)mD!&^K6y^!BtXJ=o>+y-4?X@&eF4wORrZeWQOR%xoUFwz?2B#bCr;4{?NLbzℑ z_TU3Q(c76dYDIKx+oEq&m2Tw47WZ@elNo@)ifJYrTdV9Y|n#kWMp@$v$wktL}3 zar8Pq*9MGNSgLc?7`4Cn()A7K00DEE3r%8`4=2gN7Mju#Eg)C~%W<{((vjc=D26Rh zUSN1X`M%o7(I&#eSy2~G& zRT=@JuQxzC)K*5_kTVqsJwMem$SUr@Zml3&r+VvQJNtTtam0M^_}$Q(nNuFCi^EISj=5wl&eAVOM2!)QZd-R8ZaQ1ZWH9Vdh2?ZXglPq z@DXM#zy;8xCm2HvOlU8*q67Et6bpD8Xn;QF5hm+QQ5r;>_H>3j$k|-w-0^47)KTFF zbxvSwA`gm zPDl8ke1I2JAX`OUdBobfO=SuXJenmN1^@O*D9izXlVyf;N*~?}CSZ`XCx|9T&|Sbr zgb|!W035h>&~1PLG!4oH`B@nDK>&%_U}~|0#ziOw1ne63J@qS-gE;MoZK>eAfY@Am zpb-KXRRP&GV$bCU(0(pBGN}C`{o=zgXdkCGXtxImt+EYx;aHLAC+}K>lE|fU^G9<~ z1G+~!SB{n=2RJt7o>L_+*4%kpq7xH@!(XOz0RL66B+wA33gph|Br=Sgy!I`l9Z&@M z+Ed1@?KmYb`fT^{r}fTcbEvK13$+kK*N<_KTVYF&>^%E@@Z_NZ@TTGJF$nGJ^c;GE zMId;|mIuId1ORh^Va=50$+sbQgP1XZsZ7wYnXbar1CWRo%s-BU`&g z&+X7{-lB~c9RKU94)m}w-rT0LRyIxX9XSB4E7Q@}#|%mWNV#}Uwg6`gfU&4Pa5vDr_T3<+ z5Hk*7ebmg0&fx+hjAsH%cY{fA1G>VfE#UO3lT4r0K#; zswpha;$+n^nI9|bU^DR>;9F5xQ@pXk99aJ2;in+hR@>YSCk}PVHa7aE?tlIM9V##l zjKo%S{^jtG(GDmTqniG*w0OM-R3>vVJNXi8FHrl939~m3BsldhM^g3KTf;mBeKRQ? z)sc|dL+x30oJC3R)psGLS(RKa+l#@thG?$@Tnbf{4g*i#1zPoMjyv~Mpb8cYfwp#e zrDiNzCPwm&*UrBW33Pyl$H6=uJjC+p+mOtYc&5h4npkH!x^DXVs~5!*raeH6={ISL z%xWFlJ*Jvv(aIExYRCVZ05zNC7?b6V>w$MY&}GoJ5BE=ie9m0w=vs^gcK3zagZd z^Nm%g4mY0$57Ld%PP(cx!pMAMgn;cJ9v5J^O*QA3+!VAImU9nCFL)ypTMaV74tkz+ z`9ko_f-_HfUc4uakr1e8GTK#4_QBFCN2O=bBJKPUW{1J#&@42kJlzeloK@yWBXAi>wv}=!$fnwiy`N+BNr*ndW z7G2|!XFz$NLFGn2=qmisfJPDE%yMoHxYlrixcRaVcq&MH_)@U}q6nVq9rnC6UI zyJkbC3ec`qLEqXXIy(|IpF%NRbz;zB{HnMvgcofy->Filzh-QoD*EPtKfd+`x_Obz zYgKgzoe6&Gng8=|zdoXzXb;0j4{aZA+NCaC6!aC44^oF_z|CjD>YJ9qfjSnkAk?j` zv6CM2*q$5D*#{PRtW|dRKr7=6c@EPgWA`+cz~F5P^LR7;FNiLMv}AkZ`kS20yx749 zF5kJLH;~Xg^p=#ibWptxu-AjEyG)z6atvo40&DUN8UOIde$W^&E`b6g1n+{Xq-haI z17KU)3a-5n1M-&GqRm(!C04-r)%Gc1Y z9|9|*TQ0od1}?b%R-51PH=VT?aZPp7F*9XNYGUd*YGItrMyCp07SBHe z!Rem;fmp{jHvhYzTns#?DG+FQ)q(Bj5z|!o#-WdY_B^<=-0moUs71PJ89iK1SVa5E z=s0`twoy(Vs9-7sL997}3hq_TJV(_s&zBk=z&up-pU$(>x$dAwkU7ZG%%-r{&2n=w zfMqzw68ykE@9hx4RPB_~q`l(*_qX4@GtmEccRx5A;zGY`PnCkSZIdl@-t2DRd6slT z8ni|xR&@nSNDWN7nQQUkYPsQ@mG`CBMXlaF8`6%07+7;D$d9*f8Dz2pNg}{mMXSh6c29KhHKKttvuO)W!x*}-iF?dWC;OvGNd2MnimphSa z=JMt7(qG>uD2kTI1tY%1KL27oNJV0mj7=UN@lIc3mV+-aWuKd#eme`xY z#_>f?-grC7z*Uc13C7Mm6vo1J0Tp8nhbZ9o(V%J&KyOWe4uF_8z8>iS^{h6|Iq=L6S#*NY z${16jq{9|@?G6f{3Pe}-fU-dDKHI>RGm@v%$I$}RG6K!yLg~>Su(>v?#>>Mrd-zBf z`1Rk6HF04at^w;4^RR_TI-)JfCB`){+;yKP8m0kjFMv*F1o~8^<+5LUU`N09!91G# z>4}~VBRIfwDi@7@@(X(2xg2b}-R6`0Y4TeP1>Xv_cELo4aNaFEAdln~+5`?vo;wsr zKl#ovzH~)jYx>w{FK{t&t=%)sxo0?mE#=43JuT)Q;B1Qk4?O=f6?ZY=OM{G39ep#^ zMfKWK(sTWY$hmlxqAUoW^EOmWUV1N$(P6sRcI=Yw9&R}V(r)=6hs6#aAK|ma*?U*+ zvv6Xl0lyrPA?FT4S>49}?zZs@ks=4x-b z!b1bIBJ3e-RSI)X+qOB+R_rcfK`!r00ly^y0v2JJSvPnArxX z(dDz*x6><6tXnzH7rb9A5j+W?9Lf`X`7oIGg=Zan(i!?4%@d|UWVQ}4wJLbM<{*n& z!K^^l6Qw-F=2MTM0=q^&d?p`Kd6t26mL66$y|p(0Ech=k9)*au1at8Ps{;Ym`CKP~ z^|C_n1?5MmJg8ut#uk>Tj&l#`DwIlZb=P9bD9ttKI{%_N;rkfy=Xs_ReY(BG%-;fM@VVB2`m5c39b0qcbGtkNwV z*`jh*K#bl2g-7SIRFm=LPaxW&=t0hN=e|9;rM#REdVp!K0cNZW>~G*9QZ3HB48gQd zYXt*Xfj|VP*H8&=;gp*Q!Qp@YOR&Zr(PYPo;`;LSA9aQY=5(YQzQs`swC{s?-hnv2<}^1`5O z0zwMMAK*Q&;|9}^`jS7IV<4ibHP=rzydZnc3_l?sW+(7MGTxT zFw2+%EG(eXD%GPcMRf2mQ-PtiTXi_qqtP@Upr*?38|j8fxp+HX#*@l# zpBClod!8~v^Kt;|1C@aW<9913pz+1eO=~(s1_V0s&H~s8Qn!j%o*5Mzc!y}tAOA>{ z&@sI$QnV(zi@vKL98)eSSl)*=!yOH}bT?RO*fO}F9ioUA3e(=SclO6QSyvknXbmpHC6yX-By+mxBYwLh1sM=?LOLPl> zFHNYkp)q#--^8j$8LZ~T;#vcBfvjZL29*EJuqB41_sbw-$a3(<2rW< zQ0EKV(0&GQ8j$S5WZC=xih%(9LB*g=$0vz-$|K2@alLRq*w=SG2<{o3ZkoDmC49QB zB{?JC6%DSXD!-tu)q-hHdg4^0Ksu6&T1?x)T6^M<_OecW11KA;m&nW*E{Z@3uO8nr z00Ycw&^Hf3SIG>`k=eMCuWFeqFt-KjeZ$mrRiMQTzWJSpHnHG;>SFAm<}$`B1p~Liz1v0g%7A`C?9+(H%H9EH~%A%@t`l_L!3{&4>?esqK2ou02{YS2Qg6e z;danAqqAvcKpSx4+_;VQ^bjry$)N0OT#|fTV?KEIbfZyss~zYQNf zBL<762qOGBL7RxIjnpI+Qu?s%DLb zfJ+>3;b`Mv)E?2xmmz1?JfsD_&N-id_VN#PizqyWpb26C6o`A;Hb}08?%Yd_9vUl+ zo?xiMKsr>l%oxSu2a%J^nJ+Se7HMMM%GE+zKo5u6gXQ+Ge)0~zYt4cJII9{EbJRPd z^vhpC0;VaU!SxErJ5Pv!Pfxbb0vH9fX`=rVItNYuV_g?-96ng2K@C@J`4lpFb!pUtO+KjZo=~#eX zINxbz7Fr!n{;aki;jB&*|z3gA&H`ckL#K! z^ot6p3Z>X%MReBBiZcA?M_*&GYSX=jHdqcEh=48N=5vhM3W&*?3<9dz`N_NLtgJvC z^t%eUZZoqC6rHI*-3?~8#v=lF)uQ|tm~}m;2xuRj`Nlxo$N*bn3UgE{YS45rN-RlD z+`X_ zysb`vhCe8d+3N;c0&VwCAL?(+P34W-C@2_o?R799+8vA~R{}coLni+ui211f&-*k> z1K=h=18A=)p7za)vsExLj@2xL0y<=P3=D7qT|GDsZoWOnj~)~-s7yD<<-foDgC~A} z_q*{dt^m3baIMQt8cB17X6MkcAhzL?4?=0r24k*`L@n;t&I_Gk0#sC_L3dVy<~TWK zBe3aps1~X)rGwXB;LNkV%rHODWEpn`qL};hoLmcN`Z(`7=%;*hE{j+017oJy@!F_; zw6#=hC``ebi4<^V6T4WIJij+6mDS4Z z02pFApP{j_&*7$@sv{4L&BjH3{=H9My!-~Y7>TOg2O$+SYSoy9Mr%_+(`=Ek94e3t zVp{Af)gI`$`Ark))Hz3>{TXO9PcI3x=3#VnIz#2)%l+n?mpNa6w?=E{4_Qmg#+0Kl zOn>!$UVxAGC8pjhkKS@D^BvkNYOv7(e8vhf%;Pa_wxW}X>K@Z7HuudR7ooY_#l_By zmy3Go<#NP;C|7G~ie_W;;N;a;f|#^!DP=n#W*c`Zm!JVS@In$lBaM?UIf){C72^C) z0{kP&Cr-T`&(hJc2U-v4@+YUe4#Ca=(~WOUMXfq=9gbF7{HW?th#oTUf=|^*Xe!n#yC`Z z(+JvLzA|D}1I7M}w`T?*lgGos`_3H!2llZF1qFUMbbrUYi&=rmawU!Wjtssv)QJCY162c=!-R;1|y`q~~9fzV<3e z`aU9Y0{o+2fw5wM5l|-azDYpSPPG7atj9wgdDZuK_DNy<1!Sp-hc;fv54rnE~2;ZDD~W z;}&R2-Z?Y@W(Baak1iWL^Nciw-*O>^>Dt2^TwVa{%M1z+!Np`N27Z74`)K>=HY)>E z?R{>2myRt%XQ2UmJ#)Ytq`@2Y{e6z<`Pc3h1H)p?BHSkJItc!yA}F$EzbTkvots=) zO3v`$Ie=65h%{C5fO(^sI*cNjDwAKn2OhwJ0(c8$yN4%QEWPqb1mpqkled~aIGji` zX?{N#z@7s7;8UhwwD!TMC$#Mx*tKVwqRh$9p2(Uzdl1U%F8)!*8x584tJmLF#l|JW zHQrEKIKb6`6$*U#x=QJR>hpJ?oz1O61Fzj-@1Hi+B-?;re4ic$+%1QrIX)oPtu$RU z`$is3HN@20%_6l*Vk~R5-=J{|wsHgMZX>7=BryK*&Uc(`3gv$~w^{OYjN2Q2tTHCT ziAPGnwSz#q@*vtb5IUW;B>eWhcVeqTKQlMOJTuhMj1_08eIy12s44kx&rOG^By`_; z_8{09oD07h|NMwA6dlu|74L8P#!NVtTSh4Z3A}yA-`Gxe;_^-p0=2DnqTF)0|DvJ1 zo$@Wd{kAjVz!m0^;Ms;aw;lh$wVS6b^}r6BjeoFDVT5UkVo>TM|FNgEL%SJrExv8U z-!d^AmUUuUKZ?=%w(R lit(0)) & (f.array_length(col("failed_suppliers")) < f.array_length(col("all_suppliers"))) -).select(col("order_id"), col("failed_suppliers")) +).select(col("order_id"), col("failed_suppliers"))) ``` @@ -303,14 +278,14 @@ once: ```python exec="1" source="material-block" result="text" session="aggregations" from datafusion.expr import GroupingSet -df.aggregate( +print(df.aggregate( [GroupingSet.rollup(col_type_1)], [ f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed"), f.max(col_speed).alias("Max Speed"), ], -).sort(col_type_1.sort(ascending=True, nulls_first=True)) +).sort(col_type_1.sort(ascending=True, nulls_first=True))) ``` @@ -322,14 +297,14 @@ for that row and `1` when it is aggregated across. Use `.alias()` to give the column a readable name: ```python exec="1" source="material-block" result="text" session="aggregations" -df.aggregate( +print(df.aggregate( [GroupingSet.rollup(col_type_1)], [ f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed"), f.grouping(col_type_1).alias("Is Total"), ], -).sort(col_type_1.sort(ascending=True, nulls_first=True)) +).sort(col_type_1.sort(ascending=True, nulls_first=True))) ``` @@ -340,13 +315,13 @@ With two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` p - one grand total row ```python exec="1" source="material-block" result="text" session="aggregations" -df.aggregate( +print(df.aggregate( [GroupingSet.rollup(col_type_1, col_type_2)], [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], ).sort( col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True), -) +)) ``` @@ -361,13 +336,13 @@ For our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the t by `Type 1` alone, by `Type 2` alone, and a grand total — all in one query: ```python exec="1" source="material-block" result="text" session="aggregations" -df.aggregate( +print(df.aggregate( [GroupingSet.cube(col_type_1, col_type_2)], [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], ).sort( col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True), -) +)) ``` @@ -384,13 +359,13 @@ For example, if we want only the per-`Type 1` totals and per-`Type 2` totals — full `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that: ```python exec="1" source="material-block" result="text" session="aggregations" -df.aggregate( +print(df.aggregate( [GroupingSet.grouping_sets([col_type_1], [col_type_2])], [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], ).sort( col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True), -) +)) ``` @@ -398,7 +373,7 @@ Each row belongs to exactly one grouping level. The [`grouping`][datafusion.func function tells you which level each row comes from: ```python exec="1" source="material-block" result="text" session="aggregations" -df.aggregate( +print(df.aggregate( [GroupingSet.grouping_sets([col_type_1], [col_type_2])], [ f.count(col_speed).alias("Count"), @@ -409,7 +384,7 @@ df.aggregate( ).sort( col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True), -) +)) ``` diff --git a/docs/source/user-guide/common-operations/basic-info.md b/docs/source/user-guide/common-operations/basic-info.md index 703e319e2..f4f7b49ab 100644 --- a/docs/source/user-guide/common-operations/basic-info.md +++ b/docs/source/user-guide/common-operations/basic-info.md @@ -1,28 +1,3 @@ -```python exec="1" session="basic-info" -import os -import pathlib - -import datafusion # noqa: F401 -from datafusion import ( # noqa: F401 - SessionContext, - col, - column, - lit, - literal, -) -from datafusion import functions as f # noqa: F401 -from datafusion.dataframe_formatter import configure_formatter - -# mkdocs runs from the repo root; the demo data lives at docs/source/. -for candidate in ("docs/source", ".."): - p = pathlib.Path(candidate) - if (p / "pokemon.csv").exists(): - os.chdir(p) - break - -configure_formatter(max_rows=10, show_truncation_message=False) -``` - + +# datafusion + +::: datafusion + options: + members: false + +## Submodules + +| Module | Description | +| --- | --- | +| [`catalog`](catalog.md) | Catalog, schema, and table providers | +| [`common`](common.md) | Common types shared across the API | +| [`context`](context.md) | `SessionContext`, session config, and runtime | +| [`dataframe`](dataframe.md) | `DataFrame` query builder and write options | +| [`dataframe_formatter`](dataframe_formatter.md) | HTML/text rendering for DataFrames | +| [`expr`](expr.md) | Expression tree (`Expr`, window frames, grouping sets) | +| [`functions`](functions.md) | 290+ built-in scalar, aggregate, and window functions | +| [`input`](input.md) | Input source plugins | +| [`io`](io.md) | `read_csv`, `read_parquet`, `read_json`, `read_avro` | +| [`ipc`](ipc.md) | Arrow IPC serialization for DataFrames and expressions | +| [`object_store`](object_store.md) | Object store backends (S3, GCS, Azure, local) | +| [`options`](options.md) | Read-option configuration types | +| [`plan`](plan.md) | Logical and physical plan introspection | +| [`record_batch`](record_batch.md) | `RecordBatch` and `RecordBatchStream` | +| [`substrait`](substrait.md) | Substrait plan serialization | +| [`unparser`](unparser.md) | Convert logical plans back to SQL | +| [`user_defined`](user_defined.md) | User-defined scalar, aggregate, window, and table functions | + +## Top-level names + +These names live on the `datafusion` package itself and are imported as +`from datafusion import `. + +### Column builders + +::: datafusion.col.col + +::: datafusion.col.column + +### Literal builders + +::: datafusion.lit + +::: datafusion.literal + +::: datafusion.string_literal + +::: datafusion.str_lit + +::: datafusion.literal_with_metadata + +::: datafusion.lit_with_metadata diff --git a/docs/source/reference/datafusion/input.md b/docs/source/reference/datafusion/input.md new file mode 100644 index 000000000..88f2528f2 --- /dev/null +++ b/docs/source/reference/datafusion/input.md @@ -0,0 +1,28 @@ + + +# input + +::: datafusion.input + options: + members: false + +::: datafusion.input.base + +::: datafusion.input.location diff --git a/docs/source/reference/io.md b/docs/source/reference/datafusion/io.md similarity index 100% rename from docs/source/reference/io.md rename to docs/source/reference/datafusion/io.md diff --git a/docs/source/reference/ipc.md b/docs/source/reference/datafusion/ipc.md similarity index 100% rename from docs/source/reference/ipc.md rename to docs/source/reference/datafusion/ipc.md diff --git a/docs/source/reference/object_store.md b/docs/source/reference/datafusion/object_store.md similarity index 100% rename from docs/source/reference/object_store.md rename to docs/source/reference/datafusion/object_store.md diff --git a/docs/source/reference/options.md b/docs/source/reference/datafusion/options.md similarity index 100% rename from docs/source/reference/options.md rename to docs/source/reference/datafusion/options.md diff --git a/docs/source/reference/plan.md b/docs/source/reference/datafusion/plan.md similarity index 100% rename from docs/source/reference/plan.md rename to docs/source/reference/datafusion/plan.md diff --git a/docs/source/reference/record_batch.md b/docs/source/reference/datafusion/record_batch.md similarity index 100% rename from docs/source/reference/record_batch.md rename to docs/source/reference/datafusion/record_batch.md diff --git a/docs/source/reference/substrait.md b/docs/source/reference/datafusion/substrait.md similarity index 100% rename from docs/source/reference/substrait.md rename to docs/source/reference/datafusion/substrait.md diff --git a/docs/source/reference/unparser.md b/docs/source/reference/datafusion/unparser.md similarity index 100% rename from docs/source/reference/unparser.md rename to docs/source/reference/datafusion/unparser.md diff --git a/docs/source/reference/user_defined.md b/docs/source/reference/datafusion/user_defined.md similarity index 100% rename from docs/source/reference/user_defined.md rename to docs/source/reference/datafusion/user_defined.md diff --git a/docs/source/reference/index.md b/docs/source/reference/index.md index 0eab6977f..97e5830a6 100644 --- a/docs/source/reference/index.md +++ b/docs/source/reference/index.md @@ -19,47 +19,6 @@ # API Reference -The public API of `datafusion` is exported from the top-level package. Every -symbol below is importable directly: `from datafusion import SessionContext`. - -| Symbol | Page | -|---|---| -| `Accumulator` | [User-Defined Functions](user_defined.md) | -| `AggregateUDF` | [User-Defined Functions](user_defined.md) | -| `Catalog` | [Catalog](catalog.md) | -| `CsvReadOptions` | [Options](options.md) | -| `DFSchema` | [Common](common.md) | -| `DataFrame` | [DataFrame](dataframe.md) | -| `DataFrameWriteOptions` | [DataFrame](dataframe.md) | -| `ExecutionPlan` | [Plan](plan.md) | -| `ExplainFormat` | [DataFrame](dataframe.md) | -| `Expr` | [Expr](expr.md) | -| `InsertOp` | [DataFrame](dataframe.md) | -| `LogicalPlan` | [Plan](plan.md) | -| `Metric` | [Plan](plan.md) | -| `MetricsSet` | [Plan](plan.md) | -| `ParquetColumnOptions` | [DataFrame](dataframe.md) | -| `ParquetWriterOptions` | [DataFrame](dataframe.md) | -| `RecordBatch` | [RecordBatch](record_batch.md) | -| `RecordBatchStream` | [RecordBatch](record_batch.md) | -| `RuntimeEnvBuilder` | [SessionContext](context.md) | -| `SQLOptions` | [SessionContext](context.md) | -| `ScalarUDF` | [User-Defined Functions](user_defined.md) | -| `SessionConfig` | [SessionContext](context.md) | -| `SessionContext` | [SessionContext](context.md) | -| `Table` | [Catalog](catalog.md) | -| `TableFunction` | [User-Defined Functions](user_defined.md) | -| `TableProviderFactory` | [Catalog](catalog.md) | -| `TableProviderFactoryExportable` | [Catalog](catalog.md) | -| `WindowFrame` | [Expr](expr.md) | -| `WindowUDF` | [User-Defined Functions](user_defined.md) | -| `col`, `column` | [Expr](expr.md) | -| `configure_formatter` | [DataFrame](dataframe.md) | -| `functions` | [Functions](functions.md) | -| `ipc` | [IPC](ipc.md) | -| `lit`, `literal` | [Expr](expr.md) | -| `object_store` | [Object Store](object_store.md) | -| `read_avro`, `read_csv`, `read_json`, `read_parquet` | [I/O](io.md) | -| `substrait` | [Substrait](substrait.md) | -| `udaf`, `udf`, `udtf`, `udwf` | [User-Defined Functions](user_defined.md) | -| `unparser` | [Unparser](unparser.md) | +The Python API of DataFusion is exported from the [`datafusion`](datafusion/index.md) +package. See the package landing page for an overview and a list of submodules, +or jump directly to any module from the navigation sidebar. diff --git a/docs/source/user-guide/dataframe/rendering.md b/docs/source/user-guide/dataframe/rendering.md index d6c6083a5..0b668b985 100644 --- a/docs/source/user-guide/dataframe/rendering.md +++ b/docs/source/user-guide/dataframe/rendering.md @@ -20,7 +20,7 @@ # DataFrame Rendering DataFusion provides configurable rendering for DataFrames in both plain text and HTML -formats. The [`datafusion.dataframe_formatter`](../../reference/formatter.md) module controls how DataFrames are +formats. The [`datafusion.dataframe_formatter`](../../reference/datafusion/dataframe_formatter.md) module controls how DataFrames are displayed in Jupyter notebooks (via `_repr_html_`), in the terminal (via `__repr__`), and anywhere else a string or HTML representation is needed. diff --git a/mkdocs.yml b/mkdocs.yml index a17c36f08..2a5e0a9e8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -151,20 +151,23 @@ nav: - FFI: contributor-guide/ffi.md - API Reference: - reference/index.md - - SessionContext: reference/context.md - - DataFrame: reference/dataframe.md - - DataFrame Formatter: reference/formatter.md - - Expr: reference/expr.md - - Functions: reference/functions.md - - User-Defined Functions: reference/user_defined.md - - Catalog: reference/catalog.md - - I/O: reference/io.md - - IPC: reference/ipc.md - - Object Store: reference/object_store.md - - Options: reference/options.md - - Plan: reference/plan.md - - RecordBatch: reference/record_batch.md - - Substrait: reference/substrait.md - - Unparser: reference/unparser.md - - Common: reference/common.md + - datafusion: + - reference/datafusion/index.md + - catalog: reference/datafusion/catalog.md + - common: reference/datafusion/common.md + - context: reference/datafusion/context.md + - dataframe: reference/datafusion/dataframe.md + - dataframe_formatter: reference/datafusion/dataframe_formatter.md + - expr: reference/datafusion/expr.md + - functions: reference/datafusion/functions.md + - input: reference/datafusion/input.md + - io: reference/datafusion/io.md + - ipc: reference/datafusion/ipc.md + - object_store: reference/datafusion/object_store.md + - options: reference/datafusion/options.md + - plan: reference/datafusion/plan.md + - record_batch: reference/datafusion/record_batch.md + - substrait: reference/datafusion/substrait.md + - unparser: reference/datafusion/unparser.md + - user_defined: reference/datafusion/user_defined.md - Links: links.md diff --git a/python/datafusion/__init__.py b/python/datafusion/__init__.py index f4b50980d..889d3898b 100644 --- a/python/datafusion/__init__.py +++ b/python/datafusion/__init__.py @@ -33,20 +33,18 @@ calls. Build with [`col`][datafusion.col.col] and [`lit`][datafusion.lit]. - **functions** -- 290+ built-in scalar, aggregate, and window functions. -Quick start ------------ - ->>> from datafusion import SessionContext, col ->>> from datafusion import functions as F ->>> ctx = SessionContext() ->>> df = ctx.from_pydict({"a": [1, 2, 3], "b": [4, 5, 6]}) ->>> result = ( -... df.filter(col("a") > 1) -... .with_column("total", col("a") + col("b")) -... .aggregate([], [F.sum(col("total")).alias("grand_total")]) -... ) ->>> result.to_pydict() -{'grand_total': [16]} +Examples: + >>> from datafusion import SessionContext, col + >>> from datafusion import functions as F + >>> ctx = SessionContext() + >>> df = ctx.from_pydict({"a": [1, 2, 3], "b": [4, 5, 6]}) + >>> result = ( + ... df.filter(col("a") > 1) + ... .with_column("total", col("a") + col("b")) + ... .aggregate([], [F.sum(col("total")).alias("grand_total")]) + ... ) + >>> result.to_pydict() + {'grand_total': [16]} User guide and full documentation: https://datafusion.apache.org/python diff --git a/python/datafusion/expr.py b/python/datafusion/expr.py index 5e9b0cc5d..167f8f15a 100644 --- a/python/datafusion/expr.py +++ b/python/datafusion/expr.py @@ -447,8 +447,9 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: worker process for distributed evaluation. When ``ctx`` is supplied, encoding routes through that session's - installed [`LogicalExtensionCodec`][LogicalExtensionCodec] (so settings like - `with_python_udf_inlining` take effect). + installed logical extension codec (set via + [`with_logical_extension_codec`][datafusion.context.SessionContext.with_logical_extension_codec]), + so settings like `with_python_udf_inlining` take effect. When ``ctx`` is ``None``, the default codec is used (Python UDF inlining on, no user-installed extension codec). From 28874fe7d791662b119807c491288447faff4008 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Mon, 8 Jun 2026 12:22:20 +0200 Subject: [PATCH 14/18] docs: resolve mkdocs build warnings and tighten public API surface Bring the strict mkdocs build to zero actionable warnings. Cross-reference fixes: - Replace ~20 unqualified `[X][X]` refs with fully-qualified targets (sort/sort_by, cube/rollup/grouping_sets, array_* aliases, register_*, Volatility, Serde, Producer, max_rows, metrics, etc.). - Fix typos: `Dataframe` -> `DataFrame`, `datafusion.Expr.to_bytes` -> `datafusion.expr.Expr.to_bytes`. - Drop links to non-Python symbols: `SessionState`, `CreateExternalTable`, `multiprocessing.Pool`, `cloudpickle`; point the `ObjectStore` reference at the module page; link `LogicalExtensionCodec` to `with_logical_extension_codec`. - Update relative link in `dataframe_formatter.md` for the subdirectory move. Public surface and reference page coverage: - Add `__all__` to `context`, `dataframe`, `dataframe_formatter`, `io`, `record_batch`, `user_defined`, `input/base`, `input/location` so the public surface is explicit. - Document the newly-declared public symbols on the corresponding reference pages (Compression, Volatility, *Exportable Protocols, ArrowStreamExportable, etc.). - Update `object_store.md` to render the PyO3 class aliases via explicit per-class directives (whole-module discovery skips re-assigned PyO3 bindings). - Allow `__next__`/`__anext__` through the formatter filter on `RecordBatchStream` so the iterator protocol is documented. Docstring fixes: - Convert overload-impl Args blocks (ScalarUDF.udf, AggregateUDF.udaf, WindowUDF.udwf) to free-form prose: their actual signatures are `*args, **kwargs`, which griffe was flagging. - Fix Args continuation indent in the formatter, the `idx::` typo, and a `with_extension` Args entry whose continuation lost its indent. - Re-wrap doctring lines that pushed past the 88-column limit after the cross-ref qualifications. mkdocstrings config: add `docstring_options: warn_unknown_params: false` so future overload-impl patterns don't trip the build. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/source/reference/datafusion/context.md | 8 ++ docs/source/reference/datafusion/dataframe.md | 2 + .../datafusion/dataframe_formatter.md | 2 +- .../reference/datafusion/object_store.md | 12 ++ .../reference/datafusion/record_batch.md | 5 + .../reference/datafusion/user_defined.md | 12 ++ .../common-operations/aggregations.md | 4 +- .../user-guide/common-operations/windows.md | 2 +- mkdocs.yml | 4 + python/datafusion/catalog.py | 2 +- python/datafusion/context.py | 56 +++++--- python/datafusion/dataframe.py | 28 +++- python/datafusion/dataframe_formatter.py | 27 +++- python/datafusion/expr.py | 21 +-- python/datafusion/functions.py | 16 +-- python/datafusion/input/base.py | 2 + python/datafusion/input/location.py | 2 + python/datafusion/io.py | 10 +- python/datafusion/ipc.py | 22 ++-- python/datafusion/plan.py | 14 +- python/datafusion/record_batch.py | 10 +- python/datafusion/substrait.py | 4 +- python/datafusion/user_defined.py | 123 +++++++++++------- 23 files changed, 266 insertions(+), 122 deletions(-) diff --git a/docs/source/reference/datafusion/context.md b/docs/source/reference/datafusion/context.md index d9447c409..dbbe27bfc 100644 --- a/docs/source/reference/datafusion/context.md +++ b/docs/source/reference/datafusion/context.md @@ -11,3 +11,11 @@ ::: datafusion.context.SQLOptions ::: datafusion.context.RuntimeEnvBuilder + +::: datafusion.context.ArrowStreamExportable + +::: datafusion.context.ArrowArrayExportable + +::: datafusion.context.TableProviderExportable + +::: datafusion.context.PhysicalOptimizerRuleExportable diff --git a/docs/source/reference/datafusion/dataframe.md b/docs/source/reference/datafusion/dataframe.md index c411e8af6..e2db4f2c0 100644 --- a/docs/source/reference/datafusion/dataframe.md +++ b/docs/source/reference/datafusion/dataframe.md @@ -16,6 +16,8 @@ ::: datafusion.dataframe.ExplainFormat +::: datafusion.dataframe.Compression + ## DataFrame Formatter See [DataFrame Formatter](dataframe_formatter.md) for the full formatter API diff --git a/docs/source/reference/datafusion/dataframe_formatter.md b/docs/source/reference/datafusion/dataframe_formatter.md index 2e93a44a8..d9530586f 100644 --- a/docs/source/reference/datafusion/dataframe_formatter.md +++ b/docs/source/reference/datafusion/dataframe_formatter.md @@ -2,7 +2,7 @@ The `datafusion.dataframe_formatter` module controls how DataFrames render in notebooks and HTML contexts. See the user-guide -[Rendering](../user-guide/dataframe/rendering.md) page for worked examples. +[Rendering](../../user-guide/dataframe/rendering.md) page for worked examples. ::: datafusion.dataframe_formatter.configure_formatter diff --git a/docs/source/reference/datafusion/object_store.md b/docs/source/reference/datafusion/object_store.md index 2b5b6794d..7012c1482 100644 --- a/docs/source/reference/datafusion/object_store.md +++ b/docs/source/reference/datafusion/object_store.md @@ -1,3 +1,15 @@ # Object Store ::: datafusion.object_store + options: + members: false + +::: datafusion.object_store.AmazonS3 + +::: datafusion.object_store.GoogleCloud + +::: datafusion.object_store.Http + +::: datafusion.object_store.LocalFileSystem + +::: datafusion.object_store.MicrosoftAzure diff --git a/docs/source/reference/datafusion/record_batch.md b/docs/source/reference/datafusion/record_batch.md index 60e23439f..c02b786ef 100644 --- a/docs/source/reference/datafusion/record_batch.md +++ b/docs/source/reference/datafusion/record_batch.md @@ -3,3 +3,8 @@ ::: datafusion.record_batch.RecordBatch ::: datafusion.record_batch.RecordBatchStream + options: + filters: + - "!^_" + - "^__next__$" + - "^__anext__$" diff --git a/docs/source/reference/datafusion/user_defined.md b/docs/source/reference/datafusion/user_defined.md index 912190c58..99e8889ad 100644 --- a/docs/source/reference/datafusion/user_defined.md +++ b/docs/source/reference/datafusion/user_defined.md @@ -1,5 +1,7 @@ # User-Defined Functions +::: datafusion.user_defined.Volatility + ::: datafusion.user_defined.ScalarUDF ::: datafusion.user_defined.AggregateUDF @@ -19,3 +21,13 @@ ::: datafusion.user_defined.udwf ::: datafusion.user_defined.udtf + +::: datafusion.user_defined.ScalarUDFExportable + +::: datafusion.user_defined.AggregateUDFExportable + +::: datafusion.user_defined.WindowUDFExportable + +::: datafusion.user_defined.LogicalExtensionCodecExportable + +::: datafusion.user_defined.PhysicalExtensionCodecExportable diff --git a/docs/source/user-guide/common-operations/aggregations.md b/docs/source/user-guide/common-operations/aggregations.md index c38cfd59f..ad0971747 100644 --- a/docs/source/user-guide/common-operations/aggregations.md +++ b/docs/source/user-guide/common-operations/aggregations.md @@ -453,8 +453,8 @@ The available aggregate functions are: You can ship custom aggregations to the engine by subclassing [`Accumulator`][datafusion.user_defined.Accumulator] and registering it via -[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`][datafusion.user_defined] for -the accumulator interface and worked examples. +[`udaf`][datafusion.user_defined.udaf]. See [`user_defined`](../../reference/datafusion/user_defined.md) +for the accumulator interface and worked examples.

Note

diff --git a/docs/source/user-guide/common-operations/windows.md b/docs/source/user-guide/common-operations/windows.md index 3b7eb6d24..f121705da 100644 --- a/docs/source/user-guide/common-operations/windows.md +++ b/docs/source/user-guide/common-operations/windows.md @@ -208,7 +208,7 @@ The possible window functions are: You can ship custom window functions to the engine by subclassing [`WindowEvaluator`][datafusion.user_defined.WindowEvaluator] and registering it -via [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`](../../reference/user_defined.md) +via [`udwf`][datafusion.user_defined.udwf]. See [`user_defined`](../../reference/datafusion/user_defined.md) for the evaluator interface and worked examples.
diff --git a/mkdocs.yml b/mkdocs.yml index 2a5e0a9e8..92f839506 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -68,6 +68,10 @@ plugins: - https://docs.pola.rs/api/python/stable/objects.inv options: docstring_style: google + docstring_options: + warn_unknown_params: false + returns_named_value: false + returns_multiple_items: false show_source: false members_order: source inherited_members: true diff --git a/python/datafusion/catalog.py b/python/datafusion/catalog.py index b7c5dda25..3c862492f 100644 --- a/python/datafusion/catalog.py +++ b/python/datafusion/catalog.py @@ -239,7 +239,7 @@ class TableProviderFactory(ABC): @abstractmethod def create(self, cmd: CreateExternalTable) -> Table: - """Create a table using the [`CreateExternalTable`][CreateExternalTable].""" + """Create a table using the `CreateExternalTable`.""" ... diff --git a/python/datafusion/context.py b/python/datafusion/context.py index f0149f033..e1b4e245c 100644 --- a/python/datafusion/context.py +++ b/python/datafusion/context.py @@ -102,6 +102,18 @@ ) +__all__ = [ + "ArrowArrayExportable", + "ArrowStreamExportable", + "PhysicalOptimizerRuleExportable", + "RuntimeEnvBuilder", + "SQLOptions", + "SessionConfig", + "SessionContext", + "TableProviderExportable", +] + + class ArrowStreamExportable(Protocol): """Type hint for object exporting Arrow C Stream via Arrow PyCapsule Interface. @@ -340,7 +352,7 @@ def with_extension(self, extension: Any) -> SessionConfig: Args: extension: A custom configuration extension object. These are - shared from another DataFusion extension library. + shared from another DataFusion extension library. Returns: A new `SessionConfig` object with the updated setting. @@ -597,7 +609,7 @@ def register_object_store( Args: schema: The data source schema. - store: The [`ObjectStore`][datafusion.object_store.ObjectStore] to register. + store: The [object store][datafusion.object_store] to register. host: URL for the host. """ self.ctx.register_object_store(schema, store, host) @@ -623,7 +635,7 @@ def register_listing_table( """Register multiple files as a single table. Registers a [`Table`][datafusion.catalog.Table] that can assemble multiple - files from locations in an [`ObjectStore`][datafusion.object_store.ObjectStore] + files from locations in an [object store][datafusion.object_store] instance. Args: @@ -921,7 +933,9 @@ def register_table_provider( ) -> None: """Register a table provider. - Deprecated: use [`register_table`][register_table] instead. + Deprecated: use + [`register_table`][datafusion.context.SessionContext.register_table] + instead. """ self.register_table(name, provider) @@ -975,9 +989,11 @@ def register_record_batches( def read_batch(self, batch: pa.RecordBatch) -> DataFrame: """Return a `DataFrame` reading a single batch. - Convenience wrapper around [`read_batches`][read_batches] for the single-batch - case. Unlike [`register_batch`][register_batch], this does not register the - batch as a named table; it returns an anonymous + Convenience wrapper around + [`read_batches`][datafusion.context.SessionContext.read_batches] for the + single-batch case. Unlike + [`register_batch`][datafusion.context.SessionContext.register_batch], this + does not register the batch as a named table; it returns an anonymous [`DataFrame`][datafusion.dataframe.DataFrame] directly. Args: @@ -1326,8 +1342,9 @@ def deregister_udwf(self, name: str) -> None: def udf(self, name: str) -> ScalarUDF: """Look up a registered scalar UDF by name. - Returns the same ``ScalarUDF`` wrapper that [`register_udf`][register_udf] - accepts, so it can be invoked as an expression in the DataFrame API + Returns the same ``ScalarUDF`` wrapper that + [`register_udf`][datafusion.context.SessionContext.register_udf] accepts, + so it can be invoked as an expression in the DataFrame API or re-registered into a different `SessionContext`. Built-in scalar functions from the session's function registry are also looked up. @@ -1372,8 +1389,9 @@ def udf(self, name: str) -> ScalarUDF: def udaf(self, name: str) -> AggregateUDF: """Look up a registered aggregate UDF by name. - Returns the same ``AggregateUDF`` wrapper that [`register_udaf`][register_udaf] - accepts. Built-in aggregate functions such as ``sum`` or ``avg`` are + Returns the same ``AggregateUDF`` wrapper that + [`register_udaf`][datafusion.context.SessionContext.register_udaf] accepts. + Built-in aggregate functions such as ``sum`` or ``avg`` are also discoverable through this lookup. See `udf` for a worked late-binding example; the pattern is identical for aggregates. @@ -1402,8 +1420,9 @@ def udaf(self, name: str) -> AggregateUDF: def udwf(self, name: str) -> WindowUDF: """Look up a registered window UDF by name. - Returns the same ``WindowUDF`` wrapper that [`register_udwf`][register_udwf] - accepts. Built-in window functions such as ``row_number`` or ``rank`` + Returns the same ``WindowUDF`` wrapper that + [`register_udwf`][datafusion.context.SessionContext.register_udwf] accepts. + Built-in window functions such as ``row_number`` or ``rank`` are also discoverable through this lookup. See `udf` for a worked late-binding example; the pattern is identical for window functions. @@ -1618,13 +1637,13 @@ def add_physical_optimizer_rule( The rule is imported via its ``__datafusion_physical_optimizer_rule__`` PyCapsule, typically produced by a separate compiled extension. The - underlying [`SessionState`][SessionState] is rebuilt from its current state + underlying `SessionState` is rebuilt from its current state with the new rule appended, so previously registered tables, UDFs, and catalogs are preserved. Args: - rule: Object exposing ``__datafusion_physical_optimizer_rule__``, - a [`PhysicalOptimizerRuleExportable`][PhysicalOptimizerRuleExportable]. + rule: Object exposing ``__datafusion_physical_optimizer_rule__`` — a + [`PhysicalOptimizerRuleExportable`][datafusion.context.PhysicalOptimizerRuleExportable]. Examples: >>> from datafusion import SessionContext @@ -1782,7 +1801,7 @@ def read_parquet( schema: pa.Schema | None = None, file_sort_order: Sequence[Sequence[SortKey]] | None = None, ) -> DataFrame: - """Read a Parquet source into a [`Dataframe`][datafusion.dataframe.Dataframe]. + """Read a Parquet source into a [`Dataframe`][datafusion.dataframe.DataFrame]. Args: path: Path to the Parquet file. @@ -1921,7 +1940,8 @@ def read_empty(self) -> DataFrame: """Create an empty `DataFrame` with no columns or rows. See Also: - This is an alias for [`empty_table`][empty_table]. + This is an alias for + [`empty_table`][datafusion.context.SessionContext.empty_table]. """ return self.empty_table() diff --git a/python/datafusion/dataframe.py b/python/datafusion/dataframe.py index 5a74a1b40..6b24cdda7 100644 --- a/python/datafusion/dataframe.py +++ b/python/datafusion/dataframe.py @@ -88,6 +88,16 @@ from enum import Enum +__all__ = [ + "Compression", + "DataFrame", + "DataFrameWriteOptions", + "ExplainFormat", + "InsertOp", + "ParquetColumnOptions", + "ParquetWriterOptions", +] + class ExplainFormat(Enum): """Output format for explain plans. @@ -867,7 +877,7 @@ def sort(self, *exprs: SortKey) -> DataFrame: Note that any expression can be turned into a sort expression by calling its ``sort`` method. For ascending-only sorts, the shorter - [`sort_by`][sort_by] is usually more convenient. + [`sort_by`][datafusion.dataframe.DataFrame.sort_by] is usually more convenient. Args: exprs: Sort expressions or column names, applied in order. @@ -907,7 +917,9 @@ def limit(self, count: int, offset: int = 0) -> DataFrame: """Return a new `DataFrame` with a limited number of rows. Results are returned in unspecified order unless the DataFrame is - explicitly sorted first via [`sort`][sort] or [`sort_by`][sort_by]. + explicitly sorted first via + [`sort`][datafusion.dataframe.DataFrame.sort] or + [`sort_by`][datafusion.dataframe.DataFrame.sort_by]. Args: count: Number of rows to limit the DataFrame to. @@ -1066,7 +1078,7 @@ def join( When non-key columns share the same name in both DataFrames, use `col` on each DataFrame **before** the join to obtain fully qualified column references that can disambiguate them. - See [`join_on`][join_on] for an example. + See [`join_on`][datafusion.dataframe.DataFrame.join_on] for an example. Args: right: Other DataFrame to join with. @@ -1321,7 +1333,7 @@ def union_distinct(self, other: DataFrame) -> DataFrame: """Calculate the distinct union of two `DataFrame`. See Also: - [`union`][union] + [`union`][datafusion.dataframe.DataFrame.union] """ return self.union(other, distinct=True) @@ -1388,8 +1400,9 @@ def except_all(self, other: DataFrame, distinct: bool = False) -> DataFrame: def union_by_name(self, other: DataFrame, distinct: bool = False) -> DataFrame: """Union two `DataFrame` matching columns by name. - Unlike [`union`][union] which matches columns by position, this method - matches columns by their names, allowing DataFrames with different + Unlike [`union`][datafusion.dataframe.DataFrame.union] which matches + columns by position, this method matches columns by their names, + allowing DataFrames with different column orders to be combined. Args: @@ -1460,7 +1473,8 @@ def sort_by(self, *exprs: Expr | str) -> DataFrame: This is a convenience method that sorts the DataFrame by the given expressions in ascending order with nulls last. For more control over - sort direction and null ordering, use [`sort`][sort] instead. + sort direction and null ordering, use + [`sort`][datafusion.dataframe.DataFrame.sort] instead. Args: exprs: Expressions or column names to sort by. diff --git a/python/datafusion/dataframe_formatter.py b/python/datafusion/dataframe_formatter.py index 5495b7bf3..dfbdf6cba 100644 --- a/python/datafusion/dataframe_formatter.py +++ b/python/datafusion/dataframe_formatter.py @@ -32,6 +32,19 @@ from collections.abc import Callable +__all__ = [ + "CellFormatter", + "DataFrameHtmlFormatter", + "DefaultStyleProvider", + "FormatterManager", + "StyleProvider", + "configure_formatter", + "get_formatter", + "reset_formatter", + "set_formatter", +] + + def _validate_positive_int(value: Any, param_name: str) -> None: """Validate that a parameter is a positive integer. @@ -218,12 +231,12 @@ class DataFrameHtmlFormatter: max_rows: Maximum number of rows to display in repr output repr_rows: Deprecated alias for max_rows enable_cell_expansion: Whether to add expand/collapse buttons for long cell - values + values custom_css: Additional CSS to include in the HTML output show_truncation_message: Whether to display a message when data is truncated style_provider: Custom provider for cell and header styles use_shared_styles: Whether to load styles and scripts only once per notebook - session + session """ def __init__( @@ -343,8 +356,9 @@ def repr_rows(self) -> int: """Get the maximum number of rows (deprecated name). .. deprecated:: - Use [`max_rows`][max_rows] instead. This property is provided for - backward compatibility. + Use + [`max_rows`][datafusion.dataframe_formatter.DataFrameHtmlFormatter.max_rows] + instead. This property is provided for backward compatibility. Returns: The maximum number of rows to display @@ -356,8 +370,9 @@ def repr_rows(self, value: int) -> None: """Set the maximum number of rows using deprecated name. .. deprecated:: - Use [`max_rows`][max_rows] setter instead. This property is provided for - backward compatibility. + Use the + [`max_rows`][datafusion.dataframe_formatter.DataFrameHtmlFormatter.max_rows] + setter instead. This property is provided for backward compatibility. Args: value: The maximum number of rows diff --git a/python/datafusion/expr.py b/python/datafusion/expr.py index 167f8f15a..811372a8f 100644 --- a/python/datafusion/expr.py +++ b/python/datafusion/expr.py @@ -1126,7 +1126,7 @@ def initcap(self) -> Expr: def list_distinct(self) -> Expr: """Returns distinct values from the array after removing duplicates. - This is an alias for [`array_distinct`][array_distinct]. + This is an alias for [`array_distinct`][datafusion.functions.array_distinct]. """ from . import functions as F @@ -1309,7 +1309,7 @@ def atanh(self) -> Expr: def list_dims(self) -> Expr: """Returns an array of the array's dimensions. - This is an alias for [`array_dims`][array_dims]. + This is an alias for [`array_dims`][datafusion.functions.array_dims]. """ from . import functions as F @@ -1348,7 +1348,7 @@ def ceil(self) -> Expr: def list_length(self) -> Expr: """Returns the length of the array. - This is an alias for [`array_length`][array_length]. + This is an alias for [`array_length`][datafusion.functions.array_length]. """ from . import functions as F @@ -1405,7 +1405,7 @@ def char_length(self) -> Expr: def list_ndims(self) -> Expr: """Returns the number of dimensions of the array. - This is an alias for [`array_ndims`][array_ndims]. + This is an alias for [`array_ndims`][datafusion.functions.array_ndims]. """ from . import functions as F @@ -1430,7 +1430,7 @@ def sinh(self) -> Expr: return F.sinh(self) def empty(self) -> Expr: - """This is an alias for [`array_empty`][array_empty].""" + """This is an alias for [`array_empty`][datafusion.functions.array_empty].""" from . import functions as F return F.empty(self) @@ -1708,7 +1708,8 @@ def rollup(*exprs: Expr | str) -> Expr: [30, 30, 60] See Also: - [`cube`][cube], [`grouping_sets`][grouping_sets], + [`cube`][datafusion.expr.GroupingSet.cube], + [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets], [`grouping`][datafusion.functions.grouping] """ args = [_to_raw_expr(e) for e in exprs] @@ -1730,7 +1731,7 @@ def cube(*exprs: Expr | str) -> Expr: Examples: With a single column, ``cube`` behaves identically to - [`rollup`][rollup]: + [`rollup`][datafusion.expr.GroupingSet.rollup]: >>> from datafusion.expr import GroupingSet >>> ctx = dfn.SessionContext() @@ -1744,7 +1745,8 @@ def cube(*exprs: Expr | str) -> Expr: [30, 30, 60] See Also: - [`rollup`][rollup], [`grouping_sets`][grouping_sets], + [`rollup`][datafusion.expr.GroupingSet.rollup], + [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets], [`grouping`][datafusion.functions.grouping] """ args = [_to_raw_expr(e) for e in exprs] @@ -1787,7 +1789,8 @@ def grouping_sets(*expr_lists: list[Expr | str]) -> Expr: [3, 3, 4, 2] See Also: - [`rollup`][rollup], [`cube`][cube], + [`rollup`][datafusion.expr.GroupingSet.rollup], + [`cube`][datafusion.expr.GroupingSet.cube], [`grouping`][datafusion.functions.grouping] """ raw_lists = [[_to_raw_expr(e) for e in lst] for lst in expr_lists] diff --git a/python/datafusion/functions.py b/python/datafusion/functions.py index 1d372098f..c1e0ad18a 100644 --- a/python/datafusion/functions.py +++ b/python/datafusion/functions.py @@ -3225,7 +3225,7 @@ def list_distinct(array: Expr) -> Expr: """Returns distinct values from the array after removing duplicates. See Also: - This is an alias for [`array_distinct`][array_distinct]. + This is an alias for [`array_distinct`][datafusion.functions.array_distinct]. """ return array_distinct(array) @@ -3234,7 +3234,7 @@ def list_dims(array: Expr) -> Expr: """Returns an array of the array's dimensions. See Also: - This is an alias for [`array_dims`][array_dims]. + This is an alias for [`array_dims`][datafusion.functions.array_dims]. """ return array_dims(array) @@ -3271,7 +3271,7 @@ def list_empty(array: Expr) -> Expr: """Returns a boolean indicating whether the array is empty. See Also: - This is an alias for [`array_empty`][array_empty]. + This is an alias for [`array_empty`][datafusion.functions.array_empty]. """ return array_empty(array) @@ -3320,7 +3320,7 @@ def list_length(array: Expr) -> Expr: """Returns the length of the array. See Also: - This is an alias for [`array_length`][array_length]. + This is an alias for [`array_length`][datafusion.functions.array_length]. """ return array_length(array) @@ -3529,7 +3529,7 @@ def list_ndims(array: Expr) -> Expr: """Returns the number of dimensions of the array. See Also: - This is an alias for [`array_ndims`][array_ndims]. + This is an alias for [`array_ndims`][datafusion.functions.array_ndims]. """ return array_ndims(array) @@ -3607,7 +3607,7 @@ def list_pop_back(array: Expr) -> Expr: """Returns the array without the last element. See Also: - This is an alias for [`array_pop_back`][array_pop_back]. + This is an alias for [`array_pop_back`][datafusion.functions.array_pop_back]. """ return array_pop_back(array) @@ -3616,7 +3616,7 @@ def list_pop_front(array: Expr) -> Expr: """Returns the array without the first element. See Also: - This is an alias for [`array_pop_front`][array_pop_front]. + This is an alias for [`array_pop_front`][datafusion.functions.array_pop_front]. """ return array_pop_front(array) @@ -4263,7 +4263,7 @@ def empty(array: Expr) -> Expr: """Returns true if the array is empty. See Also: - This is an alias for [`array_empty`][array_empty]. + This is an alias for [`array_empty`][datafusion.functions.array_empty]. """ return array_empty(array) diff --git a/python/datafusion/input/base.py b/python/datafusion/input/base.py index f67dde2a1..52b170b12 100644 --- a/python/datafusion/input/base.py +++ b/python/datafusion/input/base.py @@ -25,6 +25,8 @@ from datafusion.common import SqlTable +__all__ = ["BaseInputSource"] + class BaseInputSource(ABC): """Base Input Source class. diff --git a/python/datafusion/input/location.py b/python/datafusion/input/location.py index 779d94d23..3390d0587 100644 --- a/python/datafusion/input/location.py +++ b/python/datafusion/input/location.py @@ -23,6 +23,8 @@ from datafusion.common import DataTypeMap, SqlTable from datafusion.input.base import BaseInputSource +__all__ = ["LocationInputPlugin"] + class LocationInputPlugin(BaseInputSource): """Input Plugin for everything. diff --git a/python/datafusion/io.py b/python/datafusion/io.py index 9e32b08a3..f85d586e3 100644 --- a/python/datafusion/io.py +++ b/python/datafusion/io.py @@ -34,6 +34,14 @@ from .options import CsvReadOptions +__all__ = [ + "read_avro", + "read_csv", + "read_json", + "read_parquet", +] + + def read_parquet( path: str | pathlib.Path, table_partition_cols: list[tuple[str, str | pa.DataType]] | None = None, @@ -43,7 +51,7 @@ def read_parquet( schema: pa.Schema | None = None, file_sort_order: list[list[Expr]] | None = None, ) -> DataFrame: - """Read a Parquet source into a [`Dataframe`][datafusion.dataframe.Dataframe]. + """Read a Parquet source into a [`DataFrame`][datafusion.dataframe.DataFrame]. This function will use the global context. Any functions or tables registered with another context may not be accessible when used with a DataFrame created diff --git a/python/datafusion/ipc.py b/python/datafusion/ipc.py index ed6389fca..1d23d3bf5 100644 --- a/python/datafusion/ipc.py +++ b/python/datafusion/ipc.py @@ -18,7 +18,7 @@ """Driver- and worker-side setup for distributing DataFusion expressions. When a [`Expr`][datafusion.expr.Expr] is shipped to a worker process (e.g. through -[`Pool`][multiprocessing.Pool] or a Ray actor), the worker reconstructs the +`Pool` or a Ray actor), the worker reconstructs the expression against a `SessionContext`. If the expression references UDFs imported via the FFI capsule protocol — or any UDF the worker would otherwise resolve from its registered functions rather than from inside @@ -42,7 +42,7 @@ def init_worker(): .. note:: Serialization model Expressions containing Python UDFs (scalar, aggregate, window) are - serialized using [`cloudpickle`][cloudpickle]. The callable itself travels + serialized using `cloudpickle`. The callable itself travels **by value** (bytecode and closure cells inlined), but any names the callable resolves via ``import`` are captured **by reference** and must be importable on the receiving worker. @@ -51,10 +51,11 @@ def init_worker(): ``(major, minor)`` version. Loading on a different minor version raises [`ValueError`][ValueError] with an actionable message — cloudpickle payloads are not portable across Python minor versions. See - [`to_bytes`][datafusion.Expr.to_bytes] for examples of what travels by + [`to_bytes`][datafusion.expr.Expr.to_bytes] for examples of what travels by value vs. by reference. -On the driver side, call [`set_sender_ctx`][set_sender_ctx] to control how +On the driver side, call +[`set_sender_ctx`][datafusion.ipc.set_sender_ctx] to control how [`dumps`][pickle.dumps] encodes expressions — for example, to apply `with_python_udf_inlining` to every pickled expression on this thread: @@ -77,12 +78,13 @@ def init_worker(): ``ctx``. The thread-local sender context holds a strong reference to the -installed `SessionContext` until [`clear_sender_ctx`][clear_sender_ctx] is -called or the thread exits. Long-running driver threads that install a sender -context once and never clear it will retain that session for the -lifetime of the thread; pair [`set_sender_ctx`][set_sender_ctx] with -[`clear_sender_ctx`][clear_sender_ctx] (e.g. in a ``try``/``finally``) when the -sender context is only needed for a bounded scope. +installed `SessionContext` until +[`clear_sender_ctx`][datafusion.ipc.clear_sender_ctx] is called or the thread +exits. Long-running driver threads that install a sender context once and never +clear it will retain that session for the lifetime of the thread; pair +[`set_sender_ctx`][datafusion.ipc.set_sender_ctx] with +[`clear_sender_ctx`][datafusion.ipc.clear_sender_ctx] (e.g. in a +``try``/``finally``) when the sender context is only needed for a bounded scope. """ from __future__ import annotations diff --git a/python/datafusion/plan.py b/python/datafusion/plan.py index 592bdbc97..474c584ba 100644 --- a/python/datafusion/plan.py +++ b/python/datafusion/plan.py @@ -235,8 +235,9 @@ def collect_metrics(self) -> list[tuple[str, MetricsSet]]: Each entry in the returned list corresponds to one operator that recorded metrics. The first element of the tuple is the operator's description string — the same text shown by - [`display_indent`][display_indent] — which identifies both the operator type - and its key parameters, for example ``"FilterExec: column1@0 > 1"`` + [`display_indent`][datafusion.plan.ExecutionPlan.display_indent] — which + identifies both the operator type and its key parameters, for example + ``"FilterExec: column1@0 > 1"`` or ``"DataSourceExec: partitions=1"``. Returns: @@ -263,10 +264,11 @@ class MetricsSet: """A set of metrics for a single execution plan operator. A physical plan operator runs independently across one or more partitions. - [`metrics`][metrics] returns the raw per-partition `Metric` objects. - The convenience properties (`output_rows`, [`elapsed_compute`][elapsed_compute], - etc.) automatically sum the named metric across *all* partitions, giving a - single aggregate value for the operator as a whole. + [`metrics`][datafusion.plan.MetricsSet.metrics] returns the raw per-partition + `Metric` objects. The convenience properties (`output_rows`, + [`elapsed_compute`][datafusion.plan.MetricsSet.elapsed_compute], etc.) + automatically sum the named metric across *all* partitions, giving a single + aggregate value for the operator as a whole. """ def __init__(self, raw: df_internal.MetricsSet) -> None: diff --git a/python/datafusion/record_batch.py b/python/datafusion/record_batch.py index afdeb3918..a24b80b9d 100644 --- a/python/datafusion/record_batch.py +++ b/python/datafusion/record_batch.py @@ -32,6 +32,9 @@ import datafusion._internal as df_internal +__all__ = ["RecordBatch", "RecordBatchStream"] + + class RecordBatch: """This class is essentially a wrapper for [`RecordBatch`][pyarrow.RecordBatch].""" @@ -79,7 +82,12 @@ def __init__(self, record_batch_stream: df_internal.RecordBatchStream) -> None: self.rbs = record_batch_stream def next(self) -> RecordBatch: - """See [`__next__`][__next__] for the iterator function.""" + """Return the next batch. + + See + [`__next__`][datafusion.record_batch.RecordBatchStream.__next__] for the + iterator function. + """ return next(self) async def __anext__(self) -> RecordBatch: diff --git a/python/datafusion/substrait.py b/python/datafusion/substrait.py index ffc6d10bb..6cbba8e2b 100644 --- a/python/datafusion/substrait.py +++ b/python/datafusion/substrait.py @@ -49,8 +49,8 @@ def __init__(self, plan: substrait_internal.Plan) -> None: """Create a substrait plan. The user should not have to call this constructor directly. Rather, it - should be created via [`Serde`][Serde] or py[`Producer`][Producer] classes - in this module. + should be created via [`Serde`][datafusion.substrait.Serde] or + [`Producer`][datafusion.substrait.Producer] classes in this module. """ self.plan_internal = plan diff --git a/python/datafusion/user_defined.py b/python/datafusion/user_defined.py index 07cb29349..93bf5e03a 100644 --- a/python/datafusion/user_defined.py +++ b/python/datafusion/user_defined.py @@ -37,6 +37,26 @@ from collections.abc import Callable, Sequence +__all__ = [ + "Accumulator", + "AggregateUDF", + "AggregateUDFExportable", + "LogicalExtensionCodecExportable", + "PhysicalExtensionCodecExportable", + "ScalarUDF", + "ScalarUDFExportable", + "TableFunction", + "Volatility", + "WindowEvaluator", + "WindowUDF", + "WindowUDFExportable", + "udaf", + "udf", + "udtf", + "udwf", +] + + class Volatility(Enum): """Defines how stable or volatile a function is. @@ -225,7 +245,7 @@ def udf( def udf(func: ScalarUDFExportable) -> ScalarUDF: ... @staticmethod - def udf(*args: Any, **kwargs: Any): # noqa: D417 + def udf(*args: Any, **kwargs: Any): """Create a new User-Defined Function (UDF). This class can be used both as either a function or a decorator. @@ -240,22 +260,24 @@ def udf(*args: Any, **kwargs: Any): # noqa: D417 When you do so, it will be assumed that the nullability of the inputs and output are True and that they have no metadata. - Args: - func (Callable, optional): Only needed when calling as a function. - Skip this argument when using `udf` as a decorator. If you have a Rust - backed ScalarUDF within a PyCapsule, you can pass this parameter - and ignore the rest. They will be determined directly from the - underlying function. See the online documentation for more information. - input_fields (list[pa.Field | pa.DataType]): The data types or Fields - of the arguments to ``func``. This list must be of the same length - as the number of arguments. - return_field (_R): The field of the return value from the function. - volatility (Volatility | str): See `Volatility` for allowed values. - name (Optional[str]): A descriptive name for the function. - - Returns: - A user-defined function that can be used in SQL expressions, - data aggregation, or window function calls. + **Parameters:** + + - `func` (`Callable`, optional): Only needed when calling as a function. + Skip this argument when using ``udf`` as a decorator. If you have a Rust + backed ScalarUDF within a PyCapsule, you can pass this parameter + and ignore the rest. They will be determined directly from the + underlying function. See the online documentation for more information. + - `input_fields` (`list[pa.Field | pa.DataType]`): The data types or Fields + of the arguments to ``func``. This list must be of the same length + as the number of arguments. + - `return_field` (`pa.Field | pa.DataType`): The field of the return value + from the function. + - `volatility` (`Volatility | str`): See + [`Volatility`][datafusion.user_defined.Volatility] for allowed values. + - `name` (`str`, optional): A descriptive name for the function. + + **Returns:** a user-defined function that can be used in SQL expressions, + data aggregation, or window function calls. Examples: Using ``udf`` as a function: @@ -530,7 +552,7 @@ def udaf(accum: AggregateUDFExportable) -> AggregateUDF: ... def udaf(accum: _PyCapsule) -> AggregateUDF: ... @staticmethod - def udaf(*args: Any, **kwargs: Any): # noqa: D417, C901 + def udaf(*args: Any, **kwargs: Any): # noqa: C901 """Create a new User-Defined Aggregate Function (UDAF). This class allows you to define an aggregate function that can be used in @@ -607,22 +629,23 @@ def udaf(*args: Any, **kwargs: Any): # noqa: D417, C901 ... "total")[0].as_py() 16.0 - Args: - accum: The accumulator python function. Only needed when calling as a - function. Skip this argument when using ``udaf`` as a decorator. - If you have a Rust backed AggregateUDF within a PyCapsule, you can - pass this parameter and ignore the rest. They will be determined - directly from the underlying function. See the online documentation - for more information. - input_types: The data types of the arguments to ``accum``. - return_type: The data type of the return value. - state_type: The data types of the intermediate accumulation. - volatility: See [`Volatility`][Volatility] for allowed values. - name: A descriptive name for the function. - - Returns: - A user-defined aggregate function, which can be used in either data - aggregation or window function calls. + **Parameters:** + + - `accum`: The accumulator python function. Only needed when calling as a + function. Skip this argument when using ``udaf`` as a decorator. + If you have a Rust backed AggregateUDF within a PyCapsule, you can + pass this parameter and ignore the rest. They will be determined + directly from the underlying function. See the online documentation + for more information. + - `input_types`: The data types of the arguments to ``accum``. + - `return_type`: The data type of the return value. + - `state_type`: The data types of the intermediate accumulation. + - `volatility`: See + [`Volatility`][datafusion.user_defined.Volatility] for allowed values. + - `name`: A descriptive name for the function. + + **Returns:** a user-defined aggregate function, which can be used in either + data aggregation or window function calls. """ # noqa: E501 W505 def _function( @@ -748,7 +771,7 @@ def get_range(self, idx: int, num_rows: int) -> tuple[int, int]: # noqa: ARG002 etc) Args: - idx:: Current index + idx: Current index num_rows: Number of rows. """ return (idx, idx + 1) @@ -952,7 +975,7 @@ def udwf( ) -> WindowUDF: ... @staticmethod - def udwf(*args: Any, **kwargs: Any): # noqa: D417 + def udwf(*args: Any, **kwargs: Any): """Create a new User-Defined Window Function (UDWF). This class can be used both as either a function or a decorator. @@ -1001,19 +1024,21 @@ def udwf(*args: Any, **kwargs: Any): # noqa: D417 >>> df.select(biased_numbers(col("a")).alias("result")).to_pydict() {'result': [10, 11, 12]} - Args: - func: Only needed when calling as a function. Skip this argument when - using ``udwf`` as a decorator. If you have a Rust backed WindowUDF - within a PyCapsule, you can pass this parameter and ignore the rest. - They will be determined directly from the underlying function. See - the online documentation for more information. - input_types: The data types of the arguments. - return_type: The data type of the return value. - volatility: See [`Volatility`][Volatility] for allowed values. - name: A descriptive name for the function. - - Returns: - A user-defined window function that can be used in window function calls. + **Parameters:** + + - `func`: Only needed when calling as a function. Skip this argument when + using ``udwf`` as a decorator. If you have a Rust backed WindowUDF + within a PyCapsule, you can pass this parameter and ignore the rest. + They will be determined directly from the underlying function. See + the online documentation for more information. + - `input_types`: The data types of the arguments. + - `return_type`: The data type of the return value. + - `volatility`: See + [`Volatility`][datafusion.user_defined.Volatility] for allowed values. + - `name`: A descriptive name for the function. + + **Returns:** a user-defined window function that can be used in window + function calls. """ if hasattr(args[0], "__datafusion_window_udf__"): return WindowUDF.from_pycapsule(args[0]) From 9b28a494366c85b7b38e6fb9581f131af4ac3198 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Mon, 8 Jun 2026 12:24:26 +0200 Subject: [PATCH 15/18] docs: fix unrecognized relative links across user guide and ffi page Replace trailing-slash URL-style links left over from the Sphinx site with explicit `.md` paths so mkdocs can resolve and validate them. Also point the DataFusion 52 upgrade-guide reference at the FFI contributor page (the bare `[ffi](ffi)` link no longer pointed anywhere). Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/source/contributor-guide/ffi.md | 6 +++--- docs/source/user-guide/common-operations/windows.md | 6 +++--- docs/source/user-guide/concepts.md | 2 +- docs/source/user-guide/data-sources.md | 6 +++--- docs/source/user-guide/dataframe/index.md | 2 +- docs/source/user-guide/sql.md | 2 +- docs/source/user-guide/upgrade-guides.md | 2 +- 7 files changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/source/contributor-guide/ffi.md b/docs/source/contributor-guide/ffi.md index cccb6a957..ea3ca5b29 100644 --- a/docs/source/contributor-guide/ffi.md +++ b/docs/source/contributor-guide/ffi.md @@ -28,9 +28,9 @@ when doing these integrations and the approach our project uses. ## The Primary Issue Suppose you wish to use DataFusion and you have a custom data source that can produce tables that -can then be queried against, similar to how you can register a [CSV](../../user-guide/io/csv/) or -[Parquet](../../user-guide/io/parquet/) file. In DataFusion terminology, you likely want to implement a -[Custom Table Provider](../../user-guide/io/table_provider/). In an effort to make your data source +can then be queried against, similar to how you can register a [CSV](../user-guide/io/csv.md) or +[Parquet](../user-guide/io/parquet.md) file. In DataFusion terminology, you likely want to implement a +[Custom Table Provider](../user-guide/io/table_provider.md). In an effort to make your data source as performant as possible and to utilize the features of DataFusion, you may decide to write your source in Rust and then expose it through [PyO3](https://pyo3.rs) as a Python library. diff --git a/docs/source/user-guide/common-operations/windows.md b/docs/source/user-guide/common-operations/windows.md index f121705da..008456679 100644 --- a/docs/source/user-guide/common-operations/windows.md +++ b/docs/source/user-guide/common-operations/windows.md @@ -65,7 +65,7 @@ print(df.select( ### Partitions A window function can take a list of `partition_by` columns similar to an -[Aggregation Function](../aggregations/). This will cause the window values to be evaluated +[Aggregation Function](aggregations.md). This will cause the window values to be evaluated independently for each of the partitions. In the example above, we found the rank of each Pokemon per `Type 1` partitions. We can see the first couple of each partition if we do the following: @@ -166,7 +166,7 @@ print(df.filter(col('"Type 1"') == lit("Bug")).select( ## Aggregate Functions -You can use any [Aggregation Function](../aggregations/) as a window function. Here +You can use any [Aggregation Function](aggregations.md) as a window function. Here is an example that shows how to compare each pokemons’s attack power with the average attack power in its `"Type 1"` using the [`avg`][datafusion.functions.avg] function. @@ -202,7 +202,7 @@ The possible window functions are: - [`lag`][datafusion.functions.lag] - [`lead`][datafusion.functions.lead] 3. Aggregate Functions - : - All [Aggregation Functions](../aggregations/) can be used as window functions. + : - All [Aggregation Functions](aggregations.md) can be used as window functions. ## User-Defined Window Functions diff --git a/docs/source/user-guide/concepts.md b/docs/source/user-guide/concepts.md index 4acc9a2e1..2cb896819 100644 --- a/docs/source/user-guide/concepts.md +++ b/docs/source/user-guide/concepts.md @@ -76,7 +76,7 @@ For more details on working with DataFrames, including visualization options and ## Expressions -The third statement uses [Expressions](../common-operations/expressions/) to build up a query definition. You can find +The third statement uses [Expressions](common-operations/expressions.md) to build up a query definition. You can find explanations for what the functions below do in the user documentation for [`col`][datafusion.col.col], [`lit`][datafusion.lit], [`round`][datafusion.functions.round], and [`alias`][datafusion.expr.Expr.alias]. diff --git a/docs/source/user-guide/data-sources.md b/docs/source/user-guide/data-sources.md index 6236b7291..168d447a0 100644 --- a/docs/source/user-guide/data-sources.md +++ b/docs/source/user-guide/data-sources.md @@ -24,8 +24,8 @@ DataFusion provides a wide variety of ways to get data into a DataFrame to perfo ## Local file -DataFusion has the ability to read from a variety of popular file formats, such as [Parquet](../io/parquet/), -[CSV](../io/csv/), [JSON](../io/json/), and [AVRO](../io/avro/). +DataFusion has the ability to read from a variety of popular file formats, such as [Parquet](io/parquet.md), +[CSV](io/csv.md), [JSON](io/json.md), and [AVRO](io/avro.md). ```python exec="1" source="material-block" result="text" session="data-sources" ctx = SessionContext() @@ -208,7 +208,7 @@ Features that are available in PyIceberg but not yet in Iceberg Rust will not be ## Custom Table Provider You can implement a custom Data Provider in Rust and expose it to DataFusion through the -the interface as describe in the [Custom Table Provider](../io/table_provider/) +the interface as describe in the [Custom Table Provider](io/table_provider.md) section. This is an advanced topic, but a [user example](https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example) is provided in the DataFusion repository. diff --git a/docs/source/user-guide/dataframe/index.md b/docs/source/user-guide/dataframe/index.md index 084f57fb3..c11448999 100644 --- a/docs/source/user-guide/dataframe/index.md +++ b/docs/source/user-guide/dataframe/index.md @@ -77,7 +77,7 @@ DataFrames can be created in several ways: ``` For detailed information about reading from different data sources, see the [I/O Guide](../io/index.md). -For custom data sources, see [io_custom_table_provider](../../io/table_provider/). +For custom data sources, see [io_custom_table_provider](../../user-guide/io/table_provider.md). ## Common DataFrame Operations diff --git a/docs/source/user-guide/sql.md b/docs/source/user-guide/sql.md index 6be197f6a..2409c5410 100644 --- a/docs/source/user-guide/sql.md +++ b/docs/source/user-guide/sql.md @@ -115,7 +115,7 @@ object. Those objects will be cast into a [PyArrow Scalar Value](https://arrow.apache.org/docs/python/generated/pyarrow.Scalar.html). Using `param_values` will rely on the SQL dialect you have configured -for your session. This can be set using the [configuration options](../configuration/) +for your session. This can be set using the [configuration options](configuration.md) of your [`SessionContext`][datafusion.context.SessionContext]. Similar to how [prepared statements](https://datafusion.apache.org/user-guide/sql/prepared_statements.html) work, these parameters are limited to places where you would pass in a diff --git a/docs/source/user-guide/upgrade-guides.md b/docs/source/user-guide/upgrade-guides.md index c8ea98493..d8c020963 100644 --- a/docs/source/user-guide/upgrade-guides.md +++ b/docs/source/user-guide/upgrade-guides.md @@ -91,7 +91,7 @@ let codec = unsafe { data.as_ref() }; ## DataFusion 52.0.0 -This version includes a major update to the [ffi](ffi) due to upgrades +This version includes a major update to the [ffi](../contributor-guide/ffi.md) due to upgrades to the [Foreign Function Interface](https://doc.rust-lang.org/nomicon/ffi.html). Users who contribute their own `CatalogProvider`, `SchemaProvider`, `TableProvider` or `TableFunction` via FFI must now provide access to a From 2ebd1e582cd66fd43f0ee5e8a22306ae331ce047 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Mon, 8 Jun 2026 12:37:22 +0200 Subject: [PATCH 16/18] docs: prefer DataFrame.show() over print() in user-guide examples Convert ~50 `print(df.)` calls in the user-guide pages to `df..show()`, which is the idiomatic way to display a DataFusion DataFrame and matches what users would actually write. Lines whose chain returns a non-DataFrame (`.schema()`, `.to_pandas()`) keep the explicit `print()`. Also strip `result="text"` from executable blocks that produce no output (variable assignments only) and hide any remaining empty markdown-exec result containers via a CSS rule so the page no longer shows distracting empty boxes after assignment-only examples. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/source/_static/theme_overrides.css | 7 +++ .../common-operations/aggregations.md | 58 +++++++++---------- .../common-operations/basic-info.md | 4 +- .../common-operations/expressions.md | 36 ++++++------ .../user-guide/common-operations/functions.md | 28 ++++----- .../user-guide/common-operations/joins.md | 2 +- .../common-operations/select-and-filter.md | 8 +-- .../common-operations/udf-and-udfa.md | 2 +- .../user-guide/common-operations/windows.md | 24 ++++---- 9 files changed, 88 insertions(+), 81 deletions(-) diff --git a/docs/source/_static/theme_overrides.css b/docs/source/_static/theme_overrides.css index a72b28bbb..88602898a 100644 --- a/docs/source/_static/theme_overrides.css +++ b/docs/source/_static/theme_overrides.css @@ -52,6 +52,13 @@ display: none !important; } +/* markdown-exec emits an empty `
` for executable + * code blocks that produce no stdout. Hide them so pages don't show + * distracting empty boxes after assignment-only examples. */ +.result:empty { + display: none; +} + /* Notebook code output (e.g. `df.show()` ASCII tables) often exceeds * the content width. Force a non-wrapping pre with horizontal scroll * so wide tables stay legible instead of wrapping mid-row. diff --git a/docs/source/user-guide/common-operations/aggregations.md b/docs/source/user-guide/common-operations/aggregations.md index ad0971747..1d4bb3dee 100644 --- a/docs/source/user-guide/common-operations/aggregations.md +++ b/docs/source/user-guide/common-operations/aggregations.md @@ -33,14 +33,14 @@ col_type_2 = col('"Type 2"') col_speed = col('"Speed"') col_attack = col('"Attack"') -print(df.aggregate( +df.aggregate( [col_type_1], [ f.approx_distinct(col_speed).alias("Count"), f.approx_median(col_speed).alias("Median Speed"), f.approx_percentile_cont(col_speed, 0.9).alias("90% Speed"), ], -)) +).show() ``` @@ -48,28 +48,28 @@ When `group_by` is `None` or an empty list, the aggregation is done over the who [`DataFrame`][datafusion.dataframe.DataFrame]. For grouping the `group_by` list must contain at least one column. ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [col_type_1], [ f.max(col_speed).alias("Max Speed"), f.avg(col_speed).alias("Avg Speed"), f.min(col_speed).alias("Min Speed"), ], -)) +).show() ``` More than one column can be used for grouping ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [col_type_1, col_type_2], [ f.max(col_speed).alias("Max Speed"), f.avg(col_speed).alias("Avg Speed"), f.min(col_speed).alias("Min Speed"), ], -)) +).show() ``` @@ -80,7 +80,7 @@ operation. These can also be overridden using the builder approach to setting an parameters. When you use the builder, you must call `build()` to finish. For example, these two expressions are equivalent. -```python exec="1" source="material-block" result="text" session="aggregations" +```python exec="1" source="material-block" session="aggregations" first_1 = f.first_value(col("a"), order_by=[col("a")]) first_2 = f.first_value(col("a")).order_by(col("a")).build() ``` @@ -94,14 +94,14 @@ sort the Pokemon by their attack in increasing order and take the first value, w Pokemon with the smallest attack value in each `Type 1`. ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [col('"Type 1"')], [ f.first_value( col('"Name"'), order_by=[col('"Attack"').sort(ascending=True)] ).alias("Smallest Attack") ], -)) +).show() ``` @@ -112,9 +112,9 @@ time each. Suppose we want to create an array of all of the `Type 2` for each `T Pokemon set. Since there will be many entries of `Type 2` we only one each distinct value. ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [col_type_1], [f.array_agg(col_type_2, distinct=True).alias("Type 2 List")] -)) +).show() ``` @@ -128,14 +128,14 @@ df.filter(col_type_2.is_not_null()).aggregate( [col_type_1], [f.array_agg(col_type_2, distinct=True).alias("Type 2 List")] ) -print(df.aggregate( +df.aggregate( [col_type_1], [ f.array_agg(col_type_2, distinct=True, filter=col_type_2.is_not_null()).alias( "Type 2 List" ) ], -)) +).show() ``` @@ -163,14 +163,14 @@ df.aggregate( ], ) -print(df.aggregate( +df.aggregate( [col_type_1], [ f.first_value( col_type_2, order_by=[col_attack], null_treatment=NullTreatment.IGNORE_NULLS ).alias("Lowest Attack Type 2") ], -)) +).show() ``` @@ -185,13 +185,13 @@ Filter takes a single expression. Suppose we want to find the speed values for only Pokemon that have low Attack values. ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [col_type_1], [ f.avg(col_speed).alias("Avg Speed All"), f.avg(col_speed, filter=col_attack < lit(50)).alias("Avg Speed Low Attack"), ], -)) +).show() ``` @@ -278,14 +278,14 @@ once: ```python exec="1" source="material-block" result="text" session="aggregations" from datafusion.expr import GroupingSet -print(df.aggregate( +df.aggregate( [GroupingSet.rollup(col_type_1)], [ f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed"), f.max(col_speed).alias("Max Speed"), ], -).sort(col_type_1.sort(ascending=True, nulls_first=True))) +).sort(col_type_1.sort(ascending=True, nulls_first=True)).show() ``` @@ -297,14 +297,14 @@ for that row and `1` when it is aggregated across. Use `.alias()` to give the column a readable name: ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [GroupingSet.rollup(col_type_1)], [ f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed"), f.grouping(col_type_1).alias("Is Total"), ], -).sort(col_type_1.sort(ascending=True, nulls_first=True))) +).sort(col_type_1.sort(ascending=True, nulls_first=True)).show() ``` @@ -315,13 +315,13 @@ With two columns the hierarchy becomes more apparent. `rollup(Type 1, Type 2)` p - one grand total row ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [GroupingSet.rollup(col_type_1, col_type_2)], [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], ).sort( col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True), -)) +).show() ``` @@ -336,13 +336,13 @@ For our Pokemon data, `cube(Type 1, Type 2)` gives us stats broken down by the t by `Type 1` alone, by `Type 2` alone, and a grand total — all in one query: ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [GroupingSet.cube(col_type_1, col_type_2)], [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], ).sort( col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True), -)) +).show() ``` @@ -359,13 +359,13 @@ For example, if we want only the per-`Type 1` totals and per-`Type 2` totals — full `(Type 1, Type 2)` detail rows or the grand total — we can ask for exactly that: ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [GroupingSet.grouping_sets([col_type_1], [col_type_2])], [f.count(col_speed).alias("Count"), f.avg(col_speed).alias("Avg Speed")], ).sort( col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True), -)) +).show() ``` @@ -373,7 +373,7 @@ Each row belongs to exactly one grouping level. The [`grouping`][datafusion.func function tells you which level each row comes from: ```python exec="1" source="material-block" result="text" session="aggregations" -print(df.aggregate( +df.aggregate( [GroupingSet.grouping_sets([col_type_1], [col_type_2])], [ f.count(col_speed).alias("Count"), @@ -384,7 +384,7 @@ print(df.aggregate( ).sort( col_type_1.sort(ascending=True, nulls_first=True), col_type_2.sort(ascending=True, nulls_first=True), -)) +).show() ``` diff --git a/docs/source/user-guide/common-operations/basic-info.md b/docs/source/user-guide/common-operations/basic-info.md index f4f7b49ab..967d44500 100644 --- a/docs/source/user-guide/common-operations/basic-info.md +++ b/docs/source/user-guide/common-operations/basic-info.md @@ -40,7 +40,7 @@ print(df) Use [`limit`][datafusion.dataframe.DataFrame.limit] to view the top rows of the frame: ```python exec="1" source="material-block" result="text" session="basic-info" -print(df.limit(2)) +df.limit(2).show() ``` @@ -62,5 +62,5 @@ print(df.to_pandas()) [`describe`][datafusion.dataframe.DataFrame.describe] shows a quick statistic summary of your data: ```python exec="1" source="material-block" result="text" session="basic-info" -print(df.describe()) +df.describe().show() ``` diff --git a/docs/source/user-guide/common-operations/expressions.md b/docs/source/user-guide/common-operations/expressions.md index 0e12bd654..79f6cc569 100644 --- a/docs/source/user-guide/common-operations/expressions.md +++ b/docs/source/user-guide/common-operations/expressions.md @@ -40,7 +40,7 @@ The type of the object passed to the [`lit`][datafusion.lit] function will be us In the following example we create expressions for the column named `color` and the literal scalar string `red`. The resultant variable `red_units` is itself also an expression. -```python exec="1" source="material-block" result="text" session="expressions" +```python exec="1" source="material-block" session="expressions" red_units = col("color") == lit("red") ``` @@ -51,7 +51,7 @@ When combining expressions that evaluate to a boolean value, you can combine the It is important to note that in order to combine these expressions, you *must* use bitwise operators. See the following examples for the and, or, and not operations. -```python exec="1" source="material-block" result="text" session="expressions" +```python exec="1" source="material-block" session="expressions" red_or_green_units = (col("color") == lit("red")) | (col("color") == lit("green")) heavy_red_units = (col("color") == lit("red")) & (col("weight") > lit(42)) not_red_units = ~(col("color") == lit("red")) @@ -71,7 +71,7 @@ from datafusion import col ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5, 6]]}) -print(df.select(col("a")[0].alias("a0"))) +df.select(col("a")[0].alias("a0")).show() ``` @@ -87,7 +87,7 @@ Starting in DataFusion 49.0.0 you can also create slices of array elements using slice syntax from Python. ```python exec="1" source="material-block" result="text" session="expressions" -print(df.select(col("a")[1:3].alias("second_two_elements"))) +df.select(col("a")[1:3].alias("second_two_elements")).show() ``` @@ -100,7 +100,7 @@ from datafusion.functions import array_empty ctx = SessionContext() df = ctx.from_pydict({"a": [[], [1, 2, 3]]}) -print(df.select(array_empty(col("a")).alias("is_empty"))) +df.select(array_empty(col("a")).alias("is_empty")).show() ``` @@ -115,7 +115,7 @@ from datafusion.functions import cardinality ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5, 6]]}) -print(df.select(cardinality(col("a")).alias("num_elements"))) +df.select(cardinality(col("a")).alias("num_elements")).show() ``` @@ -130,7 +130,7 @@ from datafusion.functions import array_cat ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3]], "b": [[4, 5, 6]]}) -print(df.select(array_cat(col("a"), col("b")).alias("concatenated_array"))) +df.select(array_cat(col("a"), col("b")).alias("concatenated_array")).show() ``` @@ -145,7 +145,7 @@ from datafusion.functions import array_repeat ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3]]}) -print(df.select(array_repeat(col("a"), literal(2)).alias("repeated_array"))) +df.select(array_repeat(col("a"), literal(2)).alias("repeated_array")).show() ``` @@ -171,7 +171,7 @@ ctx = SessionContext() df = ctx.from_pydict({"a": [[1, 2, 3], [4, 5]]}) df.select(f.array_transform(col("a"), lambda v: v * 2).alias("doubled")) df.select(f.array_filter(col("a"), lambda v: v > 2).alias("big_only")) -print(df.select(f.array_any_match(col("a"), lambda v: v > 3).alias("has_big"))) +df.select(f.array_any_match(col("a"), lambda v: v > 3).alias("has_big")).show() ``` @@ -184,7 +184,7 @@ If you need explicit control over parameter names, build the lambda with from datafusion import lit double_fn = f.lambda_(["v"], f.lambda_var("v") * lit(2)) -print(df.select(f.array_transform(col("a"), double_fn).alias("doubled"))) +df.select(f.array_transform(col("a"), double_fn).alias("doubled")).show() ``` @@ -231,9 +231,9 @@ df.filter(f.in_list(col("shipmode"), [lit("MAIL"), lit("SHIP")])) # Option 3: array_position / make_array. Useful when you already have the # set as an array column and want "is in that array" semantics. -print(df.filter( +df.filter( ~f.array_position(f.make_array(lit("MAIL"), lit("SHIP")), col("shipmode")).is_null() -)) +).show() ``` @@ -256,14 +256,14 @@ df = ctx.from_pydict( {"priority": ["1-URGENT", "2-HIGH", "3-MEDIUM", "5-LOW"]}, ) -print(df.select( +df.select( col("priority"), f.case(col("priority")) .when(lit("1-URGENT"), lit(1)) .when(lit("2-HIGH"), lit(1)) .otherwise(lit(0)) .alias("is_high_priority"), -)) +).show() ``` @@ -276,13 +276,13 @@ df = ctx.from_pydict( {"volume": [10.0, 20.0, 30.0], "supplier_id": [1, None, 2]}, ) -print(df.select( +df.select( col("volume"), col("supplier_id"), f.when(col("supplier_id").is_not_null(), col("volume")) .otherwise(lit(0.0)) .alias("attributed_volume"), -)) +).show() ``` @@ -303,7 +303,7 @@ Python dictionary style objects. This expects a string key as the parameter pass ctx = SessionContext() data = {"a": [{"size": 15, "color": "green"}, {"size": 10, "color": "blue"}]} df = ctx.from_pydict(data) -print(df.select(col("a")["size"].alias("a_size"))) +df.select(col("a")["size"].alias("a_size")).show() ``` @@ -336,5 +336,5 @@ started_young = start_age < lit(18) can_retire = age_col > lit(65) long_timer = started_young & can_retire -print(df.filter(long_timer).select(col("name"), renamed_age, col("years_in_position"))) +df.filter(long_timer).select(col("name"), renamed_age, col("years_in_position")).show() ``` diff --git a/docs/source/user-guide/common-operations/functions.md b/docs/source/user-guide/common-operations/functions.md index 272beae50..cef71e65e 100644 --- a/docs/source/user-guide/common-operations/functions.md +++ b/docs/source/user-guide/common-operations/functions.md @@ -24,7 +24,7 @@ In here we will cover some of the more popular use cases. If you want to view al We'll use the pokemon dataset in the following examples. -```python exec="1" source="material-block" result="text" session="functions" +```python exec="1" source="material-block" session="functions" ctx = SessionContext() ctx.register_csv("pokemon", "pokemon.csv") df = ctx.table("pokemon") @@ -38,9 +38,9 @@ DataFusion offers mathematical functions such as [`pow`][datafusion.functions.po ```python exec="1" source="material-block" result="text" session="functions" from datafusion import str_lit, string_literal -print(df.select( +df.select( f.pow(col('"Attack"'), literal(2)) - f.pow(col('"Defense"'), literal(2)) -).limit(10)) +).limit(10).show() ``` @@ -49,7 +49,7 @@ print(df.select( There 3 conditional functions in DataFusion [`coalesce`][datafusion.functions.coalesce], [`nullif`][datafusion.functions.nullif] and [`case`][datafusion.functions.case]. ```python exec="1" source="material-block" result="text" session="functions" -print(df.select(f.coalesce(col('"Type 1"'), col('"Type 2"')).alias("dominant_type")).limit(10)) +df.select(f.coalesce(col('"Type 1"'), col('"Type 2"')).alias("dominant_type")).limit(10).show() ``` @@ -58,24 +58,24 @@ print(df.select(f.coalesce(col('"Type 1"'), col('"Type 2"')).alias("dominant_typ For selecting the current time use [`now`][datafusion.functions.now] ```python exec="1" source="material-block" result="text" session="functions" -print(df.select(f.now())) +df.select(f.now()).show() ``` Convert to timestamps using [`to_timestamp`][datafusion.functions.to_timestamp] ```python exec="1" source="material-block" result="text" session="functions" -print(df.select(f.to_timestamp(col('"Total"')).alias("timestamp"))) +df.select(f.to_timestamp(col('"Total"')).alias("timestamp")).show() ``` Extracting parts of a date using [`date_part`][datafusion.functions.date_part] (alias [`extract`][datafusion.functions.extract]) ```python exec="1" source="material-block" result="text" session="functions" -print(df.select( +df.select( f.date_part(literal("month"), f.to_timestamp(col('"Total"'))).alias("month"), f.extract(literal("day"), f.to_timestamp(col('"Total"'))).alias("day"), -)) +).show() ``` @@ -85,21 +85,21 @@ In the field of data science, working with textual data is a common task. To mak DataFusion offers a range of helpful options. ```python exec="1" source="material-block" result="text" session="functions" -print(df.select( +df.select( f.char_length(col('"Name"')).alias("len"), f.lower(col('"Name"')).alias("lower"), f.left(col('"Name"'), literal(4)).alias("code"), -)) +).show() ``` This also includes the functions for regular expressions like [`regexp_replace`][datafusion.functions.regexp_replace] and [`regexp_match`][datafusion.functions.regexp_match] ```python exec="1" source="material-block" result="text" session="functions" -print(df.select( +df.select( f.regexp_match(col('"Name"'), literal("Char")).alias("dragons"), f.regexp_replace(col('"Name"'), literal("saur"), literal("fleur")).alias("flowers"), -)) +).show() ``` @@ -108,10 +108,10 @@ print(df.select( Casting expressions to different data types using [`arrow_cast`][datafusion.functions.arrow_cast] ```python exec="1" source="material-block" result="text" session="functions" -print(df.select( +df.select( f.arrow_cast(col('"Total"'), string_literal("Float64")).alias("total_as_float"), f.arrow_cast(col('"Total"'), str_lit("Int32")).alias("total_as_int"), -)) +).show() ``` diff --git a/docs/source/user-guide/common-operations/joins.md b/docs/source/user-guide/common-operations/joins.md index 6318da06d..4f38005e7 100644 --- a/docs/source/user-guide/common-operations/joins.md +++ b/docs/source/user-guide/common-operations/joins.md @@ -30,7 +30,7 @@ DataFusion supports the following join variants via the method [`join`][datafusi For the examples in this section we'll use the following two DataFrames -```python exec="1" source="material-block" result="text" session="joins" +```python exec="1" source="material-block" session="joins" ctx = SessionContext() left = ctx.from_pydict( diff --git a/docs/source/user-guide/common-operations/select-and-filter.md b/docs/source/user-guide/common-operations/select-and-filter.md index 170ebdc83..32d762b90 100644 --- a/docs/source/user-guide/common-operations/select-and-filter.md +++ b/docs/source/user-guide/common-operations/select-and-filter.md @@ -28,7 +28,7 @@ which you can download [here](https://d37ci6vzurychx.cloudfront.net/trip-data/ye ```python exec="1" source="material-block" result="text" session="select-and-filter" ctx = SessionContext() df = ctx.read_parquet("yellow_tripdata_2021-01.parquet") -print(df.select("trip_distance", "passenger_count")) +df.select("trip_distance", "passenger_count").show() ``` @@ -36,7 +36,7 @@ For mathematical or logical operations use [`col`][datafusion.col.col] to select operations using [`alias`][datafusion.expr.Expr.alias] ```python exec="1" source="material-block" result="text" session="select-and-filter" -print(df.select((col("tip_amount") + col("tolls_amount")).alias("tips_plus_tolls"))) +df.select((col("tip_amount") + col("tolls_amount")).alias("tips_plus_tolls")).show() ``` @@ -52,7 +52,7 @@ column selection use [`select`][datafusion.dataframe.DataFrame.select] without d For selecting columns with capital letters use `'"VendorID"'` ```python exec="1" source="material-block" result="text" session="select-and-filter" -print(df.select(col('"VendorID"'))) +df.select(col('"VendorID"')).show() ``` @@ -61,5 +61,5 @@ To combine it with literal values use the [`lit`][datafusion.lit] ```python exec="1" source="material-block" result="text" session="select-and-filter" large_trip_distance = col("trip_distance") > lit(5.0) low_passenger_count = col("passenger_count") < lit(4) -print(df.select((large_trip_distance & low_passenger_count).alias("lonely_trips"))) +df.select((large_trip_distance & low_passenger_count).alias("lonely_trips")).show() ``` diff --git a/docs/source/user-guide/common-operations/udf-and-udfa.md b/docs/source/user-guide/common-operations/udf-and-udfa.md index 618a68adc..11fac828b 100644 --- a/docs/source/user-guide/common-operations/udf-and-udfa.md +++ b/docs/source/user-guide/common-operations/udf-and-udfa.md @@ -131,7 +131,7 @@ ways: first with a native expression, then with a UDF that computes the same result. The filter itself is simple on purpose so we can compare the plans side by side. -```python exec="1" source="material-block" result="text" session="udf-and-udfa" +```python exec="1" source="material-block" session="udf-and-udfa" import os import tempfile diff --git a/docs/source/user-guide/common-operations/windows.md b/docs/source/user-guide/common-operations/windows.md index 008456679..125c9f120 100644 --- a/docs/source/user-guide/common-operations/windows.md +++ b/docs/source/user-guide/common-operations/windows.md @@ -28,7 +28,7 @@ The window functions are available in the [`functions`][datafusion.functions] mo We'll use the pokemon dataset (from Ritchie Vink) in the following examples. -```python exec="1" source="material-block" result="text" session="windows" +```python exec="1" source="material-block" session="windows" ctx = SessionContext() df = ctx.read_csv("pokemon.csv") ``` @@ -38,7 +38,7 @@ Here is an example that shows how you can compare each pokemon's speed to the sp previous row in the DataFrame. ```python exec="1" source="material-block" result="text" session="windows" -print(df.select(col('"Name"'), col('"Speed"'), f.lag(col('"Speed"')).alias("Previous Speed"))) +df.select(col('"Name"'), col('"Speed"'), f.lag(col('"Speed"')).alias("Previous Speed")).show() ``` @@ -50,7 +50,7 @@ You can control the order in which rows are processed by window functions by pro a list of `order_by` functions for the `order_by` parameter. ```python exec="1" source="material-block" result="text" session="windows" -print(df.select( +df.select( col('"Name"'), col('"Attack"'), col('"Type 1"'), @@ -58,7 +58,7 @@ print(df.select( partition_by=[col('"Type 1"')], order_by=[col('"Attack"').sort(ascending=True)], ).alias("rank"), -).sort(col('"Type 1"'), col('"Attack"'))) +).sort(col('"Type 1"'), col('"Attack"')).show() ``` @@ -71,7 +71,7 @@ Pokemon per `Type 1` partitions. We can see the first couple of each partition i the following: ```python exec="1" source="material-block" result="text" session="windows" -print(df.select( +df.select( col('"Name"'), col('"Attack"'), col('"Type 1"'), @@ -79,7 +79,7 @@ print(df.select( partition_by=[col('"Type 1"')], order_by=[col('"Attack"').sort(ascending=True)], ).alias("rank"), -).filter(col("rank") < lit(3)).sort(col('"Type 1"'), col("rank"))) +).filter(col("rank") < lit(3)).sort(col('"Type 1"'), col("rank")).show() ``` @@ -112,13 +112,13 @@ two preceding rows. ```python exec="1" source="material-block" result="text" session="windows" from datafusion.expr import Window, WindowFrame -print(df.select( +df.select( col('"Name"'), col('"Speed"'), f.avg(col('"Speed"')) .over(Window(window_frame=WindowFrame("rows", 2, 0), order_by=[col('"Speed"')])) .alias("Previous Speed"), -)) +).show() ``` @@ -139,7 +139,7 @@ it's `Type 2` column that are null. ```python exec="1" source="material-block" result="text" session="windows" from datafusion.common import NullTreatment -print(df.filter(col('"Type 1"') == lit("Bug")).select( +df.filter(col('"Type 1"') == lit("Bug")).select( '"Name"', '"Type 2"', f.last_value(col('"Type 2"')) @@ -160,7 +160,7 @@ print(df.filter(col('"Type 1"') == lit("Bug")).select( ) ) .alias("last_with_null"), -)) +).show() ``` @@ -171,7 +171,7 @@ is an example that shows how to compare each pokemons’s attack power with the power in its `"Type 1"` using the [`avg`][datafusion.functions.avg] function. ```python exec="1" source="material-block" result="text" session="windows" -print(df.select( +df.select( col('"Name"'), col('"Attack"'), col('"Type 1"'), @@ -183,7 +183,7 @@ print(df.select( ) ) .alias("Average Attack"), -)) +).show() ``` From 03ab28df4aadd9a1f74802b780a7187e5df8ada0 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Mon, 8 Jun 2026 12:38:20 +0200 Subject: [PATCH 17/18] Add test to ensure documentation site coverage --- python/tests/test_docs_coverage.py | 111 +++++++++++++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 python/tests/test_docs_coverage.py diff --git a/python/tests/test_docs_coverage.py b/python/tests/test_docs_coverage.py new file mode 100644 index 000000000..2c6929018 --- /dev/null +++ b/python/tests/test_docs_coverage.py @@ -0,0 +1,111 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Drift guards between the public API surface and the mkdocs reference site. + +Two checks: + +1. Every symbol in ``datafusion.__all__`` is covered by an ``mkdocstrings`` + ``:::`` directive somewhere under ``docs/source/reference/``. Coverage may + be direct (``::: datafusion.dataframe.DataFrame``) or by whole-module + autodoc (``::: datafusion.functions``). +2. Every ``:::`` directive in the reference pages resolves to a real, + importable Python object. Renames or removals that orphan a stub fail the + suite instead of silently producing an empty doc page. +""" + +from __future__ import annotations + +import importlib +import inspect +import re +from pathlib import Path + +import datafusion + +REF_DIR = Path(__file__).resolve().parents[2] / "docs" / "source" / "reference" +_DIRECTIVE_RE = re.compile(r"^:::\s+(\S+)\s*$", re.MULTILINE) + + +def _all_directives() -> set[str]: + paths: set[str] = set() + for md in REF_DIR.rglob("*.md"): + paths.update(_DIRECTIVE_RE.findall(md.read_text())) + return paths + + +def _is_covered(qual: str, directives: set[str]) -> bool: + if qual in directives: + return True + parent = qual.rsplit(".", 1)[0] if "." in qual else None + return parent in directives if parent else False + + +def test_public_api_documented() -> None: + directives = _all_directives() + assert directives, f"no ::: directives found under {REF_DIR}" + + missing: list[str] = [] + for name in datafusion.__all__: + obj = getattr(datafusion, name) + if inspect.ismodule(obj): + mod_name = obj.__name__ + if mod_name in directives or any( + d.startswith(mod_name + ".") for d in directives + ): + continue + missing.append(f"{name} (module {mod_name})") + continue + + module = getattr(obj, "__module__", None) or "datafusion" + qual = f"{module}.{name}" + if _is_covered(qual, directives) or module in directives: + continue + missing.append(f"{name} (expected '::: {qual}' or '::: {module}')") + + assert not missing, ( + "Public API symbols missing from docs/source/reference/*.md:\n " + + "\n ".join(missing) + ) + + +def test_directive_targets_resolve() -> None: + bad: list[str] = [] + for path in sorted(_all_directives()): + parts = path.split(".") + obj = None + remainder: list[str] = [] + for i in range(len(parts), 0, -1): + try: + obj = importlib.import_module(".".join(parts[:i])) + remainder = parts[i:] + break + except ImportError: + continue + if obj is None: + bad.append(f"{path} (no importable prefix)") + continue + try: + for attr in remainder: + obj = getattr(obj, attr) + except AttributeError: + bad.append(f"{path} (attribute chain broken)") + + assert not bad, ( + "Doc ::: directives reference symbols that no longer exist:\n " + + "\n ".join(bad) + ) From 0a67654f5e9bec4ba31ac2f406eac7cc9065c6c0 Mon Sep 17 00:00:00 2001 From: Tim Saucer Date: Mon, 8 Jun 2026 13:02:11 +0200 Subject: [PATCH 18/18] docs: use sphinx cross-ref roles in docstrings for IDE rendering JetBrains and other IDEs do not understand mkdocstrings autoref syntax (`[name][path]`) and display it as literal text in docstring hovers. Switch docstring cross-references to sphinx-style roles (:func:, :class:, :meth:, :attr:, :mod:, :exc:) which IDEs render natively as clickable links. A new griffe extension (`docs/griffe_extensions.py`) rewrites the sphinx roles back into mkdocstrings autorefs before mkdocstrings parses each docstring, so the docs site continues to produce working cross-reference links. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/griffe_extensions.py | 59 +++++++ mkdocs.yml | 2 + python/datafusion/__init__.py | 4 +- python/datafusion/catalog.py | 2 +- python/datafusion/context.py | 90 +++++----- python/datafusion/dataframe.py | 96 +++++----- python/datafusion/dataframe_formatter.py | 4 +- python/datafusion/expr.py | 98 +++++----- python/datafusion/functions.py | 216 +++++++++++------------ python/datafusion/io.py | 2 +- python/datafusion/ipc.py | 18 +- python/datafusion/plan.py | 20 +-- python/datafusion/record_batch.py | 10 +- python/datafusion/substrait.py | 4 +- python/datafusion/user_defined.py | 32 ++-- 15 files changed, 359 insertions(+), 298 deletions(-) create mode 100644 docs/griffe_extensions.py diff --git a/docs/griffe_extensions.py b/docs/griffe_extensions.py new file mode 100644 index 000000000..aef19250e --- /dev/null +++ b/docs/griffe_extensions.py @@ -0,0 +1,59 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Griffe extensions for datafusion-python docs. + +`SphinxRefsToAutorefs` rewrites sphinx-style cross-reference roles +(``:func:`~path`, :class:`~path``, etc.) inside docstrings into +mkdocstrings autoref syntax (``[`tail`][path]``) so that the same +docstring renders as a clickable cross-reference both in JetBrains-style +IDEs (which understand sphinx roles) and on the published docs site +(which understands mkdocstrings autorefs). +""" + +from __future__ import annotations + +import re +from typing import Any + +from griffe import Extension, Object + +_ROLE_RE = re.compile( + r":(?:py:)?(?Pfunc|class|meth|attr|mod|obj|exc|const|data)" + r":`(?P~?)(?P[\w.]+)`" +) + + +def _rewrite(text: str) -> str: + def repl(match: re.Match[str]) -> str: + target = match.group("target") + tail = target.rsplit(".", 1)[-1] + return f"[`{tail}`][{target}]" + + return _ROLE_RE.sub(repl, text) + + +class SphinxRefsToAutorefs(Extension): + """Convert sphinx-style cross-references into mkdocstrings autorefs.""" + + def on_object(self, *, obj: Object, **_: Any) -> None: + docstring = obj.docstring + if docstring is None: + return + new = _rewrite(docstring.value) + if new != docstring.value: + docstring.value = new diff --git a/mkdocs.yml b/mkdocs.yml index 92f839506..6e0e50d28 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -67,6 +67,8 @@ plugins: - https://arrow.apache.org/docs/objects.inv - https://docs.pola.rs/api/python/stable/objects.inv options: + extensions: + - docs/griffe_extensions.py:SphinxRefsToAutorefs docstring_style: google docstring_options: warn_unknown_params: false diff --git a/python/datafusion/__init__.py b/python/datafusion/__init__.py index 889d3898b..3f6aed590 100644 --- a/python/datafusion/__init__.py +++ b/python/datafusion/__init__.py @@ -27,10 +27,10 @@ - **SessionContext** -- entry point for loading data, running SQL, and creating DataFrames. - **DataFrame** -- lazy query builder. Every method returns a new DataFrame; - call [`collect`][datafusion.dataframe.DataFrame.collect] or a ``to_*`` + call :meth:`~datafusion.dataframe.DataFrame.collect` or a ``to_*`` method to execute. - **Expr** -- expression tree node for column references, literals, and function - calls. Build with [`col`][datafusion.col.col] and [`lit`][datafusion.lit]. + calls. Build with :func:`~datafusion.col.col` and :func:`~datafusion.lit`. - **functions** -- 290+ built-in scalar, aggregate, and window functions. Examples: diff --git a/python/datafusion/catalog.py b/python/datafusion/catalog.py index 3c862492f..30f6fee11 100644 --- a/python/datafusion/catalog.py +++ b/python/datafusion/catalog.py @@ -220,7 +220,7 @@ def __repr__(self) -> str: @staticmethod @deprecated("Use Table() constructor instead.") def from_dataset(dataset: pa.dataset.Dataset) -> Table: - """Turn a `dataset` ``Dataset`` into a [`Table`][datafusion.catalog.Table].""" + """Turn a `dataset` ``Dataset`` into a :class:`~datafusion.catalog.Table`.""" return Table(dataset) @property diff --git a/python/datafusion/context.py b/python/datafusion/context.py index e1b4e245c..52ee3811e 100644 --- a/python/datafusion/context.py +++ b/python/datafusion/context.py @@ -20,17 +20,17 @@ A `SessionContext` holds registered tables, catalogs, and configuration for the current session. It is the first object most programs create: from it you register data, run SQL strings -([`sql`][datafusion.context.SessionContext.sql]), read files -([`read_csv`][datafusion.context.SessionContext.read_csv], -[`read_parquet`][datafusion.context.SessionContext.read_parquet], ...), and construct -[`DataFrame`][datafusion.dataframe.DataFrame] objects in memory -([`from_pydict`][datafusion.context.SessionContext.from_pydict], -[`from_arrow`][datafusion.context.SessionContext.from_arrow]). +(:meth:`~datafusion.context.SessionContext.sql`), read files +(:meth:`~datafusion.context.SessionContext.read_csv`, +:meth:`~datafusion.context.SessionContext.read_parquet`, ...), and construct +:class:`~datafusion.dataframe.DataFrame` objects in memory +(:meth:`~datafusion.context.SessionContext.from_pydict`, +:meth:`~datafusion.context.SessionContext.from_arrow`). Session behavior (memory limits, batch size, configured optimizer passes, -...) is controlled by [`SessionConfig`][datafusion.context.SessionConfig] and +...) is controlled by :class:`~datafusion.context.SessionConfig` and `RuntimeEnvBuilder`; SQL dialect limits are controlled by -[`SQLOptions`][datafusion.context.SQLOptions]. +:class:`~datafusion.context.SQLOptions`. Examples: >>> ctx = dfn.SessionContext() @@ -609,7 +609,7 @@ def register_object_store( Args: schema: The data source schema. - store: The [object store][datafusion.object_store] to register. + store: The :mod:`~datafusion.object_store` to register. host: URL for the host. """ self.ctx.register_object_store(schema, store, host) @@ -634,8 +634,8 @@ def register_listing_table( ) -> None: """Register multiple files as a single table. - Registers a [`Table`][datafusion.catalog.Table] that can assemble multiple - files from locations in an [object store][datafusion.object_store] + Registers a :class:`~datafusion.catalog.Table` that can assemble multiple + files from locations in an :mod:`~datafusion.object_store` instance. Args: @@ -667,7 +667,7 @@ def sql( param_values: dict[str, Any] | None = None, **named_params: Any, ) -> DataFrame: - """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from SQL query text. + """Create a :class:`~datafusion.dataframe.DataFrame` from SQL query text. See the online documentation for a description of how to perform parameterized substitution via either the ``param_values`` option @@ -676,7 +676,7 @@ def sql( Note: This API implements DDL statements such as ``CREATE TABLE`` and ``CREATE VIEW`` and DML statements such as ``INSERT INTO`` with in-memory default implementation.See - [`sql_with_options`][datafusion.context.SessionContext.sql_with_options]. + :meth:`~datafusion.context.SessionContext.sql_with_options`. Args: query: SQL query text. @@ -732,7 +732,7 @@ def sql_with_options( param_values: dict[str, Any] | None = None, **named_params: Any, ) -> DataFrame: - """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from SQL query text. + """Create a :class:`~datafusion.dataframe.DataFrame` from SQL query text. This function will first validate that the query is allowed by the provided options. @@ -760,7 +760,7 @@ def create_dataframe( """Create and return a dataframe using the provided partitions. Args: - partitions: [`RecordBatch`][pyarrow.RecordBatch] partitions to register. + partitions: :class:`~pyarrow.RecordBatch` partitions to register. name: Resultant dataframe name. schema: Schema for the partitions. @@ -770,7 +770,7 @@ def create_dataframe( return DataFrame(self.ctx.create_dataframe(partitions, name, schema)) def create_dataframe_from_logical_plan(self, plan: LogicalPlan) -> DataFrame: - """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from an existing plan. + """Create a :class:`~datafusion.dataframe.DataFrame` from an existing plan. Args: plan: Logical plan. @@ -783,7 +783,7 @@ def create_dataframe_from_logical_plan(self, plan: LogicalPlan) -> DataFrame: def from_pylist( self, data: list[dict[str, Any]], name: str | None = None ) -> DataFrame: - """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from a list. + """Create a :class:`~datafusion.dataframe.DataFrame` from a list. Args: data: List of dictionaries. @@ -797,7 +797,7 @@ def from_pylist( def from_pydict( self, data: dict[str, list[Any]], name: str | None = None ) -> DataFrame: - """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from a dictionary. + """Create a :class:`~datafusion.dataframe.DataFrame` from a dictionary. Args: data: Dictionary of lists. @@ -813,7 +813,7 @@ def from_arrow( data: ArrowStreamExportable | ArrowArrayExportable, name: str | None = None, ) -> DataFrame: - """Create a [`DataFrame`][datafusion.dataframe.DataFrame] from an Arrow source. + """Create a :class:`~datafusion.dataframe.DataFrame` from an Arrow source. The Arrow data source can be any object that implements either ``__arrow_c_stream__`` or ``__arrow_c_array__``. For the latter, it must return @@ -857,7 +857,7 @@ def from_polars(self, data: pl.DataFrame, name: str | None = None) -> DataFrame: # https://github.com/apache/datafusion-python/pull/1016#discussion_r1983239116 # is the discussion on how we arrived at adding register_view def register_view(self, name: str, df: DataFrame) -> None: - """Register a [`DataFrame`][datafusion.dataframe.DataFrame] as a view. + """Register a :class:`~datafusion.dataframe.DataFrame` as a view. Args: name (str): The name to register the view under. @@ -871,7 +871,7 @@ def register_table( name: str, table: Table | TableProviderExportable | DataFrame | pa.dataset.Dataset, ) -> None: - """Register a [`Table`][datafusion.catalog.Table] with this context. + """Register a :class:`~datafusion.catalog.Table` with this context. The registered table can be referenced from SQL statements executed against this context. @@ -934,7 +934,7 @@ def register_table_provider( """Register a table provider. Deprecated: use - [`register_table`][datafusion.context.SessionContext.register_table] + :meth:`~datafusion.context.SessionContext.register_table` instead. """ self.register_table(name, provider) @@ -944,7 +944,7 @@ def register_udtf(self, func: TableFunction) -> None: self.ctx.register_udtf(func._udtf) def register_batch(self, name: str, batch: pa.RecordBatch) -> None: - """Register a single [`RecordBatch`][pyarrow.RecordBatch] as a table. + """Register a single :class:`~pyarrow.RecordBatch` as a table. Args: name: Name of the resultant table. @@ -990,11 +990,11 @@ def read_batch(self, batch: pa.RecordBatch) -> DataFrame: """Return a `DataFrame` reading a single batch. Convenience wrapper around - [`read_batches`][datafusion.context.SessionContext.read_batches] for the + :meth:`~datafusion.context.SessionContext.read_batches` for the single-batch case. Unlike - [`register_batch`][datafusion.context.SessionContext.register_batch], this + :meth:`~datafusion.context.SessionContext.register_batch`, this does not register the batch as a named table; it returns an anonymous - [`DataFrame`][datafusion.dataframe.DataFrame] directly. + :class:`~datafusion.dataframe.DataFrame` directly. Args: batch: Record batch to wrap as a DataFrame. @@ -1011,11 +1011,11 @@ def read_batches(self, batches: Iterable[pa.RecordBatch]) -> DataFrame: """Return a `DataFrame` reading the given batches. All batches must share the same schema. Any iterable of - [`RecordBatch`][pyarrow.RecordBatch] is accepted (list, tuple, generator); + :class:`~pyarrow.RecordBatch` is accepted (list, tuple, generator); it is materialized into a list before being handed to the underlying Rust binding. Unlike `register_record_batches`, this does not register the batches as a named table; it returns - an anonymous [`DataFrame`][datafusion.dataframe.DataFrame] directly. + an anonymous :class:`~datafusion.dataframe.DataFrame` directly. Args: batches: Record batches to wrap as a DataFrame. @@ -1295,7 +1295,7 @@ def register_arrow( ) def register_dataset(self, name: str, dataset: pa.dataset.Dataset) -> None: - """Register a [`Dataset`][pyarrow.dataset.Dataset] as a table. + """Register a :class:`~pyarrow.dataset.Dataset` as a table. Args: name: Name of the table to register. @@ -1343,7 +1343,7 @@ def udf(self, name: str) -> ScalarUDF: """Look up a registered scalar UDF by name. Returns the same ``ScalarUDF`` wrapper that - [`register_udf`][datafusion.context.SessionContext.register_udf] accepts, + :meth:`~datafusion.context.SessionContext.register_udf` accepts, so it can be invoked as an expression in the DataFrame API or re-registered into a different `SessionContext`. Built-in scalar functions from the session's function registry are @@ -1390,7 +1390,7 @@ def udaf(self, name: str) -> AggregateUDF: """Look up a registered aggregate UDF by name. Returns the same ``AggregateUDF`` wrapper that - [`register_udaf`][datafusion.context.SessionContext.register_udaf] accepts. + :meth:`~datafusion.context.SessionContext.register_udaf` accepts. Built-in aggregate functions such as ``sum`` or ``avg`` are also discoverable through this lookup. See `udf` for a worked late-binding example; the pattern is identical for aggregates. @@ -1403,7 +1403,7 @@ def udaf(self, name: str) -> AggregateUDF: Examples: Look up a built-in aggregate by name and use it in - [`aggregate`][datafusion.dataframe.DataFrame.aggregate]: + :meth:`~datafusion.dataframe.DataFrame.aggregate`: >>> ctx = dfn.SessionContext() >>> sum_fn = ctx.udaf("sum") @@ -1421,7 +1421,7 @@ def udwf(self, name: str) -> WindowUDF: """Look up a registered window UDF by name. Returns the same ``WindowUDF`` wrapper that - [`register_udwf`][datafusion.context.SessionContext.register_udwf] accepts. + :meth:`~datafusion.context.SessionContext.register_udwf` accepts. Built-in window functions such as ``row_number`` or ``rank`` are also discoverable through this lookup. See `udf` for a worked late-binding example; the pattern is identical for window @@ -1494,7 +1494,7 @@ def table_exist(self, name: str) -> bool: return self.ctx.table_exist(name) def empty_table(self) -> DataFrame: - """Create an empty [`DataFrame`][datafusion.dataframe.DataFrame].""" + """Create an empty :class:`~datafusion.dataframe.DataFrame`.""" return DataFrame(self.ctx.empty_table()) def session_id(self) -> str: @@ -1643,7 +1643,7 @@ def add_physical_optimizer_rule( Args: rule: Object exposing ``__datafusion_physical_optimizer_rule__`` — a - [`PhysicalOptimizerRuleExportable`][datafusion.context.PhysicalOptimizerRuleExportable]. + :class:`~datafusion.context.PhysicalOptimizerRuleExportable`. Examples: >>> from datafusion import SessionContext @@ -1655,7 +1655,7 @@ def add_physical_optimizer_rule( self.ctx.add_physical_optimizer_rule(rule) def table_provider(self, name: str) -> Table: - """Return the [`Table`][datafusion.catalog.Table] for the given table name. + """Return the :class:`~datafusion.catalog.Table` for the given table name. Args: name: Name of the table. @@ -1801,7 +1801,7 @@ def read_parquet( schema: pa.Schema | None = None, file_sort_order: Sequence[Sequence[SortKey]] | None = None, ) -> DataFrame: - """Read a Parquet source into a [`Dataframe`][datafusion.dataframe.DataFrame]. + """Read a Parquet source into a :class:`~datafusion.dataframe.DataFrame`. Args: path: Path to the Parquet file. @@ -1941,14 +1941,14 @@ def read_empty(self) -> DataFrame: See Also: This is an alias for - [`empty_table`][datafusion.context.SessionContext.empty_table]. + :meth:`~datafusion.context.SessionContext.empty_table`. """ return self.empty_table() def read_table( self, table: Table | TableProviderExportable | DataFrame | pa.dataset.Dataset ) -> DataFrame: - """Creates a [`DataFrame`][datafusion.dataframe.DataFrame] from a table.""" + """Creates a :class:`~datafusion.dataframe.DataFrame` from a table.""" return DataFrame(self.ctx.read_table(table)) def execute(self, plan: ExecutionPlan, partitions: int) -> RecordBatchStream: @@ -1963,7 +1963,7 @@ def _convert_file_sort_order( Each ``SortKey`` can be a column name string, an ``Expr``, or a ``SortExpr`` and will be converted using - [`sort_list_to_raw_sort_list`][datafusion.expr.sort_list_to_raw_sort_list]. + :func:`~datafusion.expr.sort_list_to_raw_sort_list`. """ # Convert each ``SortKey`` in the provided sort order to the low-level # representation expected by the Rust bindings. @@ -2073,17 +2073,17 @@ def with_python_udf_inlining(self, *, enabled: bool) -> SessionContext: to rebuild Python UDFs rather than call ``cloudpickle.loads`` on untrusted input. - The setting affects [`to_bytes`][datafusion.expr.Expr.to_bytes] and + The setting affects :meth:`~datafusion.expr.Expr.to_bytes` and `from_bytes` whenever this session is passed as the - ``ctx`` argument. [`dumps`][pickle.dumps] and [`loads`][pickle.loads] + ``ctx`` argument. :func:`~pickle.dumps` and :func:`~pickle.loads` do not pass a context, so to apply the setting through pickle, register this session with - [`set_sender_ctx`][datafusion.ipc.set_sender_ctx] on the sender and - [`set_worker_ctx`][datafusion.ipc.set_worker_ctx] on the receiver. + :func:`~datafusion.ipc.set_sender_ctx` on the sender and + :func:`~datafusion.ipc.set_worker_ctx` on the receiver. .. warning:: Security This setting narrows only `from_bytes`. Calling - [`loads`][pickle.loads] on untrusted bytes remains unsafe + :func:`~pickle.loads` on untrusted bytes remains unsafe regardless of the toggle. Returns a new `SessionContext` with the toggle applied; diff --git a/python/datafusion/dataframe.py b/python/datafusion/dataframe.py index 6b24cdda7..460f07c7f 100644 --- a/python/datafusion/dataframe.py +++ b/python/datafusion/dataframe.py @@ -14,23 +14,23 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. -"""[`DataFrame`][datafusion.dataframe.DataFrame] — lazy, chainable query representation. +""":class:`~datafusion.dataframe.DataFrame` — lazy, chainable query representation. A `DataFrame` is a logical plan over one or more data sources. -Methods that reshape the plan ([`select`][datafusion.dataframe.DataFrame.select], +Methods that reshape the plan (:meth:`~datafusion.dataframe.DataFrame.select`, `filter`, `aggregate`, -`sort`, [`join`][datafusion.dataframe.DataFrame.join], +`sort`, :meth:`~datafusion.dataframe.DataFrame.join`, `limit`, the set-operation methods, ...) return a new `DataFrame` and do no work until a terminal method such as -`collect`, [`to_pydict`][datafusion.dataframe.DataFrame.to_pydict], +`collect`, :meth:`~datafusion.dataframe.DataFrame.to_pydict`, `show`, or one of the ``write_*`` methods is called. DataFrames are produced from a -[`SessionContext`][datafusion.context.SessionContext], typically via -[`sql`][datafusion.context.SessionContext.sql], -[`read_csv`][datafusion.context.SessionContext.read_csv], -[`read_parquet`][datafusion.context.SessionContext.read_parquet], or -[`from_pydict`][datafusion.context.SessionContext.from_pydict]. +:class:`~datafusion.context.SessionContext`, typically via +:meth:`~datafusion.context.SessionContext.sql`, +:meth:`~datafusion.context.SessionContext.read_csv`, +:meth:`~datafusion.context.SessionContext.read_parquet`, or +:meth:`~datafusion.context.SessionContext.from_pydict`. Examples: >>> ctx = dfn.SessionContext() @@ -358,7 +358,7 @@ class DataFrame: """Two dimensional table representation of data. DataFrame objects are iterable; iterating over a DataFrame yields - [`RecordBatch`][datafusion.RecordBatch] instances lazily. + :class:`~datafusion.RecordBatch` instances lazily. See user_guide_concepts in the online documentation for more information. """ @@ -366,13 +366,13 @@ class DataFrame: def __init__(self, df: DataFrameInternal) -> None: """This constructor is not to be used by the end user. - See [`SessionContext`][datafusion.context.SessionContext] for methods to - create a [`DataFrame`][datafusion.dataframe.DataFrame]. + See :class:`~datafusion.context.SessionContext` for methods to + create a :class:`~datafusion.dataframe.DataFrame`. """ self.df = df def into_view(self, temporary: bool = False) -> Table: - """Convert ``DataFrame`` into a [`Table`][datafusion.Table]. + """Convert ``DataFrame`` into a :class:`~datafusion.Table`. Examples: >>> from datafusion import SessionContext @@ -438,7 +438,7 @@ def describe(self) -> DataFrame: return DataFrame(self.df.describe()) def schema(self) -> pa.Schema: - """Return the [`Schema`][pyarrow.Schema] of this DataFrame. + """Return the :class:`~pyarrow.Schema` of this DataFrame. The output schema contains information on the name, data type, and nullability for each column. @@ -452,7 +452,7 @@ def column(self, name: str) -> Expr: """Return a fully qualified column expression for ``name``. Resolves an unqualified column name against this DataFrame's schema - and returns an [`Expr`][datafusion.expr.Expr] whose underlying column reference + and returns an :class:`~datafusion.expr.Expr` whose underlying column reference includes the table qualifier. This is especially useful after joins, where the same column name may appear in multiple relations. @@ -487,17 +487,17 @@ def column(self, name: str) -> Expr: return self.find_qualified_columns(name)[0] def col(self, name: str) -> Expr: - """Alias for [`column`][datafusion.col.column]. + """Alias for :func:`~datafusion.col.column`. See Also: - [`column`][datafusion.col.column] + :func:`~datafusion.col.column` """ return self.column(name) def find_qualified_columns(self, *names: str) -> list[Expr]: """Return fully qualified column expressions for the given names. - This is a batch version of [`column`][datafusion.col.column] — it resolves each + This is a batch version of :func:`~datafusion.col.column` — it resolves each unqualified name against the DataFrame's schema and returns a list of qualified column expressions. @@ -534,7 +534,7 @@ def select_exprs(self, *args: str) -> DataFrame: return self.df.select_exprs(*args) def alias(self, alias: str) -> DataFrame: - """Assign a table alias to this [`DataFrame`][datafusion.dataframe.DataFrame]. + """Assign a table alias to this :class:`~datafusion.dataframe.DataFrame`. Replaces the qualifiers of the output columns with ``alias``. Useful for self-joins and any situation that needs an unambiguous table-style @@ -562,11 +562,11 @@ def alias(self, alias: str) -> DataFrame: def select(self, *exprs: Expr | str) -> DataFrame: """Project arbitrary expressions into a new `DataFrame`. - String arguments are treated as column names; [`Expr`][datafusion.expr.Expr] + String arguments are treated as column names; :class:`~datafusion.expr.Expr` arguments can reshape, rename, or compute new columns. Args: - exprs: Either column names or [`Expr`][datafusion.expr.Expr] to select. + exprs: Either column names or :class:`~datafusion.expr.Expr` to select. Returns: DataFrame after projection. It has one column for each expression. @@ -654,10 +654,10 @@ def filter(self, *predicates: Expr | str) -> DataFrame: Rows for which ``predicate`` evaluates to ``False`` or ``None`` are filtered out. If more than one predicate is provided, these predicates will be combined as a logical AND. Each ``predicate`` can be an - [`Expr`][datafusion.expr.Expr] created using helper functions such as - `col` or [`lit`][datafusion.lit], or a SQL expression string + :class:`~datafusion.expr.Expr` created using helper functions such as + `col` or :func:`~datafusion.lit`, or a SQL expression string that will be parsed against the DataFrame schema. If more complex logic is - required, see the logical operations in [`functions`][datafusion.functions]. + required, see the logical operations in :mod:`~datafusion.functions`. Examples: >>> ctx = dfn.SessionContext() @@ -706,8 +706,8 @@ def parse_sql_expr(self, expr: str) -> Expr: def with_column(self, name: str, expr: Expr | str) -> DataFrame: """Add an additional column to the DataFrame. - The ``expr`` must be an [`Expr`][datafusion.expr.Expr] constructed with - [`col`][datafusion.col.col] or [`lit`][datafusion.lit], or a SQL expression + The ``expr`` must be an :class:`~datafusion.expr.Expr` constructed with + :func:`~datafusion.col.col` or :func:`~datafusion.lit`, or a SQL expression string that will be parsed against the DataFrame schema. Examples: @@ -734,8 +734,8 @@ def with_columns( By passing expressions, iterables of expressions, string SQL expressions, or named expressions. - All expressions must be [`Expr`][datafusion.expr.Expr] objects created via - `col` or [`lit`][datafusion.lit], or SQL expression strings. + All expressions must be :class:`~datafusion.expr.Expr` objects created via + `col` or :func:`~datafusion.lit`, or SQL expression strings. To pass named expressions use the form ``name=Expr``. Example usage: The following will add 4 columns labeled ``a``, ``b``, ``c``, @@ -816,18 +816,18 @@ def aggregate( By default each unique combination of the ``group_by`` columns produces one row. To get multiple levels of subtotals in a single pass, pass a - [`GroupingSet`][datafusion.expr.GroupingSet] expression + :class:`~datafusion.expr.GroupingSet` expression (created via - [`rollup`][datafusion.expr.GroupingSet.rollup], - [`cube`][datafusion.expr.GroupingSet.cube], or - [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets]) + :meth:`~datafusion.expr.GroupingSet.rollup`, + :meth:`~datafusion.expr.GroupingSet.cube`, or + :meth:`~datafusion.expr.GroupingSet.grouping_sets`) as the ``group_by`` argument. See the aggregation user guide for detailed examples. Args: group_by: Sequence of expressions or column names to group by, or ``None`` for aggregation over the whole DataFrame. - A [`GroupingSet`][datafusion.expr.GroupingSet] expression may + A :class:`~datafusion.expr.GroupingSet` expression may be included to produce multiple grouping levels (rollup, cube, or explicit grouping sets). aggs: Sequence of expressions to aggregate. @@ -877,7 +877,7 @@ def sort(self, *exprs: SortKey) -> DataFrame: Note that any expression can be turned into a sort expression by calling its ``sort`` method. For ascending-only sorts, the shorter - [`sort_by`][datafusion.dataframe.DataFrame.sort_by] is usually more convenient. + :meth:`~datafusion.dataframe.DataFrame.sort_by` is usually more convenient. Args: exprs: Sort expressions or column names, applied in order. @@ -893,7 +893,7 @@ def sort(self, *exprs: SortKey) -> DataFrame: >>> df.sort("a").to_pydict() {'a': [1, 2, 3], 'b': [20, 30, 10]} - Sort descending using [`sort`][datafusion.expr.Expr.sort]: + Sort descending using :meth:`~datafusion.expr.Expr.sort`: >>> df.sort(col("a").sort(ascending=False)).to_pydict() {'a': [3, 2, 1], 'b': [10, 30, 20]} @@ -918,8 +918,8 @@ def limit(self, count: int, offset: int = 0) -> DataFrame: Results are returned in unspecified order unless the DataFrame is explicitly sorted first via - [`sort`][datafusion.dataframe.DataFrame.sort] or - [`sort_by`][datafusion.dataframe.DataFrame.sort_by]. + :meth:`~datafusion.dataframe.DataFrame.sort` or + :meth:`~datafusion.dataframe.DataFrame.sort_by`. Args: count: Number of rows to limit the DataFrame to. @@ -976,7 +976,7 @@ def collect(self) -> list[pa.RecordBatch]: computation. Returns: - List of [`RecordBatch`][pyarrow.RecordBatch] collected from the DataFrame. + List of :class:`~pyarrow.RecordBatch` collected from the DataFrame. """ return self.df.collect() @@ -1078,7 +1078,7 @@ def join( When non-key columns share the same name in both DataFrames, use `col` on each DataFrame **before** the join to obtain fully qualified column references that can disambiguate them. - See [`join_on`][datafusion.dataframe.DataFrame.join_on] for an example. + See :meth:`~datafusion.dataframe.DataFrame.join_on` for an example. Args: right: Other DataFrame to join with. @@ -1170,8 +1170,8 @@ def join_on( ) -> DataFrame: """Join two `DataFrame` using the specified expressions. - Join predicates must be [`Expr`][datafusion.expr.Expr] objects, typically - built with [`col`][datafusion.col.col]. On expressions are used to support + Join predicates must be :class:`~datafusion.expr.Expr` objects, typically + built with :func:`~datafusion.col.col`. On expressions are used to support in-equality predicates. Equality predicates are correctly optimized. Use `col` on each DataFrame **before** the join to @@ -1190,7 +1190,7 @@ def join_on( ... ).sort(col("x")).to_pydict() {'a': [1, 2], 'x': ['a', 'b'], 'b': [1, 2], 'y': ['c', 'd']} - Use [`col`][datafusion.col.col] to disambiguate shared column names: + Use :func:`~datafusion.col.col` to disambiguate shared column names: >>> left = ctx.from_pydict({"id": [1, 2], "val": [10, 20]}) >>> right = ctx.from_pydict({"id": [1, 2], "val": [30, 40]}) @@ -1228,7 +1228,7 @@ def explain( verbose: If ``True``, more details will be included. analyze: If ``True``, the plan will run and metrics reported. format: Output format for the plan. Defaults to - [`INDENT`][datafusion.dataframe.ExplainFormat.INDENT]. + :attr:`~datafusion.dataframe.ExplainFormat.INDENT`. Examples: Show the plan in tree format: @@ -1299,7 +1299,7 @@ def repartition_by_hash(self, *exprs: Expr | str, num: int) -> DataFrame: return DataFrame(self.df.repartition_by_hash(*exprs, num=num)) def union(self, other: DataFrame, distinct: bool = False) -> DataFrame: - """Calculate the union of two [`DataFrame`][datafusion.dataframe.DataFrame]. + """Calculate the union of two :class:`~datafusion.dataframe.DataFrame`. The two `DataFrame` must have exactly the same schema. @@ -1333,7 +1333,7 @@ def union_distinct(self, other: DataFrame) -> DataFrame: """Calculate the distinct union of two `DataFrame`. See Also: - [`union`][datafusion.dataframe.DataFrame.union] + :meth:`~datafusion.dataframe.DataFrame.union` """ return self.union(other, distinct=True) @@ -1400,7 +1400,7 @@ def except_all(self, other: DataFrame, distinct: bool = False) -> DataFrame: def union_by_name(self, other: DataFrame, distinct: bool = False) -> DataFrame: """Union two `DataFrame` matching columns by name. - Unlike [`union`][datafusion.dataframe.DataFrame.union] which matches + Unlike :meth:`~datafusion.dataframe.DataFrame.union` which matches columns by position, this method matches columns by their names, allowing DataFrames with different column orders to be combined. @@ -1474,7 +1474,7 @@ def sort_by(self, *exprs: Expr | str) -> DataFrame: This is a convenience method that sorts the DataFrame by the given expressions in ascending order with nulls last. For more control over sort direction and null ordering, use - [`sort`][datafusion.dataframe.DataFrame.sort] instead. + :meth:`~datafusion.dataframe.DataFrame.sort` instead. Args: exprs: Expressions or column names to sort by. @@ -1804,7 +1804,7 @@ def __arrow_c_stream__(self, requested_schema: object | None = None) -> object: supported through this interface. Args: - requested_schema: Either a [`Schema`][pyarrow.Schema] or an Arrow C + requested_schema: Either a :class:`~pyarrow.Schema` or an Arrow C Schema capsule (``PyCapsule``) produced by ``schema._export_to_c_capsule()``. The DataFrame will attempt to align its output with the fields and order specified by this schema. diff --git a/python/datafusion/dataframe_formatter.py b/python/datafusion/dataframe_formatter.py index dfbdf6cba..f74697f3e 100644 --- a/python/datafusion/dataframe_formatter.py +++ b/python/datafusion/dataframe_formatter.py @@ -357,7 +357,7 @@ def repr_rows(self) -> int: .. deprecated:: Use - [`max_rows`][datafusion.dataframe_formatter.DataFrameHtmlFormatter.max_rows] + :attr:`~datafusion.dataframe_formatter.DataFrameHtmlFormatter.max_rows` instead. This property is provided for backward compatibility. Returns: @@ -371,7 +371,7 @@ def repr_rows(self, value: int) -> None: .. deprecated:: Use the - [`max_rows`][datafusion.dataframe_formatter.DataFrameHtmlFormatter.max_rows] + :attr:`~datafusion.dataframe_formatter.DataFrameHtmlFormatter.max_rows` setter instead. This property is provided for backward compatibility. Args: diff --git a/python/datafusion/expr.py b/python/datafusion/expr.py index 811372a8f..e9682ab83 100644 --- a/python/datafusion/expr.py +++ b/python/datafusion/expr.py @@ -17,19 +17,19 @@ """`Expr` — the logical expression type used to build DataFusion queries. -An [`Expr`][datafusion.expr.Expr] represents a computation over columns or literals: a +An :class:`~datafusion.expr.Expr` represents a computation over columns or literals: a column reference (``col("a")``), a literal (``lit(5)``), an operator combination (``col("a") + lit(1)``), or the output of a function from -[`functions`][datafusion.functions]. Expressions are passed to -[`DataFrame`][datafusion.dataframe.DataFrame] methods such as -[`select`][datafusion.dataframe.DataFrame.select], -[`filter`][datafusion.dataframe.DataFrame.filter], -[`aggregate`][datafusion.dataframe.DataFrame.aggregate], and -[`sort`][datafusion.dataframe.DataFrame.sort]. +:mod:`~datafusion.functions`. Expressions are passed to +:class:`~datafusion.dataframe.DataFrame` methods such as +:meth:`~datafusion.dataframe.DataFrame.select`, +:meth:`~datafusion.dataframe.DataFrame.filter`, +:meth:`~datafusion.dataframe.DataFrame.aggregate`, and +:meth:`~datafusion.dataframe.DataFrame.sort`. Convenience constructors are re-exported at the package level: -[`col`][datafusion.col.col] / [`column`][datafusion.col.column] for column references -and [`lit`][datafusion.lit] / [`literal`][datafusion.literal] for scalar +:func:`~datafusion.col.col` / :func:`~datafusion.col.column` for column references +and :func:`~datafusion.lit` / :func:`~datafusion.literal` for scalar literals. Examples: @@ -262,8 +262,8 @@ def ensure_expr(value: Expr | Any) -> expr_internal.Expr: """Return the internal expression from ``Expr`` or raise ``TypeError``. This helper rejects plain strings and other non-`Expr` values so - higher level APIs consistently require explicit [`col`][datafusion.col.col] or - [`lit`][datafusion.lit] expressions. + higher level APIs consistently require explicit :func:`~datafusion.col.col` or + :func:`~datafusion.lit` expressions. See Also: `coerce_to_expr` — the opposite behavior: *wraps* non-``Expr`` @@ -276,7 +276,7 @@ def ensure_expr(value: Expr | Any) -> expr_internal.Expr: The internal expression representation. Raises: - TypeError: If ``value`` is not an instance of [`Expr`][datafusion.expr.Expr]. + TypeError: If ``value`` is not an instance of :class:`~datafusion.expr.Expr`. """ if not isinstance(value, Expr): raise TypeError(EXPR_TYPE_ERROR) @@ -295,7 +295,7 @@ def ensure_expr_list( A flat list of raw expressions. Raises: - TypeError: If any item is not an instance of [`Expr`][datafusion.expr.Expr]. + TypeError: If any item is not an instance of :class:`~datafusion.expr.Expr`. """ def _iter( @@ -316,7 +316,7 @@ def _iter( def coerce_to_expr(value: Any) -> Expr: """Coerce a native Python value to an ``Expr`` literal, passing ``Expr`` through. - This is the complement of [`ensure_expr`][ensure_expr]: where ``ensure_expr`` + This is the complement of :func:`~ensure_expr`: where ``ensure_expr`` *rejects* non-``Expr`` values, ``coerce_to_expr`` *wraps* them via `literal` so that functions can accept native Python types (``int``, ``float``, ``str``, ``bool``, etc.) alongside ``Expr``. @@ -355,7 +355,7 @@ def _to_raw_expr(value: Expr | str) -> expr_internal.Expr: value: Candidate expression or column name. Returns: - The internal [`Expr`][datafusion._internal.expr.Expr] representation. + The internal :class:`~datafusion._internal.expr.Expr` representation. Raises: TypeError: If ``value`` is neither an `Expr` nor ``str``. @@ -443,12 +443,12 @@ def variant_name(self) -> str: def to_bytes(self, ctx: SessionContext | None = None) -> bytes: """Serialize this expression to bytes for shipping to another process. - Use this — or [`dumps`][pickle.dumps] — to send an expression to a + Use this — or :func:`~pickle.dumps` — to send an expression to a worker process for distributed evaluation. When ``ctx`` is supplied, encoding routes through that session's installed logical extension codec (set via - [`with_logical_extension_codec`][datafusion.context.SessionContext.with_logical_extension_codec]), + :meth:`~datafusion.context.SessionContext.with_logical_extension_codec`), so settings like `with_python_udf_inlining` take effect. When ``ctx`` is ``None``, the default codec is used (Python UDF inlining on, no user-installed extension codec). @@ -464,8 +464,8 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: .. warning:: Security Bytes returned here may embed a cloudpickled Python callable (when the expression carries a Python UDF). - Reconstructing them via [`from_bytes`][datafusion.expr.Expr.from_bytes] or - [`loads`][pickle.loads] executes arbitrary Python on the + Reconstructing them via :meth:`~datafusion.expr.Expr.from_bytes` or + :func:`~pickle.loads` executes arbitrary Python on the receiver. Only accept payloads from trusted sources. .. warning:: Portability @@ -473,7 +473,7 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: stable across Python minor versions**. A payload produced on Python 3.11 will fail to load on Python 3.12. The wire format stamps the sender's ``(major, minor)``; - `from_bytes` raises a [`ValueError`][ValueError] naming + `from_bytes` raises a :exc:`~ValueError` naming both versions on mismatch. cloudpickle captures the UDF callable **by value** — @@ -527,13 +527,13 @@ def double(x): def from_bytes(cls, buf: bytes, ctx: SessionContext | None = None) -> Expr: """Reconstruct an expression from serialized bytes. - Accepts output of `to_bytes` or [`dumps`][pickle.dumps]. + Accepts output of `to_bytes` or :func:`~pickle.dumps`. ``ctx`` is the `SessionContext` used to resolve any function references that travel by name (e.g. FFI UDFs, or Python UDFs sent with inlining disabled via `with_python_udf_inlining`). When ``ctx`` is ``None`` the worker context installed via - [`set_worker_ctx`][datafusion.ipc.set_worker_ctx] is consulted; if no worker + :func:`~datafusion.ipc.set_worker_ctx` is consulted; if no worker context is installed, the global `SessionContext` is used (sufficient for built-ins and Python UDFs, plus any UDFs registered on the global context). @@ -548,7 +548,7 @@ def from_bytes(cls, buf: bytes, ctx: SessionContext | None = None) -> Expr: cloudpickle payloads are **not portable across Python minor versions**. The wire format stamps the sender's ``(major, minor)``; if it does not match the current - interpreter, this method raises [`ValueError`][ValueError] + interpreter, this method raises :exc:`~ValueError` naming both versions. Modules the UDF imports must also be importable on the receiver — see `to_bytes` for by-value vs. by-reference details. @@ -568,17 +568,17 @@ def __reduce__(self) -> tuple[Callable[[bytes], Expr], tuple[bytes]]: """Pickle protocol hook. Lets expressions be shipped to worker processes via - [`dumps`][pickle.dumps] / [`loads`][pickle.loads]. Built-in functions + :func:`~pickle.dumps` / :func:`~pickle.loads`. Built-in functions and Python UDFs (scalar, aggregate, window) travel inside the pickle bytes; only FFI-capsule UDFs require pre-registration on the worker. The worker's `SessionContext` for resolving those references is looked up via - [`set_worker_ctx`][datafusion.ipc.set_worker_ctx], falling back to the + :func:`~datafusion.ipc.set_worker_ctx`, falling back to the global `SessionContext` if none has been installed on the worker. .. warning:: Security - [`loads`][pickle.loads] on the returned tuple executes + :func:`~pickle.loads` on the returned tuple executes arbitrary Python on the receiver, including any cloudpickled UDF callable embedded in the payload. Only unpickle expressions from trusted sources. @@ -597,17 +597,17 @@ def __reduce__(self) -> tuple[Callable[[bytes], Expr], tuple[bytes]]: 'a * Int64(2)' The encoding side honors a driver-side sender context installed - via [`set_sender_ctx`][datafusion.ipc.set_sender_ctx] — that is how + via :func:`~datafusion.ipc.set_sender_ctx` — that is how `with_python_udf_inlining` propagates through ``pickle.dumps``. The sender context is read by - ``__reduce__``, so [`copy`][copy.copy] and [`deepcopy`][copy.deepcopy] + ``__reduce__``, so :func:`~copy.copy` and :func:`~copy.deepcopy` — which also go through ``__reduce__`` — pick it up too. """ return (Expr._reconstruct, (self.to_bytes(get_sender_ctx()),)) @classmethod def _reconstruct(cls, proto_bytes: bytes) -> Expr: - """Internal entry point used by [`__reduce__`][__reduce__] on unpickle. + """Internal entry point used by :func:`~__reduce__` on unpickle. Examples: >>> from datafusion import Expr, col, lit @@ -692,11 +692,11 @@ def __getitem__(self, key: str | int) -> Expr: If ``key`` is a string, returns the subfield of the struct. If ``key`` is an integer, retrieves the element in the array. Note that the element index begins at ``0``, unlike - [`array_element`][datafusion.functions.array_element] which begins at ``1``. + :func:`~datafusion.functions.array_element` which begins at ``1``. If ``key`` is a slice, returns an array that contains a slice of the original array. Similar to integer indexing, this follows Python convention where the index begins at ``0`` unlike - [`array_slice`][datafusion.functions.array_slice] which begins at ``1``. + :func:`~datafusion.functions.array_slice` which begins at ``1``. """ if isinstance(key, int): return Expr( @@ -1126,7 +1126,7 @@ def initcap(self) -> Expr: def list_distinct(self) -> Expr: """Returns distinct values from the array after removing duplicates. - This is an alias for [`array_distinct`][datafusion.functions.array_distinct]. + This is an alias for :func:`~datafusion.functions.array_distinct`. """ from . import functions as F @@ -1309,7 +1309,7 @@ def atanh(self) -> Expr: def list_dims(self) -> Expr: """Returns an array of the array's dimensions. - This is an alias for [`array_dims`][datafusion.functions.array_dims]. + This is an alias for :func:`~datafusion.functions.array_dims`. """ from . import functions as F @@ -1348,7 +1348,7 @@ def ceil(self) -> Expr: def list_length(self) -> Expr: """Returns the length of the array. - This is an alias for [`array_length`][datafusion.functions.array_length]. + This is an alias for :func:`~datafusion.functions.array_length`. """ from . import functions as F @@ -1405,7 +1405,7 @@ def char_length(self) -> Expr: def list_ndims(self) -> Expr: """Returns the number of dimensions of the array. - This is an alias for [`array_ndims`][datafusion.functions.array_ndims]. + This is an alias for :func:`~datafusion.functions.array_ndims`. """ from . import functions as F @@ -1430,7 +1430,7 @@ def sinh(self) -> Expr: return F.sinh(self) def empty(self) -> Expr: - """This is an alias for [`array_empty`][datafusion.functions.array_empty].""" + """This is an alias for :func:`~datafusion.functions.array_empty`.""" from . import functions as F return F.empty(self) @@ -1621,7 +1621,7 @@ def __init__(self, case_builder: expr_internal.CaseBuilder) -> None: """Constructs a case builder. This is not typically called by the end user directly. See - [`case`][datafusion.functions.case] instead. + :func:`~datafusion.functions.case` instead. """ self.case_builder = case_builder @@ -1672,12 +1672,12 @@ class GroupingSet: """Factory for creating grouping set expressions. Grouping sets control how - [`aggregate`][datafusion.dataframe.DataFrame.aggregate] groups rows. + :meth:`~datafusion.dataframe.DataFrame.aggregate` groups rows. Instead of a single ``GROUP BY``, they produce multiple grouping levels in one pass — subtotals, cross-tabulations, or arbitrary column subsets. - Use [`grouping`][datafusion.functions.grouping] in the aggregate list + Use :func:`~datafusion.functions.grouping` in the aggregate list to tell which columns are aggregated across in each result row. """ @@ -1708,9 +1708,9 @@ def rollup(*exprs: Expr | str) -> Expr: [30, 30, 60] See Also: - [`cube`][datafusion.expr.GroupingSet.cube], - [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets], - [`grouping`][datafusion.functions.grouping] + :meth:`~datafusion.expr.GroupingSet.cube`, + :meth:`~datafusion.expr.GroupingSet.grouping_sets`, + :func:`~datafusion.functions.grouping` """ args = [_to_raw_expr(e) for e in exprs] return Expr(expr_internal.GroupingSet.rollup(*args)) @@ -1731,7 +1731,7 @@ def cube(*exprs: Expr | str) -> Expr: Examples: With a single column, ``cube`` behaves identically to - [`rollup`][datafusion.expr.GroupingSet.rollup]: + :meth:`~datafusion.expr.GroupingSet.rollup`: >>> from datafusion.expr import GroupingSet >>> ctx = dfn.SessionContext() @@ -1745,9 +1745,9 @@ def cube(*exprs: Expr | str) -> Expr: [30, 30, 60] See Also: - [`rollup`][datafusion.expr.GroupingSet.rollup], - [`grouping_sets`][datafusion.expr.GroupingSet.grouping_sets], - [`grouping`][datafusion.functions.grouping] + :meth:`~datafusion.expr.GroupingSet.rollup`, + :meth:`~datafusion.expr.GroupingSet.grouping_sets`, + :func:`~datafusion.functions.grouping` """ args = [_to_raw_expr(e) for e in exprs] return Expr(expr_internal.GroupingSet.cube(*args)) @@ -1789,9 +1789,9 @@ def grouping_sets(*expr_lists: list[Expr | str]) -> Expr: [3, 3, 4, 2] See Also: - [`rollup`][datafusion.expr.GroupingSet.rollup], - [`cube`][datafusion.expr.GroupingSet.cube], - [`grouping`][datafusion.functions.grouping] + :meth:`~datafusion.expr.GroupingSet.rollup`, + :meth:`~datafusion.expr.GroupingSet.cube`, + :func:`~datafusion.functions.grouping` """ raw_lists = [[_to_raw_expr(e) for e in lst] for lst in expr_lists] return Expr(expr_internal.GroupingSet.grouping_sets(*raw_lists)) diff --git a/python/datafusion/functions.py b/python/datafusion/functions.py index c1e0ad18a..a86f6d07c 100644 --- a/python/datafusion/functions.py +++ b/python/datafusion/functions.py @@ -14,15 +14,15 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. -"""Scalar, aggregate, and window functions for [`Expr`][datafusion.expr.Expr]. +"""Scalar, aggregate, and window functions for :class:`~datafusion.expr.Expr`. -Each function returns an [`Expr`][datafusion.expr.Expr] that can be combined +Each function returns an :class:`~datafusion.expr.Expr` that can be combined with other expressions and passed to -[`DataFrame`][datafusion.dataframe.DataFrame] methods such as -[`select`][datafusion.dataframe.DataFrame.select], -[`filter`][datafusion.dataframe.DataFrame.filter], -[`aggregate`][datafusion.dataframe.DataFrame.aggregate], and -[`window`][datafusion.dataframe.DataFrame.window]. The module is conventionally +:class:`~datafusion.dataframe.DataFrame` methods such as +:meth:`~datafusion.dataframe.DataFrame.select`, +:meth:`~datafusion.dataframe.DataFrame.filter`, +:meth:`~datafusion.dataframe.DataFrame.aggregate`, and +:meth:`~datafusion.dataframe.DataFrame.window`. The module is conventionally imported as ``F`` so calls read like ``F.sum(col("price"))``. Examples: @@ -449,7 +449,7 @@ def array_join(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for [`array_to_string`][datafusion.functions.array_to_string]. + This is an alias for :func:`~datafusion.functions.array_to_string`. """ return array_to_string(expr, delimiter) @@ -458,7 +458,7 @@ def list_to_string(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for [`array_to_string`][datafusion.functions.array_to_string]. + This is an alias for :func:`~datafusion.functions.array_to_string`. """ return array_to_string(expr, delimiter) @@ -467,7 +467,7 @@ def list_join(expr: Expr, delimiter: Expr | str) -> Expr: """Converts each element to its text representation. See Also: - This is an alias for [`array_to_string`][datafusion.functions.array_to_string]. + This is an alias for :func:`~datafusion.functions.array_to_string`. """ return array_to_string(expr, delimiter) @@ -506,7 +506,7 @@ def lambda_(params: list[str], body: Expr) -> Expr: Args: params: Ordered lambda parameter names. body: Body expression that references the parameters via - [`lambda_var`][datafusion.functions.lambda_var]. + :func:`~datafusion.functions.lambda_var`. Examples: >>> ctx = dfn.SessionContext() @@ -550,7 +550,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: ``transform`` may be a Python callable, which is converted to a lambda automatically (its parameter names become the lambda parameters), or an - explicit lambda built with [`lambda_`][datafusion.functions.lambda_]. + explicit lambda built with :func:`~datafusion.functions.lambda_`. Examples: Using a Python callable: @@ -562,7 +562,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("d")[0].as_py() [2, 4, 6] - Using an explicit lambda built with [`lambda_`][datafusion.functions.lambda_]: + Using an explicit lambda built with :func:`~datafusion.functions.lambda_`: >>> double_fn = F.lambda_(["v"], F.lambda_var("v") * lit(2)) >>> df.select( @@ -571,7 +571,7 @@ def array_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: [2, 4, 6] See Also: - `array_any_match`, [`lambda_`][datafusion.functions.lambda_]. + `array_any_match`, :func:`~datafusion.functions.lambda_`. """ return Expr(f.array_transform(array.expr, _to_lambda(transform).expr)) @@ -580,7 +580,7 @@ def list_transform(array: Expr, transform: Expr | Callable[..., Any]) -> Expr: """Transform each element of a list with a lambda. See Also: - This is an alias for [`array_transform`][datafusion.functions.array_transform]. + This is an alias for :func:`~datafusion.functions.array_transform`. """ return array_transform(array, transform) @@ -602,7 +602,7 @@ def array_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("m")[0].as_py() True - Using an explicit lambda built with [`lambda_`][datafusion.functions.lambda_]: + Using an explicit lambda built with :func:`~datafusion.functions.lambda_`: >>> predicate = F.lambda_(["v"], F.lambda_var("v") > lit(2)) >>> df.select( @@ -611,7 +611,7 @@ def array_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: True See Also: - `array_transform`, [`lambda_`][datafusion.functions.lambda_]. + `array_transform`, :func:`~datafusion.functions.lambda_`. """ return Expr(f.array_any_match(array.expr, _to_lambda(predicate).expr)) @@ -620,7 +620,7 @@ def any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Return ``True`` if any element of an array satisfies a predicate. See Also: - This is an alias for [`array_any_match`][datafusion.functions.array_any_match]. + This is an alias for :func:`~datafusion.functions.array_any_match`. """ return array_any_match(array, predicate) @@ -629,7 +629,7 @@ def list_any_match(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Return ``True`` if any element of a list satisfies a predicate. See Also: - This is an alias for [`array_any_match`][datafusion.functions.array_any_match]. + This is an alias for :func:`~datafusion.functions.array_any_match`. """ return array_any_match(array, predicate) @@ -652,7 +652,7 @@ def array_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: ... ).collect_column("f")[0].as_py() [3, 4, 5] - Using an explicit lambda built with [`lambda_`][datafusion.functions.lambda_]: + Using an explicit lambda built with :func:`~datafusion.functions.lambda_`: >>> predicate = F.lambda_(["v"], F.lambda_var("v") > lit(2)) >>> df.select( @@ -661,7 +661,7 @@ def array_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: [3, 4, 5] See Also: - `array_transform`, `array_any_match`, [`lambda_`][datafusion.functions.lambda_]. + `array_transform`, `array_any_match`, :func:`~datafusion.functions.lambda_`. """ return Expr(f.array_filter(array.expr, _to_lambda(predicate).expr)) @@ -670,7 +670,7 @@ def list_filter(array: Expr, predicate: Expr | Callable[..., Any]) -> Expr: """Keep the elements of a list for which a predicate is ``True``. See Also: - This is an alias for [`array_filter`][datafusion.functions.array_filter]. + This is an alias for :func:`~datafusion.functions.array_filter`. """ return array_filter(array, predicate) @@ -864,8 +864,8 @@ def count_star(filter: Expr | None = None) -> Expr: def case(expr: Expr) -> CaseBuilder: """Create a case expression. - Create a [`CaseBuilder`][datafusion.expr.CaseBuilder] to match cases for the - expression ``expr``. See [`CaseBuilder`][datafusion.expr.CaseBuilder] for + Create a :class:`~datafusion.expr.CaseBuilder` to match cases for the + expression ``expr``. See :class:`~datafusion.expr.CaseBuilder` for detailed usage. Examples: @@ -883,8 +883,8 @@ def case(expr: Expr) -> CaseBuilder: def when(when: Expr, then: Expr) -> CaseBuilder: """Create a case expression that has no base expression. - Create a [`CaseBuilder`][datafusion.expr.CaseBuilder] to match cases for the - expression ``expr``. See [`CaseBuilder`][datafusion.expr.CaseBuilder] for + Create a :class:`~datafusion.expr.CaseBuilder` to match cases for the + expression ``expr``. See :class:`~datafusion.expr.CaseBuilder` for detailed usage. Examples: @@ -1315,7 +1315,7 @@ def ifnull(x: Expr, y: Expr) -> Expr: y: Fallback expression to return when ``x`` is NULL. See Also: - This is an alias for [`nvl`][datafusion.functions.nvl]. + This is an alias for :func:`~datafusion.functions.nvl`. """ return nvl(x, y) @@ -1340,7 +1340,7 @@ def instr(string: Expr, substring: Expr | str) -> Expr: """Finds the position from where the ``substring`` matches the ``string``. See Also: - This is an alias for [`strpos`][datafusion.functions.strpos]. + This is an alias for :func:`~datafusion.functions.strpos`. """ return strpos(string, substring) @@ -1669,7 +1669,7 @@ def position(string: Expr, substring: Expr | str) -> Expr: """Finds the position from where the ``substring`` matches the ``string``. See Also: - This is an alias for [`strpos`][datafusion.functions.strpos]. + This is an alias for :func:`~datafusion.functions.strpos`. """ return strpos(string, substring) @@ -1694,7 +1694,7 @@ def pow(base: Expr, exponent: Expr | int | float) -> Expr: # noqa: PYI041 """Returns ``base`` raised to the power of ``exponent``. See Also: - This is an alias of [`power`][datafusion.functions.power]. + This is an alias of :func:`~datafusion.functions.power`. """ return power(base, exponent) @@ -2329,7 +2329,7 @@ def current_timestamp() -> Expr: """Returns the current timestamp in nanoseconds. See Also: - This is an alias for [`now`][datafusion.functions.now]. + This is an alias for :func:`~datafusion.functions.now`. """ return now() @@ -2361,7 +2361,7 @@ def date_format(arg: Expr, formatter: Expr | str) -> Expr: """Returns a string representation of a date, time, timestamp or duration. See Also: - This is an alias for [`to_char`][datafusion.functions.to_char]. + This is an alias for :func:`~datafusion.functions.to_char`. """ return to_char(arg, formatter) @@ -2573,7 +2573,7 @@ def datepart(part: Expr | str, date: Expr) -> Expr: """Return a specified part of a date. See Also: - This is an alias for [`date_part`][datafusion.functions.date_part]. + This is an alias for :func:`~datafusion.functions.date_part`. """ return date_part(part, date) @@ -2603,7 +2603,7 @@ def extract(part: Expr | str, date: Expr) -> Expr: """Extracts a subfield from the date. See Also: - This is an alias for [`date_part`][datafusion.functions.date_part]. + This is an alias for :func:`~datafusion.functions.date_part`. """ return date_part(part, date) @@ -2634,7 +2634,7 @@ def datetrunc(part: Expr | str, date: Expr) -> Expr: """Truncates the date to a specified level of precision. See Also: - This is an alias for [`date_trunc`][datafusion.functions.date_trunc]. + This is an alias for :func:`~datafusion.functions.date_trunc`. """ return date_trunc(part, date) @@ -2776,7 +2776,7 @@ def make_list(*args: Expr) -> Expr: """Returns an array using the specified input expressions. See Also: - This is an alias for [`make_array`][datafusion.functions.make_array]. + This is an alias for :func:`~datafusion.functions.make_array`. """ return make_array(*args) @@ -2785,7 +2785,7 @@ def array(*args: Expr) -> Expr: """Returns an array using the specified input expressions. See Also: - This is an alias for [`make_array`][datafusion.functions.make_array]. + This is an alias for :func:`~datafusion.functions.make_array`. """ return make_array(*args) @@ -2969,13 +2969,13 @@ def get_field(expr: Expr, *names: Expr | str) -> Expr: of nested struct/map fields in a single ``get_field`` call. For a single static-string name, ``expr["field"]`` is a convenient shorthand; use ``get_field`` when the field name is a dynamic - [`Expr`][datafusion.expr.Expr] or when traversing multiple levels at + :class:`~datafusion.expr.Expr` or when traversing multiple levels at once. Args: expr: The struct or map expression to read from. *names: One or more field names (``str``) or expressions - ([`Expr`][datafusion.expr.Expr]). + (:class:`~datafusion.expr.Expr`). Examples: Single-level lookup: @@ -3085,7 +3085,7 @@ def row(*args: Expr) -> Expr: """Returns a struct with the given arguments. See Also: - This is an alias for [`struct`][datafusion.functions.struct]. + This is an alias for :func:`~datafusion.functions.struct`. """ return struct(*args) @@ -3124,7 +3124,7 @@ def array_push_back(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for [`array_append`][datafusion.functions.array_append]. + This is an alias for :func:`~datafusion.functions.array_append`. """ return array_append(array, element) @@ -3133,7 +3133,7 @@ def list_append(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for [`array_append`][datafusion.functions.array_append]. + This is an alias for :func:`~datafusion.functions.array_append`. """ return array_append(array, element) @@ -3142,7 +3142,7 @@ def list_push_back(array: Expr, element: Expr) -> Expr: """Appends an element to the end of an array. See Also: - This is an alias for [`array_append`][datafusion.functions.array_append]. + This is an alias for :func:`~datafusion.functions.array_append`. """ return array_append(array, element) @@ -3166,7 +3166,7 @@ def array_cat(*args: Expr) -> Expr: """Concatenates the input arrays. See Also: - This is an alias for [`array_concat`][datafusion.functions.array_concat]. + This is an alias for :func:`~datafusion.functions.array_concat`. """ return array_concat(*args) @@ -3225,7 +3225,7 @@ def list_distinct(array: Expr) -> Expr: """Returns distinct values from the array after removing duplicates. See Also: - This is an alias for [`array_distinct`][datafusion.functions.array_distinct]. + This is an alias for :func:`~datafusion.functions.array_distinct`. """ return array_distinct(array) @@ -3234,7 +3234,7 @@ def list_dims(array: Expr) -> Expr: """Returns an array of the array's dimensions. See Also: - This is an alias for [`array_dims`][datafusion.functions.array_dims]. + This is an alias for :func:`~datafusion.functions.array_dims`. """ return array_dims(array) @@ -3271,7 +3271,7 @@ def list_empty(array: Expr) -> Expr: """Returns a boolean indicating whether the array is empty. See Also: - This is an alias for [`array_empty`][datafusion.functions.array_empty]. + This is an alias for :func:`~datafusion.functions.array_empty`. """ return array_empty(array) @@ -3280,7 +3280,7 @@ def array_extract(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for [`array_element`][datafusion.functions.array_element]. + This is an alias for :func:`~datafusion.functions.array_element`. """ return array_element(array, n) @@ -3289,7 +3289,7 @@ def list_element(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for [`array_element`][datafusion.functions.array_element]. + This is an alias for :func:`~datafusion.functions.array_element`. """ return array_element(array, n) @@ -3298,7 +3298,7 @@ def list_extract(array: Expr, n: Expr | int) -> Expr: """Extracts the element with the index n from the array. See Also: - This is an alias for [`array_element`][datafusion.functions.array_element]. + This is an alias for :func:`~datafusion.functions.array_element`. """ return array_element(array, n) @@ -3320,7 +3320,7 @@ def list_length(array: Expr) -> Expr: """Returns the length of the array. See Also: - This is an alias for [`array_length`][datafusion.functions.array_length]. + This is an alias for :func:`~datafusion.functions.array_length`. """ return array_length(array) @@ -3377,7 +3377,7 @@ def array_contains(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for [`array_has`][datafusion.functions.array_has]. + This is an alias for :func:`~datafusion.functions.array_has`. """ return array_has(array, element) @@ -3386,7 +3386,7 @@ def list_has(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for [`array_has`][datafusion.functions.array_has]. + This is an alias for :func:`~datafusion.functions.array_has`. """ return array_has(array, element) @@ -3395,7 +3395,7 @@ def list_has_all(first_array: Expr, second_array: Expr) -> Expr: """Determines if there is complete overlap ``second_array`` in ``first_array``. See Also: - This is an alias for [`array_has_all`][datafusion.functions.array_has_all]. + This is an alias for :func:`~datafusion.functions.array_has_all`. """ return array_has_all(first_array, second_array) @@ -3404,7 +3404,7 @@ def list_has_any(first_array: Expr, second_array: Expr) -> Expr: """Determine if there is an overlap between ``first_array`` and ``second_array``. See Also: - This is an alias for [`array_has_any`][datafusion.functions.array_has_any]. + This is an alias for :func:`~datafusion.functions.array_has_any`. """ return array_has_any(first_array, second_array) @@ -3413,7 +3413,7 @@ def arrays_overlap(first_array: Expr, second_array: Expr) -> Expr: """Returns true if any element appears in both arrays. See Also: - This is an alias for [`array_has_any`][datafusion.functions.array_has_any]. + This is an alias for :func:`~datafusion.functions.array_has_any`. """ return array_has_any(first_array, second_array) @@ -3422,7 +3422,7 @@ def list_overlap(first_array: Expr, second_array: Expr) -> Expr: """Returns true if any element appears in both arrays. See Also: - This is an alias for [`array_has_any`][datafusion.functions.array_has_any]. + This is an alias for :func:`~datafusion.functions.array_has_any`. """ return array_has_any(first_array, second_array) @@ -3431,7 +3431,7 @@ def list_contains(array: Expr, element: Expr) -> Expr: """Returns true if the element appears in the array, otherwise false. See Also: - This is an alias for [`array_has`][datafusion.functions.array_has]. + This is an alias for :func:`~datafusion.functions.array_has`. """ return array_has(array, element) @@ -3466,7 +3466,7 @@ def array_indexof(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for [`array_position`][datafusion.functions.array_position]. + This is an alias for :func:`~datafusion.functions.array_position`. """ return array_position(array, element, index) @@ -3475,7 +3475,7 @@ def list_position(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for [`array_position`][datafusion.functions.array_position]. + This is an alias for :func:`~datafusion.functions.array_position`. """ return array_position(array, element, index) @@ -3484,7 +3484,7 @@ def list_indexof(array: Expr, element: Expr, index: int | None = 1) -> Expr: """Return the position of the first occurrence of ``element`` in ``array``. See Also: - This is an alias for [`array_position`][datafusion.functions.array_position]. + This is an alias for :func:`~datafusion.functions.array_position`. """ return array_position(array, element, index) @@ -3507,7 +3507,7 @@ def list_positions(array: Expr, element: Expr) -> Expr: """Searches for an element in the array and returns all occurrences. See Also: - This is an alias for [`array_positions`][datafusion.functions.array_positions]. + This is an alias for :func:`~datafusion.functions.array_positions`. """ return array_positions(array, element) @@ -3529,7 +3529,7 @@ def list_ndims(array: Expr) -> Expr: """Returns the number of dimensions of the array. See Also: - This is an alias for [`array_ndims`][datafusion.functions.array_ndims]. + This is an alias for :func:`~datafusion.functions.array_ndims`. """ return array_ndims(array) @@ -3552,7 +3552,7 @@ def array_push_front(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for [`array_prepend`][datafusion.functions.array_prepend]. + This is an alias for :func:`~datafusion.functions.array_prepend`. """ return array_prepend(element, array) @@ -3561,7 +3561,7 @@ def list_prepend(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for [`array_prepend`][datafusion.functions.array_prepend]. + This is an alias for :func:`~datafusion.functions.array_prepend`. """ return array_prepend(element, array) @@ -3570,7 +3570,7 @@ def list_push_front(element: Expr, array: Expr) -> Expr: """Prepends an element to the beginning of an array. See Also: - This is an alias for [`array_prepend`][datafusion.functions.array_prepend]. + This is an alias for :func:`~datafusion.functions.array_prepend`. """ return array_prepend(element, array) @@ -3607,7 +3607,7 @@ def list_pop_back(array: Expr) -> Expr: """Returns the array without the last element. See Also: - This is an alias for [`array_pop_back`][datafusion.functions.array_pop_back]. + This is an alias for :func:`~datafusion.functions.array_pop_back`. """ return array_pop_back(array) @@ -3616,7 +3616,7 @@ def list_pop_front(array: Expr) -> Expr: """Returns the array without the first element. See Also: - This is an alias for [`array_pop_front`][datafusion.functions.array_pop_front]. + This is an alias for :func:`~datafusion.functions.array_pop_front`. """ return array_pop_front(array) @@ -3639,7 +3639,7 @@ def list_remove(array: Expr, element: Expr) -> Expr: """Removes the first element from the array equal to the given value. See Also: - This is an alias for [`array_remove`][datafusion.functions.array_remove]. + This is an alias for :func:`~datafusion.functions.array_remove`. """ return array_remove(array, element) @@ -3665,7 +3665,7 @@ def list_remove_n(array: Expr, element: Expr, max: Expr | int) -> Expr: """Removes the first ``max`` elements from the array equal to the given value. See Also: - This is an alias for [`array_remove_n`][datafusion.functions.array_remove_n]. + This is an alias for :func:`~datafusion.functions.array_remove_n`. """ return array_remove_n(array, element, max) @@ -3714,7 +3714,7 @@ def list_repeat(element: Expr, count: Expr | int) -> Expr: """Returns an array containing ``element`` ``count`` times. See Also: - This is an alias for [`array_repeat`][datafusion.functions.array_repeat]. + This is an alias for :func:`~datafusion.functions.array_repeat`. """ return array_repeat(element, count) @@ -3738,7 +3738,7 @@ def list_replace(array: Expr, from_val: Expr, to_val: Expr) -> Expr: """Replaces the first occurrence of ``from_val`` with ``to_val``. See Also: - This is an alias for [`array_replace`][datafusion.functions.array_replace]. + This is an alias for :func:`~datafusion.functions.array_replace`. """ return array_replace(array, from_val, to_val) @@ -3770,7 +3770,7 @@ def list_replace_n(array: Expr, from_val: Expr, to_val: Expr, max: Expr | int) - specified element. See Also: - This is an alias for [`array_replace_n`][datafusion.functions.array_replace_n]. + This is an alias for :func:`~datafusion.functions.array_replace_n`. """ return array_replace_n(array, from_val, to_val, max) @@ -3840,7 +3840,7 @@ def list_sort(array: Expr, descending: bool = False, null_first: bool = False) - """Sorts the array. See Also: - This is an alias for [`array_sort`][datafusion.functions.array_sort]. + This is an alias for :func:`~datafusion.functions.array_sort`. """ return array_sort(array, descending=descending, null_first=null_first) @@ -3889,7 +3889,7 @@ def list_slice( """Returns a slice of the array. See Also: - This is an alias for [`array_slice`][datafusion.functions.array_slice]. + This is an alias for :func:`~datafusion.functions.array_slice`. """ return array_slice(array, begin, end, stride) @@ -3917,7 +3917,7 @@ def list_intersect(array1: Expr, array2: Expr) -> Expr: """Returns an the intersection of ``array1`` and ``array2``. See Also: - This is an alias for [`array_intersect`][datafusion.functions.array_intersect]. + This is an alias for :func:`~datafusion.functions.array_intersect`. """ return array_intersect(array1, array2) @@ -3949,7 +3949,7 @@ def list_union(array1: Expr, array2: Expr) -> Expr: Duplicate rows will not be returned. See Also: - This is an alias for [`array_union`][datafusion.functions.array_union]. + This is an alias for :func:`~datafusion.functions.array_union`. """ return array_union(array1, array2) @@ -3972,7 +3972,7 @@ def list_except(array1: Expr, array2: Expr) -> Expr: """Returns the elements that appear in ``array1`` but not in the ``array2``. See Also: - This is an alias for [`array_except`][datafusion.functions.array_except]. + This is an alias for :func:`~datafusion.functions.array_except`. """ return array_except(array1, array2) @@ -4002,7 +4002,7 @@ def list_resize(array: Expr, size: Expr | int, value: Expr) -> Expr: filled with the given ``value``. See Also: - This is an alias for [`array_resize`][datafusion.functions.array_resize]. + This is an alias for :func:`~datafusion.functions.array_resize`. """ return array_resize(array, size, value) @@ -4025,7 +4025,7 @@ def list_any_value(array: Expr) -> Expr: """Returns the first non-null element in the array. See Also: - This is an alias for [`array_any_value`][datafusion.functions.array_any_value]. + This is an alias for :func:`~datafusion.functions.array_any_value`. """ return array_any_value(array) @@ -4050,7 +4050,7 @@ def list_distance(array1: Expr, array2: Expr) -> Expr: """Returns the Euclidean distance between two numeric arrays. See Also: - This is an alias for [`array_distance`][datafusion.functions.array_distance]. + This is an alias for :func:`~datafusion.functions.array_distance`. """ return array_distance(array1, array2) @@ -4073,7 +4073,7 @@ def list_max(array: Expr) -> Expr: """Returns the maximum value in the array. See Also: - This is an alias for [`array_max`][datafusion.functions.array_max]. + This is an alias for :func:`~datafusion.functions.array_max`. """ return array_max(array) @@ -4096,7 +4096,7 @@ def list_min(array: Expr) -> Expr: """Returns the minimum value in the array. See Also: - This is an alias for [`array_min`][datafusion.functions.array_min]. + This is an alias for :func:`~datafusion.functions.array_min`. """ return array_min(array) @@ -4119,7 +4119,7 @@ def list_reverse(array: Expr) -> Expr: """Reverses the order of elements in the array. See Also: - This is an alias for [`array_reverse`][datafusion.functions.array_reverse]. + This is an alias for :func:`~datafusion.functions.array_reverse`. """ return array_reverse(array) @@ -4143,7 +4143,7 @@ def list_zip(*arrays: Expr) -> Expr: """Combines multiple arrays into a single array of structs. See Also: - This is an alias for [`arrays_zip`][datafusion.functions.arrays_zip]. + This is an alias for :func:`~datafusion.functions.arrays_zip`. """ return arrays_zip(*arrays) @@ -4189,7 +4189,7 @@ def string_to_list( """Splits a string based on a delimiter and returns an array of parts. See Also: - This is an alias for [`string_to_array`][datafusion.functions.string_to_array]. + This is an alias for :func:`~datafusion.functions.string_to_array`. """ return string_to_array(string, delimiter, null_string) @@ -4197,7 +4197,7 @@ def string_to_list( def gen_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: """Creates a list of values in the range between start and stop. - Unlike [`range`][datafusion.functions.range], this includes the upper bound. + Unlike :func:`~datafusion.functions.range`, this includes the upper bound. Examples: >>> ctx = dfn.SessionContext() @@ -4225,10 +4225,10 @@ def gen_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: def generate_series(start: Expr, stop: Expr, step: Expr | None = None) -> Expr: """Creates a list of values in the range between start and stop. - Unlike [`range`][datafusion.functions.range], this includes the upper bound. + Unlike :func:`~datafusion.functions.range`, this includes the upper bound. See Also: - This is an alias for [`gen_series`][datafusion.functions.gen_series]. + This is an alias for :func:`~datafusion.functions.gen_series`. """ return gen_series(start, stop, step) @@ -4263,7 +4263,7 @@ def empty(array: Expr) -> Expr: """Returns true if the array is empty. See Also: - This is an alias for [`array_empty`][datafusion.functions.array_empty]. + This is an alias for :func:`~datafusion.functions.array_empty`. """ return array_empty(array) @@ -4282,7 +4282,7 @@ def make_map(*args: Any) -> Expr: - ``make_map(k1, v1, k2, v2, ...)`` — from alternating keys and their associated values. - Keys and values that are not already [`Expr`][datafusion.expr.Expr] + Keys and values that are not already :class:`~datafusion.expr.Expr` are automatically converted to literal expressions. Examples: @@ -4415,7 +4415,7 @@ def element_at(map: Expr, key: Expr) -> Expr: Returns ``[None]`` if the key is absent. See Also: - This is an alias for [`map_extract`][datafusion.functions.map_extract]. + This is an alias for :func:`~datafusion.functions.map_extract`. """ return map_extract(map, key) @@ -4429,7 +4429,7 @@ def approx_distinct( This aggregate function is similar to `count` with distinct set, but it will approximate the number of distinct entries. It may return significantly faster - than [`count`][datafusion.functions.count] for some DataFrames. + than :func:`~datafusion.functions.count` for some DataFrames. If using the builder functions described in ref:`_aggregation` this function ignores the options ``order_by``, ``null_treatment``, and ``distinct``. @@ -4654,7 +4654,7 @@ def quantile_cont( """Computes the exact percentile of input values using continuous interpolation. See Also: - This is an alias for [`percentile_cont`][datafusion.functions.percentile_cont]. + This is an alias for :func:`~datafusion.functions.percentile_cont`. """ return percentile_cont(sort_expression, percentile, filter) @@ -4668,7 +4668,7 @@ def array_agg( """Aggregate values into an array. Currently ``distinct`` and ``order_by`` cannot be used together. As a work around, - consider [`array_sort`][datafusion.functions.array_sort] after aggregation. + consider :func:`~datafusion.functions.array_sort` after aggregation. [Issue Tracker](https://github.com/apache/datafusion/issues/12371) If using the builder functions described in ref:`_aggregation` this function ignores @@ -4730,9 +4730,9 @@ def grouping( aggregate spans all values of that column). This function is meaningful with - [`GroupingSet.rollup`][datafusion.expr.GroupingSet.rollup], - [`GroupingSet.cube`][datafusion.expr.GroupingSet.cube], or - [`GroupingSet.grouping_sets`][datafusion.expr.GroupingSet.grouping_sets], + :meth:`~datafusion.expr.GroupingSet.rollup`, + :meth:`~datafusion.expr.GroupingSet.cube`, or + :meth:`~datafusion.expr.GroupingSet.grouping_sets`, where different rows are grouped by different subsets of columns. In a default aggregation without grouping sets every column is always part of the key, so ``grouping()`` always returns 0. @@ -4743,7 +4743,7 @@ def grouping( filter: If provided, only compute against rows for which the filter is True Examples: - With [`rollup`][datafusion.expr.GroupingSet.rollup], the result + With :meth:`~datafusion.expr.GroupingSet.rollup`, the result includes both per-group rows (``grouping(a) = 0``) and a grand-total row where ``a`` is aggregated across (``grouping(a) = 1``): @@ -4760,7 +4760,7 @@ def grouping( [30, 30, 60] See Also: - [`GroupingSet`][datafusion.expr.GroupingSet] + :class:`~datafusion.expr.GroupingSet` """ filter_raw = filter.expr if filter is not None else None return Expr(f.grouping(expression.expr, distinct=distinct, filter=filter_raw)) @@ -4976,7 +4976,7 @@ def covar(value_y: Expr, value_x: Expr, filter: Expr | None = None) -> Expr: """Computes the sample covariance. See Also: - This is an alias for [`covar_samp`][datafusion.functions.covar_samp]. + This is an alias for :func:`~datafusion.functions.covar_samp`. """ return covar_samp(value_y, value_x, filter) @@ -5017,7 +5017,7 @@ def mean(expression: Expr, filter: Expr | None = None) -> Expr: """Returns the average (mean) value of the argument. See Also: - This is an alias for [`avg`][datafusion.functions.avg]. + This is an alias for :func:`~datafusion.functions.avg`. """ return avg(expression, filter) @@ -5211,7 +5211,7 @@ def stddev_samp(arg: Expr, filter: Expr | None = None) -> Expr: """Computes the sample standard deviation of the argument. See Also: - This is an alias for [`stddev`][datafusion.functions.stddev]. + This is an alias for :func:`~datafusion.functions.stddev`. """ return stddev(arg, filter=filter) @@ -5220,7 +5220,7 @@ def var(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the sample variance of the argument. See Also: - This is an alias for [`var_samp`][datafusion.functions.var_samp]. + This is an alias for :func:`~datafusion.functions.var_samp`. """ return var_samp(expression, filter) @@ -5261,7 +5261,7 @@ def var_population(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the population variance of the argument. See Also: - This is an alias for [`var_pop`][datafusion.functions.var_pop]. + This is an alias for :func:`~datafusion.functions.var_pop`. """ return var_pop(expression, filter) @@ -5302,7 +5302,7 @@ def var_sample(expression: Expr, filter: Expr | None = None) -> Expr: """Computes the sample variance of the argument. See Also: - This is an alias for [`var_samp`][datafusion.functions.var_samp]. + This is an alias for :func:`~datafusion.functions.var_samp`. """ return var_samp(expression, filter) @@ -6052,7 +6052,7 @@ def lead( return the 3rd following value in column ``b``. At the end of the partition, where no further values can be returned it will return the default value of 5. - Here is an example of both the ``lead`` and [`lag`][datafusion.functions.lag] + Here is an example of both the ``lead`` and :func:`~datafusion.functions.lag` functions on a simple DataFrame:: +--------+------+-----+ @@ -6128,7 +6128,7 @@ def lag( will return the 3rd previous value in column ``b``. At the beginning of the partition, where no values can be returned it will return the default value of 5. - Here is an example of both the ``lag`` and [`lead`][datafusion.functions.lead] + Here is an example of both the ``lag`` and :func:`~datafusion.functions.lead` functions on a simple DataFrame:: +--------+------+-----+ diff --git a/python/datafusion/io.py b/python/datafusion/io.py index f85d586e3..9d72ae744 100644 --- a/python/datafusion/io.py +++ b/python/datafusion/io.py @@ -51,7 +51,7 @@ def read_parquet( schema: pa.Schema | None = None, file_sort_order: list[list[Expr]] | None = None, ) -> DataFrame: - """Read a Parquet source into a [`DataFrame`][datafusion.dataframe.DataFrame]. + """Read a Parquet source into a :class:`~datafusion.dataframe.DataFrame`. This function will use the global context. Any functions or tables registered with another context may not be accessible when used with a DataFrame created diff --git a/python/datafusion/ipc.py b/python/datafusion/ipc.py index 1d23d3bf5..71e2c2098 100644 --- a/python/datafusion/ipc.py +++ b/python/datafusion/ipc.py @@ -17,7 +17,7 @@ """Driver- and worker-side setup for distributing DataFusion expressions. -When a [`Expr`][datafusion.expr.Expr] is shipped to a worker process (e.g. through +When a :class:`~datafusion.expr.Expr` is shipped to a worker process (e.g. through `Pool` or a Ray actor), the worker reconstructs the expression against a `SessionContext`. If the expression references UDFs imported via the FFI capsule protocol — or any UDF the worker would @@ -49,14 +49,14 @@ def init_worker(): The serialized payload is stamped with the sender's Python ``(major, minor)`` version. Loading on a different minor version - raises [`ValueError`][ValueError] with an actionable message — cloudpickle + raises :exc:`~ValueError` with an actionable message — cloudpickle payloads are not portable across Python minor versions. See - [`to_bytes`][datafusion.expr.Expr.to_bytes] for examples of what travels by + :meth:`~datafusion.expr.Expr.to_bytes` for examples of what travels by value vs. by reference. On the driver side, call -[`set_sender_ctx`][datafusion.ipc.set_sender_ctx] to control how -[`dumps`][pickle.dumps] encodes expressions — for example, to apply +:func:`~datafusion.ipc.set_sender_ctx` to control how +:func:`~pickle.dumps` encodes expressions — for example, to apply `with_python_udf_inlining` to every pickled expression on this thread: @@ -79,11 +79,11 @@ def init_worker(): The thread-local sender context holds a strong reference to the installed `SessionContext` until -[`clear_sender_ctx`][datafusion.ipc.clear_sender_ctx] is called or the thread +:func:`~datafusion.ipc.clear_sender_ctx` is called or the thread exits. Long-running driver threads that install a sender context once and never clear it will retain that session for the lifetime of the thread; pair -[`set_sender_ctx`][datafusion.ipc.set_sender_ctx] with -[`clear_sender_ctx`][datafusion.ipc.clear_sender_ctx] (e.g. in a +:func:`~datafusion.ipc.set_sender_ctx` with +:func:`~datafusion.ipc.clear_sender_ctx` (e.g. in a ``try``/``finally``) when the sender context is only needed for a bounded scope. """ @@ -163,7 +163,7 @@ def get_worker_ctx() -> SessionContext | None: def set_sender_ctx(ctx: SessionContext) -> None: """Install this driver's `SessionContext` for outbound pickles. - Controls how `dumps` encodes [`Expr`][datafusion.expr.Expr] instances on + Controls how `dumps` encodes :class:`~datafusion.expr.Expr` instances on this thread. The most useful application is propagating a session configured with `with_python_udf_inlining` so the toggle takes diff --git a/python/datafusion/plan.py b/python/datafusion/plan.py index 474c584ba..faccc1de0 100644 --- a/python/datafusion/plan.py +++ b/python/datafusion/plan.py @@ -113,7 +113,7 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: @staticmethod def from_proto(ctx: SessionContext, data: bytes) -> LogicalPlan: - """Deprecated alias for [`from_bytes`][datafusion.expr.Expr.from_bytes].""" + """Deprecated alias for :meth:`~datafusion.expr.Expr.from_bytes`.""" warnings.warn( "LogicalPlan.from_proto is deprecated; use from_bytes instead", DeprecationWarning, @@ -122,7 +122,7 @@ def from_proto(ctx: SessionContext, data: bytes) -> LogicalPlan: return LogicalPlan.from_bytes(ctx, data) def to_proto(self) -> bytes: - """Deprecated alias for [`to_bytes`][datafusion.expr.Expr.to_bytes].""" + """Deprecated alias for :meth:`~datafusion.expr.Expr.to_bytes`.""" warnings.warn( "LogicalPlan.to_proto is deprecated; use to_bytes instead", DeprecationWarning, @@ -191,7 +191,7 @@ def to_bytes(self, ctx: SessionContext | None = None) -> bytes: @staticmethod def from_proto(ctx: SessionContext, data: bytes) -> ExecutionPlan: - """Deprecated alias for [`from_bytes`][datafusion.expr.Expr.from_bytes].""" + """Deprecated alias for :meth:`~datafusion.expr.Expr.from_bytes`.""" warnings.warn( "ExecutionPlan.from_proto is deprecated; use from_bytes instead", DeprecationWarning, @@ -200,7 +200,7 @@ def from_proto(ctx: SessionContext, data: bytes) -> ExecutionPlan: return ExecutionPlan.from_bytes(ctx, data) def to_proto(self) -> bytes: - """Deprecated alias for [`to_bytes`][datafusion.expr.Expr.to_bytes].""" + """Deprecated alias for :meth:`~datafusion.expr.Expr.to_bytes`.""" warnings.warn( "ExecutionPlan.to_proto is deprecated; use to_bytes instead", DeprecationWarning, @@ -227,7 +227,7 @@ def collect_metrics(self) -> list[tuple[str, MetricsSet]]: DataFusion executes a query as a pipeline of operators — for example a data source scan, followed by a filter, followed by a projection. After the DataFrame has been executed (via - [`collect`][datafusion.dataframe.DataFrame.collect], + :meth:`~datafusion.dataframe.DataFrame.collect`, `execute_stream`, etc.), each operator records statistics such as how many rows it produced and how much CPU time it consumed. @@ -235,7 +235,7 @@ def collect_metrics(self) -> list[tuple[str, MetricsSet]]: Each entry in the returned list corresponds to one operator that recorded metrics. The first element of the tuple is the operator's description string — the same text shown by - [`display_indent`][datafusion.plan.ExecutionPlan.display_indent] — which + :meth:`~datafusion.plan.ExecutionPlan.display_indent` — which identifies both the operator type and its key parameters, for example ``"FilterExec: column1@0 > 1"`` or ``"DataSourceExec: partitions=1"``. @@ -264,9 +264,9 @@ class MetricsSet: """A set of metrics for a single execution plan operator. A physical plan operator runs independently across one or more partitions. - [`metrics`][datafusion.plan.MetricsSet.metrics] returns the raw per-partition + :meth:`~datafusion.plan.MetricsSet.metrics` returns the raw per-partition `Metric` objects. The convenience properties (`output_rows`, - [`elapsed_compute`][datafusion.plan.MetricsSet.elapsed_compute], etc.) + :attr:`~datafusion.plan.MetricsSet.elapsed_compute`, etc.) automatically sum the named metric across *all* partitions, giving a single aggregate value for the operator as a whole. """ @@ -345,7 +345,7 @@ def value(self) -> int | datetime.datetime | None: """The value of this metric. Returns an ``int`` for counters, gauges, and time-based metrics - (nanoseconds), a [`datetime`][datetime.datetime] (UTC) for + (nanoseconds), a :class:`~datetime.datetime` (UTC) for ``start_timestamp`` / ``end_timestamp`` metrics, or ``None`` when the value has not been set or is not representable. """ @@ -353,7 +353,7 @@ def value(self) -> int | datetime.datetime | None: @property def value_as_datetime(self) -> datetime.datetime | None: - """The value as a UTC [`datetime`][datetime.datetime] for timestamp metrics. + """The value as a UTC :class:`~datetime.datetime` for timestamp metrics. Returns ``None`` for all non-timestamp metrics and for timestamp metrics whose value has not been set (e.g. before execution). diff --git a/python/datafusion/record_batch.py b/python/datafusion/record_batch.py index a24b80b9d..ebe4a38a2 100644 --- a/python/datafusion/record_batch.py +++ b/python/datafusion/record_batch.py @@ -18,7 +18,7 @@ """This module provides the classes for handling record batches. These are typically the result of dataframe -[`execute_stream`][datafusion.dataframe.execute_stream] operations. +:func:`~datafusion.dataframe.execute_stream` operations. """ from __future__ import annotations @@ -36,7 +36,7 @@ class RecordBatch: - """This class is essentially a wrapper for [`RecordBatch`][pyarrow.RecordBatch].""" + """This class is essentially a wrapper for :class:`~pyarrow.RecordBatch`.""" def __init__(self, record_batch: df_internal.RecordBatch) -> None: """This constructor is generally not called by the end user. @@ -46,7 +46,7 @@ def __init__(self, record_batch: df_internal.RecordBatch) -> None: self.record_batch = record_batch def to_pyarrow(self) -> pa.RecordBatch: - """Convert to [`RecordBatch`][pyarrow.RecordBatch].""" + """Convert to :class:`~pyarrow.RecordBatch`.""" return self.record_batch.to_pyarrow() def __arrow_c_array__( @@ -74,7 +74,7 @@ class RecordBatchStream: """This class represents a stream of record batches. These are typically the result of a - [`execute_stream`][datafusion.dataframe.DataFrame.execute_stream] operation. + :meth:`~datafusion.dataframe.DataFrame.execute_stream` operation. """ def __init__(self, record_batch_stream: df_internal.RecordBatchStream) -> None: @@ -85,7 +85,7 @@ def next(self) -> RecordBatch: """Return the next batch. See - [`__next__`][datafusion.record_batch.RecordBatchStream.__next__] for the + :meth:`~datafusion.record_batch.RecordBatchStream.__next__` for the iterator function. """ return next(self) diff --git a/python/datafusion/substrait.py b/python/datafusion/substrait.py index 6cbba8e2b..c90e4d3c0 100644 --- a/python/datafusion/substrait.py +++ b/python/datafusion/substrait.py @@ -49,8 +49,8 @@ def __init__(self, plan: substrait_internal.Plan) -> None: """Create a substrait plan. The user should not have to call this constructor directly. Rather, it - should be created via [`Serde`][datafusion.substrait.Serde] or - [`Producer`][datafusion.substrait.Producer] classes in this module. + should be created via :class:`~datafusion.substrait.Serde` or + :class:`~datafusion.substrait.Producer` classes in this module. """ self.plan_internal = plan diff --git a/python/datafusion/user_defined.py b/python/datafusion/user_defined.py index 93bf5e03a..42e56ac73 100644 --- a/python/datafusion/user_defined.py +++ b/python/datafusion/user_defined.py @@ -162,7 +162,7 @@ def __init__( ) -> None: """Instantiate a scalar user-defined function (UDF). - See helper method [`udf`][datafusion.user_defined.udf] for argument details. + See helper method :func:`~datafusion.user_defined.udf` for argument details. """ if hasattr(func, "__datafusion_scalar_udf__"): self._udf = df_internal.ScalarUDF.from_pycapsule(func) @@ -179,7 +179,7 @@ def _from_internal(cls, internal: df_internal.ScalarUDF) -> ScalarUDF: Used by `udf` to surface a function looked up from the session's function registry without re-running - [`__init__`][__init__]. + :func:`~__init__`. """ wrapper = cls.__new__(cls) wrapper._udf = internal @@ -273,7 +273,7 @@ def udf(*args: Any, **kwargs: Any): - `return_field` (`pa.Field | pa.DataType`): The field of the return value from the function. - `volatility` (`Volatility | str`): See - [`Volatility`][datafusion.user_defined.Volatility] for allowed values. + :class:`~datafusion.user_defined.Volatility` for allowed values. - `name` (`str`, optional): A descriptive name for the function. **Returns:** a user-defined function that can be used in SQL expressions, @@ -493,7 +493,7 @@ def _from_internal(cls, internal: df_internal.AggregateUDF) -> AggregateUDF: Used by `udaf` to surface a function looked up from the session's function registry without re-running - [`__init__`][__init__]. + :func:`~__init__`. """ wrapper = cls.__new__(cls) wrapper._udaf = internal @@ -641,7 +641,7 @@ def udaf(*args: Any, **kwargs: Any): # noqa: C901 - `return_type`: The data type of the return value. - `state_type`: The data types of the intermediate accumulation. - `volatility`: See - [`Volatility`][datafusion.user_defined.Volatility] for allowed values. + :class:`~datafusion.user_defined.Volatility` for allowed values. - `name`: A descriptive name for the function. **Returns:** a user-defined aggregate function, which can be used in either @@ -785,13 +785,13 @@ def evaluate_all(self, values: list[pa.Array], num_rows: int) -> pa.Array: This function is called once per input *partition* for window functions that *do not use* values from the window frame, such as - [`row_number`][datafusion.functions.row_number], - [`rank`][datafusion.functions.rank], - [`dense_rank`][datafusion.functions.dense_rank], - [`percent_rank`][datafusion.functions.percent_rank], - [`cume_dist`][datafusion.functions.cume_dist], - [`lead`][datafusion.functions.lead], - and [`lag`][datafusion.functions.lag]. + :func:`~datafusion.functions.row_number`, + :func:`~datafusion.functions.rank`, + :func:`~datafusion.functions.dense_rank`, + :func:`~datafusion.functions.percent_rank`, + :func:`~datafusion.functions.cume_dist`, + :func:`~datafusion.functions.lead`, + and :func:`~datafusion.functions.lag`. It produces the result of all rows in a single pass. It expects to receive the entire partition as the ``value`` and @@ -926,7 +926,7 @@ def _from_internal(cls, internal: df_internal.WindowUDF) -> WindowUDF: Used by `udwf` to surface a function looked up from the session's function registry without re-running - [`__init__`][__init__]. + :func:`~__init__`. """ wrapper = cls.__new__(cls) wrapper._udwf = internal @@ -1034,7 +1034,7 @@ def udwf(*args: Any, **kwargs: Any): - `input_types`: The data types of the arguments. - `return_type`: The data type of the return value. - `volatility`: See - [`Volatility`][datafusion.user_defined.Volatility] for allowed values. + :class:`~datafusion.user_defined.Volatility` for allowed values. - `name`: A descriptive name for the function. **Returns:** a user-defined window function that can be used in window @@ -1132,7 +1132,7 @@ def _wrap_session_kwarg_for_udtf(func: Callable[..., Any]) -> Callable[..., Any] The Rust call site forwards a ``datafusion._internal.SessionContext``, but UDTF authors expect to interact with the public - [`SessionContext`][datafusion.SessionContext] wrapper. This closure wraps the + :class:`~datafusion.SessionContext` wrapper. This closure wraps the internal object once per call before delegating to ``func``. """ @@ -1172,7 +1172,7 @@ def __init__( ``with_session=True`` is only supported for pure-Python callables. Passing it together with an FFI-exported table function (one exposing ``__datafusion_table_function__``) raises - [`TypeError`][TypeError]. + :exc:`~TypeError`. Registry mutations performed through the injected session (such as registering tables or UDFs) propagate to the caller's

x6C^NjF2J3m(Je<2+;640kKUYK6*|wB&(X9( zc+|DKqnGYGn-Y=k43a*nxCCS)M}E6Kf&?_e75hTGK?9r&iu~$cxNmBx*g?gr?VgG| z?R6I7N2Xw>;&tY^+lpx&@PUhDA&$?Imy&45EjKQ)k$bvWV6|_5WYMKeWP{1%CSCW4 ztSfF1>V+yHl*X7f8S8gnN4<}Kd^4?W_kBVfj>A+vyjeA9+CRKTW56kR<0W|B^$9*( ziQUPL@pAcEUg)Qpe_M{Ab3uTFMF)8{kq4Ws|0MKObDcf6q-j7>c0#+09=`t~9V>TT ziK9)M_!afuRi3&3H6HvHtRj$=d*jBzp-t`$DBUSY8axZ}6f!Vkf_kyyU+=V_<~wvM zJ@V-)KfV?zO+@YeN6xk^*6d%>kQx5R@bvfbqJ`JPtBwWKx=i572gWrhEN9M|5m|6< zfe{a1?>~}USMlVvH1vR-6V68E6Lffe_K!u(2$|aEOXQ4Bo{s5gbb3czH-nnhv$*JC zrRP9fPpS4iG!*#yWxUsyH-BgVGP$_q%+}Bk7=dqfMLaD?8B`(IbAkJQqIA&)A{UN* zP)i5Xq8w>Ozca4>xU?X4IZJQ4h>1HG7~H674O(v>`~?OCl-Be9+{QD!zV?I(gK(4JrnGM~!oqn#>C9HY+jj3SPxGIhRdepA z!@5s#@vVBym4xTgQG+yo_Mu^*;z`+U2XD7%^;P3u8r-JJ7phXh;^T_<|@zai#{t7P!GBPrsG4j3o zkujKVu`{}jS6rFay&ZZwZE@BWmSa>RugwHWE_XdoyKx|ug113(7r=YRI6%hl`+c_2c#nI@UJ!7Wl0+%w~$f1+I z8dOSEm$i`zSN#@30CnVMHQPw@xXwZ0`*GytvsrcxqruN{b@SI2wxiHT+p zbjV9NZ}p1|Ax4I_1qmfo!E)HDHb&nkx6U;zYb%IBucR1K2-gX2x+7wpi~H+nTi!a6 zDAPoVChYX>xQoOrtDR>mVGM+QMm%rNr=(`;mCLJ&T&}juS3=xYuO2nkkQ_27XNK-4 z1dO|=%#U{|%cC|X;@@4k8RP0foMjryYZqS6>DN_2R(Q28$I1D~gJ<7-OW*T-|LxFG zMxuhZSaV`mpqG86IVuXYD)HEE47hV@sGHf3dI3I)Y~Z;=H@!eW|F~^cXGE16PhS;{m`g=(WIDwhppCA3GZ5GVQ6qY$d_5iNO z-${h6oR@xhZP#(#?Jg&cIu!FED=W{Vr2VwHX4Xfmzea9%y@g_uv7jw0OV?q9l>r@x zuqW!OMc3w_tpEs(_BTb4%0Ezru-k3u79@kX!Pt$uJQ&2t;g(<_w84QFhMj+;yU7S_ z0$g8#wi5eHVZ%;ygXX6LwInxGoCamwfx9mJw20!8%_gc0Aar9D+}PV-v2!0?n*rm%1-)iTh>$wOEgI^p_s)%K0v-^3|eLr3i9n@0hAS-&T(lEmk6oz zxfUpw$Z({a?<28r+nFILz|q~|LiEQI`%mFVsjy$c+Y0q+lhR)ANHe`W$8z-ab_Y^q zW(OwT@u1rnX|f<9Zsq25H~JiJ&)-*XYLj^87Ve80Q4#npcgv5(O4o~kc}2yU5dDnc_k89x{bIGKX@m8>EUf(TPTq z&U-F-rE1t*43Sl6CBn-)f8LK6_pd9X;Qh|NR63>3KcB+;WxE~(=d0gqW6dcbIGsYO zSqRr@;Bq3$61(5-C@>OJwSK?;#KFaHc~nf$`eHWUJPS|W^lf$igLXsW;CHq;A4et6 z)}MmrdwfzPIp7pS-wRtl6?Fg`IY-H%DUsX?^#Wc*FG5p^PdluQ_phUrHE1tHV5i3| z;xB9M{WFQP$X62Sv#cLETYwd|A5EF7&QSgU%;@mX0nW{ssDSqFSA|BIO#FsI%O1ab z2lr73Mv2RPs2Y*%BNbG3t+t!M<(YBWFo$R8?;%`=Nl6MZ?pC|>+?t&>-Cx6d4_nEj zQ~|X<1TI8Wbr_*S;xC5LCvv0E@{dE)5=8N-q#KJ?qYE+a0EB=S>Ozb9V1)Vi;{RevZi5ed`s!s@LX^s&fg<`lzt)N6*Ni&DW*;(F$ZNud2cS8VSK z{j3`dYAs~yll^_B`OoE_%Tz}qJ_AX?Q$<=o9h2Hm%!RBvO}i8vplm*tOk&2zM&$9~ zvxR~gl>X0(tIBU-Wbez-{QEPxa6a~w@}##k5WU^l6AZM*pFHZ+K)|L@AlXIT9fxNx zya=W+u`MT;d;JXIl#L>4PaUQr+y5T^%FPyU(@cxtCSLuzmquA;R*=(i1xnQ4J8h^V!!bViOwO%^Vq)U( zNri>js_ND6G+%_pV8NSJLYp*DaTQQeNA|3_p&z{%riEk;(Z>+POtZ z6;vZr<;Lc)UKjr>k#wRzes#YnMX-%i9wf%A;da`6ZzCwd!mazxd7E;sG3&u&!@-$y zt;q~QW6NUlLs^3+ielqs3EM-mQu7Ug?S^s=(dr>VEuk9NdcA_$sO7V!|y6#Wdq- z8|`49$V+&}-tHy{G#|nrpO7BmZa{hA4proL9YC5hv0$?Jc>TyDV&*i2nBf=eEQTZQ z?&Elh6p!8qvs8Pj%4LZ=c7*P0OtW!Y21rvC%Eor$5kd{Lo*m?hsv^*arEhkmZWB5DLa#kQM|L02lTXp$IdLS*= zVT=c5zJDv>m(D@y_el$g|CW9~4CwdOVsyp+XOYqWzp}`yKp(goAD{F$p7ndIF-D%z zL*rNZS)+f4F_O=8K*fHfE_T?zjidKi0x}@>|C5CK*C@z`E^H9{BHB!>e<$|QQCLOh zCxL$(_^54y1;qY>Qhes$SPy9uRP6g@kNjJ31$=@{DvDiUcuOC~#-C*H@;!`a03{=mJd3 z)Z4r~pJ*{PX7SFg!H@k0EUi-ZtSeX2J^z}dc$HedNYxBZ^Ge47(E=N(ci)JgaK_j* zpDj-U=s4p$Xn>LoRN#S6f8#P?;yO7vvSc34l>mnD_~czE)-TkyT-l+7&M+s5(>H`u z1Aj3rz#bh21W_8@n+^kP1*x&WfqY2e44?v!xs9C1pH`tJWg?x;#QY$`11JweRBf^n z)D;9hf!$5`^&lWG3?<}{-_*l+XRhqMH;gD={ycd8rhX~R5v5jt4W@n24x6l1UJ-vf zkbL_cSZYY3CZIlR)IiOf`v)MD{v0*ww9KuOgR)HcqG^y~s}VJ5wE>_M;bLL-G)zQW zedBsYTscj>2W!mi#SKK1TRv07V>zfgUlmwbF3-7p&kYV4O!s3GEx`oPr+;Z~Eig8! zVPecBY07BM)9MF>sVJNBe}Hht#`@ClwWNQ*!_xjKAyC; zR-|^2UyFEy;-G-ItQaQ?5$yufi$j$^U{s>%F-p%Wg8FWKKTJKhOyh?3;!P!mw&iSaREIPR2tC@dmN;#JxoJs#u`mSTN0F)6;(S+9n6!&HsnFt5#% z@+?|Zjp7)|)u*m%Y z3`J`~EhH?vwrR^QaDqR52R-}%~879i}JMua)^^~&l+#=wFoLv&M?5AtdVc6u ze+60!17rqfAO7Ol-hQmGcSdnrJW!~XAx$WJ!k)Ah;AO@fI`FO+x$QSquFP*|ffZr~ zvEehu5!qPntWCxkR`I6EeXLLf8L^6jF1tLD%2BjzF+Kg1lB`m=QVfp!5Iv%DcE@YQ zyKL?2p!Y!krS)>l@4=mOuWip|slH_XS;uA03FkCNz@W5{^5gNU+{;R}M(5Iz3LN>u zMG{mn`x4D$S{-$g@gK(K=j4|8T@za{I+uOlEZ^&*vKHCkH))bb?^6DQ!sL#@-O`Z3 zH6#kxa6vVGd+_+`7Bvir7?QU?EYtN{%aN%xn=;bMK91AK;EfZmv)(UEv3djMz!=WW zfD#2i3Z4o-Fg^fhmh%tyE7a-zp$2_F=(S-Dpf99ssNmZvvS{ws7Jm)+O}?&2b7jt; zHy=p(<fgx&>VlJm`eR1NV#-^mGm{t4qP$97`XhhB&K4&cSLmj*40Z(6F!|PJakzMxM`;D^4UpZSrb((BAH>)! zo&8GUUAY_(-!V=f#A~Y>7=1AKIIQpBtZ}1iU_((D?aTul>*q4{m7aR<&uog5jVlaI z-qo;W^rAajXx`On?<}4sr=NwK8V+VXI)-t5#R$cM$4N_WIT=E=1~U)JkDCrD z7gUqC2Zh9Oyq6_m7hAR5ND*Y^8FLdwrB`Q~XSjmsS>!3$U8$O&n16W=Xrjfv>d7X8 z$A%4X(d6d)`DO44o}9n-c)~qFH0$Lws=A&0V`5UW2z)7FPFCb!1qh%nDi^bXV{# z!3#Zz3shd=(k0wa`?$|zh4qnCsKU(){7=q>bdB#c4Y?xFm7qxVW~(2ZcV9LCP;r-v zw5+h_qWz*}>WK19%g7?}CjsU1$0{{!4C?^V)e||0@jLZz)^hf{U1yD*Ola@@pZuZ%8INVfVCHAvjvgqw~=~(HB8j{;a&tlWhp1C@vFVQFi zf<*tjXxSZl9u+jm=MuOW$mPPXb41&EqXm(O)f>Yf-JQz$p_$w=zA`tpDkZWyQA$-_tL+P4u;HaSUjwa4`cfxsVriqLe^IBs z|LYGQ!YD75a0=V#pN~5NPeLec`m3sn9KQ2o+k(>E7sdP21Ypc|rvv3)fZCm&bY<#% zg-aVw+&j+iK)#c>L%|-ghG=^qY13SHP$JO3(wA{xU~mdlfRN?z-+;p5J@``QZTI8L zwPH%+hYE%aE4u{xj1{6;0F?D$N@-oF;Q{dv{JF@p59G31)-Nq4Dwt*qW~4IIOR?gD zUa0QB#I$x-CwVYS>7HGr^^^G#nd|=jr6BLPYm?JsOx}R5nU~s;Tg?~SFb{LiW4jgv zT_2=Ejp7Q-lO9Z%Gk<$Be?LGVzd-qQ2c^59-oQlDJ*0NuldSo%?Y!$Mj|dOB5rW&) z6eR~Qw}NHlBtcO>{95Pf<8*G>=lQ%yyGRyX4OL~+F+}ee==7NJC^&&HW3Q;qK4P%-39f@x~*hl}` zlf+<=Mps-axmZ#)@KKN ze)Ihj(mLKVaEcn6I)eO_Jdwva?sqT|9kW~8Pr+sxhxt&Jgzl}zS?x2k{@vIze?I@) z9nZ2$6?1ahjeF1_fJnx{!kT7R~Hh| zoPZCy5?RQ|f*RIr*_Uy`yBN4soOApK?$hmxY8^ZIL7cPE;^(V5)}2k*w#{>zk^rJb zyj&d2vN_D5l1;eGmw`}G{1AGSPzG8*Qpd)W1*4Bli2@w(3;8W$n7bGR?HPd+${J1s z7$i{`G!u5y-h#GinT6)&Irb zTSis+w(H)4h;+xKg(*mPHzE^|4(V?fP3EebzgCwzvDO%UAQSNrGsDlKaIxQjxnpN{*79TX?G+- zlr)@MH*K@$C!kf6P{$m0IE0VO9^$`CQWq~xVQS=V@_vJOjZEaXxp4f= z8<<$-Unoe_iq39wm8W9T<*)FaFVNbdb8mK#b$JV910$>Xo?SsUD&>16>H^W$hNjX; zrwSCJ2kVi2L5G{IXBAXme^JBuItWuG#|G_(pgqw-$6bGn_ix=?TG!xgy%0kRaf(9% z@1SDI9=_VeX8qQPn66EuWk8*#u3p3wmV-~`aip%^ zFmLPe$7~VqZ*!x#1?^5JQqv`wlBf!$7t{lzX2!RFtt^~rS;e~F8tq4SlwcdQt3YdE zYHe31+tGYFq;F;#EqtF#*!e2w*g!QM|K^)_&L>=AFJ&K`6p;Q&oFs4j7I7{u^@s6N z&CWcZFm!o%p}2?AdY;$T26V?gjGA7q@~$XbMh9VW+F<~0jn`Y_=eR@=`V1c7s)q^! z9yP+YC&2;XQ3ZAmZ)@-LW0B*)2=Pd-FPeB6xceb&Z~5nNr(z6g-7pUm%d)P8?68z$ z1`4|7!Gejbc(r*$=dbb~V22TqKXqM` z@3xx!@QyJWMyun!;O85Xr4g6rmg0!D-}Vs=2ylIv@5v)uHR*X99!SaqHE+|oAL>;4 zcfe{Tj@Gs{%aI%*^bwgC?7T+P$_F$+jLp@99I1yD`D4g1vzEdjQ)|mhF2_{~Oli2= zue#;T-m&T!>;5h-TQkU_#orH_ z@(pHH`w97|Y=L;X0i`zl+46(|EbPI%vdOXxE{|4zmSxP$$7RSmauNqCgR1(>!y?#K zlR-AAF%-97bLz~0jWjpyglLBFY@hIr+*AEIJq9ZkUn29tRn0WhzBPX5?!QWShJoVr zX3H%e-ITKc?FN-Z`MkmSO(}w|KVH{NRry+797Ts+$gTk*jt@5(L9CAZ=fGdX?Tt!t zz=85XD~)za(i;O(d5S_Ugcd}|x<%)p!$=u*6h`u>LIp%M#r#G?J}Sphqh?}c!x@yW zxbk-ezfI?RE#1@zAB`DWfujR;djaM~0{T8P9ya-z{0Mp@ep}=d(jDseU;M*r7be97 z^Mu~5JTwUrNo~Cg&25>cs@24sYYoiMV+h};85-~Gh|_++Ie4&F)m)Xvlz7>>C6KI{ z-yIYtv&$9Z@8`9+<{_KKrn3F;yrHuxT)2VK-Qx#cc@ItC%|L&dpGQ?jkk@1#vapTO zlug9J`aB_LnrmYSu}0@M_g+;vxa}mQ+K=;1jcq;QC|}|Psl{`#mA1p3E@}Ef(PGW& zh=N-2hx|V6j+xqjGite2%>#81{RqzAg{P&a? zG2|XN>B8iE&c{kX1>5IK<=D0-+jsk$m5(dmVGM8UYEr^VgO%HsGBUoEeZ3O=Y_}cG zDXI4>%<;QVk!upxE;Z^wl|W^Sy9d)X#YU2egug^3D4HGm{*Y#m8&~>Ke1eT@Wqc@B z!Fi2E-KD^&+>d#1HfORY&D|JxpL&ByILqHTGHQ6r@O@ts649wtR|rAT$N$VKb=en%}gR zBDvX(G()IcMlvv&Rk+bGn#<*@<-+GZAl{KE^d)k)&GLxD2_g=WO^h&tSR7MxR>tA*;klFoo)DyG2?4`$VZuI+oO?0 zJ=&2xNq!l~d{O*Px`*(N=tjJ}9XAL_*v?IPzxs=9)z9NbN*ByXQYvJPHLLcf=CWsC zvW=8YVZ;>&Wl)=WtcD%w?Q7!*K_MLUUV*ThGpoZXoW2=OYz|D0VIK;*hcND*P{(am zD(ADF#^s=dvMm2|&v4E)p~|)5BT-OOEU50JcJSu8k|}==CUEu>_Rsr`i#Tzr0@6K` z@2pX9v&%7N6sgmP@G(nzTWaDY_T_l;zNNo*3lCj;)&g3BqdY^j*G}$i1_8Up-iw#W z*xq~0q}kKPFFyCaWotBDw+gQ)^S5=Bj0xwI*_GB~0b`9<(vaH1Ss6PDvpKdjv*ibK zoOUUD^|=uFz}=(=f#O)pmCx%5F8Y@lBJPt83dhc-Yz9)(N&bAegneFWqa4Csa|&~a zgfFd2U1{Z9vgupT?#o|>jXH%3s%MR`8qAl z$&YzQKNJH(!#S;Z9hQI4!yLwa`hHlCA&udl`CC9dXWYUV-LRp#JI4^iDEUs?EdEx) zdSWWsTRzp?yc9&@8bFedSZWD^wv>JO_w$u!n*_h=Q^Yw6*l1y9yv^Z{c1^6E{OiMG zONRVV%pj$BD}4KGrRFS(IZOC4tq;XupE8l?TytF|i41xQNEV_g=}PE*jc+Wu?v1&_ zyb}?m|9qw>S43L3C@B{`@p>>rGI8etFZ`Ow&z-im;(Kos6O}X?tj5TYPF(xEcnT(R zbASIa4|};(%-2OlVrNHMas7+AOW}O4JZf9xzJ7bR?+1nZ8&`zS)Pn;Bn`bCZY26oX z+Ux!>*bZd;@+gbWZlJLW=e)$rCy3me;Nhi-5Np(VgAYts)kC)Bc|RD^DYwaMP+hH4 zd*q>Tv?V7maTzjsij-(J9G`?QrB>pt!7nJ#hz)*WgQez%vrVoaTW^cj z-)dUuR0%y3UB6JjejJkQnRqS0D#8PPk(f^Jx#K(VmnHA(SoPI^w5ls@LKv1}kq`!dF3tf~1qzmLpuY8l|! zjD1pD^Ev9Je1D+-=Y_Wd&3#u|a0s~&<;}R?9eLJxx}y8fgPI|BG^#p3k8{r@e$w|Q zp363y((jGrjLu`}kM`d31hYw}kn~CuwJ=LZIWcKAD;&q=%E1Twtr1!sY2FdhbM;ZL z>3lNmLoxj&s`r0j_}>;M88Qn~hX@`*cdww6mAo9iRygD<6yH`|5>hlpC=S5xoP(wW>k-`-0h{<9+HqQ+yo$S=*g zua7!;9;;W?S8Goe@g(8s^eg1?pzHkv%~r~ZqfqNV8pH9MSD4$R)&yX(re*3cZw3(2+CSMX`}feo+KwGY)mx(17RV@4Vti zIysRcxHEYn|IVJ)GKoY7l0a{U$!9Yk03kkfuHE7fBb^lDRy$7d1djjmsb_*JYVEsI zEVM;d*JW_Qz|I%2e$^-gG4-}`{wF@Y4QDu~IwFXl&@)?w=j=<$t9)uldBeQ@Qo9S& zz6!LTzKC)HFL5fW7q*0!R%t1!l)+UOopIY1YB}sRjaWX2!8>N;;iE#2e$Zj>!Z0%L zQaM^rrJp=Z`8MJcxBk=|rXQfww$gX$VD_ca*56>Enq|CU|4FN}C%4PMoRZ^0Oe&AV zkjfK_Ni2iEis2Q?D(RhT)|;x}iOjx03iQp9zKpG_<(xhP?@{ex$zT86MI@v1z8L5+ zz)htaU8(`E*WGs1*L6mM+MGaygIU02fBY3@tp&N#qOI|^g48|E91IIcR^b#O8vr?Y(*6=l06EGaaok@8$PB% z_BZFe4H{3?L6S})^Wc;6wEasK%=DL6eblTITNgW(?B&czINjg4GIKS zvK?NuI?&nkGeer4Bjas2By#yCdg85nNHqGY4{V3J~X=ai--C zSWys3Xnreeqg6EfLcn(wvFC0Dj3-M6fnJyge82xp5x-IcFDVcK(LPpZtciy{2yLw;vFAACkl3}mlL4QMPx|$h0r?oblyIGK$U}bON3kgWR?TKOAIppTg z0nRjw6%Mc8sU-OW=^j`XN8To%uaXC$+{cjYlO2$FOitP82r5h(gzm@MBzHdfVb4yo z2*}v^EjA^zL|>0L%qGcEsgRg%Qm&dapG!>&)||oLq2btjwJ07Z&2Huc3;d_!ItAH7 zg{AYG_rrsvY4-`$9|)3Ze=_>qbsBV+fb~L0FRTw6A#!|?NnBFeM%rHv+sCK(Tbd9- zO$D<;{s;B0O706$8;|UWe>B|`4NkYMsH`ptDTw{rq160cFpz&E+q%(;_#FP)oRz&? zo+;^MQGz=$u%39wexk4Vd!K{tzSwBtIPPL2n6Q)Nt;MY-!5r}=G4)zYW?fc^nt#0y zcrl3XnQUK+(uI-@U8r|ls)Le}*HfKaO@dxLLm)8CV?^d`hwwhX;E%tO4=^8$|E9?>0ITmr|jy@dX$mGCCZ!e{v4<>+L{ z`9`5i)KvE%4MvWLSsfrLx)rUNP6OIbFJ8wF1pFyCA31KYWq17=LovV3@b}zs_OSJM zi3xxLE zb55>=tj&S%Y;)AB`rk#*^k#`OhEG1Pt7=i|H^@g`OaT)Lu_O)K^p!a+ zzWa0udW>0{qfRM221UjtEn1HhECw(t{wkT1-Xz+HV^_4_UMl>3-}c~(6whaC>r>#g zB0R77#N4ggZ;7Mqpl~cpLYBgp1j)=|ieyAG?_Dx}zUz~!(clzEGuS91KLcJrQ`V`B zK9ZuOh^Yqd+$wl9@dI;-AL5X zF8omP?p*M$c$sPvoGd_qJz(vCHaiFSC2V``|A)2k4rBindQMNAKkoUa&>GdNtk`d=b zB0L+t038~kikVYR27P})`Sg%bF!IiSntcq@V)y_1Nc!lV!j?Pm9L*xl5|Q?kr=a|4 zmSQMv#5q*_KGRlEZsKo+y;M0b4yqdsj?c20sFQ-E2*a~vm)1WshSCMl%U_OCX>pN* z$Bvj$+dAAEA5y$G{CNZ1%om-uzzr6{(r8f7fRb*>G)fUdpt&*B0whxzeL8k`djblI&57x zIvMvx6x}>Wb+bcb<)#04r7a`3wiR`LK7C?A0*;3XZHPagAMxcx=?I~DP>U)TXC=8s zBHDurlUzyHWN%`&yY zA)#$M7>=4&#j^jjP0_J7ruxbI@C)9`If9*aj8%#TTiOJpkzKkiKUo5by5A4bwYug@ z+y%amuKRy(-};YyD0pRToAQjG>{E34cxfkVyvC<~`~h}GxO>I4m{q)nZ1 z+fb2#Ghuj>M_{@S*&Tdqy!Z~%@tVdhy}(rkmnBiwr&E)us7ljMRLS8|5`U)lh6zA?8jZXkl9X3Z-jR>>XYisfq3#09Rovx1Zq5an&I1?{5AY5a@>vF)4 z@!Db-#BBn=hRb?H{K>FX#e@Mf$^L(ZPArC{$fUE9{?b%71g&;H>4 zxnr8>9K_Tman@{t8{8w=$ zT9<)uP%@WMJ*%->FL@nG6sluT*y6((UV*ZTofq56N&>dVGDF<$*T=sA!zzDTq9|2% zzKBY&-)CjqHUxj8@KJ|X`^(j?wN4GNkFK?sT&HSC$Xf-lT_CS@GdM!n;!<#yv?@|Y z20fBp76ajWbvhbSLd_@!9OXXUyMx-P&STOzYhHs&-55LE_*QzKFrovsyIsp{1GVwm zN+9*7@FGLstNdhkn$|DLVr(5FWS6}M6|VOERvsmcWf=fyxiR{yx&Bk)ims9e9cQnR z6N>sKZnJEYTb6`=bEfydFB1kHhWzTnS4S4E|MPFNN_# z{3(?E^wCmtUO_X;&FeH=q=nJ>snOtBm+bi*RWFtMFiPDa`^Au|Vnr!(VIJ$+ueR~;ukpB88gdUkUUAcL55O~(y z+nRj;A7mR?Ar*rSE5&$KpO)ezLokXPS6Gsd90T!D0t>NhC_*QT4%9;CQ4;JMVAjX6 z%U1PfDswqm3i!%vEa}nf8=4kC=0`zkC@I_j_-mNfyVI>9v(ws*I#;kM|K|(|!2W7m zl%repo|67=AF%B=+k5!fC$Nvo-UgtnR$7=ucUB~AFKAZI`4(x@lnPn{jFG@q1 z-HZt1jrs(tJ5@oC_-CoT{I6SLLjbo2Cg|OrUmO##ODKw z(uo1PI=N-|`@w|%7N-EI7x&wiXE5?}nZj*h+zTaDcd)^5pl+;+Cha&3?fmeL7StTV z_(fVy&)4I|2*kJZ4qske-Q)3=d)1*&5oPl?-61$D&hBLUkrHgg%fuB9f^!L%GSEA+>sp(YQgg(mK|MXAw8!4)Hpd==- z=(|^$wB>2_3d^F&cgT1-5sce6Phq=&mevsWNnnr$yLqz0C>^wxZ3Uh|6nXRuQ0ex& zBy_GyKfqY@JIyByZ~#t-0lmbEv%9s6mP?kzQl=LrywG&W{^+}mqZ<%(QGCp`Uo zk^8r_*<{P_K@iC;-GR3vvGx+{_iV4_&k(i^hO6{3f0iq7-ZY`@a!>b=pO81w7D-FU z=(}Nzpj~t2I?X-F+9E47Xo}F%XS2lk0Sd!8;!?-!^_g0}Mpx+*6!5twFgNFFH3ooJ zihHQ)pELzeTUJ!yMlEIgdGDqr+_%1)iaBEPTh}rW#d5@$+TK}kRLzS+E&{%}4s`(V z{)wtB!CP7;pK{oO`A#9<m4H6Dvw8&Fa$aDk0uT06$}$3hQz)+gyNpv-~~ zwU!@v5M;pDl){SEL5E-R{ZSThB^%#)LKXYA6W0fCQ>4&6UiZ<=O5x254A6Srk?#6) z50!}nPY{FmJ?51ipGa1Hgx1CdlPKVoofnC;cxm_h1swf*+$Vu>TGCkS3Snyso$9dV z@i_ktX!OvLgZz;M*IRzv`#cy$cmg!%u3?`?N@dg-yG!a`lJzYvpEf@3RD*bDTMvj> zHin|28H)JuLa^P7R4y6Eg>I@^PZ!`5LRe2l5eBC_ack3dwH*VFo(3{C`m6z@9d}*< zgMZ4-Kmg>9@Y?tsg;yE)QFgahzLO2Kz$p8>>_%%_f4{Je$WCA!%Uz@CB2W}P?O+@a zA8=CpN|RHxcT2WPs#UL*I6UYorgl6TO^Rz59Pr-^ed@zwNHs$emdH#f6kXndl%M}3 zI=%~qs-qU+#CdA(}cHIMFCbiyaK&^7jl!acly|AGl{ac*jxh) z2hIk)u0ai^u!%)@#CsOWjXv|wqZGDXh%7+#dpr!f3-x?9XjtcR2ymJQ_Bvm5*?6~2 zGSO7%b}%(P-(jhWp(We1(jO^V??)tO`ThP(m9vI5<@eHD^&i0k{m(N{!o(RXm!qsL z)*2%7@OI$>o0l3zq%T~Z@Id2{9Y_#LpUPL0f=A(PLg|}Y_DVv}3phnB=}=9_!7 zxyG`RXA`9H~XBuNhY)0dbt;&M|12 zOrj*pH4CPobg5Nr4c1Y$Mcu@iuMdvjo4iuix~1S*yUjOQ0^v2$Q257V+|$|_9H=+Z z!VkV`YrGKQD7C^`9NpHNy^KJ!gg(|R1F4XF(lFMpOSj?IjOmi-6(PO8PPxH28 zv7t{O%(4Clek+;Z=xue!PaE3Fh zspUuUqgh7Dll6XgrLUmvPy6%RtG)SG{VP1n3}QrzthP?b$JnoW#xBoHNKX#e#8~lf zPMXqo8n;zZo7UdeTL!nE^pgD*A>BUkAn&bO;i$98pIL6vo(R#nIWnPq?J)ZZw2-?v zRt6nZoez4DLcpL|+Ncey)!O=~KmOf&fi5h<8D>648NMaDLMkQm4v&384zQt`-`;Mlo+w^41BSLbu3u0Yp;X(1LUzL} zdo&aV#Ezw6F6XW?`H8G4>7Da6>GN%BwYam2X*(lFBh!-6KC7Adw=uPOUoF&YQScyy zeV6vkM2`BDqg|byaA*g|&RP!wYE%;Y(^AI~0|noUh6%z-Ga&)*G-EISylr_KUmMNu zY!vz`@>PD>_tx{ZXk3k#%h-f&G@acCs3uD<7XGO`l(L^loK-k6sP^mb=3j~Ozou9o zYBFTos==uAr!lQ4qZZIhYWnpfgpfP1mSO;GgzSs!vmc=Q+Y2|#&ebWvH?cd=v)s@leq!~ruPtl)$lLWuUz~q`*kW%o9iRu8GYMswH zcB1S}F;yQ&QOz^{k{7Qv5&7Jrum6&KjyS`6bSTb%_z|FLO`xE=v<+t#@@~PEJlycQ|TD=+a(<2ToGnwfiD<+ zw2+%%K-`2u6}La=Qd;NcY8N){c8yUkIW)|N2UU31-X&RXev&bp40T+s zto5z`983HRPIR_u3*zo2hCblMw4mw0GU{Tzh7kduZ!h;KdD3@JUQN z%X~}DzrK`NlUfgS`6_L8szA%?%M;bMo>&AD&w*W8RIGTmucTTxL9@B&g+T6wD(wDv zj|e}NSUm_XBE!rB2d~n!vT^Z5GNiW<4=B_3RsvsLwB&n&uNY3!`pLG^Q+r!1MCH}xzk=KMWn6XDVLZsyo%NC!MOwSu>IV{_CF?YplLyorf zH5I)V0@FJth2k_N&$6yG4 zG%v{{O@1Zk10Kz|k9zM@GEFpmRDA_%g?=iG39XYP7ke*RRT1Y&V+%Bvk5rJgxYHS(2h;b~`G9 zP^9&aOrc-TxWWp9T!ieUZ~t3RBJzfNMqnW|^2${A{yO5+XrL!sPSaooI9vvc{xPOW z49x!Hc{PJl4Ag%cAsDQDr|!fGtqlZamiH%qQx5h9B9E=c9b5~0AjnP6R&WxxAv$y) zoI)ckre8v(*c90ekK0CR4HC@ETHpHe2&a(u``a@K+VmpUEImli9<+5&g4`}jUsyuh z4kD9cbx~$5SOmYTpkA;1Xm$Xjo zje8lzRVMB5y|pvArI)orGtGArj}^{uXGG1jp_BRLZGso7@9OqNZTYZ}e5YbU`J~Ik zrb6M#G9!;_#!<~b@&a)7*nSZBTq~Cz;?aF}vYeK$&15_97|mf%B+Mzv0^VbwlxZ6y zG?TWeyu{3XYj%tYt!_C_fU#9;zq$GFTVQP%ulyMli>l_i4u)_5C0@_zQjIVk)S%Vk zj-VpiZuImCRm>A9K}RRNPW=A+#vG&Vnfp4oQ=gBbAfxlt4W5M<*B&s{%}7hLNy1?6 z9ewh&2Q^Z&d&_3H4Q#zh3l6kw+S-~@%0$8H!N436%-%X1PC4+ z>IxM^AG%D_pOJ)rphLSHA0F(Yu4H2zVN=uxIhBU*J=6{igLfjZ;O}Igr3)iicWSt> z46=!w%<<$=%7aNg5ee4B8t~>tit6@}_>j4=zSMlE1WM5qgy^hfF3IFkZ*+u@PvI5( z^0J}@KY>7&GscXsm-+1KV=MX`P3_}*B4>;cG=oL-qrGDfKAVXnzAr4}VfA5> z#vKKbzO!V@j4!diBZD{7G#6j`&ASIwld(ggJ8>vt{63|75B{tJp7oF7!1~e`pJBG1 zO3#t)<;iU`IS=*fX|&&J&1X#;oDAEAcd$*u?)1{K26)eKf2g!kPq}S5%_O$i-9=SO zdX-)e4y7HozNqP?=pClM^`AwaS~{9z^eKS3fkrhJTO~+{@&#HW%;b|5(c;4 zdQDa4Fbl}pUcNW~&-8=Vg%(B*A(y+G$!yv;;)bR`zk-49W>d)wlc>S`76JnHUeO0K z{C&wFkCnZHa_c>Zs^a-5HH|)F0CfdnRd;JAjL@U>jg`PBdV@Mk)ZR5`ShG&`vCLov%Cx;gmrtT(gJ8+Dtz`b#W3R;ieVN8#G$O4e~ zi$R**t}TnuAI-Tlp0)nzLlV4~(t;R%v8vf#ziRr!AwRF%fLjLt43*q~yegxk<1Zcr zUcYEq4}1Z#z^cd>#i$mO*4rz8+m8%}Ylbkst)qm#Pe(rj4S06=E+S8kD&(w@r+P#p z*0&GQ2Unotw!$A{?@N$OiDHw&A#SEe-_S(7!vBXOmWB8sFxXCAaZ|YV`RR7iM3DO&QcTi2epBRv+Wbh++%jpJjNtZPcf(U_&Z!e z6&=y6+DwiCDe4%|wRIiICqFYUW&rC z2QH+Hm5kv}i6Z&(c220d>MVoC{*;of4e?q^8JhwcCRXNUH4g~8BH+M`Wfc6#_c(9* z6jNf9PyAk(WMA8~uAqqsp1huDV%U~y^tq#?=J{$7Esj%he%4`^pc?C2a0GzUY`jah z_$Uqn3vr9QA&u?*G_|73EUtMEdkvhj`52^6Sd(&jaa0R{pX#$~NXl2;T)Un%BiFpH z^A2i7z9XLwnilYDy08=v44UAlorT!INP3f zVfHl^DO^{Dr?uWPr_R=w8*boVUbBWN_&3(-)^&~9vuqbPwuflGKP`rhCxnHYi0+K{oM%BCA5C z_oKe9$(jk`yc`6l;(1$15_Y3L(FMfLxUCOLk}I%)d!+B>p>9GmETC6$inr4)`;<$ z&ZCDc6%9j5oDZ}3rC+zk_F|u(KD=SvRb4s#5NJ0j2kU3D%@UHOC#ek`xCL0%3 zWIZg2)IROAIvo(jH$3%EK2Y@u&*vs<&mpC^ zt?=_i$xYwq-!TNvOVRh}QRLFAC*^pC%mr5Wp<_U2uJ>!!XW&x81*y_R#V&ywU8taq8<$U=zYAhR@Aztv)7$$+pRZcqRZ<*i>*`R3JeT5T`9ESdz&nU;b(8CdRra4=0P1ZT zh(P|Z0;wQm4~G{B%;qHUjQJ08Rmg^_`3sj%?jL!g_9{#76;#KhtHT;fs4lu1T98Fp zO@kECkw4nki;1NX9Oj;_LvV(3et4R%K=;(=UffGAPS?DD7OVh!=@+a%yz$diE!R>g zlw9#LOyf{TcFnod>v4uMOYWV`_vyJ6-U>1OSbUP-DT)^)E$lSGq)OvjSo;OxbxwOj5$r~Unk9%Cr0VFv4rhkf zGYqIczHHmwFX9IYy|{U6_|bW)1X~AZ#GjsIkTJiF%RB{uRk&iuSnr_V9glM@52$yB z^_|16$YG$u2aml3-bE_eC45d3x2dp?;^BgEoOD9*l2Ggh3w?-#c-zE||Fvy+2K zQ?f9$R-Jh4&@Bq2n$kP(9f!BioTfj=+GLC3n3?8N-h3t@6AOU&8_+;3b*2eDI++@O z)|x58FO+V?>rc?`*nX&H-BO#tcbmCO{hs9lqyFb^ z0zu3AZYv;h%uqc1S*2GK7*R@ zNJnbvi3KA*`56X4+Ey^z_$R=wgsCxu}c$u0tG>I{Ej7pJ??jXp(2{V zT3BC+IHBJ`4aKqq1D+wTmB(UN7%r}f8e&e)aJXeVGFLls1+men6+(;Ko7~CD&zlwM zlzM&%%0HXkv77jBq#F-tgn{>kY51f~;)k+2b4m@kzu;~>`tSG^v2kcX zI{|4s^F9`+i}+vhJu?&F&nhh@UN{qWy3>9mQdLXMR7mN>(UaV!y42(Q8Y>o~Uh7&F zowk@RI4Ambe(q$B`vA%^XEck*Y5iqP%r9$(LW+I;uND~Bg0ZEpUmx4}%E1PhDteni z>!0E(;DZ-&{O!9A2)1sDqcuF0Ug`mu3-l;W8N&nZd=FhlzLXRp8Qb>7OI|eoPG7wM z>x5=1e=i!oEaJRtk@56pY~kEvOs&H(9>fa36{eIs#FssYhWC;Q7^VpNdJw@+E91H{ z9(hpz2A*PF=k)-0qUZ!?l8bOIGeQWYCC}l3nXpj@O|525;WgsATZ!bgdW#6cXVu46 z2I>sKUh5?UKHFNiCcl8S0jKDDCJnK@=RPdpGK*)Q3P@?u@Kh ziDt4p4)NUTmRuIAAxaF8! zXU25m2Zc3reqjuMGloBp-vr< zd*@#2>D$t+I@h=^tt^4wrlufnp^98nM4n`a*WEd9@N$F}NLnR-d^h+U%GXxe(9k7Z z*c<7Z+D1y^9(yt>x*93*yVLv6s8?IkQb{2T^kVxH)3aES-g}qtAO6UeyqMwTuLfD4 z?p6I_DX)u1Aa~xh>&nd{b53OGh=PepOE{)rv~*Hc%&0T;Y;g}Vz>ERXh^-S|(Rdl( z_+?(MrpH;?64F|kuqhRf7LwxFlm0JpjSMVtRV&0Qlh90m0OKFs0skCngkD%vNn`Dob9T#Bp#yV)t(7t0{yl7Xij!2jj~>9FV0w zd>u&T%O^j?+tdcJbkr?3I_GSm8E^9xZP^whN~ zpP*K&`x@4A48O)R-$jQvLddzsRNPMAv|L`V2;HPDM+!Ywo9OfU2#_S^LM1UpY895a zJGOMtmlaus@C6VAQU`kMpLiE^Ty-c`KUk4am-}bNJH2emllq!GRgcTY;jAmRb)#!| zom&40NPn4Zs?QP22(!Wp7MHo=DtRjyL2P+Nc-MK)f)N`KVPj9c=wz{m~=)-;$6BNOT8--24e&?nsc-^dXlROb()i)UR3 z6BQ6JH(-}=YVPV`M@lA&(1n)IQ7mXf;h3-aHNo zE0&6RC9cmW#}>6^_-pwBWMeHE(%4nEo<>=wx#KC~BI0qu_J|Sr8@Ico$#w?>1%Q@; zrJuFtIpSm&<<8;;L^)KQX=_82V1Wv?axj5J>#egrENDua^m;6H*u{Ns?k_E-JU#yut zTb58youbpv+IrXRnUhg%AZg*na1aS@h;Xym^S;qN11YpN=9vU!_|3E_a%3l|x9w}y zPPQ)srA5aQWSfE!6G|1*{Lhum5-6<{7$#xhb~=04>9-0yOjTF_s{0Wb&wGa99qd`2mz?{ zO3pxHEY%TP4UH(4;Kwxp&`i92FKY1!CCSKo<`E${Dcj!b6CV+@^%GX1{jEKe&_Q{I zOHG25w+VN2sJL}yA`63< zmLBsMH@B;ywnsE&V_1Bdx05+yYKw1x;bgDF1Ni40u7e^6j8(*N1`d3~O-pn*41YP71TS zXxO#Nf30?4kS5N8Uie_VUX@0{1PMRI7~dRkR1$mi=^J^*GI=)kr7{nYNrmj%-MDA| zitCNHB`!<|FCvs>b+K*X=U)J#hI;0XNNyaTnP+m0pOBL(pYY3$0=(sT|J=b}OVKl8 z+Z{>4feV?ZEeH8&CGr5UwNxc~-TOVDR@;X(F{^6_*;D9N4P;fddIWqZ#!O6lcNqly zt58YztUuaLaA8l5G3f(UuSidjDn*@nJ|Qo?=`TY31-DPMkJ1ppdC&*sJd@FnY#b%m zIL3PwNOqQ}c@Ocl`>z{E5Wnvu%zcYX#r2dXe#by9eO&Q>u=kfyQMPaVF0Oz`N=i3U z0+NE#jnaseNVkB1N(loZCEeYf(j^St1JaFvfOHHD-T!Nz@7})`|2O(-zu0T9#ahpL zT(f3wt~u|UE6(F{e0~aXnoSES$sKGxlNPQny|SY?DKE>uGWo;ehLbB8R3_;g>Bu^w z25V1qQ6-1p$JCw@U2t2$bioXsyQSm>|e-?kW^4drm86aQu#7;NN|em zA@#qgE69XF9d)On{fo)qn?wk>3ZBQ{!TtANklPv5QK%>;@jp~PzBd`T3X|8L{zW`u zv;*ph!kzG66bW=FJm4z23O4>l1JOE-M7T~Ksr^IcOQ4y6s|Zpb`6r`E1rey`otGK^ zQ2AdlI>9~lFgwfi??X(7VhXGdlhe@>e}A}u`{Ek`cr44w_Wb?6mO$%P0VPzy;{7id z@&8xIN}}FK8NduOrXC4eYTQ2g?A?ZOwWZwxLnd_Z5QMwRfD^4I+wx>5CwsI8PoG=R z|6bX+c)i!uW$B2uk$!t+=^XC<%hd7nYM|Rx>CXB4|KJg#LzSQxf+&ys8xE3>8XI%q z0|QFkf-U2(6qPS*p2Qt&Cy9X;WiP&)nC7ua$m?4bc_Fqq2JS_SE3mQ|^aRl?7TN}G z=FVfvg699l-~)i-)90T^WON2eD(u{xws_To^L=M5o6+}s^AX3OrI_)uTd+A&9}p)q zAiF>58oXwR2KWUmL8!fFEiyxz(zd)Uhlfnp1G$yGgFFR`DMGU46l|9NoyA?zgRM=- z2|!(~%nl}emeJJIpx`sviT|*Yzv!2w$d{*ud~ybTQ)c`jAVz-97VMmdh+74PBj&vj z(diruM&Kl+FYUO*a-ta*w+HpXvy6=aBoQ%WaJ{8HwUCW1(@I2#2ih#%X8KstQ?8MX zB1p8DI3EzT9XmYXZUAf-1x=Yxn)SWXPXeTW0~=)D1HcyS0BS|K7IZ)lEvw9&1H6FO z{nrXjyy#ITUaC`R_%!q=my2_j&ev(HFrQ>-lku&ey6SIaytOk(UNh%Ud+Gj%P1Q}| z1SIn{7aRh7-w!6y488-)MSw_DfnZzKOF*hbeFstJGf1eOAtWC9hv{WUbxp-%)FIVU z&~rG0T++DA^bU=Aih&b`U>BrbGbrr-uI22{7~^orj;YNG8eA153eF&N@MFA?bbHXr zDU#oCoANI78|k=*lnP+`JX3fPvRvk|${wBRZ0-f}pe?~C=2s#AWd^L3wM8tE)&}sA z#kK%r=2c+s3qR?kbIY>RwXi=*g?q4EA^{Y2nkytjgtRCwyXSFa=!a&9c?r);Q6Fw=^ zUN9nC!Ycu0bts!U$fpG~!hqZt=tuI{JD~aucPfM?qwbimGR41*()T=H^?N`l`t|tI z$Y6cA;G&MnbzVRV#b;dGG_-)mK=IBsB6;y@4QD>EFPrALAWPM#WzKdJPRI<|N5WEe zUlXR?KR-1yexCCoZzMGe2pt67cWn%lg+$B3X+I9|_+_QVo86KRxytQcCh=M6AJ#PTWCS3-5k4i%_OKeSLZR`Pz?Q({ej1$ZPu z_n$Tf8r;*MjTgIP7kW2~lw9a;%z7DE)+pXH+VVy=^E)HmvdFnHz&zlAcT&QE`)hVs z%fM|lt7FjU>ux!U%>De-*DT80ISmdMBDWwLvVyzRxL&bOZ}DQ%7HAw^{ztBWAqM1%Hgx&n z40sB7vb99}?g@Ek6ZnrnPG<5&UI-6SeS-Piwv5G!+DSv7*n>_o?u}(Zx44t#5B;2^ zqGdiEEZkOU8?WMc8Z43ENk~DVV8^s~_O}Z^;I5^nHNSWTZb3;*a-xf?4a~H&PL^(! zZ5?bBa64K$4@=(vd6D-@sIYWXn${JdRqRv>ChANOb=Ktqx=pz|_vvOskboU8-30_B zcf6erNASw2KcM#C5Hz8cD^UwWY+H4`UGcwY5?ytD6JSWpo(0`VvA#`P+f z7aLTmgvkGtm8RY`Qk@YBXpu=@i}MD5Kj8{1a$5K*ADsA9i}3Gqpy;B_$?T!qhh z>E|FTUEdid2yT<300$B2cElAxoX4>;WY$bk=_)R?1b_tFgCf_~3ARGePVN$yhT@9I z{uTF4f(Rw#6OsWjAXE8~roy0LhoA}-h0c-ZnmY}10QoC?S6ilezdMnOheuR%K`5wVQzTnt4v3v62W7#=_;d1rZJG;a$Hk^>~8(( zf_uJ7_|+Nmu5Tyro*g1PC+oWct?wCD;-=qfDuW|oYvTStym%omzVyr%Ng9DJz;bd;{MnD({R{A4LS*^&gA8*PT#ca9*1&!r1 zv+0n#8>qwa+*S7(&!9a=jAyp!KL0ScTG4j)2XBgu`K%Kj&M zA@vq9BvQ%o0t9+5-BC7rfXd~<8F`}zkAyrEeC3~1;fiF}V1o_~1`{p2Guwj)_+-`1 z6sjD@K#4!WF24PADx};tpoF(7lgX~0!!$;Fer9E|> zl%b{vmZ^Fvo_}C|Q5vNQ1s=!04y&L7w5wWf-&MlGT9J+KdBLma9-?ux7=IK36In!- zFst@{#=6yY*n7HgS1K>YaEDX2H%Tam7eC>JsKVM=Qu>NV1NAcVP3z1AJi+jPigum*imn!!G_ z9`?LyaD8k)=Gav1dR=MnjBnec^lg&c8eh8Z!tMX;3!XZV6!w6%l;l6Cz(o+^E@twM z`AQ2>_|4MD1TL>j0{cH21eK1_*+t*Tp)~GosOK+PI+rZAL;8{e)e!xn!3~(0pjNDP ze7FUao%fu^jb5-be*I8oujKH8#{*O+A#M@E&_R%X??cc_{aA6qLmNB_^v;D{a1w}grjGhTEzHlV1KZ)J z$So|lw3D(NQSVp$F5VtWJ`u8$sj*B3b zdw0hOG%SwU&PQcrLDRiS^GkdG5)W7u3SsJQ3=~kKPuQcoY8bCU{`_~dR?pak<*)u~ zR%F$eYG`W<%zOae;>v>QH9o1yjUH(F;o^ZHFLa-yS{GG=y#-yXeCOljp+3Lg?DaJf z$P=p4^2A7-_j*U!4@CGO3RRi`2N!Q`pE@oit+nUGBb}Yl{YK2Ad5-|-COWLvF1E|K zM&qs}bGa7sT+Q^DSZP9EHjBw1Mr%blBq(b3AEl^cERrs=xQQL(_AIx#D2sf=yPn?6 zq-JbB-^fe+W#ecyv?2W4<06(9uBPQrO@1pfY6Lt-oMrYVUzj=1ZVn6OJk(aj6xc`O zKhfkU*+dJ@#-*E^$wOB{UqkH`Jd$3ox{Z?LlWl!XsNj9*IpN4oRFp>9*aBP(Jg^4A zlLru*%Vp9Q9Z&6Zey+X@>x?m-A<Re0b0uaI=bY@`zx3Y&R%&H9PPEb3|PwkE{w z#y4&J;YFU2Y}J2$X{38PX-~VLXk7f-5Z=)lh?sZ^L5%2_pH~MdN$mK-P({{hl5=m!qvJ*jVNag9Hb>x^#b30vp5q^#9Z@bSu zZdRl6HAg|*ie2WX)Q{4hzh${Tt2>H@atQTTT$0}-(dd4YQY<&ALxp&%z9y39Zz@*Z z?BUWF_iPorVd%%ggNzbHHfMc!J@>o&e(tQu>3QFyhwq#5t=vLsH@gF@4nVy4a&Jf= zl_09(K!j|{7DYBI9f!vpI6VM=5GGSj*U~(MauSQ|S<fzWqb(nX$%lgIh>eOr0V;wRviZFj!8A{=&|=U;-&1ug$W zcFC+ht~*O+S^MYm_49Pqd@bLXGgQ$Y*U7r8!b=C=BUwz$Bm72y>ETqssZ7*mv~KMf z3$CN9NqXynm21-Z5GJ}+l<-G`Gv*xao6u^A%^ur8OC#ga`X9uGbVsXz8gGEz}{m4Sok9*XXTCE;CLlXj~`rX z54*;tNi1v?6Ec^GSF7}sc?sR%lOD!r2q&vQvrV49U#NpYivT0minaooq|wr_Tg9~w z6&u-(hpi%`um!(qXIyAV$b3MX3G&FR-Cv*8&NaZ;w}mI+&0CW+wYTYIW!eT+VA|v> zIZ(Z{G)B0R-bG)F8+%V5fggD~_Np`O{-wib9}nC_avZvf?On1(+iA73;*@WAi^9gf z-A@#eN%FmE9kmLX3JK8+3#?tj{12b;20iBR`u^58mJJdb4ALnhYn;V?El(3DBtu7P zuW-^u{{vW>>0Lm8z);XTqR9pNw)Zz+CHHF0TDUuo;M2K~q8t{9q|g<8i7 zkBPj!tlQ}%JijDCR&(A3h&arw}BHN9Y z(5zo8m>%_26~QW*PcIR4Pu~>-H;09V1>sn{0EW5HwdhMhl%oTvWT3P6L_E*Dbaa01 zXuwIXU+UH&p_o&>K8y4xqU8e{2M@lmXX8Dc+xP|YO$9f{{MF*PX#VPk${p@QQ_n8E z*|p-%zr!`M*)g3oZr+^y5KC!e&;7rc6terC`JW~r`70oCUI(wK?snP0I$Nhg#?r1r ztyc%&G9A98I&V;exXIeQm#N|E;un5lf>WP)%#%cu-G7A3QOi+i$T+5{vl;B93HWnp zn6J}mO^Pynw1W`+e6gf z=1m_lR;tR#D({b>yUU_^&&zaBkr90Y!0cRefGh*_wc;aSFS;O zFzZ_nJ2BU>{i@C5guDG|iXzbJ)!mL4yOgw8iwXH((xZY-k4EF<6XKv;Lv8zQw2auH z_YT@4%to2;+=jTCC5d;Se2h3(LSL7JjCyFjJjiJuQz58eQyvxPx*AJ<)Y-3B&EV~m z%}7zqGkli3z2)okWOQS+%r4pvS#8Yz7tPOMG_DCB@8F^>Mmz%&xFELWjf6_E9P#y7 znBAd&E-ief`f3N3S$2uUUJB3+nP1W9T(m#TMNMHNEoj-HS`^d}{```>@=4RtC_*eG zJ%@df)jUJ{0gYoPfmnSOPVG7_MLf45?yJ+mpibW#qYHgg>C~4^L=-*GTa4PielR=C zD3hXaGxf6awID_vqT`AXx02CInSMo6C1+57GMRBe)_q*v_N}ff{lm|^!5o_M6V!EG zJ{IDSVGJRUE7a?KiHd`Z_w-YSB3+C~C?h9go6;Gc@T+}ZYYqt^-@C4vr4{oo5>Y=r zYN#yP@|-1jh8;zFo;?_Kq-ykbvp-o!U+zLHbaw0ZJ83w6BSmG=sO3nO>}nl3g|djg zIN`UG41goRUQvz=o=VXU;9PFBUVR$hcsE-NAiH>75q^%o#=oxIvavDE zvfo^M_O{09p)69{@s(nk;C-AphJ9!;`yA(C=5IH%^^WdH?L5C=Z=wfSdX0idwiPQr z;0v*$jN!@LFJ*rq+ts;+=C#S~u_DLQ;(27wLS|dOe2MKb0AZq@gI;%RyMRHO@t-l+Irar~JA~lD zmEi=hA1m)Wy#8j|(i{v)2IdoIwFCctKWx&$AM?UkIuJ^+_{AJQIN1QEmC}7Ob0SL2 zy~QPfaHeV6>%88*%-ky@Y$^CbzE)g1_2ZUNE%(s|Y{835Q{%(uvY+LK8J{RbkrhOm z+)z2N7`_zbOml5rR*>;FLuCJf9SDYTVdO44!Zi&fT$sJIh1?22wOZx~HcWbp6w{I0 zj^^n(?|eCV>YTB{@}f@AZa|REU^!ZVJ-gp!h@2%!nA+=Gf0-k93d~Q1WqAjXb(@s$ zD&M|G%09QeKukwTdq$wyNprF38|y({=*^deYZzWi^O$8qvDpbcO?xGlo+Kt#91H)J z%7z7Nmj{IE9E#BM<#$5kq`T7C#?I7CQfY~U;@)Qx4u#>p>uoQd-jjXa8!#P^1J!6G zqlX?at{vMRJiyg;s0s^6_Z%oarI7?PZ~f5-Ka?q*E@CI&bvNFgT1>{LI|~Wz@!LlB zevjrPA9XdpBoiB?Pe-QZm_s`Vp6a`vRRi)QxQdp1b82-nD&XpkxaM}BE>+U|CYGjK zXN}lsyt+Von>YRDazAyNRhv-FcfnR5(KYs1L21a*Ta8OL)qVlD4liLyTzX;sDFg~~ z8)sV#XAX`Od1z{$zM*1v+fKFL zaqn~p(F&Y)IgXF&2KO-kla7Y4 zhT(PH6)QbxprZgonXkz%-a>@}bV=}2R$+s^qqz2e^#J1<{h`3!{KlZ*U4XV<=ErM- z0vM5|s`ab5^GnFQ>E=+J7#1*iEa8)BUR+GC{Ia-q89f`|8#>CbStRnV1$OAomCH+> z`wF*MlTT#I7_C*md@`PO9@cCCgd+C5hy%w82yt2ntiAP19T8&&nX~oKCD@s=iYJ+xK8VT6>!Z+J`c{p&d#+;D zyPR^0j@9(8<15oshW?)5O(Ik4ejaymc-LxvXAs*t#|@dpj}}jPHb~XlM z^%aK9e=d-{v2B*qwO~yLglK@H~UrWsoL-(2@U-tNSGH`je4d(`!EdU zGAgE;j76T=*nt%!hha^jJLiLF=&^^9f+!ImvD?YgQBmQksgQ9$`ycxB?i(7z``sb4 zG0C-Z#7j?~np{}o*racwLdaD;+ctepNW#b_nIA2L%;NUj9EM@%XFXVP2m7G09NRIr zOHILt_Arj=o)nXY%F{79|HE1LwQBEOVXMZ{U_8~!d5qVqfv@uf?`iHS$Ne zaQacO{C8yWdY&tkrij~@-dE5PCYvU6^xkqWGS;Hu2$oHDvB(h4d zcZC&vBPc{DuKey9YIF^!lzt{WT0>6Yzxwls&-K7F$tq0KGdglA!aq7s@+ZI znsIeIZQgXP7f-(~hE4(Lsg5hxN}H{hR-W&oVl;0QHyf&2O7fC;oC*~vlD+?JGKkp! zTE^97lcTak+DI4{79?a76A9DrL)+NRdl^!YG7{WeegzXl`wx)+bO~DW{!Uy&>7NcX zC|aW$9x*MTl4L_-jp%CaW9@7k^c&L&l$BbvVzsZ{7-V*Y`tfaXuKpmGHd>#3im8K3i*ap}MK zFqefRINC!{7H+I?mR`nC`27x*=lGmNXy1#lAw*J1!9wwoWid5koSdSGC&M}a^~wkL z@KaPmHSSy$b%tj>y4GlC)iNAfxi=XwU8g8)LfBciT}1pQ=CZp(Sp49@g2bx4NC13& zE*`{Ss}Pmj^S%D+D6f6ee5LT*>3~W+No6}((hT}4mZd!bH=Dy$h|@;H6eIpf5o|y! z-cHTZRntN!TNdkHS>cn7M?8ITF|1WrU9uH@c4H5lc~z*X+*%iCGFX1I2Yl-lN~9J& z-0IghkE&BGkVj8rS80c$dy_q2oxCyUC-kXQzNOWcDi1H`hP*T4-k9Xk|GxGX0~}q~ ziC1ig+EaI`kfcjHcZLA&nCeqmpMbEyr(e|W_@h_~FLo!_2lM_+qYNfJIQnQY*%eyI ziwt)**`mUZ|9+ZRcdz>S;P|`D59lkg^T|M;w=be7z7<+1rj$e`UN0cP5WKQr8fP6$ zW^CkmG&YxuqkEI;0mW!kfvOYxI4Ye9d|&Bz*~sOzrU>$|!_4D=}L69Ad`zAMNkzB}t^-$dWPKF553u$j(n zmTosj3tqk3Y#l8m9jD(DHzE>~>1h^no7d2t`?9$*tQS;(k(Cp+wQ(2~70}GJX32cR z+_dVpXfICA3gE84&sY^(T;cHC!dl(@N_Ad1WCgjROZ3{#6F$nGDG*e;iXp79ez+@y z;1`ibZBCsMP-$)c3E(~UOW9zO*iG7x8x>?uY>J#Xc9)E5u`E{_voYqA>7|rUtp`7P zjQLEKiG|IkD;gziZl|wZO0pKVH zy&wd6K{~&~Jz52%3bEyyy_&(%pS#4=nD|zY@QI~DM!Cb3B8iAJWU7}eu~Gyyu#=)5 zJC&V2E(HU8idf*bcw{<@vl8(RNM>GjO|8!Rg4j3hr*uwoyiqlL<~Ll zYO0$%-)_L^vZVqeQQ3=A4eX24lkL12h^Ag1XN5~U>%(oF%<4KDD1E=Ui`s&wBi0a4 zds|vKqDLne05=B|G4j#XGHpf&2mU2b+*m!<_v?_9%>t#CMi7ViYKFGXxZCu{@0uaT zT>^lu+3!j)+w2w8W(mQm+=euKG>qmQg&DzQLKHIp$J4+=|Kn+B*~Lw#Lbr8DSJ@Rr z6j`QoI%6(#0^%icbe(C<>jk$o(CPJ8Mj!oO%%v7dq=jZTahXSGD2FkH=!&e)5^;Z= z*3#seCb_jkLb-@6wJ7E?uDes(%vwyV@b5FT<$P{tYLn%DESTrZk0^0zVWB?7=S=(DD&0hpz+F zM)onCd%4k8Zn5r=6!0K%^#tOu=i)-GAzqp79Ok{to_u?)&ueKg(0PQ)a&psazkFJU zaN0*0{NW7gVV(SGWxS5Ln1gMPoxM3Jv=BG+ZOCmsH6Gf;8P05-2zJ5xq*;d!Tr)H% zwo_c;c5-fX3w-wA6!W7Fw^aXVD5tgEa=@7ZINW!x|8-#vZ4dg1kg>YMYM&NGjVRN+ z**C@-i%9{=mQ0EZb2ty3A)ty7+(S6qLhN3gO&+3C%=Xpb>7vhUJl7Q=d&MdGn_N$O zjH`tDaUuX4U@x21^B~8IQ3sz-8yEH02n?s30)yx0^7d+6l`kfR@2E|>I+M5n>6k4Z zh{aP~(9zHo)Jdq#6|08tfmrry!;?YOt+1@C4piSx2`LO;e8QwQ34F;Qe4MYH=-ue( z_g)9x8~KujPS7SEK}IPjW9w@gg8L8+7Yj>A?Ji}(3-w_P6beO|FkAxcTgckJ!J_a? zlig`8{PO4XMHl$q;NYNGqYG@V+j&atN1Rt}b(c>>LGeU#g!hrYTi+=WR8{PN=2GnG zQ{ODrL!JrqyL+2YZ{ORF3Hb9B8`;wrxZxotUmRwBPmpIjPM64b`8peG-49j2v!LtN zM>&vV+6cXxoucu6FJ}1e>quG6rR=r2`%|qwElk%R(hUnavFdB*C2wkqPHYcxp6YVV zr9l`wD2s!|4l!NZI1hPniVg`)HKKc3sl!pz$8SGVT!@9BB^_&bxTH zfBnvg?|U@uQqIjmq2tJf)9T?FBe!oZPErSjo=}Zq+LIe6krGy{J@`skf=m9)m5F5P zB#5KMA9pDAF4el@VoHBFY-^h{{_5s%NL;)dE_2`j6d_`^xV5S44|TbP5WS2x=7hf^7zp~Sj6STk;{zse`Ow=hUwp&i{UXc!Z>Jf)@RhgGPa zB|jBeggow;iyzx@Z+Jm>+*>74`$1?`T`sb#Mi>&H_Jq=2Vd^g#TjbE-%G=5D@{ewUzx|E{*0W?POg-|b=U%9rwb8H2c=-%ds_;eRY7c`)K3Dd8*6V|-YD-I-b6x%Z@!Z%`*zC#px6V>= zoOFZVVZ7;TZYAE`?mN;OZDdS2s()6fR#Xi`<3EoWHlz*fyUc3}$Mf#LUp^hGlEvKe zdw3N6-IxCOczewS?x@%jPKP5+!Y6YO=lV-8jUbvd?Vx!nS6NCIJ~bTkyX2b1y5*;c zEJ<#x`4HK}g8MA*nkux;_3Crkov_?)(%r?-!Gf=#G_!2=)-EX)u)771+(RC2mT8sf zsJk0J+mklebhLlG;w6{r)^c7sB&Z*q&|FlIZt4~3=#!>liI&7x2@|88eQ!g124NtQ z-2BReyOe25(_8V0W^sl(bfJp-i+}dDF%HZXPZ!6IX+`BiS$Hr(TzSh&gJG6N0A+IF z52#pYuIwMplUl26l~?unKB^&!NO4X+opb9%R4(~K6{)=~~Ri(=tE%qdxU zpu15Sf%%mpF79raTP--u^FNmxYg3vg<;{G%8^_nBl)!7QU?+SXrEJyy^&YmAK;cWg ziCQG35z&S7z_dH0a4sJ6vK39XFYFMTT$_F60?}hq|VMnc69_;zg<2P=c4x5 zZ==`X?UH6m_xPX#v|F5dBp2*vu!33Vh4_*KTx62*u9`(4H8MwS*7<@-1L~{zKj8kVJz@s->f#-NWb zIDl8uDMKWkH&eYAUDa`_M7K(cIc7gVIxhxGH=pR#oM@ceW$#zRN}}2$v$#vA%h6CJ zR@%z_dz*U^xWgS$Op22~*6!Y?!Y|Oa@S|A_4i#K@X6Dm0dv5MJg=M{@IJw~udRdC& z_^Qw`bfw?S4PK67hHIsn=W$>q4s#pu>b0;_edwjH>36w!V6N~Z0uxI*4_Xo%d&;E2 zd*3rVF6mnCO1R(N0pwSG%eNrPb^twglb+aGX%LlU0Zyex|E5^pH1c>FiA(pR%-2&3sfy4$xmIj z=QH-JuR}l_rvEXckVpC|58Ts^=QM@(RN}5n0kL=OlI{R&|y_d6L~H=Ubm}MNTMQseI#Kx z#j#qkLgkOJOmUR6+lkbEaD8_wLsjiv>X&PQbLugd{6ee~(8i_P8?$ab;xv~S;52QS z4%s@f&K5_bXd=?AHvb-G$(yT{AZTykMHlkB*!l2$)-^-Z<47CnvMHX4EyKLq?I{2k z1M`CR5qVmD7rBQ+0SE7yZEace`$FQ{4|-?~UF__t<-at$tioa)#pZUD)?N%u__xaZcMaq|LV(j|YTA-JT&{Lq!$N}SvWVfF0TFbt(z6b`+_2H%DbcSR9V#Iz;5Qf4S~oY2(Tc>^PNpF<`5smuz5P!FprHHUcebYEm$cJI7x#L|CG3DYEgJ3(6#Lb6V!x}ez;*#ESAwdaJYWV;5ny zs!NzRGQH-Pxb=rRYfooI-=Xc^L7mm=A?C+lu1)KiE!qpp3-(?Q;J~Qapb`ToYB}bs zipXAoh4a>v&_6sg6Rzp3bMZcuQ~3J*^!+}EXmr+P%}Jg*W#12-Pir+T9M3{(O7#Wp zu!SLlLe}EJdlm7Qkhd2{O?LS-)m109A)M*e0kU=lh95baNG6k9H*z85E#eo=at3&* z<6T<|8%^)VPeHWiAGkKY}jCgzL z)30{XVL4N4w;pPjKl2~#Xm>QQ%}pg*f?bMUU83#OC6Y436pFu!65JoL1;B@i2E*8 zv&;!|Sa4Vs|0gpSy&*%p-0UE|9$l11By{Lc>5;tkV@5n{ZGmKi#-Lr8Nad^DHzd*G zW|;<3dA~9EEC+F-nUxi)tQt{JT+NsKq-R0jVn=5n5j$&*BZ0`A5*^_gva79Qy26w) z{dbr}^tJ0A_XZ9C{*_pv5@wwD-q%X8vi{ge7mZbgj~)zKr>eM{?kwlwmFW;>Jt-p335$gaIKsO9$BwNGu0 z`m1?fJq{7G5{6Dk9b#{Jvx9cX0ZhW}wzaIVt%#6KxMzcPP6!N`?KNIYU`cie1mB=l zwk2J~KJ_Hz#LwBUwmkJDH`*mT`lM>iowD2XA%;DR@&acMJm9h>JBj46XlY~MZwZ<6v~fV(?4rT=_nD_wE#)yC z9ih(HWi!r%>*fk&U#55u&v(Js$-%)W*B}kWJejUUQpzLS;>ZRIgEN zCQAYhIzx8PG=MjT>;a0aCBZ{FJ|Ud3d{w+E%b^F9cGV*euRSqr0(+=;nIc1LXDpIy zyCGX%0sG#?Fa~ZI2Hg}x11T?kw~wodW+V(IosiETrKzpnQyo&uRym4o4Un4*cQg!_ z1YLw26S@?a6o(xMxpE>MfOfnyzCyIKjhj#U|AisIxpSAT68$%Q0TzSWJ#X{=I6saO zH4ZF1ev>am_7AbMIrwFJr5<;ivNn9XA8MWu^|!aP< z!pzjl<7AVB#w?LpuD{SAK2j1vC6aE0MFAMEW$5(igv_whyN_S$^dsA!vB8dy;xt%8 zb)-E1_&7NCjk=R&$X_msm@tB&7(evEzd%Pv(O-yluXD<5{sry&|K9lj4C(*dheAR* z@)%4~kE&=IYX4Gh$T4YUWhe(|^3x_gw#p=iSLpu$+;|sZzKGO(svi3oumgyTv?-Y;pHBCP2_Q)>JL{bnUh*yS!sq$o*KBF3UOm6B zzcGff?i^A~6vp?I0)Oc~@N+yv#zlIIdE}=$V;#*Qtr{KRM08q>Er9zth`kg8;CbsS zG8;%)%QTp3ntI*-S{D;cTkLlySDspiG;S3YziC+2PHgM4~iAOujENrhhvno4P|fBl_GS@x;ir7spZ={$oAu*^|1($1ZL5u{Uc4 z*iz!R|TaYNkhZT**HJXolG&hcmH<)z<)Wa+(K)dtB~dR43F#9`(hDZ+%v(y-IqH9o!s4SkjZ=<9zb+$@ShJ`&_=s0C%}>gdc8mS*w0CNS z4jKTx1qk6@!rgwGz)SMvEg9~I$>#~Xt*ivzCXA~fnSM52^-kA>q5&f6%Ylme=WV@3 ziZoDHWgqK<)|*r^XX#P&nfHM-C?J@t`5EkT=aniDA9a9vqGj?06)y_1waj(#azVTJ zX?Hs73eO*oXt1vpu>dHr1Ba6t%c)dOSuN@ZcHbm&(GoZgDNUoyFf)Y=bbQI$^^XBK zXHorn{n_Mfe>@jnmBr7>odb=a5+N$cBKVW$72e8TdDirseuS0&Tir`;l4%_N=j##D z53D~Wg$1^%5$&&t`tq~tZ)oqe+}^MboP*nq432kqQ5&dmoML#8Fe0jfbX=I&#YQ<` zlH|)u@E{w~HDcR2(JC-aXX6Tgh91JHMR2y>0kAyM(jRL{8>^i>o*k^%2oaN=U!ERm z4(*r~_G`1gQLUz*fv*G;yp8?QQQPNqU&{wDiS&(ARj`9%b^X(mj3~$iJQPBg*!GDl z50}2SD;2&$!RS+yYNx^L%#zvQ5%Itz+93jaLLddz5rjzWK15oexSh7f9YB6X=Lcm8 z7huNn?Z6ZNm0kpwmT0XD*P53*T;kpJjx8J7!#|s>Ykn9@tR)09_dLWCJ2N_x%k!!_ z0>UQ3aQce(Q{#5u*2^!1F~EZi zq)dH${TWVYFMB!Gv?t{eJ8M6aNzG||SCNu3UG{S%v*y~(m!>|qrucBEu~Y(lfAN^y zWhNXOqAvA0#Jc+Ei>u4mBEOL&-wt09@}H>HI#+Li+{i*CnwBJ0uyCYR)B^Z+@V&z@ z@!O3yM{{cq3HFTd!HK?}^9_mcy#?W!j)G~rv;g6S#F3!~QC~Do8gI_|6YD~xX`u|k zZLP%y$3-`qa@r)`cOQLW@GA$f10ylT0vn&jK*2=J3ajeA!XsqouQ8wj%c$3}wSJN$ z;f9+)%Q&t9mKI`znv{X!End;gnVBP*VaA5L61ci7yl@cau;fSD&Ed*K4e-mjCU`n4 zloMTO#p1%DY~u`Moe|#4_oaG7;Oq#N;)N5J!G>2RM{CCSrTBc;d_PsA38Gn(P~Y{# zR(O)Tg#iVUv(Q7TEH!?_cyHq}K!3>}wot8{6Rmx#nddn8wZF%-v}D}EC|o+!#yq^| zQE%(cJqjARNKZSD8Az9c~eLq(d!%kxbKq4qdPTyQ?M zvboyLWa(}IIDScfbK*9ouSCf(Y>whv!F)w^$VsSpV#M0}%rL0rNeBGH!9cKEsjCSA zPjS&BS*_$p1Qf&1vz0re{1V``^$BPzcI~g739KUTv_EO_M}gsS{Zb6B@DIJ#`SvsA zc~u=_f&Sd8A>>UI6G)7 z?|ND1PWDF?n=v>d>0DK~kCJYQ2Ob-t^}zTZ!M7+KpVFBNVdpflOGV@#*O`-y78jAq zvb|7r40E|>!%*LhNYBU@WUf}aURt$f{f%)anu@If5VP+J?#|y0?x_ZXbYFgpp~E{S zoGfUA770H1v+NCTn!*r%&w2#1-C|xNB|dwH?cXp_9}}uuLA~u~*1xaQp)@`E-Q-M0 z%6lSnAi2mOtHqTrb38Q*Hf~K?THnU<$W)CNc})Ci`nl%GTqooF4X0w_2w0<@f39U% zkzKQ{?AIFNf=wxqT^(0So^%lJgEbm)?AVDX2V;2zo$+xmdFPyj9du+zrJbjM3~*(&hy#hM6IV zHIP^R`jB0#^E3Bz6(TIwl6~#cx4KoM^mxZ|1t&q;&@9u1%^JBmXL8v%%pBJTH$M)- z2!y`LB=Cu_xUK(ehXrJB{+S7@wamHP<+^)8=-y(jcZgnr6M>m1r|^9oQo!7fk+c-u z!akVKwJiA68yyBwM4qkK+0pwF%3>8EsrivPzr&5OwV`x-*`wp=6QtNJj3_{ai7M7q zpGhMkCx|2YYFIE6jYC1~?aUOgCzd0lR?R1F1Tcr|_!i2as|x2IJ|Vn`s#$pjcV+Jh z$dlIT$y!kGQT<<88SNM1*y9pB;VCm#lBe7fHP!o3TnW)8erJQ0n|2mDbhS8AIdtj8w3&?YDg90m z#~OBTtu0k=%8Q_qy()P3L*z6KXx7#rlBD|kz-5|el_LFcg78b8J7=5sArwqN>OB^~xY}^>E4GhMur*oyg}6b1FtWf)Ll8g86}sC#|ZmXm|s? z6{MXgO-;BH@+N_U!d}QXD`uVZOBTvq(-B`xCbF*z=yQcqou#Hqw!+s+4`Zq7J|-Qg z^i$Z)nG@@N+GnkL(fO`UW?hv<)g+?5>bQ@+a_9N^%8mO%)o~B_MbN$PbfZ`Q$N?*+6mv49Bg#YgcT??# zxUDkt#r7xtH(p*!3JG_jM=gIJZo0dpmESj$v>25aLqu+n51EaLG$2(hXp`NKqmd@e zDJjzh*OgE|qJG|~;U5!(8*3i*1)>Ef4_+s+Na!h!WB;KPJHn$8@V(EJg<|K(2u}jD zhO+Qxmw;9ze{Hp4V*J)^57<-fO{%eH5ppm^krqx{E`OE~T|^X~cmhSS{Db&^Dq|tIGFDxBkiPht%`hu zN>F;j$0tN`&^oZ9?6UZd`|^-SmyY%;L(B{Qda=y63=7Y-E_{WVc6^v<1={swhaLDN z(nQ^zf!p;4U-}$B?2B(J7CX11L1dR0`uL~+){_HYr^uFSUwlIIB|pGz^uzu03|E{6 zH>JM{3QqBC($J|+eVl4vx;oAr8X23QRgWhR66B`%rDUnZ*s-nZbj~5dCx_m%SN>KI z4~!4;YcygIu^31QOW(DEXf?pw7+WP0#X3arG3~yTFV=aN1j;5SO*Nkj^V+{XIj@~# z5219Upy#fvVbdAg26lrCE>F4^Mk?8}m-8PP${ zOHD5ZlEa@ZfBQv=fS{`jCa|;DsJ4cDn8j$xPpy48BIf0ZdoB(dpN`krA8G#ct}tB;a#bRZ80&GeXBWQrr-6Et&}D>m z^w}Fk(8FZ9yl6DUg@>D?8RYB{tnIKrz**v(!mkg6C@dU1FIayQh((erLMD$8eJ*qh zHguY1tA5v+5Auu-jUP>x=xQ7)NhC_Hz`tMPqjX^O-+%RZqv`hMvLia}D906t31g^B zZ*!QQ;SYb^B9JVxb?>aB4OI}pb8>L%`hkwo@pl(F;adu6^ zS=@bEHpx|dEOPWk<6?J*A{@d5rhc*qo>R)D7rxCLI2>#YI|-C11G~2qHp}xr(rPT- zDu41KA0}~vdskqYO~kC6^tzE}En{FRaJd3RY~G)JsmS`v>i@^yTSis6_TR#Sh@^BW z(%mK9NOyNhOGzv`rKBW8y1S&MOS%MUK|;EuOVV@Qd%Vv(&WH1C$N6x+{l7W3%f0S( z-BLrdRD{;bqeYkR?Eh&m%Lj18n$P;z`unUQ4h&jc?AcBc9 zR~f7L{e!M2I1VPBFTZDV4Ej|{+B)2VMAJ{q+{**IOqEaYMTj?&b^AYzF=`<_3keWm zz48ghQ@~R)`aamKb_4RE4B3$szG7#DZ?}4$?!G6m|NZ@<-Si@&JU4{9I#D|3WtYJ4 z%+QG~^~Uta+hFEuZyBHq3W&7Q%KY45JTMcMen$N4xbR6HpQof2=abeS<#U#UsL7GI zy6ao5TpvB_ZFJ}H1C6kGL8wF?{ID?&HQrJ#lE0c5e2%V-v<<;m@98YtWnT~RWux5u zE|ewgrJ|`XxWd7n5>9Kg%yxxbwKZOM;n=~03aL8Z(%Ic!YlsUk7gg`y(AKBC`PqbK zCLi5JqD$giFZC1h(>bfJS?~y`eZXTIo5Q=&_W5&V1dX4Y!}X2?$`gIwkuxKyVYiQ` zD@yZ0<6TX&mplE(Zu@*Ln=gAGWyXVsTw{abH=x*aoaJve~j%^f)mq9{3%UU^PXEA$T&DTZ~q!8eO?Z=C|dU?_!im z5Go47I<1KR9Jjvn>be%oKl7}v968ge(c`rluBQnbHXF5I5+F?AU-x)h>qsiHu}CIjjTa$l$angg zX`)4ZVhRov#?6}k&Nv#u^m9=DH$vBHHrh{&0yK2eaO7-c_wU@jC~rw2LArpONvWJ6%*@YDgaI{ z&S3$bWa8zw)2LxVedG7-kzi3|6Vg&K;+xlr)Y;R>hz7pYmu%FPw_FUN#j>|m1*-m; z&vGFzOE?1Soevh8s!2AdUmAD9su!pe7?zrX5_M{^F%%~NT1nV69+Q%Uq1B5drzlIn zY5C`Np7B?AHvu`uI`k*iwj!LQE?x8{!z$rbNQzqRz=MgC z9nLJ#(t%e#VOV2lJCl1#CTJ1x!!yH{<-86Hje40JV=Y?mpuG{=E~J0s88ttb)Drk- zvmh;d^NcWc6)kWjG%N2$-&|PHTaMxy*(|-)6~yfHEm{b^_bwbb5ucx;SacXgSW`16 z!t#v}{ftmfnKz~H&}8vkOj|(PcH{E3rwkbG;d)+ar@5NpX zXml&0lNy_Ig%*IQzLtH^{TKlalyz?9QVfk>-X^L9ex0YN{N4X9@ZuXS=}iCOmH-M@ z0$T9hm*1VN*=d{4>+I1)k;2XPSiouHx#8}GvA~Lhs$Ux<-1{oM$n*0e30xlvVS6m5 zA3nAI80GVu^$n|A^ThjLikEcrL-{O~O#6G!1C2()^zAXc+-LLFDnd*=}>A8@{n8+9goaUWr_}w2`&wziy zbQg1h>BZ@Zt+6m1&aHEu^*!iLRS74NkQ1627Ch9+z7H48LQcMA&Cow@(rflfLns#- zMrD>!%^1=Kn`FWcdgb@}9s|UTCoa;Pn~Cr4dwlAqU%E0!=?lny8u6SNo~iLn?n3@L zVZ&j{w1uO_rW|RNQws_CJ&L3c)G=AZ!rrN_YFKu2#D6e;h+akY7 z^71yB&&6^92CzBAC@z@+8Qo{XtmN)g>m1D_GhYv=&K2S;4#xMjPJpjp1m?Vu@)fY) zoT2{tsajR-cgW)G*XQv3c28N#H;(mAY#o#1OARFx8{X0WLQGtP@WPhX-aHa6$wcNZ z1q)B1wHSxxGe{<8O}Sy~y}3Hs7!Qh)o3Qb4%+~8gZ3m?B1Dd&gp&G9e z=nZ|)|5k|P`muBVv+Y9^D#VOvgaam&7W2V)9Rk*c?wZ8}x#*d_TFf$=#zHN}(Mx4X zQwx3OvLPRCr({9T6MiGhd;Um#W|VSN*=E3V&Fb2YY<;~sxOvw4{+Uo&>*;6KtOvGL zrt9K=BJuoHyPiGU^Nq)l-Z;H+PmrZBi{=un8LK=DlFW{{dQ&+c^o4?l`1A1?K~Q)u z<~CvGoL7l(k3r>i^GY_K$7ge4JM!&q(#fhgEi+Oz;SAa|l@u6r60^Q#(UkARA3YCU z5&Y8O{OCf2)-AKV>b%Q@Q{)=9+RHnMIR7a7GQ{Lw>7+*GFq#-c>sC3UJR-1^7mLsw-Y+|J>q+9CO}VwPKj~9A zc~5-*hN%*YNZ836b?O`{PW1MVkyVr8Y-sTmkt)oVCB|G=CPH|)2KlA9x*io75Zk^2sB{WL%SK zA@8mZR&x7PTc=(hh+#bDJqS~H{-{bn*VMm^5fBZJ2_rzZ(9Yz53w-3%h9w*itn`}$ zR*RPh_1|nUzdYEpiV(c;BxmqtUUAa7gp=}&d(tZDL>a^4FXZbO{lJtPsX^>d)a;Mn z$7?Rib>~hKehVBo%VOhRq>0*DL9(5U{PQI_L~dH+By}!^8}=~)o{wC#D8DrE@E>JT z-198S8qXTpe{3;?Z_lz@Z(7@Oh$LWr6879pXJ2h(hgYc?)RB4G9t&^&w65ew$q^@W zcU;5NRf%y>7+DQ{j(Y>QUY?M=(G)yv-LuI;I*SMDfVTX_Hrp3!<{#rK>+=R`TDs>} zKtpkO1mi_Ybz73`E16E8R{at@$(cV7A3XT;=ORBI@jvlUzcP>)gw(G(F(RnU3H-%% zDQct=3dD=4n9Xg;<>R>JRpT`B6(i&0DL#Iy_(nl-AJ>!hPAySfvOxUq{xrj@!euL? z>9YMW!@cS7&|5g`GNTD--_ox#Q)&grTrxSw(kb#O7X1I*+0o!ad9Y`oi~pk44QyNt zV^CqK;Qgywy6I0atsptbV)PNJJk3X-V&e^jSgIWmnXcsdnH=Vm!+3o@~&8=#1EUH$ZVMsMT({^l^t=w^P_?fsSj)>d}V6FKNg?z#Jr z^tUK*R1?-b@IQ_;R#+#;D9ZBl;=lNAHY@>Azt@25TjaZ2x@_mk0GfE4<6Z(~5l}Pm z_m>IWt@;A4QW<(do^zLgy{!Yj=I+$TfUu&m2VCbyHHQE1?j7ARFRg397{qhWnSVI> zzM1BjDGZwJ-3`IXcbRaGKZ`xEh4{=)+z|eA z(7abs2WAo}b0PUu5+IYZv57sbqBH6t%Z?u@6C@JuPy)0LGmJ3_K58 ze2mXJWrq6>xV!8dNPC_vVnPkJLU6l0fKTq=~fj9MsrLqk?-;fzZQ%?e& z<1F9UF#)hR+c<<;8YlZ)DR2d#dkfSz;oUd5ro18i%S49vu=R8Tg|VD%^8$p5YR&^@ zZg_3Fp3_l#`MkotfS~o~BeRMk#e91zu3jRSxW|YToj^;S)X^GsQ^?as-?oZZNeCR4 z1ov43-OryNdPp~s{@MO;?0dWK=+Af~3Td48y)lm$M-|}hqr$>IVOY|n%*B-fm(dr% zPS0(lTLr@(L<`BscunOslN_u@vK~Z*Z~yQ;O-Oc;rm^_4de&MN}O{zhQy z`xOu$#2P`t(EXVgbw#kg5yQA!cFaKVK{_GeajG2kHSRgE7BrTLDmd-r^@gl8qo$EH zn>wpDGM0yZkrf`p?Gp7V57frm5=njNR0j@%&S*#AX*eJ%=$F?!)Ah1}3ZSky{J3V> zH-;5^Jdv3bP;Duy(g`M6T;-9flGLxB4J+QzDUyevj>NWuI4DKCm#` zD=D-*S+eU<#kCdFy`b@Y4*sS>`-tih5KNc|85Jk_x}hzxbqVYi=ka4mEDJ_`U;SBS zZ1k>^F zJ@%0gOF5Ni{;c8GxFCJ3+&u0{g0s@`SQu={iE?3ur<1T-TjLu&C>r!;DU;{{-KWwV z;~PmIro@aLb?Vts$7{n@fA--v7R`A_OGW8r`c*bj^NZW;K%yh|Joj(5Q9Le4FRys$ zYX)|z&cBR!&d()%04H7LnMPo2tF~xzP(bz5Q2V{oohxw7h#F+e925$jT7<)ph-xcc zvsztpom4DPrGWoZ81Tg$>0#QchYuUPhnK0PS7p^!rgx#l6Fzf=<=W_z1DNrctVs=W z@j$9wm-+`kR>>K*lW7`B{MynZvT@smw_ZoOG3}2fuAMP80<QtKY8GkOvVC?s+cy!Y{Z1|gv(q@o4DXRRVKGQ-LFh5SX(ahNdDI2;d>R57;q8<@x zNabtQT4d|^h>#*<9{jf9HP=EXdrfqOTJ#mlD~#Dtte8Odhef5B?%))s8%B0Vh6uS& z6p>up@BVS~4oT1y<4D9sYA;py2d8A~*M0qk1BbeCDeBPOmZca);XDR+=48~2o3Bq& zWaci0leVbYh!BdslTC-va$oloc0aO=gkq7R8#WE4Jzg5>pc{l?z5@51vdrCEOH0Bb zHp?2Vhxm4No-5e(UVb#Uh3|O*(S@_Gvg9^F!=aya!Ry|n^ObW*&;TiPIPX@_9(gFH z^hbu^`#ytbd$>o(h?}-nYT$?ne8XTK1yWuRKh1!;e(vf6-LC8H^DDzjm zfX*_(mq|K;R)vamT9_@G_4nI?FCBtR;?p#Dxri;|np{ zdvh8O%@tYi(Q+GH)6fefwk0cWEf`u99B7Y|LJLT!^5dfog;i;W@++_0PTRjOADLUE8ExU3|dWA2Z7?2_Z&AO*4w zh$01I`vH2Hm^WxDPv_-ilacGVVZI3!3=7@u(1+;aMHC=m}2Q zjy*vUKlrE*$sd((&j!{8Ze^rTKvfx+n*CqKpD2>^Pn zJ$D|lzs)u{WHd38Og>4%RWH_$|80{0=SgEb^TGUhL9e5z@wX{wq64N}DE0RFU#Hv$ zaZ$;ce3KWde|ywGQ?M|eCT7pd|7~F?pg@i7*kY3{zyUw!f{6{~65xH-pKU1};g0v71@`_kkSHF;kdz_shyA({oc+ zV82d16&QaYHaj@HcRgs@Dtt`p!}bZtS8V_!lA5MEGCg(=PFscyzw4b1@1tCTHtPz@ zm2P>HR)l*XV5Yq%IesS{b*Z){Z=#1BRTT5$6Vdr5c5G)IPE~vI1;ut6+;A~#t}f0B zs7ti3n1%!_72~eSunCd>=@7UE!4Gi|e&z}UuLV%T+dlA@m)N)i;DJ-~e*KS$czKo- zsIxF%+MEsAsEeAamRk_ZY*uRCnNxrnwx0O$U5_{(K~fAJ>gQNFEws0!N2 z^gfW4XR3ZV52P#*YLHaF;5>W=z@nTBOK1cpAxtl`+PckD5cI zkDt4$@PivBsqo6u0G7#-7a(^M;m^f5{!Apb*X6Y?0MYU!NGjGea+}e$Dge*pA4?TQ z8K^7qkBoJdwF^KwC%ETk#KR3pnASzYpad_1z>AF%JZ%GZD7QrkGK|#&fZXY};Z5Gc zV*o|TPc{G%dBv8^^)Y}+UNBd6!hbIE2Tt7ujFqh%h9CxF2b}w|0Orxt1D(s0>Y(zC zUjbbj`ra9j!)V`JZ6pqHhfDtg_o}fvyQ@dESWj*_uk7J!@2i;K5 z;$nQLAQ*86u#*a?%ylYy-us^?fwd#&Ua;Z^BeD-~9$r%z0}M^=j1kn~9R*OJojvIO zumPH9`X7^pKl)xMZml(*RFSdYx_xgSgWi)m48U(oVA5xfg45q1MyFw(VuY~`4CSS1mhF{(9F{; z7wo1%`G6pDk+1utP2L3kwElKf9plR+w8b|@pSKlJ4k`6fG7&?pTXn^L0zA)` z{@37-ObE5G0Z?0+uQ2~HfNk9Azn4V@WXCA+c7-&v4f%fi2jqjw{6({9zR>vBfSWkr ze{X^p^Z_WCYxg|v9|m3_)27Elch2>&Mbh{O@WIe*C}7K>FNx-!r&H~N@FDetiZ$>i zn^`XU>#(y)Y)kV>i9l29-2{-Gs0pRh`3Gaolbsy_K=*|%85E<>*V0J9gd=uqtY<^O#q%>rI=Ia0; zO6Qq}h;%TxCh9)TNTwQ@MVlB|lOg1h6wV@z8dA^oxD0PcATZehZ6+RD4FY+R53a^< z02ZoL=ni^x)$W6xhS|gngx!)Y;mf8CradSWz=Fu+v#btm zT63}iV4O~4PZ(G)jF|%g48xM)Iw8Bkm+%Sg>CQv8;kPMAjmYGknY!*lbR>N@`w2Wp z%TZ*aO^(ewg%(E1e7yqWxb;A`IAP)LcMq^6GjTFJam`j5JJ1|#Q&Ny92U4CSy69-4 z9&Rs_`kswnZ5$|51ac3BHYcA%L$F-U+50cvK{WzHcZ!Z)>U^ktt4$eC;YWI2X>~&# zCnup(Kk6zQExHVU+Z1KY&{HPp9(wc(^qJcw+)$X2ScZat>BCb9#3E;VudCCz)P;l< zgm7K9qPe!C?C&K`(#3&_h^#;xC~~)>Tu{!E)g<8uj(e*TH2*{Gwq4EV7&L_JpDkQ@ z)9`dlzHQgzqkQQmzDFUKu(e5z1+@e_e8>YorIne&Tbe5tRPvSdE@vBsS0Oj4d?BAz zX^q&+Y8XcWL^sc;-AMO~e=EidUX{QPSnwCe{2Y%XKLUwN1*i8}z_E(*gCi$H;0~2a z54yGvJN=#7lfWgUY2 zC#(J1bABDVGJZRN=p}u<0$T9}yyuCYM7M{3?j2`6{T0$h$-n=qD(cWgYHq=1oEc)e zF*n~H2`a|?FOH1$UA zy`GYpkA`t~;|$;}6|%YQCM-e#b;a!+XeHTx8iQwR?wWDgzm7v^>FYie=GJt5r`k`j zB+u`J@@PF-LDE+w!Z(f`gEI)DZv2uyK{KPqr~BE`xAVFLU(?|4d@-#(WY8!O7P}8# zph&z^N)z+7AT)$0ueE5Mv81D+Pk<4^=Il^E(bu5tRVhgtH*%!er~8l~M-c~hDXMOz z4G&FR#Uf0y}G}cz@x2Iwrnc zyWZ{0^@7Xz@ycyGy{o3etF$%oG~`@!|B9X9_EOXoPJJoOb`SUM6^#AePSL66)AyZM7XaBh z!v5^FRwK2S0ifK1F{0D&^7vKdsh&R2 zpsv`xJSqP(H;27#v@dC+2%{NsR-HlZ9df!Z1_LxaWcpLnnU1w(A>rt801f_sloTZ$mMQ zp#eTgJ2O6@|bmU$%rOKC?s$fX}|N67m>tY+YKME z?x!T(|2P(&ZmSsg(R~!N{3$rDcBnDYpGE&5TQ=7T#_2eMTORaTQq7NNocX@ZmD*FV z+S^q%=o}lUw^G}%_q4+7LBP5Cv{(8wf`ejy^cP)sKwT|lh))-y-^1bWDVP~Y#OYa- z&}wQZO?DU}U#Qd^=YFDs7m*0w>9RWjT=@X6U^oji&~KE^m^8N^_PkHU)#bwN)nhZ8 z@abhVQe|K`x!Jn;!coq>Ds1QZWOt<*b0!EWk|=o^csh1@EqqfnA(Yfwp($iCTj~~E zKeVsBOtYnPL3lRJLHqSp&XJy&j2|ligQgLdz@F;9%WYYX%Yx0{k%p_QDqU@_R!+d; z%fDmbe|K&e^J3Tcn{~q|xL{OlJzbY}sCygV!a{}_bMe-#lT8pVICHA|%_3;23{|d@ zF{M~mV&vhDn?VYW59ju#Ah&$*EzA)Gq6zqyIWIXC&;w0}5ev+wx)o7m`Y6Yj&Fn0a z`QbzMMwPB`_dY6Y_vI}Sqy7C8e3--g*iC_#bLKu-z8>HY7XA%jw!5{3dUYEi=43cK~dUH3N zzVBlqu*o1F@TsE(zWLLoaM>(oG09AOvd>kpalAwM=46mI>G08=1pMIxU;YI*?VABu z)5s5>rRwI`{t|}sy;bk>=kR=&;R#>qi_#}bI1hkmq)#E_M(lbow1hIYB}Jk{^5;#j zN_Rdk z((_f{og#T*x#bA<6lf4{d4{JwE`PWtI2MS<62zEUq+{3U@}$@F^9OZJBPZhn z19-NLj9h{WCN+4v2{w~A85o&vz#hr@PIf5$GIRM@m`%MGY(+EyEAWRLhtx2oY8|(f zf>suENySWUG@Mn#Q9O80;tQ9(IhAmq&uKGMk;ZQ-Y56EpQMB=N>NsWYif&bz_rjcv3qT;D>Es&!eSley22C5eveH7*0$HEa5xb#>54|kma*iu2 z#zQz5m{1A7Z;@-Os9JI9if$ZZu4~TX416NRV$+<>$(4ATuHv&nzJPpU$Fu0F?pL~C zb)jaUzyH1OxRinsfjAx1TOGnQU*E`fUm2(jpGv2?>g<+#xvcbAiQLRp)VWA%!JKR; zUajEl@aUeoQ}Yts4dCgv_S4Cw-Iva!28|57_Uf>q%~N>OprxsF!qP7w*Q)Y#L{f4s zZJSn(UF>iOXd!cOy;Oz zIlfiKPRpBU1nS{ES0A{P76I#k3dd&5kDo( zdu=>j(-+ESQ@CAtW)8X^SV`V*xEqSH6DD?OW;N#wuFvq)BPf-&p)K^vbQ&=F0mTdEE$Q`vW9|V6y7qOp73Ta8=&++K{NJ ze@$%3|J@~HeSRbywl7cj6gFX@U6%(&icDo}i$eECvFU3hlPIEP9Xs}SFJ;~_%3LoX zU91wHl$Sbffj3(i>fGW^RCtIZ^*F(*GWHq%%pkUx;T|8d0yhMJDtf%0u!<5j)F>Kw zO7*x-+CW(#G3?tHjsCCy$SCZINT?B!yN9G7H}!k0))fv6J+@VlEkor0v2sc@5CmUV zxaJ6eCXqmcE3RK;bjpk~g{7Nh4G4};bvX4=z6vE?qai}l7nr5FD((9021=<>M5CZl zhW>-X&3JBq7-zP}dqSN}6k-Gwr$tcgvu%MG$g;ag-sw3wCqZrV9P5ZR1vm#e&IHwU zMsXIlJ?eK+k8F0P*2#lAru&#^D|YDdpDW*j3QS&=x4Y4{2HJEkoiE=*JP zk--k-g@cqCzob{)s4JSp2wl+a@$7!uohQA+)g_eaX_(G0j@b&o=GQ)RaY(qyzKYVB zaVbZ=`n3wh8Zx3JdNq})5}NA1|IA)1V-|~QHvbl4h4;Mt6{19!Si14uonYjLq?(4V znEuI*MF)N)X-4ql%U!QOS9F9%-WjB5zWDWW1@GLzr}m5)Gh6L2q$()E+B;071xhKx660uvR zy);+Tl-V4Mxv8K#Ebadwn0bSp`~-fyzw&)#AtmdQ%PK;yBhNbuR0(M`CN{U1QVA`x zy~zw1(B^BBdX3&Id{P3yMaGU+TYsG^YwcIex7LbrAtBiCO+cjcnO)y?DUZDKRWAU= z6`lmPic4U6OSt+*ZGwtU{=>M!jm+ly`rKH=teLr}AS%8SBH1r9`{ZL1TUYbbnIgHbmQmDQ1 zoi;2307)vAhu{1wIQ!mNSv~7I+$Ezno_)BytNDf3p7DMvC?jYweYfYb>@L`dUQ#5e zU4MoT6N|!lTh8Id~V0z*p#5b-;Wj0mDG& zzM$*Hcra;u@qpS2?dR5epdnF21)Q5Heh@U?KX^8e9V#O|QNt)J@1S!d5qcOb>4_Fad@>i>ozm19GOv zSXnQ5=B@KIG^R?bs^mkaZ9SR%HzDwr`SX;TS;d;DDRLYDPuuDMrd-e^*N$Ta@@?ra zBgJeX;CR^gJ6^Oh?I@7!eVZF|m*G6P0-~B+bIp-DxsVAK2*E1RQKpGm`fRKdT-5ct zK~SqPkCEuq4AOKREdtC&jlH{kFZFjdkmMD1VUnZYw7>MKwa`bxn&96nR~J^wK~m@y zb^j@y$3%zI1;mdFO%==D@;y)}G#;nskZtM`p3+JCoAzy{d?*w->QDq(0WYwtksL3m zxaLWL11mzM&64NVezUKsYG%ZHSI?*mD3e?k67x-R3^S>dFk0>vzL+(rU0|JwmVHd( zVKRGu9CRJ-6(PzWk{0ajFZc%_XL?BXYAb#9J_}vgmF7BVb`wBGn1{k1(F7j3w1Ka*Jhi=JXOE{*cOK))EuFkcFnqSGbS`{Pf({>2C4d(y zYhj7H;ZoF*c8RldiC24POG@|W0Ch!rYq2!?*B33MI#q5@QlZ_z=8XoJFwDQ_dcaA~? zs&6q59B#!0-sAB&>|Z}B|Kgbs+sQ*1f%kmDhPoB@m}*A|$I4F86!XPtO&Ys=n*>Ql~>e%J5!MBF6xgo<*pnsVn>A z%13SD2Q_r_t3KTN9oOOY;C`Nj9lCZcuxwxWgj z3J|uz)$UX~gDAxcfpL~@uQf{tiIl(prs-#q2!YMFWo`u)fS}P_CaN}|NeGGo-A=|A zqOCjA<{veD7>bmFb9E4?t=i7V_P1h365babwnF7I=A`gMy=e z@a%rhQZv>`usV8J#iLW*oW}azHH{ZN*X3&|ahi%jM7?!EO;!)CDX~I7#z^to+v~e9 z6OyV{DrvXtP~6|3Lq#0cz$02F{AI>(?5Gxx@oa?YGK(otmsjuj-!%>I z-{FA+V;*=)5qAk8uTCw24Nmqf**{ka(>cDn3zA#S)rF!B$srglw3Jpw)O;ctJH>=G zRy=ZHSOgL!1Spda4ww$?hr6v_4D*SeXY0xSg-9KDAQVC+*w;23CHWvj9#>N$g37@ zpD}dl?C6@o#b4ahvmvt~N=>^QUX2{j4yPL*MLU)zsVXiV)F>H}fTa~L(yy>QHk`eq zA7N#^d-IK3+Ed-3yh#&%fFgJW(PX@p5UC$U;d{;iX@gcdo!)69X&`Ysspg5sZV)qn-my8eS$;t z22`f7cAoZ@t_)p!J-S$(m;CKtOHV9vBgzdhjT=dbT^uc~dxhpksU-1JqR(ZlbsPZl z&VJSJMbeA=Tz|Jiu*XGHw;fnE9SWX;bm{ZkcxO_?-`2E>Oe_>tN3Zf_!>EB8d38N< zW_6YugQ3Rx>DrI13LXpzn~>6y|oCr>tyi>I4s z*w==udsY6etg_u}Bxti{UJOrnfao|@`nk+<{!`~RMF*xAQ9gBEF{b_vg#)E}_!Nzz zzflOw2Tw!c%O;Fi2ts}7Dw6^b?j^`7e{UyKT8jk?T=xG=QR4ro<2pa5^#~aGBx{@V znFTw>Noj;O*@`h(AqeMxz1zb_joOoOk0oLZ>Pg#!ksj5l|ny@V-}j+ecgR$u0Tx5fMEnLRtvxkCWzw4cwrgr4tg8 z_|*K>Y=+=wl7{iFI!f%y6P&&uBQz7bk1LLRo5+6m49%9&WQnpouyeMNzRQ{|nQ}*B zV0-%t|8+%h$xMUb7w~F5FK(}r#iCckt*qcqfI(Wpo_?j?+D3&{2~P2|ZFjSyfNb0( z9d^-G!5|z70ff@GR`ait)eG%h9^wpT&%Ql@@jYGtQPMJMHYrV}y)fyXKbog3^(V#8 zXlbkB2H!>bkwHv6sTqU8rTXPYVw2ApOiFd~@~rk|)ILGcp-ya>I8Oo|Wi&ZMzG;TY zZp5LPb8Y8D_hZ7tHO=eA#@~IP)G7ccWDdh01LcvW6f#<7B=@y&-CzQzf8F>1sddM( zvsyX&B#gz98ZA@8OwQy4x#zpANK?;meJ|s=} zt_Y-j1}T}`N*ptOmrw3_L8D#~!MhQ_*Kk?w8PoBml?`E_2^d*u-Q@``Rq^nBIwMlnujT?iwDgJ~ZS zf*Wt&Umg2oJrY3WaofR)@p$RiXaJH9Czba26aVNq_2Lab$^{-dgST&*9?K*mVu*@t%h<&`iws-W_pMCXfdVvXSX&bA1z){HcJfWi~pcAr6 z(?B}(TCxG|_jErN`vo`I%~qoUmpsxv^7Ennxi2EUW1lJH`DVq2Ud!EpjdX=xlh=jh z<kMeI7$P;1D;4e-No|`e1sx!(_zg z{%VVMzP#%j1B-0P7juRLBcc9`0lc_lEsJiPk1=izOB0}V;bkKO76sU4O*t0Rk`Zk1=D(h{f2$Pna8Hk}7uCu*jeBw<_E_NJ1HSi}T=-zPy&lL#A5f(~6RQWHhlvUl2Y0LRv% zWS}my3+{)IXZ5&;`hFRptfogOK19*hvv7-;6SJUgp?928uS1_6<=@A($L(4+0>?yl zNXxwQk(xb$#}e?G+$q3z1QMawKDS+xrhGr?%KKUp0?#^TK}l5OV3loaoh8bC!~7gJ zrCM;*l*<20)lMNllVl7`y2_5Cqz7-6uODb`aQSlWN?eDUo@pjccA<(g2omML!q=L$ z+3k9TXhm;r3>tu%N3YCYBcbVYk0%uw6(jc16S6UP+@4IU-)D-$_c9Gkczk*@E3jDs z8Z1ubsa+S7`6yzbo1v)&bTf=d(Xn`D^r;e(!McJjl_c0f@ex~=3qfG)GTGjj#$L8m z1aXq^FXU^BujwrRBqBzhp}GVqn|t@_MVB1;B}Js@ZFsQrc!_{A2HV+WqW_2wqnBw? z!{bmcB&n!^i@k8Aje-;;zrXR5weqJp86WLs-mj||i15KXNXCx#XWwTYANK?HK(U)C zR++r}TJYG(i159**F8)Hz+O~Z>JvLI3jb z;Rt`olE4CH5NtTEEKxHzEI1>ZinHdZew+hm7Ew8y1gdQlo|X=I&Oc&fUEiL} z4y-=NQ8j$7Dc|%@3vI=W5(l063~VKW!L(#d;WC`AN^dw8Sy3RctH6zq#PZ?+e#6;$ zACC1+393pW)T^dUSA)hG)LCUAk)k$x`d-CYMO&pWG({qy=1XU2I}dQhqX0b(Q=LS_ z8|?#~7<@q#PbM#O4kdLdX>-Qv)K<_Di3-g=5ovT1TyFM5&i$a67;`GMGGNaunj9b=iiieH z#U9{>@bBYpG7q(qTu`t*{L2SL;bGv*r&W)W|MI&bp&}6Q?n{~={`&xd2irv|9w^OQ zW&hiJ3$y_ec-#5NPPBg&%0u5bg|$sr16htXpZ@(F#ft7BkN}IzOGR}_E#sF`|2|e= zZdj**w_hKgn)$au5M>0T_4)1i%fF7nJPnxk_PK?$e|xFlAjR<~6E3X3j#fRS?oTV; z`RlZ=1G;D}Mnc7xf14)R_Fz8E)4cZj>n!;Msryl+Z~*uA??3&2hVpN-?f+sZJEK(0 zg@AU`2HfEsZ{`$~LF*?wJj6>55s!g3ZeJI_W#7vJ0PJQ*T}D|6QbLfcZ)Hj;()#|{ zq_XnqsDPu_+^|t1zLCdDN%MLBVpz@9{A_z7FrohN4<>su#Hc*qwpkeL2%4x|!O>gz zVn&B@A6(>JCi9dBYoYcXWytx~3p7M&+@pJ|5C3nXchC}uM!A>04D`)^1kgFSQ(KC! z>#&mv15Mkd4|qvDdwZO>HKDoHhwZVQ6E*>xHB5rK3;z)@Lv7DHpx^x9`@1z>XV9}+ z0Q8wc%+)>EHj&Krg#8~UpTYESbv5oGJ>eu|wtfb*!W;mJ@MlDYTiLD!%H%p=*Qm&f zcgO;G31qn4x4{lcIrya@7(+-QSTwnacL;jMQvzNz>JsAKaLx##i~PQcfCdkxAPO<0~# zX5c@}g^(FsF0>P})&@E)Bs%^|(4guJS*-e$P%-)cu9W!cy?kTTydE#l-{HNVy*MT9 z@*gjNZ%aUk%C9=3XZy(+wCrChSNiVT3iQ)9n|UfG7{~#+w&yiuXsrSd-LRo6_3O0> z$NWW5boj*q`}VAlQfMI6Nh!A+>eI`IoaR5zzIgtUZ?J*D6a-~u#XY~m0C($fFOP3W zBUZwk}+`_~a zO#GHeFf387M+lOT!?-H|*$_}YL64+9=P;mxjN*goe)`@VD($`e)nGJP|C{e8_Uj zNvKRx&OSN%mf^rS@N^TkAD@Z}AYlUk$`~|Ug1xHdl=A-=06d$_mQcO}<>@eY-=H#C zGUT|=KV&}ox!`Q%aI#PMEjz-`pwLpRN8*RzZ@4o6g%B;9eGBDlWX=fH#js6Ac~KZUv>;~-l2$&lpZ*5$^^yaFYew3DRicNe zxcFGb^4)aw?|U!INN;qMEvR9~P+ZY?OFfvSS&&kG(#6NM2TM91TKf-Y2@H9T`oAG~ zU_c(;nf4q*yWp~3b72Hom5GnfvDT62|8kE}75oE;i99Z{o;=B5NPAr1WO6&&KzP)8 zJzY790ow;X)}uSXR$mS>7MMN$4RoFE35e;6zHCD~AS}VvzT?Hj8^*vAs!jzX+)Zgx zU)QbrF|V0g?7o@MajGLAy5InY=A^skBpX7APIPNw$P6`z*fVOnmz z{hCrffHM0QB9?}?Qqjc$YM3$WG@?{7pl6w4F#$aMg=v)h0>0pCu_7SoxpFmaf}&T9 zHu(7$2{v7(Pk;ZF>baky_Wha7}*BfK>RmK9;;%R$8XW-V09s zACpXFZ-CmdGF=f^a@+w+G4veFok>59%E}hwOw07Fk_e{j9wc7x5Kw%z1c!!8ilCfC=ZjIuSbd_Dui;t**id-+S+$+~Y;u5)vc=Tt{~em5O-Bu3Kog9Q zI9bGrBt_^dJDSyZ@6S_bTMN_CV@WmRK29b#-4Hsrtvx^7P7`A)Ns$yCK&PeI%h*lJ zY2pG*=@rYG06H4rL>Jybd0Q4-^9=I|p#7yJ9WGK#I#^v<2sVO-J%qo;V@ZNTNbrw| z$1jq3UNjg5DH0u4=d;4Z2erd4U&upkfaH6`9dHIz271ySz$+u`e-ZZ9QB{5I+CL)F z-JMc`q;yJ4cPO1AUD6<_!lqm4l#uR{+;o?8w}5oRJNNTD&-p#$ea|@G{}`}*T;w{;MO@z>;*|g|7kEJZkGGd2Vf~J`NdLxc3!6Xg6o-W`G zLtFjMzZyY<6!wL*HnHt0&L?N|9cWM28)`Sg!N6gv2eu`3IjL)s68s5hxmP>be^)PK z+FVx+H7=0q^VEQ*l|&AXg;)O%_D+{scQr`8aNEQW;0&xM9@G*_=GKduI;XDM_#9TU z6*x$s=Zfe&^KnSW(^3__ffg-E-}6_N5vd(Z<;~*A=@3Hpk`>~l$!Cgb@`^3=P*q)C zI>$C-RFjpH0rusueb+GcYhHO$(BKzoy-^leYFZxGXa2J{IZEFO!%7TCaJON$50xUD zmoVOTL!v`7uvXqz%=vH?5aO$|mWNkiL_ zSLWgzxZn|$viNP)&1n>n>!!6f*`Z>v95h)0NOZEB>B*d4kjO8SD^9`p+<|gP& zWc_U&ZZbOjq*#4MI4^QH%f0*@I*`x2_SqXpA0a=xB{`}g&@?}%>7&hiHCcf;4BEEQ zuw8mY1F8byLIlolE}KrUyBVZ}*TtqD{2Glmzou(@1hzTO)Usxf1d*q;#G^dSm8$hH zjw&zDvn%54TkRE(>?j|=zmpQ3t}`olzuoaun6TR7kSce7ioJ=qGH)9cLGm5$<<_q- z@1#xT2`PHAB#gD4BUwgUEWf+DtWnr2ai!cqM_l;A)q~xG2$nc<;+=I@xr&O2*C|%L5`G1##O5x9je0OtZJz6yZW(`Vf?V|!172oJW0J97Q)pcQ`g zy>bLYR^JE@7fQSgLcB6^Bu739y>l8Z%VXmA=7G%mhp{L>&HP)KkI_$1X>LBi>{O~( zL&*|v>)*(j^s#1q=$)4|rym@CrHJta8V;6qYrRKWO?1AKsU~hRTKr5{Q8NK#?{i;c zlUy{s7ry>cCJIo6qs24cdgx~Rhu;WpSG)-F(2 zv8%dI#H9z}LO&Tl2_dpa=!pHM%#&zvO&zD~k{kCCcS)gY=G~32o86oFa6I@V`2#AGLWC>e@-!rjFXT#|21`@3je9(<3p>f- z1-#l2`2l7?bt0aJ=`9FvCRK;t4R8N*Z19dY(&kF%Ak$!H=?ar!5406#)7}& z37*Vw!hC_u#v49=>QzCOlhKL^N#KxBlb8|@t`Ss?#>S4;DARdr^FBm8;ez(E*eGKE zs2$}GsCzBG50rc>M=dkF{{^_3ynVMwjYyw$C7KM3zn~1HGfD0kqzDWvVfc{aMxKM; z`g#swiR>oQlWgfii07?qK3iQ@Z^t|Idrk=S#1AOV!$n<@D|a>M{uUzkyTQD=u(+SH z#9N%TNSgHBv}JJ=8>(cmFPy3;ZKM6tS-$}uBc5V8f}#CK!GUNwWbo&r%TB@i4eBC9 zP7Bj`nHkVOKIm@Pxm$>eZegsg?Hth#T)0_^W`y+jemAQ)x-Y1qtu4Vly}VA=1{7pc zg1~D^3xT-PxJ2v7o+&0Xz?Ey?q7ll4-+1Jm=73uDR=k3)xEbATMZI?p^O;J8N{Sx4ro^U zywUnd+%);%yRl^1RQ(+yAcdG2+r`+I8_s%;?5+i5C6c>)^uHMJ^^y#|tP6!#^hHO9 z?zgsQrVet-`-z@fl~0UFP;Qd#V{Zk#=DaJ%?aZXsaoq?i2M$`aoR0RW#8jEpbU%9c z@?Sw{!x2TN73;yViV(`g)kH7ePHylTJ?%yq>r#Shb=S4eyu(L%N_U3Cx}UwKGW^;} z!Ga;2J*PL%!g=?IJE%89aPxFLH51Tt2W*KE-gy?pt>X~;aM;8T8_%;nTceC?ut3B| zb(cV%-?i;g{0)O-B-!6)j2Q1&Jbz7rEU8n*er~HRe8RCXw+o3N#<%7Ct9zdO`P0Cmv6@w{*T21z%lA6oA17?3! z>EoF#*u6!4G(hcSiB-|(iEG*L0D1<>y02w-3VOq8_NVn-hQ{HTON;G0D+P}-Ow2=L zV;v{eWb=qxcBH2U1b^kUtck~Ul;F*#X`^)2qtfUvdZhxJo)tto31}a^qlMJh zBS+}{CC^ZUM5Mr6uI9@Kyl-%^K0%IvjZ7fd=}}F;vG%(3fv+M3zvPw z7sjRUU{Gz)G^~$+Z|h7EIFl>_JraJrW8iofDDAgPzK#fKEPfLs-Rj7#P2|YaUV?Vm zUS)ibZIhfiC@DvRf%jDZ@XgcC7HssMh7YnLi26<{CDc1B<*n%9Ql{dn1leXolH|j2 zqNpWu8Lpb<&xRgt8Q;Zp8+b6Pf^#dH5^%Ve=jJHtVTwqLO0;AlD5^V>nfd$`O4&dh z_npyv7^`J^W_12z00D!1)Q7fMUVSKs5F7qZ5L2aGI3}s~Li`L2o~~5>>SkMX*=5JL zNr>v*segsJ5fTFYb{_xGJ_}e}~S$ zScu#VtX0LGF*vFiqi(C~zYUb;Iq-AiB~%%k7UCh5veag~Wv-Mk)=isL^9}o(7+sbn zMS&yFJ^Tyo8GaaQKjgE0ne8FZjKV(9Ie12Cn*bLUri+#&O1W8mvsawdPcZE+v(;_u z55LVcAROHDYz|JCxfzHyURce`lDB$K*R(sg+RGSqx!h(na_Gy?4E3V2bCgHw-McNE zq{LspZN9!-GXEmsbv?*FL;REOPsQ$Y?1NbD%KR#PhK~+6!Ne@M9V-Nv$76_Ubs`rP zQ7=cMG!Fq|%=QVYF~|VCmb65R2!#8lF9G)pcFfs7%JBDT^e2HPiGG zNHV#Y$`D$sM@^v^0ESF|0^iyF_0;eQ2;Ygu-okTy9oWW)#Wdf6fnSB@_>`DN$9#1f zqNh9c$utb{t12IMFB#X`kKVwJPVf_&_vSKE8sapuBcha2oPnb!AQ6qiY$ZTwh{*3YHe zB*=@(B2C#NcQ*b!ws5bio9A}T8dIwnxjmgg zB+_@?f>%0om({T$p$$@l+m{V^5sTxrp^kO*M!y3En=`{gR z^8mZM7~QCY3?bqt;!$oH)K%+`o>qL9FJs7OMk?XV%vT-Qv{=_%DiA(ZQR(;U<`s9S(1PH-w^6-3J2|GV zH845v^ff)(0i_%50hyWCG}oYHYv=|?*dw!sxk-w?!OXKw!u=tR2-4Tg)9Yg?IqfEh zI-&vsW_^nc+++ zIL2&qE{|9V1vh2!Q-!gT;>A0=X=9D8Xj+v@liHs0Xal*ox%v#L-l7%sq3G=_W;StQ z)z0(HI}sc8qVdD=2V7!dGakVYVAP!t2(b}3hkT54rqvnNUPRk+Lu)-Dfs-Q6ZQhG- z*B=>#Cf|1tj$kVX)22AYpA>2EZh4YQOKTfmcEX3CpF-;R#?_D_?AmIBk zm7j?J!g-2f$-QCkLyIXPvnMFHNUTURNi4NXGBYx1CraIGm8^9N41&}6f`utVDVWw% zB5dAA!D)Y~K5bAJR-p)&W~Hb|FY$JFk>*&&a8pDe(0grqR;RmExdqp6jV{L7SmaIR zF>bP4I99+Tq7og&QWVBIN7x?~-SoV4fZ6xH+!FKBxvP`-B?Lp&JTui0G$#k1W2@4b zmzAbX4Q%QywutZog3~o#Db5Mr@MXCKR#4DgBr?e1m1*+`ZloHaoZKe19A~5*Eqg6G zZN7qSJ@|7TlxYwZlBQ0+Y1U%1geD`=&3KJ|3+yjRBP#{A-z4mHd%x+g(quBX_Q1@> zzyT!vL0^*&FrK4!k4Ram2HM}XMh0o^!!*R4FrS8m{96-a(&Hk;948ED0q3m`vFG1- zunk%Y9eoM^_fs;`g4L=vHW6pA$tzdS#zWwLEth!I1U8V5S%)&^ zaR;SM^TI}Zwd53khU|}m>~}?3l=!FN?+$3qGA)Y1Xknv?rr4eoYardNep|QU8G=XX zvHAhu#%YPKWJ9+AQpfyv2Y1W5=J zPGN!BeUzLCR5hL~@T}Pu)DL-M%axd~Jzl)un>mRGQ&NtKjaPbR5-l2t5ttYM+OGIF zKib=*cLvSUmmZ(2qy;~Kr2mZQ$FCg9_R4fWq^!aboS7!OBymwnS_^2R$F}(HAo@@J zCAUG&9!N?;LfP0L-w0`zXXdZZaM+#r9hvZ5!VwJ?zcOf9(0OiOT1C&aT-Q_Rw5{GG z@%;IQ{%2QqjF0gZN!8Fxi#3;FVQ{)=X_%t;Ty|Z}oMtB14}F%QvV$Yg1ks&1DmDs? zF1C^9l%d(Zin>d=g3v8tnj{qyitYdKYZwL`%AxCt9ubJCgBLHMB#92jE<8-Xb^|INdq>=vZ36!# zn#j|N7J%psJi*-MB126oa;xI}q9WNM2bRCMc|D}|)ak#6B(vOa_~`Nyq;j^4hqXJ^${N}5V%-j-8mCnz!-14PKIB$ubo~UV>tM!<2 zh#T{2YP5YzYiyY7vk#-{S^dI+mGv}hPn6F6mrfc1iS?P64(+hw{MK~GvGUX=r+eO4 z2RDS54S|Gy)t`Ovf?xbUp8t!6lsEvBIM92>p zhu9p~!03y-;k*V-eXTqV}rm{{OKedW)nZxGHEuD=5S6-vRGIJR6h{XPZiVJzazBzil0{h@V2{JALIgm}{A0zJ!0`gP-Au?&kXL z^Lca0oVl_b4t4@(uw*E$9pB+!3zLc|#vZaDy>lsi6GK3OH7?P1WGtye+ka101k&1Iu*Cf_8{a~sig zXFXz3Hdm_CZ}@(Dns11nMIHE6P=O1=f?Og=y#IdnrhJ&f#$c)7Mz?j#JZ+rkq_ZhV zn%wSf+UTpbitbOWmhc<BN}}JUr0hK5|P*hYr%o2Ga+I>hUKdE zvMgdTaxIFF7@1wQnl<*sWJd4N{=*>6r!hQV7KyYDE@#0%LIk_@X~Y43VI``n#}t*d z)*8r3RxS93XKK2+X-@En!o{VH9~z!Nb$e2ET!@9t)NY{D{MEIoC&nV=+=D&jSwAZz z3c01(;H=+(_<{+`CVx^TCV}OL*EzFUsJYfE`0^_Kx^!PcG>65=teCB|=Q(gYFii!` z@}j;v?_Y+%*}|B(b5-jFJgTR82+cBj!#r4jy$~%qhL+GFt%K{bk+e6!8w2>bN3r>~GR= z?AQGe*0xat-Y=Lo4?Y}X;G;!d^g)Tny&R+R3*yMUx{mlG*i-z3C=-kjs0;f#rpl*A3`v@s zSKhoF5BqMx2hT`WU{)Qy1^gVaoGCIXiW^vAo#V zMHHzW-1Tl<0-h0iT$x+2-%w1Qe|8FoaMDpC5P~Ctel4s=^ty?ueX7V+LosRXhZfWG zkxO+V4#bXbtEq1H!#DZXEvDmSmZ|DVR8em-m0*gLTmfI}bMOG=F9EOBJlWKRix^`D zd37w#4L%zEk;G$uC^y8qV_wSsdCGw5Z6DvMD`TaYMOX>YRQ}6R+=6n}^j@Fn}s@<8xFj%U|w)va!g1K;UOmS zJz+3|vTHdBF)$5ZrdofCK@SDdk;;=$p2~0fXwFLYtos(iZKmVM*WmO|9zD%}0$42H zpPd?*^ofymay;!ajrDhV262JH^nx&VFajmk+}+K5a>13 z@NxG>f{;(BY=1&#OUdIKxXVDMUX<&=?ae2T5#Pm^mfORq9+aLq5L_iP2IKM>J^i`c z*tibUjtwrDd6Km$k_9U(nf!JyVm62ZX3|8EZ97H%p=1XA1zz7%PyZW9svFgw`6tlk z!G5R+_C66WbH!;Z)xKg^BT)=eU7Ren{*=>aLvDwY3QH{<0*TF{@#iCMtxKf)4N#$k z9w)JcQe87%->H}EKF%{uO7~PZ51eTJ0JGgGu^2e_2`}-NY5TEDwtNL8PI+e0J!lQ7 z(PT~famUw*ES0tnDIwA3SRYhomnQT@S%AZhw(CMKte3X07lqKWjOPGe-OxK>oggxV zD609OQ#?x0t+E5uq8^`M8I^grtQWgoAxB1jc%YB(A20FyFWw{Yd@rwJ81Of}G6D)3 zH~9MgER9G4f+xit`EK4P?Qh+vM?-1R!k<|RZLR_@JZ6?f$Mu<7+I=XQ!yn`-M2gg4 z+G1toLE92w*!+C8={v^~iJpGk%#a(mWf^b)9A6}K0^{JJljcDMlwQTju!HGA#A%L_ z3T?YdHN@ap1QR6qdo{EM=WL~!Ku6=1H$Bm9;=F`ncDSmFp|s(GNMaLejM>#4f>e1J zrK+lM24GpTpuFY_3uD!%>+UE;4&2k%--7Sh6V$%Cm63bB;w%1M6~hN>GNC=j8+!Ta z9l`?J8R{w`@=Wa5u?&-%mD(~(S|NW@lf3yBbt%a=2~m$MjMt+#xr`##imvFx!0E2e zSsR%#MQL;c*L1L(@&|o%*~HhZC-NPQBq5$gj6}g>tvO%*5IDXhUPe6NLu4jKqCw^x z^nr)?rr-$o+Onow{&xScg_klbFgRZ*D6QQZin}aWWpR3roDva4gbaVqS?YIEBjS=Y zhCEcf%z1E$2gtekNy=S&JlEDc#>t2n7c3UM zG!v3bXO@6IY9J}(`l0Om4HP#GB?w2)dqYm5o9@UoV`@MmAS1p8L?N0$d%iI_VA!_< zF6!PfT!HFyv*s;l$&+hvFwH=NXQ}?=GDYqSr(c=PHZ$p@fK)yBMepu?Z?>am91?2e zSiNGn*UV3wRZTItG;)qGB48(Ya_|kKq3GG8CNenFz(ew{xTz&W5chOMn5veaie`+o z*8+UTVTs5Q3HvV6Vasas@U&{94zLwUGz8d)my!01DsIckU7J$@g z0)VCONJBYJLoQR=yrH(O`IkQ-uOuHg)2eC-^n|&p^k61I!xK6QTWNUR!{(rMD3)Oo zx6FRghquglECc#098dCWCh3?k)UPufGhH z@Yfr*=e@{Uwk_$sHvgsJL{(>o0?Zejq}i@yK7GkfIl2dD(sm6|ZJ$*c&xSF2-lL^V zbq`YLN2nWda3c|QAt3ziRY%v5T>*zUEEo~lKXWO%IjkahrkM=XCS)ZYiENx>xzRCp z_oaReSLYoC3PYTz)&)zkx>4kb;te<{A+Hu5K8u5jwj!D_xSbVV4=N3VyO=ZcjPOxA z^GTe2>du-_eBxjVqk#w3H8n`d)zUioW_r^DyULXc{DdOVOTQu#SExGzD}_~Jw(M(x z;24B+B4|S|IURaYFRH@}{hs=0Q6x8}11betspXoz^RKgd#GAOpm(fIts#z4BAuPU& z(KXW~Sn))de;wor^fnkB6u+H(ZVN3uZwOJ>(2oM!ZjeGig?`}$FBqX2zbZ(*hMCjb zF;4#4N<&;vYCC>3pOz&B&@s5Z=A*Q=U*CF7Evu-H# zK7~HW*%P=RvKT}~WYdOQy>9E|?t&D0wE8IcMmdYd%U=uN^XpUf7v1T%Qk8sLz}j*; zy{~6rb#95gu zT$6ajQZdqxpZ!EoJMur3C!8u(jUvSuan1X7V({5EhJeP29jtLecBAdEpb!&s83E(R z*Z0c2%x;p2yE5fSHDf}?p82L2u78{pxS`F>`0TSAdgsv^rDow_(-(!+C10oBa{(a`Q4{s-uK;jNyUq1gDh@t$iZ{u0+h#zCnsy zqn%+~oi`J|o|%(yK<(FySdu3uYQyJD*cA(%YnXM3p%*AVy9!3~XHQIdJr)9Z6ED8k z`oLJ(&ZiEiBwk9sSpj(5Yw@_o3~=|V9a0OsD&W{54_oGsWdl6z(XtLzP;D63kcfg5 zIcq`+=Yj%CH^gfpS?ks{_jLwlP|yyURHJ=CBwWAwGzK74!keHm8=w5h*6DA&m3-f< zRWQ(UmHCkOJI$Fv_^g+`4OA8yL^hVv_ZA+WQQchEV~$wm_XylT>UWGlYSWtP0U_To z%AniLfYd^UZHe(jhxZ!q0{{R5aNy(FLzChMM8am!(GoA zTrw=WNzR2Td~1A`j3k`RaR~jwsDhWmj404c@!c}Z86z&Ph;6(9%QrKcW$Th`3#qUxfw^r-mgv> zHrL1E52xc+&H1BR&y|y+EHi)9W80PVu6w;$U|KA>$jTyu6<-HIe{;%h5c}KcFnbXy$Xc``1qJ zsm88*);B^BUu=);UM@LfV)#~#&e87+HrDe_NyC@Q{-hS?9<83RmfF4o-}~wWU6JC2 zue`E|oC20zT1=w8YgzD$l_9=1dzj5C7< z%7!NvQe+QZjG|)?+nI0?K<2L~_+*ZDk7+v85@XX)hI_paJ(Q<6C7S&ukj*|dlAnjx z^ty2p>jE_|Vr#8Vm!Rhxo`+oK$|Am#y8qVH!=iA=sPw}%iR9A$G;;`9f=nw`hoBk) z?bZhP*Z}S->dr;$IH8=r;ey@&$;&)omll*ijlJGR|E3A@L*Nc?9gn*Mb60PW=y5ex z3g}k)@pXP6u%oC+)o6S9H4r-@iVlHX?PuRf!O$H6Vg*O7uvrW@P z9Tr&OAEJ7?<-R*QQUxFFDX=xoM~M%45HY> zh>@y)|N4IrB!i}~zN6FX&HMl1hyQV%oN+wN7JhS_`{~U0HGh+NG-A(f8H=e%*79 zesDJt&buk@|8!wE40Y-b+0Xe0E+#;Y0Yn>2Gl~89#}5ZL*%x6eMI(6jQdczQA0B&j z+Yi{|+@pX0@Q;rZ78#I8SQ1754^K%3hQg__zdrq^Z~;qZ*yF4=_GSI2r?dxqoL(s3 z{Qv1f1=!=vd$w=(kB<`uZ%rzU-u+J#(SO`vfSx>fCs<$VQYrjT6U%=L7lpy`z`l416u#n1qQGsRA>9nDIf?+({j0g? zHGU{L3w$;NtcOztV6?g%@B--1U9_=RyaKvP_q8K{{&$H-`TyT}2&6mU54wUlsa${{ zv5QPxtq~(-YiJV2n;nu+kfN zI&tIp8-Kb_6RgA12vBayl<@MaAGpa02(~O%78N(D6H>;%kNPg$zo8D#%Nw&X!fKH` zqq{YtU9byQ0M_`hT%&|>02c7$Cr~|J_3Q!>iUL@o%t2Jb&ro@re1nBX+VK9zf5n9T zS7buQzy0_dG(~NJSN-nnh~EXoMBI=aEahBfp#5am2<~g3*$((;Sp&0wmS<3f^@eqZzbeUl(nR$Vb%d>!6t98jn0LcCi*PE zvi$7oS3|Au&u3JCyvD8&{pUJ%SXU6HAQOep-*z{({7qtc{Z@kx3AC5-Y=8%^UI|1~ zRxq4v-UY1EU8Tu95QL9^WlaCgw+r)(+y$``EbCdLfKpjb{TD>pj2SWj1R|=?Spb2I zgCyWN?uP~E*e7EZJ(TZWcp-Ui!T1b{%|j4$kXqWfS9DTYH7rjzcEI=&v$0PJz#xfYkyFhLdvC*?AyQ9zFC)7jb?*j`3<@#$-3(%M| zz`~g+-T^R4mc9<`(*+G&hIp)NrnFgZl~p|ZHToYe0Rcnbs|C3w6T9e(-US)x5(qHpeCbd+8OH?L(X-((klJ?zNL;7R(OV zf^$F=Wzzj&II7Bun*=-Dwujs}NtG6)wYeVTdw?ml0QV2eIu1mjS&Mal}8#e&# zoc$cc7S6^t0Cpx&YLu+%uoBMq44Z8p1bXIyUKfMhBdn)Y;3dkddld7wf_GQrIe(1H zlA^`!!%je@mrU_`1cN06U0N<1-GcRu;e;VXYV7^AIEnh3m-hXkI z+c!dMvwo6){B6<=z@y*M3uKkb!pB|xy>l1(Rq3T@WQeL|9^EXca|C_kbf2;_coBm< zlzA6jC)T1RF;SpHnxu%oA$raopB})znjI#k)Q=4~;Aa8q_KyjiDQQ*HR(hc(L91my zyOiZ&=Lg~7Wv}~isPsbBdqTEAh4Gb|HcnqU?7wg1j@_F9@;Q7y__ShWa4HZW>Wgzl@V91Fs4qyy&?c} z_j_@qa*^X|MKBH08zzi@!)#!{p3S|!lOm8$$Z*lnaw=My66NKW?e=v5R+NnQE)NL% zP(hmYzTjb3Vb<4hUgyxFSZ4=PNDUq%M^NkiunJz{k z7k=4I9`W1qNQ?s(h1s7EB-Of49OlmjmJymIQRKZ5htKfh#V~1u<;97~(!sPW-ORNE z<3V)oA{_6)hlK|<57z89Ta1?|`D@y({XIeNA$7HD^Oqiv`LRA2B|M2k8;m9lgKDfF z)lHj0tH|o}`uhUppH0Jp`}n|7cqC9q!0w@gnjK-Ca7cU}#PrHB0HTmuBwJKe8l#pPdKe8NLmN-6nPr`0kjjZDV2`7y-B zveoq`H_QqJT27~uk2^V#q3)p^Zv)PE8&MWleOzuH&uMll1gH;J%OwhWI8dMO|+-jJyyz zEB0P{P?R^*s!4me0r59M2dEi&RUP6Pe!zATgY=pbt4ID4a@HnJE_Q>?#Pp&E)-BzQQEHh)^&nslUyj3 zi0KHXcSg%2QY^Tw7JjvP2_Q*BCL?D)j!#FXz1jK#vT1D%=qRdFdXd&%Z3;)fpW(~o z)s50h8%)&vNrzP{jk&YFd7lmI?fbD~M409RoCgqReA1o!nbpGDGp7H>BvxLSZXe6H zb5)N?kZ`xUh>&kCC2Kk>#c#&NI&Bek!X3!&_SLnySaOxngs;O`b)C&&clK5h*3MK` z^ofV7)0f>xb`T!lMZSBS1GIBN!o8=M)Ya#}1gb37dd&=a--Jfy@I`!eeo`jxl6T~i zV}$P9XOcJNBHk*Y)jPR0$@>;CP5AxJOBqehJZe+W4Sc>yf*XCPI&4ixD$3`p?~!3s zkV~ucW3=yKjqvTj3&Pwjw1Eix(OtIEyYal*n62~vR7X^H$E<^NZ7}5y3Hs>MpR7)= zzh%UsPr_KQp5zE`@~%I+yUJmAe=jfXsiYmN(kWyAWNC~uYuHG^avc3=&9JWCIAJ0V zf&`WEym~0>{0#5C`H^31*>u9e4_^W^?8!8?-eoArRFXF0US3v;z@zQQVd{w#MlTu2Cwk$hV#iSgZJ$hylx z6Q*R=#g81m_HpTC1+uQHsdeF_NY3Pf&aUlo@rLMnOfDs-%X@(FFKksw2Q7>$kXDb3!QvQN+i;Fe(jUr+R2ybw3TFYYGlriI-$({c-UyjgEsNdDiaPm9mh1dcZHZO3Nf zkFZev8(wo9Bv*vp=3L#0O2C=XcSWY@ND^M!USbOwb5fT3HhgSG&(O%rqLd#;*!jH6 zUKq!~`%#Xh^2?5~ThbeSWnv?BnF^YLC_C$?E+?QM;3ucLq;EKz6BU35^0)I(qHXcd zmgF+H&E4LJH3(%qvbenrSG#?&)iAAnDnfte#=8{0%~^OhH%;|KY*^eV9xt0)SHM*9 zcrAj(%{Nd?X1R=U^xTvD+*AFr_#2eM zMPmdrFEjapjV;kEXFXE(-)blA!<-UtPV;Mv2I3m*cx~5}%_j5@IwI#zcy=SJ1Z`q^uibKsV8*SWUHR}nD?*V9-Dgxni+_yF+sWhw9ImMR^gEOGI`@^5RW-}7Y-CGVr5LCu&)oZ z7eCuO35DJPgMLdUk#{M6=SP+H%dU=h!0&Vy zLr?qY-tRH(ej>Ki_O2+pzNO~Y)~ofiFQv&V`R{|rb~o)OB3aKVuw5+MI%isMxvGy& z-y65-K~Ih%;vh-7$8WCY8e2W!G>5RXKg_y0`?7uJ_Q=PKA?zU|+n%jBeczicQhryk z_r)qM{Nu_zxsftYZ_*~)p0==4&PcF-fAkb0?rM9h1~o!Z<`Q)F-U69oV|#;WIY!r9ry($gdrDMW(DV4ZZhiT@)cslK6m0ieF+@_yM27#hQ0&ps zk=>&(*_ib>>|=k@SqUjDy&m!6I>2-OE|y9|*vW|Ln<}EIlu7hj+0eu99kZie+GTPp zThI+)pM8n$#QS5B>wRZ`V%i^c`ZHwH!yk@Iu?Lbch8vcEs=?~}=bN)jsCztXFxB&oT%G!I z_Vg7jZ-@MyTATHIEG+!I2LmGkNy4lYb+-(=_^7m`lVh&oyAxYIzXs7GdGVBx9+6P1 zIk*9$j13FjW4mzwB@RdV6NE-mo4b@b;z}l+IkpbnF~4Ig6WjZ$E_um)7Gm+={gRPU z`z0ft{J-2Sf=-5KL^A4XM=s03CdsbSXjd^Kcw5+)8qY*qW7ehrzB^0TcV0nooI6y! zApmEV+j`%8Wmf4_=qS3}`!fzEzuz;a+j=nwoRior!}3|D-!r-ci^Y}SWT7p5)tgR~ z7>aM9*X_vEs1!q|{5R0`KvtrAackr=_#Mih8K;6N z{B!Bz?6mx&$N8@02PjR#-Cn`u2O=Y)b?-F;{YeZDGvlw_Mo9@D?7B(i(kIlq`@>bV z4WMf+xjr64YKKF7MZ`YNaY7n9oBd+L#oHw8s}$~jde`q+Q7ATAT>BX1)2S5MXM_0L&7zL`(A`K-35 z_g2iW!tSE4-tp~CxgS%+BdgMXH5rbRvPwDp;0>T;>mw1R9IQ|F+0~DFNu^=)Ace_J&-TvT);~7ycSlQK5$-1+;7RkMnriQ4?p*Y< zAR-w^WOsiGy&no=99aHA`DZD*2Bb&QS4~b&(24SId+uULx)n1ul2$g;yuf58-hO#L z-$M+YSje^WI|yili-~LG@Ts8L2W^9&{Q}x^#&;Z@aPrn&l=W9PhA0OZF``k^$bUJ! zi>nNthR7&$%<1_;B3v5GY@0C3Jul{vFYRVJjJb+_H!wBD8~?6(HCe_M+r(G41I12` zV+&=SVLI>I+Hkm?Mhj&FV1hzRwnnvcGuMQ)4v@si0 zgw>j_c)J`f6)H6-({j%bJcg|UTBlB8)njjLBG;R&CZ?k;4tTCFBFXJCj`Be<3L7xQ z7!llf%c;LVP{)%S4n`AP;w3`sMvgy?jSub>7)4k@9Pf<(qVkcG_{?@t8z$f`ziY-# zI8`rYigd|b zjLE{PD~{jkVk2*a)iykAxaHn4E>#2^mz&q3faO!Ef9^`TDL*P|=14f8)VAqD%W}cq zx%xDcbMVRYIQmcPp9HLLKQ6R#UqbtM=>d|zBTfYg!ZM?Dr5$WnEbMI{o`I0k&?kJ( ztNgk9|J7qnnjiX4TBF$3BQ}*j)E+HuIWS9Ve^1>$qnh6C@;Y>HUZ((~=GQQjyGUx$ zPbvt7%mHy3H_6nJ>Gq)Sg=w!TPuL=UMj-(yap~Xu8rLq=^lFdwi63~c{Yxjq$PO{e ze4C6~3lP?k`cazWj~Qgz`(isLot8K>w`&dvGB>J}@0wiZO4aG4$mBI%h!iLO6|cFw zA&{~=**6l84CH1sd_qO$tH>!Y8F_`o#AfAB`HtBN_O z>^|&N=P-%0~$K&|7(1bF$z0BW2)HefIF0 ztZu>TPPm%Wyqm>1;=7SRze~mkf1mm9rg-C&jm(d~rC@i*TI&9sM zp|Sj^&~{bBY0k-(f2iI=qA~B$)lYw++=VmTUq9SFQZ}dn)O`;2=XQk1|23HW8vk=J zZPm}7c?H1|c|H(*+d95(8cR1p=73i5rl|Cz7D&2cRJ5}29M)E__T;CsrBKZajmk}E z2k;NJ4Bm47dkXR_qv;nzXoT=d*?t83BJ-Llf5*D~+wk*WONQb`!6sBfQXeAiCYu*O z=_Ka5YS^y(+-04T6r7p%o@D*qNp+F9lWMA9kd`?l&#}E^AIw*k#c}A(rp@j8+!cb#64C6zW^G8w<4y9NGGSS)1xVn@5|UFqq9s69N1THZ9BDZSlTm2DnB1xFw?1FSc> zIF+Z}m%%a5j;yOn-!)!Uff7+raClWG>Se!fI7VWWU|we4v0I=9D3^EzG8Ol`@MV=o z>UDgkTg-j+uk#{Fau_yiME(zZZyi-twDt>&pwb;%LPBW-Hb^%}ODiE=0@4lA-Jx`Y zfQTS1<)&fNA>Aq6-EikV?>+b4@s9J|bH4xY7<(}IW3yOmt~sAMpXd37Yc8Y$hBR8z zTQD}m|E&)lrYv`ekyJ8@Dxw_Z8Y2k%7 zuuWBW&x<;P#*_*4ddOvd(KMLlaTs6uSzY1FE((#KyWWyKrtj&w=R+k9$0^!`x598V z$jAs82s~Jq-+gwmnbDsx)0)=0JehB8mmduCQF$Ctlf^`6P79b8dII|K}8n19L_2A0JU>vo52oiJ9J}KR=Cw zRYX9>=@=!5qkf+4FnUJ&D`L(-1e)vV*UGsZJ+0~dM|C((+?&vQD-q_j9oqV; zmP)2Op;YIha^>{*GXszcMNL0Ta?~Pb3+ZeeTEDED4EME^u^_KBKN4fb8}umdqE5&4 zKWE*zh$C0+O|Zf%&%TJNirJd;1v}WKbsQ# ztOTb*D2+HnB3P_MHWf&Se7<}DAbP9_mmdM02l*rt#6HMQR{Md(_;W_w zlPu$uzQ64r4=+8AGHU`?-Y_bLbf@9G8(*7~5iAYt&)(}?6935pjy+O!HQeDgnj^_y zL`e7I4Th=){9Vsgd>lFUZc^>b6_GZh9<>#BeI@tOoZuYbeLh>FG0+=9uYpYLe-0Zv zuP%z-u>x_zTzxODzDmSv&Z-R_BL0_lQ}2HrSOdkYD-M`{0_piZ{h_@KiNp%( z2%;ePx4n$m1L(fi@W^@EVoM~wQ=Hfw8M0?6H?wxH^vocCj79HEm9SS{Kz;cS9yczb z0Po^VvKqy0Z?nsK?mGF?xZ>i1y=AL+^z(J$LjiQcs;~FF!t#FEvpWB_=`&g%TX#Ym z9ZTUg1LqEv{|W-4YFd%g8kb$Dg=LP3oF3=u6JxzDFyWBri`Tj$QnDHSaEfw=*m6oQ zZZC<9$y#>d#=Yd#>e7E6HfBGL4#O6}8YU9ALldKnf2}zd+-$zb>Gjck|6>=ceg!Kcd#Y@gNBX%^(|p zI`z8@f)#d?41kS)_pGd#I}Gdl)2a?O<#kQU>cS#WN8U9o%5 zLp{AUzeg=N&nlXt>^T#f@*95^mBWuA;MD!wa23uJrxnj?<=JPl$J|%|7C*z_3IoWA zWrK~S6`@I$PE8=u}~0y{2Q7)E=YJ#&CA0@yF}amLh90>qVF8mP(W; zuu6rw9u4wOKgo?H;)SlArDUd^1Tqu0{(Ce)vUF4(P(OC2?-i)w9#DE(A@#=x91XP4 z*h$X$#XV{&YK+Mp`=E)QAG5&w?rM07ypilBL-Cu_= znV|C1@z~WTBE?@qgGh>}92}1(YQWP5RZ|G2P)9uJD>b3nXJ~79GQu9rqVK9ni{E>> zo~4g%@$G;%%Xm*rFF>CzPgct>(*G9 z^+oKVZX{QLG;82`+r_mPEUB-$@+&Q^TZblT=2-3SDunP^mC(%}eTDba6^A)2bp;5= z3KjL4f2Ol0#BlXz<#^Gz#ZFbXhbFDjy5Yerdvi6$nAK{_87g+TN6Q-MS_}qFS^I)G zjW16Ik#C9MS@j~!Wb@RfoAvzP+ucbgY;B#a-*2u^Xq&e=-jZ53M1>e2Dai^vbe4O# z5OQQQUW>Di9x3TnYc5h^CbF9|Hu&}}#{R*whNFG43HLw3MCX4G6Mxn<=N3~6*pB>- z=9)ilp;M zVe{y7Pi~iZ^LZo^eV#nZ;G{N4_RVE58(xdjv{o07I{wz+7Y;KTGA#2`jUM7=SvZCh z4Y$MV3_*!6iE)~bjagT@wRzt$>w32NLDH>v>kQpn%JuX8#UnYaNb~W@oT28pB-61% z5q-mkhv*UylZLaqnU+-rir ztXP#?{kafGUSk2FJ=~bjyP9h}Z!!{y6dqrq zK{KJn-}3cM7w!IjmR2|I3c+Rv%>1H{$rt@W7I*z z8I$PKiEm2FbvZF#V>)tLGC6x_^kjdJVri8hMOC~O>ugRQ1c-85U!0stm>Uio@9;YN z<81HGiDP5liK=d?Iwi1}%!yc2tev$X4i z{7Oh0C3P!u&^oSSykOPipM{q8>X^px@&=*(NGw(W9*U(Nh#JY;mG5Uh=9|bAMqS)l zzm{ul;*|cTw|L`+-7Ju*tbXYBc>NW#gz!_0l?}Nx#ikuB`p6Lblf*nkO^=)l(WB?- z*VA4P5luRT-^`93^%kBA6%uLbJ^AyFC!N#8U5Im&1a=NYJm__b1f4cTpL zi8puMkg&@%YqUB2{A$GouH~0bB-D){rG14>O;^qVwMDjh@Ph>58f0B>zt2MCFKN;I z5!Uz8#y;Y!XN~INcWI%IRJ)QSx^Lt4)C@s9(h@w$!J(KOj8Vl?i%)oI1&o7NA{2P$ zUU}Yw{;~27z-lddZ0YkcvI3bjB^LgYSI>y@Iq0p}7T&#!Wu-NJ#N2~qH`(DezpZzU zzQAX;8I{g&@~8TUnkf@d5JGlM=_yOzkqQv!8wU*T0|n8ZuR2=z!@v63!g|ZpBnzEz zI`i)mdT%3C3&~s(vN z`PBPowoM}PfS*@>>n{c(oDLv&%TYFWzgQo?kB|#*;`21S7JEYR*3D(KaF|mh-8DCg z8-4ReC;=X|%s1$+r~-ec>mL37C|yI`~Gx9l2P zE0>mzgYimyQ>dK^G)?(S#cnq%lv({`rwL}U4$pNjpT%!wZZ!_ECWH^g=A}mx<>S3d zfxQ(ES?}H=+mvCe%Y0zn3<$8~-UqXZ@b+YdD;vqiW>+XtA9tZ|$Wt^Noa^6uFDuN}ap!Eo9{0B9)O zT8X7;(4SDB(CWDb%de=H1IQDhsT)sI(N!Zb^CZqk5;(!gr!JTlX48idAV8%3)yA_= ztX&I}*$~)C$(nbhC=tWZZ)a}UMv>i|wXNvPEd0`QkGEsM60KjM|7DBNRFqp!u;YU} zAauT0@-65g8poJiXuH^64zmS36gBDRGnL(reGm>}I1$1mzKxfAe%*&N#z43Q@;s6I znEuKS+!l-Q+a|MEQMHUiEmn%Y@DRbj)P`zx@3|uHZ|^St*E$VGbi#K}?}7gO_`uwr zU5Ef-zJDV?3UVE#jAfSNG9=g=&dc*npgHB z{<`vO>C;2&%ueA<`qQQ#_Q!9sd)8SQvSyuq)k6HQsQ)-7WBw%+B21OHbZJn{K0Kf-87 zDA!-66!(-@(Pz4S6`5;xwf#k#MW-;o;bJ3vNOk0|@dkDk?gZ0;dDi~$yey-)KDM0T z10vfbL9=ZD;XtbsO>L6le!qIIO0=`l{9rNeVXJ*&LH*#x4d_(5zxZpmb?MJm^qdjr zXqKCtVb$_n=67pPF#qAwXI^Y22LKM+sEjn#NlJsLc)JuU$!nX<7~3jlMen=WS8y=D z^Fe#p89r|jW@dc8B2ws)f5;SQXwhRfv+gmu{)dzFn?m)9mOb(L+$`rVHZxS!Xwz}O zeG#JlLu9bX^kBKwv3COLI{Jy08b$V;AKocEx>89!sp6xTX8G~xsa9L&@6r`Hv!JcU zm4gWD60uujE8G~ei+)W*LT~Gmrne)LmEE4y^i_@!%NfuEpZ0g7{rT%@pKj{B;jAaU z926aCO}7|R35ey-M2-Lg+Tm%aUL1esU9!FE$l>8-;>v#YpfM!^61b|_mIxPeeQL`R zC7_q;YWV3{E9y5B&D$r&=x`E7A)e0yLjj$~Qfo9Pj?OAns1cTUW>z;w6Dgw}$sW3& zSNlKaR|pkTF$ioebJx@<9B9_f?b@uz_4T=`zb;$$6I?j?#)IYFaGbHPGrJ+hL+bIY zw?$P2TQdUTRZzUKxn(MNhABnD@C2j9lH7J)U~W^f!Q(U$$3}&rR8r%zqq>vT~!vzP3&@YSicjFe7>f3J@GzMhXP!<%WGHs#%q4L+*7?6=iR6P^!a^&QiVd}`6VNia&;8Gm&tgg6h> zWH}6Jy%t4*OJ^6|!_#JBDa2t%_=Mr*_gt6EKcdG*4hhX-2)ln>}VH`YMB)p9%-|4^e%3H5# zs2=`{54-(C4ig3PI@Vfp)J6Bzg*a8#DV9ATXd3j6u()hlh?)*k$L=Bv?htX45~9{e zZnay*|KdX6>g;HUYx!4{l(V44AC|X89F){9hwt3(1N5}yMK|5jb;ZN4X++?PevJ6a zX@Zg$^;M{$zW?HbDJvRWOx?1{j`-*Q=HqvWA4_cs^K^Fkci%+(V`{XnSgsem6mUyemF=_18X$0qoSf?TcHKd=XSZZ^PhZK0`anC?D*dxruV=37{E}NB*p>mLtvhH^FO*ejUBQd zFk)GYVgb1dhA;-{@*Q!{=ZTFKPJro&xw(YCNYj+of&~1)z0-i+@+EAQV3w%!}afu zk~)A%2Z9V&4bjdF<~u~#lP@EPP~4LS0Y-q!^>mGp)3pzgvfdnoh_KkWRHhNdc+9SR zVD&l9%t4SmmWXXUSIBtgdl0PmWx#R4+`L=&xhtpzr<@75zC8Tcni%hs+>R(4r`%<& z!B6V;z1s5*s!7jgoEg#(ezYB}t>X3z*Yh>CGjn^~)s~y>w|JW2yLLWKQX(VW^jSJL zfUm{EJ!{u?z_a!Aon9_JxnN_j+s6Z$TYSGuy6k$9h1G)Dh~_+0_L6(x=A>4Y;-SE# zR3DAn$LXhXy})`E>%Nu;i||G5BlZKCntn$IiM|f7XQAytxVS8rxHecG2GR=}=DiuH zX!yDZ{|4~Whax~VG@L+@H|GN3t8jB#>Vp;Jdvy9a#?$1mEJUc7b&z1;9O3Z9i^$na zJy)4K09US(sR_re896+PXP)=(<#vcW zFV@{L>%Lp3KzMG9aD8pZOB)qWz&|6e%>eAUZx@$?K6=GKh^E6a^Eh33{W1RN4KMJ& z3P;#xwaEC2IgCes;nJeCAQ@yj@2|WHpuEYhMfCM9pIr@w5CevxY&np{2yPmcl(`=* z5a!z92SB#)Cx91GXread0$SRHh)Qbh83H2@TYSWpEGK*g+dyQw%VId6lzCTyV2+v< z;P*ro+ylx#UQa?GVv0n78Sv~^XLfsyixsdOf0+r)7skKovkIi(BH$}xe}>jF-C5a- zspQKkY}&HrNw0)8{ZkUkA`L*b1w0t|eEwSnBSd0l9^B zBH*fa9RbW3JUS4+{o~jS@fq^|gU!Xb4dEGP^8)jZZ9wGyX7NRfW@Lr;jxi+}xz?S> zuQl67-RGzG*FN{Qbk`R^@3=NQ!shDc+H8@gJk#d6?04`M ziQ(u6#xK&1lF6umBb0nOoYc|;2SJ3SfIl16UK1Ymf~!<%Dcss_19>a?#x_;yo?1R+!2dWRr8 z#&04Do&Nh@x^d0b))M2Yy&jrItXRj*9!9&e09H=HHjD1xzXu>S2u$?eY_>e^dvo!V^nATRmpc4e;cJ+*$2NIb;mYoC`SQW7S z*l{Jj1L;2#q(!C*&OdeC1iPYZ0Xa#`L1^uuEC-zJWxTsJ-6@edf0g0 zz&1RJPtTnn!O z-?szQvf{ZOr#lVHb^yL$hJYXgt@*+}sB@T%Tu?A%KyZ@PaIUX7K3#2}9A+f+Gc?c0 z?d16lUn43a=;ROw|3gb)cgLo#WtyOaZ{6hgKngj22!R`9rYlqEeP?t}m@{4q zqp?A)2EcZ#n9j?MKxpOZAgbXxDCJZ<%#U@(WjKdWDXJ~^h)WuFOalIsy1|@Oj%6_3 zQCP(;h`}eKiDq?1?4Q5-X#*|R2zszGkS{YQD8Y;pY`6{F*Iu|%|FedaosNiJ8ZwCi z)Da09#nt6RnzMp~OqtS8Xj)6&o+H<4ahqtFw;-%rmKqcNhixDd#E&kT zfVoKr-~>Z@@M#@c`xxLz^( zgl)Twf@J5yDPS56Ik~nHi9MFd_^><3cGhJOJXNN7#!X@_v_n~mV6*x&eM1pHMT!XvX)v* zBMMtVaWsw0()!|E(8JTf0Jo7rKOy@Mojn00+Z>|EmK zBP8YTdFdfJn*ICNOg)4J&=EHFees&+kAhdc@MZOC1_qU8`78RNUwyyN&|=*Cj9V)e zE;(%a(QX6#8yqYWXUEp%Tz zEo}?m3I){cYKOqD`$9)u&nw|&OvqO>4~^-XVW_v*cnYwt*j0BgO@`AOM$#UCCu za~Kl9UM=S*yOFGTwRk1tD*DR)=!U6d_TIz1QolxWgi2t*LDPm5^Gzy>X)F$2IXR{l zo5xhIcrE@Ud)K#2!|yGaN_UU?R+w zB}eDyY!z_Ap7z`|-ckrPU(1C+QM-NO(q>=Gq6GgyxXLXFJl+B^#w%Oj#1wS}ms8RB z!6E$1B@J|L7N0-fY#!wa`W4#?(EaCoo3ugc;vE@XEGWh9vIsL=9pEPSfhtZXQ|N|G z2*gTpkovKj3ctw`-LtX^cNJ0`{^EjCA0L(RXWQOr1soarZ2Ng5za*bed$ec<<OGdnNJhb zZY1m(VqW(&SI~E@wI!^nGIgt--eMkRhgJ@um4EIXXt@}U)CUF>S7sOqAK~sE(ycJk ziW|2_^c3mNZ(v8Ar8{M}o_B!x^!+P(^W)VRlh_B2gz;UdQuK6RAQSz@!lqqEx0W`- zet(w0UiFc8AE?VNEdyyn{fo@2kP*ri)_DBl9a>mUe}YMu9A4U>wVNtu$ZOQ|O%1}y z*)Lw;Ze73ZYaX3Ozk4#?Xdjs;gv80q-f4JB=t#0{<^Zq)PZh}Iw5QdRMUL*Q1KGPF zEMeq$5&Jl4Yk7y|S95Q&sAvbr3d8S#%u2djq2`WspGhJNr;~%Ul{iV6uIYf zpmiMgC_Q{SO8R_%b!Q)XnsHu>o7#;r(XtT`@{5E}&xxNqx!Mb~Y&PA?1p=%AcI5a5 zc<7cgaRaf9>>JY=kO2`8wtCt=*DI9&bgzTk+M>>?s>m>|PJt z+zw@FWSx5=+KW)C44gN;F_9UGSL3SNpP zl}t4sJa`~4q2U85lP+|~+8w1e-5}NJLo=bR2>q%(>d2fvW_!C7bYQ)xd1@~VpZ$p3?KkMY4(S(tMmd-~xK`4Kw0xjq|LJy5>c18Y zdr8$L)gaqoc^(Cb-)&Df9skTw@dY$atf;U?wkZq=Xw}a1uR`k;#NT)Fa|y%=u6G>s?QTG2%BHA*_>cz5hnEWndl?Bo*4Me11-?k5&W#C_)d@XTFIrBCk3{jfT} z`aI*>!uH>oC4pd0!qfTW*ZDaPRBvt6CWEk6ML11V%9VSlNpzi5F8*enr!5_fex9fUAUU$gwJ926FKV*tp= zTkPwnSivGBmtrlSf2>r8_Z6HkK(AEcsPpb1m9qIY!(8$sY7b+=bn(M8ZySxHA`RXz zzV#DT07?62yeucX{9Q2Mi2&HglYE=4uYd7qVLmo8Jbeu+--s2%&w25}J0bunPCouR zR73FcT=QZ@bpfiW^_}I6^Qpy=L1$SPr}Z#*Q{H-p%^NGnY{id4E31BjwZYtK&nl|=*rNLS3?t;>G<%+c z`(a=55;HmBZA292&Ie3C8i9!r_Dh3~mjNw}66mj|wr>kGhK;*&*`zez$^Dj~ENgj8 z#zLy;+K+n6Z_JuEfEM6F5?ltSMhzjls&nTyaeq=AkJnS#9cAg2He&K-^F6mbtx(ZC zLK295yvpMvpo!XaK>fTEE+rNFMk-1ggh6ra2hlTLBxtnZ4XcED^p>X$eLLrBL|&wF z@5`SDjm!ll~ikk#nUz-@*tC$T{Zrp=?vHbf!$=~G%0p$|) z`}&T->?LjKQ|c|b1*Gr9diiK+*fEjTYWu8V_F}=shB5-&*urW>=;RnE2Lq(~LszaI= zR*@Rh4<4}55x#*&68hR(9|!md&Zg*KGpj>)++cjDue(Q}Hi6~F8e^t&>#!k(KaPC{rd;po8?ox~cq2Xzyk48v7{hMc* z2dKfCECccHth$W+#r67*Gb;?<;p%`K#rAEcP&{&^jdPLq6s;W&)Ib?MO}mJ#Rx7-hW?2Y z%hI}C33i_cw7q6wa^A3>_`$)7LSYW{|-+lz(b7Mw1^tImSbKW9k8Bi zIgDiYGShly{mHQPF&T5+VYOkbxSQ%gOwP!F(tHTi#6JBe_mThwNdT3g;^*27G_q4R zTu2RdM1=CX>TYe8V0kNE0LJbEWJ;JByDtsjs`Dh@%Ka5UpBbA%WlJQ7@$7G z-J8V=8}gxvEL2`L9I6nG+dY{op7u{+M4FnHGB?vgGGm5pDw#Vv1{gQd40=eU8QDDp zMrxutFGatb@rJm!K%dZ~JIXZ|K3rwVv3S~Mu<R?l- z!PrIcyjXMa=X(=Re+|0%8-^J-B@)9<*w>et((MyN`&0DImEi<5yU%aEVWbfP2CU+w zSM1X+b#{3F)B;dk8#%f6q_5)h5<%Aq>_XzTlDVRO;IZ~s=A&eyev3VjRzi6c+dsh< z%+V@~AHa2N&lX>>Np6nQS2$jq~X(4cu-61?mUy`MvZOh{uIuh)o}lm+N&MA2S!+{&w@yT1$V-8>4?2 zWD{Wj{WZ#mAm1p7Uo@8La(Q0D5Q636(Bw8|4Cot5rI`<(tY}~jM1Dm8_gKl3VPjv}YWgN~R@ZDn^0rc1e&z;0ax%TG>T(4y>un~`XOl94U^T|#7ZeW`eg`39gu5J^*b{_^h;Kl4ljHO4-L zCB5opoA2NAGzWTaMYr|Tw{Bu&C!BVb3B5X{*6z^HToOE;f27MRlDN6|vTm5{t{AU8 z*zEZR(x@YBFJ>M0)4Z>OxVdfcZ@hMp7D1ZC}Ssl9#H3w!Vb+mJ9e@77;Xnj`M79G+~KXHSaGdOPL*b3;d0hQ2pJ;y(MI7!n|9F_A8fWkx17&hzHHH z7G7WWI*%*H4lZBVh3pnUK!bm{o zewWc%B5;e7-(-Gtxxq0uqCOwI)V7%Vu#7sfFvw&DL{Z2*a7sjkqo1|ZhPochz$LN^ z`&Nps%y{<*WITpt>4P?SJWGq7Z`iTRlvYoFe39@1^;H5EwlA-L3M;*Rcv-h}^pOey zEP%c~V0Lo2G_{$%+fAS5VWoKUS`~*$zg!q*63z%&!5Tr|VQ3fphh3x;uejS`sk@Pt zIrg_asE4Fg*Kz`s_b(O8z(E9b$dX;EWAo##Lr&CYy=+8F6!+)5m-!1)gMOUPIKr2F zz#8C04q+rsqNLW?$z zG&0#QQ0m)&HfLidPj=axx#AQ(q;MV*Jzd6b#=oi-?hMK{$q6J9Z6H5Ae>=QWt#V{J zFQ1<7YOqvCfuXlsE%eT1KDL92fwRO2VmxGY!^@WQaQ9Z4o2uL)T*G84@433$)y>o| zuP4h6ADcA{rSau|)-3>?m0_ZP$d(*>k(=FNA4XIg`LeGP z0~>N~m)sq8TIemZB1O)wS%}+tPe&p3Xj=90oKBKu74*}UW#6)^2=xJT-!g2uBP%?3 zabyYR51kiayu7_?O!lShY?qVYB8C3Nqx|Z0sP$N){e3N?{CdL5YqRVq=x6F)nZA4a z%$iY>$dM|azF(@=PhG>!n2pu;&6m&?eFM)~9Fu1GJU@BaN@vU3@2#t(VmGJcjYAGE*8%cP359E$0NSXVt3N$UyT!07w6B%c?QesR#NH~3fMK#iO4FgmC_ zlfgBz8qqxjC=15nl?KHO>Z1OHmq*Ad8GWK1_8&?^Pa}o-u@s4{_1*;p6y`@zNWF~V zy_noORypYh#h`K+R(=}Vf$v7pfleUNi`SSO;XwmgHQTPtZ=xgU65@tQW_uBm6W)?|UUjKI1xhpqWF8i zv%~2;?2KP{rKe(tviaLLU*ARAwu78JRgfhBah3;y?GL%`24vW zSX*iu^w>lv)i0h0ei-1JQ z(+Hoo&Dy0Ivt+Vbc$tl_P4X&~7+pAF8mNeOCWOCovo`KJkP`JN_8J6&3U5|nsgCG; z0tH?icGW%V1v2!vwMUjkUk=z`vC;Ns-JfLc9)9HKbXYa?LPr2E$jaX6FfPd9;JkI1cKI8_EH72>y%4XQN*0c!2vav4Ig8fu_BC1<3 z|9t2=di=vJU((Z-n>q`STJ-<)2)QF0c*d*=X`78k#4PcUtE}ZXiQa6Pd0!uR9R83w zd+zMKdPP`yzTknOogue695Vc(nm?S__9B?l7%da+#aa@7LA9<@wycE(!Qi*<)F9FD zwxDa5=ccpZgZG?M9K-q=$Z82B}S8nd=iNo z6Rbtic-^6LAN=ClI;G>>N^=@gy%pk*G|K!MOP~`egI0aPEUkp z3Gw{%MvX|BJX6(=ta@km-(6}8*aFqdReE;6^q7? ze;|k{yx&%nQ}8CpV5ol%H>5@Qi(3tD2AO7Sg*mh&%cIyQmic*S|H7H;wk@xn&=%iz zOk&ACWqIl9ZAH0VS+znB4JCJ)k z{V;;O^^{#xal2vJdfzp3#QEhhkfPEi{;-ptC|qnUT#!;yXa)0QI#2)Xqm9=;OR;W1 zV4K}qo_@zg>n)k&)2Yt%6Evpt&?W@|Z&fyK|NIl^5|bMwx0OF=Me*XTml?sDF^UDD za-c)}`PZHUMfHez!-?#Eld&HS4qe+n70#_KZt3e8zotM3*G?tcy4pcIRf=OUC;mLy zbRHbg5|L)0UOrc}_ReerXA`ElvdLuPJQpN_sbBGEY2?b|K~Yt>@x|juMvS?!#{`;` z!JW+K({4v`G3nw1M>5}~)nBv($jQ<7o5{g5^hIzB)k-FR6jcYaBx+52{Ca~Fm6PBR zEz}BjdE1N4E$He(TL(Hc-i#4yOLOKfk;o`L8U6L#5XWP(`C|^Z;WaNe@x8DB;d=!( zug;=AQiZMW?o?OZE*cW!W#{7s)EJTv1JnaA{sIckZR`1{h`xvlOPRx;kh?><4M1|# z?_^J0ZPlnC9Tq3G|l=-*LDb`dg8Kwsz+=;C&gEW^1Sa$qsBmPYM?g<;?bDQ8ipzO3k!!OpA*#I zGrnjGNU4q$v$P=>7jL@iC8Oz=iLL)I$KAuS4%Ky?eQ|av^TgSUYoE&al+S1-UFYsb zQ)t3=JzBS9CM`aZc-8MyuO{QE=XLpapO|kxF7*c#V|azVzv5k-9#&cN(m;cMYDaY> zz>xLSx}Z%)1OZ)uTO|1DL(=8bRKoS*;5G~2g!8~pePK9kou2)rvHcV)H8X-1gti}i zguMm{%$bfay1oe`dEpIwd5x>-Joav-t!RPRrCu@QEv?=+{nQ?K(j&clK{iI{txECr z;pdNA^8zA0g0u;g)&zdS0H!-e-@lnrG!VNQwfCC1ZQiJqy^MTa&ukQ~XSi-496c9s zSRncMK=F&-{cN7dFGuWaJ}3xd)3!jM`_cR$;~{|!-F!7ZZGwx=j52?&m7%!btA^KO z9?qS5oYqWIJ+YFH%1!^)MUN2L{k10tv*l5RAnuYQf?Ktqb`NF}-8}+4Q~PQkuk8zY zga%3#mx(?0akxacub6)eX6p~ZG<|Ar_3G%n z{}PnH_qommFNqwiHB9WWnj)Xc^I^SXO@FrO)kSkZ{~6>fKt4e~m<3j-{GCRD&73}J z3~6gf%Piv;!^u;rU<#K>zq}0ar;$N~MVxyhiaCMOi0%R1neBV@t#M&5`^ZUrtI(54WGjGXcp9FT>KK&{n@j zY_s@D(Jrq`;m_5h9v#zePiWbm8P=UQT+}bO!CAS0qJyQ?(ig*Id|Q;$A}>`7EoihK zdDsF)s$8jcKXU~{LaR$2o;E+#e{p$7*2$%<`%)qi@BMUEBaT5nmRv#l1@u#i@KpI) za_7EbB(2Lcue&7CxaHZkffHv0_Nx*uK{j!H$>c>_RI!N>_SguU>b~LXbm3-lFr+zV zWYbjyaIgg%!+A0<>_hW8^Bt^SmEU?^oiiT^Yivt+l6tYV)($&y72^I7KC!X6X`5V7 zUqT2Vm&}`D=x0if_$-aS&db!K7uPCxAoYsI?ER5YDbqYn{uB6o3|nLs@JkRoK52bG zv%zf@6rV9C0C&~>Y`T7s6{+DeYv0LUZSwo&tHVz^>4oNMowOh~u}6k2W*+Fs^zgR- z4%7VUtAfG@jI08ZoGzeCKEjR91Nwj-DC1^=ydfTXXX{PJv*I?;Zly}#UB`UepyRd_ zgv{&)s*#554|-!@(BVXLTrSo*oqvZ0Y=dZ zN5pnHilDL7(h(tNK$nS$iAPt#D@Ro~L>nVtK#ixPN#%6&mqEpmWi47c&3Cfixn4g`+&BZ*pA(V>;{$KlBjV36H zyhiAoXOoCp1YH%1%DY~AEpldoWo)0W!7iv)YIL~b$f#k=-**4ar%)~ZdT5rnJ-?Oj znj7E=ayPNMa&T;I`MxzSZdfjX^}_N~_?jA#jJ)%_1TRH6TzYHl;l?XSO&X`Y?)N_z z)4jW{YEwfYM!41MLGRXy`5vo7en5DpuT5^`yp6llQypHbI@(dTbd-LtUIBQD5jsEw zsFKgl+Mak?0V@gIc(D-#yNLAxWNQQ>7Q>1nlKPJ%{Maypx5XJ&86VVPswknie2Kd6 zYvPB_Z$@llmtokxcH!JvqU`b=<3%kX)Nc=S?H$K);Ul9+a_uvT;`JKLMgqRL|0_Ma zKoloW1`bqc2)nGPy)S+&Z=6%8(a} zFLG5*UgpDZdfgAU=s@1(E54v~9?`PaNV!ykc=ESfvVW5E*R-ghC@) zpz~#JbHi)e0$mT>Cv;tDEi5h6G~Q=fMWQu?*v|==c@eucF%B5gL4G|oo?m5kWd!2x zErinK+s3LZF9g)H0#H)p6Z6i(*$L4hnaS5W00Z|w+B@s0s=jvLOGty#Al)E{gl5ll<1C+WFGB-^H{jW~&hnxeEH$qcpDs$D zvtU6&PGd-66- z*#DW&wfd((I-5lZTUsO4jBxt!l$p&Vg;s73)Ae>%S`E^NfjW{zkDU(lqr>2G@C=Mp1feUMs34a; zA=$!+R@Ir>zW>y&%CP&;&vYI55fG2?0t*2;c03`lk069b%r0#U*7j>&*S@ZtxU>55 z5+>9^aAzV!@CqrQ!`4}!wmOfzJ^>08if0#Ox+HC6<@_|K0&COelgrbkv2Hv@|UKb36cEZ-dbw{gh=PGVaSDL{ZmHg8=`i!E$M5-9#u3bcsN^OEZ@qG6eOL) z>V_arIeQl}yIJ3=iQUJ9l1&mnIr~A<v5o|%bu_<%m{naJs*3Yab)XW8foGy*nUe~{c%-i+1%lb$m{gbWy%1; ztH&ejpK1xiY$oYeSD_CN>;;Y z4o>HP_+Z5y=aYbVK>OX?;}gi0Y-cfoNU)Fl>5L!$2;fJ#{!NqEU^WhHDg%`prn@8| z+;)Mj5n~e(pGWUI#MPDIM2?UJfAD%LLJ+*T*PNF*czG+o$YRq?FA=Bh3_S>WEx-Bh zvh`Pkd}pFfarmt&mhkffdpR~mR^cCa`Z;oJ^3IE}^M_K|d-D)iysR2UH@{z>+h{vA zXmMa<{H}H-Y)lRILlj6Drv9Yw-ZZd?V$nxW3C&mKiI5lRf8ZuJ^{Pedlx#3vZpAqD zCrz6@1*@Q7VwwPs<%S|2gU{H}J##{N{Slv|a3qyExK^p4UZ=R`Um4`gO4YlhIK1)T z8V1k1j+0%CxxX&Ipl<@_zt zdX@bk&su>^?~u0bS{P6K*RdjbnP3vn(F*+t7ni9dy?&j4B*e)${`kx-WM4zYFd^4+ z_WkJt8B zs_xHQ5kR_5*ylBs|Lj`10)0DlX{}{2dub#W(B>P8dm{qdPZOKvsb;U#Ru4dnDh4nPoAWOl6T`P|7^$p1MUW#tXVO)bDx+`TYh3l>pP&-yySb+qugQ3VdUl zAN6$p2{-3nqkPE`0dr^qu4k*%Hc=}X3f)ZVH0~eMhmtgsd7#>W!oJ-*ppq2a$2V+| zrII06Bc&SCM>00{VLKBk3pXf zFkfkv4HuknadYVj9~s0$VzwB=#mWB8hQo7J>|NndeY@7~(Xl~EepNrS%sX6dSo4x= zIUCL7#0}@>A(0+g*9Me~tRgCbJNDGu39V?XQF-VosGIq4Gh*lw_ zX5m=)Sj(0j>5$-3Lc({M#!wBaZc-1M>Hz~%I`}Ns z5N}tOLq>?1?J&AU@i;tv1b~zwN%!hr&ah^Bo#wMhH;1QU-YS!Npu#A+GWk07^>Sj> zx9*F-(IrT81=#k*%d02IeWCYyUwe3yl*#Tknsv5%%n&3-#9|Ws#TxtGVO(&D6rF|Y zM$9b6j+h!a;p~UJJg5>p`6+?U+##0?H8#&Ma9HXaF`ZQZ*K~5cH(-nhyveul*5mK( zUoWQse!CnpZ4|AS-zgM)pu*fv!{nL#>Mzt)?-_E@rtxR?r;L?Kx@3{>kVSFn+8<}A z;|r%?bOgn*P|JOSe^+bz6uhi09sLeh2p6v|_g_P)i2bYB%uH)~E;?k6+tn1TcJ)C_ z*+tB!X8JkyzrSn$^>yh~z6gA4=@tc^81bSZyj(UAUkIp{{y@jCf3)GB{BR}l8SF{V zkhb*fzs8vLCJ@_XmQ0y99J0u)+z4}<1L`US`PU(DV;}uc8V0=bFwlyiFqgS<4vu*M zgSX5Pw&*#Y3`+-`I~*zanF_CvFQqt)HgHo?xVByc6pwNtLxR#`dH+}e_X@f3wR>Vt zmL-7o{)vVKzsf}e1bH;O(B>Y&eN=>7&r%q5SY^P`)JpZtx3BLIx8P7><;&#n9a zv5Uo6B-GD1{#GB;BCF7r{&>lzQd7*y_BXnj7P%6T-~}C7DSziK8l^`pjJvn%{?5Vw zbK?KgOaIdq|5v-BLg5#}?z9XDOcC+b4TqXQ3K6(T@B^=dj~$`5o-2Q;dHqFnbGY4h z@-@8$4y4Vj2o;)T#Qwy);*98^f0$^#vmAtb-((@=WoBv8WBUAkwmPjyPlfw=O_S2? zS^j#=e!lvp`{zpI5G-JuYk;$S@FIY5J3zppZ3SXU`GD~~-QK!ZkQ7GnlfjB!N70qE zw+bM+hd^4uFHGBnTJPF8fd@|8Il= z=*=&5QH;($Al)fMhM{sP2z+pY5s~E8~zKpgtLPpzXwx8dcV?08N6xSM6Wipf)Rw_(jB?WWMv#NBaA$u1<`lQ z=^ybPD=9I!G;V7L17Fqrvj$;y-RqgjAZ*T>^qPYo*uemvoEWb-itiFOvBKl3Oc)uA z#6W=P9AUyo-CXLopms`Q?yRc%3H5;wIuT;@gMp>1h>)`Yz>I@O6@y>Lub6ed>x~DGFkt{e}w4VX3FNkwaq5ZWl`dq^wA6?4LH^ zwd)9i+5qf?Y&(V`Y*c>@tpZru#t!US&3kQCm?CR;i8KQppC2~~O{6KzSjN4Hm%o6+ zZ1-}QOM~H%z}W9sdXC6h{ozv0b%Q@ykk+vgc%y`n(MHZKz|%zx0lKX9Z#NFBJmTt| zk#7Jw$yk)V=@>x(0uGM%$OwbmGVmWU!f5cxM-)8%bk~{DRP`j2C`5k9g#?kGZeucM z`K?-q^>GJCzHw6eBfF*mKHD)UXAXLq^a9lP1p)4y-%9XwOeGi>GS#wQ+@yleg}n zAPEL#KA+U$b*tuo;t5c9>`qcypO8Dk`uALcuMf$i z69yK^=Fg{1)AW3X);^5N^F1N!n*;ZdZcKZ)9m#G8MDK!f^6{Tdt-j>`A5uk@BtrBZ0*h1=@%7<}9C zggE$Hpf4R^{F^ItJV6}B^Lk*%TUZ?-zQ|{fDBj1~?l$hWXwN~2z3@G3QCaFy^s@Jz z(S0YUG|!rK>|Mh=@6Dll-1;s4`{^r z7Y8jUg{ufHjkaEiPTc;4cV)f@luPXrU-5xE&Vj#NZ)h3$3bOh_IwOlC&W{V?jRgK@ z5Ox;m=Qr2gSQE(~C>8h-I$QQK-y2Plv*5VYCntu?{1gao9>)oVA%SgF_q0HRxPGs$ zkZE+|6Go{;Z%L{#n|t^V z6Xy;#WIKJ#k#sRocF*RB(fPczKVIT>+Fo3U4c2`9qP`lRC^WOOvuNOKIEPF&QWem&t)7a3Qw-pA5QE?57ZtoEU zXKd$H3SuOJN8p7w1&+p3U6R)U(|VKaZ5wzFHDUO^wajAmve$hM2cX`;eF^L+Q`XR& z#i1>1)v}}SpkElr+Pg1P+a^b-aXxUD#@)*1XF6 zF-Xg+!}1!VL&ERD2%S=|)crFw)1R(;6f1IS?5R4Xn64vm#ms&xr?Izcm$3>Q!Dp6j zR`{$b&kBz+Mq2#+v3*Q#$x!xkf`oozM^|gzhBPH=|oM?ViBw|8Lru21_ zWp~+a)p*HY>i$~M&JeIlV5#qRAtYaR^mTqmKZ=?0T@&IU@wysK%z#hCH-HuQ2Ovef7VDJ65C~!XKuwsOr)8(O7 z68|AL8|S$400_bej}gZx$s0^sT6X_nm4zxXvl+f0Jykx<`AxP<>T8G3x=5$|LWPa# z$>c4#Z`5+W#EO#@?~p%Uv?WfpRpA8g8^hGX+t4Abjda+r%hKb4I=aH}hS9<4Tll`I z7OTXvYO`_oBMmS^HyC3q1v%A@e8|AjAo2oW{mHSqx^et~)jC&D3$;cNv-xC9Ki$o25D`IYlihVQs^jWNFa1t#udL4C{UEQu$k6 z7oHR_)!P4HSki8RlcxZ$u&;9YJJ=<5%ekLw_E-92^JR+%_XOJtS^VYTn-@|gHV27+5w@Ir&b_D2p;@XH`!>jgz!9# zX_ALQ9U~xtEuS#cle=7h=nitpQrHYh7m4X*`bWhb$H&uWgaYAlxzS>J=WkFxqtL{2 zo||F6GCw*z3LYcqlSlg-AShiaNS)r&p&(pkjxaVW*osePxL?cJhU@QmVi`5fjwnwW$**lUVXa7rf6mj?P7b7hxBvAlk8^4HkJu zN{9Bu@l|iqyt<8^(ve?AZ-orZ6xrxU&^Ogbvm_oaJoidr`HH=dAyZZPPvJD6B9Mbn zF9$w;T{FMrBJ2uTk&6ZeLZxg`+%IP>`i)XT!md?9A4YLm8@4 zhxqcOtM-DMBdRj&vFAOG7$KUFrYD{GoGk17EFErPN`Gb(XlABb1Dqc()_xZY6sIIu z;dayJY>0k&N!=cy>R%*ikhM!OIfag{VHv*SHfJUJZd2!t#sd-m_A&WQ@@W<(pNrhl?|hOmXfP`TQsiGd1dsSRIPq#_&vhivxu}RsWRH zAZ9MU(c1pSrJu{S>>1h7R40v&8FQ`h2s9y&ZbPr*!5iWb@uGWN4^+6>dqkbQsx;xRVtQZdmnU&DpMnTwyI(4I)3nuN6cvEA=~xyzc!ztGuU)A?$Jpu(3T z@>f(Kww*X;)US_$BCHtnfq_WR#l@L6C6wM8>V&l7O5$q}c@hz;9zA@E`nLaL5WRuc z!}j~HJZp4sue1CGqZ4X3=6E;eYI{nS77}=66^BYpbL$HW3QvUJ|2$qMkIS(X{gL5C z3@!dXag86EdjtDjPH#C?F5TAz922U)Ua|`B1~}Q#v%?4jd4Dfk&G!&N>slthy$?PY zAM#0ke`=|!VUgFEGRjM7>Ugfrk4H&?)baGxHgv|Jwl0dcrP`cLzn1983HIca_{Oz1 zkKTP;k5;nKHlv8l?6c4l@5`PH(%Gy6^5jfVViN)h9=@5)C^-8cE^;pcf-vDj6_9Ey$EfRU7YvT*hx{f;AVA=6%6SG*AWb1_!Io1+E z<97mzUeywMQ+0>7jvGgm&)6dv+^+{AK+6qh8Yf21Y?)t8fC)6{dU!(Js<2YF$!FesAA5F_;IiM9L9 zVV8&Ecip#P=iLn9Cp#t4xKa#)=k!Y~y_>4MN=}XC*MIVRONI9#aSR$plB9~tW|6e_ zkT7s0#NZ#wKNLQCXO=|7ZET6Et;o54ll=92pz#-9T_-C!CtPu1DH_idKH^C~Aj0;( z-=gGHp9o5@aMk$de9_xu?HlewdOlrfh0o&$d)=L8w>uwU&(FL{hVSRRFKmalVrB1j z;qj=@X|C1vMFh4V7bzR{7$@#O#h!f;enHvk5g1g|lS(=i%!Hbo6W47HAVftLTYOXyYP^BF#|iRe4eo*b+l^5EoMSEL}Tp;H>S)J_^h)rVH%C$?A@}Z=&)@Dl$iEfTA#5**k}eU6ff zTV1K5O7U$IBaw!(7TF@nePK?$C|Yq6&o?@{Yf_^7IDfoACW_;=vP22FX8Fvqr7E=Z zqUo_D92MH+JzHfR>y&YIXgKEY9`x71hz9&MFsQihe@<}WN)zbX?w#X@>NtKqQE4iq z43=GCR2@QDE=9>Vu&qqQ)F|7sj6w?=?ZuQ-Pk8OooTYaSgw1BySHK|Iyvj6HJ!LJF zeUV4t2fya4A(_Lr%Y(Rzwl(uzt*bGeQryIs&%!sQG#3X8h!%hE&Q@zs{iwo!NLFQ| z1^s3xrQ&`(z!$v1&^c}`7yxU-XJs!xp$aRWec>{YedJQz`|)#?xwo6i_d)5%HKJK) z)%^J|nd0IHSoCwR=xxgq%kjKae8emWMhDD-vTg;K7yQK*QtFpDt>p`&bXUsHG{%TDD5U+66NaAnGtH?p9NEl*zM5sn|2YdDVChVyD zjuzzi6ZdQsZ0;vT<&L;hAJ)|Q(^9VlMcOkEPz3X{h6Z73w71_ZU#Sv9m1kU6HqW0= zE8|Yf$#8B)t2IQ`pD1;4GTWVBs~52kOdTu|V8xE=df&!A$usmM5EbD>S2L2V{=}VD zcS^S75wVfuDSG2Crh#EbqE6I4!Io#>mV{=(6LoZ+XS3eCzn@577+P-G8L+4ui~rK- zMR)&bciL$q=2zUq>7_*6O*^IK$Y{3m04)!C)0qB(4rIAiwS6~8IZaHYXRGA!rb*jI3!OQb#fJ288hFt9j5ReCKU zstlDoHOEBNBWQ7}fM1ifJXuw!&+Y`4Icqwz=)4uGHN*7tnTW;tn^njgWX{4u6{8-H zw&NkuG+Z&5oKhvnIcCyo+;9ZXI#X5>sg5gMz19n3ClPl}(fzJMc- zd&ui1j}yk7MI?65Ka=XzOinW&$U)j5)lW%y+4eQwX9_9>o!XYEDr=`q;ml<|aoeli zh+05bKF9c~o4%izHx+PV-4tSIM3|W;;IMvC+yr+&JHdG@zjo8a`9oZVUP*%wBJnh# zOl`0tf3Tm6z45SIjI?<$&Rs_j_^yQmD4<$);j(t#IsKd(Rfu(tjBB_o`25|{$DQUh-?0+NR zT{wf5#lqH?lvoZn_&z;{91HOF{D`Ap`wahWsP zi_jGmB-r)9fYsE6*f5caJJn zc{x<)c)eTALJnb@#wmK{vNzXY!)aK~wF0&u3Z6R-`))PA&d+DeC}x*FM!O6B)Kcn{ zb*0|O-t)EPI&$qtoj5#ZrMQTLrfvTWv;AzU0V@ZQe?Oup*s)${5ZG4{l`?F)*waW= ztr!kHuyC%Z^%tXnrGYty;kWJC&NP%(%ORi0IC0B1fz-E7Vj9o4jL_^6O=u&R)`wnH zPhncafO2|SG&{)X;|&>%%yTAi-HNB8aFKa!}MdPh?U2H~~L*Dd;;8Q(U zLz=yP{{=?(<{PwUBpSTu)%{G4;gKP6uXx+XA1I}%IMJBS&jIB=nF?0p?@SCyV5u~w zxDC|-69DvQ`U)^@3QmRHkm363Z%dsTalUd`D!59W$q>oA3VU;OMEd|a`|#m`U25?} zwpjy^<&Cbz@BdtW!DxPYw8mQAsUq2-Hlt)@uC=ow>ShtCT=uwUG-yGR$!+?es%Gn5 zmAKvkI zffCe8$wgHd)Q#~Gh%KB_GOD5C?-jX8zT8Yk{S}tWiut++Y#O47#=#|WS0+so@5 zTSYY+Tim=UxKgm>%DnzSozs) zvcq`AuN@b!LYkYFq2t`=y9@Bm_j|Krcy32EiS8oS{>8R?(5-nTN+xfA@aEo4y}j8a zK}yQe7#jzRYKi(@5H(o+8T*a-}r_k0!(4Np{V ze;Q=H^;f#Cte)c^WzFAP@=|S5A z(p`mE9G+g*KAvvD>GahC6D3}K8+(UqHO_4EVWP