Add async support for Dataverse SDK by abelmilash-msft · Pull Request #171 · microsoft/PowerPlatform-DataverseClient-Python

abelmilash-msft · 2026-04-28T05:56:26Z

Introduces the complete aio/ async package mirroring the sync SDK:
- HTTP layer: _AsyncHttpClient wrapping aiohttp with identical retry, backoff, and timeout logic
- Auth: _AsyncAuthManager for async Azure Identity token acquisition
- OData client: _AsyncODataClient — full CRUD, SQL-over-API, table/column metadata, file upload, and relationship operations
- Batch: _AsyncBatchClient with _SyncResponseWrapper bridging the async HTTP response to the shared sync multipart parser in _BatchBase
- Operation namespaces: records, tables, query, files, batch, dataframe — all mirroring their sync counterparts
- Top-level client: AsyncDataverseClient with lazy init, async context manager, and session lifecycle management
Adds pytest-asyncio (asyncio_mode = auto) and aiohttp as an optional dependency (pip install PowerPlatform-Dataverse-Client[async]).

Test plan

1,758 unit tests pass (389 new async tests, ~95% coverage of the async layer)
black --check passes on all files

Copilot

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

- Reset to main (which is now PR #175) - Re-apply: _ODataBase, _BatchBase, _QueryBuilderBase, _BatchContext Protocol, _operation_context in base, Self type annotation - Re-export multipart helpers from _batch.py for test compatibility - Update test_sql_parse.py patch target to _odata_base.urlparse Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Reset to main via refactoring branch - Restore full async implementation: aio/ client, HTTP, batch, OData, relationships, upload, query builder, fetchxml, operations, tests, examples - Re-export multipart helpers from _batch.py for test compatibility - All 2166 tests passing Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ontent _AsyncResponse buffers the body in ._body; it has no .content attribute (unlike requests.Response used by the sync client). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…replace downloads Two download paths after replace uploads still used resp_r.content and resp_rc.content instead of ._body, causing AttributeError. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Validates 7 properties of the async client against a live environment: 1. Non-blocking reads — canary confirms event loop stays free during GETs 2. Read throughput — concurrent reads via gather() beat sequential 3. Write concurrency — concurrent POSTs beat sequential (POST path) 4. Pagination non-blocking — async generators yield between page fetches 5. Mixed fan-out — different op types run simultaneously without serialization 6. Error resilience — one failure in gather() does not kill other calls 7. Real-world fan-out — metadata for multiple tables fetched in parallel Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Each test now has a brief explanation of what it runs, what property it validates, and what a failure would indicate. Also clarifies that speedup measures async-sequential vs async-concurrent, not async vs sync. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Populate __all__ in aio, aio.models, and aio.operations __init__.py files so public symbols are importable directly from the package namespace. Update all async examples, README, and skill docs to use the shorter import paths. Add test_async_package_exports.py mirroring the sync test_package_exports.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

'dv' is not in _ALLOWED_SKILLS (added to config.py on main). The PR build tests the merged result so it picked up the validation. Update to use 'dv-data' which is an allowed value. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…c-phase2

Sync side was updated to use the new CreateEntities API in PR #183. The async client has its own copy of _create_entity and _create_table, so we need to mirror those changes manually: - _AsyncODataClient._create_entity(): URL EntityDefinitions -> CreateEntities, payload wrapped in Entities[0] array with @odata.type ComplexEntityMetadata - _AsyncODataClient._create_table(): pass complex=True to _attribute_payload calls so attribute metadata uses the Complex*Metadata variants required by CreateEntities The shared base (_odata_base.py) changes from PR #183 — including the _attribute_payload(complex=...) parameter and Complex*Metadata output — flow into the async client automatically via inheritance. Also picks up PR #181 (OperationContext key/value allowlisting) via the same main merge. Tests: 2187 pass, black clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

This reverts commit c1f1237.

Sync side was updated to use the new CreateEntities API in PR #183. The async client has its own copy of _create_entity and _create_table, so we need to mirror those changes manually: - _AsyncODataClient._create_entity(): URL EntityDefinitions -> CreateEntities, payload wrapped in Entities[0] array with @odata.type ComplexEntityMetadata - _AsyncODataClient._create_table(): pass complex=True to _attribute_payload calls so attribute metadata uses the Complex*Metadata variants required by CreateEntities The shared base (_odata_base.py) changes from PR #183 -- including the _attribute_payload(complex=...) parameter and Complex*Metadata output -- flow into the async client automatically via inheritance. This is a clean re-apply of the legitimate diff after reverting commit c1f1237, which accidentally committed many untracked scratch/build files alongside this change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

The sync side added test_url_targets_create_entities and updated several _create_table assertions to use Entities[0] in PR #183. Those tests only cover the sync _create_entity; the async client has its own copy. Add three tests to cover the async-side behavior: - test_create_entity_url_targets_create_entities: URL ends with /CreateEntities - test_create_entity_payload_wraps_in_entities_array: body has Entities[0] with @odata.type=ComplexEntityMetadata - test_create_table_posts_complex_attribute_metadata: _create_table forwards complex=True so the posted Attributes use Complex*AttributeMetadata variants Tests: 2190 pass (up from 2187), _async_odata.py coverage at 98%. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

PR #183 added a test_picklist_table scenario to the sync functional testing script demonstrating local OptionSet creation via an Enum subclass, label-cache resolution on write, and FormattedValue annotation on read. Mirror the same flow to examples/aio/basic/functional_testing.py so async users have parity coverage: - New async test_picklist_table() using await client.tables.create(), client.records.create(), client.records.retrieve(), client.records.list() - cleanup_test_data() takes an optional picklist_table_schema_name and prompts for the picklist table teardown - Wire test_picklist_table into main() with corresponding summary line Unit tests: 2190 pass (unchanged; this is an example-only script). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

github-actions · 2026-05-24T22:22:03Z

Warning
ADO PR pipeline YAML change detected

This PR modifies .azdo/ci-pr.yaml. After merge, Azure DevOps may disable or require approval for the PR validation pipeline.

Action required (post-merge): Re-enable / approve the updated YAML for:

DV-Python-SDK-PullRequest (definitionId=29922)

https://dev.azure.com/dynamicscrm/OneCRM/_build?definitionId=29922

Please resolve this comment after completing the post-merge steps.

sagebree · 2026-05-25T17:13:17Z

+                t = self.default_timeout
+            else:
+                m = (method or "").lower()
+                t = 120 if m in ("post", "delete") else 10


can we make 120 and 10 constants? This is hard to maintain.

Also we should fix for both sync and async: In practice Patch can also be slow operation

t = 120 if m in ("post", "patch", "delete") else 10

sagebree · 2026-05-25T17:39:08Z

+        if self._session is not None:
+            await self._session.close()
+            self._session = None
+        self._closed = True


call this in line 211 before any other await.
Reason: _closed is not set to True until after both await calls complete if there is concurrent request.

output of the repro:

(.venv) PS C:\repo\DV-Python-SDK-2\examples\advanced> python .\test11.py
_odata.close() called 2x (expected 1)
_session.close() called 2x (expected 1)
(.venv) PS C:\repo\DV-Python-SDK-2\examples\advanced>

Repro unit test:

import asyncio
from unittest.mock import MagicMock
from azure.core.credentials_async import AsyncTokenCredential
from PowerPlatform.Dataverse.aio.async_client import AsyncDataverseClient

async def main():
credential = MagicMock(spec=AsyncTokenCredential)
client = AsyncDataverseClient("https://aurorabapenve4a25.crmtest.dynamics.com", credential)

odata_close_calls = 0 session_close_calls = 0 # Mock _odata with a slow close that yields to the event loop mock_odata = MagicMock() async def slow_odata_close(): nonlocal odata_close_calls odata_close_calls += 1 await asyncio.sleep(0) # yield — simulates real aiohttp teardown mock_odata.close = slow_odata_close client._odata = mock_odata # Mock _session with a slow close mock_session = MagicMock() async def slow_session_close(): nonlocal session_close_calls session_close_calls += 1 await asyncio.sleep(0) mock_session.close = slow_session_close client._session = mock_session # Two concurrent aclose() calls await asyncio.gather(client.aclose(), client.aclose()) print(f"_odata.close() called {odata_close_calls}x (expected 1)") print(f"_session.close() called {session_close_calls}x (expected 1)")

asyncio.run(main())

sagebree · 2026-05-25T17:54:30Z

+                    continue
+                cs_requests: List[_RawRequest] = []
+                for op in item.operations:
+                    cs_requests.append(await self._resolve_one(op))


This can be addressed after GA but please create a work item to asyncio.gather multiple _resolve_one. In async, there is an opportunity to make this performant by batching them.

abelmilash-msft force-pushed the users/abelmilash/async-phase2 branch from e491241 to b848324 Compare May 11, 2026 23:53

abelmilash-msft marked this pull request as ready for review May 12, 2026 06:29

abelmilash-msft requested a review from a team as a code owner May 12, 2026 06:29

Copilot AI review requested due to automatic review settings May 12, 2026 06:29

Copilot AI reviewed May 12, 2026

View reviewed changes

abelmilash-msft force-pushed the users/abelmilash/async-phase2 branch from b32987a to ad9297e Compare May 14, 2026 17:28

github-actions Bot reviewed May 15, 2026

View reviewed changes