UN-3479 [FIX] Provide service-account access to resources for org-migration via python client - #1987
Merged
chandrasekharan-zipstack merged 9 commits intoMay 25, 2026
Conversation
…l instances Enables the unstract-python-client SDK migration subpackage to drive org-to-org data migration purely through admin-issued Platform API keys. Specifically: - adapter_processor_v2/models.py: AdapterInstanceModelManager.for_user returns non-frictionless adapters for service-account callers (was: all()) - permissions/permission.py: IsFrictionLessAdapter grants access to service accounts on non-frictionless adapters, keeping the friction-first check - prompt_studio/permission.py: PromptAcesssToUser short-circuits to True for service accounts so Platform API can GET/POST prompts - tool_instance_v2/views.py: get_queryset scopes via Workflow.for_user so service accounts see all tool instances under workflows they can access Plan: org-to-org data migration v1 (KB: zipstuff/org-data-migration/05).
The SDK migration subpackage relies on list-by-name GET as the cross-run idempotency check (Layer 2). Without this filter, every re-run would re-create adapters that already exist on the target org. Plan: org-to-org data migration v1 (KB: zipstuff/org-data-migration/05).
Contributor
WalkthroughThis PR adds service-account bypass checks in permissions, restricts adapter visibility for service accounts, moves tool instance access to workflow-scoped querysets, introduces exact-match filters for SDK flows (api_name, pipeline_name, workflow name), adjusts connector serializer read-only flags, and enables full CRUD plus name filtering for tags. ChangesAuthorization and Access Control Enhancements
|Adapter & Connector list filtering and serializer tweaks |ToolInstance workflow-scoped access SDK Get-or-Create Query Parameter Filters
Tag Collection CRUD and Filtering
Sequence Diagram(s)No additional sequence diagrams beyond the hidden artifact were generated. 🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Mirror the adapter pattern from c05dc05: SDK migrator uses name-based GET against target to detect existing rows before POST. - ConnectorInstanceViewSet.get_queryset: thread CIKey.CONNECTOR_NAME through FilterHelper. - TagViewSet: declare filterset_fields=["name"] so DjangoFilterBackend honors ?name=. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tag URLs previously bound only GET on list/detail, so the migrator (and any other API consumer) got 405 on POST tags/. Wire create/partial_update/ destroy through the same TagViewSet — permission_classes already cover the auth path; no behavior change for callers that only GET. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ConnectorInstanceSerializer.to_representation overrides connector_mode from the catalog every response, so any client-supplied value is silently discarded. Make that explicit via extra_kwargs so DRF OPTIONS reports the field as read-only and round-trippers don't trip the choice validator (catalog mode 'FILESYSTEM' vs model choice 'FILE_SYSTEM'). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Needed by the org-to-org migration SDK's idempotent get-or-create flow: without a name filter the SDK had to list-all-then-linear-search every time. Adds WorkflowKey.WF_NAME to the existing FilterHelper.build_filter_args call — same shape as the recently-added adapter_name / connector_name / tag name filters. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Needed by the org-to-org migration SDK: after CustomTool migration the target's PromptStudioRegistry gets a freshly-minted prompt_registry_id, and downstream ToolInstance migration needs to remap source.tool_id -> target.tool_id (both are registry UUIDs). With this filter the SDK can GET /prompt-studio/registry/?custom_tool=<tool_id> on either side to resolve the registry id without needing to know it up front. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the existing exact-match name filters added for adapter, connector, tag, and workflow lists. Migration SDK's get-or-create flow queries by name on the target before deciding fresh vs adopt; without this the SDK had to list-all-then-linear-scan every time. Pipeline keeps the icontains ?search= alongside; api_deployment keeps the icontains ?search= alongside. New filters are exact-match only. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
chandrasekharan-zipstack
marked this pull request as ready for review
May 24, 2026 07:00
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
backend/prompt_studio/prompt_studio_registry_v2/views.py (1)
30-30: 💤 Low valueConsider using a constant for "custom_tool" to match the existing pattern.
Line 29 uses
PromptStudioRegistryKeys.PROMPT_REGISTRY_IDconstant, while the new filter key is a string literal. Using a constant would improve consistency and reduce the risk of typos.♻️ Proposed refactor using a constant
Define the constant in
backend/prompt_studio/prompt_studio_registry_v2/constants.py:class PromptStudioRegistryKeys: PROMPT_REGISTRY_ID = "prompt_registry_id" CUSTOM_TOOL = "custom_tool" # Add thisThen use it in the filter:
def get_queryset(self) -> QuerySet | None: filterArgs = FilterHelper.build_filter_args( self.request, PromptStudioRegistryKeys.PROMPT_REGISTRY_ID, - "custom_tool", + PromptStudioRegistryKeys.CUSTOM_TOOL, )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/prompt_studio/prompt_studio_registry_v2/views.py` at line 30, Replace the string literal "custom_tool" with a constant to match the existing pattern: add CUSTOM_TOOL = "custom_tool" to the PromptStudioRegistryKeys class (where PROMPT_REGISTRY_ID is defined) and update the filter list in PromptStudioRegistryView (the list that currently contains "custom_tool") to use PromptStudioRegistryKeys.CUSTOM_TOOL instead; this keeps keys consistent and avoids typos.backend/api_v2/api_deployment_views.py (1)
263-266: ⚡ Quick winConsider using FilterHelper.build_filter_args for consistency.
The manual query-param extraction and filtering pattern here duplicates the logic provided by
FilterHelper.build_filter_args, which is used inbackend/workflow_manager/workflow_v2/views.py(lines 81-86) andbackend/connector_v2/views.py(lines 48-51). Using the helper would improve consistency and reduce code duplication.♻️ Proposed refactor using FilterHelper
+ from utils.filtering import FilterHelper + # Search by display name search = self.request.query_params.get("search", None) if search: queryset = queryset.filter(display_name__icontains=search) - # Exact-match api_name filter for migration SDK's get-or-create flow. - api_name = self.request.query_params.get("api_name") - if api_name: - queryset = queryset.filter(api_name=api_name) + # Apply exact-match filters for migration SDK's get-or-create flow. + filter_args = FilterHelper.build_filter_args(self.request, "api_name") + if filter_args: + queryset = queryset.filter(**filter_args) return queryset🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/api_v2/api_deployment_views.py` around lines 263 - 266, Replace the manual api_name extraction and filter with FilterHelper.build_filter_args: import FilterHelper if missing, call FilterHelper.build_filter_args(self.request.query_params, ...) to build the filter dict (including an exact-match for api_name) and then apply queryset = queryset.filter(**filter_args) so filtering is consistent with other views (e.g., workflow_v2/views.py and connector_v2/views.py) and removes the duplicated manual logic.backend/pipeline_v2/views.py (1)
80-83: ⚡ Quick winConsider using FilterHelper.build_filter_args for consistency.
This manual query-param extraction and filtering duplicates the logic provided by
FilterHelper.build_filter_args, which is correctly used inbackend/workflow_manager/workflow_v2/views.py(lines 81-86). Using the helper would improve consistency across the codebase and reduce duplication.♻️ Proposed refactor using FilterHelper
+ from utils.filtering import FilterHelper + # Search by pipeline name search = self.request.query_params.get("search", None) if search: queryset = queryset.filter(pipeline_name__icontains=search) - # Exact-match name filter for migration SDK's get-or-create flow. - pipeline_name = self.request.query_params.get(PK.PIPELINE_NAME) - if pipeline_name: - queryset = queryset.filter(pipeline_name=pipeline_name) + # Apply exact-match filters for migration SDK's get-or-create flow. + filter_args = FilterHelper.build_filter_args(self.request, PK.PIPELINE_NAME) + if filter_args: + queryset = queryset.filter(**filter_args) # Apply default ordering: last_run_time desc (nulls last), then created_at desc🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/pipeline_v2/views.py` around lines 80 - 83, Replace the manual extraction and filter for pipeline_name with FilterHelper.build_filter_args to keep behavior consistent: call FilterHelper.build_filter_args(self.request.query_params) to get filter_args, ensure it includes an exact-match key for PK.PIPELINE_NAME, then apply queryset = queryset.filter(**filter_args) instead of manually reading pipeline_name and calling queryset.filter(pipeline_name=...). Update the code around the existing pipeline_name/queryset usage in the view method (where pipeline_name is read) to use filter_args and apply them to the queryset.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/tags/views.py`:
- Line 23: The view currently sets filterset_fields = ["name"], which is mutably
defined and triggers Ruff RUF012 even though Tag.name exists; change this class
attribute to an immutable tuple (e.g. filterset_fields = ("name",)) in the same
view so the filter still performs an exact-match on Tag.name while eliminating
the lint warning; ensure this change is made where filterset_fields is declared
in the view containing the DjangoFilterBackend usage.
In `@backend/tool_instance_v2/views.py`:
- Around line 97-100: The current ToolInstanceViewSet.get_queryset uses
Workflow.objects.for_user(self.request.user) which includes read-shared
workflows, permitting unintended mutations; restrict write-scoped queryset or
add owner-only permission: modify ToolInstanceViewSet.get_queryset to branch on
request.method (or use viewset actions) so that for unsafe methods (update,
partial_update, destroy, reorder) you filter by workflows owned by the user (use
Workflow.objects.owned_by(self.request.user) or equivalent) instead of
Workflow.objects.for_user, and/or add/override get_permissions to apply
IsOwner() (same as WorkflowViewSet) for mutation actions so only owners can
mutate ToolInstance objects.
---
Nitpick comments:
In `@backend/api_v2/api_deployment_views.py`:
- Around line 263-266: Replace the manual api_name extraction and filter with
FilterHelper.build_filter_args: import FilterHelper if missing, call
FilterHelper.build_filter_args(self.request.query_params, ...) to build the
filter dict (including an exact-match for api_name) and then apply queryset =
queryset.filter(**filter_args) so filtering is consistent with other views
(e.g., workflow_v2/views.py and connector_v2/views.py) and removes the
duplicated manual logic.
In `@backend/pipeline_v2/views.py`:
- Around line 80-83: Replace the manual extraction and filter for pipeline_name
with FilterHelper.build_filter_args to keep behavior consistent: call
FilterHelper.build_filter_args(self.request.query_params) to get filter_args,
ensure it includes an exact-match key for PK.PIPELINE_NAME, then apply queryset
= queryset.filter(**filter_args) instead of manually reading pipeline_name and
calling queryset.filter(pipeline_name=...). Update the code around the existing
pipeline_name/queryset usage in the view method (where pipeline_name is read) to
use filter_args and apply them to the queryset.
In `@backend/prompt_studio/prompt_studio_registry_v2/views.py`:
- Line 30: Replace the string literal "custom_tool" with a constant to match the
existing pattern: add CUSTOM_TOOL = "custom_tool" to the
PromptStudioRegistryKeys class (where PROMPT_REGISTRY_ID is defined) and update
the filter list in PromptStudioRegistryView (the list that currently contains
"custom_tool") to use PromptStudioRegistryKeys.CUSTOM_TOOL instead; this keeps
keys consistent and avoids typos.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d89b6769-7920-4432-91dc-24c0ffaf2477
📒 Files selected for processing (13)
backend/adapter_processor_v2/models.pybackend/adapter_processor_v2/views.pybackend/api_v2/api_deployment_views.pybackend/connector_v2/serializers.pybackend/connector_v2/views.pybackend/permissions/permission.pybackend/pipeline_v2/views.pybackend/prompt_studio/permission.pybackend/prompt_studio/prompt_studio_registry_v2/views.pybackend/tags/urls.pybackend/tags/views.pybackend/tool_instance_v2/views.pybackend/workflow_manager/workflow_v2/views.py
Contributor
|
| Filename | Overview |
|---|---|
| backend/permissions/permission.py | Adds service-account short-circuit to IsFrictionLessAdapter before the ownership check; consistent with IsOwner/IsOwnerOrSharedUser pattern already in the file. |
| backend/tool_instance_v2/views.py | Service accounts now query ToolInstances via Workflow.objects.for_user scope; regular users retain the previous created_by=self.request.user filter. Comment in code explicitly documents the intentional asymmetry. |
| backend/tags/urls.py | Exposes POST on tag_list and PATCH/DELETE on tag_detail; existing IsOrganizationMember permission applies to detail actions at object level. |
| backend/adapter_processor_v2/models.py | Service-account queryset now filters out frictionless adapters (is_friction_less=False), consistent with IsFrictionLessAdapter permission gating. |
| backend/connector_v2/serializers.py | connector_mode marked read_only=True in extra_kwargs, correctly reflecting that the field is server-derived; no functional change for existing callers. |
Sequence Diagram
sequenceDiagram
participant SDK as Migration SDK
participant Auth as Auth Middleware
participant Perm as Permission Class
participant View as ViewSet.get_queryset()
participant DB as Database
SDK->>Auth: Request with Platform API Key
Auth->>Auth: "Set user.is_service_account = True"
Auth->>Auth: Block DELETE for all API keys
Auth->>Perm: Forward request
alt Adapter retrieve/update
Perm->>Perm: IsFrictionLessAdapter.has_object_permission
Perm->>Perm: is_friction_less? blocked regardless
Perm->>Perm: is_service_account? True allow
else Prompt/Note access
Perm->>Perm: PromptAcesssToUser.has_object_permission
Perm->>Perm: is_service_account? True allow
end
View->>DB: "filter(is_friction_less=False) for service account"
DB-->>View: org adapters no frictionless
View->>DB: Workflow.objects.for_user org-scoped
DB-->>View: accessible workflows
View->>DB: ToolInstance filter workflow in accessible
DB-->>SDK: Filtered results
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
backend/tags/urls.py:12-18
**Tag PATCH/DELETE accessible to any org member**
`TagViewSet.permission_classes` is `[IsAuthenticated, IsOrganizationMember]`. `IsOrganizationMember.has_object_permission` only verifies that `obj.organization == user_organization` — it does not check authorship. With PATCH and DELETE now routed, any authenticated org member can modify or delete a tag created by someone else. For the migration SDK use-case (create-if-not-exists), this is harmless, but it also means a regular user who discovers `PATCH /tags/<id>/` can rename or remove tags they don't own. Consider whether an `IsOwnerOrOrganizationAdmin`-style check is needed for the mutation verbs, or document that tags are intentionally unowned org-level resources.
Reviews (2): Last reviewed commit: "fix(tool-instance): scope queryset widen..." | Re-trigger Greptile
Greptile flagged on PR #1987 that the prior switch from `created_by=user` to `workflow__in=Workflow.for_user(user)` silently widened tool-instance visibility for regular users with shared-workflow access — they would start seeing every member's tool instances in those workflows, not just their own. Restrict the widened queryset to `is_service_account` callers so the migration SDK still gets org-wide enumeration via Platform API keys, while regular users keep their original per-creator scope. Reported-by: greptile-apps[bot] (PR #1987, P1)
Contributor
Test ResultsSummary
Runner Tests - Full Report
SDK1 Tests - Full Report
|
|
chandrasekharan-zipstack
requested review from
Deepak-Kesavan and
kirtimanmishrazipstack
May 24, 2026 11:45
Deepak-Kesavan
approved these changes
May 25, 2026
Deepak-Kesavan
left a comment
Contributor
There was a problem hiding this comment.
Changes looks good
kirtimanmishrazipstack
approved these changes
May 25, 2026
chandrasekharan-zipstack
added a commit
that referenced
this pull request
May 25, 2026
* feat(platform-api): service-account access for adapters, prompts, tool instances Enables the unstract-python-client SDK migration subpackage to drive org-to-org data migration purely through admin-issued Platform API keys. Specifically: - adapter_processor_v2/models.py: AdapterInstanceModelManager.for_user returns non-frictionless adapters for service-account callers (was: all()) - permissions/permission.py: IsFrictionLessAdapter grants access to service accounts on non-frictionless adapters, keeping the friction-first check - prompt_studio/permission.py: PromptAcesssToUser short-circuits to True for service accounts so Platform API can GET/POST prompts - tool_instance_v2/views.py: get_queryset scopes via Workflow.for_user so service accounts see all tool instances under workflows they can access Plan: org-to-org data migration v1 (KB: zipstuff/org-data-migration/05). * feat(platform-api): support ?adapter_name= filter on adapter list The SDK migration subpackage relies on list-by-name GET as the cross-run idempotency check (Layer 2). Without this filter, every re-run would re-create adapters that already exist on the target org. Plan: org-to-org data migration v1 (KB: zipstuff/org-data-migration/05). * feat(platform-api): support name filter on connector + tag list Mirror the adapter pattern from c05dc05: SDK migrator uses name-based GET against target to detect existing rows before POST. - ConnectorInstanceViewSet.get_queryset: thread CIKey.CONNECTOR_NAME through FilterHelper. - TagViewSet: declare filterset_fields=["name"] so DjangoFilterBackend honors ?name=. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(platform-api): expose Tag POST/PATCH/DELETE handlers Tag URLs previously bound only GET on list/detail, so the migrator (and any other API consumer) got 405 on POST tags/. Wire create/partial_update/ destroy through the same TagViewSet — permission_classes already cover the auth path; no behavior change for callers that only GET. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(platform-api): mark connector_mode as read-only ConnectorInstanceSerializer.to_representation overrides connector_mode from the catalog every response, so any client-supplied value is silently discarded. Make that explicit via extra_kwargs so DRF OPTIONS reports the field as read-only and round-trippers don't trip the choice validator (catalog mode 'FILESYSTEM' vs model choice 'FILE_SYSTEM'). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(platform-api): support ?workflow_name= filter on workflow list Needed by the org-to-org migration SDK's idempotent get-or-create flow: without a name filter the SDK had to list-all-then-linear-search every time. Adds WorkflowKey.WF_NAME to the existing FilterHelper.build_filter_args call — same shape as the recently-added adapter_name / connector_name / tag name filters. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(platform-api): allow filtering PromptStudioRegistry by custom_tool Needed by the org-to-org migration SDK: after CustomTool migration the target's PromptStudioRegistry gets a freshly-minted prompt_registry_id, and downstream ToolInstance migration needs to remap source.tool_id -> target.tool_id (both are registry UUIDs). With this filter the SDK can GET /prompt-studio/registry/?custom_tool=<tool_id> on either side to resolve the registry id without needing to know it up front. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(platform-api): support ?pipeline_name= and ?api_name= filters Mirrors the existing exact-match name filters added for adapter, connector, tag, and workflow lists. Migration SDK's get-or-create flow queries by name on the target before deciding fresh vs adopt; without this the SDK had to list-all-then-linear-scan every time. Pipeline keeps the icontains ?search= alongside; api_deployment keeps the icontains ?search= alongside. New filters are exact-match only. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-instance): scope queryset widening to service accounts only Greptile flagged on PR #1987 that the prior switch from `created_by=user` to `workflow__in=Workflow.for_user(user)` silently widened tool-instance visibility for regular users with shared-workflow access — they would start seeing every member's tool instances in those workflows, not just their own. Restrict the widened queryset to `is_service_account` callers so the migration SDK still gets org-wide enumeration via Platform API keys, while regular users keep their original per-creator scope. Reported-by: greptile-apps[bot] (PR #1987, P1) * feat(prompt-studio): stream Prompt Studio file uploads to storage [UN-3479] `upload_for_ide` / `upload_converted_for_ide` previously buffered the entire payload in memory before writing — for a 100 MB PDF that meant ~100 MB held per concurrent upload per worker. Replaces the single-shot write with chunked streaming via the underlying fsspec handle when the input is a file-like (Django UploadedFile / IOBase). - New `streaming_writer.write_streaming(fs_instance, path, data)` helper in its own module (kept Django-import-free so it is unit-testable without booting full settings). - Bytes inputs stay single-shot — preserves caller contract for the converted-PDF flow that passes already-materialised bytes. - Opens the destination with `block_size=8 MB` so providers use their native multipart upload primitive (GCS resumable, S3 multipart). - Iterates via the source's own `chunks()` when available, otherwise fixed-size `read()` blocks. - On mid-stream exception, attempts provider-level multipart abort (`abort_mpu` for s3fs, `discard` for gcsfs) so partial uploads don't leak orphaned parts. Unrecognised providers log and rethrow. Benefits the SDK org-migration files phase (companion PR in unstract-python-client) and the FE concurrent-upload path simultaneously — no API surface change, no behavior change for callers. Branched off `feat/org-migration-platform-api-gaps` so the full stack (platform-API filters + service-account perms + streaming write) can be tested together. PR targets the same parent branch. 6 unit tests cover: bytes fast-path, chunks() streaming with multipart-hint propagation, read() fallback for sources without chunks(), abort_mpu on stream error, discard fallback when abort_mpu absent, and a no-coalesce assertion that catches the "collapse to one buffer" regression. * test(prompt-studio): pure-pytest streaming-writer tests, drop unittest.mock Rewrites the streaming-writer test suite as bare pytest: - Removes ``unittest.mock.MagicMock`` / ``patch`` imports. - Hand-rolled ``FakeHandle`` / ``FakeFs`` / ``FakeStorage`` classes expose only the surface the helper actually touches (write, fs.open, abort_mpu/discard, close). - Module-level test functions with pytest fixtures, no test-class grouping needed. - Adds one extra test (``test_unknown_abort_method_does_not_mask_original_error``) that asserts the original exception propagates and the leak warning is emitted when no abort hook is exposed. Self-sufficient under the existing ``uv run pytest`` runner; no new dev dependency. * fix(streaming_writer): propagate close() errors on the success path Provider-native multipart commits (s3fs, gcsfs) happen inside close() — a close failure on the success path means the upload did not finalize. The previous code logged-and-swallowed any close error in finally, so a silent partial upload looked like a successful write. Track success/failure via a flag: on success, let close() raise; on the exception path, keep the swallow-and-log behaviour so a close failure cannot mask the original error reaching the caller. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(streaming_writer): use callable() to detect chunks method hasattr() returns True even for a non-callable data attribute named ``chunks``, which would then raise TypeError when invoked. Switch to callable(getattr(...)) so plain attributes correctly fall through to the read-based fallback. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(streaming_writer): drop multipart abort, rm file on failure Replace _safe_abort's fsspec-internal introspection (abort_mpu / discard) with a uniform fs.rm cleanup on the public API. Local partial files are removed; orphaned S3/GCS multipart parts are left to the bucket's AbortIncompleteMultipartUpload lifecycle rule. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(streaming_writer): tighten cleanup envelope to include chunks_iter Move chunks_iter construction inside the try so a hypothetical non-generator chunks() that raises eagerly still triggers cleanup. No behavioural change for current callers (UploadedFile.chunks is a generator; BytesIO fallback uses iter() with a sentinel). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chandrasekharan-zipstack
added a commit
that referenced
this pull request
Jun 15, 2026
…management + name filters (#2044) * feat(platform-api): service-account access for adapters, prompts, tool instances Enables the unstract-python-client SDK migration subpackage to drive org-to-org data migration purely through admin-issued Platform API keys. Specifically: - adapter_processor_v2/models.py: AdapterInstanceModelManager.for_user returns non-frictionless adapters for service-account callers (was: all()) - permissions/permission.py: IsFrictionLessAdapter grants access to service accounts on non-frictionless adapters, keeping the friction-first check - prompt_studio/permission.py: PromptAcesssToUser short-circuits to True for service accounts so Platform API can GET/POST prompts - tool_instance_v2/views.py: get_queryset scopes via Workflow.for_user so service accounts see all tool instances under workflows they can access Plan: org-to-org data migration v1 (KB: zipstuff/org-data-migration/05). * feat(platform-api): support ?adapter_name= filter on adapter list The SDK migration subpackage relies on list-by-name GET as the cross-run idempotency check (Layer 2). Without this filter, every re-run would re-create adapters that already exist on the target org. Plan: org-to-org data migration v1 (KB: zipstuff/org-data-migration/05). * feat(platform-api): support name filter on connector + tag list Mirror the adapter pattern from c05dc05: SDK migrator uses name-based GET against target to detect existing rows before POST. - ConnectorInstanceViewSet.get_queryset: thread CIKey.CONNECTOR_NAME through FilterHelper. - TagViewSet: declare filterset_fields=["name"] so DjangoFilterBackend honors ?name=. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(platform-api): expose Tag POST/PATCH/DELETE handlers Tag URLs previously bound only GET on list/detail, so the migrator (and any other API consumer) got 405 on POST tags/. Wire create/partial_update/ destroy through the same TagViewSet — permission_classes already cover the auth path; no behavior change for callers that only GET. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(platform-api): mark connector_mode as read-only ConnectorInstanceSerializer.to_representation overrides connector_mode from the catalog every response, so any client-supplied value is silently discarded. Make that explicit via extra_kwargs so DRF OPTIONS reports the field as read-only and round-trippers don't trip the choice validator (catalog mode 'FILESYSTEM' vs model choice 'FILE_SYSTEM'). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(platform-api): support ?workflow_name= filter on workflow list Needed by the org-to-org migration SDK's idempotent get-or-create flow: without a name filter the SDK had to list-all-then-linear-search every time. Adds WorkflowKey.WF_NAME to the existing FilterHelper.build_filter_args call — same shape as the recently-added adapter_name / connector_name / tag name filters. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(platform-api): allow filtering PromptStudioRegistry by custom_tool Needed by the org-to-org migration SDK: after CustomTool migration the target's PromptStudioRegistry gets a freshly-minted prompt_registry_id, and downstream ToolInstance migration needs to remap source.tool_id -> target.tool_id (both are registry UUIDs). With this filter the SDK can GET /prompt-studio/registry/?custom_tool=<tool_id> on either side to resolve the registry id without needing to know it up front. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(platform-api): support ?pipeline_name= and ?api_name= filters Mirrors the existing exact-match name filters added for adapter, connector, tag, and workflow lists. Migration SDK's get-or-create flow queries by name on the target before deciding fresh vs adopt; without this the SDK had to list-all-then-linear-scan every time. Pipeline keeps the icontains ?search= alongside; api_deployment keeps the icontains ?search= alongside. New filters are exact-match only. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tool-instance): scope queryset widening to service accounts only Greptile flagged on PR #1987 that the prior switch from `created_by=user` to `workflow__in=Workflow.for_user(user)` silently widened tool-instance visibility for regular users with shared-workflow access — they would start seeing every member's tool instances in those workflows, not just their own. Restrict the widened queryset to `is_service_account` callers so the migration SDK still gets org-wide enumeration via Platform API keys, while regular users keep their original per-creator scope. Reported-by: greptile-apps[bot] (PR #1987, P1) * chore(platform-api): tighten comments to be generic and concise Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(platform-api): allow service accounts to manage groups Group CRUD, member add/remove, and the admin-only resources action were gated on the org-admin role, which is deliberately False for service accounts — blocking platform-key automation (org-to-org clone needs to recreate group shells + memberships). Service accounts bypass authorization here for the same reason they do in ShareAuthorizationService: they already bypass other access controls. - Add _is_admin_or_service_account() write gate in group_views and use it in IsOrgAdminForWrite plus the inline members/remove_member/resources checks and the list ?member=<id> filter. - is_org_admin() in sharing_helpers is intentionally unchanged; it drives resource-visibility semantics where service accounts are handled separately. - Verified GET /users/ already works for platform keys (no admin gate; returns id/email/role/is_admin) — no change needed. - Expose is_service_account in the GET /users/ and groups/{pk}/members/ listing rows so API clients can distinguish platform-key identities without inferring from the email suffix. The users listing itself still excludes service accounts (unchanged); select_related the user to avoid N+1 in member serialization. - Tests: service-account create/add/remove/resources/delete-group, non-admin member 403 matrix, listing flag/exclusion cases (23 green via manage.py test). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Apply suggestions from code review Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Signed-off-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> * UN-2977 [REFACTOR] Extract `_is_service_account` predicate Mirror the standalone `_is_org_admin` helper so the write gate composes two named predicates instead of inlining the service-account check (review nit). Behavior unchanged; `tenant_account_v2` suite green (23/23). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Platform API gaps surfaced while building the org-to-org data-migration SDK (companion:
Zipstack/unstract-python-client#15). Backend-only; no UI changes.0648934ac):AdapterInstanceModelManager.for_userfiltersis_friction_less=Trueout of the service-account queryset (frictionless onboarding adapters never migrate).IsFrictionLessAdaptershort-circuits to allow service accounts on non-frictionless adapters.PromptAcesssToUserpasses service accounts through (Prompt Studio access via Platform API key).tool_instance_v2.views.get_querysetroutes throughWorkflow.for_userso service accounts see workflow-scoped tool instances.?adapter_name=filter on adapter list (c05dc0561) viaFilterHelper.build_filter_args.tags/urls.py(5af6baac1).?name=filter on connector + tag list (5e02e99d5).connector_moderead-only inConnectorInstanceSerializer(9b1d6c526) — server-derived, must not be settable from client.?workflow_name=filter on workflow list (34e9db70a).?custom_tool=filter onPromptStudioRegistrylist (4538894f1).?pipeline_name=/?api_name=filters on pipeline + api-deployment list (f53959e98).Why
The SDK migrates resources by
list (filter by name) → adopt-if-exists or POSTagainst the target org. Without these the service-account-issued Platform API key either 403s on non-owned rows or can't filter by name. Tag CRUD was previously read-only — the SDK needs to create tags on the target.connector_modebeing writable is a latent bug (server-derived) and was tightened while touching the serializer.Tracked in UN-3479.
How
get_queryset(), exact-match, behind explicit param presence checks. Existing flows (no param) are unchanged._is_service_account(user)— regular user flows continue to hit the existingIsOwner/IsOwnerOrSharedUser/IsOwnerOrSharedUserOrSharedToOrgchecks.Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)
No. Changes are strictly additive for end users:
is_service_account=True; regular user authorisation is untouched.connector_modebecoming read-only matches what the backend already enforces inperform_create— there is no caller in the OSS or cloud frontends that sets it.Database Migrations
None.
Env Config
None.
Relevant Docs
~/Documents/Obsidian Vault/zipstuff/org-data-migration/Related Issues or PRs
Zipstack/unstract-python-client#15(feat/org-migration).Dependencies Versions
None.
Notes on Testing
org_Q4qgjLWIbaJlfSts→org_migration_target): adapters, connectors, tags, custom tools, workflows, tool instances, workflow endpoints, 12 ETL/TASK pipelines, 3 API deployments — all create on first run, adopt idempotently on re-run. Auto-provisioned keys verified 1:1 on target.unstract-migrate migrate --source-url ... --source-org ... --target-url ... --target-org ...(Platform API keys via env:UNSTRACT_MIGRATION_SOURCE_KEY/UNSTRACT_MIGRATION_TARGET_KEY).Screenshots
N/A — backend-only.
Checklist
I have read and understood the Contribution Guidelines.