Skip to content

[FEAT] Platform API gaps for org-to-org clone: service-account group management + name filters - #2044

Merged
chandrasekharan-zipstack merged 15 commits into
mainfrom
feat/org-migration-platform-api-gaps
Jun 15, 2026
Merged

[FEAT] Platform API gaps for org-to-org clone: service-account group management + name filters#2044
chandrasekharan-zipstack merged 15 commits into
mainfrom
feat/org-migration-platform-api-gaps

Conversation

@chandrasekharan-zipstack

Copy link
Copy Markdown
Contributor

What

Service-account (platform-key) support for org-to-org clone automation:

  • Service accounts allowed through group CRUD/membership write gates
  • ?name= filters on adapter, connector, tag, workflow, pipeline, api-deployment list endpoints
  • Tag write handlers + is_service_account flag exposed on GET /users/ and group members listing
  • N+1 fix: select_related("user") on group-member queries

Why

The org-to-org clone CLI (unstract clone in unstract-python-client) and customer automation authenticate with platform keys → service accounts. UN-2977 group CRUD/membership writes were admin-gated; is_org_admin() deliberately returns False for service accounts, blocking group cloning and post-clone membership automation. Same bypass doctrine as ShareAuthorizationService: service accounts bypass authorization — they already bypass other access controls.

How

  • _is_admin_or_service_account helper in tenant_account_v2/group_views.py gates group writes
  • is_service_account property on User model, exposed on list endpoints
  • Users list endpoint still EXCLUDES service accounts (behavior unchanged)
  • Merge of origin/main (UN-2977 group sharing) resolved 7 conflicts; dropped one branch-side change (tool-instance queryset widening, now covered by Workflow.for_user)

Can this PR break any existing features. If yes, please list possible items. If no, please explain why.

No. Service accounts were previously blocked from group writes and are now allowed through explicit gates. The gates mirror existing authorization patterns (admin checks). Service account filtering/exposure is read-only and optional (new flags only returned if service account checks pass). Users list endpoint behavior unchanged.

Database Migrations

None required.

Env Config

None required.

Relevant Docs

  • Org-to-org clone CLI: companion PR in Zipstack/unstract-python-client (branch feat/unstract-cli-groups) consumes these endpoints

Related Issues or PRs

  • UN-2977: group sharing foundational work (merged, conflicts resolved on this branch)
  • unstract-python-client branch: feat/unstract-cli-groups (companion PR)

Dependencies Versions

None changed.

Notes on Testing

manage.py test tenant_account_v2 → 23 tests OK:

  • Service-account create/add/remove group membership
  • Service-account resource listing with is_service_account flag
  • Service-account delete-group
  • Non-admin 403 matrix validation
  • Listing flag/exclusion behavior

E2E on chandru-unstract-dev namespace: full unstract clone run cloned 3 groups (including empty group), memberships email-matched (missing users skipped), group/user/org share state replicated and verified via API. Idempotent rerun adopted everything.

Screenshots

N/A

Checklist

I have read and understood the Contribution Guidelines.

🤖 Generated with Claude Code

chandrasekharan-zipstack and others added 12 commits May 24, 2026 00:27
…l instances

Enables the unstract-python-client SDK migration subpackage to drive
org-to-org data migration purely through admin-issued Platform API keys.

Specifically:
- adapter_processor_v2/models.py: AdapterInstanceModelManager.for_user
  returns non-frictionless adapters for service-account callers (was: all())
- permissions/permission.py: IsFrictionLessAdapter grants access to service
  accounts on non-frictionless adapters, keeping the friction-first check
- prompt_studio/permission.py: PromptAcesssToUser short-circuits to True
  for service accounts so Platform API can GET/POST prompts
- tool_instance_v2/views.py: get_queryset scopes via Workflow.for_user so
  service accounts see all tool instances under workflows they can access

Plan: org-to-org data migration v1 (KB: zipstuff/org-data-migration/05).
The SDK migration subpackage relies on list-by-name GET as the cross-run
idempotency check (Layer 2). Without this filter, every re-run would
re-create adapters that already exist on the target org.

Plan: org-to-org data migration v1 (KB: zipstuff/org-data-migration/05).
Mirror the adapter pattern from c05dc05: SDK migrator uses name-based
GET against target to detect existing rows before POST.

- ConnectorInstanceViewSet.get_queryset: thread CIKey.CONNECTOR_NAME
  through FilterHelper.
- TagViewSet: declare filterset_fields=["name"] so DjangoFilterBackend
  honors ?name=.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tag URLs previously bound only GET on list/detail, so the migrator (and
any other API consumer) got 405 on POST tags/. Wire create/partial_update/
destroy through the same TagViewSet — permission_classes already cover
the auth path; no behavior change for callers that only GET.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ConnectorInstanceSerializer.to_representation overrides connector_mode
from the catalog every response, so any client-supplied value is silently
discarded. Make that explicit via extra_kwargs so DRF OPTIONS reports the
field as read-only and round-trippers don't trip the choice validator
(catalog mode 'FILESYSTEM' vs model choice 'FILE_SYSTEM').

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Needed by the org-to-org migration SDK's idempotent get-or-create flow:
without a name filter the SDK had to list-all-then-linear-search every
time. Adds WorkflowKey.WF_NAME to the existing FilterHelper.build_filter_args
call — same shape as the recently-added adapter_name / connector_name /
tag name filters.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Needed by the org-to-org migration SDK: after CustomTool migration the
target's PromptStudioRegistry gets a freshly-minted prompt_registry_id,
and downstream ToolInstance migration needs to remap source.tool_id ->
target.tool_id (both are registry UUIDs). With this filter the SDK can
GET /prompt-studio/registry/?custom_tool=<tool_id> on either side to
resolve the registry id without needing to know it up front.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the existing exact-match name filters added for adapter,
connector, tag, and workflow lists. Migration SDK's get-or-create flow
queries by name on the target before deciding fresh vs adopt; without
this the SDK had to list-all-then-linear-scan every time.

Pipeline keeps the icontains ?search= alongside; api_deployment keeps
the icontains ?search= alongside. New filters are exact-match only.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Greptile flagged on PR #1987 that the prior switch from
`created_by=user` to `workflow__in=Workflow.for_user(user)` silently
widened tool-instance visibility for regular users with shared-workflow
access — they would start seeing every member's tool instances in those
workflows, not just their own.

Restrict the widened queryset to `is_service_account` callers so the
migration SDK still gets org-wide enumeration via Platform API keys,
while regular users keep their original per-creator scope.

Reported-by: greptile-apps[bot] (PR #1987, P1)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…tform-api-gaps

Conflict resolutions:
- permissions/permission.py, prompt_studio/permission.py: kept main's
  owner/admin/group-share logic; branch's service-account allowances
  already merged cleanly above the conflicts.
- adapter_processor_v2/models.py: took main's admin short-circuit +
  group-share visibility in for_user (defines group_shared_ids used below).
- connector_v2/serializers.py: kept main's shared_users/shared_to_org
  read-only kwargs alongside branch's read-only connector_mode.
- tool_instance_v2/views.py: dropped branch's service-account-only
  queryset widening as redundant — main's created_by | accessible-workflows
  union already grants org-wide access to service accounts via
  Workflow.for_user's short-circuit.
- api_v2/api_deployment_views.py, pipeline_v2/views.py: comment-only
  conflicts; kept branch's generic phrasing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Group CRUD, member add/remove, and the admin-only resources action were
gated on the org-admin role, which is deliberately False for service
accounts — blocking platform-key automation (org-to-org clone needs to
recreate group shells + memberships). Service accounts bypass
authorization here for the same reason they do in
ShareAuthorizationService: they already bypass other access controls.

- Add _is_admin_or_service_account() write gate in group_views and use it
  in IsOrgAdminForWrite plus the inline members/remove_member/resources
  checks and the list ?member=<id> filter.
- is_org_admin() in sharing_helpers is intentionally unchanged; it drives
  resource-visibility semantics where service accounts are handled
  separately.
- Verified GET /users/ already works for platform keys (no admin gate;
  returns id/email/role/is_admin) — no change needed.
- Expose is_service_account in the GET /users/ and groups/{pk}/members/
  listing rows so API clients can distinguish platform-key identities
  without inferring from the email suffix. The users listing itself
  still excludes service accounts (unchanged); select_related the user
  to avoid N+1 in member serialization.
- Tests: service-account create/add/remove/resources/delete-group,
  non-admin member 403 matrix, listing flag/exclusion cases (23 green
  via manage.py test).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d6940476-93fc-4f5e-8043-3a88efa92f5f

📥 Commits

Reviewing files that changed from the base of the PR and between a47116c and 2b7d206.

📒 Files selected for processing (1)
  • backend/tenant_account_v2/group_views.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • backend/tenant_account_v2/group_views.py

Summary by CodeRabbit

  • New Features
    • Service accounts can now manage groups and group membership (including membership changes and group-shared resource access) alongside organization admins.
    • Member and group-member API responses now include an is_service_account boolean so clients can distinguish service accounts from human users.
  • Bug Fixes / Improvements
    • Improved efficiency when retrieving organization members with service-account filtering.
  • Tests
    • Added coverage for service-account permissions, group sharing access, and user/group member listing (including is_service_account values).

Walkthrough

This PR extends service account authorization for group management operations. Service accounts can now perform group creation, member management, and resource listing operations alongside organization admins. The API serializers expose is_service_account fields so clients can distinguish service accounts from human users. A new authorization helper gate enables writes for both service accounts and org admins across affected endpoints.

Changes

Service Account Authorization for Group Operations

Layer / File(s) Summary
Service account authorization helpers and permission gates
backend/tenant_account_v2/group_views.py
Introduced _is_service_account and _is_admin_or_service_account helpers that permit write access when request.user.is_service_account is true, otherwise defer to existing org-admin check. Updated IsOrgAdminForWrite.has_permission and all group endpoints (list, members POST, remove_member, resources) to use the expanded authorization gate.
Service account field exposure in API serializers
backend/tenant_account_v2/group_serializers.py, backend/tenant_account_v2/serializer.py
GroupMemberSerializer and OrganizationMemberSerializer now expose is_service_account as read-only boolean fields sourced from user.is_service_account, enabling API consumers to identify service accounts in responses.
Query optimization for member listing
backend/tenant_account_v2/organization_member_service.py
get_members() now uses select_related("user") when filtering out service accounts, combining user data in a single database query.
Service account authorization and member listing tests
backend/tenant_account_v2/tests.py
Added test utilities imports and two new test classes: GroupViewSetServiceAccountTests verifies service accounts can create groups, manage members, and list resources while non-admin humans receive 403 on write endpoints, and OrganizationMembersListingTests validates user listing excludes service accounts with correct is_service_account flags.
Comment clarifications
backend/api_v2/api_deployment_views.py, backend/connector_v2/serializers.py, backend/pipeline_v2/views.py
Updated inline comments clarifying exact-match query filtering and field sourcing in three viewset/serializer methods; no logic changes.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main changes: service-account group management and name filters for org-to-org clone automation.
Description check ✅ Passed The description comprehensively covers all required template sections with substantial detail: What (service-account group support + name filters), Why (org-to-org clone requirements), How (implementation approach), feature impact assessment, testing results, and related issues.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/org-migration-platform-api-gaps

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chandrasekharan-zipstack
chandrasekharan-zipstack marked this pull request as ready for review June 12, 2026 04:32
@greptile-apps

greptile-apps Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR opens group CRUD and membership write endpoints to service accounts (platform-key auth) for org-to-org clone automation, and exposes is_service_account on group-member and user-listing responses. The auth change is implemented via a _is_admin_or_service_account helper that mirrors the bypass doctrine already in ShareAuthorizationService.

  • Adds _is_service_account / _is_admin_or_service_account helpers in group_views.py and threads them into all five write gates (create group, bulk-add member, remove member, delete group, resources view), replacing the prior admin-only checks.
  • Exposes is_service_account: bool on GroupMemberSerializer and OrganizationMemberSerializer; the /users/ endpoint still excludes service accounts from the queryset so the flag is always False there — it is meaningful in the group-members listing where service accounts can appear.
  • Applies a select_related(\"user\") N+1 fix to OrganizationMemberService.get_members(); the sibling get_members_by_role() was not updated and still issues per-row user queries.

Confidence Score: 5/5

Safe to merge; the gate change is narrowly scoped to service accounts and mirrors the existing bypass pattern in ShareAuthorizationService, with no write paths left ungated.

All five write gates are consistently updated to _is_admin_or_service_account. The is_service_account field additions are read-only and additive. The test suite covers the full permission matrix including 403 enforcement for plain members. No logic regressions identified.

backend/tenant_account_v2/organization_member_service.py — the select_related N+1 fix applied to get_members() was not carried over to the sibling get_members_by_role().

Important Files Changed

Filename Overview
backend/tenant_account_v2/group_views.py Adds _is_service_account + _is_admin_or_service_account helpers and wires them into all group write gates; the pattern mirrors the existing ShareAuthorizationService bypass doctrine.
backend/tenant_account_v2/organization_member_service.py Applies select_related("user") N+1 fix to get_members() only; the sibling get_members_by_role() still lacks it and issues per-row user queries when callers traverse member.user.
backend/tenant_account_v2/group_serializers.py Adds is_service_account read-only BooleanField to GroupMemberSerializer; already backed by select_related("user") in the members action queryset so no additional query.
backend/tenant_account_v2/serializer.py Exposes is_service_account on OrganizationMemberSerializer; always False in the /users/ listing since get_members() already filters out service accounts, but keeps the schema consistent with the group-members endpoint.
backend/tenant_account_v2/tests.py Adds 9 focused test cases covering the service-account write matrix, 403 enforcement for plain members, is_service_account flag in listings, and idempotent bulk-add; patch target is correct for the lazy-import pattern used in group_views.py.
backend/api_v2/api_deployment_views.py Minor comment reword on the api_name exact-match filter; no logic change.
backend/connector_v2/serializers.py Simplified inline comment on connector_mode's read_only rationale; no logic change.
backend/pipeline_v2/views.py Minor comment reword on the pipeline_name exact-match filter; no logic change.

Reviews (4): Last reviewed commit: "Merge branch 'main' into feat/org-migrat..." | Re-trigger Greptile

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@backend/tenant_account_v2/organization_member_service.py`:
- Around line 62-66: The get_members() query in OrganizationMemberService
currently only filters out service accounts and does not constrain by
organization, causing cross-tenant member leakage; update the call chain so
AuthenticationController.get_organization_members_by_org_id(org_id) accepts and
forwards an organization_id and change OrganizationMemberService.get_members to
accept an organization_id parameter and filter the queryset by
organization=organization_id (e.g.,
OrganizationMember.objects.select_related("user").filter(organization_id=organization_id,
user__is_service_account=False)), and ensure
OrganizationUserViewSet.get_organization_members passes the organization_id
through to AuthenticationController.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9cd2284f-454c-4dc1-abbc-7036aa706438

📥 Commits

Reviewing files that changed from the base of the PR and between cafcca7 and 60a5d08.

📒 Files selected for processing (8)
  • backend/api_v2/api_deployment_views.py
  • backend/connector_v2/serializers.py
  • backend/pipeline_v2/views.py
  • backend/tenant_account_v2/group_serializers.py
  • backend/tenant_account_v2/group_views.py
  • backend/tenant_account_v2/organization_member_service.py
  • backend/tenant_account_v2/serializer.py
  • backend/tenant_account_v2/tests.py

Comment thread backend/tenant_account_v2/organization_member_service.py
Comment thread backend/tenant_account_v2/group_views.py Outdated
Comment thread backend/tenant_account_v2/organization_member_service.py
Comment thread backend/tenant_account_v2/group_serializers.py Outdated
Comment thread backend/tenant_account_v2/organization_member_service.py Outdated
Comment thread backend/tenant_account_v2/serializer.py Outdated
Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com>
Signed-off-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com>
chandrasekharan-zipstack and others added 2 commits June 15, 2026 18:25
Mirror the standalone `_is_org_admin` helper so the write gate composes two
named predicates instead of inlining the service-account check (review nit).
Behavior unchanged; `tenant_account_v2` suite green (23/23).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

Unstract test results

Per-group results

Status Group Tier Passed Failed Errors Skipped Duration (s)
unit-connectors unit 64 12 0 3 17.1
unit-core unit 0 0 2 0 1.2
unit-platform-service unit 9 0 1 0 1.4
unit-prompt-service unit 15 0 0 0 19.9
unit-rig unit 53 0 0 0 3.3
unit-runner unit 11 0 0 0 3.2
unit-sdk1 unit 390 0 0 0 20.7
unit-tool-registry unit 0 0 1 0 1.3
unit-workers unit 0 0 0 0 18.4
TOTAL 542 12 4 3 86.5

Critical paths

⚠️ Critical paths not yet covered

  • auth-login — User can log in and obtain a session cookie. (entry: POST /api/v1/auth/login; declared coverage: no groups declared)
  • adapter-register-llm — Register and validate an LLM adapter. (entry: POST /api/v1/adapter/; declared coverage: no groups declared)
  • workflow-create-execute — Create a workflow, configure source+destination, execute, poll, fetch result. (entry: POST /api/v1/workflow/{id}/execute/; declared coverage: e2e-workflow)
  • api-deployment-run — Deploy a workflow as an API, POST a document, receive structured JSON. (entry: POST /deployment/api/{org}/{name}/; declared coverage: e2e-api-deployment)
  • prompt-studio-fetch-response — Prompt Studio: create project, add prompt, run single-pass, get response. (entry: POST /api/v1/prompt-studio/prompt-studio-tool/{id}/fetch_response/; declared coverage: e2e-prompt-studio)
  • pipeline-etl-execute — Run an ETL pipeline from source connector to destination. (entry: POST /api/v1/pipeline/{id}/execute/; declared coverage: no groups declared)
  • usage-token-tracking — Per-execution token usage is recorded and retrievable. (entry: GET /api/v1/usage/get_token_usage/; declared coverage: no groups declared)
  • workflow-execution-fan-out — Multi-file workflow execution fans out to file-processing workers and rejoins. (entry: internal: backend → rabbitmq → workers/file_processing; declared coverage: no groups declared)
  • callback-result-delivery — Async results are posted back via the callback worker. (entry: internal: workers/callback → backend /internal endpoints; declared coverage: no groups declared)
✅ Covered critical paths
  • tool-sandbox-exec — covered by unit-runner

@chandrasekharan-zipstack
chandrasekharan-zipstack merged commit 01d56dc into main Jun 15, 2026
9 checks passed
@chandrasekharan-zipstack
chandrasekharan-zipstack deleted the feat/org-migration-platform-api-gaps branch June 15, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants