Skip to content

feat: create project from cloud with initial data support#9194

Merged
AdityaHegde merged 26 commits intomainfrom
feat/create-project-from-cloud
Apr 23, 2026
Merged

feat: create project from cloud with initial data support#9194
AdityaHegde merged 26 commits intomainfrom
feat/create-project-from-cloud

Conversation

@AdityaHegde
Copy link
Copy Markdown
Collaborator

@AdityaHegde AdityaHegde commented Apr 7, 2026

Adds support for creating a project from cloud. We show the same welcome page (with some features temporarily missing) as in rill dev. Creating a project will land the user on a generated canvas for now. In future this will land the user to the edit page.

Run localStorage.setItem("rill:welcome:enabled", "true") in the console with UI open and refresh to enable this. This is a temporary hack of a feature flag since our usual feature flag is project level.

Closes APP-800

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

Comment thread admin/server/projects.go Outdated
return id, nil
}

func (s *Server) createEmptyManagedRepo(ctx context.Context, org *database.Organization, project, ownerID string) (githubRepoID, instID *int64, mgdGitRepoID *string, remote, primaryBranch string, resErr error) {
Copy link
Copy Markdown
Member

@k-anshul k-anshul Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be simplified to just creating a managed repo.
We don't need to create a branch. If a branch does not exist on remote, runtime will create it. I added a change yesterday so that it pushes that branch immediately as well : #9229

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Comment thread admin/server/projects.go Outdated
runtime.EditTrigger,
)
}
if permissions.ManageDev { // TODO: add safeguard for editable deployments
Copy link
Copy Markdown
Member

@k-anshul k-anshul Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I didn't get context around this. This is set to false for deployments :

rill/admin/permissions.go

Lines 218 to 219 in 7249ecc

ManageDev: false,
ReadProvisionerResources: true,

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Borrowed this change from the bigger cloud editing PR. Without this I get action not allowed for the long standing watch resources/files endpoint.

@AdityaHegde AdityaHegde force-pushed the feat/create-project-from-cloud branch from 7bfabe4 to d04e305 Compare April 20, 2026 12:03
@AdityaHegde AdityaHegde marked this pull request as ready for review April 20, 2026 15:05
@AdityaHegde AdityaHegde requested a review from k-anshul April 20, 2026 15:06
Copy link
Copy Markdown
Contributor

UQXA:

  1. Trying to upload a local file led to this error message which wasn't directionally helpful and also, the Back button is a dead click so i'm stuck in this view

    Screenshot 2026-04-21 at 9.49.56 AM.png
  2. Actually, the connectors aren't working (not connecting) but i'm assuming it's unrelated to this ticket?

  3. If I enter a project name string, it will create the project. If I hit back on the browser back button, I see the Project Create page with a filled in auto-generated name: new_project. It should just be blank (I'm chatting with @Di7design right now on what we want to do here given we don't have the ability to set a Display Name for projects atm)

@AdityaHegde
Copy link
Copy Markdown
Collaborator Author

AdityaHegde commented Apr 22, 2026

1 & 2 doesnt always happen and never happend to me. My guess it is a local server resource issue. Lets keep an eye on stage.
The dead back click is on main as well. Will raise a separate PR. (#9275)

Will revisit the naming page and add display name in a follow up.

Copy link
Copy Markdown
Contributor

@ericpgreen2 ericpgreen2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Product / Design / UX

In priority order:

1. Working branch should probably be main, not dev. Before the first publish there's nothing in prod to protect, so starting on a dev branch is ceremony without governance value — and it's what's driving the stubbed-out machinery in publish-project.ts. Consider working directly on main with editable=true until first publish, then flipping main to non-editable and requiring dev branches for subsequent edits. The tradeoff is a dual-mode main (editable pre-publish, protected post-publish), but it collapses onboarding into a single commit path. Worth checking with the Platform team on whether this is feasible before more of publish-project.ts gets built out.

2. Commit message "Initial dashboard commit" is sometimes inaccurate. Tracing the code: ConnectorForm.svelte:saveConnector wires "Save" → onCloseonDonepublishProjectAndRedirect, so a user can publish after creating only a connector (no source, no dashboard). Suggest "Initial project setup" instead — covers all paths honestly.

3. No exit on project welcome. Once a user lands on /-/welcome, the +layout.ts forces them back every time they try to navigate elsewhere (until the welcome branch is cleared). If a user creates a project by mistake, there's no discoverable escape hatch. Product may have intentionally chosen to force users through the funnel — flagging for discussion. If they do want an exit, a "Cancel" or "Delete this project" affordance would do it.

4. Form field labeled "URL". CreateProjectForm.svelte labels the name input "URL" with a prefix of https://ui.rilldata.com/{org}/. Users are naming a project, not entering a URL — the label should be "Project name". The URL preview is a nice affordance but the label is misleading.

5. "Create new" button copy + placement. In ProjectCards.svelte the button lives inside a sentence ("Check out your projects below.") and is labeled just "Create new". It reads awkwardly next to the hint text and would be clearer as its own heading row with a full "Create project" label. Same button on [organization]/+page.svelte.

6. allowEmpty={false} on project welcome. The cloud welcome hides the "Start a blank project" card, which is reasonable since the user already named the project. But there's no "skip and start empty in code" path visible. Intentional for now?


Engineering Implementation

In priority order:

1. Bugs.

  • web-common/src/features/add-data/manager/GenerateDashboardStatus.svelte:185onClick={() => onDone} returns the function reference instead of invoking it. Should be onClick={onDone}.
  • web-common/src/features/add-data/form/ConnectorForm.svelte:95if (config) return goto(...) is a no-op gate: config is a required prop so it's always truthy. Probably intended if (!config.skipNavigation). The current code navigates to /files/... in the cloud flow, which is a broken route.
  • web-common/src/features/add-data/manager/steps/connector.tsmaybeInitProject drops the olapEngine detection entirely and now always unpacks with the default (DuckDB). This also affects SourceForm.svelte, which is used outside the cloud onboarding flow. If a rill-dev user adds ClickHouse as their first connector, olap_connector will no longer auto-route. Intentional or collateral?

2. setTimeout(() => goto(...)) is being papered on. This PR adds two new instances (publish-project.ts:24, create-project/+page.svelte:46), and the pattern already exists in -/welcome/organization/+page.svelte, OrgNameSettings.svelte, and ProjectNameSettings.svelte. Five workarounds for the same "navigation cancelled" symptom suggests something structural is wrong — likely a load function throwing, an in-flight query being invalidated, or a reactive statement re-running. Worth tracing the root cause in a follow-up rather than adding a sixth.

3. projectWelcomeStatusStores — is sessionStorage the right backing? Welcome state is tracked in a per-tab sessionStorage store, which means: open the project in another tab, no welcome; close the tab after abandoning, welcome state is lost; can't resume on another device. The backend already knows whether a project is unpublished (skipDeploy, no primary deployment). Consider deriving "show welcome" from server state rather than maintaining client state that can drift from reality.

4. FileAndResourceWatcher duplication with PR 8912's edit route. Both the welcome layout and the edit layout wrap children in FileAndResourceWatcher + runtime resolution. Not a blocker for this PR — once both land we should consolidate into a shared layout or component so the authoring surfaces share one runtime-watching entry point.

5. config.skipNavigation naming is confusing. The flag is used in two opposite-looking conditions across GenerateDashboardStatus.svelte (if (!config.skipNavigation) return goto(...) vs. {#if config.skipNavigation} around the Skip button). Each may be individually defensible, but together they're hard to reason about. Consider renaming to something intent-based (e.g. autoNavigateOnDone for the auto-redirect, showSkipButton for the button visibility), and splitting if they're independent concerns.

6. Reliability nits.

  • projectWelcomeStatusStores.welcomeStatus is a Map keyed by project name that grows unbounded per session. If we keep sessionStorage, skip the in-memory cache — storage is already the source of truth.
  • ProjectGlobalStatusIndicator.svelte adds optional chaining to projectParser.state.parseErrors. Likely defensive-fixing a crash in the new flow (parser state not yet populated). Worth a one-line comment or finding the root cause so this doesn't silently swallow future bugs.

7. Test coverage. Not a blocker for merging, but before we ship to users this flow needs a component test for CreateProjectForm (happy path + form error mapping) and an e2e covering the full welcome → connect → publish loop. The e2e can wait until the Cloud Editing PR is in.


Developed in collaboration with Claude Code

@AdityaHegde
Copy link
Copy Markdown
Collaborator Author

AdityaHegde commented Apr 22, 2026

Thanks @ericpgreen2

Addressing UXQA items

  1. Changing the environment from dev to prod will basically be rebuilding the project. This step will not publish the project, rather there will be an explicit user action. Project that do not have a prod deployment will show this https://www.figma.com/design/JtG3sbaopjO0xQlyeCjmho/RILL-WIP?node-id=7406-48218&m=dev (Will do a follow up that integrates with your PR and include this) I have renamed publishProjectAndRedirect to checkpointProjectAndRedirect to match the intent.
  2. Updated.
  3. Ya there is no cancel/delete right now. User can delete from list of projects in the future: https://www.figma.com/design/JtG3sbaopjO0xQlyeCjmho/RILL-WIP?node-id=7535-232261&m=dev
  4. Updated to Name
  5. There is a greater refactor of the page planned in a follow up: https://www.figma.com/design/JtG3sbaopjO0xQlyeCjmho/RILL-WIP?node-id=7535-232238&m=dev . This is a temporary button.
  6. Start blank project is meant to take the user directly to the editor, will be added in the integration PR along with create data using AI block.

Addressing code issues,

  1. a,b updated. c is intended, passing olap here is meant to be a way to create rill managed olap. Our add data forms do not support this yet. Passing olap: clickhouse when user creates a clickhouse connector actually breaks the flow by creating a self managed clickhouse 1st.
  2. Ya will do a follow up.
  3. This is meant to allow user to go back to org page and come back to welcome screen. Checking presence of only an editable instance will be in a follow up that implements https://www.figma.com/design/JtG3sbaopjO0xQlyeCjmho/RILL-WIP?node-id=7406-48219&m=dev . Once I integrate with your changes, there can be an additional check very similar to rill-dev for presence of rill.yaml.
  4. Yes this is meant to just bootstrap this PR. Will have everything under edit routes in the integration PR.
  5. This is a temporary variable, with your PR the routes will automatically redirect to the correct edit pages. The onDone can just do a checkpoint commit.
  6. Done for both.
  7. Once we figure out github support in E2E we can add this. Component tests for CreateProjectForm is not needed, we can update org-management.spec.ts to include project creation and initialisation.

@AdityaHegde AdityaHegde requested a review from ericpgreen2 April 22, 2026 11:35
Copy link
Copy Markdown
Contributor

@ericpgreen2 ericpgreen2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, though I do think we should come back to my point 1. I think it should be possible for us to provision the main branch first as an editable dev deployment, then later promote it (and scale it up) to an non-editable prod deployment.

@AdityaHegde
Copy link
Copy Markdown
Collaborator Author

AdityaHegde commented Apr 22, 2026

@ericpgreen2 Sure, I can updated it to be on main branch. But we will need to recreate deployments.

I am seeing some issues when using main as non prod deployment, changes are getting erased and no dashboard gets created. I have made it simple to swap the branch so we can revisit this.

@AdityaHegde AdityaHegde force-pushed the feat/create-project-from-cloud branch 2 times, most recently from d0f5e1e to 9421d07 Compare April 22, 2026 15:59
@AdityaHegde AdityaHegde force-pushed the feat/create-project-from-cloud branch from 9421d07 to ae84952 Compare April 22, 2026 16:13
@AdityaHegde AdityaHegde merged commit a19614d into main Apr 23, 2026
12 checks passed
@AdityaHegde AdityaHegde deleted the feat/create-project-from-cloud branch April 23, 2026 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants