From 8167504ed4fee1e6dd6f4d5bd7950dad9e2a37ff Mon Sep 17 00:00:00 2001 From: Danny Chiao Date: Thu, 4 Aug 2022 15:22:07 -0500 Subject: [PATCH 1/6] docs: Add Web UI documentation on repo structure Signed-off-by: Danny Chiao --- ui/CONTRIBUTING.md | 97 +++++++++++++++++++++++++++++++++++++++++ ui/PUBLISHING_TO_NPM.md | 13 ------ ui/README.md | 36 +-------------- 3 files changed, 98 insertions(+), 48 deletions(-) create mode 100644 ui/CONTRIBUTING.md delete mode 100644 ui/PUBLISHING_TO_NPM.md diff --git a/ui/CONTRIBUTING.md b/ui/CONTRIBUTING.md new file mode 100644 index 0000000000..74e2f3a834 --- /dev/null +++ b/ui/CONTRIBUTING.md @@ -0,0 +1,97 @@ +

Table of contents

+ +- [General contributor](#general-contributor) + - [NPM package project structure](#npm-package-project-structure) + - [Tests](#tests) + - [Yarn commands](#yarn-commands) + - [`yarn install`](#yarn-install) + - [`yarn start`](#yarn-start) + - [`yarn test`](#yarn-test) +- [Release process](#release-process) + - [1) Publishing the Feast Package to NPM](#1-publishing-the-feast-package-to-npm) + - [Requirements](#requirements) + - [Steps for Publishing](#steps-for-publishing) + - [2) Linking to the new UI in the Feast Python SDK](#2-linking-to-the-new-ui-in-the-feast-python-sdk) + +# General contributor +In this doc, we describe how to contribute both to the Feast Web UI NPM package as well as the embedded Feast UI in the Python SDK (i.e. what's run when you run `feast ui`) + +## NPM package project structure +The Web UI is powered by a JSON registry dump from Feast (running `feast registry-dump`). Running `yarn start` launches a UI +powered by test data. +- `public/` contains assets as well as demo data loaded by the Web UI. + - There is a `projects-list.json` which represents all Feast projects the UI shows. + - There is also a `registry.json` which is the registry dump for the feature repo. +- `feature_repo/` contains a sample Feast repo which generates the `registry.json` +- `src/` contains the Web UI source code. + - `src/contexts` has React context objects around project level metadata or registry path metadata to inject into pages. The contexts are static contexts provided by [FeastUISansProviders.tsx](src/FeastUISansProviders.tsx) + - `src/parsers` parses the `registry.json` into in memory representations of Feast objects (feature views, data sources, entities, feature services). + - This has ~1:1 mappings to the protobuf objects in [feast/protos/feast/core](https://github.com/feast-dev/feast/tree/master/protos/feast/core). + - There are also "relationships" which create an in-memory lineage graph which can be used to construct links in pages. + - This generates state which pages will load via React queries (to the registry path). + - `src/pages` has all individual web pages and their layouts. For any given Feast object (e.g. entity), there exist: + - an **Index page** (which is the first page you hit when you click on that object). This loads using a React query the in memory representation of all objects (parsed from `src/parsers`) and embeds: + - a **Listing page** (i.e. listing all the objects in the registry in a table). This creates links to the instance pages + - an **Instance page** (which shows details for an individual entity, feature view, etc). This embeds: + - a default Overview tab, which shows all the Feast metadata (e.g. for a given entity) + - custom tabs from `src/custom-tabs`. + - Other subdirectories: + - `src/components` has common React components that are re-used across pages + - `src/custom-tabs` houses custom tabs and a custom tab React context which exist on the core pages. There is a `TabsRegistryContext` which is also supplied by the [FeastUISansProviders.tsx](src/FeastUISansProviders.tsx), and if there are custom tabs, the Feast UI will embed them as a new tab in the corresponding page (e.g. feature view page). + - `src/graphics` houses icons that are used throughout the UI + - `src/hooks` has React hooks. The most complex hooks here define the bulk of the search / filter functionality. + +## Tests +There are very few tests for this UI. There is a smoke test that ensures pages can load in [FeastUISansProviders.test.tsx](src/FeastUISansProviders.test.tsx) + + +## Yarn commands + +If you would like to simply try things out and see how the UI works, you can simply run the code in this repo. First: + +### `yarn install` + +That will install the all the dependencies that the UI needs, as well as development dependencies. Then in the project directory, you can run: + +### `yarn start` + +Runs the app in the development mode.\ +Open [http://localhost:3000](http://localhost:3000) to view it in the browser. + +The page will reload if you make edits.\ +You will also see any lint errors in the console. + +### `yarn test` + +Launches the test runner in the interactive watch mode.\ +See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information. + +# Release process +There are a couple of components in Feast that are tied to the Web UI: +1. the npm package +2. the Feast Python SDK, which bundles in a compiled version of the Feast Web UI which is run on a `feast ui` CLI command. + +**TODO(adchia): Update Feast's release process to automatically bump versions and publish npm packages as part of semantic release** + +## 1) Publishing the Feast Package to NPM + +The Feast UI is published as a module to NPM and can be found here: https://www.npmjs.com/package/@feast-dev/feast-ui + +### Requirements + +To publish a new version of the module, you will need: +- to be part of the @feast-dev team in NPM. Ask `#feast-development` on http://slack.feast.dev to add you if necessary. +- to [login to your NPM account on the command line](https://docs.npmjs.com/cli/v8/commands/npm-adduser). + +### Steps for Publishing + +1. Make sure tests are passing. Run tests with `yarn test` in the ui directory. +2. Bump the version number in `package.json` as appropriate. +3. Package the modules for distributions. Run the library build script with `yarn build:lib`. We use [Rollup](https://rollupjs.org/) for building the module, and the configs are in the `rollup.config.js` file. +4. Publish the package to NPM. Run `npm publish` +5. [Check NPM to see that the package was properly publish](https://www.npmjs.com/package/@feast-dev/feast-ui). + +## 2) Linking to the new UI in the Feast Python SDK +You can see the logic in [../sdk/python/ui](../sdk/python/feast/ui/). This instance is loaded in [../sdk/python/ui_server.py](../sdk/python/feast/ui_server.py). + +To ensure this gets released in the next Feast SDK patch release, update the `package.json` file in [../sdk/python/ui](../sdk/python/feast/ui/) (specifically bumping the version of @feast-dev/feast-ui to the new version). \ No newline at end of file diff --git a/ui/PUBLISHING_TO_NPM.md b/ui/PUBLISHING_TO_NPM.md deleted file mode 100644 index 0ab1af3923..0000000000 --- a/ui/PUBLISHING_TO_NPM.md +++ /dev/null @@ -1,13 +0,0 @@ -# Publishing the Feast Package to NPM - -The Feast UI is published as a module to NPM and can be found here: https://www.npmjs.com/package/@feast-dev/feast-ui - -To publish a new version of the module, you will need to be part of the @feast-dev team in NPM. Ask Tony to add you if necessary. You will also need to [login to your NPM account on the command line](https://docs.npmjs.com/cli/v8/commands/npm-adduser). - -## Steps for Publishing - -1. Make sure tests are passing. Run tests with `yarn test` in the ui directory. -2. Bump the version number in `package.json` as appropriate. -3. Package the modules for distributions. Run the library build script with `yarn build:lib`. We use [Rollup](https://rollupjs.org/) for building the module, and the configs are in the `rollup.config.js` file. -4. Publish the package to NPM. Run `npm publish` -5. [Check NPM to see that the package was properly publish](https://www.npmjs.com/package/@feast-dev/feast-ui). diff --git a/ui/README.md b/ui/README.md index 713d1c17c5..e91a8741ec 100644 --- a/ui/README.md +++ b/ui/README.md @@ -4,22 +4,10 @@ This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app). -## Project structure -The Web UI is powered by a JSON registry dump from Feast (running `feast registry-dump`). Running `yarn start` launches a UI -powered by test data. -- `public/` contains assets as well as demo data loaded by the Web UI. - - There is a `projects-list.json` which represents all Feast projects the UI shows. - - There is also a `registry.json` which is the registry dump for the feature repo. -- `feature_repo/` contains a sample Feast repo which generates the `registry.json` -- `src/` contains the Web UI source code. This parses the registry json blob in `src/parsers` to make this data -available for the rest of the UI. -- `src/custom-tabs` includes sample custom tabs. This is a WIP plugin system where users can inject their own tabs and -data to the UI. - ## Usage There are three modes of usage: -- via the 'feast ui' CLI to view the current feature repository +- via the `feast ui` CLI to view the current feature repository - importing the UI as a module - running the entire build as a React app. @@ -124,28 +112,6 @@ const tabsRegistry = { Examples of custom tabs can be found in the `/custom-tabs` folder. -### Alternative: Run this Repo - -If you would like to simply try things out and see how the UI works, you can simply run the code in this repo. First: - -### `yarn install` - -That will install the all the dependencies that the UI needs, as well as development dependencies. Then in the project directory, you can run: - -### `yarn start` - -Runs the app in the development mode.\ -Open [http://localhost:3000](http://localhost:3000) to view it in the browser. - -The page will reload if you make edits.\ -You will also see any lint errors in the console. - -### `yarn test` - -Launches the test runner in the interactive watch mode.\ -See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information. - - ## On React and Create React App This project was bootstrapped with Create React App, and uses its scripts to simplify UI development. You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started). From d3a6fb8a526663beeac1ce2b36f1fc0f6deb5144 Mon Sep 17 00:00:00 2001 From: Danny Chiao Date: Thu, 4 Aug 2022 16:06:32 -0500 Subject: [PATCH 2/6] fix Signed-off-by: Danny Chiao --- sdk/python/feast/ui/package.json | 2 +- ui/CONTRIBUTING.md | 18 +++++++++++++----- ui/package.json | 2 +- 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/sdk/python/feast/ui/package.json b/sdk/python/feast/ui/package.json index 883c19660b..358aa2cdd2 100644 --- a/sdk/python/feast/ui/package.json +++ b/sdk/python/feast/ui/package.json @@ -6,7 +6,7 @@ "@elastic/datemath": "^5.0.3", "@elastic/eui": "^57.0.0", "@emotion/react": "^11.9.0", - "@feast-dev/feast-ui": "^0.20.5", + "@feast-dev/feast-ui": "latest", "@testing-library/jest-dom": "^5.16.4", "@testing-library/react": "^13.2.0", "@testing-library/user-event": "^13.5.0", diff --git a/ui/CONTRIBUTING.md b/ui/CONTRIBUTING.md index 74e2f3a834..32521c4a44 100644 --- a/ui/CONTRIBUTING.md +++ b/ui/CONTRIBUTING.md @@ -1,6 +1,7 @@

Table of contents

-- [General contributor](#general-contributor) +- [General contributor notes](#general-contributor-notes) + - [`feast ui` command](#feast-ui-command) - [NPM package project structure](#npm-package-project-structure) - [Tests](#tests) - [Yarn commands](#yarn-commands) @@ -8,14 +9,19 @@ - [`yarn start`](#yarn-start) - [`yarn test`](#yarn-test) - [Release process](#release-process) - - [1) Publishing the Feast Package to NPM](#1-publishing-the-feast-package-to-npm) + - [Publishing the Feast Package to NPM](#publishing-the-feast-package-to-npm) - [Requirements](#requirements) - [Steps for Publishing](#steps-for-publishing) - [2) Linking to the new UI in the Feast Python SDK](#2-linking-to-the-new-ui-in-the-feast-python-sdk) -# General contributor +# General contributor notes In this doc, we describe how to contribute both to the Feast Web UI NPM package as well as the embedded Feast UI in the Python SDK (i.e. what's run when you run `feast ui`) +## `feast ui` command +You can see the logic in [../sdk/python/ui](../sdk/python/feast/ui/). This instance is loaded in [../sdk/python/ui_server.py](../sdk/python/feast/ui_server.py). + +Under the hood, what happens is that the Feast SDK spins up a server which exposes an endpoint to the registry. It then mounts the UI on the server and points it to fetch data from that registry. + ## NPM package project structure The Web UI is powered by a JSON registry dump from Feast (running `feast registry-dump`). Running `yarn start` launches a UI powered by test data. @@ -71,9 +77,11 @@ There are a couple of components in Feast that are tied to the Web UI: 1. the npm package 2. the Feast Python SDK, which bundles in a compiled version of the Feast Web UI which is run on a `feast ui` CLI command. +The bundled Web UI in the Python SDK always compiles in the latest npm version, so we only need to ensure the npm package is published correctly + **TODO(adchia): Update Feast's release process to automatically bump versions and publish npm packages as part of semantic release** -## 1) Publishing the Feast Package to NPM +## Publishing the Feast Package to NPM The Feast UI is published as a module to NPM and can be found here: https://www.npmjs.com/package/@feast-dev/feast-ui @@ -92,6 +100,6 @@ To publish a new version of the module, you will need: 5. [Check NPM to see that the package was properly publish](https://www.npmjs.com/package/@feast-dev/feast-ui). ## 2) Linking to the new UI in the Feast Python SDK -You can see the logic in [../sdk/python/ui](../sdk/python/feast/ui/). This instance is loaded in [../sdk/python/ui_server.py](../sdk/python/feast/ui_server.py). + To ensure this gets released in the next Feast SDK patch release, update the `package.json` file in [../sdk/python/ui](../sdk/python/feast/ui/) (specifically bumping the version of @feast-dev/feast-ui to the new version). \ No newline at end of file diff --git a/ui/package.json b/ui/package.json index 22128cc968..c5eddbfd2e 100644 --- a/ui/package.json +++ b/ui/package.json @@ -1,6 +1,6 @@ { "name": "@feast-dev/feast-ui", - "version": "0.20.5", + "version": "0.23.0", "private": false, "files": [ "dist" From 94ab0fbc70c43b9d78e7ad9ca1800bbd26149d10 Mon Sep 17 00:00:00 2001 From: Danny Chiao Date: Thu, 4 Aug 2022 16:51:28 -0500 Subject: [PATCH 3/6] update release docs Signed-off-by: Danny Chiao --- ui/CONTRIBUTING.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ui/CONTRIBUTING.md b/ui/CONTRIBUTING.md index 32521c4a44..5c72a84f5f 100644 --- a/ui/CONTRIBUTING.md +++ b/ui/CONTRIBUTING.md @@ -9,7 +9,7 @@ - [`yarn start`](#yarn-start) - [`yarn test`](#yarn-test) - [Release process](#release-process) - - [Publishing the Feast Package to NPM](#publishing-the-feast-package-to-npm) + - [(Advanced) Manually publishing the Feast Package to NPM](#advanced-manually-publishing-the-feast-package-to-npm) - [Requirements](#requirements) - [Steps for Publishing](#steps-for-publishing) - [2) Linking to the new UI in the Feast Python SDK](#2-linking-to-the-new-ui-in-the-feast-python-sdk) @@ -73,15 +73,15 @@ Launches the test runner in the interactive watch mode.\ See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information. # Release process -There are a couple of components in Feast that are tied to the Web UI: +There are a couple of components in Feast that are tied to the Web UI. These are all automatically handled during the release GitHub action: 1. the npm package + - The release process for Feast automatically bumps the package version (see [bump_file_versions.py](../infra/scripts/release/bump_file_versions.py)) and releases the new NPM package (see [publish.yml](../.github/workflows/publish.yml) in the `publish-web-ui-npm` job) 2. the Feast Python SDK, which bundles in a compiled version of the Feast Web UI which is run on a `feast ui` CLI command. + - The bundled Web UI in the Python SDK always compiles in the latest npm version -The bundled Web UI in the Python SDK always compiles in the latest npm version, so we only need to ensure the npm package is published correctly +## (Advanced) Manually publishing the Feast Package to NPM -**TODO(adchia): Update Feast's release process to automatically bump versions and publish npm packages as part of semantic release** - -## Publishing the Feast Package to NPM +This generally should not be necessary, since new package versions are released with the overall Feast release workflow (see [publish.yml](../.github/workflows/publish.yml) in the `publish-web-ui-npm` job) The Feast UI is published as a module to NPM and can be found here: https://www.npmjs.com/package/@feast-dev/feast-ui From 0a3945489dc35ba21aa0719f4e418163ed63984a Mon Sep 17 00:00:00 2001 From: Danny Chiao Date: Thu, 4 Aug 2022 16:52:36 -0500 Subject: [PATCH 4/6] update release docs Signed-off-by: Danny Chiao --- CONTRIBUTING.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6e575b31a0..ce2a8684ee 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -245,6 +245,8 @@ The services with containerized replacements currently implemented are: You can run `make test-python-integration-container` to run tests against the containerized versions of dependencies. +## Feast UI +See [Feast contributing guide](ui/CONTRIBUTING.md) ## Feast Java Serving See [Java contributing guide](java/CONTRIBUTING.md) From b0f06db22d7c62ad1c58c049749a74e61341f057 Mon Sep 17 00:00:00 2001 From: Danny Chiao Date: Thu, 4 Aug 2022 16:55:50 -0500 Subject: [PATCH 5/6] remove unnecessary step Signed-off-by: Danny Chiao --- ui/CONTRIBUTING.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/ui/CONTRIBUTING.md b/ui/CONTRIBUTING.md index 5c72a84f5f..35918db6de 100644 --- a/ui/CONTRIBUTING.md +++ b/ui/CONTRIBUTING.md @@ -12,7 +12,6 @@ - [(Advanced) Manually publishing the Feast Package to NPM](#advanced-manually-publishing-the-feast-package-to-npm) - [Requirements](#requirements) - [Steps for Publishing](#steps-for-publishing) - - [2) Linking to the new UI in the Feast Python SDK](#2-linking-to-the-new-ui-in-the-feast-python-sdk) # General contributor notes In this doc, we describe how to contribute both to the Feast Web UI NPM package as well as the embedded Feast UI in the Python SDK (i.e. what's run when you run `feast ui`) @@ -97,9 +96,4 @@ To publish a new version of the module, you will need: 2. Bump the version number in `package.json` as appropriate. 3. Package the modules for distributions. Run the library build script with `yarn build:lib`. We use [Rollup](https://rollupjs.org/) for building the module, and the configs are in the `rollup.config.js` file. 4. Publish the package to NPM. Run `npm publish` -5. [Check NPM to see that the package was properly publish](https://www.npmjs.com/package/@feast-dev/feast-ui). - -## 2) Linking to the new UI in the Feast Python SDK - - -To ensure this gets released in the next Feast SDK patch release, update the `package.json` file in [../sdk/python/ui](../sdk/python/feast/ui/) (specifically bumping the version of @feast-dev/feast-ui to the new version). \ No newline at end of file +5. [Check NPM to see that the package was properly publish](https://www.npmjs.com/package/@feast-dev/feast-ui). \ No newline at end of file From f4961bf1fe0d915d864ad3e1a5b310adbb503f1f Mon Sep 17 00:00:00 2001 From: Danny Chiao Date: Thu, 4 Aug 2022 17:25:52 -0500 Subject: [PATCH 6/6] fix nits Signed-off-by: Danny Chiao --- ui/CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ui/CONTRIBUTING.md b/ui/CONTRIBUTING.md index 35918db6de..de5cfc66ea 100644 --- a/ui/CONTRIBUTING.md +++ b/ui/CONTRIBUTING.md @@ -17,7 +17,7 @@ In this doc, we describe how to contribute both to the Feast Web UI NPM package as well as the embedded Feast UI in the Python SDK (i.e. what's run when you run `feast ui`) ## `feast ui` command -You can see the logic in [../sdk/python/ui](../sdk/python/feast/ui/). This instance is loaded in [../sdk/python/ui_server.py](../sdk/python/feast/ui_server.py). +You can see the logic in [../sdk/python/feast/ui](../sdk/python/feast/ui/). This instance is loaded in [../sdk/python/feast/ui_server.py](../sdk/python/feast/ui_server.py). Under the hood, what happens is that the Feast SDK spins up a server which exposes an endpoint to the registry. It then mounts the UI on the server and points it to fetch data from that registry. @@ -96,4 +96,4 @@ To publish a new version of the module, you will need: 2. Bump the version number in `package.json` as appropriate. 3. Package the modules for distributions. Run the library build script with `yarn build:lib`. We use [Rollup](https://rollupjs.org/) for building the module, and the configs are in the `rollup.config.js` file. 4. Publish the package to NPM. Run `npm publish` -5. [Check NPM to see that the package was properly publish](https://www.npmjs.com/package/@feast-dev/feast-ui). \ No newline at end of file +5. [Check NPM to see that the package was properly published](https://www.npmjs.com/package/@feast-dev/feast-ui). \ No newline at end of file