Skip to content

Commit 34d3635

Browse files
tokokotokoko
authored
chore: Change dev workflow to use uv (and pixi) both for lock and install (feast-dev#4166)
* change dev workflow to use uv (and pixi) Signed-off-by: tokoko <togurg14@freeuni.edu.ge> * pin ibis versions Signed-off-by: tokoko <togurg14@freeuni.edu.ge> * remove redundant make command Signed-off-by: tokoko <togurg14@freeuni.edu.ge> --------- Signed-off-by: tokoko <togurg14@freeuni.edu.ge>
1 parent e88f1e3 commit 34d3635

11 files changed

Lines changed: 657 additions & 1082 deletions

File tree

‎Makefile‎

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ install-python-ci-dependencies-uv:
4747
python setup.py build_python_protos --inplace
4848

4949
lock-python-ci-dependencies:
50-
python -m piptools compile -U --extra ci --output-file sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
50+
uv pip compile --system --no-strip-extras setup.py --extra ci --output-file sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
5151

5252
package-protos:
5353
cp -r ${ROOT_DIR}/protos ${ROOT_DIR}/sdk/python/feast/protos
@@ -60,13 +60,15 @@ install-python:
6060
python setup.py develop
6161

6262
lock-python-dependencies:
63-
python -m piptools compile -U --output-file sdk/python/requirements/py$(PYTHON)-requirements.txt
63+
uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py$(PYTHON)-requirements.txt
6464

6565
lock-python-dependencies-all:
66-
pixi run --environment py39 --manifest-path infra/scripts/pixi/pixi.toml "python -m piptools compile -U --output-file sdk/python/requirements/py3.9-requirements.txt"
67-
pixi run --environment py39 --manifest-path infra/scripts/pixi/pixi.toml "python -m piptools compile -U --extra ci --output-file sdk/python/requirements/py3.9-ci-requirements.txt"
68-
pixi run --environment py310 --manifest-path infra/scripts/pixi/pixi.toml "python -m piptools compile -U --output-file sdk/python/requirements/py3.10-requirements.txt"
69-
pixi run --environment py310 --manifest-path infra/scripts/pixi/pixi.toml "python -m piptools compile -U --extra ci --output-file sdk/python/requirements/py3.10-ci-requirements.txt"
66+
pixi run --environment py39 --manifest-path infra/scripts/pixi/pixi.toml "uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py3.9-requirements.txt"
67+
pixi run --environment py39 --manifest-path infra/scripts/pixi/pixi.toml "uv pip compile --system --no-strip-extras setup.py --extra ci --output-file sdk/python/requirements/py3.9-ci-requirements.txt"
68+
pixi run --environment py310 --manifest-path infra/scripts/pixi/pixi.toml "uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py3.10-requirements.txt"
69+
pixi run --environment py310 --manifest-path infra/scripts/pixi/pixi.toml "uv pip compile --system --no-strip-extras setup.py --extra ci --output-file sdk/python/requirements/py3.10-ci-requirements.txt"
70+
pixi run --environment py311 --manifest-path infra/scripts/pixi/pixi.toml "uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py3.11-requirements.txt"
71+
pixi run --environment py311 --manifest-path infra/scripts/pixi/pixi.toml "uv pip compile --system --no-strip-extras setup.py --extra ci --output-file sdk/python/requirements/py3.11-ci-requirements.txt"
7072

7173
benchmark-python:
7274
FEAST_USAGE=False IS_TEST=True python -m pytest --integration --benchmark --benchmark-autosave --benchmark-save-data sdk/python/tests
@@ -359,9 +361,6 @@ kill-trino-locally:
359361
install-protoc-dependencies:
360362
pip install --ignore-installed protobuf==4.24.0 "grpcio-tools>=1.56.2,<2" mypy-protobuf==3.1.0
361363

362-
install-feast-ci-locally:
363-
pip install -e ".[ci]"
364-
365364
# Docker
366365

367366
build-docker: build-feature-server-python-aws-docker build-feature-transformation-server-docker build-feature-server-java-docker

‎docs/project/development-guide.md‎

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -123,43 +123,43 @@ Note that this means if you are midway through working through a PR and rebase,
123123
Setting up your development environment for Feast Python SDK / CLI:
124124
1. Ensure that you have Docker installed in your environment. Docker is used to provision service dependencies during testing, and build images for feature servers and other components.
125125
- Please note that we use [Docker with BuiltKit](https://docs.docker.com/develop/develop-images/build_enhancements/).
126-
2. Ensure that you have `make`, Python (3.8 and above) with `pip`, installed.
126+
2. Ensure that you have `make` and Python (3.9 or above) installed.
127127
3. _Recommended:_ Create a virtual environment to isolate development dependencies to be installed
128128
```sh
129129
# create & activate a virtual environment
130130
python -m venv venv/
131131
source venv/bin/activate
132132
```
133-
4. Upgrade `pip` if outdated
134-
```sh
135-
pip install --upgrade pip
136-
```
137-
5. (M1 Mac only): Follow the [dev guide](https://github.com/feast-dev/feast/issues/2105)
138-
6. Install pip-tools
139-
```sh
140-
pip install pip-tools
141-
```
142-
7. (Optional): Install Node & Yarn. Then run the following to build Feast UI artifacts for use in `feast ui`
133+
4. (M1 Mac only): Follow the [dev guide](https://github.com/feast-dev/feast/issues/2105)
134+
5. Install uv
135+
It is recommended to use uv for managing python dependencies.
136+
```sh
137+
curl -LsSf https://astral.sh/uv/install.sh | sh
138+
```
139+
or
140+
```ssh
141+
pip install uv
142+
```
143+
6. (Optional): Install Node & Yarn. Then run the following to build Feast UI artifacts for use in `feast ui`
143144
```
144145
make build-ui
145146
```
146-
8. Install mysql (needed for ci dependencies)
147+
7. (Optional) install pixi
148+
pixi is necessary to run step 8 for all python versions at once.
147149
```sh
148-
brew install mysql
150+
curl -fsSL https://pixi.sh/install.sh | bash
149151
```
150-
9. Install development dependencies for Feast Python SDK / CLI
152+
8. (Optional): Recompile python lock files
153+
If you make changes to requirements or simply want to update python lock files to reflect latest versioons.
151154
```sh
152-
pip install -e ".[dev]"
153-
```
154-
155-
This will allow the installed feast version to automatically reflect changes to your local development version of Feast without needing to reinstall everytime you make code changes.
156-
157-
10. Compile the protubufs
155+
make lock-python-dependencies-all
156+
```
157+
9. Install development dependencies for Feast Python SDK / CLI
158+
This will install package versions from the lock file, install editable version of feast and compile protobufs.
158159
```sh
159-
make compile-protos-python
160+
make install-python-ci-dependencies-uv
160161
```
161-
162-
11. Spin up Docker Image
162+
10. Spin up Docker Image
163163
```sh
164164
docker build -t docker-whale -f ./sdk/python/feast/infra/feature_servers/multicloud/Dockerfile .
165165
```

0 commit comments

Comments
 (0)