chore: Fix Dockerfiles to install feast correctly (feast-dev#2635)

achals · web-flow · commit cef4d3cc713c · 2022-05-04T10:50:32.000-07:00
* chore: Fix Dockerfiles to install feast correctly

Also removes an unsued python script since we use the pypa/gh-action-pypi-publish action to build and publish wheels now.

Signed-off-by: Achal Shah &lt;achals@gmail.com&gt;

* more fixes

Signed-off-by: Achal Shah &lt;achals@gmail.com&gt;

* add a verify step before we publish to pypi wheel

Signed-off-by: Achal Shah &lt;achals@gmail.com&gt;

* more fixes

Signed-off-by: Achal Shah &lt;achals@gmail.com&gt;
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
@@ -127,7 +127,7 @@ jobs:
 
   publish-python-sdk:
     runs-on: ubuntu-latest
-    needs: [build-python-sdk, build-python-sdk-macos-py310]
+    needs: [verify-python-wheel]
     steps:
       - uses: actions/download-artifact@v2
         with:
@@ -139,20 +139,35 @@ jobs:
           password: ${{ secrets.PYPI_PASSWORD }}
 
 
+  verify-python-wheel:
+    runs-on: ubuntu-latest
+    needs: [build-python-sdk, build-python-sdk-macos-py310]
+    steps:
+      - name: Setup Python
+        id: setup-python
+        uses: actions/setup-python@v2
+        with:
+          python-version: "3.8"
+          architecture: x64
+      - uses: actions/download-artifact@v2
+        with:
+          name: wheels
+          path: dist
+      - name: Install wheel
+        run: pip install dist/*38*linux*.whl
+      - name: Install sdist
+        run: pip install dist/*tar.gz
+
   build-python-sdk:
     name: Build wheels on ${{ matrix.os }}
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
         os: [ ubuntu-latest, macos-10.15 ]
-
     steps:
       - uses: actions/checkout@v2
-
       - name: Build wheels
         uses: pypa/cibuildwheel@v2.4.0
-        with:
-          package-dir: sdk/python
         env:
           CIBW_BUILD: "cp3*_x86_64"
           CIBW_SKIP: "cp36-* *-musllinux_x86_64 cp310-macosx_x86_64"
@@ -195,13 +210,12 @@ jobs:
           make install-go-ci-dependencies
       - name: Build
         run: |
-          cd sdk/python
           python3 setup.py sdist bdist_wheel
 
       - uses: actions/upload-artifact@v2
         with:
           name: wheels
-          path: sdk/python/dist/*
+          path: dist/*
 
 
   publish-java-sdk:
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -91,7 +91,7 @@ pip install --upgrade pip
 
 4. Install development dependencies for Feast Python SDK / CLI
 ```sh
-pip install -e "sdk/python[dev]"
+pip install -e ".[dev]"
 ```
 
 ### Code Style & Linting
diff --git a/infra/scripts/publish-python-sdk.sh b/infra/scripts/publish-python-sdk.sh
diff --git a/java/serving/src/test/resources/docker-compose/feast10/Dockerfile b/java/serving/src/test/resources/docker-compose/feast10/Dockerfile
@@ -6,7 +6,7 @@ COPY requirements.txt ./
 RUN pip install --no-cache-dir -r requirements.txt
 
 RUN git clone https://github.com/feast-dev/feast.git /root/feast
-RUN cd /root/feast/sdk/python && pip install -e '.[redis]'
+RUN cd /root/feast/ && pip install -e '.[redis]'
 
 WORKDIR /app
 COPY . .
diff --git a/sdk/python/feast/infra/aws.py b/sdk/python/feast/infra/aws.py
@@ -347,7 +347,7 @@ def _get_docker_image_version() -> str:
                 f"version {get_version()}. If you want to update the Feast SDK version, make "
                 "sure to first fetch all new release tags from Github and then reinstall the library:\n"
                 "> git fetch --all --tags\n"
-                "> pip install -e sdk/python"
+                "> pip install -e '.'"
             )
         return version
 
diff --git a/sdk/python/feast/infra/transformation_servers/Dockerfile b/sdk/python/feast/infra/transformation_servers/Dockerfile
@@ -6,10 +6,14 @@ COPY sdk/python/feast/infra/transformation_servers/app.py app.py
 # Copy necessary parts of the Feast codebase
 COPY sdk/python sdk/python
 COPY protos protos
+COPY go go
+COPY setup.py setup.py
+COPY pyproject.toml pyproject.toml
 COPY README.md README.md
 
+
 # Install dependencies
-RUN pip3 install -e 'sdk/python'
+RUN pip3 install -e '.'
 
 # Start feature transformation server
 CMD [ "python", "app.py" ]
diff --git a/ui/feature_repo/README.md b/ui/feature_repo/README.md
@@ -18,7 +18,7 @@ git clone https://github.com/feast-dev/feast.git
 Install a dev build of feast
 ```bash
 cd feast
-pip install -e "sdk/python[dev]"
+pip install -e ".[dev]"
 ```
 
 Then for this demo, you'll actually need to fix a bug by adding this to `type_map.py#L144`:

Original file line number	Diff line number	Diff line change
`@@ -347,7 +347,7 @@ def _get_docker_image_version() -> str:`
`347`	`347`	`f"version {get_version()}. If you want to update the Feast SDK version, make "`
`348`	`348`	`"sure to first fetch all new release tags from Github and then reinstall the library:\n"`
`349`	`349`	`"> git fetch --all --tags\n"`
`350`		`- "> pip install -e sdk/python"`
	`350`	`+ "> pip install -e '.'"`
`351`	`351`	`)`
`352`	`352`	`return version`
`353`	`353`