Skip to content

Commit b6c600d

Browse files
jdye64charlesbluca
jdye64
and
charlesbluca
authored
Add conda.yaml baseline workflow file (apache#281)
* point to custom upstream arrow-datafusion with updated build script * Add conda.yaml baseline workflow file * Change conda recipe path * Add some debug output * fix syntax issues * update rustfmt path * test invoking rustfmt * Allow for RUSTFMT environment to pass through to the conda build * conda build updates * github workflow syntax issue * workflow syntax * more syntax * updates * use minimal rustup profile * use minimal rustup profile * use minimal rustup profile * use minimal rustup profile * use minimal rustup profile * changes * changes * trigger workflow rebuild * Add container * Ouput Rust env varialbes * full rustup command path * Point to typify fork for testing * Adjust conda CI workflow * Adjust conda build instead of mambabuild * Adjust command * Install conda-build * Some initial changes to get noarch builds working * Remove beta rust toolchain from CI * Add libprotobuf to host/build deps * Bump libprotobuf version * Revert back to 3 for libprotobuf --------- Co-authored-by: Charles Blackmon-Luca <20627856+charlesbluca@users.noreply.github.com>
1 parent fb3c6e2 commit b6c600d

6 files changed

Lines changed: 85 additions & 36 deletions

File tree

‎.github/workflows/conda.yml‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
name: Build conda nightly
2+
on: [push, pull_request]
3+
4+
# Cancel any already running instances of this build
5+
concurrency:
6+
group: conda-${{ github.head_ref }}
7+
cancel-in-progress: true
8+
9+
# Required shell entrypoint to have properly activated conda environments
10+
defaults:
11+
run:
12+
shell: bash -l {0}
13+
14+
jobs:
15+
conda:
16+
name: Build (and optionally upload) the conda nightly
17+
runs-on: ubuntu-latest
18+
steps:
19+
- uses: actions/checkout@v3
20+
with:
21+
fetch-depth: 0
22+
- name: Set up Python
23+
uses: conda-incubator/setup-miniconda@v2.2.0
24+
with:
25+
miniforge-variant: Mambaforge
26+
use-mamba: true
27+
python-version: "3.10"
28+
channel-priority: strict
29+
- name: Install dependencies
30+
run: |
31+
mamba install boa conda-verify
32+
33+
which python
34+
pip list
35+
mamba list
36+
- uses: actions-rs/toolchain@v1
37+
with:
38+
profile: minimal
39+
toolchain: stable
40+
override: true
41+
components: clippy, rustfmt
42+
- name: Build conda package
43+
run: |
44+
# suffix for nightly package versions
45+
export VERSION_SUFFIX=a`date +%y%m%d`
46+
47+
conda mambabuild conda/recipes \
48+
--no-anaconda-upload \
49+
--output-folder .
50+
# - name: Upload conda package
51+
# if: |
52+
# github.event_name == 'push'
53+
# && github.repository == 'apache/arrow-datafusion-python'
54+
# env:
55+
# ANACONDA_API_TOKEN: ${{ secrets.DASK_CONDA_TOKEN }}
56+
# LABEL: ${{ github.ref == 'refs/heads/datafusion-sql-planner' && 'dev_datafusion' || 'dev' }}
57+
# run: |
58+
# # install anaconda for upload
59+
# mamba install anaconda-client
60+
61+
# anaconda upload --label $LABEL linux-64/*.tar.bz2

‎.github/workflows/test.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,6 @@ jobs:
3636
- "3.10"
3737
toolchain:
3838
- "stable"
39-
- "beta"
4039
# we are not that much eager in walking on the edge yet
4140
# - nightly
4241
# build stable for only 3.7

‎Cargo.lock‎

Lines changed: 6 additions & 6 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎conda/recipes/build.sh‎

Lines changed: 0 additions & 20 deletions
This file was deleted.

‎conda/recipes/meta.yaml‎

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -15,32 +15,41 @@
1515
# specific language governing permissions and limitations
1616
# under the License.
1717

18-
{% set version = environ.get('GIT_DESCRIBE_TAG', '0.0.0.dev').lstrip('v') + environ.get('VERSION_SUFFIX', '') %}
19-
{% set minor_version = version.split('.')[0] + '.' + version.split('.')[1] %}
18+
{% set name = "datafusion" %}
19+
{% set major_minor_patch = environ.get('GIT_DESCRIBE_TAG', '0.0.0.dev').split('.') %}
20+
{% set new_patch = major_minor_patch[2] | int + 1 %}
21+
{% set version = (major_minor_patch[:2] + [new_patch]) | join('.') + environ.get('VERSION_SUFFIX', '') %}
22+
2023

2124
package:
22-
name: datafusion
25+
name: {{ name|lower }}
2326
version: {{ version }}
2427

2528
source:
2629
git_url: ../..
2730

2831
build:
2932
noarch: python
33+
script: {{ PYTHON }} -m pip install . -vv
3034
number: 0
3135

3236
requirements:
3337
host:
34-
- {{ compiler('rust') }}
35-
- python
36-
- setuptools
37-
- maturin
38+
- python >=3.6
39+
- maturin >=0.14,<0.15
40+
- libprotobuf =3
41+
- pip
3842
run:
39-
- python
43+
- python >=3.6
44+
- pyarrow >=11.0.0
4045

4146
test:
4247
imports:
4348
- datafusion
49+
commands:
50+
- pip check
51+
requires:
52+
- pip
4453

4554
about:
4655
home: https://arrow.apache.org/datafusion

‎pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ classifier = [
4343
"Programming Language :: Rust",
4444
]
4545
dependencies = [
46-
"pyarrow>=6.0.1",
46+
"pyarrow>=11.0.0",
4747
]
4848

4949
[project.urls]

0 commit comments

Comments
 (0)