python: refactor README and remove Kafka dev dependency

snkas · snkas · commit a309e51951d7 · 2025-11-11T14:09:13.000Z
README changes:
- Example usage
- Explain environment variables
- General restructuring

Removes the `kafka-python-ng` Python dev dependency along with the
Kafka-related test environment variables as they are not used anywhere
currently.

Signed-off-by: Simon Kassing &lt;simon.kassing@feldera.com&gt;
diff --git a/python/README.md b/python/README.md
@@ -1,73 +1,139 @@
 # Feldera Python SDK
 
-Feldera Python is the Feldera SDK for Python developers.
+The `feldera` Python package is the Python client for the Feldera HTTP API.
 
-## Installation
+The Python SDK documentation is available at: https://docs.feldera.com/python
+
+## Getting started
+
+### Installation
 
 ```bash
 uv pip install feldera
 ```
 
-### Installing from Github
+### Example usage
+
+The Python client interacts with the API server of the Feldera instance.
+
+```python
+# File: example.py
+from feldera import FelderaClient, PipelineBuilder, Pipeline
+
+# Instantiate client
+client = FelderaClient()  # Default: http://localhost:8080 without authentication
+# client = FelderaClient(url="https://localhost:8080", api_key="apikey:...", requests_verify="/path/to/tls.crt")
+
+# (Re)create pipeline
+name = "example"
+sql = """
+CREATE TABLE t1 (i1 INT) WITH ('materialized' = 'true');
+CREATE MATERIALIZED VIEW v1 AS SELECT * FROM t1;
+"""
+print("(Re)creating pipeline...")
+pipeline = PipelineBuilder(client, name, sql).create_or_replace()
+pipeline.start()
+print(f"Pipeline status: {pipeline.status()}")
+pipeline.pause()
+print(f"Pipeline status: {pipeline.status()}")
+pipeline.stop(force=True)
+
+# Find existing pipeline
+pipeline = Pipeline.get(name, client)
+pipeline.start()
+print(f"Pipeline status: {pipeline.status()}")
+pipeline.stop(force=True)
+pipeline.clear_storage()
+```
 
+Run using:
 ```bash
-uv pip install git+https://github.com/feldera/feldera#subdirectory=python
+uv run python example.py
 ```
 
-Similarly, to install from a specific branch:
+### Environment variables
+
+Some default parameter values in the Python SDK can be overridden via environment variables.
+
+**Environment variables for `FelderaClient(...)`**
 
 ```bash
-uv pip install git+https://github.com/feldera/feldera@{BRANCH_NAME}#subdirectory=python
+export FELDERA_HOST="https://localhost:8080"  # Overrides default for `url`
+export FELDERA_API_KEY="apikey:..."  # Overrides default for `api_key`
+
+# The following together override default for `requests_verify`
+# export FELDERA_TLS_INSECURE="false"  # If set to "1", "true" or "yes" (all case-insensitive), disables TLS certificate verification
+# export FELDERA_HTTPS_TLS_CERT="/path/to/tls.crt"  # Custom TLS certificate
 ```
 
-Replace `{BRANCH_NAME}` with the name of the branch you want to install from.
+**Environment variables for `PipelineBuilder(...)`**
 
-### Installing from Local Directory
+```bash
+export FELDERA_RUNTIME_VERSION="..."  # Overrides default for `runtime_version`
+```
+
+## Development
 
-If you have cloned the Feldera repo, you can install the python SDK as follows:
+Development assumes you have cloned the Feldera code repository.
+
+### Installation
 
 ```bash
-# the Feldera Python SDK is present inside the python/ directory
 cd python
-# If you don't have a virtual environment, create one
+# Optional: create and activate virtual environment if you don't have one
 uv venv
-source .venv/activate
-# Install the SDK in editable mode
-uv pip install .
+source .venv/bin/activate
+# Install in editable mode
+uv pip install -e .
 ```
 
-## Documentation
+### Formatting
 
-The Python SDK documentation is available at
-[Feldera Python SDK Docs](https://docs.feldera.com/python).
+Formatting requires the `ruff` package: `uv pip install ruff`
 
-To build the html documentation run:
+```bash
+cd python
+ruff check
+ruff format
+```
 
-Ensure that you have sphinx installed. If not, install it using `uv pip install sphinx`.
+### Tests
 
-Then run the following commands:
+Running the test requires the `pytest` package: `uv pip install pytest`
 
 ```bash
-cd docs
-sphinx-apidoc -o . ../feldera
-make html
+# All tests
+cd python
+uv run python -m pytest tests/
+
+# Specific tests directory
+uv run python -m pytest tests/platform/
+
+# Specific test file
+uv run python -m pytest tests/platform/test_pipeline_crud.py
 ```
 
-To clean the build, run `make clean`.
+For further information about the tests, please see `tests/README.md`.
 
-## Linting and formatting
+### Documentation
 
-Use [Ruff] to run the lint checks that will be executed by the
-precommit hook when a PR is submitted:
+Building documentation requires the `sphinx` package: `uv pip install sphinx`
 
 ```bash
-ruff check python/
+cd python/docs
+sphinx-apidoc -o . ../feldera
+make html
+make clean  # Cleanup afterwards
 ```
 
-To reformat the code in the same way as the precommit hook:
+### Installation from GitHub
 
+Latest `main` branch:
 ```bash
-ruff format
+uv pip install git+https://github.com/feldera/feldera#subdirectory=python
 ```
 
-[Ruff]: https://github.com/astral-sh/ruff
+Different branch (replace `BRANCH_NAME`):
+```bash
+uv pip install git+https://github.com/feldera/feldera@BRANCH_NAME#subdirectory=python
+```
diff --git a/python/feldera/testutils.py b/python/feldera/testutils.py
@@ -43,10 +43,6 @@ def _get_effective_api_key():
     or os.environ.get("FELDERA_BASE_URL")
     or "http://localhost:8080"
 )
-KAFKA_SERVER = os.environ.get("FELDERA_KAFKA_SERVER", "localhost:19092")
-PIPELINE_TO_KAFKA_SERVER = os.environ.get(
-    "FELDERA_PIPELINE_TO_KAFKA_SERVER", "redpanda:9092"
-)
 FELDERA_REQUESTS_VERIFY = requests_verify_from_env()
 
 
diff --git a/python/pyproject.toml b/python/pyproject.toml
@@ -38,7 +38,6 @@ Issues = "https://github.com/feldera/feldera/issues"
 
 [tool.uv]
 dev-dependencies = [
-    "kafka-python-ng==2.2.2",
     "pytest-timeout>=2.3.1",
     "pytest-xdist>=3.8.0",
     "pytest>=8.3.5",
diff --git a/python/tests/__init__.py b/python/tests/__init__.py
@@ -4,8 +4,6 @@
     enterprise_only,
     API_KEY,
     BASE_URL,
-    PIPELINE_TO_KAFKA_SERVER,
-    KAFKA_SERVER,
     FELDERA_REQUESTS_VERIFY,
 )
 
@@ -15,7 +13,5 @@
     "enterprise_only",
     "API_KEY",
     "BASE_URL",
-    "PIPELINE_TO_KAFKA_SERVER",
-    "KAFKA_SERVER",
     "FELDERA_REQUESTS_VERIFY",
 ]
diff --git a/python/uv.lock b/python/uv.lock

Original file line number	Diff line number	Diff line change
`@@ -43,10 +43,6 @@ def _get_effective_api_key():`
`43`	`43`	`or os.environ.get("FELDERA_BASE_URL")`
`44`	`44`	`or "http://localhost:8080"`
`45`	`45`	`)`
`46`		`-KAFKA_SERVER = os.environ.get("FELDERA_KAFKA_SERVER", "localhost:19092")`
`47`		`-PIPELINE_TO_KAFKA_SERVER = os.environ.get(`
`48`		`- "FELDERA_PIPELINE_TO_KAFKA_SERVER", "redpanda:9092"`
`49`		`-)`
`50`	`46`	`FELDERA_REQUESTS_VERIFY = requests_verify_from_env()`
`51`	`47`
`52`	`48`