pathwaycom/pathway repository overview

⁠What is Pathway?

Pathway⁠ is an open framework for high-throughput and low-latency real-time data processing. It is used to create Python code which seamlessly combines batch processing, streaming, and real-time API's for LLM apps. Pathway's distributed runtime (🦀-🐍) provides fresh results of your data pipelines whenever new inputs and requests are received.

Explore our solution at Pathway⁠ and dive into our GitHub repositories⁠:

• Framework⁠: The core of efficiency in real-time data processing.
• Examples⁠: Practical demonstrations of Pathway's adaptability.
• Benchmarks⁠: Performance metrics showcasing speed and reliability.

⁠How to use this image

⁠Image Variants

Currently, we have two types of images that can be used according to the user's needs:

• pathway:<version>-slim

This image contains only the Pathway package and the dependencies required to use it.

• pathway:<version>/pathway:latest

This image includes all the dependencies necessary to run Pathway, along with indexer examples.

⁠Create a Dockerfile in your Python app project

FROM pathwaycom/pathway:latest

WORKDIR /app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD [ "python", "./your-script.py" ]

You can then build and run the Docker image:

docker build -t my-pathway-app .
docker run -it --rm --name my-pathway-app my-pathway-app

⁠Run a single Python script

When dealing with single-file projects, creating a full-fledged Dockerfile might seem unnecessary. In such scenarios, you can execute a Python script directly using the Python Docker image. For example:

docker run -it --rm --name my-pathway-app -v "$PWD":/app pathwaycom/pathway:latest python my-pathway-app.py

⁠License

Pathway is distributed on a BSL 1.1 License which allows for unlimited non-commercial use, as well as use of the Pathway package for most commercial purposes, free of charge. Code in this repository automatically converts to Open Source (Apache 2.0 License) after 4 years. Some public repos which are complementary to this one (examples, libraries, connectors, etc.) are licensed as Open Source, under the MIT license.