Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions docs.feldera.com/docs/connectors/sources/nats.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,6 @@ with the `nats_input` connector.
The NATS input connector supports exactly-once [fault
tolerance](/pipelines/fault-tolerance) using JetStream's ordered pull consumer.

:::warning

NATS support is still experimental, and it may be substantially modified in the future.

:::

## How it works

The NATS input connector uses JetStream's **ordered pull consumer**, which provides:
Expand Down
10 changes: 8 additions & 2 deletions docs.feldera.com/docs/formats/json.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,8 +196,8 @@ data change event formats in JSON: (1) the raw format and (2) the insert/delete
### The insert/delete format

A data change event in this format is represented as a JSON object with a
single key, which must be equal to `insert` or `delete`. The associated value
represents the table row to be inserted or deleted, encoded using the format
single key, which must be equal to `insert`, `update`, or `delete`. The associated value
represents the table row to be inserted, updated, or deleted, encoded using the format
documented above. Example row insertion event:

```json
Expand All @@ -210,6 +210,12 @@ Example row deletion event:
{"delete": {"part": 1, "vendor": 2, "price": 10000}}
```

Full or partial updates are also supported if the table has a [primary key](/sql/grammar/#creating-tables). Example update event for a vendors table:
```json
{"update": {"vendor_id": 2, "vendor_address": "123 Feldera Place, San Francisco, CA"}}
```
The `vendor_id` column must be a primary key. The vendors table may have more columns, but only the `vendor_address` column will be updated by the update event above. An `update` payload that contains all of the table columns is equivalent to an `insert`.

### The raw format

This format is applicable to append-only event streams where rows can only
Expand Down
4 changes: 0 additions & 4 deletions docs.feldera.com/docs/pipelines/checkpoint-sync.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Checkpoint Sync to Object Store

:::caution Experimental feature
Synchronizing checkpoints to object store is a highly experimental feature.
:::

Feldera can synchronize pipeline checkpoints to an S3-compatible object store
and restore them at startup. This enables disaster recovery scenarios where
local storage may be lost, and supports running standby pipelines that can
Expand Down
6 changes: 0 additions & 6 deletions docs.feldera.com/docs/pipelines/modifying.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,6 @@
This feature is only available in Feldera Enterprise Edition.
:::

:::warning
This is an experimental feature and may undergo significant
changes, including non-backward-compatible modifications, in future releases of
Feldera.
:::

## Overview

Feldera supports **incremental pipeline modification** through a feature called **bootstrapping**.
Expand Down
6 changes: 0 additions & 6 deletions docs.feldera.com/docs/pipelines/transactions.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,6 @@ import TabItem from '@theme/TabItem';

# Efficient Bulk Data Processing using Transactions

:::warning
Transaction support is an experimental feature and may undergo significant
changes, including non-backward-compatible modifications in future releases of
Feldera.
:::

Transactions enable Feldera pipelines to ingest and process large volumes of data atomically—in
one logical unit of work rather than piece-by-piece. Transactions are used to achieve:

Expand Down
7 changes: 3 additions & 4 deletions docs.feldera.com/docs/sql/grammar.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,6 @@ columnConstraint
: PRIMARY KEY
| FOREIGN KEY REFERENCES identifier '(' identifier ')'
| LATENESS expression
| WATERMARK expression
| DEFAULT expression

tableConstraint
Expand Down Expand Up @@ -272,9 +271,9 @@ latenessStatement

See [Streaming SQL Extensions, LATENESS](streaming.md#lateness-expressions)

### WATERMARKS
<!-- ### WATERMARKS

See [Streaming SQL Extensions, WATERMARKS](streaming.md#watermark-expressions)
See [Streaming SQL Extensions, WATERMARKS](streaming.md#watermark-expressions) -->

## Creating user-defined functions.

Expand Down Expand Up @@ -545,7 +544,7 @@ hintOption

:::warning

These hints are considered still experimental, and they may change
These hints are considered experimental, and they may change

:::

Expand Down
4 changes: 2 additions & 2 deletions docs.feldera.com/docs/sql/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ lateness time units out of order.
See the [Time Series Analysis Guide](/tutorials/time-series#timestamp-columns-and-lateness)
for details.

## `WATERMARK` expressions
<!-- ## `WATERMARK` expressions

:::warning

Expand All @@ -31,7 +31,7 @@ or substantially modified in the future.
of the input rows by a constant amount of time.

See the [Time Series Analysis Guide](/tutorials/time-series#delaying-inputs-with-watermark)
for details.
for details. -->

## `append_only` tables

Expand Down
7 changes: 0 additions & 7 deletions docs.feldera.com/docs/sql/system.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,6 @@ built-in. Their contents is pre-populated by the runtime.

## `ERROR_VIEW`

:::warning

The `ERROR_VIEW` is still experimental; the schema and name of this
view may change.

:::

`ERROR_VIEW` is a view that signals runtime errors. The schema is
given by the following (illegal) SQL declaration:

Expand Down
15 changes: 2 additions & 13 deletions docs.feldera.com/docs/sql/udf.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,10 @@ AS (str LIKE ('%' || CAST(value AS VARCHAR) || '%'));
CREATE VIEW V0 AS SELECT contains_number(CAST('YES: 10 NO:5' AS VARCHAR), 5)
```

## User-defined functions written in Rust

:::warning Experimental feature

Rust UDF support is currently experimental and may undergo significant changes, including
non-backward-compatible modifications, in future releases of Feldera.

:::

:::danger

## User-defined functions written in Rust

* Feldera's incremental query engine assumes that all computations are deterministic. Using
a non-deterministic UDF is likely to result in incorrect outputs. The SQL compiler cannot verify
that Rust functions meet this requirement. Therefore, it is the responsibility of the UDF
Expand Down Expand Up @@ -254,10 +247,6 @@ curl -i -X PUT http://127.0.0.1:8080/v0/pipelines/udf_api_test \

### SQL type representation in Rust

:::caution Experimental feature

:::

The following table shows the Rust representation of standard SQL data
types. A nullable SQL type is represented by the corresponding rust
`Option<>` type. Notice that some of these types are not standard
Expand Down
33 changes: 0 additions & 33 deletions docs.feldera.com/docs/tutorials/monitoring/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,39 +70,6 @@ A Feldera Instance Monitoring dashboard will be created in Grafana.
The dashboard is a template and may need to be adjusted to fit your specific needs.
Look for the `feldera_*` metrics in Grafana to add more metrics to the dashboard.

## Tracing with Jaeger

A Feldera pipeline can be traced using Jaeger. Tracing is useful if you need to analyze
throughput or latency bottlenecks of your pipeline, as it instruments every step of the
pipeline execution and provides detailed information about the execution time of each
step.

### Setup

1. **Jaeger:** You must have [Jaeger installed](https://www.jaegertracing.io).
2. Start Jaeger using the `jaeger-all-in-one` script:
```bash
./jaeger-all-in-one
```
3. **Tracing a Feldera pipeline**
- Enable/disable tracing and specify the Jaeger endpoint in the pipeline `runtime_config`:
```bash
curl -i -X PATCH http://127.0.0.1:8080/v0/pipelines/tracing-pipeline \
-H 'Content-Type: application/json' \
-d '{
"runtime_config": {"tracing": true, "tracing_endpoint_jaeger": "host.docker.internal:6831", <other config settings> }
}'
```
- Use `host.docker.internal:6831` if you are running Feldera in docker or
`127.0.0.1:6831` if you run Feldera directly.
- Make sure to specify other settings you changed from non-default values in the `runtime_config` as well.
`runtime_config` can be retrieved with:
```bash
curl -X 'GET' \
'http://127.0.0.1:8080/v0/pipelines/tracing-pipeline' \
-H 'accept: application/json'
```

## DBSP Profiles

A DBSP profile is a graph of the pipeline's circuit where each node represents an
Expand Down
9 changes: 5 additions & 4 deletions docs.feldera.com/docs/tutorials/time-series.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,8 +147,9 @@ tables and views:
* **LATENESS does NOT delay computation**. Feldera does not delay computation
until all out-of-order records have been received. It always computes the
output of the queries given all inputs received so far and incrementally updates
these outputs as new out-of-order inputs arrive. (See [below](#delaying-inputs-with-watermark)
for an experimental feture that allows delaying inputs on demand).
these outputs as new out-of-order inputs arrive.
<!-- (See [below](#delaying-inputs-with-watermark)
for an experimental feature that allows delaying inputs on demand). -->

* **Inputs that violate lateness are discarded.** When a program receives a record
that is more than lateness time units behind the most recent timestamp value
Expand Down Expand Up @@ -399,7 +400,7 @@ The use of `emit_final` is subject to the following restrictions:
for the view used as output. If the view is used in defining
other views, these derived views will receive the non-delayed data.

## Delaying inputs with `WATERMARK`
<!-- ## Delaying inputs with `WATERMARK`

:::warning

Expand Down Expand Up @@ -442,7 +443,7 @@ CREATE TABLE purchase_watermark (
The current version of the SQL compiler does not support multiple
`WATERMARK` columns in a single table.

:::
::: -->

## Append-only tables

Expand Down
Loading