diff --git a/docs.feldera.com/docs/connectors/sources/nats.md b/docs.feldera.com/docs/connectors/sources/nats.md index b2265d78215..b0a1507e839 100644 --- a/docs.feldera.com/docs/connectors/sources/nats.md +++ b/docs.feldera.com/docs/connectors/sources/nats.md @@ -6,12 +6,6 @@ with the `nats_input` connector. The NATS input connector supports exactly-once [fault tolerance](/pipelines/fault-tolerance) using JetStream's ordered pull consumer. -:::warning - -NATS support is still experimental, and it may be substantially modified in the future. - -::: - ## How it works The NATS input connector uses JetStream's **ordered pull consumer**, which provides: diff --git a/docs.feldera.com/docs/formats/json.md b/docs.feldera.com/docs/formats/json.md index 75b09797139..9160e4945d5 100644 --- a/docs.feldera.com/docs/formats/json.md +++ b/docs.feldera.com/docs/formats/json.md @@ -196,8 +196,8 @@ data change event formats in JSON: (1) the raw format and (2) the insert/delete ### The insert/delete format A data change event in this format is represented as a JSON object with a -single key, which must be equal to `insert` or `delete`. The associated value -represents the table row to be inserted or deleted, encoded using the format +single key, which must be equal to `insert`, `update`, or `delete`. The associated value +represents the table row to be inserted, updated, or deleted, encoded using the format documented above. Example row insertion event: ```json @@ -210,6 +210,12 @@ Example row deletion event: {"delete": {"part": 1, "vendor": 2, "price": 10000}} ``` +Full or partial updates are also supported if the table has a [primary key](/sql/grammar/#creating-tables). Example update event for a vendors table: +```json +{"update": {"vendor_id": 2, "vendor_address": "123 Feldera Place, San Francisco, CA"}} +``` +The `vendor_id` column must be a primary key. The vendors table may have more columns, but only the `vendor_address` column will be updated by the update event above. An `update` payload that contains all of the table columns is equivalent to an `insert`. + ### The raw format This format is applicable to append-only event streams where rows can only diff --git a/docs.feldera.com/docs/pipelines/checkpoint-sync.md b/docs.feldera.com/docs/pipelines/checkpoint-sync.md index 67f232c652a..9ad687104fe 100644 --- a/docs.feldera.com/docs/pipelines/checkpoint-sync.md +++ b/docs.feldera.com/docs/pipelines/checkpoint-sync.md @@ -1,9 +1,5 @@ # Checkpoint Sync to Object Store -:::caution Experimental feature -Synchronizing checkpoints to object store is a highly experimental feature. -::: - Feldera can synchronize pipeline checkpoints to an S3-compatible object store and restore them at startup. This enables disaster recovery scenarios where local storage may be lost, and supports running standby pipelines that can diff --git a/docs.feldera.com/docs/pipelines/modifying.md b/docs.feldera.com/docs/pipelines/modifying.md index df1532bcb8a..81647db9e9b 100644 --- a/docs.feldera.com/docs/pipelines/modifying.md +++ b/docs.feldera.com/docs/pipelines/modifying.md @@ -4,12 +4,6 @@ This feature is only available in Feldera Enterprise Edition. ::: -:::warning -This is an experimental feature and may undergo significant -changes, including non-backward-compatible modifications, in future releases of -Feldera. -::: - ## Overview Feldera supports **incremental pipeline modification** through a feature called **bootstrapping**. diff --git a/docs.feldera.com/docs/pipelines/transactions.md b/docs.feldera.com/docs/pipelines/transactions.md index 56abac4c5e8..be03d953596 100644 --- a/docs.feldera.com/docs/pipelines/transactions.md +++ b/docs.feldera.com/docs/pipelines/transactions.md @@ -3,12 +3,6 @@ import TabItem from '@theme/TabItem'; # Efficient Bulk Data Processing using Transactions -:::warning -Transaction support is an experimental feature and may undergo significant -changes, including non-backward-compatible modifications in future releases of -Feldera. -::: - Transactions enable Feldera pipelines to ingest and process large volumes of data atomically—in one logical unit of work rather than piece-by-piece. Transactions are used to achieve: diff --git a/docs.feldera.com/docs/sql/grammar.md b/docs.feldera.com/docs/sql/grammar.md index 334b55e95f6..f3f10e90ef8 100644 --- a/docs.feldera.com/docs/sql/grammar.md +++ b/docs.feldera.com/docs/sql/grammar.md @@ -73,7 +73,6 @@ columnConstraint : PRIMARY KEY | FOREIGN KEY REFERENCES identifier '(' identifier ')' | LATENESS expression - | WATERMARK expression | DEFAULT expression tableConstraint @@ -272,9 +271,9 @@ latenessStatement See [Streaming SQL Extensions, LATENESS](streaming.md#lateness-expressions) -### WATERMARKS + ## Creating user-defined functions. @@ -545,7 +544,7 @@ hintOption :::warning -These hints are considered still experimental, and they may change +These hints are considered experimental, and they may change ::: diff --git a/docs.feldera.com/docs/sql/streaming.md b/docs.feldera.com/docs/sql/streaming.md index 6365a519f99..57eb6924e30 100644 --- a/docs.feldera.com/docs/sql/streaming.md +++ b/docs.feldera.com/docs/sql/streaming.md @@ -18,7 +18,7 @@ lateness time units out of order. See the [Time Series Analysis Guide](/tutorials/time-series#timestamp-columns-and-lateness) for details. -## `WATERMARK` expressions + ## `append_only` tables diff --git a/docs.feldera.com/docs/sql/system.md b/docs.feldera.com/docs/sql/system.md index 3861ffc3491..fd2a8c41322 100644 --- a/docs.feldera.com/docs/sql/system.md +++ b/docs.feldera.com/docs/sql/system.md @@ -5,13 +5,6 @@ built-in. Their contents is pre-populated by the runtime. ## `ERROR_VIEW` -:::warning - -The `ERROR_VIEW` is still experimental; the schema and name of this -view may change. - -::: - `ERROR_VIEW` is a view that signals runtime errors. The schema is given by the following (illegal) SQL declaration: diff --git a/docs.feldera.com/docs/sql/udf.md b/docs.feldera.com/docs/sql/udf.md index 90f7a0112d1..7f5d6e89926 100644 --- a/docs.feldera.com/docs/sql/udf.md +++ b/docs.feldera.com/docs/sql/udf.md @@ -15,17 +15,10 @@ AS (str LIKE ('%' || CAST(value AS VARCHAR) || '%')); CREATE VIEW V0 AS SELECT contains_number(CAST('YES: 10 NO:5' AS VARCHAR), 5) ``` -## User-defined functions written in Rust - -:::warning Experimental feature - -Rust UDF support is currently experimental and may undergo significant changes, including -non-backward-compatible modifications, in future releases of Feldera. - -::: - :::danger +## User-defined functions written in Rust + * Feldera's incremental query engine assumes that all computations are deterministic. Using a non-deterministic UDF is likely to result in incorrect outputs. The SQL compiler cannot verify that Rust functions meet this requirement. Therefore, it is the responsibility of the UDF @@ -254,10 +247,6 @@ curl -i -X PUT http://127.0.0.1:8080/v0/pipelines/udf_api_test \ ### SQL type representation in Rust -:::caution Experimental feature - -::: - The following table shows the Rust representation of standard SQL data types. A nullable SQL type is represented by the corresponding rust `Option<>` type. Notice that some of these types are not standard diff --git a/docs.feldera.com/docs/tutorials/monitoring/index.md b/docs.feldera.com/docs/tutorials/monitoring/index.md index 4fc0c46e970..b40e51397d4 100644 --- a/docs.feldera.com/docs/tutorials/monitoring/index.md +++ b/docs.feldera.com/docs/tutorials/monitoring/index.md @@ -70,39 +70,6 @@ A Feldera Instance Monitoring dashboard will be created in Grafana. The dashboard is a template and may need to be adjusted to fit your specific needs. Look for the `feldera_*` metrics in Grafana to add more metrics to the dashboard. -## Tracing with Jaeger - -A Feldera pipeline can be traced using Jaeger. Tracing is useful if you need to analyze -throughput or latency bottlenecks of your pipeline, as it instruments every step of the -pipeline execution and provides detailed information about the execution time of each -step. - -### Setup - -1. **Jaeger:** You must have [Jaeger installed](https://www.jaegertracing.io). -2. Start Jaeger using the `jaeger-all-in-one` script: - ```bash - ./jaeger-all-in-one - ``` -3. **Tracing a Feldera pipeline** - - Enable/disable tracing and specify the Jaeger endpoint in the pipeline `runtime_config`: - ```bash - curl -i -X PATCH http://127.0.0.1:8080/v0/pipelines/tracing-pipeline \ - -H 'Content-Type: application/json' \ - -d '{ - "runtime_config": {"tracing": true, "tracing_endpoint_jaeger": "host.docker.internal:6831", } - }' - ``` - - Use `host.docker.internal:6831` if you are running Feldera in docker or - `127.0.0.1:6831` if you run Feldera directly. - - Make sure to specify other settings you changed from non-default values in the `runtime_config` as well. - `runtime_config` can be retrieved with: - ```bash - curl -X 'GET' \ - 'http://127.0.0.1:8080/v0/pipelines/tracing-pipeline' \ - -H 'accept: application/json' - ``` - ## DBSP Profiles A DBSP profile is a graph of the pipeline's circuit where each node represents an diff --git a/docs.feldera.com/docs/tutorials/time-series.md b/docs.feldera.com/docs/tutorials/time-series.md index 37c038b220b..1595c8ba173 100644 --- a/docs.feldera.com/docs/tutorials/time-series.md +++ b/docs.feldera.com/docs/tutorials/time-series.md @@ -147,8 +147,9 @@ tables and views: * **LATENESS does NOT delay computation**. Feldera does not delay computation until all out-of-order records have been received. It always computes the output of the queries given all inputs received so far and incrementally updates - these outputs as new out-of-order inputs arrive. (See [below](#delaying-inputs-with-watermark) - for an experimental feture that allows delaying inputs on demand). + these outputs as new out-of-order inputs arrive. + * **Inputs that violate lateness are discarded.** When a program receives a record that is more than lateness time units behind the most recent timestamp value @@ -399,7 +400,7 @@ The use of `emit_final` is subject to the following restrictions: for the view used as output. If the view is used in defining other views, these derived views will receive the non-delayed data. -## Delaying inputs with `WATERMARK` + ## Append-only tables