Fixes: #5799

Follows the same scheme as the kubernetes secret resolver.
To resolve an env var successfully, the string must start with: `${env:`
and end with `}`.

The env var name then has to start from a valid alphabet, or underscore.
The name then may contain numbers.

Example: `"table": "${env:TABLE_NAME}"`

Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>

Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>

[ci] apply automatic fixes

Signed-off-by: feldera-bot <feldera-bot@feldera.com>

apply suggestions from code review

Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>

Signed-off-by: feldera-bot <feldera-bot@feldera.com>

Signed-off-by: Mattias Matthiesen <mattias.matthiesen@eviny.no>

Signed-off-by: Mattias Matthiesen <mattias.matthiesen@eviny.no>

Signed-off-by: Mattias Matthiesen <mattias.matthiesen@eviny.no>

…eConfig

DevTweaks Option<f64> fields (bloom_false_positive_rate, balancer_balance_tax,
etc.) fail to deserialize when nested inside PipelineConfig, which uses
triggered by the workspace-wide arbitrary_precision feature: the Content
buffer represents numbers as maps, producing "invalid type: map, expected f64".

Apply the existing serde_via_value::deserialize workaround (already used by
ResourceConfig.cpu_cores_min/max) to all four Option<f64> fields in DevTweaks.

Also add the chrono "alloc" feature to fix a pre-existing compile error in
adapter_stats.rs (to_rfc3339_opts returns String, which requires alloc).

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>

Signed-off-by: feldera-bot <feldera-bot@feldera.com>

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>

This reverts commit d23c627.

This reverts commit c4c0fe2.

This reverts commit 0be9804.

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>

The connector already had retry logic in some places, but mostly relied on
delta-rs for retries. This wasn't always enough and we saw timeouts and expired
token errors bubbling up.

This commit adds retry loops around all object store accesses. The loops are
controlled by the new `max_retries` setting, similar to the output connector.
By default, it will retry forever. The retry loops set health status to UNHEALTHY
while retrying.

If the pipeline is stopped and restarted during a retry, the connector resumes
from the last successfully ingested table version.  After exhausting retry
attempts the connector fails permanently with a fatal error, which eliminates
the possibility of data loss.

There is an important caveat:

Because retries may occur after partial progress (e.g., after partially
processing a Delta log entry), the same data may be ingested more than once.
This is consistent with the connector’s at-least-once delivery guarantee.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>