Skip to content

Prevent throwing RuntimeException when invalid proto is received - #166

Merged
feast-ci-bot merged 1 commit into
feast-dev:masterfrom
pradithya:proto_error
Mar 29, 2019
Merged

Prevent throwing RuntimeException when invalid proto is received#166
feast-ci-bot merged 1 commit into
feast-dev:masterfrom
pradithya:proto_error

Conversation

@pradithya

Copy link
Copy Markdown
Collaborator

Follow up to fix #163

@zhilingc

Copy link
Copy Markdown
Collaborator

/lgtm

.print(rowExtended));
} catch (InvalidProtocolBufferException e) {
return KV.of(
rowExtended.getRow().getEntityName(),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we assuming that rowExtended.getRow().getEntityName() was not the cause of the exception?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, at worst, that line will return empty string.

@thirteen37

Copy link
Copy Markdown
Collaborator

/approve

1 similar comment
@pradithya

Copy link
Copy Markdown
Collaborator Author

/approve

@feast-ci-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pradithya, thirteen37

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@feast-ci-bot
feast-ci-bot merged commit a0111bb into feast-dev:master Mar 29, 2019
Vperiodt pushed a commit to Vperiodt/feast that referenced this pull request Aug 10, 2026
…feast-dev#166)

* feat: Multi-arch publish for feast operator image

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* feat: Allow users to have protected project on shared registry

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>

* fix: Defer feature-freshness thread to post-fork to avoid Gunicorn deadlock (feast-dev#6648)

feast_metrics.start_metrics_server() starts a feature-freshness thread
in the Gunicorn master process, before Gunicorn forks its worker(s).
That thread's first action, with no delay, is update_feature_freshness()
-> store.list_feature_views(), which lazily builds the registry for the
first time. For registry backends needing a lazy DBAPI import (e.g. the
SQL registry importing pymysql via SQLAlchemy's create_engine()), this
means a thread in the master can be mid-import, holding CPython's
per-module import lock, at the exact moment Gunicorn forks a worker.

POSIX fork() only duplicates the calling thread into the child process;
every other thread in the parent, including this one, simply ceases to
exist in the worker. If the fork lands while that thread holds a
module's import lock, the lock stays permanently held in the new
worker, since there is no longer any thread that can finish the import
and release it. The worker's own later attempt to build its registry
then deadlocks forever with no error - the process just hangs at
"Waiting for application startup." This is intermittent by nature: it
only manifests if the fork lands inside that narrow timing window.

Resource monitoring already avoids this correctly (start_resource_monitoring=
not uses_gunicorn plus the post_worker_init hook calling
init_worker_monitoring()), but the freshness thread was not given the
same treatment. This applies the identical pattern: start_metrics_server
gains a start_freshness_monitoring flag (deferred exactly like resource
monitoring's), and FeastServeApplication's post_worker_init hook now
also calls the new init_worker_freshness_monitoring(store) after the
fork, instead of feast_metrics.py starting it unconditionally beforehand.

Fixes feast-dev#6647

Signed-off-by: Carlos Sánchez <carlos.sancheza@cabify.com>
Co-authored-by: Carlos Sánchez <carlos.sancheza@cabify.com>

---------

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
Signed-off-by: Carlos Sánchez <carlos.sancheza@cabify.com>
Co-authored-by: ntkathole <nikhilkathole2683@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Carlos Sánchez <i52saarc@uco.es>
Co-authored-by: Carlos Sánchez <carlos.sancheza@cabify.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ingestion retrying an invalid FeatureRow endlessly

4 participants