You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Dec 10, 2025. It is now read-only.
I'm currently using bigtable for archival of tsdb data. When attempting to restore, I find that I have to frequently release the table.scan() cursor, or my memory usage skyrockets (will consume 32gb within an hour). With checkpointing in place and new cursors, I am not seeing this issue. Is this working as intended?
I've created 2 gists to demonstrate the behavior, both of which assume they're running on a GCE instance with local instance credentials granted the necessary BT roles. https://gist.github.com/clusterfudge/3fe40f7f55e8a2a80c65dbd8e8d6764e - implements checkpointing, has a reasonable and stable memory footprint
I'm currently using bigtable for archival of tsdb data. When attempting to restore, I find that I have to frequently release the table.scan() cursor, or my memory usage skyrockets (will consume 32gb within an hour). With checkpointing in place and new cursors, I am not seeing this issue. Is this working as intended?
I've created 2 gists to demonstrate the behavior, both of which assume they're running on a GCE instance with local instance credentials granted the necessary BT roles.
https://gist.github.com/clusterfudge/3fe40f7f55e8a2a80c65dbd8e8d6764e - implements checkpointing, has a reasonable and stable memory footprint
https://gist.github.com/clusterfudge/3a2e6b484d24070b6f43fd3fd989e229 - simple single call to scan, will consume all memory available, stopped only by the heat death of the universe.